As some of you may know, I recently shut down my old company Ameliant, and have joined Confluent – the Good People behind Apache Kafka – as a Solutions Architect. So far, after 5 weeks I like what I have seen. A group of very nice (important), smart people, and a company that has learned the lessons of those who came before them with a strong plan for the future.
As with any company that is experiencing growth, there are certain problems that emerge that are Good To Have. In particular, on the Professional Services team that I am on, we have a shortage of Solutions Architects to go out on site and help clients get set up, and comfortable with our software platform. To that end I wanted to describe what it is that I do in my role, in the hope that you might decide that this is something that might appeal to you, and entice you to join us.
As an SA, my role is extremely client facing. Clients would engage us for a Professional Services engagement once they have decided to take up the Confluent Platform (more on this later). Our engagements are paid for, and follow discussions with Sales and Sales Engineers, who have convinced them of the technical merits of our streaming platform. Our engagements are usually in the order of a few days out on site, and are usually followed up by a consulting report. I am based in EMEA, and travel to clients mostly arond Europe (we are looking for SAs in North America and Europe at this stage). The travel demands are a couple of days a week, for most weeks – we have a considerable pipeline of work.
The Solutions Architect role can be a bit of a nebulous term, but is loosely equivalent to that of a Principal Consultant in other product companies. Our responsibility is to make clients comfortable at the design/early implementation phase of a streaming project, and clear up any gaps in their understanding. Clients typically engage us after they have had some training on Apache Kafka/Confluent Platform and have had an opportunity to kick the tires at the Proof of Concept stage. As such, they already have a working understanding of at least Apache Kafka, and a handful of use cases put together. Our work usually involves looking at what they have done, and making sure that they have all of the information that they need to take it to the next phase of the project – usually Production.
On an average client engagement we talk about:
- their physical infrastructure requirements
- platform monitoring
- existing code; code reviews are fairly typical – usually Java, potentially Scala, sometimes other platforms (.Net etc.)
- better ways to implement certain things that they may already have done; an understanding of messaging pattern, streaming is useful
As such, the Solutions Architect is a mixed role. We need an appreciation of infrastructure from cloud all the way down to disk. We need a solid programming background to take a look at often novel applications on the client side – multi-threading and ability to reason about object lifecycles, garbage collection and network interactions is frequently needed. In addition, we need to be able to communicate with clients – we typically speak to groups of 4-10 people within different functions – from development, right through operations, systems architecture and occasionally C-level executives.
Our platform is vast, and composed of:
- Apache Kafka
- Kafka Connect
- Kafka Streams – an embeddable Java DSL for Stream processing
- KSQL (if you haven’t seen this, you absolutely should – it’s awesome)
- Confluent Control Center (C3) – a monitoring and management console
- REST Proxy
- Confluent Schema Registry
We don’t expect you to have an understanding or background in these, as we have a solid ramp-up program and you won’t be thrown off the deep end with clients on starting. We have a strong onboarding and mentoring process, where you will be trained up on the products, and shadow other SAs while they take a lead on jobs. After a while, comes a period of reverse-shadowing, where you take the lead, while backed up by another SA. At all times, you are connected back into and supported by Confluent Engineering and Operations.
Jobs can be unpredictable – you might get called in for a health check of a client’s setup before they are given the green light to launch, and end up talking about alternative ways to structure their applications based on what you have seen. An appreciation of distributed systems, messaging or integration will make your life much easier. If you have worked with things like Camel, Spark, Cassandra then you might already have a background in the general area. The sheer variety of work in terms of clients and use cases is extremely interesting, and professionally very satisfying.
Our interview process probably won’t bother you too much if you are still reading. We don’t do any silly puzzle questions – we typically ask you how you might go about building a piece of software infrastructure like a link shortener, or a real-time console based on changing data in a number of back-end systems, in order to start a conversation. We will talk around a bunch of the design decisions and trade offs that you want to take with respect to scaling out, perfomance, storage, infrastructure. The point isn’t to catch you out, but to assess where your strengths and weaknesses lie.
If this sounds like in might be interesting to you, feel free to apply directly, or ping me on Twitter and we can have a chat.