We rebuilt our own hiring pipeline with AI

We tell clients to start with the boring internal thing they can measure. So here is ours.

We recruit senior engineers at volume, and two stages of that operation ran on scarce specialist time: assessing whether a candidate is actually as good as their résumé says, and turning a qualified candidate into a profile a client will read. Both took specialist time and got slower as we grew. We redesigned them around AI. Here is what is inside.

Stage one: the assessment was capped by the interviewer

Technical screening needed recruiters with enough engineering depth to evaluate experienced developers across several technologies. That is a narrow hiring profile, and even a strong one has boundaries: someone fluent in Python and Django ends up evaluating a senior Node engineer, and a conversation that starts in Python moves into cloud infrastructure, databases, security, distributed systems.

The sharper failure was subtler. When a candidate knew a technology better than the interviewer, the interviewer could not reliably tell real depth from an answer that merely sounded right. So the depth of an assessment was partly determined by who happened to run it.

Coordinating two calendars for each interview also limited assessment capacity.

What we built instead

We did not automate the interview. We asked what the interview was for.

For senior engineers, the useful question is not whether they can solve an isolated exercise. It is how they think: how they approach uncertainty, what alternatives they weigh, whether they can name trade-offs, why one architecture over another, what they do when a solution introduces a new constraint. Coding challenges answer none of that, and they ask experienced candidates to spend hours writing code early in a process they have not committed to.

So we built a conversational assessment. The candidate is given a technical situation and explains, in their own words, how they would approach it. There is no fixed questionnaire: the system reads each answer and decides where to go next. A discussion that starts in Python and produces a cloud architecture can be pushed on that architecture; if the design leans on a database strategy, the conversation moves into design, performance, consistency. It follows the candidate's own reasoning across domains, which is how you separate theoretical familiarity from hands-on experience — and it removes the requirement that one interviewer personally master every technology that might come up.

Assessments run on demand and concurrently, so candidates can pick a time when they are prepared instead of coordinating two calendars.

The output is structured rather than anecdotal: demonstrated depth, a corresponding seniority, and where the candidate sits relative to everyone else we have evaluated. Spoken English is scored from the same conversation. Both land in the recruiter's view next to the profile, as inputs a person reads. The AI scores; it does not make hiring decisions. No candidate advances or is rejected on a score alone.

Stage two: the client-ready profile

Qualified candidates still had to be presented to clients in our standard format — experience, responsibilities, technologies, real participation in each project. Since early 2025, we have used an agent in production to turn interview transcripts into client-ready profiles.

It was never just writing. To represent a senior engineer accurately you have to understand how each technology was used in each project, what the candidate personally implemented versus what their team shipped, and which details a client will actually care about. Without that technical context, things get omitted, flattened, or quietly overstated. Each profile took two to six hours.

The agent takes the transcript and extracts a fixed schema: technologies per role and the verb attached to each — built, maintained, evaluated, migrated; work the candidate implemented personally, held separate from team output; role and real level of participation; the technical detail relevant to presenting them. Then it writes the profile in our format. Not a summary of the interview — a document with one job: making a senior engineer legible to someone deciding whether to meet them.

The hard part was pronouns. Interviews are told in "we." A profile has to be honest about "I." A model asked to summarise will happily turn "we migrated the payments service" into a claim that the candidate migrated the payments service. Getting it to interrogate every sentence for attribution — and to drop the claim when the transcript does not support it — was most of the build, and it is the difference between a profile a client trusts and one they skim.

Two to six hours became about ten minutes per profile.

What production feedback taught us

We iterated on real output: errors, omissions, inconsistencies and edge cases. Each correction became a rule: how to handle a project described only in the plural, what to do with a technology mentioned once in passing, when "led" means led and when it means attended.

The rules we had written in advance were mostly wrong about where the problems would be. Production feedback showed us which rules the system actually needed.

What we measure

Preparing each profile went from two to six hours to about ten minutes. These figures describe our own profile-preparation workflow.

Hours per profile give us a concrete unit to measure. On the assessment side, technical and spoken-English scores are inputs for a recruiter, alongside the profile. They are not automatic hiring decisions.

What we would tell anyone doing this

Ask what the step is for before you automate it. We could have built a faster version of the interview we already ran. The interview we actually needed was a different one.

Transcript quality is the ceiling. Everything downstream — the profile, the seniority, the English level — inherits the transcript's errors. We spent more time there than on any model choice.

Let the corrections write the rules. Review real output for errors, omissions and attribution. Those corrections expose edge cases that design alone can miss.

Measure the unit. Hours per profile. Interviews per day. Not "productivity."

We built this for ourselves. Start by finding where specialist availability, scarce knowledge or manual handoffs limit an operation. Then define a concrete unit to measure before redesigning the workflow.