We rebuilt our own hiring pipeline with AI

We tell clients to start with the boring internal thing they can measure. So here is ours.

We recruit senior engineers at volume, and two stages of that operation ran on scarce specialist time: assessing whether a candidate is actually as good as their résumé says, and turning a qualified candidate into a profile a client will read. Both were slow, both were expensive, and both got worse as we grew. We redesigned them around AI. Here is what is inside.

Stage one: the assessment was capped by the interviewer

Technical screening needed recruiters with enough engineering depth to evaluate experienced developers across several technologies. That is a narrow hiring profile, and even a strong one has boundaries: someone fluent in Python and Django ends up evaluating a senior Node engineer, and a conversation that starts in Python moves into cloud infrastructure, databases, security, distributed systems.

The sharper failure was subtler. When a candidate knew a technology better than the interviewer, the interviewer could not reliably tell real depth from an answer that merely sounded right. So the depth of an assessment was partly determined by who happened to run it.

And there was a hard ceiling: about six technical interviews per recruiter per day, with two calendars to align for each one.

What we built instead

We did not automate the interview. We asked what the interview was for.

For senior engineers, the useful question is not whether they can solve an isolated exercise. It is how they think: how they approach uncertainty, what alternatives they weigh, whether they can name trade-offs, why one architecture over another, what they do when a solution introduces a new constraint. Coding challenges answer none of that, and they ask experienced candidates to spend hours writing code early in a process they have not committed to.

So we built a conversational assessment. The candidate is given a technical situation and explains, in their own words, how they would approach it. There is no fixed questionnaire: the system reads each answer and decides where to go next. A discussion that starts in Python and produces a cloud architecture can be pushed on that architecture; if the design leans on a database strategy, the conversation moves into design, performance, consistency. It follows the candidate's own reasoning across domains, which is how you separate theoretical familiarity from hands-on experience — and it removes the requirement that one interviewer personally master every technology that might come up.

Two things changed as a consequence. Assessments run on demand and concurrently, so candidates pick a time when they are prepared instead of negotiating calendars, and evaluation capacity stops being a function of headcount. And when hundreds of people apply, we no longer have to pick who gets one of the scarce interview slots based mostly on a résumé — we can evaluate the whole pool and make the next decision on technical evidence.

The output is structured rather than anecdotal: demonstrated depth, a corresponding seniority, and where the candidate sits relative to everyone else we have evaluated. Spoken English is scored from the same conversation. Both land in the recruiter's view next to the profile, as inputs a person reads.

Stage two: the profile that took six hours

Qualified candidates still had to be presented to clients in our standard format — experience, responsibilities, technologies, real participation in each project. We had a dedicated team writing those.

It was never just writing. To represent a senior engineer accurately you have to understand how each technology was used in each project, what the candidate personally implemented versus what their team shipped, and which details a client will actually care about. Without that technical context, things get omitted, flattened, or quietly overstated. Each profile took four to six hours; a full candidate pool for one client took two to three days.

The agent takes the transcript and extracts a fixed schema: technologies per role and the verb attached to each — built, maintained, evaluated, migrated; work the candidate implemented personally, held separate from team output; role and real level of participation; the technical detail relevant to presenting them. Then it writes the profile in our format. Not a summary of the interview — a document with one job: making a senior engineer legible to someone deciding in five minutes whether to meet them.

The hard part was pronouns. Interviews are told in "we." A profile has to be honest about "I." A model asked to summarise will happily turn "we migrated the payments service" into a claim that the candidate migrated the payments service. Getting it to interrogate every sentence for attribution — and to drop the claim when the transcript does not support it — was most of the build, and it is the difference between a profile a client trusts and one they skim.

Four to six hours became five to ten minutes. A candidate pool that took two to three days is ready in under two.

How we earned the autonomy

We did not switch it on and walk away.

For the first three weeks the operation went from three writers to one reviewer, whose job was to validate every generated profile and flag errors, omissions, inconsistencies and edge cases. Each correction became a rule: how to handle a project described only in the plural, what to do with a technology mentioned once in passing, when "led" means led and when it means attended.

Over the next three to four weeks we iterated on production feedback. As the corrections approached zero, the dedicated review step stopped being necessary. Three writers → one reviewer → no dedicated operators, in about seven weeks.

The rules we had written in advance were mostly wrong about where the problems would be. The review step was not overhead. It was how the system got built.

The numbers

Profile production went from four to six hours to five to ten minutes. Output went from about 90 profiles a month to about 890 — the same operation, an order of magnitude more candidates presented. On the assessment side the constraint that disappeared is harder to put in a single figure: evaluation capacity is no longer tied to how many technical recruiters we have, when they are free, or which technologies they happen to know.

The point was not automating two tasks. It was that quality stopped depending on which individual performed each step — the same standard now applies to every candidate, and it holds as volume grows.

What we would tell anyone doing this

Ask what the step is for before you automate it. We could have built a faster version of the interview we already ran. The interview we actually needed was a different one.

Transcript quality is the ceiling. Everything downstream — the profile, the seniority, the English level — inherits the transcript's errors. We spent more time there than on any model choice.

Let the corrections write the rules. Not your assumptions. A reviewer marking up real output for a few weeks will teach you more about the edge cases than any amount of design.

Measure the unit. Hours per profile. Interviews per day. Not "productivity."

We built this for ourselves before we built anything like it for anyone else, and it is the same pattern we now put inside other companies' operations: find where specialist availability, scarce knowledge or manual handoffs cap the business, and redesign that part around what the models are actually good at.