I built a full-stack AI interview coach: upload a resume, paste a job description, and an agent runs an adaptive interview grounded in a real gap analysis between the two, then hands you a structured report. It was the take-home for Sigma's technical assessment, and it is public on GitHub. This is the walkthrough of the decisions I actually had to make, not a feature list.
Stack: Next.js frontend (static export), FastAPI backend, Anthropic SDK (Claude Haiku), SQLModel over SQLite, packaged as a single Docker process. In production, FastAPI serves the built frontend directly, so the whole thing runs on one port.
The problem: interviews drift
Most "AI interviewer" demos read your resume and the job posting on every turn and improvise from there. The output wanders, repeats itself, and forgets what it already asked. The fix I wanted was to make the interview reason about a structured artifact instead of re-reading raw text each turn. Compute the gap between candidate and role once, then let every question trace back to a specific item in that gap. That one decision shaped the rest of the build.
Architecture: a four-step pipeline
Resume + Job Description
|
[1] Extraction -> ResumeProfile, JobProfile (structured, no guessing)
|
[2] Gap Analysis -> strong matches, weak/missing (by priority), claims to verify
|
[3] Gap-driven interview -> one question per turn, each targeting a gap item
|
[4] Report -> score, strengths, gaps (each traceable to evidence)The interview agent (step 3) never sees the raw resume or job description again. It only receives the gap analysis plus the conversation so far. That is the core design decision in the project, and everything below follows from it.
Decision 1: the gap analysis is the only source of truth
Re-reading source documents every turn is how these agents lose the plot. Instead, extraction (step 1) turns each document into a structured profile once, with an explicit instruction to only use what is in the text and to leave fields empty rather than guess. Extraction is also lazy and cached: it runs at session creation and the result is written back onto the resume/JD row, so reusing the same resume in a later session does not pay for extraction again.
From there, the interview agent works off the gap analysis alone. Every question has to target a specific gap item by id, worked through by priority: high-priority missing requirements first, then medium, then low, then claims worth verifying out loud. When those are exhausted but more questions are still owed, it verifies strong matches instead (is a certification still current, is the candidate available for the exact shift). The final report is held to the same standard: every strength and gap it lists has to trace back to either the gap analysis or something the candidate actually said. No free-floating praise.
Decision 2: structured output by forced tool use, not "please return JSON"
Every LLM call in the app returns a validated Pydantic object, and it gets there by forcing the model to call a tool whose input_schema is that Pydantic schema, rather than asking it to emit JSON in prose. If the response fails validation, the error is fed back to the model for exactly one retry before the call gives up.
# The tool's input_schema IS the target Pydantic schema; tool_choice forces
# the model to call it. That is how the structured output is guaranteed.
tool = {
"name": "record_output",
"description": "Record the result in the required structure.",
"input_schema": schema.model_json_schema(),
}
...
response = client.messages.create(
model=MODEL,
max_tokens=max_tokens,
system=system,
messages=messages,
tools=[tool],
tool_choice={"type": "tool", "name": "record_output"},
)
...
block = _first_tool_use(response)
return schema.model_validate(block.input)Anyone who has used Vercel AI SDK's generateObject with a Zod schema will recognize the shape. This is the same idea, implemented by hand against the raw Anthropic SDK instead of a library, because one goal of the project was to keep every part of the agent loop visible and readable rather than delegated to a framework. All of it lives in one gateway module (client.py) that every agent call routes through, so the cross-cutting rules (the mock switch, cost logging, the API key, forced tool use, retry) live in exactly one place instead of drifting across four copies.
Decision 3: the model does not get to decide when to stop
I did not want interview length left entirely to the model. So termination is a hybrid rule enforced in code: a floor, a ceiling, and the agent's own judgment only in between.
MIN_QUESTIONS = 5
MAX_QUESTIONS = 12
def should_end_interview(should_end: bool, question_count: int) -> bool:
if question_count < MIN_QUESTIONS:
return False
if question_count >= MAX_QUESTIONS:
return True
return should_endFive lines, but it is the difference between an interview that ends after one question because the model felt done, and one that behaves predictably every time. The agent still emits a should_end signal, it just does not get the final word.
Decision 4: mock mode is the default, and that is the real cost story
AI_MOCK=true is the default. In that mode every AI call reads from a local JSON fixture instead of hitting the API, and each fixture is validated against the same Pydantic schema a real response would have to pass, so a drifted fixture fails loudly. Same pipeline, same code paths, zero API spend, no key required.
This is not a toy switch. The entire app was developed against it, which kept the whole build at zero cost while still exercising the full pipeline end to end. Flip AI_MOCK=false and it makes real calls, logs input/output token counts and cost to the database after every call, and prints a running cumulative total. So the cost discipline is not a claim you have to take on faith, it is visible in the code and in the logs. A full live interview runs in cents on Haiku.
Two smaller decisions in the same spirit: the model is Claude Haiku, chosen deliberately for the cost profile the task allowed, and the API key is read from SIGMA_ANTHROPIC_API_KEY rather than ANTHROPIC_API_KEY, on purpose, so it cannot be picked up silently by other tooling on the machine.
Decision 5: the backend is stateless, state lives in the database
A session id lives in the URL as a query parameter, and one endpoint returns the full session state (gap analysis, turns so far, current question) on every load. The frontend never treats its own component state as the source of truth, so refreshing mid-interview reloads from the database instead of losing progress. On the backend, each answer submission rebuilds the interview context from persisted rows, which means a server restart mid-interview loses nothing. Statelessness here was not for scale, it was for correctness: there is no in-memory session that can get out of sync with what the user sees.
Decision 6: a security fix nobody asked for
Production serves the built frontend through a hand-rolled catch-all route (the Next static export produces flat files, which Starlette's StaticFiles does not resolve the way I needed). A hand-rolled file route is exactly where path traversal creeps in: a request for ../../etc/passwd, or its percent-encoded form, would otherwise escape the static directory. The guard resolves the candidate path first to collapse any .., then rejects anything that lands outside the static root before serving it. Small, but it is the kind of thing a reviewer notices is missing, so I would rather it be there.
What I would improve with more time
Being honest about the edges is part of the point:
- Streaming responses instead of waiting for each full AI call to finish.
- Automated tests: the pure logic (the 5/12 boundary, history formatting) plus route-level tests against FastAPI's TestClient in mock mode, which would cover the session loop and the recovery paths in CI at zero API cost.
- Auth and accounts, so sessions tie to a user instead of a URL id. That also unlocks resuming an interview after a dropped connection.
- Bounded retry with backoff for transient API errors (rate limits, 5xx); right now only schema-validation failures retry.
- A managed database (Postgres or similar) with real migrations, instead of local SQLite with dev-time drop-and-recreate.
See it or read it
- Repo: github.com/Brafy11/ai-interviewer
- Quickstart: about two minutes from clone to a live interview (
QUICKSTART.mdin the repo).
The fastest way to judge it is live mode with a real key: upload a resume, give a deliberately vague answer at some point, and watch the follow-up.