Adrián Sacristán

Private beta

Benchd

Benchd is a voice mock interview for commercial roles in hospitality tech: sales, partnerships, account management and leadership. A candidate sets a target role and seniority, can add a CV and a job description, and sits a timed spoken interview. The report scores six competencies and quotes the candidate's own words.

Benchd voice mock interview for a Partnerships Manager role, with a live transcript and interview stages

The problem

Practising a commercial interview with a friend produces a feeling, not a rubric. Benchd keeps the questioning, the transcript and the score in one system, so the feedback is evidence from that conversation.

What the product does

  • A marketing site with a simulated interview preview, a how-it-works section, an FAQ, and a beta interest form that writes to interest_signups.
  • Email and password, plus Google and LinkedIn. A legal gate records age confirmation and acceptance of the terms before the product opens.
  • Onboarding is one screen: name, target role, seniority. A CV can be a PDF, DOCX, TXT or Markdown file up to 5 MB. Text is extracted locally. A pasted job description can be turned into title, company and seniority by one model call that stores nothing.
  • A new interview snapshots the job. Later edits to the job do not rewrite a past report. The clock starts on the first voice connection or answer, so consent and the microphone check do not eat the twenty minutes.
  • The call room shows captions in step with the audio, a voice-state pill, barge-in, and retry. After the call, the report has an overall score, six weighted categories, per-answer reviews, strengths, improvements and a practice plan. A failed analysis is shown as unscored, not as a zero pretending to be a judgement. Progress over attempts lives on a dashboard with a score trend, a skill radar and badges.

Architecture

It is a pnpm and Turborepo monorepo. apps/landing and apps/app are Next.js. apps/api is FastAPI with SQLAlchemy 2, Alembic and Pydantic v2. Shared TypeScript packages are consumed as source, with no separate build, through Next's transpilePackages. The browser talks to the API with a cookie session and TanStack Query. Zustand holds only wizard and draft UI state.

Benchd architecture The Next.js landing and candidate app call a FastAPI service. Voice audio goes to ElevenLabs over a signed WebSocket. ElevenLabs calls Benchd's custom language-model endpoint, which runs the interview engine and OpenAI. A signed webhook then stores the transcript and queues the report. Postgres and CV storage sit behind the API. Landing Candidate app FastAPI REST, custom LLM, webhook Interview engine one brain, every mode Postgres RLS on, 18 migrations ElevenLabs agent signed URL, Flash v2, turn model OpenAI gpt-4o-mini strict JSON schema Grounding, then the report CV files: local disk or Supabase Storage
Audio and turn-taking sit with ElevenLabs. The questions, the transcript and the score stay in Benchd.

Voice and the interview engine

Text chat, a legacy speech path, and ElevenLabs Conversational AI all call the same two functions: save the answer, then ask the next question. The default voice path is a signed WebSocket from the browser to ElevenLabs. ElevenLabs then calls Benchd's OpenAI-compatible endpoint, so every spoken turn still runs through Benchd's engine. The browser client is custom. It captures the microphone, downsamples to 16 kHz, and turns that into 16-bit PCM. Captions follow alignment data, so they match what was spoken, including after a barge-in trims the turn.

The engine plans stages from the CV, the profile and the job description, then asks one follow-up that names a detail from the answer. Invalid or failed model output falls back to a transcript-aware line rather than a generic prompt. On the voice path, provider retries are off, so a waiting caller is not stacked with timeouts. Before the model answers, the stream sends a short spoken acknowledgement so synthesis can start, and that filler is stored on the turn so the transcript matches the audio. Concurrent retries take a row lock on the session. A stable answer id, hashed from the shape of the turn, means a repeated call does not advance the interview twice.

There is no vector store. The CV summary, the job description, the question plan and a question bank go straight into the prompt and the agent's dynamic variables. The agent configuration itself lives in the repository as JSON: English, a patient turn model, a 15-second turn timeout, backchannels ignored as interruptions, a 25-minute ceiling, and a skip-turn tool when the candidate asks for a moment. A sync command can show a dry-run diff or apply it, and it only patches the agent with the expected name.

A spoken turn

Benchd voice turn The browser asks the API for a signed session. Audio goes to ElevenLabs. ElevenLabs posts the conversation to Benchd's custom language-model endpoint. Benchd locks the session, calls OpenAI for the next question, and streams it back. After the call, a signed webhook stores the transcript and the report is scored, then repaired by a grounding pass. Signed session URL Microphone audio to ElevenLabs Custom LLM endpoint merge fragments, hash the answer id Early acknowledgement, then the question Row lock, save turn, log the call HMAC webhook after the call Score, then grounding
ElevenLabs handles the audio. Benchd decides the next question and keeps the transcript idempotent.

Scoring

Only a real interview is scored. Clarifications, apologies and fragments are not treated as answers. The rubric always emits all six categories: Communication Clarity and Commercial Reasoning at 20 each, and Leadership, Stakeholder Management, Strategic Thinking and Interview Readiness at 15. Analysis runs at temperature 0 with a fixed seed and a strict schema. A deterministic pass then repairs the model's draft: quotes must be the candidate's words, generic praise is removed, delivery is scored apart from substance, and empty answers are capped. A persistent failure is stored as unscored. Reports keep the prompt, rubric and analysis versions, so a later regenerate is explicit.

A calibration command can score a pack of benchmark interviews against the live model and write a report. In continuous integration, recorded proxy outputs are replayed through the grounding code, so that repair logic is tested without calling a model. Every model, speech-to-text and speech-synthesis call writes latency, tokens and an estimated cost. Those figures are telemetry, not invoices. There is no external error tracker in the repository.

Data, security and delivery

Eighteen Alembic migrations build the schema, and a test refuses more than one head. Sessions store only a hash of an opaque token. The cookie is HttpOnly. Passwords are bcrypt. OAuth state is signed. Row Level Security is enabled on every public table, and Supabase's anonymous and authenticated roles are revoked, so the Data API cannot read them. The API connects as the owner. OpenAPI docs are not mounted in production. Interview audio is streamed and dropped. It is not stored. Voice telemetry is purged after 90 days. Inactive accounts are flagged and later deleted. The warning email for that path is not wired yet.

The API is a Render web service that migrates, then starts. The frontends are on Vercel. GitHub Actions, pinned to commit SHAs, runs lint, typecheck and the app tests, and runs the API suite against Postgres 16 with a fake model and fake speech services. There are 39 pytest modules and 47 frontend test files. There is no browser end-to-end suite, and Python lint is not in that workflow.

Where it is going

The product is in early beta, with auth, consent, privacy controls, an admin console and cost telemetry already in the path a candidate uses. The code points next at a staged switch of the voice backend, outbound mail for inactivity warnings, and cohort comparisons once there is a cohort. Peer benchmarking is not live.

Open Benchd