AI Engineer Interview Questions — Senior Level

The word doing the work is shipped. They are separating people who have deployed something and watched it behave from people who have a notebook.

Give the shape in ninety seconds: what it does, who uses it, roughly how much traffic, and what you own when it breaks at 2am. If the honest answer is an internal tool used by nine people, say that — an internal tool that is genuinely in use beats a side project with a landing page, and everybody in the room knows it.

The second half is the question. Anybody can list model names.

Give a reason that has a constraint in it — cost per request, latency at the ninety-fifth percentile, a context window you needed, a data-residency rule that ruled out an API, an open-weights model because the workload could not leave your own infrastructure. Any of those makes you sound like somebody who chose. "It was the best one" makes you sound like somebody who read a leaderboard.

Small talk, and the most consequential question in the screen: the answer decides which loop they put you in. Get it wrong and you spend four hours being assessed against a job you did not apply for.

The distinction people actually hire on, stripped of the argument: an ML engineer is accountable for a model, an AI engineer is accountable for a system that contains somebody else's model. One owns the training data, the objective and the retraining schedule. The other owns retrieval, context, evaluation, failure behaviour, latency and the bill — and reaches for fine-tuning last rather than first. Neither is the senior one, and apologising for never having trained anything from scratch is a strange move in a job whose premise is that somebody else already did.

There is a third job hiding under the title. When those seven thousand postings were sorted by what the work actually is, about seven in ten were building features on foundation models, about a quarter were platform work for other people's AI teams, and roughly one in twenty was traditional machine learning renamed. If the advert talks about enabling teams rather than shipping a feature, expect infrastructure questions.

Say which one you are in a sentence, then say which parts of the others you can hold. That second clause is what stops it sounding defensive.

A currency check wearing friendly clothes. Have one real reversal ready with the reason attached — an approach you dropped, a tool you stopped defending, a technique you over-used. "Nothing comes to mind" is a bad answer in a field that moves this fast.

Give a researched range whose floor is a number you would sign. Titles in this field are noise — the same three words cover a €55k support role and a €190k platform role — so anchor on the scope you were just describing, not on the title in the advert.

Asked everywhere now, scored as a competency at several of the companies above, and answered badly in both directions. The candidate who says they do not use them sounds incurious. The candidate who says the tools write most of their code sounds like somebody who has never had to debug what came back.

The answer that lands names the split. Where you hand work over — scaffolding, tests from a spec you wrote, a first pass at unfamiliar syntax, migrations. Where you do not — anything you will be paged about at 3am, anything touching money, and the design decision itself. Then one sentence on how you check what comes back, because that is the actual competency and everybody skips it.

If you have been burned, say so. A specific story about a plausible-looking function that was subtly wrong is worth more than any position on the subject.

Everybody's demo works. The question is whether you have been through the part that comes after, when real inputs arrive and the distribution you designed for turns out to have been a guess.

They want three things: how you noticed, how fast, and what you changed so that the next one gets noticed sooner. The second is the one people leave out, and it is the one that separates a candidate who had monitoring from one who had a customer complaint.

It is worth seeing the two versions side by side, because most candidates give the first and think it landed.

A question you only get from somebody who has been burned, and a gift when you get it, because it invites the answer most candidates never think to give.

Good material: the model answered correctly and the answer was useless because it arrived after the user had left; the retrieval was right and the source document was two years out of date; the classification was accurate and the threshold sent everything to a queue nobody read. Say what you changed, and note whether the change was in the model at all. Usually it was not.

There is a specific person they are hiring against here: the engineer who says yes to everything and then quietly ships something that half-works with a disclaimer under it.

Give a request you refused or reshaped, the reason in one sentence a non-engineer could repeat, and what you offered instead. The offering-instead part is the whole answer — refusing without an alternative is not judgement, it is just refusing.

The integrity question, and in this field it has teeth, because the numbers are unusually easy to get wrong in your own favour without meaning to. Test examples that were in the few-shot prompt. An eval set built from the same documents the retriever indexes. A baseline you compared against that was never tuned. A judge model scoring outputs from its own family.

Pick a real one, say how it was caught, and say what changed in how you measure. Candidates who volunteer a contamination story are, in practice, the ones who have run enough of these to have found one.

Do not perform the eye-roll they are half-inviting. The workable answer is to convert the headline into a test on your own data within a day — cheap, specific, and it ends the argument with evidence instead of seniority.

Asked casually, and it is a real question about taste. A candidate who has deleted a reranker that was not earning its latency, or an agent step that was just a function call wearing a costume, has been through the phase of the work where things get simpler.

The canonical question of this role, and the reason it is canonical is that a candidate can talk for twenty minutes without saying anything checkable.

Start by refusing to build anything for two minutes. Who asks, and what do they do with the answer? How stale can a document be before the answer is harmful? Are there documents this user is not allowed to see — because if there are, permission-aware retrieval is the architecture, not a feature you add later. Those two minutes are themselves the answer — start drawing boxes instead and the mark is already gone.

Then the pipeline, briefly: ingestion and what you do about the twelve file formats nobody warned you about; chunking with a reason attached; embeddings and a store; a retrieval step you should assume is hybrid rather than pure vector; a reranker if the latency budget can pay for it; generation with the retrieved passages and their citations; and a refusal path for when nothing retrieved clears the bar.

Spend the last third on the parts that are actually hard: how you evaluate retrieval separately from generation, how a document that changes gets re-indexed without a nightly full rebuild, and what the user sees when the system does not know. A design without a refusal path is a design that will hallucinate in front of a customer, and the interviewer is waiting to see whether you bring it up unprompted.

A diagnostic question, and it is the sharpest one in the set, because it cannot be answered from a blog post. Either you have debugged one of these or you have not.

Do not produce a list. Produce an order.

First: is the passage actually in the context the model saw, or did it get truncated when you assembled the prompt? Log the final prompt. Half of these end here and everyone is embarrassed.

Second: is the answer in the retrieved chunk, or in the chunk next to it? A chunk boundary through the middle of a definition is the most ordinary cause of this failure and the least often suspected.

Third: is there a competing passage saying something else — an old policy, a superseded price — and the model picked the wrong one? That is a data problem wearing a model's clothes, and no amount of prompting fixes it.

Fourth: does the instruction actually tell the model to prefer the retrieved text over what it already believes? Models will happily answer from memory when the passage is thin.

Only then start suspecting the model.

A short question with a long trap. The wrong answer is a number.

The answer is that it depends on what a complete thought looks like in the corpus, and that you would look at the documents before deciding. Contracts chunk by clause. API documentation chunks by endpoint. Support threads chunk by message, and the summary of the thread is a separate document. Say you would overlap, say why, and say how you would test it — retrieval quality on a labelled set, not a feeling.

Three questions that are one question in different costumes, and you will usually get one, not all: which embedding model, which vector store, and whether you would add a reranker.

Prepare them together, because the honest answer to all three has the same shape: the choice barely matters at your scale, the switching cost is what matters, and the thing that actually improved results was not the choice. Say which one you used and what would make you change it — a language your embedding model handles badly, a corpus that outgrew a single machine, a latency budget that cannot pay for a second model pass.

The one place to be specific is multilingual retrieval. If the corpus is not in English, embedding quality varies enormously between models and it is the one part of this stack where the choice is genuinely load-bearing. An interviewer whose corpus is Arabic or mixed-script will notice you know that.

Asked to see whether you reach for the same hammer regardless of the nail.

It is the wrong answer when the question is aggregate rather than lookup — "how many contracts expire this quarter" is a query, not a retrieval, and a pipeline that answers it by stuffing forty chunks into a prompt will be confidently wrong. It is the wrong answer when the corpus is small enough to fit in context. It is the wrong answer when the task needs reasoning over structure the documents do not carry, in which case somebody should have built a table.

The word lie is doing deliberate work. They are not asking you to design a loop. They are asking what you do about the failure mode everybody has now seen: the model reports that it did the thing, and it did not do the thing.

The answer they are listening for is that you do not trust the model's account of what happened. The tool's return value is the truth; the model's narration is a claim. So: every tool call is logged with its arguments and its raw response, the final answer is assembled from the returns rather than from the model's summary of them wherever that is possible, and any step that touched money or state is verified by reading it back rather than by asking the model whether it worked.

Then the boring parts, which are where these actually fall over: a step budget so the loop terminates, idempotency so a retry does not double-charge anybody, a schema check on every argument before the call goes out, and a plan for the partial failure — two tools succeeded, the third did not, and the user is owed a truthful account of that rather than a cheerful summary.

If you have run one of these in production, say what it did wrong the first week. That sentence is worth more than the rest of the answer.

They want to know whether you have ever looked. A candidate who has will answer in components and will know which one is the surprise.

The shape: embedding the query, the retrieval call, the rerank if you have one, time-to-first-token, then generation which scales with the output length and not much else. The surprises are usually that reranking costs more than the retrieval it improves, and that a chatty system prompt is paid for on every single request. Say what you would cut first, and say that you would stream so that the perceived latency and the real one stop being the same number.

A real question, asked more often every quarter, and it is one of the few in this set with an order of operations that is close to universal.

Measure first — which route, which prompt, which tenant. Then: cache what repeats, and be specific about whether you mean an exact-match cache, a semantic one, or the provider's own prompt caching, because they fail differently. Then trim the context, which is usually where the money actually is: the retrieved passages are longer than they need to be and nobody has checked in months. Then route by difficulty, sending the easy majority to a smaller model and keeping the expensive one for the cases that need it — and say how you would know which is which, because that is the part people wave at. Batch anything asynchronous.

Then say the unpopular thing: some of the cost is a product decision, and if the feature is not worth fourteen thousand dollars a month the engineering answer is not the only one available.

Start by disposing of the false premise. Non-determinism does not make testing impossible; it makes exact-match assertions the wrong tool. Plenty of the system is still deterministic and should be tested as such — did retrieval return the document, did the tool get called with the right arguments, did the output parse as the schema you promised.

Then the graded part, in layers. Deterministic checks first, because they are free and they catch most regressions: format, schema, required fields present, forbidden content absent, citations resolving to documents that exist. Then reference-based scoring where you genuinely have references. Then a model-graded layer for the judgements a string comparison cannot make, with all the caveats in the next question. Then human review on a small sample, which is expensive and irreplaceable and the thing everybody drops first.

The two sentences that make this answer credible are that you fix the temperature and the seed where the provider lets you, so at least the test is repeatable, and that you run each case more than once and look at the variance — because a case that passes four times out of five is not a passing case, and a single run will tell you it is.

Then say where the eval set came from. That is the real question underneath, and it leads directly to the third question in this section.

Half a challenge. They are not against it — they use it — they want to know whether you know how it fails.

The case for: it scales, it handles judgements no string comparison can make, and it correlates well enough with human ratings on many tasks to be useful.

The failures worth naming, and naming two of them with a number attached is the whole test. Verbosity bias: in published measurements, judge models preferred the longer response upwards of nine times in ten. Self-enhancement bias: models rate their own family's output higher, measured at around ten percentage points of win rate for one model and twenty-five for another. Position bias: the same judge changes its answer when you swap which candidate comes first, and one widely used judge did so in roughly seven cases out of ten.

The mitigations are unglamorous and specific: a written rubric with binary calls rather than "rate this out of ten", because the difference between a three and a four is nobody's opinion twice; positions swapped and averaged; and a calibration set of a hundred or two human-labelled cases you re-run whenever the judge model changes underneath you, scored on how often it agrees with the human rather than on a single accuracy figure. Say the calibration part out loud. A judge you have never validated against a person is a number generator with good manners.

An invitation, not an attack — and you win it by attacking your own number before they do. The reasons a score like that is usually inflated, roughly in order of how often it is the culprit:

The set was drawn from the same place as the build. Sampled the week you built the feature, same tenant, same format. It measures the system on the world you already looked at.

The examples leaked — into the few-shot prompt, or out of documents the retriever indexes, so the system is reproducing something it can see.

Public benchmark contamination. If any of it rests on a well-known public set, the model may have met that set in training and the score is partly memory. That is why a private, freshly written, held-back set beats a large one.

The judge was never calibrated, so 94% is one model's opinion of another model's output with no person anywhere in the loop.

The failures are not uniform. The 6% is a cluster — long inputs, one language, one document type — and an aggregate hides precisely the thing that will hurt you.

Then say what you would want instead: a held-out set built after the system was, stratified by the dimensions you expect to matter, with the failing cases read rather than counted. And have this ready for the follow-up about why evaluation work takes so long — practitioners who publish on it put the share of development time spent on error analysis and evals at sixty to eighty percent.

One story makes the point better than the argument. When a frontier lab open-sourced a take-home it had used for two years, it reported that on the first day after release, no submission below a certain score was actually valid: in every case a model had modified the tests to make the problem easier. The scores were real. The results were not.

Prompts are code that nobody version-controls, and this question is aimed straight at that.

Prompts in the repository, changed by pull request, with the eval suite running in CI on the diff. Then the part that makes it real: a report of what changed per case, not an aggregate, because an aggregate that stayed at 91% can be hiding nine fixes and nine new breakages. If you have ever shipped a prompt change that fixed the reported problem and broke something nobody was watching, say so.

The answer that lands is a list you could actually debug from six weeks later: the input, the retrieved passage identifiers and their scores, the assembled prompt, the model and version, the parameters, the raw output, the latency split by stage, the token counts, and the trace identifier that ties all of it to the user's session.

Then the sentence that separates you from a candidate reciting a list: what you do about the personal data in there. Redaction before storage, a retention window, and the fact that a debugging log full of customer messages is a data-protection problem before it is an engineering one. In some markets it is also a legal reason you cannot send that traffic to an API outside the country at all.

Not a number in the abstract — a threshold tied to what a failure costs and what the fallback is. "Ninety percent with a refusal path and a human queue" is a real answer. "Ninety-five percent" on its own is a number somebody made up in a meeting.

Soft phrasing, real question. What the interviewer wants is one sentence proving you know the rules attach to the use, not to the technology — the same model is unregulated in one product and high-risk in another.

And there is a trap in it right now. The AI Act entered into force on 1 August 2024 and applies in stages: prohibitions and the AI-literacy duty from 2 February 2025, general-purpose model obligations from 2 August 2025, general application from 2 August 2026. Almost every guide written before this summer then says the high-risk regime landed with it in August 2026. That is no longer the timetable. An amending regulation — the AI Omnibus — entered into force on 27 July 2026 and pushed the obligations for stand-alone high-risk systems to 2 December 2027, and for high-risk AI embedded in products already covered by EU product-safety law to 2 August 2028.

Why that matters to you rather than to a lawyer: recruitment and candidate-evaluation systems sit in the high-risk annex, alongside education, biometrics and credit — and so do systems that monitor how people work. If the company builds hiring software, HR tooling or anything that scores people, the deferral is a live topic in their planning meetings, and "high-risk obligations started in August 2026" tells the room you read an article and stopped.

Then land it on engineering. The obligations are work you have either done or not: documented data governance, logging you can produce on request, human oversight that is real rather than a checkbox, and technical documentation written before deployment instead of after an auditor asks. Name which of those you have built, and stop there — nobody expects article numbers, and candidates who try usually get one wrong.

The question this set exists for.

There is a specific person being hired against: the engineer who raised a concern in a document nobody read, shipped anyway, and now describes that as having flagged it. Flagging is not refusing.

What earns credit is a refusal with a cost attached. You held a launch, or took a feature out of one, or said no about something that was not strictly yours to decide. Say who was unhappy about it. Say what it cost — a date, a relationship, a quarter — because a refusal that cost nothing was not a refusal, it was a preference.

Then say what you built so the next one would not need a refusal at all: a threshold, a queue, a check that runs before release rather than an opinion voiced during it.

An organisational question that arrives looking technical, and the answer separates somebody who ran an evaluation practice from somebody who ran an evaluation.

The interesting half is what happens when the set disagrees with a person — a product manager who wants the feature out, a founder who tried it once and liked it, a customer whose complaint is real and is not in the set. Say who can add a case, who can remove one, and whether removing one requires anybody's agreement. A set that anyone can quietly prune measures nothing by the end of a quarter.

The strongest version of this answer names something you gave up: a case you agreed to drop and why, or a threshold you moved under pressure and what you asked for in exchange.

If you have run one of these for long enough, this has already happened to you, and the question comes from somebody it has happened to as well.

A migration plan is the answer anybody can produce. What they are listening for is that you know the migration is not the hard part.

The hard part is that your eval set was tuned against the old model's failure modes, so a straight swap gives you a number that is not comparable to the one you have been reporting all year. Say that first. Then: run both against the same set in parallel, look at the cases where they disagree rather than at the aggregate, and expect prompts to need rewriting, because the instructions you sharpened were sharpened against one model's particular stubbornness.

And say what you would tell the business, and when. A deprecation with three months' notice is a schedule you own. A deprecation you mention in month three is a surprise you caused.

Not a question about the regulation, which the section above handles at length and where the dates have already moved once. This is a question about what exists in your repository.

So answer in artefacts. What you log and for how long, and what gets stripped before it is written. What the system refuses to answer, and how the refusal is actually implemented — a classifier, a threshold, a list, or nothing. Who looks at the cases where it refused wrongly. What a person can do when the output is wrong, and whether that path exists in the product or only in your head.

Then the sentence that makes the rest credible: name the gap. There is always a gap, and a candidate who describes a complete posture is describing a system they have not operated.

Keep the legal framing short. The obligations move, and the counsel at the company interviewing you has a more current reading of them than you do.

Asked because a prompt change can reach production without the review a code change would get, and the answer says whether prompts are code where you work or a text box in somebody's dashboard.

The workable answer: you do not read the diff and form an opinion, because nobody can. You ask what it was meant to fix, whether a case in the eval set fails before and passes after, and what the per-case report says moved elsewhere. If the answer to the second is no, the change is not reviewable and the review is theatre.

Knowing the number is not the skill. Owning it means having explained it to somebody who does not care how it works, and that is what is being asked.

Give the figure, the direction it was moving, and the thing you had to say out loud — that a feature was not worth its bill, that one customer was unprofitable, that the cheapest available fix was making the product do less. Then say what was decided, including the times you lost the argument.

Three questions that arrive together when the job is inheriting something rather than building it, which is what a senior opening on an existing system is: what would you read first, what would you turn off, and what would you refuse to promise in your first month.

Prepare them as one answer with three exits. What you read first should be operational rather than architectural — the logs, the last three incidents, the eval set if one exists, and the bill. What you would turn off is a real question with small honest answers: a reranker nobody measured, a retry loop that makes incidents worse, a scheduled rebuild whose output nobody reads.

The third is where senior candidates lose it. Refusing to promise a number before you have seen the system is correct, and saying so without sounding evasive takes one specific sentence: name what you would measure in week one, and the date you would come back with a number.

A bus-factor question asked gently. If the answer is nobody, say nobody and say what you have done about it — the system you would be inheriting there has the same problem, and that is what they are hiring against rather than the number itself.

The taste question. Naming a sophisticated architecture is easy at this level; what is scarce is somebody who declined to build one and can say why without sounding defeated.

Good material: keyword search that turned out to be good enough, a queue instead of an agent, a lookup table instead of a model, one prompt instead of a chain of four. Give the thing you gave up and the thing you bought — and what you bought is a system somebody else can fix at two in the morning.

Refusal without the story attached, asked to find out whether you have a position or only a toolkit.

Have two ready. One where the cost of being wrong lands on a single identifiable person rather than being spread thin across many. One where the task is really a query over structured data that nobody has got round to modelling. Say what you would build instead in both cases — refusing without an alternative is not judgement at this level either.

This is a people question, and answering it purely technically is the failure.

The answer that lands names the cheapest experiment that would settle it — a hundred labelled cases, both approaches, a day — and says who decides if it does not settle it. Then the part senior candidates skip: what you would do if they turned out to be right. An engineer who has never been overruled by their own evidence is describing a team where the evidence never actually runs.

Removing a feature, a model, a pipeline or a metric costs political capital and earns no credit, so naming one means you stayed somewhere long enough for your own decisions to come back to you.

Asked of somebody who will be interviewing, because a senior hire does, and the answer describes your model of the job more honestly than anything above it.

Say what you would test and what you would refuse to test. Somebody who would run a four-hour algorithm loop for a job that is mostly retrieval and evaluation has just told you what they think the job is. Name the round you would add, name the one you would cut, and say what you would do about candidates using assistants — it is the live question, and having no view on it has itself become a view.