Stop interviewing engineers like it's 2022
Keep, kill, add - the AI-era interview playbook for leaders.
👋 Hey, it’s Stephane. I help engineers become great engineering managers - whether you want to become one or are already leading a team.
To accelerate your growth see:
Get Hired as an EM (CV & job search to actually get interviews)
AI Behavioral Interview Coach (the best prep for your interviews)
Paid subscribers get 50 Notion Templates, The EM’s Field Guide, and access to the complete archive.
In the second half of 2025, candidates using unauthorized AI in coding interviews jumped from 15% to 35%. In technical roles specifically, it hit 48%. And 61% of those cheaters passed to the next stage.
Six in ten people who cheated successfully progressed.
If your reaction is “we need better proctoring”, I think that’s the wrong instinct. You’d be racing against invisible screen overlays that the candidate’s grandmother could install in five minutes.
The companies that have actually fixed this didn’t fix detection. They redesigned their interview completely.
This is what’s happening today. I’ll give you a simple “keep, kill, add” that works for companies like Google, Canva, Shopify, Meta, and Anthropic.
The wrong instinct (both versions of it)
Engineering leaders have two instincts.
Instinct 1: Ban it. There is software that create invisible overlays which reads the interviewer’s audio and feed the candidate a responses in under two seconds. And although you might think you’ll easily catch it, you might be optimising for the candidates who are best at cheating.
Instinct 2: Allow it on your existing interview questions. Google ran the experiment for you, and the lesson, is that if all you do is allow AI on your existing problems, you’ve changed nothing.
Where the serious companies actually landed
Look at who has moved publicly and where they ended up.
Google is piloting an AI assistant (Gemini) inside the coding round itself. The format is a three-panel layout: file explorer, code editor, AI chat. The AI can’t edit files. The round focuses on reading, debugging, and optimising existing code, not writing from scratch. (Exponent) The signal: the future is “review and verify”, not “produce”.
Canva killed its “CS Fundamentals” round in June 2025 and replaced it with an AI-Assisted Coding round. Candidates are expected to use Copilot, Cursor, or Claude. (summary) The new questions are “complex, ambiguous, realistic”, which is the opposite of one-shot prompts.
Shopify lets candidates bring their own AI tools. The interviews require integrating AI-generated snippets into unfamiliar codebases. (Hello Interview) Reading and grafting code matters more than writing it.
Meta runs an AI-Enabled Coding Round with a rubric of problem solving, code quality, verification, and communication. (Hello Interview) Communication is back on the scorecard.
Anthropic reversed an early ban. AI is allowed for the application and prep, but forbidden during the live round. (Fortune) Even AI companies decided that uncontrolled AI in live interviews is a signal problem.
Nobody serious is doing “business as usual”. The actual split is between allow with structure (Google, Canva, Shopify, Meta) and block during live, allow around it (Anthropic).
Keep / Kill / Add.
At the recruiter screen, keep the 15-minute phone call. Kill the “do you use AI tools?” question, because everyone says yes. Add one specific prompt instead: “Walk me through the last time AI got something wrong for you. What did you do?”
For the CV and portfolio, keep the GitHub link and prior impact. Kill LeetCode contest scores and hackathon badges. Add one required line in the application: a 200-word “how I work with AI” statement.
At the first technical round, keep a live, conversational format. Kill the 45-minute algorithm puzzle. Add a 45-minute read-and-fix round on a real, messy file from your codebase.
The take-home stays gone. Keep nothing. Kill the take-home, all of it.
At the onsite coding round, keep one pair-programming session. Add a live AI-assisted round, where you score them on direction, not output.
At the onsite system-design round, keep the whiteboard. Kill “design Twitter from scratch”. Add a 45-minute “review this architecture and tell me what’s wrong” round.
For the onsite behavioural round, keep your real values prompts. Add: “Tell me about a time you chose not to use AI on something. Why?”
A few of those are spicy. The take-home being killed is the spiciest. If you want to know why, ask yourself: when a candidate hands their take-home to Claude and the result is good, what did you actually measure?
The five signals that survive
If you only redesign one thing, redesign your scorecard. The signals every serious company is now scoring converge on five dimensions:
1. Problem decomposition. Before they touch a keyboard, do they clarify scope, edge cases, and constraints? Or do they jump straight to prompting?
2. Control over the AI. Are they directing the model, or being led by it? Strong candidates write targeted prompts, reject unhelpful suggestions, and choose when not to use AI at all.
3. Verification habits. Do they read the diff line by line, run the code, write a test, catch the hallucinations? Or accept the first plausible output and move on?
4. Architectural judgment. Can they reason about blast radius, cost, scale, trade-offs? This is the hardest thing to fake with AI in 90 seconds, which makes it the most valuable thing to test.
5. Communication. Can they narrate while juggling the AI? Can they hold a conversation with you while typing? Or do they go silent and emerge with a finished blob?
The signals you stop scoring matter too. Clean variable names. Speed to first plausible answer. Syntactic elegance. AI does all of those for free now.
I wrote more about this shift in Stop interviewing like a mid-level engineer and Why you can’t hire great senior engineers. Both rest on the same idea: you’re hiring for judgment, not output.
The junior pipeline
The LeadDev AI Impact Report 2025 found that 18% of engineering leaders expect to hire fewer juniors in the next 12 months. 54% think AI will reduce junior hiring long-term. 38% say AI has already reduced mentoring.
If you redesign your loop only around “judgment and verification”, you’ve designed juniors out of it.
You can consider:
Separate the junior loop. Same five signals, applied at a junior level. Junior verification looks like “did you actually read the code?”, not “can you spot a subtle architectural flaw?”. Make them clear a junior bar that still maps to the real signals.
Score learning velocity explicitly. Add a 20-minute round where you teach the candidate something new on the call and watch how they use it. This is the round AI can’t help with.
I’ve made this case in different shapes in 11 Interview questions I ask every engineer and The new engineering manager hiring bar.
A 90-day rollout plan
You don’t need executive sponsorship for this. You need a few months to put it in place.
Weeks 1 to 2: Audit. Pull the last 20 hires through your current loop. For each one ask: did the loop actually surface the five signals? Where did it miss? Where did it overweight noise?
Weeks 3 to 4: Redesign the pairing stage. Take your existing questions and replace them with read-and-fix problems pulled from your real codebase.
Weeks 5 to 6: Retrain interviewers. New questions with old interviewer instincts produce old outcomes. Two 90-minute sessions: one on scoring AI usage, one on running the new format.
Weeks 7 to 8: Calibrate. Run the new loop on three internal “candidates”, engineers from another team. Score them, compare scores across interviewers, and adjust the rubric where you diverge.
Weeks 9 to 12: Ship and measure. Run the redesigned loop. Track: offer-to-acceptance rate, 90-day performance correlation, time-to-decision.
What this looks like in 2027
Here’s where I think we’re headed.
In eighteen months, the live coding interview is no longer a coding interview. It’s a collaboration interview. You and the candidate sit in front of a real problem, both with AI tools, and the candidate’s job is to drive while you watch. The whiteboard system-design round becomes the most predictive round in the loop, because it’s the one AI can’t take over.
Take-home tasks are completely gone. Algorithm puzzles are gone. Behavioural rounds get harder, not easier. The candidates who win are the ones who can hold a conversation, hold a system in their head, and hold the AI accountable.
Most leaders will arrive there in 2027 because they were forced to. The leaders who arrive in 2026 will be the ones who hired the best people for an entire year.
The one thing to take away: Your interview loop has a redesign deadline, and it’s not “when you have time”. Pick the keep/kill/add row that’s hurting you most and change one thing this month.
P.S. If you want to see how this connects to onboarding (because a bad redesigned loop still misses the next Soham), I wrote about that here: Would your onboarding process catch a Soham?. The interview is the front door. Onboarding is the test of whether the door worked.
If you enjoyed this article, consider subscribing to get:
✉️ Free: 1 original post every Tuesday, my favourite posts of the week every Sunday + 10 Notion Templates for Engineering Managers
🔒 Paid: Full archive + 50+ EM templates & playbooks + The EM Field Guide
See you in the next one,
~ Stephane