Recently, I was tagged on a LinkedIn discussion about system design interviews.

The interviewer had a problem I suspect a lot of technical interviewers are having these days. A candidate proposes a technology or vendor that you don’t know particularly well as a solution to your question and makes a factual claim about it. For example, that a database supports multiple secondary indexes efficiently or a vendor does a specific part of the desired workflow perfectly.

You have no idea if that’s true. Do you challenge them? Nod along? Google it while they’re talking?

The factual claim that they are making isn’t usually the interesting part. Whether it’s true may matter to the design, but I don’t need to know the answer yet.

If the candidate says, “I’d use X because it supports efficient secondary indexes,” I can treat that as an assumption in a design they are thinking about and start digging a little deeper. I can always verify a specific feature later if I’m curious or it ends up being important. During the interview, I’m much more interested in what happens when I pressure test the reasoning.

System design interviews should be testing engineering judgment, not shared trivia.

I’ve thought about this distinction for years, but AI makes it even more relevant.

When you give an AI a half-baked idea, there’s a good chance it will hand you back something much cleaner than what you gave it. The assumptions become organized, the language gets sharper, and the missing transitions appear between ideas. Depending on the model and how you ask, it may also politely reinforce the premise you started with instead of telling you about any gaps, or even that the premise might be faulty to begin with.

More often than not, this is an incredibly useful capability that AI provides. It’s a big part of my toolkit and I use it often to help me clarify my thinking.

It can also be a problem if you are interviewing someone.

AI can improve the presentation of an idea much faster than it improves someone’s understanding of an idea.

For a long time, those things were more closely connected. If someone could walk me through a coherent technical design, there was at least a reasonable chance they had spent enough time building things, breaking things, or thinking about similar problems to understand why the design worked and where it could fail.

The polished answer was never proof of understanding. It was a correlation, and it held up well enough and for long enough that most of us stopped noticing it was a correlation.

AI broke that correlation enough to expose how much of the interview depended on it. We thought we were measuring understanding, but part of what we were measuring was whether someone could produce the kinds of outputs we associated with understanding. It was, for a long time, a reliable proxy.

There’s a quote I’ve loved for years: “Experience is the knowledge you get just after you need it.”

A lot of engineering experience works exactly like that. You make a decision. Six months later you discover the thing you failed to consider. You build a system that works beautifully until the access pattern changes. You discover that the clever abstraction nobody worried about is now the thing stopping the team from shipping. You choose technology based on a capability you thought mattered and learn that the operational cost mattered much more.

You remember those lessons because they hurt. Pain is a great teacher.

AI can now give someone many of the outputs associated with those lessons without requiring them to have gone through the painful part first.

It can give you a list of tradeoffs. It can tell you what tends to fail and why. It can give you a sensible set of architecture choices and explain the pros and cons of each. It can turn a messy thought into a convincing recommendation.

That doesn’t mean the person reading the answer has developed the judgment that would have produced it.

It also means that two people can now produce surprisingly similar-looking answers. One might understand why the answer works and be able to defend it under scrutiny. The other may be very good at using AI to produce something polished without having the same depth underneath it.

Both of these people can be valuable hires. I’m not trying to prove every candidate has to have deep understanding. But I do want to know how much understanding sits underneath their output, and whether that’s what the role requires.

Boring Interview Problems

One of my favorite ways to interview engineers and technical product people is to ask them to design a system for managing a library.

There’s nothing particularly clever about the problem. That’s part of why I like it. It’s simple on the surface, but like any problem, reality has a surprising amount of detail (highly recommend reading this short essay if you’ve never read it).

I don’t really care whether someone can produce the ideal library architecture. I care about their thought processes and where they look first. I ask them to make enough of their reasoning visible that I can follow the choices they’re making and poke at the ones that help me determine their level of understanding.

Do they ask what already exists before deciding what they might need to build? Are they aware of existing library systems like the Dewey Decimal System or are they about to reinvent one? Do they think about the librarian operating the system or the person asking for a book? Do they think about librarian edge cases like removing books, managing damaged inventory, or transferring something between branches? Do they think about book reservations? Search? Late returns? Multiple copies of the same title with different quality levels? Digital books? Or do they immediately start drawing service architectures? None of these answers prove anything by itself. But they do give me something to pressure test.

That was a lot of example questions and there are going to be a lot more. They aren’t specifically an interview script. But I do want to show examples of how I’m digging deeper into a topic. The specific questions you are asking matter less than knowing the reason that you are asking them. You don’t need an endless library (pun intended) of interview questions if you’re willing to keep asking why.

I want to be clear that I’m not evaluating someone’s entire solution. I’m seeing what problems they think through and how they think them through.

Eventually, with an engineer, we’ll get into the technical decisions. What kind of database are they using? What are the access patterns? What is the scale? Why that database? What kind of indexes do they need? Are they building one application or separating out parts of the system? Where do they think the boundaries are? Which parts need strong consistency? Where would additional complexity actually buy them something?

With a product manager, I stay much longer in the workflows, assumptions, users, and prioritization.

The point isn’t the library, the vendor, or the specific tool. It’s giving someone a whole problem to sink their teeth into.

I could use a coffee shop. Or a pizza shop. Or a video rental store (I know I am dating myself with this reference, but I had to make one with COBOL in college and it’s still a great end-to-end problem to think through). I could ask the candidate to choose whatever business they understand and walk me through the capabilities it needs.

In fact, if someone knows in advance that I use the library problem, I don’t particularly care.

They can prepare. And if you’ve never done this as a thought exercise, I highly recommend giving it a try.

A system design question shouldn’t depend on surprising the candidate. If knowing the question in advance destroys the interview, there’s a good chance you were testing recall rather than judgment.

Someone can easily prepare a polished library architecture if they know that it’s going to be part of the interview. But if that prepared architecture survives scrutiny, who cares that it was prepared? And if they can do that convincingly, preparation isn’t a problem anymore (it’s actually a benefit knowing that they know how to prepare). They’re ultimately demonstrating the understanding I wanted to see in the first place.

Why did you separate those two concepts/systems?
Why wouldn’t you just use what the library already has?
You said search is the most important workflow. Why?
What happens if the inventory data is stale?
Who is allowed to change this?
You chose this database. What property of it do you actually need?
What would have to become true before you’d split this system apart?

None of these interview mechanics falls apart if a candidate proposes a technology I’ve never used or heard of. If the whole system design collapses when I question one database feature, that tells me how much weight the assumption was carrying. If they know three other ways to solve the same problem, that tells me something else. If they realize halfway through that they’ve optimized for a requirement that doesn’t matter, better still.

I’m not trying to catch them being wrong. Good engineers are wrong all the time. I want to know whether they understand which parts of their design depend on being right.

I also want to hear an occasional “I’m not sure” or “it depends,” once we get to a certain level of depth. “I think this database supports that, but I’d need to verify it before I committed to the design” is a perfectly reasonable answer, and it’s more reasonable now than it used to be. Pretending to know something you could easily check in thirty seconds isn’t impressive.

Let them use AI

I also don’t think technical interviews should pretend people aren’t going to use AI once they have the job.

If somebody is going to use AI while designing systems, working through product decisions, writing implementation plans, or checking technical details, I’d rather see some version of how they actually work.

So I’m fine telling someone: use AI during the interview.

But you’re going to need to “check its work.”

I’ll also take notice of how they prompt and how they work with AI. Some of that is useful to see even if it isn’t the main thing I’m looking for.

I’m interested in what they ask.

I’m interested in what they don’t ask.

I’m interested in which answer(s) they accept immediately and which one(s) makes them suspicious.

If AI proposes a technology, can the candidate explain why it belongs there?

If AI gives them five alternatives, does the candidate have a way of narrowing them down?

If AI contradicts something it said ten minutes earlier, does the candidate notice?

If the answer sounds polished but doesn’t fit the system we’ve been discussing, can the candidate say no?

AI can produce answers. Judgment shows up in which answers (and parts of answers) someone does and does not accept.

AI also still has pretty predictable defaults. Ask it to design something and you’ll get a design that works before you get one that performs. Cost, operability, migration paths, and what happens when the data model has to change tend to show up only if you ask for them.

Those gaps are useful to poke at because they are the places the candidate has to supply their own thoughts and experience. If a design handles the functional requirements cleanly and nobody has mentioned what happens at ten times the volume, I know where I can point the next question.

There’s another useful constraint here: interviews happen in real time.

A candidate cannot practically send every thought to an AI, wait for the answer, read it, decide whether it makes sense, all the while continuing to have a coherent hour-long technical conversation.

They have to choose what’s worth asking. And what they choose to ask is also very insightful.

An experienced engineer might say, “I’m comfortable with the persistence model, but I’m not positive whether this database supports the query model and indexing behavior I’m assuming. I would need to check on that.”

That tells me they have a model of the system in their head and have identified some specific uncertainties.

Someone else may need to ask AI which database to use, then what access patterns matter, then which index to create, then what alternative architecture they should consider.

Both people are using AI. They are not demonstrating the same thing.

How someone asks AI for help tells you something about what they know they don’t know.

For the kind of judgment-based interview I’m describing, that often tells me more than whether they remembered the exact answer themselves.

Experience Changes What Makes You Nervous

There’s another set of questions I like asking that AI makes more interesting, not less.

What has bitten you in the ass before?
When have you made a choice like this and regretted it?
What technology or operating model has surprised you?
What do you look at differently now because of something that went wrong previously?

AI can give you a list of failure modes. Experience changes which ones make you nervous.

If you’ve operated a distributed system that failed in an ugly way, you may react differently to a design pattern that introduces another network boundary. If you’ve watched a (supposedly) temporary customer customization become permanent infrastructure, you may ask different questions about a proposed exception. If you’ve spent six months trying to unwind a data model that was optimized around the wrong access pattern, some decisions will bother you earlier than others.

That doesn’t make experienced people automatically right. Sometimes scar tissue creates bad instincts too. Someone can overlearn a lesson from one company and apply it everywhere.

But hearing categorical statements like that often gives me another thing to poke at:

You seem really concerned about this particular failure mode. Why?
How likely is it in the example you just gave?
What would convince you that this situation is different from the one you experienced before?

That’s judgment too: not just having prior experience, but knowing when your prior experience should be challenged or heavily applied.

It’s worth remembering whose scars we’re talking about. Someone who has shipped a lot of AI-generated code has been burned in ways others haven’t. They know which things the model gets confidently wrong, what happens to a codebase when you accept a refactor you didn’t read, and where generated code quietly rots. That’s also valuable experience to have. It’s newer, and the questions that surface it are different.

Not Just a Technology Problem

None of this is unique to system design interviews or to engineering.

It’s a useful pattern for assessing judgment, experience, and insight anywhere. Give a product manager an ambiguous problem and ask what they’d want to understand before proposing anything. The questions change. But how to test assumptions doesn’t.

Take a non-technical example. Give a marketing candidate a business with a product to sell and ask them to design a campaign around it. You can even let them choose the business or the campaign goals, giving them the best chance of success. This also could work really well if they pull on something that they’ve already done and you can poke at the reasoning.

Why those channels?
What are you measuring and why?
Which early numbers would make them excited?
Which KPI are you now skeptical of and what happened?
What have they seen produce great-looking campaign metrics without producing much business value?
What has gone wrong before?

AI can make all of those candidates sound smart. It can generate a clean marketing plan, an implementation checklist, a product strategy, or a technical architecture. The interview needs enough messiness and ambiguity to find out whether the person can tell you why any of it makes sense or doesn’t.

The New Bad Proxy

There can be a risk in interviewing this way.

Once you ask candidates to reason out loud, use AI, absorb answers, and react to follow-ups in real time, fast verbal processing starts looking a lot like judgment. They aren’t the same thing.

Some people can hear a new constraint, think about it, and start forming a useful answer almost immediately.

Others need time to think and process.

They may be every bit as good at the job that you are interviewing them for. They just don’t do their best thinking while someone is staring at them waiting for the next sentence.

Interviews are already a medium that tends to reward people who think out loud comfortably and operate well on the spot. Adding, “show me how you work with AI in real time” can make that bias worse if you’re not careful.

I think it’s worth asking people directly. Are you someone who tends to process by talking, or do you generally need a little time to think before answering?

Neither answer bothers me. Organizations need people who can react quickly in a live incident or a difficult customer meeting. They also need people who will go away for an hour, think carefully, and come back having noticed the thing everyone else missed.

Don’t replace one bad proxy for competence with another. Fast processing isn’t the same thing as good judgment.

As an aside, there’s a scaling problem with this kind of interviewing too. A rubric travels. Judgment doesn’t. If multiple people are running these interviews, I want feedback that shows where the interviewer pushed and what happened when they did. Where was the candidate uncertain, what did they check, what did they challenge, what did they get wrong and then catch? “Strong judgment” in the candidate summary doesn’t tell me any of that. While a full interview transcript can reduce the interviewer burden of remembering the conversation, it doesn’t tell me which moments the interviewer thought mattered. And none of this should turn into a documentation tax on the interviewer.

What I’m Actually Measuring

The outputs still matter. The code still needs to work. The architecture still needs to make sense. The campaign plan must still be coherent. But AI is making all of those easier to produce without all of the understanding we used to infer from them during the interview process.

I don’t want or need AI banned from the interview to recover that signal. What I’m trying to find is the candidate’s model of the problem: what they believe is true, what they’re assuming, where they’re uncertain, what deserves verification, and what new information would change their mind.

A good judgment interview keeps pulling on the threads until you can tell how much understanding sits underneath the answer.

The problem doesn’t stop once the candidate gets the job. The same gap that sits between a polished artifact and the understanding behind it, shows up in design docs, RFCs, product specs, campaign plans, and incident write-ups.

That’s also one way AI slop gets generated inside a company. AI can produce enough polished material that neither the person generating it, nor the person receiving it wants to inspect every claim. Verbosity raises the cost of scrutiny. That’s why I care whether someone actually reads, challenges, throws away, or takes ownership of what AI gives them, not just whether they can produce something good with it.

I’ve hired people whose biggest contribution in their first six months was showing everyone else a better way to do something legacy or routine. AI can create the conditions that engender that kind of change. That might be exactly the person I’m looking to hire. But if they’re going to teach the rest of the team a new way of working, I want them to understand what they’re passing along, not just repeat what the model gave them.

This isn’t only an interviewing skill. Leaders increasingly have to know when to accept the artifact and when to pull on the assumptions underneath it. Who understands why this decision was made? What breaks if an assumption is wrong? Who can defend it when the context changes?

The interview is your opportunity to get underneath the polished answer and understand what’s actually there before the gap between output and understanding becomes your company’s problem.