7 AI Coding Interview Questions & Answers (Junior, Mid & Senior)

AI coding question number one. An AI agent reviewed and tested its sound PR, but after deployment, you found production bugs. How would you fix and avoid this in the future? So, an AI agent you and test it some PR. Well, usually the problem with agents is that the way they are built, they try to game whatever goal we give it. And so when an agent has context to build the implementation and then it has to write a test and then review its own code, usually, it's just kind of working against those three conditions and it's going to do whatever it takes to make it pass.

And so it's a better strategy, for example, if you have three independent agents working on these three different tasks. And so, you know, if you have agent number one doing the coding, agent number two, maybe writing the test, and then agent number three reviewing everything. It's better if agent number three and two and one, they're another way of each other that's clear context boundaries. And so when agent number two is being told, hey, you need to test the feature this agent built, they cannot be tricked by the original agent.

And so a lot of the a good example where when agents game themselves, it's, for example, when they write a shallow test that don't mean anything. Just to tell you, hey, I tested a feature and then you go into the test and you realize there's the test doesn't make any sense. And it's pretty much the same with Q uh with a QA kind of agent. So it's much better to have them isolated, not only from the perspective of context isolation, but also because from the context engineering perspective, it's easier for them to pay attention to what's important.

If they only have to do one specific task, and if we're giving them three competing objectives. AI coding question number two, how do you ensure code quality of AI written code? When it comes to code quality of AI written code, things get a bit more complex than when we are coding manually. I believe also because of the volume, we are shipping 10x or 20x more pulley requests. possible to even read the code.

So, a couple of things that are being developed and that I've seen working are, number one, we got to default to the traditional code quality measures, things like we're going to have to obviously have a lint. Uh I do prefer languages that are type safe because it means all my code it's by default programmed to interfaces. That's kind of the mindset, so it's easier to isolate modules. It's easier to even read the code than if you're using um you know a language that it's dynamically typed.

Uh so basically you want to make sure that your type checker, you know, there's no errors in the in the typing or uh also make sure that agents do not jump to the goals. Uh you'll see that sometimes agents that add exception uh lines, you know, they'll go ignore this file with TS lint. And that's how you they make your linter pass. That's very important, you want to prohibit that specifically in the agent instructions.

But usually, you want to lint, you want to have your type safety and then I would go deeper into static analysis of the code. Um basically, this means that you can run a certain uh you can measure your code and your code complexity by certain metrics that can be extracted deterministically. So for example, you could detect the amount of duplication in your modules. And so there's tools like Fallow that you can run over your code base or over what you change and it will tell you, hey, there's a 15% duplication here.

Uh when you're building front end, for example, and you're building components, there's a normal tendency for agents to put logic that is reusable inside the component. And so you have the same function 15 times. And I know there's a lot of people online that are saying, oh, the application doesn't matter because it will be an agent that gets the code anyhow. But when you look at your token spend or when you look at the context window, the more duplicated code you have in your code base, the more context you provide in every call, the worst answers you'll get.

So the applications of code is not a good thing even in the AI age. Uh so the application will be number one. Uh the other anti pattern I see a lot is dead code. A lot of times what happens is the agent well clean up. We'll set the component, but it doesn't clean up the dependencies. So you end up with dead modules and a lot of dead code. And again, that code usually gets stripped down, it doesn't get to production because we have what we call in module bundlers tree shaking.

You are still having to deal with that code when we develop with the agent. So again, it's policy or context. So that code is one of them. And then uh one of the measures uh that's a bit more aggressive is cyclomatic complexity. Basically this will analyze how complex a function is by the different logical paths you can go through in that function. Now, of course we can have a full conversation about that, but complexity is never good.

Complexity never scales. It's very hard to read complex functions, so when you have a lot of cyclomatic complexity you can instruct your agents to break that function into smaller modules that you know, apply solid principles and and break it down. And so basic the results of all those analysis that I usually add to my CICD pipeline, they're usually being fed back to a isolated review agent that then will implement uh the fixes over a couple of turns.

What about testing? So the thing with testing is that for the most part they found n and tests to be the best when working with AI. And n and tests are more about I'd say software quality than code quality itself. You could have very clean code that then breaks in production because the feature wasn't well specified or we forgot an edge case. So in terms of testing, I do prefer end to end tests and the reason is when you change code so fast, writing unit tests that are very short lived and very small scoped, it's just increasing your token spend a lot uh without any benefits because at the end of the day, the kind of mistakes AI does are usually not catchable with the unit test.

It's very easy to game the unit test. So unit test became this routine. I think unit test do make human code better, human written code. AI coding question number three. From these two, which agent command do you prefer and why? And the agent commands are clear and compact. So which agent command do I prefer? Well, clear will basically clear the context window, so it will send everything that we have in the current window to trash and the problem there is that you're basically starting over.

So you fall back to whatever you have in your in your agents MD and and maybe your code. what you have to start over, so I don't use clear unless I'm done with the task. And compact makes a summary. So it will go through the whole context window and then it will try to just as I've say, you know, extract the most important things and give you a smaller context window and start a new session. The problem that you get a lot with compact is that normally in a session, you start maybe with something very important, which is your prompt of your intent, and then usually maybe you do some planning where you go back and forth with an agent.

It's not so relevant. And then maybe you have, you know, you finish your plan, so that's the final thing and then you start maybe executing on it. So that's also kind of relevant information as you're executing. And then you're out of the context window. And so when you compact, you kind of get the same portion of all that conversation. But not all of it not all of it is relevant. So for example, this red part over here, I'm not really interested in it because it's just the back and forth with an agent.

So why would I get that to my new conversation? So I think a better way, it's usually to do a handoff. Uh there's different skills for that. In this case, I I work a lot with cloud code, but I don't think agents have a standard for handoff. But you can basically say that, okay, can you summarize the context window and extract, you know, the current progress on the tasks we are building and the architecture decisions.

So, when you do that and you hand off, you are actually getting the relevant parts, right? So basically the one I the the overall plan and the status of the implementation. So, I would say none of them and I believe, you know, hand off when you're doing this long coding session, a hand off, it's usually a lot All right. AI coding question number four. Are you familiar with the Agendic software factory? How does it work and how would you build one? Right, so basically the Agendic software factory, it's a tent where people are adding as many agents to their software lifecycle. Right, to automate as much as possible. So, usually we start with uh the planning part and refinement. And so, you could have an agent connected to your to your Jira ticket that can do uh planning and refinement. Agents are really good for example at producing mugs. And once they're done with the planning and refinement, we could have another agent kind of doing the prioritization and estimate. Which in my opinion, it's one of the worst tasks to give to agents, but of course there are people that hide that. So that would be planning and pride. Right? And then we finally get into the implementation. Right? So, you'd have basically an agent pulling your code base, reading the ticket that was written by the previous agent and then kind of implementing everything, and then from the implementation, you could go into quality. Which would be you get an agent to reproduce the issue. And once quality passes, then you could even get it to deployment. So basically that would be kind of getting that to deployment, let me just down that really quick, deploy. And then this is in itself a cycle because once you deployed, you have your agents kind of listening to sentry, listening to all your logs and errors, and if a bug comes through, they classify that back, put it in planning and refinement, um, you know, refine it, plan it and go into implementation. And usually as an engineer, you are kind of working here, so after there was some QA by the agent, you work at the final gate which is approving PRs. Because no matter how well you set up this fa

Actually, no matter how much you tune your agents, there are a lot of tickets where you'll have to take a look. Maybe there's some low value ticket where you're just changing some frontend and it doesn't have a big blast radius that you could just merge directly. But for the most part, you need to review big changes. So that's kind of the idea, that's why people are going. A lot of people like experimenting with this.

I think right now there's a lot of hype around it. But of course, all those things that you see here, they kind of break, so usually you need an engineer to go in and adjust things. A lot of people are talking about the dark factory, where you don't even look, you set it up and you go away. Uh, I haven't seen anyone that had good results without other than hyping it. But when it comes to production, valuable software that you deploy to users, usually you you need human supervision in it.

Okay, let's suppose you join a software team and they have an agentic software factory set up. Where would you spend most of your time looking at from all these different stages of the factory? So I probably spend a lot of time in the PR kind of reviewing code. Now, it's unrealistic to review all the code. So I will sample it or just look at one in three PR requests, for example. And see, can we also like write tickets in a better way are the test cases defined, not only the code that comes out.

But also what's everything that a ticket has? Because that's usually the context that moves from all the stages. Basically the branch of the changes and the ticket as it was specified. So that's number one, I would definitely read PRs. And then I'll definitely spend a lot of time adjusting the agents. So, agents are built with a specific prompt that usually you start with something very vague. But then you can get more and more specific or you can even add agent skills.

If you're building with React, you can add them React skills. So when they do when they read the React code, they are a bit better at it. So that's what I would look at. And also at the Q&A part, there's usually quite a lot of effort into making this fast and efficient. Because as I mentioned in the other questions, end-to-end tests the best when working with agents, but also very expensive to run and not easy to set up.

You need to make sure all your environment is reproducible for the agent, to spin up all your back in dependency, mock all your six external services, and then run the test. And even if you manage to do that, it will consume a lot of CIA resources. So right now, to give you an example that here we have 950 entry and trend tests. They consume, every time you run them, they consume around $1.5 in GitHub actions. It sounds little, but when you are deploying 10 times a day, it sounds up very fast.

So you start as an engineer to really dull all this engineering of the factory itself. And by the way, folks, if you're interested in the agentic software factory concept and how to build one, there is one video on our channel where Buler and I go deep exactly into that. AI coding question number five, the performance of an AI coding agent decreases as context grows. Why? The thing with agents is that at the end of the day, they are just a recursive call to a model.

So we have our our AI model, right, that will probably implement a transformer architecture. Uh so that's our model, right, there's the big chunk here and just to represent it very vaguely, the attention layers are what we call attention layers are over here. And so the attention layers are the ones that decide from context, which is what we pass in, what it's the most relevant part. So to explain it very shortly, when you ask the model, what is the capital of France, the model has to understand, to look for a cluster where the word capital and France are very important, because if not, they would look for a capital city and maybe they say Washington, but then France has to wait 10

and be like, okay, well, if I have capital and France together, then it's Paris. If I only have France together, then maybe it's a different question about France. All that is calculated for every single token we pass in. So we have an attention budget like a weight you put on, right? So every single token gets a weight, and based on those weights, the way models work, it's the model will try to generate this last token.

And so the problem is what happens when you have a lot of context? It's you have, you know, imagine, right, this would be a a little context and when we pass in a lot of context, we just have a lot more dose. So I have a lot more tokens that go in. And so what happens is the context window and the context budget of this model is the same. You didn't change we have a constant window, which means each one of those tokens gets less attention.

They weight less in the final answer. That's why when you pass a lot of context, you end up having hallucinations. Because every word you said before has less importance and the model doesn't really know which one is the most important. They all kind of flatten out, they all you spread out your attention budget between too many tokens, so the model doesn't know what to pay attention with. And it will come up with what we call hallucinations.

What would be a solution or a way to mitigate this? So, in order to do this, you want to keep your context as low as practically possible, usually under 10 to 20% of the context window. So, if you're working with a model like Opus 4 5.5, that has the context window of 1 million, you want to keep your context around 100k to 200k tokens. And if you want to do that effectively, you can use the handoff skill, as I mentioned before, clear between sessions and then use handoff quite early.

Normally models will auto-compact when you're at 90% context window, but that's a bit too late already. AI coding question number six, you are building a feature that includes several calls to an LLM. How do you spend the least amount of tokens per call? So, to spend the least amount of tokens per call, I think we need to optimize both the input and the output. Uh, but there's also, you know, in this complex continue models.

thinking tokens? I I think I'll leave that out. But when it comes to input tokens, probably it's all has to do with the way you design your prompt. And the reason is, usually we have a prompt, a system prompt that we pass. We want the LLM to do a certain function, we take some user input. And so the way it mostly works is we have our system prompt that the model has. We cannot change that, then we add our stuff, our generic, whatever the functionality of our product is.

I want you to act as X and do X for the user, and then the user the prompt somewhere here. So that would be what goes into the context window. And an anti pattern when working with LLMs is to whenever you make this call, to change the context window at the very early point. So I'm going to draw this arrow here. You basically want to change things here at the bottom. And the reason for that is because when we change something here, between different LLM calls, all this gets recycled in what we call the KV cache.

So it basically hits KV cache. So that's the number one optimization you can do. KV caching is basically where you cache what I mentioned before, the attention budget. So this attention budget, the weight of every token gets recomputed every time you hit the model internally. But if you don't change your prompt, it's going to reuse as much as possible for all the tokens before, because attention is calculated only for the new stuff you add.

So number one, for example, a very bad thing you could do is to embed a time stamp here. Maybe the users want a question that it's today, you want to add that time stamp at the very bottom. So it doesn't end up recomputing all the attention before. So that's number one and the number two, look into prompt caching too, to see if you could cache the answer to the full prompt. Uh probably use some semantic cache and see how much of every prompt you could reuse.

Those are the optimizations I would do at the LLM layer, and of course you can add caching in your application all the way to the top with Redis and so on, the traditional caching. And probably the last one, I if you have flexibility, also have have a model router. I think there's a couple of open source model routers that will evaluate the prompt really quickly with a very fast model and route it to a more expensive or less expensive model depending on the difficulty they rank they they assigned to.

AI coding question number seven, when integrating LLMs into a product, how can we avoid or minimize hallucinations? Uh, this is a tough one, I believe, because hallucinations are as much of a bug as a feature, right? So models, whenever they answer a question, they hallucinate over their knowledge. They they interpolate and give you an original answer based on their knowledge and we as humans know what's going around, right? We're like, oh this is actually oh it's so smart and it's like it's hallucinating, but the model does the same thing.

They just generate on top of what they have, for them is the same. The model doesn't know it's hallucinating. If it do it, it wouldn't do it in the first place. It's a feature. You you could set the the temperature of the model to zero, so it literally gives you literal answers, but then it's not that useful anymore because it's basically just a glorified text query. It will literally look into the text has been trained on and give the exact passage.

So that's not going to work. What when you're building software with LLMs, what I've been using lately and I think it's the future of that is basically two things. Number one, it would be uh static interfaces, so you've probably heard of models like Jeff where you basically give it context and then a decision to make, and it will return to you either a yes or no or something in the format of a static type.

So you know it's not going to hallucinate that's a probability there that gets eliminated. Because you could be like Jeff is not going to tell you it can tell you this is 0.962 or false. You that to a binary and you're like okay this is true or false. So you go from unstructured input to a structured output. So that's number one, and number two would be what? They're basically developing now, which is formal verification.

So this is kind of a high level of language where you write rules that cannot be bent in your software. They say bent language that's being developed, we'll see where that goes. But basically you could say something like, a user shouldn't be charged twice, and then models will look at that rule and then scan your software and find all the dependencies where this happened and make sure that this rule does not get violated.

So that's how you kind of validate business logic at scale. But again, all this is kind of experimental, so I'm just going to leave it on the board as an idea. Right, so thank you, for the folks watching us, if you are interviewing right now when you want to go beyond AI coding questions, there's a free Tangle assessment where you can actually assess your skills and find out your gaps across the full stack way beyond AI fluency, and if you're looking to become an AI native senior engineer, then check out the mentorship that I offer.

We might be able to help you level up and stay relevant for decades to come. With that being said, thank you so much, and we will see you folks in the next one.