How to Talk so Your AI Will Listen

TL;DR below.

AI assistants are everywhere, and we interact with them in natural language. We know, of course, that they are machines—but they don’t always behave in a strictly mechanical way. That’s one of the most important things to get the hang of when working with them.

Let me explain:

One of the characteristics of  LLMs is context windows. Think of an LLM’s context window like its short-term memory. It’s the amount of text the model can “see” and keep in mind at once. A small context window—say, a few thousand tokens—means it can handle shorter conversations, code snippets, or documents before it starts to forget earlier details. A large window lets the model work with entire books, big codebases, or long conversations without losing track. Bigger windows don’t make the model smarter, but they do let it connect ideas over longer stretches, which often seems like deeper understanding.

Context window sizes have increased dramatically with model releases, with early versions having a context window of less than 5000 tokens, compared to hundreds of thousands or even millions of tokens in recent models.

So now our LLMs seem smarter, and can operate across larger datasets, considering an application’s whole codebase when creating new modules or functions, for instance. We can also give them more instructions.  As well as longer prompts, we can provide them with a set of default instructions to use with every task we ask them to do. For writing code, this can be useful for enforcing naming standards, architectural patterns (and anti-patterns), and code security. For instance, we can instruct our LLM to always use pinned versions of dependencies, mitigate against specific threats, or perform security scans using an MCP-enabled scanner. Our model will then take our specific request (e.g., “write a function to read a JSON-formatted file, extract some values, and put them in a PostgreSQL database”) and add in default instructions about authentication standards, avoiding SQL injection flaws, and variable names, etc.

At least it probably will, if it ‘remembers’ the instructions. Or if they aren’t too long and tedious. 

Because this is where things get interesting. Yes, we are interacting with a machine, but one that sometimes mimics human behaviors, and not always the good ones. 

A long time ago, I attended some presentation training. Once I had overcome my initial irritation that someone thought my brilliance could possibly be improved by spending time in a room with a bunch of other drones and a trainer, I found it quite interesting. One of the topics covered was “Primacy and Recency”: people remember the first and last things you tell them better than the guff in the middle. 

Well, it turns out that our robotic LLM  friends are the same. There’s a lot of talk about “U-Shaped Attention” and “Contextual Position” floating around in LLM research circles, along with a growing realization that large context sizes increase the chance that some instructions can get ‘lost’ in a large text body. Seems familiar? Another thing is that the clearer we make our instructions using paragraphs, succinct instructions, and bullet points, the more likely the model is to remember them. 

What does this mean for us? 

Practically, it means creating prompts and default instructions as if you were writing for a smart, but busy human. Put the most important things first, don’t make it too long (use references for supporting information), keep things clear, and summarize key instructions at the end. Also, never trust that all your instructions have been followed: always have systems to verify the output, especially in critical areas like code security.

Intellectually, it's more nuanced. LLMs are different. They are a mix of super-human speed and scalability, combined with some very human frailties. It’s probably philosophically,  and certainly technically, wrong to anthropomorphize LLMs, but it’s also quite a handy presupposition (let’s just act like we’re working with a human) when it comes to providing prompts and verifying outputs. 

TL;DR:LLMs don’t think like humans, but their context window—the text they can “remember” at once—shapes how well they follow instructions. Bigger windows let them handle longer codebases or documents, but important details can still get lost in the middle. To get the best results, write prompts like you’re briefing a smart but distracted human: keep instructions clear, put the most important points first and last, and always verify critical instructions like security rules are followed.