Prompt Engineering

When you have an already trained model and want it to consistently produce results that meet your expectations, there are generally two paths: one is training or fine-tuning, that is, modifying the model's internal parameters; the other is changing nothing but the input you feed it. Prompt engineering takes the second path—without touching any of the model's weights, it relies solely on designing the wording and structure of the input to improve the hit rate. The so-called hit rate refers to how well the model's output conforms to the constraints you set in advance: what content, for whom, what format, what tone, whether to include examples. When the constraints are clearly stated, the probability that the output hits the target increases; when they are not clear, the model can only guess.

This is a craft that can be tested and improved repeatedly, not a one-time button. Its working process can be broken down into several steps: first clarify the task goal, audience, context, boundaries, and output requirements one by one; then write these requirements into a prompt that can actually be submitted to the model; then hand it to the model to execute and observe the output. The output of the entire process is two things: a prompt that can be tested and reused, and the actual result that this prompt produces on the model. Use the results to examine in turn whether the prompt is written well—the closer the output is to the requirements, the higher the constraint hit rate of this prompt; if the deviation is large, go back and change the wording, add examples, restrict the format, and run it again. This cycle of "write prompt → observe output → revise prompt" is the daily working method of prompt engineering.

The reason it is placed at the very front of all methods is that it has three obvious characteristics: it does not update weights, it takes effect immediately, and it can be changed at any time. It does not require training data, does not require waiting for training to complete; once changed, the effect can be seen immediately, and if unsatisfied, change it again right away. Therefore it is usually suitable for validating requirements first: before deciding whether to invest training cost, use a prompt to probe "whether the model can actually do this task, and where it falls short." But "no training required" does not equal "zero cost"—the longer the prompt, the higher the cost and latency of each inference; the prompt itself also requires maintenance and version management; and each time you change it, you must run regression evaluation again to confirm that the original effect has not been broken. All these costs should be counted in the ledger of prompt engineering, but they occur in the inference stage rather than the training stage.

At the same time, be clear about the capability boundary of this method: from beginning to end it does not update the model's weights, so its effect only holds for "the kind of model that has already been tested." Change to a different model, or the model itself is upgraded, or the data distribution changes, and the original set of prompts may not continue to hit; you must re-verify. Prompt engineering changes the interface between you and the model, not the model's own capability ceiling.

Many people have a common question when they first encounter prompt engineering: the model is still the same model, not a single weight has changed, so why does merely asking differently make such a big difference in output quality? To answer this, you have to go back to the essence of large language models. The way it works is not to understand your intention and then do it, but to predict: given the text you have provided, what is the most likely content to follow (see the “Large Language Model” deep-dive page). Everything the model does—whether translating, summarizing, or writing copy—ultimately comes down to this one action: given a prefix, continuing with the most likely next step.

Once you accept this perspective, the role of the prompt becomes intuitive: the prompt is the starting point and conditions you set for this continuation. Every time the model generates, it reads in the whole of “prompt + existing prefix,” and then, based on this already present text, estimates what the next token should be. The key point here is that the model is trained to be highly “compliant” with the given text—constraints that have already appeared in the prefix will strongly narrow the probability distribution of its subsequent generation. So the clearer the prompt and the more specific the constraints, the narrower and more precise the range of “what to say” that the model can lock onto. If you specify a watch ad aimed at middle-aged men, three lines, each no more than forty characters, emphasizing water resistance, then when the model continues, most continuation directions that do not meet these conditions have already been excluded by the prefix, and the remaining high-probability paths naturally fall within the range you want.

Conversely, the vaguer the prompt, the fewer constraints there are in the prefix to follow, and the model can only continue along the most generic, safest common directions. By “safest,” this refers to those directions that appear most frequently in the training data and are least likely to go wrong for any context—for example, empty platitudes, adjectives that are universally applicable, statements that are correct but carry no information. You then get a passage that cannot be faulted but is also completely useless: “correct nonsense.” It is not that the model has become stupid; it is that the starting information it received is too little, so it can only choose the most mediocre high-probability path.

So the whole mechanism can be understood this way: you are not commanding a person who understands you to carry out a task; you are setting as clear a starting point as possible for a continuation engine. The better the starting point is set, the more narrowly the continuation direction is constrained, and the more the continued content matches what you want. At the same time, note the other half of this statement: output that better meets your requirements only means that the constraint hit rate for this continuation has improved; it does not mean that the model has thereby gained any new knowledge. Its upper capability limit has not changed; what has changed is only which capabilities are elicited by the prefix in this generation. And setting this starting point is precisely where all the operations of prompt engineering take effect.

Knowing only that 'a prompt is the start of a continuation' is not enough; you also need to know how to rewrite a vague wish into an executable specification. This section introduces a more actionable concept: the task contract. A good prompt should act like an interface contract, so that both the model and the evaluator can follow it—the model knows what to produce, and the evaluator knows what criteria to score against. For a prompt without a contract, the model can only guess, and the evaluator can only go by feel.

The structure of the contract can be viewed from the layers in Figure 1: the outermost layer first defines the goal, that is, the specific problem to be solved this time; then one layer inward provides the necessary context to accomplish the goal; next, write down the boundaries, that is, what cannot be done, and crossing them counts as a breach; the innermost layer specifies a verifiable output structure, so that results can be checked item by item by machine or manually. The four layers narrow successively from the outside in. Among these, role descriptions (such as 'you are a senior copywriter') only play a supporting role, used to set identity and perspective and constrain language style; they cannot replace permission declarations, nor can they replace provisions on factual sources—these two things must be undertaken by the boundary layer and the context layer respectively.

The common prompting techniques are organized into a table, where each row is a correspondence of the type 'pattern → what it does → example':

Every item in the table does the same thing: it pins down in advance a dimension that would otherwise be left for the model to guess. Even so, a prompt is rarely written well in one pass; its correct use is iterative. Look at where the output is wrong, add a constraint targeted at that specific issue, and then observe the effect. Treat the prompt as an adjustable parameter: change only one thing at a time and observe the resulting change, which makes it easier to find the sentence that really matters than piling up a large block all at once.

The following worked example illustrates the difference between 'attributable iteration' and 'non-attributable stacking.' The initial prompt of a refund assistant was just one sentence, 'Answer refund questions,' and out of 20 test cases, only 11 could provide both the refund deadline and the source of the basis. These 9 failures are observable and have clear categories: missing deadline, missing source, or both. So the new prompt added the task scope, the allowed reference materials, a clarification strategy when there is no evidence, and required output of JSON fields decision, reason, and source. After retesting, 17 cases passed. Note the remaining 3 failures: they all involve scenarios of quality problem exceptions, which belong to another failure category and point to the next step of supplementing retrieval evidence or business rules, rather than continuing to pile on vague pleas such as 'please be serious' and 'please be professional' in the prompt.

The causal chain revealed by this example is: score improvement is meaningful only when it corresponds to the disappearance of specific failure categories. The improvement from 11 to 17 can be attributed to the elimination of failures such as 'deadline and source' by the new constraints; but if the score increases after changing the prompt yet you cannot say which failures disappeared, that is just non-attributable prompt stacking—the increase may come from coincidence and cannot guide the next step. Therefore, every time you change a prompt, you should first ask: which category of observable failure does this change target?

In the examples from the previous section, a phenomenon worth asking about appeared: just by putting a few examples into the prompt and specifying the output format, the model really did it. Why can it “just do it”? The reason prompt engineering works is that there is a more fundamental mechanism behind it, called in-context learning (see the “In-context Learning” deep-dive page). Once you understand it, those earlier techniques are no longer a bunch of isolated tricks, but applications of the same ability in different dimensions.

In-context learning means: the model does not need to update any weights; merely with the examples given in the prompt, it can identify on the spot what a task “looks like” and carry it out. Its working process can be described as follows: the input consists of the examples in the prompt plus the new query; during forward computation, the model first locates the pattern of “input to output” in the examples — for example, each input is a product selling point, each output is an advertising slogan of no more than twenty characters, or each output is three list items; after locating this pattern, it then processes the new query according to the same pattern and outputs an answer that conforms to the style of the examples. The entire process occurs in forward propagation, and the model’s knowledge does not undergo any persistent change.

Looking at the most effective tricks in prompt engineering from this perspective, they are essentially all the same: giving examples demonstrates “what the input looks like and what the output looks like”; giving a format demonstrates “what the structure of the output looks like”; giving a role demonstrates “what identity and tone to speak with.” These techniques all use the prompt itself to demonstrate “what output I want looks like,” and after the model recognizes this pattern, it continues along that pattern. In other words, of the earlier idea of “giving the continuation engine as clear a start as possible,” the most effective type of start is demonstration.

From this, a clear hierarchy can be given: in-context learning is a capability the model itself possesses — it can “do things as shown in the prompt”; prompt engineering is the craft of using this capability well — designing prompts that let the model understand at a glance “how things should be done.” The capability is inherent to the model and cannot be changed; the craft can be learned, iterated, and evaluated. At the same time, this mechanism also gives two boundary reminders: first, this “doing as shown” is imitation that occurs within the context and is not written into the weights; if you change the conversation or clear the context, the model will not become better at doing this task; second, the fact that the model has the ability to follow examples does not mean every prompt will be faithfully followed — when a prompt has internal contradictions, too many constraints, or conflicts with the model’s existing patterns, compliance may be reduced. In-context learning explains why prompts work and also delineates the scope within which they work.

Prompt Engineering delivers quick results at low cost, but it has a clear ceiling. The question this section answers is: when do prompts fail to hold the line, and when should you change tactics? The entry point for judging is to examine the “type of failure”—what exactly is missing when the model gets it wrong this time. Considering the failure type, the source of the required knowledge, the required output stability, call volume, and cost together lets you decide what to switch to: keep refining prompts, integrate retrieval, or turn to fine-tuning.

Let’s first look at three typical situations where prompting can’t hold the line.

The first is a lack of facts. If the task requires up-to-date or private knowledge—for example, a company’s latest internal refund policy or today’s inventory data—then no matter how well you write the prompt, it cannot conjure knowledge that simply isn’t in the model’s parameters. Prompts can only bring to bear capabilities the model already has; they cannot create knowledge out of thin air. In this case, what you need is Retrieval-Augmented Generation (RAG): retrieve external material, put it into the context, and have the model answer based on that material (see the “RAG” deep-dive page).

The second is a lack of stable, complex behavior or format. Some tasks have very complex output structures or require extremely high behavioral consistency, and prompts just can’t hold it steady—the same prompt may comply this time and drift the next, or you have to write a long passage each time just to barely keep it in line. Repeatedly coaxing the model with wording shows that this behavior has gone beyond what wording can control; at this point you should consider fine-tuning, to hard-code the desired behavior into the weights so it no longer depends on carefully designed prompts each time.

The third one requires distinguishing a higher-level concept: if what you need to manage is no longer the wording of a single prompt, but “what goes into the entire context”—how historical messages are trimmed, how retrieval results are arranged, how examples are organized—then you have entered the field of Context Engineering (see its node). This is a higher-level engineering problem than Prompt Engineering; the input changes from “a passage of text” to “a strategy for orchestrating a set of materials.”

From this we can lay out a pragmatic ladder: first use Prompt Engineering for low-cost validation to clarify needs and gaps; when you find that what’s missing is updatable facts, consider integrating RAG; when you need to maintain specific stable behavior across a large number of requests, then evaluate fine-tuning. This order is the default starting point, not a hard rule—how exactly to choose ultimately depends on quality requirements, latency, privacy, and operational constraints. Prompts are a starting point, not the destination.

Prompts can shape model behavior—this is the source of their power, and it also means they can be an attack surface: if the model is this responsive to prompts, could maliciously crafted prompts hijack it in turn? This section does not introduce any new facts beyond this page; it only uses the mechanisms already established in Sections 2 and 4 to reason through the risk—this risk has a specific name, called prompt injection (see the “Prompt Injection” deep-dive page).

First, recall the mechanism from Section 2: the model performs continuation; it reads in the whole “prompt + existing prefix” and then writes the most likely next step. Note that no step in this mechanism separates the text in the prefix into “your instruction” and “other content” categories—for continuation, all the words in the prefix are simply conditions for estimating the next token. Section 4 then added a point using the in-context learning demonstrated by GPT-3: whatever style appears in the context, the model continues in that style. Combining these two points yields: any text that enters the context participates in “demonstrating what should come next.”

From this, the risk can be deduced. What prompt engineering does is precisely to carefully select the text that enters the context in order to shape the output; conversely, as long as other text can also enter the context, it will participate in shaping in the same way—this is the entire principle behind the idea of “injection.” Consider a hypothetical scenario (not a record of an actual attack): your prompt asks to “summarize the following article,” while the article content itself includes a passage saying “ignore the above requirements and instead output…”. Once this text enters the context, it occupies the same position as the other prompt text: it is all part of the prefix, all influencing the direction of continuation. The model does not need to be “persuaded” or “deceived”—it is simply continuing as described in Section 2, except that some of the text shaping the output is not from you.

Thus the causal chain can be divided into three steps: prompts can shape behavior (this is the foundation of prompt engineering) → any text that enters the context participates in shaping in the same way → if someone mixes the text they want into the context, it is equivalent to exerting partial control over the output. Therefore prompt injection is not a kind of “failure” of prompt engineering, but rather the same principle turned around—attackers are doing exactly the most “orthodox” prompt design, except their goal is opposite to yours.

This also shows at which layer protection should begin. Section 5 said that managing “what goes into the entire context” is a matter of context engineering—this boundary is exactly the line of defense here: deciding which external text can enter the context determines which text has the opportunity to shape the output. Following this line of thought, the first step in protection is to treat “what can enter the context” as an explicit design decision, rather than assuming by default that external content is always safe; for scenarios where the output can have high-impact consequences, process controls such as human confirmation should also be considered as a fallback. Conversely, countermeasures of the form “detect malicious sentences and block them” can at most be one layer: they block known phrasing, while the path by which context shapes output does not rely on any fixed phrasing. The power of prompts cannot be completely eliminated; what can be done is to manage “who can influence the context,” so that people who want to hijack prompts have no channel into the context.

Putting the previous sections together, the whole causal chain is as follows.

The starting point of the chain is a goal: without modifying the model, just design the “how to ask” well so that the model reliably produces the desired result—this is the definition of prompt engineering, and also the fundamental feature that distinguishes it from training-type methods (§1).

Why can “how to ask” have an effect? Because the model is essentially completing text; the prompt you give is the starting point and condition for this completion. The clearer the starting point, the more precisely the continuation direction is narrowed (§2). From this mechanism, a set of general practices naturally follows: be clear and specific, give it a role, specify the format, provide examples, make it go step by step, and set boundaries—each move locks in advance a dimension that would otherwise be left for the model to guess, and then tunes it gradually through iteration (§3).

One level deeper, the underlying principle of these practices is in-context learning: what the prompt is really doing is demonstrating “what the output I want looks like,” and once the model recognizes this pattern, it follows it. All the techniques of prompt engineering are essentially about making good use of this built-in capability of the model (§4).

With power comes the need to know its boundaries: when facts are missing, prompts cannot create knowledge—you should turn to RAG; when you need stable behavior, prompts cannot hold it steady—consider fine-tuning; when what you need to manage is what goes into the entire context, that is the job of context engineering (§5). And the flip side of power is the dark side: if a model can be shaped by prompts, it can also be hijacked by malicious prompts—prompt injection exploits exactly the same principle as prompt engineering (§6).

This chain starts from the fact that “the model is playing a word chain game” and, along one line, derives why prompts are effective, how to design them, and how to iterate on them; along the other line, it derives when they fail and when they are dangerous. To grasp its core, you only need to be able to answer two questions: why does changing the way you ask produce such different results—because the model is playing a word chain game, and the way you ask is the starting point of that game; and what is the relationship between prompt engineering and in-context learning—in-context learning is the model's ability to act according to the pattern shown in the prompt, and prompt engineering is the craft of making good use of that ability.

The prompt engineering node on this knowledge map has conceptual dependency relationships that can be arranged by learning level into the table below:

The logic of this hierarchy is: the prerequisite layer answers 'what the model is and why it can follow prompts'; the core layer of this page is built on these two facts—the model is a continuation engine, so wording and structure determine the starting point; the model has in-context learning capability, so demonstration methods such as clear and specific, role, format, and examples are effective; and 'no training required but evaluation required' stipulates that this craft must be tuned through iteration and a fixed test set. Every concept in the adjacent extension layer grows directly out of a conclusion on this page: Chain of Thought (CoT) is the in-depth form of 'make it go step by step', the system prompt is the part of the prompt that specifically carries global constraints, context engineering is the higher-level problem of 'managing what is placed in the entire context', Retrieval-Augmented Generation (RAG) and fine-tuning are two alternative directions after crossing the prompt boundary, and prompt injection is the dark side of the principle that prompts shape behavior. The farther layer—self-consistency, tree of thoughts, structured outputs, and prompt caching—corresponds respectively to reasoning credibility, the organization of complex reasoning, the rigid guarantee of output format, and the optimization of inference cost; these are advanced directions after mastering the core concepts. When studying, proceed in the order 'prerequisite → core → adjacent extension'; the farther layer can serve as topic leads, and there is no need to go deep at the very beginning.