Prompt engineering and context engineering

Generated by Claude through GigAI 0.1.11.10 on 2026-10-11.

Concept: Prompt engineering · also covers Context engineering · Step 14 of 34 in AI forward deployed engineer · version 1

Practice this

On this page

Prompt engineering is the practice of writing instructions a model can follow reliably; context engineering is deciding what information occupies the window at each step and in what form. Together they are the main control surface for model behavior that does not require training. The discipline is empirical: define success, write a first version, measure, and change one thing at a time.

Read more

How it is done in practice

A deployed team starts by writing down success criteria with the customer and collecting a few dozen real inputs, including the awkward ones. The first prompt is plain and explicit; it is run against the cases, failures are read one by one, and each revision targets a named failure class. Prompts live in the repository as templates with typed variables, are reviewed like code, and are tagged with the model id they were tuned for. Customer documents, user input and tool results are wrapped in delimiters and treated as data, never spliced into the instruction text. When context grows, the team measures which sections the answers depend on and removes or summarizes the rest before reaching for a larger window.

Technologies

Anthropic Console WorkbenchOpenAI PlaygroundJinja2PromptfooLangSmithLangfuseDSPyPydantic AI

Diagram

flowchart TD
  A[Success criteria and real cases] --> B[Prompt template]
  C[Stable context: role, rules, tools] --> D[Assemble window]
  E[Selected documents and history summary] --> D
  F[Current user request] --> D
  B --> D
  D --> G[Model call]
  G --> H[Score against cases]
  H -->|failures by class| B
  H -->|pass| I[Release with pinned model id]
flowchart TD
  A[Success criteria and real cases] --> B[Prompt template]
  C[Stable context: role, rules, tools] --> D[Assemble window]
  E[Selected documents and history summary] --> D
  F[Current user request] --> D
  B --> D
  D --> G[Model call]
  G --> H[Score against cases]
  H -->|failures by class| B
  H -->|pass| I[Release with pinned model id]

Code

python
SYSTEM = """You are a support analyst for a billing platform.
Answer only from the material inside <documents>.
If the answer is not there, reply exactly: NOT_FOUND.
Give the answer inside <answer> tags, one short paragraph."""

def build_user_turn(docs: list[str], question: str) -> str:
    blocks = "\n".join(
        f"<document index=\"{i}\">\n{d}\n</document>" for i, d in enumerate(docs)
    )
    # long reference material first, the question last
    return f"<documents>\n{blocks}\n</documents>\n\nQuestion: {question}"

Subtopics

Anatomy of a production prompt

Parts and their jobs

A dependable prompt separates concerns. The system or developer message carries the role, the task, the audience, constraints and the definition of done. Reference material goes in clearly delimited sections, commonly with XML-style tags or headed blocks, so the model can tell instructions from data. The user turn carries the specific request. An output specification states the format and what to do when the answer is unknown. Variable parts are filled from a template so the fixed text stays identical across calls and can be versioned and cached.

Writing instructions that hold

Be explicit and direct, as if briefing a capable colleague with no background on the account. State what to do instead of only what to avoid, and give the reason behind a rule, since models generalize better from intent than from a bare prohibition. Put long documents near the top and the question after them. Remove contradictory or leftover instructions; most erratic behavior traces back to a prompt that asks for two incompatible things.

Examples, reasoning and chaining

Few-shot examples

Zero-shot prompting gives only instructions; few-shot prompting adds worked input and output pairs. Examples are the strongest signal for format, tone and edge-case handling, and the model will imitate them closely, so make them varied and representative. A set of three to five diverse examples usually outperforms one, and near-identical examples cause the model to copy surface details. Include at least one example of the hard or negative case, such as the correct response when the input lacks the needed information.

Reasoning and decomposition

Asking the model to reason step by step before answering, known as chain of thought, improves accuracy on multi-step problems at the cost of more output tokens; models with a built-in reasoning mode expose this as a setting. Keep the reasoning separate from the final answer so code can parse only the answer. When one prompt tries to do too much, split it into a chain of smaller calls, each with a single job and its own check, which makes failures easy to locate.

Context engineering: selecting and assembling what the model sees

Selection, compression and ordering

The model can only use what is in the window, and everything in the window competes for attention. Context engineering treats each call as an assembly step: choose the minimal set of facts, documents, history and tool results that the current step requires; compress the rest into summaries or references that can be fetched on demand; and order the result so stable content comes first and the live question last. For long-running work, keep durable state outside the window in files or a database and load slices of it, instead of carrying an ever-growing transcript.

Iterating on prompts with evaluations

Beyond the basic loop

Prompts are code and should be handled as such: stored in version control, changed through review, and gated by a regression run on a fixed case set before release. Two additions make the loop sturdier. First, re-run the full set whenever the model id changes, since a prompt tuned for one model version can regress on the next. Second, keep failures found in production as new cases, so the set tracks the real input distribution instead of the one imagined at design time.

Sources

Related lessons

Practice this
How LLMs work inside applications: tokens, context windows and model APIsStructured outputs and tool calling