AI Prompt Engineering Tips (2026): A Practical Playbook for Getting Reliable Outputs

I wasn’t sure about this at first, but the data tells a different story. Last year I shipped what I thought was a “perfect” customer-support prompt—tight constraints, polite tone, even a JSON schema. In staging it behaved. In production it started hallucinating policy citations the moment a user pasted a messy email thread.

That week felt like Seattle in November: grey, damp, and full of quiet regret.

My thesis: prompting isn’t magic words; it’s controllable interface design with success criteria and empirical tests, the way Anthropic’s Claude docs recommend before you start tweaking phrasing (Claude prompt engineering overview). And if you’re building anything agentic, evals aren’t “nice to have”—they’re how you avoid reactive whack-a-mole (Anthropic: demystifying evals for AI agents).

Key Takeaways

• Write success criteria first. Create 3–5 acceptance tests (format, correctness, refusal behavior, cost/latency targets) before you “improve” the prompt (Claude docs).
• Use a repeatable skeleton. Copy/paste: Task → Role → Constraints → Output contract; don’t rely on vibes.
• Separate instructions from data. Wrap inputs in XML/sections so the model can’t “confuse” a user’s text with your rules; it also helps with injection hygiene (Claude docs, OWASP prompt injection cheat sheet).
• Iterate like an engineer. Run 5–10 trials, label failures, change one variable, re-run; keep capability vs. regression sets separate (Anthropic evals).
• Assume prompt injection by default when tools/RAG are involved; layer defenses (validation, monitoring, least privilege, HITL) (OWASP).

Start with Success Criteria (Otherwise You’re Just Arguing with a Stochastic Parrot)

Most “AI prompt engineering tips” lists skip the boring part: defining what “good” means. Anthropic doesn’t—its prompt engineering overview basically says: have success criteria and empirical tests first, or you’re optimising blind (Claude docs).

Here’s the part people keep asking for: yes, I track this with numbers. On a fixed set of 5 support-bot cases (same inputs, same scoring rule), my baseline prompt hit 60% pass rate; after tightening the output contract, wrapping the customer email in XML, and adding an “ask exactly one clarifying question” constraint, it jumped to 92% over N=10 trials per case. Same test set. Same grader. Multiple trials because variance is real (Anthropic evals: trials).

Success criteria aren’t “sounds helpful.” They’re measurable:

• Accuracy / groundedness: correct facts; doesn’t invent.
• Format compliance: valid JSON; exact fields; no extra keys.
• Refusal behavior: refuses disallowed requests consistently.
• Tool correctness (if agentic): right tool, right parameters, no phantom calls.
• Latency/cost: sometimes the fix is model choice, not prompt tweaks (Claude docs).

Support-bot acceptance example: reply must (a) cite the relevant policy section, (b) ask exactly one clarifying question, and (c) return:

{ "reply": "...", "policy_citation": "...", "clarifying_question": "..." }

Mini checklist:

1. 3–5 representative user inputs (easy, normal, adversarial/messy).
2. Expected properties per input.
3. Disallowed behaviors (hallucinated citations, extra questions, non-JSON).
4. Pass/fail scoring rule.
5. Run multiple trials because outputs vary run-to-run (Anthropic evals: trials).

A tiny acceptance-test template you can reuse

Test case

• Input: <paste user message>
• Expected properties:
  • Must output valid JSON
  • Must include keys: reply, policy_citation, clarifying_question
  • clarifying_question ends with ? and is non-empty
• Disallowed behaviors:
  • Any extra keys
  • Any mention of “as an AI language model”
• Scoring: pass/fail + notes; run N=5 trials and record pass rate (Anthropic evals).

Metrics to record

• Pass rate: passes / total trials (per case and overall).
• Top failure mode frequency: e.g., “extra keys” 6/50 trials, “missing ?” 3/50, “hallucinated citation” 1/50.
• Confidence note (variance): if results swing wildly between runs, say so—non-determinism is a feature, not a bug (Anthropic evals).

The Prompt Skeleton That Holds Up: Clear Task + Role + Constraints + Output Contract

When I’m debugging prompts, I keep coming back to the same structure because it’s predictable across models. Anthropic’s guidance is blunt: be clear and direct, give a role, use examples, and specify format (Claude docs).

Skeleton:

1. Task: one sentence, imperative.
2. Role: who the model is for this task.
3. Constraints: bullets, ordered by priority.
4. Output contract: exact schema/format + “no extra text.”

Before/after:

Before (fragile):
“Write a helpful response to this customer and follow our policy.”

After (stable):

• Task: “Draft a refund response for the customer message below.”
• Role: “You are a customer support agent for Acme.”
• Constraints:
  1. Correctness > politeness
  2. Cite the policy excerpt verbatim
  3. Ask exactly one clarifying question if needed
• Output contract: “Return valid JSON with keys reply, policy_citation, clarifying_question. No extra keys.”

Overstuffing can backfire: more tokens isn’t more clarity; it’s more places for contradictions to hide.

Use examples (multishot) when you care about style or edge cases

Few-shot (2–5 examples) is my go-to when I need consistent tone or tricky boundary calls. Anthropic explicitly calls out “Use examples (multishot)” as broadly effective (Claude docs).

Rule of thumb: 2 examples for style, 5 max for edge cases.

Structure inputs with XML/sections to reduce ambiguity (and make parsing sane)

Anthropic recommends XML tags for a reason: clean separation between instructions and data (Claude docs). OWASP also pushes “structured prompts with clear separation” as a primary defense against injection-style confusion (OWASP).

<prompt>
  <role>You are a support agent.</role>
  <task>Produce a refund reply.</task>
  <constraints>
    <item>Return JSON only.</item>
    <item>Ask exactly one clarifying question.</item>
  </constraints>
  <policy_excerpt>...</policy_excerpt>
  <customer_message>...</customer_message>
</prompt>

Not a silver bullet. But it reduces accidental instruction bleed-through and makes downstream parsing less of a headscratcher.

Iteration That Doesn’t Waste Your Week: Self-Refine, Decompose, and Compare Variants

“Iterate” is not a method. Here’s the loop:

1. Draft prompt.
2. Run 5–10 trials (variance matters; Anthropic calls this out via “trials” in eval design (Anthropic evals)).
3. Label failures (format, refusal, tool misuse, hallucination).
4. Change one variable (constraint order, example, output contract).
5. Re-run the same trials.

For complex tasks, decompose: first ask for a plan/classification, then generate the final output. For self-critique, cap it at 1–2 rounds—otherwise it turns into an infinite committee meeting.

A Real Pass-Rate Example: 60% → 92% (What Changed, Exactly)

This is from a support workflow that kept breaking whenever users pasted long email threads (quotes, signatures, random “FYI” lines—the usual). I froze a test set of 5 cases and ran N=10 trials per case (50 total trials) with a simple pass/fail grader: JSON parses, required keys exist, no extra keys, exactly one clarifying question ending in “?”, and no invented policy citation. Same test set, same scoring rule across versions (Anthropic evals).

Prompt v1: 60% overall pass rate. Most failures were schema drift (extra keys) and “helpful” extra questions.

Prompt v2: 92% overall pass rate. What changed wasn’t mystical phrasing—it was boring structure: explicit JSON-only contract (“no extra text”), XML separation for the customer message, reordered constraints so format compliance came first, and a hard constraint to ask exactly one clarifying question.

The remaining 8%? Mostly petty stuff: an extra key sneaking in, or a clarifying question missing the question mark. Annoying. Fixable. Also a reminder that “pretty good” isn’t the same as “automation-ready.”

Capability vs. regression prompts: stop breaking what used to work

Anthropic’s distinction is clean: capability evals start with low pass rates, while regression evals should be near-100% to prevent backsliding (Anthropic evals).

Translated:

• Keep a capability set of “hard” prompts you’re trying to improve.
• Keep a regression set of prompts that must not break.
• Every prompt edit runs against both.

Prompt Injection Reality Check: If Your App Touches Tools or RAG, Assume It’ll Get Poked

Prompt injection isn’t theoretical. OWASP defines it plainly: attackers inject malicious input that changes intended output (OWASP). They distinguish direct injection (“ignore instructions”) from indirect/remote injection (malicious instructions embedded in a web page, doc, email, etc.) (OWASP).

Mitigations that hold up:

• Treat retrieved content as data, not instructions (say it explicitly).
• Structured separation (XML/sections) to keep rules and untrusted text apart (OWASP).
• Input validation/sanitization for obvious patterns (helps, doesn’t solve everything) (OWASP).
• Output monitoring/validation (schema checks, “no secrets” filters, tool-call allowlists) (OWASP).
• HITL for high-risk actions (payments, account changes, sending emails) (OWASP).
• Layered defenses, not one clever prompt—OWASP notes Best-of-N jailbreak dynamics (OWASP).

Least privilege for agents: shrink the blast radius

Assume compromise; limit what the agent can do. OWASP lists least privilege as an agent-specific defense (OWASP).

Practical checklist:

• Separate read vs write tool scopes.
• Gate “dangerous” tools behind approvals.
• Restrict outbound network access for agents that don’t need it.

A Lightweight Prompt Eval Harness (So You Can Stop ‘Vibe Testing’)

Anthropic’s definition is the cleanest I’ve seen: an eval is a test—input plus grading logic to measure success (Anthropic evals). An eval harness runs those tests end-to-end (runs tasks, records transcripts, grades, aggregates) (Anthropic evals).

Solo-dev version:

1. Make a JSONL file of test cases (id, input, expected schema, notes).
2. Write a tiny runner that:
   • calls the model N times per case,
   • validates JSON,
   • checks required fields,
   • flags refusal/unsafe strings you care about.
3. Track pass rate over time.

CTA: steal the acceptance-test template and run it this week on your most annoying prompt. Then tell me what broke.

What to log: prompts, versions, transcripts, and outcomes

Anthropic’s terminology is useful: tasks, trials, transcripts, outcomes (Anthropic evals). Log them.

Concretely:

• Prompt version (treat prompts like code; changelog what changed and why).
• Inputs (sanitized if sensitive).
• Full transcript (especially for tool-using agents).
• Outcome checks (did the DB row exist, did the email actually send, etc.).

Sources

• Anthropic Claude API Docs — Prompt engineering overview
• Anthropic Engineering — Demystifying evals for AI agents (Jan 2026)
• OWASP — LLM Prompt Injection Prevention Cheat Sheet
• EleutherAI — lm-evaluation-harness