# Prompts are code: eval them like code

> A prompt that cannot be regression-tested is a script nobody dares to refactor. Versioned, evaluated, diffed — the same hygiene as any deployed artefact.

Shipping an agent means changing the prompt every week. Without an eval harness, each change is a coin flip — the fix for one failure quietly breaks two behaviours nobody checked.

## The eval set is the contract

Fifty to two hundred representative inputs with expected outputs — or at least expected properties — turn 'seems better' into a number. Build it from real transcripts, not from what the team imagines users ask.

## Judge with a judge, spot-check with eyes

An LLM-as-judge scores each run against rubric criteria and scales to every change; a weekly human read of ten random outputs catches what the rubric forgot to ask. Neither alone is enough.

## Version the prompt like the code

Prompt text in version control, eval scores in CI on every change, and a rollback that takes minutes rather than a meeting. The prompt is a deployed artefact and deserves the same hygiene.

## What to take away

- Build the eval set from real transcripts, not guesses.
- LLM-judge at scale, human spot-check weekly.
- Prompt changes go through CI like any other code.
- Keep rollback trivial — prompts deploy too.

## Tags

LLM, Evals, CI, Prompting

## We build this for clients

https://dfieldsolutions.com/en/services/ai-automation

## More from the lab

- https://dfieldsolutions.com/en/lab/rag-grounding.md — RAG that cannot quote a wrong price
- https://dfieldsolutions.com/en/lab/n8n-self-hosted-vs-zapier.md — Self-hosted n8n or Zapier — where the bill actually differs
- https://dfieldsolutions.com/en/lab/mcp-servers-production.md — MCP servers in production, not in the demo
- https://dfieldsolutions.com/en/lab/invoice-pipeline-automation.md — The invoice pipeline that ended the Friday admin block

---

Source: https://dfieldsolutions.com/en/lab/prompt-evals
DField Solutions — Dunakeszi, Hungary — dezso@dfieldsolutions.com
Booking: see https://dfieldsolutions.com/en/contact