AI Gateway

The successor to prompt engineering: deliberately curating what enters the model's context window — system prompt, retrieved docs, tools, memory. Goal is max accuracy on the fewest tokens. A model only knows what you put in front of it.

Send each request to the cheapest model that can handle it: a small model for easy queries, a frontier model for hard ones — often decided by a classifier. Cuts inference cost dramatically, frequently 5-10× on real traffic.

A RAG variant that retrieves over a knowledge graph (entities + relationships) instead of flat text chunks. Lets the model answer multi-hop questions ("how is X connected to Y?") that pure vector search misses.

How an AI agent persists state across turns and sessions: short-term (the context window), long-term (a vector store / DB of facts), and episodic. The difference between an agent that forgets and one that learns your business.

Model-generated training and eval data for when real data is scarce, sensitive (GDPR), or imbalanced. Useful, but you must check quality and diversity — otherwise you bake the model's own blind spots into your system.

An AI architecture where the model retrieves relevant documents from your own data before answering, and only reasons over that context. Kills ~80% of hallucinations.