Vector database

The successor to prompt engineering: deliberately curating what enters the model's context window - system prompt, retrieved docs, tools, memory. Goal is max accuracy on the fewest tokens. A model only knows what you put in front of it.

A proxy layer between your app and LLM providers (OpenAI, Anthropic): routing, retries, caching, rate-limits, key management, cost tracking and failover. One place to see your whole AI bill - and no lock-in to a single vendor.

Send each request to the cheapest model that can handle it: a small model for easy queries, a frontier model for hard ones - often decided by a classifier. Cuts inference cost dramatically, frequently 5-10× on real traffic.

A RAG variant that retrieves over a knowledge graph (entities + relationships) instead of flat text chunks. Lets the model answer multi-hop questions ("how is X connected to Y?") that pure vector search misses.

How an AI agent persists state across turns and sessions: short-term (the context window), long-term (a vector store / DB of facts), and episodic. The difference between an agent that forgets and one that learns your business.

Model-generated training and eval data for when real data is scarce, sensitive (GDPR), or imbalanced. Useful, but you must check quality and diversity - otherwise you bake the model's own blind spots into your system.