# Auspex: Know how well you actually know

> Pure, tested calibration math: Brier score + confidence-binned reliability diagram + an over/under-confidence read. Boundary cases pinned by 8 unit tests.

Auspex lets you record predictions with a confidence level and then measures how well-calibrated you actually are. The math is pure and tested: a Brier score, a confidence-binned reliability diagram, and an over/under-confidence read. It is a from-scratch, dependency-light build whose boundary cases are pinned by 8 unit tests, and you can download and run it locally.

## About the app

Statistics + data viz

## What to take away

- Pure, tested calibration math: Brier score + confidence-binned reliability diagram + an over/under-confidence read. Boundary cases pinned by 8 unit tests.
- Download: Auspex (ZIP)

## Tags

Free software, Auspex

## Read the full write-up

/downloads/auspex.zip

## Terms used in this piece

- https://dfieldsolutions.com/en/glossary/llm-evaluation.md

## We build this for clients

https://dfieldsolutions.com/en/services/full-stack

## More from the lab

- https://dfieldsolutions.com/en/lab/sylex.md — Sylex: Make sense of the slashes
- https://dfieldsolutions.com/en/lab/kindra.md — Kindra: Miss a beat, not the rhythm
- https://dfieldsolutions.com/en/lab/voxara.md — Voxara: Argue better. Lose gracefully
- https://dfieldsolutions.com/en/lab/wraithward.md — Wraithward: Spot the bait before you bite

---

Source: https://dfieldsolutions.com/en/lab/auspex
DField Solutions — Torony köz 5-1.a, 2120 Dunakeszi, Hungary — +36 20 722 0002 — dezso@dfieldsolutions.com
Booking: see https://dfieldsolutions.com/en/contact