Noesion Radarnoesion.ai
Release2026-08-03

Prime Radiant Inc published smevals, an MIT-licensed open-source Python framework for running evaluations against small and large language models, distributed on PyPI and installable via uv or pip. The framework organizes work into Evals (directories containing an eval.yaml plus tasks/, configs/, graders/ and checkers/), Runs produced by executable Runner programs, and Grades produced by Graders composed of ordered Checks. Runners and Checkers communicate through environment variables such as SMEVALS_MODEL, SMEVALS_PROMPT and SMEVALS_RUN_DIR, with stdout captured as output.txt; non-zero Runner exits mark harness failures that are never graded or counted toward the -n sample target. Commands include run, grade, report, serve (a live web UI on port 7001) and build (a static site). Runs are immutable on disk and each Grade stores a byte-for-byte snapshot of its Grader, allowing regrading without re-running models. The repository showed 238 stars and 40 commits.

Send this to someone who needs it

Shares the story and its sources. Nothing about you.

What does this mean for your job?

This is the story as everyone gets it. Once a week we send you the version written for your role — what changed, why it matters for the work you actually do, and one thing to try. Free while we tune it.