Braintrust Blog
npub12rqy8jsv0musnr2lyartgmwrddn7yyqw29f0zv2x5rd5l2gx4glsvc7zqz@drss.io
Latest insights, tutorials, and updates from the Braintrust team. Learn about AI evaluation, LLM observability, and best practices for building reliable AI products.
Follow into
Compare Kimi K3 and DeepSeek V4
Save into
12 Aug 2026
Kimi K3 and DeepSeek V4 Flash are now built-in models, alongside GLM-5.2, in playgrounds, prompts, and scorers.
Testing whether language model harnesses transfer the wrong strategy
Save into
7 Aug 2026
An RLM harness can carry a problem-solving strategy from one task to another. This eval tests what happens when the strategy it carries is wrong.
Trace and improve Cloudflare Agents
Save into
5 Aug 2026
Cloudflare Agents traces can go to Braintrust via OpenTelemetry, or through first-class JavaScript instrumentation that runs in the Cloudflare Workers runtime.
Behavior specs, an open standard for supervising long-horizon agents
Save into
29 Jul 2026
Behavior specs are an open standard from Braintrust and Basis for defining and evaluating how an agent should behave across a whole trajectory, so you can supervise the process and not only the
Kimi K3 vs Claude Sonnet 5 for frontend design agents
Save into
27 Jul 2026
Kimi K3 in place of Sonnet on the same Paper vs Figma design eval. It matches on quality when it finishes, but is slower and times out more often.
Paper MCP vs Figma MCP for frontend agents
Save into
20 Jul 2026
An independent eval of the Paper and Figma MCP servers as design tools for a coding agent, scored on visual quality, consistency, and cost.
How we chose the model behind Topics with Baseten
Save into
15 Jul 2026
How Braintrust and Baseten evaluated small models for Topics, using evals to improve Issues recall while keeping trace intelligence affordable at scale.
Evaluating the GPT-5.6 family
Save into
10 Jul 2026
I mapped the GPT-5.6 family across task families and difficulty, with an Anthropic comparison, to find the cheapest model that clears your reliability bar.
Evaluating speech-to-text models
Save into
9 Jul 2026
I ran a controlled eval across six speech-to-text providers, 240 audio clips, and eight content domains to find where voice agents break and which STT model comes out ahead.
Faster phrase search with shingled bloom filters in Brainstore
Save into
7 Jul 2026
How trigram shingles in Brainstore's bloom filters keep phrase search fast as trace data grows large and cold.
Evaluating the USA vs Belgium World Cup matchup
Save into
6 Jul 2026
Applying the best-performing Parallel configuration from the World Cup eval to inspect the USA vs Belgium matchup as a source-backed research map.
From World Cup matchups to research maps: evaluating Parallel's web research agents
Save into
2 Jul 2026
We used Braintrust to evaluate six Parallel web research configurations across 48 World Cup matchups
Agent observability for startups
Save into
30 Jun 2026
A new program that gives early-stage teams access to the same evals and observability platform that leading AI companies run in production.
How to eval stateful agents
Save into
26 Jun 2026
Stateful agents accumulate context, take real actions, and depend on live systems. Here's how to build evals that account for all three.
Using Braintrust to evaluate agentic setups from large-scale Hugging Face data
Save into
24 Jun 2026
We pulled 1,781 real agent traces from Hugging Face into Braintrust, scored every run, and found that the wrapper around your model explains 7× more variation in success than the model itself.
How to test agent cost-efficiency with Braintrust
Save into
17 Jun 2026
How the cheapest model is not always the cheapest system, and how evals plus control logic reduce cost per resolved request.
How to use Braintrust with any framework or provider
Save into
16 Jun 2026
Instrument your app once with SDKs or OpenTelemetry, then keep using the agent frameworks and model providers you already prefer.
How we made continuous trace intelligence possible at scale
Save into
4 Jun 2026
A deep dive into the architecture of Topics.
AI observability is active observability
Save into
1 Jun 2026
Topics is now generally available. It marks a shift from AI observability to active observability, where Braintrust works continuously in the background to surface answers before you think to ask for
Automate pattern discovery with Topics, now generally available
Save into
1 Jun 2026
Topics reads your production AI traces and automatically clusters them into named categories, so you can go from pattern discovery to validated fix without switching tools.
The six generations of AI agents and how to eval them
Save into
21 May 2026
Agent architectures have evolved through six generations, and each one demands a different eval strategy. Walk through every generation with a single SRE incident response example.
How to improve your golden datasets with human review
Save into
21 May 2026
Turn production traces into golden datasets by adding human review to your eval workflow, then use that ground truth to improve scorers over time.
How to evaluate multi-turn conversations
Save into
11 May 2026
Learn how to score multi-turn conversations by combining per-turn and per-conversation evals, then automating it all in production.
Why your traces and evals belong in the same place
Save into
11 May 2026
What you can catch, fix, and automate when traces and evals live in the same platform.
How to earn stakeholder trust with evals and observability
Save into
28 Apr 2026
How PMs can use Braintrust dashboards, custom trace views, and Loop to turn AI evals and production behavior into something stakeholders can read.
How to prepare for AI compliance and governance
Save into
13 Apr 2026
The EU AI Act and ISO/IEC 42001 are raising the bar for AI governance. AI observability helps teams meet these requirements with production-level evidence.
Agentic eval development with the Braintrust CLI
Save into
8 Apr 2026
Use coding agents and the Braintrust CLI to debug failing evals, iterate on prompts, and close the loop between observability and code.
How Brainstore works: architecture for AI observability at scale
Save into
6 Apr 2026
A deep dive into the architecture of Brainstore, Braintrust's custom database built for AI observability workloads.
Braintrust CLI and MCP
Save into
3 Apr 2026
Learn when to use the Braintrust CLI and MCP depending on where you are in the AI development workflow.
Evals are the new PRD
Save into
27 Mar 2026
Why AI product managers should replace traditional PRDs with evals, and how the eval flywheel becomes the operating system for AI product development.
What is AI observability?
Save into
19 Mar 2026
AI observability is a new infrastructure category built on traces, evals, and feedback loops. Learn what it means, why it's technically hard, and how it changes AI product development.
Evals for PMs: A practical guide to AI product quality
Save into
17 Mar 2026
Everything a product manager needs to know about evals, from building datasets and scoring criteria to running experiments and integrating evals into your product development process.
Keep building with the Starter plan
Save into
16 Mar 2026
Starter is a new Braintrust plan with no platform fee, designed to scale with your needs.
Supporting privacy and compliance for EU teams
Save into
12 Mar 2026
Braintrust's decoupled architecture gives EU teams control over where their AI data lives, simplifying GDPR compliance and data residency requirements.
How to build your first offline eval
Save into
10 Mar 2026
A 10-step guide to going from a vibe to a working eval system, using a real Mermaid diagram generation project as an example.
