Blog — OpenRouter Blog

Confidence Thresholds for Model Escalation Routing

Confidence-based escalation keeps most requests on a cheap model and sends only the ones it is unsure about to a stronger one. This guide covers forcing a numeric confidence field with structured outputs, setting the threshold from error rates on your own traffic, tuning it against accuracy, cost, and latency, routing the escalation in your code, and re-tuning it after launch.

October 1, 2026

Cost vs. Quality Tradeoff Framework for Agent Models

The cheapest model that clears your quality bar is usually not the model at the top of a leaderboard. This framework sets the bar for one task, measures cost per quality point across a cheap, a mid-tier, and a frontier model on your own examples, and picks the cheapest one that clears the bar with margin. It uses live catalog prices and the cost each OpenRouter response reports.

October 1, 2026

How to Gate Pull Requests on LLM Evals in CI

A one-line prompt change can ship an agent that tells customers the wrong refund window, and nothing in a normal CI pipeline checks what the model says. This guide builds a fixed eval set for a support agent, a script that exits non-zero below a measured threshold, and a GitHub Actions job that blocks the merge when the pass rate drops.

October 1, 2026

AI Agent Regression Testing After a Prompt or Model Change

An agent's behavior can change when you edit a prompt, swap a model, change a tool schema, or change what retrieval returns. This guide covers the locked case set, the per-case behavioral contract, and how to run the same suite against two concrete model slugs through OpenRouter so the diff shows what the change did.

September 30, 2026

Building a Golden Eval Dataset from Production Traffic

A golden eval dataset is a curated set of production inputs with reviewed expected outputs, versioned in Git and run before every deploy. This guide covers the five steps to build one from live traffic and how to run the same set against many candidate models through one API.

September 30, 2026

How to Test Tool-Calling Accuracy in AI Agents

An agent can call the wrong tool, or call the right tool with the wrong arguments. This guide covers three ways to test each failure mode, a Python harness that grades both, and how to run the same test cases against several tool-capable models through OpenRouter.

September 30, 2026

Image-to-Video AI Models Compared: Cost, Resolution, and Control

If you already have the image a video should start from, the model choice comes down to what has to happen after that frame. This post compares the Veo 3.1, Seedance, Kling, and Grok Imagine Video lines on duration, resolution, first-frame and last-frame control, generated audio, reference inputs, and current per-second price, then shows how to submit an image-to-video job with the TypeScript SDK.

September 29, 2026

Best Embedding Models in 2026

An embedding model decides what your retrieval system can find. We shortlisted the embedding models in our catalog for English RAG, multilingual retrieval, code search, text-and-image retrieval, and low-cost indexing, sent live requests to each one, and recorded their prices, context windows, and default vector sizes so you can pick a candidate and test it on your own data.

September 23, 2026

How to Use Jev: Moderation with the Jev API in TypeScript

A method for putting Jev to work on a new problem, worked end to end on marketplace listing moderation: what stays in code, what Jev sees, how to phrase the questions, and how to turn its probabilities into publish, hold, or reject.

September 23, 2026

Batch API: half-price inference by bundling requests

Send a whole workload in one POST, collect the results within 24 hours, and typically pay half the per-token price. Across 230k+ batches that completed over our two week beta period, the median finished in 7 minutes.

September 22, 2026

View all posts RSS

Confidence Thresholds for Model Escalation Routing

OpenRouter is Joining Stripe

Ori Eval: Find the Best Model for What You're Building

Surpassing Frontier Performance with Fusion

Cost vs. Quality Tradeoff Framework for Agent Models

How to Gate Pull Requests on LLM Evals in CI

AI Agent Regression Testing After a Prompt or Model Change

Building a Golden Eval Dataset from Production Traffic

How to Test Tool-Calling Accuracy in AI Agents

Image-to-Video AI Models Compared: Cost, Resolution, and Control

Best Embedding Models in 2026

How to Use Jev: Moderation with the Jev API in TypeScript

Batch API: half-price inference by bundling requests