Skip to main content
RapidClaw

Evaluate Microsoft Foundry Model Router

Use Microsoft's evaluation toolkit to compare Model Router with a direct model deployment on the prompts your team actually uses.

Run the sample evaluation

Compare Model Router and a fixed baseline using Microsoft's synthetic sample data. Your browser calculates the summary without making model calls or asking for Azure credentials.

Microsoft Foundry Model Router selects an eligible model for each request. Its routing mode influences how it balances quality and cost. Microsoft's open-source Model Router Auto Evaluation toolkit sends the same prompts to the router and a fixed baseline, then reports response quality, estimated token cost, latency, and which models the router selected.

Start with the sample report

Follow the quickstart to generate a sample dashboard with synthetic data and no Azure credentials. Open results/demo/dashboard.html and look through the quality, cost, and latency charts. The sample numbers illustrate the report format. Microsoft's quickstart

Benchmark your own workload

  1. Choose representative, approved prompts. Include routine tasks such as summarizing records, harder tasks such as explaining exceptions, and tool-call examples if your workload depends on them. Use synthetic or redacted business data.
  2. Configure your Foundry Model Router deployment and the direct deployment you use today as the baseline. Use a separate judge deployment where practical. Confirm the configured token rates for your region and deployments before interpreting the cost estimates.
  3. Run a small live evaluation first. Live runs make billable calls to the configured endpoints. Keep the dataset, baseline, judge, and request settings fixed when comparing two routing modes.

Read the results by task

  • Quality. Use the judge scores to identify differences, then review losses and ties against your own acceptance criteria. Check critical tasks individually, including whether any required tool calls and arguments were correct.
  • Cost. Compare the estimated token cost for the same batch of prompts. Use the model distribution to see which models handled the work.
  • Latency. Look at the median and p95, the response time that 95% of measured requests met or beat. Averages can hide slow responses on complex tasks.

Use the results for RapidClaw model selection

Run the toolkit separately in your Azure environment. RapidClaw Setup deploys a verified direct-model plan by default, and Command Center lets you assign a deployed model per agent. Use the benchmark to decide which models warrant a controlled agent test. In that test, verify the full workflow, including connector permissions, approvals, tool calls, and observed usage.

Model Router Auto Evaluation repository · Microsoft Foundry Model Router documentation

Related products and guides