Reproducible product benchmark · 19 August 2026
32 visits in.
32 dispatch-ready.
One synthetic operating workbook passed intake, public-road routing, optimization, independent proof, exact replay, recovery, and export with no dropped work.
Observed outcome
100%
of input work survived intake, planning, proof, replay, and export.
- Input work
- 32 workbook visits
- Assigned
- 32 exactly once
- Unassigned
- 0 explicit misses
- Hard-rule proof
- 13/13 categories passed
- Pickup → delivery
- 8/8 pairs preserved
- Repeat run
- Exact same plan on replay
Agentic Harness A/B · same road matrix
Proof added in 1.5 ms. Route quality unchanged.
Ten sequential runs held the workbook, directed OSRM matrix, rules, and route candidate constant. RouteThai then added independent validation, a seven-tool execution trace, replay binding, and evidence export.
Controlled disruption
One vehicle removed. All 8 missed visits recovered.
The direct plan covered 24/32 visits. RouteThai evaluated explicit recovery options and restored 32/32 in 453 ms. The change remained a preview until operator approval.
Direct plan24/328 unassigned
Verified recovery32/32453 ms
200-stop search benchmark · 10 complete workloads
41.7× faster search. Within 0.48% median route distance.
Partitioning creates the fast plan; neighbor-pair repair then improves every incumbent before independent validation. Every workload keeps all 200 stops in the denominator.
This is the tested 200-stop tier—not a claim of 10,000-order scale. The comparison holds the synthetic workload and directed matrix constant; its fleet-wide comparator is bounded search, not a proven global optimum.
Why an operator should care
Watch the agent work. Inspect every decision.
Input, assignment, and export counts reconcile. Any miss must remain visible.
Capacity, time, tags, zones, breaks, depots, route limits, and precedence are independently checked.
The workbook, road matrix, route result, and validation evidence can be downloaded and inspected.
Business workflow acceptance · 22 August 2026
Eight operating patterns. One verified planning contract.
Retail delivery, courier pairs, field service, cold chain, multi-depot distribution, reverse collection, home healthcare, and a multi-day backlog all ran through intake, typed rules, directed roads, planning, independent proof, exact replay, and export.
12 stops · 2 vehicles · proof passed
8 stops · 2 vehicles · proof passed
10 stops · 2 vehicles · proof passed
10 stops · 2 vehicles · proof passed
14 stops · 4 vehicles · proof passed
12 stops · 3 vehicles · proof passed
10 stops · 2 vehicles · proof passed
30 stops · 6 vehicles · proof passed
These are synthetic functional acceptance runs, not customer ROI or a competitor ranking. The large-file scenario keeps the 180-row source intact and plans one explicit 30-stop service cohort; it does not imply a 5,000-stop optimization run.
Independent hard-window dataset · 2 September 2026
100 hard-window stops. 100 accounted for.
RouteThai planned the pinned Solomon C101 fixture from the DIMACS VRPTW controller, then an independent validator recomputed capacity, hard windows, depot return, and complete customer accounting.
Download complete conformance evidence100 customers · JSONInspect the pinned upstream fixtureSolomon C101 · source
This is customer-neutral functional conformance, not customer ROI. RouteThai's minute clock rounds each travel leg up, so this lane deliberately makes no best-known-solution comparison; solver-quality gaps stay in the separate CVRPLIB lane.
Dispatch output
Four routes. One accountable result.
85.7 road km · 2h 36m aggregate drive · 53.75% mean load
RouteVisitsRoad kmDriveLoad
east815.838m56.7%
north817.535m51.7%
south834.348m50.0%
west818.235m56.7%
How the claim is verified
Five gates between upload and handoff.
- 01Server-verified intakePassed
The uploaded workbook, selected Stops sheet, and 32 prepared rows reconcile to this run.
- 02Directed road authorityPassed
A complete 33-node public-OSRM matrix was saved, checked, and used without straight-line fallback.
- 03Independent hard-rule proofPassed
All 13 supported rule categories and all 8 pickup-delivery pairs passed.
- 04Deterministic replayExact
The identical road matrix produced the exact same route plan twice.
- 05Export reconciliationPassed
Every workbook visit appeared exactly once in the coordinate-free evidence export.
Inspect the evidence
Reproducible by design.
Download the same source and evidence used for these numbers. Coordinates remain in the synthetic workbook; the result evidence and matrix artifact contain no coordinate fields.
Public benchmark portfolio
27 public problems. Valid every time.
Median route distance landed 3.01% above the proven optimum across the complete CVRPLIB Set A. Every instance—from 31 to 79 customers—used the same code, search budget, single thread, and independent validator. No instance was omitted.
Release improvement · A-n32-k5 regression anchor
2.45% lower benchmark route distance than the previous public release.
The same public instance, objective, single thread, and 200,000-move ceiling were independently validated on the same Apple M2. This is a release regression result—not the 27-instance portfolio average.
| Measure | Previous | Current | Change |
|---|---|---|---|
| Objective distance | 816 | 796 | 2.45% lower |
| Gap to optimum | 4.08% | 1.53% | 62.50% narrower |
Download complete benchmark results27 instances · JSONCheck the official instance catalogCVRPLIB catalog
Mean gap: 5.52%. The widest observed gap—34.85% on A-n45-k6—remains included. A-n32-k5 finished 1.53% from optimum.
Same-machine, one-second references keep the score honest: PyVRP solved 27/27 at 0.13% mean gap; OR-Tools solved 26/27 and its no-solution runs remain in the denominator. RouteThai wraps that optimization step in an agentic workflow you can inspect: file, rules, road plan, proof, dispatch, and execution evidence.
Three evidence lanes
Benchmark the route. Benchmark the agent. Then benchmark the operation.
Each lane answers a different buyer question without blending synthetic speed, mathematical quality, and customer ROI into one flattering number.
Complete public instance sets, proven optima, equal machine budgets, and independent route validation.
Real file parsing, directed roads, hard-rule proof, Harness trace, recovery, deterministic replay, and reconciled export.
The same workflow measured with design partners for planning time, distance, violations, dispatch adoption, completion, and cost.
Give the agent your next file
Put real work in motion.
Upload the work. The agent will map, route, prove, and prepare the next decision in your private workspace.