Support tooling
Compare runs
Pick two runs to see where the models agree, where they differ, and which one matched the gold label.
| Metric | A · anthropic | B · openai |
|---|---|---|
| Category accuracy | 68% | 45% |
| Severity exact | 77% | 73% |
| Severity MAE | 0.23 | 0.27 |
| Failed | 0 | 0 |
| Output tokens | 10.6K | 13.1K |
| Median latency | 11.9s | 8.0s |
Input tokens are left out on purpose: each provider uses its own tokenizer, so counts are not comparable. Output tokens include hidden reasoning tokens.
Tickets
| Ticket | Gold | A · anthropic | B · openai |
|---|---|---|---|
| T005 | bug high | bug (correct) medium (wrong) | config (wrong) medium (wrong) |
| T011 | billing critical | billing (correct) critical (correct) | config (wrong) high (wrong) |
| T013 | bug high | bug (correct) high (correct) | config (wrong) high (correct) |
| T018 | bug medium | bug (correct) medium (correct) | config (wrong) medium (correct) |
| T022 | config medium | how_to (wrong) low (wrong) | bug (wrong) medium (correct) |
| T023 | bug high | bug (correct) high (correct) | config (wrong) medium (wrong) |
| T024 | bug high | how_to (wrong) medium (wrong) | config (wrong) medium (wrong) |
| T026 | bug high | how_to (wrong) medium (wrong) | config (wrong) medium (wrong) |
| T027 | config medium | bug (wrong) medium (correct) | config (correct) medium (correct) |
| T031 | how_to medium | how_to (correct) medium (correct) | config (wrong) medium (correct) |