# 2026 email triage benchmark: GPT-4o vs DeBERTa 95th percentile latency vs recall

Carson Drake · November 11, 2026

> DeBERTa beats GPT-4o on p95 latency and cost per decision in 2026 email triage, while GPT-4o retains higher recall on threaded and ambiguous mail.

| Takeaway | Detail |
| --- | --- |
| DeBERTa wins p95 latency and cost per decision | On 2026 email triage, the DeBERTa-family encoder beats GPT-4o on 95th-percentile end-to-end latency and cost per decision, while GPT-4o keeps recall on threaded and ambiguous mail. |
| GPT-4o keeps recall on threaded and ambiguous mail | The encoder's recall loss is concentrated in threaded and ambiguous mail, so the split point must use per-class recall rather than aggregate recall. |
| Gate release on two same-host, same-holdout numbers | Measure p95 end-to-end latency and recall on the needs-action class at a fixed precision floor on the same holdout and same host. |
| Route only the classifier's low-confidence cases | Compute the split point from p95 latency and per-class recall, with latency averaged over 20 runs per tool per day on one host. |

On 2026 email triage, DeBERTa beats GPT-4o on p95 end-to-end latency and cost per decision, while GPT-4o keeps higher recall on threaded and ambiguous mail.

The routed-pipeline answer holds only when p95 latency and needs-action recall at a fixed precision floor are measured on the same holdout and host.

![2026 email triage benchmark](https://static.mm-ais.com/article-images-ai/2026-email-triage-benchmark-gpt-4o-vs-de-ai-c5157d11.jpg)

## How p95 email triage latency is clocked

The latency clock for this triage system starts at SMTP accept or webhook receipt and stops when the mail API acknowledges the label write; encoder inference is only one segment of that interval, so each message needs both timestamps logged rather than a model-call timer alone. The per-class recall denominator is the set of holdout messages whose ground-truth label is needs-action, measured at the fixed precision floor, so recall is true positives divided by that denominator rather than by all messages or predicted positives.

For the DeBERTa path, the measured work is tokenize subject plus first 256 body tokens, one forward pass, and argmax over labels; the series must include cold starts, retries, and queue waits alongside warm calls, because excluding them makes the p95 a fiction. If a retry changes the final label, the retried attempt still belongs in the same message's end-to-end interval.

For the GPT-4o path, clock prompt assembly, time-to-first-token, completion, and any retry or backoff through the same label-commit acknowledgement; do not report model latency as if prompt construction and API round trips were free. Both paths should be run on the same host and holdout so the p95 comparison is not confounded by infrastructure differences.

Report p95 over the full end-to-end series, not over a warmed subset, and pair it with needs-action recall at the precision floor from the same run; the release gate built on these two numbers is defined in the comparison table below, so this section only fixes how each number is clocked and scored.

Log, per message, receive timestamp, enqueue timestamp, model start, model end, label-write request, and API acknowledgement; the gap between receive and acknowledgement is the latency the user experiences, while intermediate stamps isolate queueing from inference. Reconcile missing acknowledgements as failures in the latency series rather than dropping them, because a dropped label has no completed end time.

![How p95 email triage latency is clocked — 2026 email triage benchmark](https://static.mm-ais.com/article-images-ai/2026-email-triage-benchmark-gpt-4o-vs-de-ai-2c64fe63.jpg)

## What published benchmark methods settle

The convergence finding is that independent benchmarks do not compare systems on anecdotes: they measure repeated identical tasks from one fixed host and report distribution statistics, so a cross-model claim survives only when the task set, host, and repeat count are held constant.

| Benchmark | Fixed element | Repeated element | Reported statistic |
| --- | --- | --- | --- |
| Hugging Face ai-coding-assistants-benchmark-2026 | Frankfurt VPS, 1 Gbps uplink | 20 runs per tool per day | First-token latency average |
| Assistant Benchmark | Same 15 tasks | 27 of 132 assistants tested | 1-10 real-use score |
| UBOS OpenClaw guide | Predefined workload | Repeatable runs | Latency, throughput, cost, accuracy, resource use |

Hugging Face's ai-coding-assistants-benchmark-2026 measures first-token latency from a Frankfurt VPS with a 1 Gbps uplink, averaged over 20 runs per tool per day. Copy that fixed-host, N-repeats structure, not its latency numbers, which belong to coding assistants rather than mail triage; the check is whether your own host, network path, and repeat count are written down before the first run.

Assistant Benchmark scored 27 of 132 assistants on the same 15 tasks with a 1-to-10 real-use score. The transferable lesson is the identical task set: if the two triage paths do not face the same messages, the comparison is a story rather than a measurement, and a reader cannot tell whether the model or the sample changed.

UBOS frames benchmarking as a repeatable, quantitative process that evaluates an agent against a predefined workload, measuring latency, throughput, cost per inference, accuracy, and resource use. That is the same discipline as the fixed-host repeat runs above: define the workload once, rerun it, and compare distributions rather than single observations; a one-off run is a demo, not a benchmark.

![What published benchmark methods settle — 2026 email triage benchmark](https://static.mm-ais.com/article-images-pixabay/2026-email-triage-benchmark-gpt-4o-vs-de-fb503b27.jpg)

## DeBERTa vs GPT-4o: the comparison table

The comparison is a measurement sheet, not a reputation score: every cell is filled from the same holdout and host, and the last column states how the number is checked rather than who claims it. Fill it by measurement, not reputation—no vendor claim, benchmark anecdote, or model name substitutes for a number produced on your mail, your host, and your labels.

| Decision metric | DeBERTa-class encoder | GPT-4o | How you check |
| --- | --- | --- | --- |
| p95 end-to-end latency | Lower | Higher | Measure the latency distribution on the holdout and report the 95th percentile |
| needs-action recall at precision floor | Lower on threaded and ambiguous mail | Higher on threaded and ambiguous mail | Fix the precision floor, then score recall on the needs-action class |
| cost per 1,000 emails | Lower | Higher | Measure total run cost against emails triaged on the same workload |
| low-confidence rate | Defines the residual | Not the routing trigger | Count decisions below the confidence threshold divided by total decisions |

Split routing wins this table: the DeBERTa-class encoder decides the high-confidence majority, and GPT-4o re-decides only the residual below the confidence threshold. Classifier-only wins latency and cost, GPT-4o-only wins ambiguous-thread recall, and neither wins both.

The low-confidence residual rule is the split point itself: route only the classifier's low-confidence decisions to GPT-4o, and set the threshold from the p95 latency budget and per-class recall rather than from convenience. A low-confidence decision is one whose score falls below the threshold; everything at or above it stays with the encoder. The threshold is not a tuning knob for accuracy alone—it is the point where the residual's added latency and cost still leave the whole pipeline inside the p95 budget and above the recall floor.

Tie-break rule: if more than one route satisfies both gate numbers, choose the one with the lower measured cost per 1,000 emails; if no route satisfies both, hold the release. This prevents a tie from being settled by model preference: the cheaper route that still clears both gates wins, and a failed gate stops the release regardless of the other metric.

## Cost per 1,000 emails, not per token

The cost unit for this decision is cost per 1,000 triaged emails, not cost per token. A per-token figure hides thread length, output verbosity, and retry behavior; the per-1,000-email figure forces every model and route to carry the same denominator: one thousand messages moved from receipt to a committed label. Include encoder GPU time, API tokens, labeling, and escalation review in that numerator.

Compute the token portion as (input tokens + output tokens) multiplied by the current per-million-token price taken from the provider's live pricing page on the benchmark day, then add amortized GPU hours for the encoder. Do not reuse a prior-year price in a 2026 comparison; record the retrieval date and page with the result so a price change is visible rather than silent.

Charge the holdout itself as a cost line. Labeling the fixed evaluation set with two annotators plus an adjudication pass is a one-time expense, and it belongs in the benchmark budget. If it is treated as free research overhead, a low-token model can look cheaper than the routed pipeline it is being compared against.

Escalation review can dominate at scale. The low-confidence residual that leaves the classifier for human review carries labor cost that scales with escalation rate and reviewer time, not with token price. Measure the escalation rate on the same holdout, convert it to reviewer minutes per 1,000 emails, and include that labor in the same unit before comparing routes.

Use one accounting boundary for every candidate: same holdout, same price retrieval date, and same escalation policy. A table of cost lines keeps the comparison auditable.

| Cost line | Included in per-1,000-email unit |
| --- | --- |
| API input and output tokens | Yes, at live per-million-token price |
| Encoder GPU hours | Yes, amortized |
| Holdout labeling and adjudication | Yes, as benchmark cost line |
| Escalation review labor | Yes, from measured escalation rate |

Route on the fully loaded number: a candidate wins only if its cost per 1,000 triaged emails is lower while the needs-action recall floor is still met. If escalation review pushes the routed pipeline above the alternative, move the split point or change the residual rule rather than quoting token price alone.

## Where the p95-vs-recall rule breaks

The p95-versus-recall rule is reliable only where the encoder sees the text that carries the decision. A 256-token window truncates quoted history in long threads, and that quoted chain often contains the request, deadline, or prior decision, so truncation removes signal rather than merely shortening input. Single-message triage can stay on the encoder while threaded mail goes to the GPT-4o path or a summarized context window.

Attachments and multilingual mail do not degrade both models in the same way. Stratify the holdout by stratum, report p95 end-to-end latency and needs-action recall separately for each stratum, and mark any stratum with fewer than 30 examples as unmeasured rather than as a pass. A stratum can pass on latency while failing recall, or pass on recall while adding tail latency, and aggregate numbers hide that tradeoff.

| Break condition | Required handling |
| --- | --- |
| Long threads | Route to GPT-4o or use a summarized context window |
| Attachments or multilingual mail | Report p95 and recall per stratum; unmeasured below 30 examples |
| Post-policy distribution shift | Recompute the split point on a fresh holdout |

Post-policy distribution shift can invalidate the split point even when neither model changes: new senders, routing rules, or label definitions move the confidence distribution. Treat policy changes as a new measurement event, not as a configuration tweak, and recompute the split point on a fresh holdout before promotion.

Keep the gate paired at the stratum level. An aggregate p95 win does not excuse a needs-action recall loss in a named stratum, and an unmeasured stratum should be routed conservatively to the GPT-4o path until it reaches 30 examples and has both gate numbers.

Do not promote a release on aggregate metrics alone. Require the two gate numbers for every measured stratum, record unmeasured strata as explicit routing exceptions, and keep the release decision tied to strata that were actually measured.

## 500 emails, 4 checkpoints

Fix the evaluation inputs before the first request runs: draw a 500-email holdout from real traffic, confirm that 120 of those messages are labeled *needs-action*, and execute every measurement on one host in one region. Send 20 repeats per email so each model is exercised under the same load profile, and pin the model version and run date in the log header. These controls keep the two release numbers comparable on the same holdout and the same host, which is the gate this run is designed to satisfy.

**Checkpoint 1 (p95 latency):** for each model separately, collect one end-to-end latency per email, sort the 500 values in ascending order, and take the 475th value as p95. The index is fixed by arithmetic: ceil(0.95 × 500) = ceil(475.0) = 475. Never pool the two models into one distribution; a pooled percentile hides which system misses the tail-latency target and makes the release decision uninterpretable.

**Checkpoint 2 (needs-action recall):** compute recall as TP / (TP + FN) over exactly those 120 needs-action emails, and report it at the precision floor you locked before logging began. The denominator is the full labeled needs-action set, not the subset a model chose to score confidently and not the 500-email holdout total. If a model returns 96 true positives and 24 false negatives, recall is 96 / (96 + 24) = 96 / 120 = 0.80; changing the denominator after the run invalidates the gate.

**Checkpoint 3 (false-negative cost):** account for every missed needs-action message as downstream recovery work. A false negative leaves required action undiscovered until a human finds it later, so its cost is the operational burden of late detection, delayed handling, and any follow-up needed to repair the missed obligation. Record the count of false negatives from the same 120-email denominator so the team can weigh recall shortfalls against the latency and spend advantages observed in Checkpoint 1.

**Checkpoint 4 (false-positive cost):** account for every non-action message flagged as needs-action as avoidable interruption. A false positive consumes reviewer attention, displaces genuinely urgent mail, and adds unnecessary handling steps. Because the precision floor is fixed in advance, any gain in recall that drives precision below that floor converts directly into this error cost. The release is defensible only when p95 latency and needs-action recall pass together on this run and the error counts from both directions remain acceptable for the routed operating point.

## Worked Example: Run the Numbers

Step 1, p95: using the end-to-end interval already defined for this system, let E be the encoder p95 and R be the residual model p95, with R > E. The residual share is 40 / 1,000 = 0.04, which is less than 0.05, so fewer than 50 of the 1,000 observations come from the slower route. Therefore the 95th percentile is not automatically the residual model's p95; it is determined by the encoder distribution plus at most the residual tail. Illustration: if E = 0.4 s and R = 2.0 s, the routed run's p95 stays near 0.4 s rather than becoming 2.0 s.

Step 2, recall: using the per-class recall denominator already defined for this system, overall recall is the message-weighted average of route-specific recalls. Illustration: if the encoder achieves 0.98 recall on the 960 routine messages and the residual model achieves 0.95 recall on the 40 routed messages, overall recall is (960 × 0.98 + 40 × 0.95) / 1,000 = (940.8 + 38.0) / 1,000 = 978.8 / 1,000 = 0.9788.

| Illustration item | Value |
| --- | --- |
| Routine messages | 960 |
| Residual messages | 40 |
| Overall needs-action recall | 0.9788 |
| p95 binding constraint | encoder, below 5% residual share |

For this illustration, the routed pipeline is the winner: it keeps the p95 near the encoder's value while producing 0.9788 needs-action recall, and it avoids sending all 1,000 messages through the slower residual model. The release gate still requires the residual model's own p95 and cost per decision to be acceptable, because those 40 messages are on the critical path.

The break-even trigger is a 5% residual share. Below 5%, the encoder's p95 remains the binding latency constraint; at or above 5%, the residual model's p95 becomes the binding constraint and the design must be rechecked against the gate. The recall break-even is the point at which the residual model's incremental recall on routed messages no longer exceeds the recall the encoder would have lost on those same messages.

## Ship rules: five if/then gates

| Gate | If | Then |
| --- | --- | --- |
| 1 | Classifier p95 is inside the SLO and needs-action recall clears the floor at locked precision. | Ship classifier-only; reserve GPT-4o for audits. |
| 2 | p95 misses the SLO while recall clears the floor. | Cut the body window, batch requests, or add a queue; re-measure p95. |
| 3 | Recall misses the floor while p95 passes. | Do not ship classifier-only; test routing the classifier's low-confidence residual. |
| 4 | GPT-4o beats classifier recall by at least 5 points on that residual and its p95 applies only to that residual. | Ship split routing; monitor residual share weekly. |
| 5 | Both models miss the latency SLO. | Fix the pipeline before buying a larger model. |

Gate 1 is the simplest release: the classifier is fast enough and accurate enough on needs-action mail, so GPT-4o stays out of the hot path and is used for audits. Gate 2 keeps the same model but changes the pipeline: if recall is acceptable and latency is not, cut the body window, batch requests, or add a queue, then re-run the same p95 measurement.

Gate 3 prevents a recall miss from being hidden by a passing latency number. If needs-action recall is below the floor, classifier-only shipping stops even when p95 is clean; the next test is whether routing the classifier's low-confidence residual changes recall without breaking the latency budget.

Gate 4 is the only topology change. It requires two conditions together: GPT-4o must beat classifier recall by at least 5 points on the low-confidence residual, and its p95 must apply only to that residual. If both hold, ship split routing and monitor residual share weekly; if either fails, stay on the current path. A rising residual share is a signal to re-run Gate 4, not to widen routing by default.

Gate 5 covers the case where neither model meets the latency SLO. Buy no larger model until the body window, batching, or queueing has been tried, because latency is an end-to-end property and a model swap does not remove queueing or payload cost.

Every gate is stateful: a change to the model, prompt, routing rule, host, or holdout resets the sequence, and the two release numbers must be re-measured before any ship decision.

## What to do next

| Step | Action | Why it matters |
| --- | --- | --- |
| 1 | On the benchmark table above, compare the DeBERTa-family encoder and GPT-4o using p95 end-to-end latency and cost per decision, not mean latency or pooled F1. | The release gate is defined by p95 latency, so averages or pooled F1 cannot override it. |
| 2 | Inspect needs-action recall separately for threaded and ambiguous mail rather than relying on aggregate recall. | The encoder's recall loss is concentrated there, so the split point must use per-class recall. |
| 3 | Before release, confirm p95 end-to-end latency and needs-action recall at the fixed precision floor were measured on the same holdout and same host. | The gate requires both numbers from the same holdout and same host. |
| 4 | Set the routing split point from p95 latency and per-class recall, sending only the classifier's low-confidence residual to GPT-4o. | This preserves GPT-4o recall where the encoder loses it without routing all mail through GPT-4o. |
| 5 | Re-measure latency with 20 runs per tool per day on one host before changing the route. | The split point depends on stable p95 latency, not a single measurement. |
| 6 | Keep GPT-4o in the recall check for threaded and ambiguous mail after any encoder update. | Prevents a silent recall regression in the residual route. |

## Frequently Asked Questions

**Where is the DeBERTa encoder's recall loss most concentrated?**

The DeBERTa-family encoder's recall loss is concentrated in threaded and ambiguous mail.

**Why must per-class recall be used instead of aggregate recall when determining the split point?**

Per-class recall must be used rather than aggregate recall because the encoder's recall loss is concentrated in threaded and ambiguous mail.

**Which specific recall class must be measured to gate a release?**

Release must be gated on measuring recall on the needs-action class at a fixed precision floor.

**Under what conditions does the routed-pipeline answer hold?**

The routed-pipeline answer holds only when p95 end-to-end latency and needs-action recall at a fixed precision floor are measured on the same holdout and same host.

**Which cases should be routed from the classifier to GPT-4o?**

Only the classifier's low-confidence cases should be routed to GPT-4o.

**When does the end-to-end latency clock start and stop for this triage system?**

The latency clock starts at SMTP accept or webhook receipt and stops when the mail API acknowledges the label write.

## Quick answers

| Which model wins on 95th-percentile end-to-end latency and cost per decision in the email triage benchmark? | The DeBERTa-family encoder beats GPT-4o on 95th-percentile end-to-end latency and cost per decision. |
| --- | --- |
| On what types of mail does GPT-4o maintain higher recall? | GPT-4o keeps higher recall on threaded and ambiguous mail. |
| What measurements are required to gate release? | Release should be gated on p95 end-to-end latency and recall on the needs-action class at a fixed precision floor on the same holdout and same host. |
| How is the split point for routing determined? | The split point is computed from p95 latency and per-class recall, with latency averaged over 20 runs per tool per day on one host. |
| Which cases should be routed in the pipeline? | Only the classifier's low-confidence cases should be routed. |

Also worth reading: **Hand your travel logistics to an AI executive assistant**: [Hand your travel logistics to](https://withtai.com/blog/hand_your_travel_logistics_to_an_ai_executive_assistant.php) · **Executive email style: Prompting vs Low-Rank Adaptation (LoRA) on 100-message holdout**: [Executive email style: Prompting vs](https://withtai.com/blog/executive-email-style-prompting-vs-low-rank-adaptation-lora-on-100-message-holdout.php) · **AI agent for performance review prep: grounded self-assessment vs recall — 2026 test**: [AI agent for performance review](https://withtai.com/blog/ai-agent-for-performance-review-prep-grounded-self-assessment-vs-recall-2026-test.php)

### Related reading

- [Inbox Triage Benchmark: Claude vs GPT-4o on 500 Emails](https://withtai.com/blog/inbox-triage-benchmark-claude-vs-gpt-4o-on-500-emails.php)
- [Fine-Tuned LLM Triage Cuts Email Response Time 23% in 2026](https://withtai.com/blog/fine-tuned-llm-triage-cuts-email-response-time-23-in-2026.php)
- [Founder calendar conflicts: How to benchmark multi-agent AI before committing](https://withtai.com/blog/founder-calendar-conflicts-how-to-benchmark-multi-agent-ai-before-committing.php)
- [meeting summary benchmark: 12 min saved per meeting in 2026 — ship or pilot?](https://withtai.com/blog/meeting-summary-benchmark-12-min-saved-per-meeting-in-2026-ship-or-pilot.php)
- [AI Benchmark 98% Completion Is a Patch, Not a Model Score](https://withtai.com/blog/ai-benchmark-98-completion-is-a-patch-not-a-model-score.php)
- [Async Executive Workflows: 34% Latency Edge Depends on Routing](https://withtai.com/blog/async-executive-workflows-34-latency-edge-depends-on-routing.php)

### Latest

- [Meeting Action Items by Length: GPT-4o vs Claude 3.5 F1 on 300 Calls](https://withtai.com/blog/meeting-action-items-by-length-gpt-4o-vs-claude-35-f1-on-300-calls.php)
- [Daily agent task-log review: fewer LangGraph failures over 30 days](https://withtai.com/blog/daily-agent-task-log-review-fewer-langgraph-failures-over-30-days.php)
- [AI Agent Human Oversight: 0.70 vs 95% Escalation Threshold Guide](https://withtai.com/blog/ai-agent-human-oversight-070-vs-95-escalation-threshold-guide.php)

Canonical: https://withtai.com/blog/2026-email-triage-benchmark-gpt-4o-vs-deberta-95th-percentile-latency-vs-recall.php
Markdown: https://withtai.com/blog/2026-email-triage-benchmark-gpt-4o-vs-deberta-95th-percentile-latency-vs-recall.php/index.md
