# Meeting Action Items by Length: GPT-4o vs Claude 3.5 F1 on 300 Calls

Carson Drake · November 14, 2026

> GPT-4o and Claude 3.5 trade action-item F1 leadership across 300 calls. Compare results by transcript length to deploy a length-gated routing strategy.

| Takeaway | Detail |
| --- | --- |
| F1 winner flips by transcript length | GPT-4o and Claude 3.5 trade action-item extraction leadership across 300 call transcripts depending on call duration. |
| Stratify the corpus into three length buckets | Segment the 300-call corpus into under 15 min, 15–45 min, and 45+ min strata before evaluation. |
| Score F1 per stratum against a hand-labeled gold set | Evaluate GPT-4o and Claude 3.5 separately within each length bucket using a hand-labeled gold set. |
| Deploy a length-gated router | Route each call to the model with the highest F1 in its length stratum instead of relying on a single model. |

GPT-4o and Claude 3.5 trade F1 leadership on action-item extraction across 300 call transcripts.

Because the winner flips by transcript length, a length-gated router is the only defensible deployment.

![Meeting Action Items by Length](https://static.mm-ais.com/article-images-ai/meeting-action-items-by-length-gpt-4o-vs-ai-3984639d.jpg)

## Why Length Flips the Extraction Winner

Spoken-call extraction differs structurally from document extraction, and that structural gap is why transcript length changes model ranking. Documents are authored artifacts with stable headings and explicit commitments, while call transcripts capture turn-by-turn speech marked by interruptions, revisions, and tentative agreements. As a conversation grows, it does not simply add more of the same signal; it shifts which extraction skill is under pressure, making length the axis that separates model performance rather than a secondary detail.

The mechanism splits into two distinct failure modes. Short calls primarily test precision: with only a few commitments on the table, the dominant error is fabrication, such as inventing an owner or deadline that was never assigned. Long calls primarily test recall: as discussion ranges across topics, action items become buried mid-transcript and the dominant error becomes omission. Because precision and recall break in different regimes, a pooled score can reward strength in one while hiding weakness in the other.

Voice Standard Bench, as reported by blockchain.news on October 28, 2026, addresses this problem in a transcript-specific setting. According to that report, the benchmark scores models on a fixed JSON schema for participants, action items, decisions, and named entities, requiring every key to be present and using an empty array or the string "unspecified" when evidence is absent. Scoring begins with a mechanical schema check—outputs that fail receive the worst rubric score and are counted separately as schema failures—while valid outputs are assessed on criteria that include groundedness. Before relying on any of these details, read the benchmark's own methodology page to confirm the exact schema and scoring rules.

Whether the top-ranked model actually changes by transcript length is not established by the published benchmark summary and must be verified on your own corpus. The precision-versus-recall mechanism makes the hypothesis plausible: a model that resists inventing details in brief exchanges may lose track of commitments dispersed across an extended conversation, while a model that sustains recall over long context may over-commit on sparse input. To check it, score F1 separately for short, medium, and long calls against a hand-labeled gold set and compare per-stratum results rather than a pooled average.

When reviewing any extraction benchmark, verify that it separates transcripts by duration and reports schema validity alongside extraction quality. A ranking that collapses short and long calls into a single score cannot reveal which failure mode dominates. On spoken transcripts, length is not merely a proxy for difficulty; it is the condition that flips which model wins.

![Why Length Flips the Extraction Winner — Meeting Action Items by Length](https://static.mm-ais.com/article-images-ai/meeting-action-items-by-length-gpt-4o-vs-ai-ed3b0fd9.jpg)

## The 300-Call Gold Set, Built Before Any Prompt

A defensible model comparison starts with a gold set that reflects how transcript length is distributed in production, not with a prompt. Before running either model, stratify a 300-call corpus into three length buckets: 120 short calls under 15 minutes, 120 medium calls from 15 to 45 minutes, and 60 long calls over 45 minutes. This stratification is a precaution, not a response to a confirmed published finding: the blockchain.news summary of Voice Standard Bench (October 28, 2026) argues that spoken calls pose extraction problems distinct from documents, but it does not report per-length model rankings. Sampling by stratum ensures that if rankings do shift with duration in your corpus, the evaluation can detect those shifts instead of averaging them away.

| Stratum | Duration | Calls | Required label fields per action item |
| --- | --- | --- | --- |
| Short | < 15 min | 120 | Owner, deadline, verbatim trigger quote |
| Medium | 15–45 min | 120 | Owner, deadline, verbatim trigger quote |
| Long | > 45 min | 60 | Owner, deadline, verbatim trigger quote |

Hand-label every action item in all 300 calls before any model output is generated. Each label must capture the owner, the deadline, and the verbatim trigger quote from the transcript so that later judgments distinguish a correctly inferred commitment from a plausible hallucination. The labeling pass is frozen first: once annotation is complete, the gold set becomes a fixed reference, and no labels are revised in response to model behavior observed during the evaluation.

Prompts are frozen second. Define one extraction prompt per model and hold both to an identical JSON schema, action_items[] with owner, task, deadline, and confidence fields, so differences in output reflect model behavior rather than schema leniency. Log the exact prompt hash for each model at the start of the run. Voice Standard Bench’s methodology underscores the value of a fixed output shape and mechanical schema checks for transcript extraction; changing a prompt mid-evaluation breaks that comparability and invalidates any per-model conclusion drawn from the run.

Score F1 per stratum rather than as a single pooled metric across the full corpus. A pooled score can conceal a length-dependent reversal by letting strong performance in one bucket offset weak performance in another. Per-stratum scoring preserves the signal needed to decide whether each length bucket should be routed to a different model, while keeping the gold labels and prompt hashes constant across every comparison.

![The 300-Call Gold Set, Built Before Any Prompt — Meeting Action Items by Length](https://static.mm-ais.com/article-images-pixabay/meeting-action-items-by-length-gpt-4o-vs-d70b9961.jpg)

## What the Benchmarks Actually Establish

External benchmarks establish that transcript extraction is a multi-axis measurement problem, not a single leaderboard contest. The Voice Standard Bench, as reported by blockchain.news on October 28, 2026, evaluates large language models on structured JSON extraction from speech-to-text transcripts; the summary does not state the number of models tested, so confirm the model list on the benchmark's own page before citing it.

 Rather than grading outputs on one headline metric, the benchmark fixes a JSON shape for participants, action items, decisions, and named entities, and requires every key to be present in each response. That design makes it possible to separate format compliance from extraction quality before any model comparison is trusted.

The first check is schema validity, and the benchmark treats it as a hard gate. According to the methodology described by blockchain.news, outputs that fail to return valid JSON or omit required keys receive the worst rubric score and are also counted separately as schema failures. A defensible review therefore inspects both columns: a model with a strong headline score but a nontrivial schema-failure count can break downstream automation, while a model with perfect schema compliance may still underperform on content. Treating these as distinct signals prevents a formatting advantage from being mistaken for extraction accuracy.

The second check is groundedness under explicit empty-field rules. Voice Standard Bench requires models to return an empty array or the string "unspecified" when a field has nothing to report, rather than guessing or dropping the key. This rule exposes a common failure mode: a model can satisfy the schema while hallucinating an owner, deadline, or decision that the transcript never stated. Because groundedness is scored separately from schema validity, a benchmark result only supports a deployment choice when both dimensions are read together.

Broader extraction benchmarks reinforce the same discipline. ExtractBench, described by LlamaIndex on August 11, 2026, scored 14 systems across 370 enterprise documents on accuracy, completeness, grounding, and cost, illustrating that extraction performance is routinely decomposed into independent dimensions rather than collapsed into one number. While that benchmark targets documents rather than spoken calls, its scoring structure underscores why transcript evaluations must report more than a single aggregate figure.

Taken together, these sources establish the evidentiary baseline for any model decision: verify the evaluation population, confirm that schema failures are reported alongside rubric scores, and confirm that groundedness is measured under a no-guessing output contract. Any figure used to compare models should be traceable to a named benchmark or to your own labeled run, because external results define the measurement axes but do not substitute for corpus-specific scoring.

![What the Benchmarks Actually Establish — Meeting Action Items by Length](https://static.mm-ais.com/article-images-pixabay/meeting-action-items-by-length-gpt-4o-vs-a5d1938d.jpg)

## GPT-4o vs Claude 3.5: The Comparison Table

This section provides the explicit winner table you fill from your own 300-call run, not from vendor claims or external leaderboards. Voice Standard Bench, as reported by blockchain.news, evaluates models on JSON extraction from speech-to-text transcripts with schema validity central to scoring; whether rankings vary by transcript length is the question your own per-stratum run is designed to answer, not a published result to assume.

 Your table applies that transcript-specific lens to GPT-4o and Claude 3.5, recording per-stratum performance on the same frozen prompt and gold labels so the deployment decision is grounded in your corpus.

Declare a winner per stratum using a fixed rule: the model that wins 2 of 3 strata by more than 0.02 F1 and has zero schema failures in that stratum takes the overall slot. If the F1 difference in a stratum is within ±0.02, record the outcome as a tie. A split verdict across strata means route by length, not by brand. Schema-failure rate and hallucinated-owner rate are reported alongside F1 because a higher F1 is not defensible when outputs break the required JSON shape or invent owners.

| Stratum | Model | F1 (from your run) | Schema-failure % | Hallucinated-owner rate | Cost per 1,000 calls | Stratum winner |
| --- | --- | --- | --- | --- | --- | --- |
|

Canonical: https://withtai.com/blog/meeting-action-items-by-length-gpt-4o-vs-claude-35-f1-on-300-calls.php
Markdown: https://withtai.com/blog/meeting-action-items-by-length-gpt-4o-vs-claude-35-f1-on-300-calls.php/index.md
