Grok 4.7 Benchmarks: The One Vendor Claim the Independent Data Supports

Tech

Written by:

Reading Time: 6 minutes

The Grok 4.7 benchmarks worth quoting first are the two the vendor’s own page implies and rarely states side by side. On the fabrication axis, Grok 4.7 records the lowest hallucination rate of the four models in this comparison — 0.2934 against 0.5134, 0.5427 and 0.5861, lower being better, read 2026-10-07. On the same board it also records the lowest knowledge-breadth score of the four, 32.03. Both figures are independent readings, the endpoint is metered here at list rate with no markup, and the pairing is the whole article.

Those two numbers look like they contradict each other. They do not, and understanding why is worth more than any single score on the page.

The claim, and the rare thing about it

xAI’s material for Grok 4.7 attaches one specific quality claim: minimal hallucinations. Most vendor claims in this category are unfalsifiable from the outside — “state of the art”, “best-in-class reasoning”, “frontier intelligence” are positioning, not measurements, and nothing a third party publishes can confirm or refute them.

This claim is different in kind. Hallucination is something an independent evaluator can put a number on, and one does. The board’s recorded fabrication rate for Grok 4.7 is 0.2934, and it is the lowest of the four subjects here:

modelfabrication rate (lower is better)recorded
Grok 4.70.29342026-10-07
Claude Opus 5.50.58612026-10-07
GPT-6.1 Sol0.54272026-10-07
GPT-6 Astra0.51342026-10-07

That is worth pausing on for a reason that has nothing to do with Grok. It is the rare case of a vendor claim that independent measurement bears out. A claim you can check and that checks out should raise your confidence in the rest of the page it came from, and it should also set the standard: when the same vendor makes a claim you *cannot* check, the right posture is to note that the last one was checkable and this one is not.

The number that sits underneath it

Now put the second reading next to the first. On the same independent board, Grok 4.7’s knowledge-breadth score — the metric that asks how much the model actually knows across a wide set of facts — is 32.03, the lowest of the four subjects. The other three sit at 41.53, 43.40 and 46.42, read 2026-10-07.

So the model that fabricates least also knows least, on these two axes, at the same time. That is not a paradox and it is not a contradiction, and the explanation is the most useful thing on the page.

A hallucination rate is not a truthfulness score. It is a rate of confident wrong answers. A model can lower that number in two completely different ways: by knowing more, or by answering less. Grok 4.7’s profile is consistent with the second route — a lower fabrication rate alongside a lower knowledge breadth is what a model that declines, defers or produces a shorter answer instead of an invented one looks like on a scoreboard.

Whether that is good depends entirely on your workload, and the two cases are easy to tell apart:

• A retrieval-grounded pipeline — where the model is asked to answer from documents you supplied — wants exactly this profile. Refusing beats inventing, the ground truth is in the context, and a model that says less when it does not know is doing the job.

• An open-ended knowledge task — where the model is asked what it knows with no supplied source — wants breadth, and a low answer rate looks like an incomplete answer rather than a careful one.

The same pair of numbers is a strength in the first case and a limitation in the second. Choosing between them is a decision about your product, not about the model.

The record is thin, and that changes how to read it

Everything above rests on a small base, and that belongs in the same article rather than a footnote.

At the read date of 2026-10-07, Grok 4.7’s record on the third-party coverage board held 8 of 26 result rows across 3 of 8 categories, with no display score published and the model marked ineligible for ranking. Claude Opus 5.5, released one day later, held 53 of 73 rows across all 8 categories. Grok 4.7’s evidence base is roughly one-sixth the size of the model it launched beside.

Two consequences follow, and both are practical.

A thin record is thin unevenly. Three of eight categories is not a partial version of a full evaluation; it is an evaluation that stops before most categories start. The three it does cover may be none of yours, and the board does not publish a confidence interval around a category it never ran.

The aggregate is where the thinness shows most. On the reasoning index, Grok 4.7 records 46.45, the lowest of the ten models in the same pull, read 2026-10-07 — and it is measured at Xhigh on the harness while the OpenAI and Anthropic models beside it are measured at Max. A configuration one rung down on a model with a sixth of the evidence is not the same kind of number as a fully covered model’s index, and stacking them without both labels is comparing configurations.

None of that makes the fabrication finding less real. It makes it the kind of finding to hold with its scope attached: lowest fabrication rate of four, on a board that has measured this model in three categories out of eight.

Where the numbers actually diverge

Software and terminal work is where the independent sources disagree most about this model, and the disagreement is instructive:

source (all read 2026-10-07)reading
one board, terminal-task family38.00%
a second board, same benchmark generation25.76%
a third board, an earlier generation of that benchmark73.4%
the same page, a long-horizon software suite29.5%
the same page, a short-horizon software suite71.0%

Read that table as one lesson rather than five facts. The same model, on what sounds like the same task family, is published as anywhere between 25.76% and 73.4% — and every one of those figures is a real measurement taken by a real evaluator on a named harness. What separates them is not the model. It is the task generation, the scaffolding, the effort rung and what counts as a pass.

This is the practical case for the habit the rest of this cluster keeps repeating: the number is not the fact. “Grok 4.7 scores 73.4% on terminal work” is not usable. “Grok 4.7, on the second-generation terminal benchmark, at Xhigh, on that board’s harness, read 2026-10-07” is.

What to do with a Grok 4.7 benchmark table

Four checks, in the order they matter for this model.

Separate fabrication from knowledge before you read either. The two move together here and pull in opposite directions depending on your workload. Ask which one your task rewards before deciding the pair is good or bad news.

Attach the category count, not just the score. A three-of-eight record and an eight-of-eight record are different classes of evidence. Quoting the metric without the coverage is how a thin finding gets treated as a broad one.

Name the effort rung. Xhigh here against Max for the models beside it. A cross-model table without settings is a table of configurations.

Re-run the one number that decides your case. Every figure above is somebody else’s harness. The only hallucination rate that describes your product is the one you plot on your own questions with your own retrieval, and it costs an afternoon.

The takeaway

Grok 4.7 records 0.2934 on the independent board’s fabrication metric — the lowest of the four models in this comparison, read 2026-10-07 — and that makes xAI’s “minimal hallucinations” claim the rare vendor claim that independent measurement actually supports. On the same board the model records the *lowest* knowledge breadth of the four, 32.03, and the two readings are consistent rather than contradictory: a lower rate of confident wrong answers can come from knowing more or from answering less, and this profile looks like the second.

Hold the whole picture with its scope. The record behind those readings is 8 of 26 rows across 3 of 8 categories with no published display score, measured at the Xhigh effort setting against Max for the models beside it, and the software-task numbers for this model spread from 25.76% to 73.4% across named harnesses. The fabrication finding survives all of that; the aggregate does not sit comfortably beside a fully covered model’s.

Read every figure as three things stacked in one number — the metric, the harness generation and the effort rung — and quote the coverage alongside the score or you are quoting a different claim than the one the board made.

OrcaRouter carries Grok 4.7 at list rate on the same key as the rest of the xAI line, so measuring this profile against your own retrieval pipeline is a model-string change rather than a second integration.

Sourcing note: the fabrication rate of 0.2934, the knowledge-breadth score of 32.03, the Intelligence Index of 46.45, the comparison values for GPT-6 Astra, GPT-6.1 Sol and Claude Opus 5.5, and the Xhigh reasoning setting recorded for Grok 4.7 against Max for the others are from Artificial Analysis’ live model pages, read 2026-10-07. The 8-of-26 row count, 3-of-8 category coverage, the absence of a display score, the ranking-ineligible flag and the software-and-terminal readings quoted in the divergence table are from benchlm.ai’s and other named boards’ live pages, read the same day, with each harness generation distinguished. Grok 4.7’s flagship position, the “minimal hallucinations” claim and the $2 / $6 rates are xAI’s own published claims; the fabrication figure is an independent reading and is not the vendor’s. The explanation that a low fabrication rate alongside a low knowledge-breadth score reflects declining rather than inventing, the two-workload framing, and the reading that the spread across harnesses is a property of the harnesses are this article’s own analysis, not published claims. All checked 2026-10-07; re-check before 2026-11-01.