{"schemaVersion":1,"canonical":"https://codegraff.com/blog/grok-4-6-vs-4-7","markdown":"https://codegraff.com/blog/grok-4-6-vs-4-7/markdown","slug":"grok-4-6-vs-4-7","title":"Grok 4.7 uses more tokens per task than 4.6 — and scores higher","seoTitle":"Grok 4.7 uses more tokens per task than Grok 4.6","description":"In our own public evals, Grok 4.7 uses more tokens per task than Grok 4.6 — while finishing more of the checklist accurately.","publishedAt":"2026-09-22","keywords":["Grok 4.7","Grok 4.6","xAI pricing","Grok Build","codegraff","coding model eval"],"intro":"In our own, public evals, Grok 4.7 actually uses more tokens per task than Grok 4.6 — while finishing more of the work accurately. Same three codegraff bugs. Same public price. One turn each, tools off. 4.7 spent about 3.4× as many thinking tokens, hit 21/24 checklist items against 20/24, and cost 2.3× as much.","hero":{"src":"/blog/graff-frontier-harness-evals/eval-workshop.webp","alt":"Three mouse engineers test completed pieces from separate brass workbenches using one central checking jig.","caption":"Same bugs, same public price, one checking jig. The charts below are list price, not a Grok Build receipt quirk. Conceptual illustration."},"sourceNote":"Single-turn Grok Build calls, reasoning_effort=high, tools off. Both models saw the same closed-book prompt: the bug, a short pre-change excerpt, and a request for one unified diff. Gold diffs are in evals/grok-46-vs-47/gold/. Raw CLI JSON is in evals/grok-46-vs-47/runs/. Costs below use the published $2 / $6 card for both names so a 4.6 Build billing quirk is not treated as a model win. n=3. This is not an agent loop and not SWE-bench.","sections":[{"id":"simple","title":"The simple version","paragraphs":["A coding model has two kinds of writing. The patch you see, and the private scratch work it does first. That scratch work is billed like output. If a model thinks for a long time, you pay for that even when the patch is short. On this set, 4.7 did both: more tokens per task, and a slightly better score.","Grok 4.6 and Grok 4.7 share the same public price: $2 per million input tokens and $6 per million output tokens. Under 200k input, that card is the same for both names. So if 4.7 writes more scratch work, it is not because the sticker changed. It is because it used more of the expensive kind of tokens.","We asked both models to fix three small, already-merged codegraff PRs. They could not see the gold diff. They could not run tests. They got one shot. Then we scored the patch against eight things the merged PR actually did."],"sources":["results","pricing"]},{"id":"scorecard","title":"The scorecard: more tokens per task, one extra point","paragraphs":["Add the three calls together. 4.6 cost $0.285 at list price. 4.7 cost $0.669. Wall clock was 507 seconds against 1,351 seconds. Checklist hits were 20 out of 24 against 21 out of 24. That is the “more tokens, more accurate” line in numbers.","The patches you can read were about the same length: 2,761 completion tokens vs 3,145. The scratch work was not: 26,816 reasoning tokens vs 91,803. That is the whole story in one line. 4.7 is not a slower writer. Count reasoning in the output and it even decodes a bit faster (70 tokens/s vs 58). It just thinks longer before it speaks.","The one extra point is #1149: 4.7 set the TEAM.* group explicitly. The other two PRs were ties, including a shared compile miss."],"sources":["results"],"table":{"caption":"Totals for three PRs, published $2 / $6 card.","headers":["Model","Wall","Reasoning","Completion","List cost","Checklist"],"rows":[["Grok 4.6","507s","26,816","2,761","$0.285","20/24"],["Grok 4.7","1,351s","91,803","3,145","$0.669","21/24"]]}},{"id":"charts","title":"Cost and latency, one PR at a time","paragraphs":["The area chart is list-price dollars. The line chart is wall clock. Both use the published card, not the Grok Build receipt. The gap is widest on #1035, the Zig sock sweep, which is also where 4.7 spent 51k reasoning tokens."],"sources":["results"],"table":{"caption":"Per PR. List cost uses the published card. Hits are /8 against the merged behavior.","headers":["PR","Model","Wall","Reasoning","Completion","List cost","Hits"],"rows":[["1149","4.6","109s","5,461","501","$0.071","6/8"],["1149","4.7","334s","22,438","520","$0.170","7/8"],["1096","4.6","139s","7,036","1,028","$0.086","7/8"],["1096","4.7","270s","18,435","1,015","$0.149","7/8"],["1035","4.6","260s","14,319","1,232","$0.128","7/8"],["1035","4.7","748s","50,930","1,610","$0.350","7/8"]]}},{"id":"patches","title":"What the diffs did","paragraphs":["#1096 was a tie, and the interesting one. The bug: the @ menu must not treat Enter as “pick this row” until you have actually pointed at a row. Both models wrote the same helper the merged PR uses: engaged, not Shift, and Enter or Tab. Then both rewrote the native test script down to two files, which would drop the rest of the suite. 4.7 spent an extra 11k thinking tokens and still missed that.","#1149 was the one-point win. The bug: a same-team provisioning profile must not authorize someone else’s bundle. Both added a profile test and a signBundle test that fails before the signer. 4.7 also set the TEAM.* group out loud. Neither used the merged error regex, /provisioning profile/.","#1035 was a design tie and a compile miss for both. The bug: only delete a dead Accord sock file when the process that owned it is reclaimable. Both got that rule. Both wrote tests that call listen and livePost with the wrong number of arguments, so the tests would not compile. 4.7 is pickier about hex in the filename. That did not make the test compile. It cost $0.35 against $0.13."],"sources":["pr-1096","pr-1149","pr-1035","results"]},{"id":"method","title":"How we ran it","paragraphs":["Three merged PRs, picked because they are small and specific: #1149 (same-team profile tests, +20), #1096 (composer Enter must wait for a visible selection, +92), and #1035 (unlink a dead Accord sock only when the owner is reclaimable, +67).","Each model got one turn through Grok Build. grok -m grok-4.6 and grok -m grok-4.7 resolve to grok-4.6-build and grok-4.7-build. Reasoning effort was high. Tools were off. The prompt was the bug and a short excerpt from the file before the change, not the gold diff. Pairs ran at the same time. Wall time is each process’s own clock.","We scored against the merged PR’s behavior, not byte-identical patches. None of the diffs apply clean with patch -p1, because the prompt did not include the full files. That miss is shared, so it is not in the checklist."],"sources":["results","pr-1149","pr-1096","pr-1035"]},{"id":"price","title":"Same sticker, different receipts","paragraphs":["The public card for both names, under 200k prompt tokens, is $2 / 1M input, $0.50 / 1M cached input, and $6 / 1M output. Reasoning is billed as output. The charts and tables use that card, so the comparison is the same rate.","The Build receipt agreed with the card for 4.7 on every run, to the fraction of a cent ($0.170, $0.149, $0.350). It did not agree for 4.6. Those three receipts were exactly 0.34× list ($0.024, $0.029, $0.044). That is not 1×, not the 2× Fast rate, and not grok-build-0.1. If we priced 4.6 at the receipt, we would invent a discount the published card does not have. The receipt lives in the eval notes. The comparison does not use it."],"sources":["pricing","results"]},{"id":"limit","title":"What this does not say","paragraphs":["Three PRs. One turn. No tools. No “run the tests and try again.” A longer Build session might close the compile misses. It would also multiply a thinking bill that is already most of 4.7’s cost.","This is not a claim about Grok 4.7 as a chat model, or about multi-turn agents, or about SWE-bench. n=3, one turn. The finding is the token shape: at the same sticker price, high-effort 4.7 used more tokens per task and finished slightly more of the checklist."],"sources":["results"]}],"sources":[{"id":"results","title":"Eval receipts and run JSON","url":"https://github.com/justrach/codegraff/tree/main/evals/grok-46-vs-47"},{"id":"pricing","title":"xAI model pricing","url":"https://docs.x.ai/developers/models"},{"id":"pr-1149","title":"codegraff#1149, same-team profile tests","url":"https://github.com/justrach/codegraff/pull/1149"},{"id":"pr-1096","title":"codegraff#1096, composer Enter waits for a selection","url":"https://github.com/justrach/codegraff/pull/1096"},{"id":"pr-1035","title":"codegraff#1035, reclaimable Accord socks","url":"https://github.com/justrach/codegraff/pull/1035"}]}