← blog · June 28, 2026 · 10 min read
Can a 48 GB MacBook replace Claude for my workshop manuals?
The sucatisse manual pipeline runs on cloud Claude. I spent a day trying to do the whole extraction on-device on an M5 Pro instead, first with an ensemble of small models, then one big one, and measured both against the Claude output that already ships.
$ bloat--level 3Bloatin'
A while back I wrote about sucatisse, my project for turning decades-old car workshop manuals into clickable, searchable, mobile-friendly things. The extraction half of that pipeline runs on cloud Claude (Opus), one batch request per page, and the structured JSON it spits out is what the site is built from.
It works well. But every page costs tokens, and every page needs a network round-trip. So I spent a day on a different question. My MacBook is an M5 Pro with 48 GB of unified memory. Can it just do the whole thing itself, with no network and no per-token cost, and how close does it get to the Claude output that already ships in the S13 data?
To find out I built macOS-local: a self-contained pipeline that rasterizes each page, runs layout detection, hands the page to a vision model for extraction, and crops the figures. Then I iterated on the extraction stage twice, because the first idea didn’t survive contact with reality.
The setup
Everything runs on MLX, Apple’s native inference stack, via mlx-vlm. Layout detection is Surya, which I had to keep in its own isolated venv because it needs transformers<5 and mlx-vlm needs ≥5.5, and they refuse to live in the same environment. Rasterizing is PyMuPDF at 300 dpi grayscale. The reference I’m grading against is the shipped Claude extraction: 881 pages, mostly claude-opus-4-7.
One thing about local inference on this hardware: reading a page is memory-bandwidth-bound. Every token the model generates reads the entire model out of unified memory, so the model size and the quantization level set your speed, and a single GPU means you can’t run two models side by side for free. That constraint shaped both attempts.
Attempt one: the ensemble
My first idea was an inverted version of how you’d normally tier models in the cloud. Run cheap models on every page, and only pay for the expensive one where the cheap ones disagree. Two 8B vision models (Qwen3-VL-8B and InternVL3-8B, both 4-bit) read each page and vote. A judge reconciles them. If they don’t agree well enough, the page escalates to a 30B mixture-of-experts model.
That was the plan. Here’s what running it actually looked like:
Look at the agreement column. The plan was that most pages would clear the cheap path and stop. Instead the two 8B models structured pages so differently that mean agreement landed at 0.60, well under my threshold, so 19 of 20 pages escalated anyway. The cheap path almost never terminated. I was paying for the entire stack on nearly every page, which is exactly the thing the design was supposed to avoid. On top of that, 4-bit quantization seemed to be hurting digit reading.
The scores against Claude told the same story. Here’s the ensemble on 20 pages:
| metric | ensemble vs Claude |
|---|---|
| numeric recall | 0.61 |
| table-cell recall | 0.17 |
| block-structure recall | 0.52 |
| text recall / precision | 0.73 / 0.96 |
| page-type match | 60% |
Text coverage was fine and it wasn’t hallucinating (0.96 precision), but it missed about 40% of the numbers and barely reproduced tables at all. On the table-of-contents page it flattened the entire section-code index into 22 plain headings and lost every code. For a workshop manual, where the numbers are the content, that’s a fail.
Attempt two: one big model, twice
So I threw the ensemble out and did the opposite. Instead of several small models voting, use the single biggest, highest-precision model the 48 GB box can hold, and let it check its own work.
That meant Qwen3-VL-32B-Instruct at 8-bit, which is about 36 GB and fits with not much room to spare. I went 8-bit rather than 4-bit specifically to fix the digit problem from attempt one. I fed it the page at 2048 px instead of 1536 so small numbers stay legible, ran two passes (the second pass re-reads the page plus the first draft and is told to fill in anything missed), and hardened the schema so malformed blocks stopped blowing up the run.
On easy pages, it basically matches Claude
On the two front-matter pages I’d compared head-to-head with the ensemble, the solo model jumped on every axis:
| metric (front-matter pages) | ensemble | solo |
|---|---|---|
| numeric recall | 0.61 | 1.00 |
| table-cell recall | 0.00 | 1.00 |
| block-structure recall | 0.66 | 0.90 |
| text recall | 0.75 | 0.99 |
| page-type match | 1 / 2 | 2 / 2 |
It captured the table-of-contents section codes the ensemble had destroyed, and it recovered the cover title. On easy pages, this is Claude-parity.
On hard pages, the truth comes out
Front matter is the easy stuff. The real manual is spec tables, exploded parts diagrams, wire-level schematics, and diagnostic flowcharts. So I added a page selector and pointed it at 10 dense pages deep in the book, including a spec page with 316 numbers on it and an HVAC diagnostic table with 337.
Two of the ten turned out to be unusable: the local 881-page PDF and the Claude corpus drift apart around the foldout wiring pages, so those page numbers point at physically different pages with almost no text in common. Excluding those, here are the eight aligned dense pages, with the ensemble’s front-matter scores alongside for contrast:
| metric | solo (8 dense pages) | ensemble (front-matter) |
|---|---|---|
| numeric recall | 0.67 | 0.61 |
| table-cell recall | 0.56 | 0.17 |
| block-structure recall | 0.63 | 0.52 |
| text recall | 0.72 | 0.73 |
How well it does depends enormously on the page:
- Spec tables are great. One parts-data page scored 0.94 text, 0.95 numbers, and 0.96 table cells. The 316-number spec page hit 0.90 numeric recall.
- Diagrams and dense diagnostics are not. An exploded parts diagram collapsed into a single block. A wiring page managed 0.30 on numbers. That 337-number HVAC table came back at 0.17.
So the solo model fixed the ensemble’s structural collapse (tables went from 0.17 to 0.56) and nudged ahead on numbers. But the front-matter near-perfection does not carry over to dense content, where it still drops about a third of the numbers. Not Claude-parity on the hard pages, which are the pages that matter most.
The second pass earned nothing
Here’s the part I didn’t expect. Across all 12 solo pages, the verify pass added zero blocks. Whatever small value-level corrections it made (and I didn’t measure those), it recovered no missing content, while doubling the time. As built, the two-pass design isn’t paying for itself.
The wall I actually hit
There’s no dollar cost here, but there’s wall-clock, and it’s brutal:
| on the dense pages | value |
|---|---|
| avg per page (2 passes) | ~10.8 min |
| slowest single page (a wiring diagram) | ~24 min |
| 10 dense pages, total | ~107 min |
| front-matter pages (earlier) | ~2-3 min/page |
| peak RAM (no headroom to spare) | ~36 GB |
| extrapolated to the full 881-page manual | ~6 days |
The model sat at ~36 GB of RAM the whole time, with no out-of-memory crashes but no headroom either. Extrapolate the dense-page rate to the whole 881-page manual and you get roughly six days of continuous compute, three if you drop to a single pass. That settles it for bulk work. Doing a whole manual locally on one machine isn’t practical. Targeted pages, a small manual, an overnight batch: fine. The whole book on one box: no.
What I took away from it
A few things became clear by the end:
- One big 8-bit model beats a 4-bit ensemble for this. Better numbers, far better tables, a much simpler pipeline. The clever agreement-based escalation just collapsed in practice.
- Quantization precision matters for digits. Going from 4-bit to 8-bit was the single lever that moved numeric recall, which makes sense when the whole point is reading torque values off a photocopy.
- Local matches Claude on easy pages and trails on hard ones. Dense wiring, parts diagrams, and big diagnostic tables are where the gap lives.
- Throughput is the constraint, not quality or money. Eleven minutes a page doesn’t scale to a full book on one box.
- The verify pass, as designed, isn’t worth twice the time. It added nothing.
Where I’d take it next
The obvious move is to make pass two cheaper or drop it by default, and only re-verify pages that look uncertain. That’s the original escalation idea, except gated on the model’s own uncertainty rather than two models disagreeing. I want to chase the dense-page number gap with full-resolution input on spec pages, or a proper document-OCR specialist once one is turnkey on MLX. I’d like to try the 30B mixture-of-experts model at 8-bit as the primary, since it only activates ~3B parameters per token and should decode much faster. And I need to fix the page alignment between the local PDF and the Claude corpus before any larger comparison, because losing 2 of 10 deep pages to drift made them non-comparable.
For now Claude stays in the pipeline. But it’s genuinely close on the easy 80%, and the on-device version costs nothing and never touches the network, so I’m not done poking at it.
A caveat on the numbers: these are automated structural and text-overlap scores against the shipped Claude data, which isn’t ground truth itself, so they undercount output that’s semantically right but structured differently. The samples are small. The scanned page, not the Claude reference, is the real arbiter.
so the sucatisse pipeline runs on cloud claude (opus), one batch request per page, json out, site built from that. works but every page = tokens + network. macbook is an m5 pro, 48gb unified. can it just do the whole thing itself? built macOS-local to find out: rasterize → layout → vlm extract → crop figures. rewrote the extract stage twice bc idea #1 died on contact
all MLX via mlx-vlm. Surya in its own venv (transformers<5 vs ≥5.5, they hate each other). pymupdf 300dpi grayscale. grading against the shipped claude extraction: 881 pages, mostly claude-opus-4-7. note: local inference here is memory bandwidth bound, every token reads the whole model out of ram, and one gpu = no free second model
attempt 1: two 8b vlms vote per page (Qwen3-VL-8B + InternVL3-8B, both 4-bit), judge scores agreement, below 0.85 → escalate to a 30b moe. inverted cloud tiering basically
lol. mean agreement 0.60, 19 of 20 pages escalated. cheap path never fired, paid for the whole stack anyway. exactly what the design was supposed to avoid. also 4-bit seemed to hurt digit reading. scores on 20 pages:
| metric | ensemble vs Claude |
|---|---|
| numeric recall | 0.61 |
| table-cell recall | 0.17 |
| block-structure recall | 0.52 |
| text recall / precision | 0.73 / 0.96 |
| page-type match | 60% |
text ok, no hallucinating (0.96 prec), but ~40% of numbers gone, tables basically dead at 0.17. flattened the toc into 22 plain headings, lost every section code. numbers ARE the content in a manual so, fail
attempt 2: opposite. one big model twice. Qwen3-VL-32B-Instruct 8-bit, ~36gb, barely fits. 8-bit for the digits, 2048px input instead of 1536, two passes (2nd rereads page + draft, fills gaps), hardened schema. same front-matter pages:
| metric (front-matter pages) | ensemble | solo |
|---|---|---|
| numeric recall | 0.61 | 1.00 |
| table-cell recall | 0.00 | 1.00 |
| block-structure recall | 0.66 | 0.90 |
| text recall | 0.75 | 0.99 |
| page-type match | 1 / 2 | 2 / 2 |
toc codes back, cover title back. claude parity on easy pages
then 10 dense pages deep in the book (spec page w 316 numbers, hvac diag table w 337). 2 of 10 unusable, local pdf + claude corpus drift apart at the foldout wiring pages so page numbers point at different pages. the 8 aligned:
| metric | solo (8 dense pages) | ensemble (front-matter) |
|---|---|---|
| numeric recall | 0.67 | 0.61 |
| table-cell recall | 0.56 | 0.17 |
| block-structure recall | 0.63 | 0.52 |
| text recall | 0.72 | 0.73 |
spec tables great (one parts page 0.94 text / 0.95 numbers / 0.96 cells, the 316-number page 0.90 numeric). diagrams bad (exploded parts diagram → one block, wiring page 0.30 numbers, the 337 hvac table 0.17). so tables fixed 0.17→0.56 but still drops ~1/3 of numbers on the hard pages, which are the ones that matter. and the verify pass added zero blocks across all 12 pages while doubling the time. nope
the real wall is time:
| on the dense pages | value |
|---|---|
| avg per page (2 passes) | ~10.8 min |
| slowest single page (a wiring diagram) | ~24 min |
| 10 dense pages, total | ~107 min |
| front-matter pages (earlier) | ~2-3 min/page |
| peak RAM (no headroom to spare) | ~36 GB |
| extrapolated to the full 881-page manual | ~6 days |
~36gb ram whole run, no oom, no headroom. 881 pages ≈ 6 days continuous, 3 single-pass. bulk on one box: no. targeted pages / small manual / overnight batch: fine
takeaways: one big 8-bit model beats a 4-bit ensemble. quant precision is THE lever for digits. local = claude on easy pages, trails on hard. throughput is the constraint, not quality or money. verify pass worthless as built
next: drop pass 2 unless the model looks uncertain, full-res spec pages, doc-ocr specialist once turnkey on mlx, try the 30b moe at 8-bit as primary (~3b active params, should decode way faster), fix the pdf page alignment. claude stays for now but its close on the easy 80% and local is free + offline so not done poking 🔧
(scores are automated overlap vs the shipped claude data, not ground truth, small samples, the scan is the real arbiter)
The extraction half of sucatisse, my project that turns old car workshop manuals into searchable sites, runs on cloud Claude (Opus): one batch request per page, structured JSON out. It works well, but every page costs tokens and needs a network round-trip. My MacBook is an M5 Pro with 48 GB of unified memory, so I spent a day on a simple question: can it do the whole thing itself, and how close does it get to the Claude output that already ships in the S13 data?
I built macOS-local, a self-contained pipeline: rasterize each page, detect layout, extract with a vision model, crop the figures. The extraction stage went through two designs, because the first one did not survive contact with reality.
The setup
Everything runs on MLX via mlx-vlm. Layout detection is Surya, kept in its own venv because it needs transformers<5 while mlx-vlm needs ≥5.5. Rasterizing is PyMuPDF at 300 dpi grayscale. The reference is the shipped Claude extraction: 881 pages, mostly claude-opus-4-7.
One constraint shaped everything: local inference here is memory-bandwidth-bound. Every generated token reads the entire model out of unified memory, so model size and quantization set your speed, and a single GPU means two models never run side by side for free.
Attempt one: the ensemble
The first idea inverted the usual cloud tiering: run cheap models on every page, pay for the expensive one only where they disagree. Two 8B vision models (Qwen3-VL-8B and InternVL3-8B, both 4-bit) read each page and vote, a judge reconciles them, and pages below a 0.85 agreement threshold escalate to a 30B mixture-of-experts model.
Here is what running it actually looked like:
The two 8B models structured pages so differently that mean agreement landed at 0.60, and 19 of 20 pages escalated. The cheap path almost never terminated, so I paid for the entire stack on nearly every page, which is exactly what the design was meant to avoid. On top of that, 4-bit quantization seemed to hurt digit reading.
The scores against Claude, on 20 pages:
| metric | ensemble vs Claude |
|---|---|
| numeric recall | 0.61 |
| table-cell recall | 0.17 |
| block-structure recall | 0.52 |
| text recall / precision | 0.73 / 0.96 |
| page-type match | 60% |
Text coverage was fine and precision was 0.96, so no hallucinating, but it missed about 40% of the numbers and barely reproduced tables. It flattened the table-of-contents index into 22 plain headings and lost every section code. In a workshop manual the numbers are the content, so this is a fail.
Attempt two: one big model, twice
I threw the ensemble out and did the opposite: the single biggest, highest-precision model the 48 GB box can hold, checking its own work. That is Qwen3-VL-32B-Instruct at 8-bit, about 36 GB. I chose 8-bit over 4-bit specifically to fix the digit problem, fed pages at 2048 px instead of 1536 so small numbers stay legible, ran two passes (the second re-reads the page plus the first draft and fills in anything missed), and hardened the schema so malformed blocks stopped blowing up runs.
Easy pages: parity
On the two front-matter pages from the ensemble comparison, the solo model jumped on every axis:
| metric (front-matter pages) | ensemble | solo |
|---|---|---|
| numeric recall | 0.61 | 1.00 |
| table-cell recall | 0.00 | 1.00 |
| block-structure recall | 0.66 | 0.90 |
| text recall | 0.75 | 0.99 |
| page-type match | 1 / 2 | 2 / 2 |
It captured the table-of-contents section codes the ensemble had destroyed and recovered the cover title. On easy pages, this is Claude-parity.
Hard pages: the truth
The real manual is spec tables, exploded parts diagrams, wiring schematics, and diagnostic flowcharts, so I pointed the pipeline at 10 dense pages deep in the book, including a spec page with 316 numbers and an HVAC diagnostic table with 337. Two of the ten proved unusable: the local 881-page PDF and the Claude corpus drift apart around the foldout wiring pages, so those page numbers point at physically different pages. The eight aligned pages, with the ensemble’s front-matter scores for contrast:
| metric | solo (8 dense pages) | ensemble (front-matter) |
|---|---|---|
| numeric recall | 0.67 | 0.61 |
| table-cell recall | 0.56 | 0.17 |
| block-structure recall | 0.63 | 0.52 |
| text recall | 0.72 | 0.73 |
Performance depends heavily on the page:
- Spec tables are great. One parts-data page scored 0.94 text, 0.95 numbers, 0.96 table cells; the 316-number spec page hit 0.90 numeric recall.
- Diagrams and dense diagnostics are not. An exploded parts diagram collapsed into a single block, a wiring page managed 0.30 on numbers, and the 337-number HVAC table came back at 0.17.
So the solo model fixed the structural collapse (tables 0.17 to 0.56) and nudged ahead on numbers, but it still drops about a third of the numbers on dense content. Not Claude-parity on the pages that matter most.
The second pass earned nothing
Across all 12 solo pages, the verify pass added zero blocks. It recovered no missing content while doubling the time. As built, it does not pay for itself.
The wall
There is no dollar cost, but the wall-clock is brutal:
| on the dense pages | value |
|---|---|
| avg per page (2 passes) | ~10.8 min |
| slowest single page (a wiring diagram) | ~24 min |
| 10 dense pages, total | ~107 min |
| front-matter pages (earlier) | ~2-3 min/page |
| peak RAM (no headroom to spare) | ~36 GB |
| extrapolated to the full 881-page manual | ~6 days |
The model sat at ~36 GB the whole time, no crashes but no headroom. Extrapolated to the full 881 pages, that is roughly six days of continuous compute, three with a single pass. Targeted pages, a small manual, or an overnight batch: fine. The whole book on one box: no.
Takeaways
- One big 8-bit model beats a 4-bit ensemble for this. Better numbers, far better tables, a simpler pipeline.
- Quantization precision matters for digits. Going 4-bit to 8-bit was the single lever that moved numeric recall.
- Local matches Claude on easy pages and trails on hard ones. Dense wiring, parts diagrams, and big diagnostic tables are the gap.
- Throughput is the constraint, not quality or money. Eleven minutes a page does not scale to a book.
- The verify pass, as designed, is not worth twice the time.
Next
Drop the second pass by default and re-verify only pages that look uncertain: the escalation idea again, gated on the model’s own uncertainty rather than two models disagreeing. Chase the dense-page number gap with full-resolution spec pages, or a document-OCR specialist once one is turnkey on MLX. Try the 30B mixture-of-experts model at 8-bit as the primary, since it activates only ~3B parameters per token and should decode much faster. And fix the page alignment between the local PDF and the Claude corpus before any larger comparison.
For now Claude stays in the pipeline. But local is genuinely close on the easy 80%, costs nothing, and never touches the network, so I am not done poking at it.
A caveat: these are automated structural and text-overlap scores against the shipped Claude data, which is not ground truth itself, so they undercount output that is semantically right but structured differently. The samples are small. The scanned page is the real arbiter.
In the ever-evolving landscape of local AI inference, one question keeps surfacing for anyone building document pipelines: do you really need the cloud? A while back I wrote about sucatisse, my project for transforming decades-old car workshop manuals into clickable, searchable, mobile-friendly experiences. The extraction half of that pipeline currently leverages cloud Claude (Opus), sending one batch request per page and receiving structured JSON that the entire site is built from.
It’s important to note that this approach works remarkably well. However, every page costs tokens, and every page requires a network round-trip. That’s why I decided to dedicate a full day to delving into a different question entirely. My MacBook is an M5 Pro with 48 GB of unified memory, a genuinely capable machine. Could it seamlessly handle the whole extraction process itself, with no network dependency and no per-token cost? And crucially, how close could it get to the Claude output that already ships in the S13 data?
To find out, I built macOS-local: a self-contained, robust pipeline that rasterizes each page, runs layout detection, hands the page to a vision model for extraction, and crops the figures. This isn’t just a proof of concept, it’s a complete end-to-end system. I then iterated on the extraction stage twice, because, as we’ll see, the first idea didn’t survive contact with reality.
The setup
Let’s delve into the technical foundation. Everything runs on MLX, Apple’s native inference stack, via mlx-vlm. For layout detection, I leveraged Surya, which I had to keep in its own isolated venv because it needs transformers<5 while mlx-vlm needs ≥5.5, and the two simply refuse to coexist in the same environment. Rasterizing is handled by PyMuPDF at 300 dpi grayscale. The reference I’m grading against is the shipped Claude extraction: 881 pages, mostly claude-opus-4-7.
It’s worth taking a moment to understand a fundamental characteristic of local inference on this hardware: reading a page is memory-bandwidth-bound. Every single token the model generates reads the entire model out of unified memory, which means the model size and the quantization level directly set your speed. Furthermore, a single GPU means you can’t run two models side by side for free. This constraint isn’t just a technical footnote, it’s the factor that shaped both attempts.
Attempt one: the ensemble
My first idea was an inverted version of how you’d typically tier models in the cloud landscape. The strategy: run cheap models on every page, and only pay for the expensive one where the cheap ones disagree. Two 8B vision models (Qwen3-VL-8B and InternVL3-8B, both 4-bit) read each page and vote. A judge then reconciles their outputs and scores the agreement. If they don’t agree well enough, the page escalates to a 30B mixture-of-experts model. Elegant, efficient, and economical, at least on paper.
That was the plan. Here’s what running it actually looked like in practice:
Take a close look at the agreement column, because it tells a fascinating story. The plan was that most pages would clear the cheap path and stop there. Instead, the two 8B models structured pages so differently that mean agreement landed at a disappointing 0.60, well under my 0.85 threshold, so 19 of 20 pages escalated anyway. The cheap path almost never terminated. In other words, I was paying for the entire stack on nearly every page, which is exactly the outcome the design was supposed to avoid. To make matters worse, 4-bit quantization seemed to be actively hurting digit reading.
The scores against Claude told the same sobering story. Here’s the ensemble evaluated on 20 pages:
| metric | ensemble vs Claude |
|---|---|
| numeric recall | 0.61 |
| table-cell recall | 0.17 |
| block-structure recall | 0.52 |
| text recall / precision | 0.73 / 0.96 |
| page-type match | 60% |
Text coverage was fine, and it’s worth highlighting that the model wasn’t hallucinating (an impressive 0.96 precision). However, it missed about 40% of the numbers and barely reproduced tables at all. On the table-of-contents page, it flattened the entire section-code index into 22 plain headings and lost every single code. For a workshop manual, where the numbers are the content, that’s not a partial success, it’s a fail.
Attempt two: one big model, twice
So I threw the ensemble out entirely and embraced the opposite philosophy. Instead of several small models voting, why not use the single biggest, highest-precision model the 48 GB box can hold, and let it check its own work?
That meant Qwen3-VL-32B-Instruct at 8-bit, which weighs in at about 36 GB and fits with not much room to spare. I deliberately went 8-bit rather than 4-bit specifically to address the digit problem from attempt one. Additionally, I fed it the page at 2048 px instead of 1536 so small numbers stay legible, ran two passes (the second pass re-reads the page plus the first draft and is instructed to fill in anything missed), and hardened the schema so malformed blocks stopped blowing up the run. Three targeted changes, each addressing a specific weakness.
On easy pages, it basically matches Claude
On the two front-matter pages I’d compared head-to-head with the ensemble, the solo model jumped dramatically on every single axis:
| metric (front-matter pages) | ensemble | solo |
|---|---|---|
| numeric recall | 0.61 | 1.00 |
| table-cell recall | 0.00 | 1.00 |
| block-structure recall | 0.66 | 0.90 |
| text recall | 0.75 | 0.99 |
| page-type match | 1 / 2 | 2 / 2 |
It seamlessly captured the table-of-contents section codes the ensemble had destroyed, and it recovered the cover title. On easy pages, this is nothing short of Claude-parity.
On hard pages, the truth comes out
However, it’s important to note that front matter is the easy stuff. The real manual is a different beast entirely: spec tables, exploded parts diagrams, wire-level schematics, and diagnostic flowcharts. So I added a page selector and pointed it at 10 dense pages deep in the book, including a spec page with 316 numbers on it and an HVAC diagnostic table with 337.
Two of the ten turned out to be unusable: the local 881-page PDF and the Claude corpus drift apart around the foldout wiring pages, so those page numbers point at physically different pages with almost no text in common. Excluding those, here are the eight aligned dense pages, with the ensemble’s front-matter scores alongside for contrast:
| metric | solo (8 dense pages) | ensemble (front-matter) |
|---|---|---|
| numeric recall | 0.67 | 0.61 |
| table-cell recall | 0.56 | 0.17 |
| block-structure recall | 0.63 | 0.52 |
| text recall | 0.72 | 0.73 |
How well it performs depends enormously on the page in question:
- Spec tables are great. One parts-data page scored a remarkable 0.94 text, 0.95 numbers, and 0.96 table cells. The 316-number spec page hit 0.90 numeric recall.
- Diagrams and dense diagnostics are not. An exploded parts diagram collapsed into a single block. A wiring page managed only 0.30 on numbers. That 337-number HVAC table came back at a dismal 0.17.
So the solo model successfully fixed the ensemble’s structural collapse (tables went from 0.17 to 0.56) and nudged ahead on numbers. But, and this is the crucial takeaway, the front-matter near-perfection does not carry over to dense content, where it still drops about a third of the numbers. Not Claude-parity on the hard pages, which, unfortunately, are precisely the pages that matter most.
The second pass earned nothing
Here’s the part I genuinely didn’t expect. Across all 12 solo pages, the verify pass added zero blocks. Whatever small value-level corrections it made (and I didn’t measure those), it recovered no missing content whatsoever, while doubling the time. As built, the two-pass design simply isn’t paying for itself. Sometimes the data delivers a humbling verdict.
The wall I actually hit
There’s no dollar cost here, but there’s wall-clock time, and it’s nothing short of brutal:
| on the dense pages | value |
|---|---|
| avg per page (2 passes) | ~10.8 min |
| slowest single page (a wiring diagram) | ~24 min |
| 10 dense pages, total | ~107 min |
| front-matter pages (earlier) | ~2-3 min/page |
| peak RAM (no headroom to spare) | ~36 GB |
| extrapolated to the full 881-page manual | ~6 days |
The model sat at ~36 GB of RAM the entire time, with no out-of-memory crashes but no headroom either. Extrapolate the dense-page rate to the whole 881-page manual and you arrive at roughly six days of continuous compute, or three if you drop to a single pass. That settles the question for bulk work. Whether you’re processing targeted pages, a small manual, or an overnight batch, the local approach is perfectly viable. The whole book on one box, however, is simply not practical.
What I took away from it
By the end of this journey, several key insights had crystallized:
- One big 8-bit model beats a 4-bit ensemble for this. Better numbers, far better tables, and a much simpler pipeline. The clever agreement-based escalation just collapsed in practice.
- Quantization precision matters for digits. Going from 4-bit to 8-bit was the single lever that moved numeric recall, which makes intuitive sense when the whole point is reading torque values off a photocopy.
- Local matches Claude on easy pages and trails on hard ones. Dense wiring, parts diagrams, and big diagnostic tables are where the gap lives.
- Throughput is the constraint, not quality or money. Eleven minutes a page doesn’t scale to a full book on one box.
- The verify pass, as designed, isn’t worth twice the time. It added nothing.
Where I’d take it next
Looking ahead, the obvious move is to make pass two cheaper or drop it by default, and only re-verify pages that look uncertain. That’s essentially the original escalation idea, except gated on the model’s own uncertainty rather than two models disagreeing. I want to chase the dense-page number gap with full-resolution input on spec pages, or leverage a proper document-OCR specialist once one is turnkey on MLX. I’d also like to try the 30B mixture-of-experts model at 8-bit as the primary, since it only activates ~3B parameters per token and should decode much faster. And I need to fix the page alignment between the local PDF and the Claude corpus before any larger comparison, because losing 2 of 10 deep pages to drift made them non-comparable.
For now, Claude stays in the pipeline. But here’s the thing: the local approach is genuinely close on the easy 80%, and the on-device version costs nothing and never touches the network. This isn’t just an experiment, it’s a glimpse of where document extraction is heading, so I’m definitely not done poking at it.
A caveat on the numbers: these are automated structural and text-overlap scores against the shipped Claude data, which isn’t ground truth itself, so they undercount output that’s semantically right but structured differently. The samples are small. The scanned page, not the Claude reference, is the real arbiter.
In today’s fast-paced world of artificial intelligence, one question echoes louder than ever through the halls of every home lab and developer workstation: do we really need the cloud? 🤔 Buckle up, dear reader, because today we’re embarking on an epic journey to answer exactly that question, and I promise you, the results will surprise, delight, and occasionally humble us all.
Let me set the stage. A while back, I wrote about sucatisse, my passion project dedicated to transforming decades-old car workshop manuals into clickable, searchable, mobile-friendly digital experiences. It’s a labor of love, a bridge between eras, and a celebration of automotive heritage all woven into one rich tapestry. The extraction half of that pipeline currently harnesses the unparalleled power of cloud Claude (Opus), sending one batch request per page and receiving beautifully structured JSON that forms the very foundation upon which the entire site is built.
And let me be crystal clear: it works. It works wonderfully. But here’s the million-dollar question (or, more accurately, the several-dollars-in-tokens question): every single page costs tokens, and every single page demands a network round-trip. What if there were another way? What if the answer had been sitting on my desk all along?
My MacBook is an M5 Pro with a whopping 48 GB of unified memory. Could this humble laptop, this unassuming slab of aluminum, just do the whole thing itself, with zero network dependency and zero per-token cost? And the truly tantalizing question: how close could it possibly get to the Claude output that already ships in the S13 data? I dedicated an entire day to finding out, and what a day it was.
Building the Dream: The macOS-local Pipeline 🛠️
To answer these burning questions, I rolled up my sleeves and built macOS-local: a completely self-contained, end-to-end pipeline that represents a true labor of engineering craftsmanship. Here’s what this beauty does:
- Rasterizes each page: turning scanned PDF pages into crisp, processable images.
- Runs layout detection: identifying the regions and reading order that give a page its structure.
- Hands the page to a vision model: the beating heart of the operation, extracting typed JSON.
- Crops the figures: slicing out every diagram as its own image, ready for the web.
But here’s the twist in our tale: I iterated on the extraction stage not once, but twice, because, as we’re about to discover together, the first idea spectacularly failed to survive contact with reality. Isn’t that just how engineering goes sometimes? 😅
The Setup: Assembling Our Toolkit ⚙️
Let’s take a deep dive into the technological tapestry underpinning this adventure. Every component was chosen with care:
- MLX, Apple’s native inference stack: Everything runs on MLX via
mlx-vlm, unlocking the full potential of Apple Silicon. - Surya for layout detection: Surya is a phenomenal tool, but here’s a fun behind-the-scenes tidbit: I had to keep it in its own isolated venv because it needs
transformers<5whilemlx-vlmneeds≥5.5, and these two simply refuse to live together in the same environment. Dependency drama at its finest! - PyMuPDF for rasterizing: Faithfully converting pages at 300 dpi grayscale.
- The gold standard: The reference I’m grading against is the shipped Claude extraction: a massive 881 pages, mostly
claude-opus-4-7.
Now, before we go any further, it’s absolutely essential that we pause and appreciate a fundamental truth about local inference on this hardware, because it shaped everything that follows. Reading a page is memory-bandwidth-bound. What does that mean in practice? Every single token the model generates reads the entire model out of unified memory. Let that sink in for a moment. This means the model size and the quantization level directly determine your speed, and a single GPU means you can’t run two models side by side for free. This one constraint became the invisible hand guiding both attempts.
Attempt One: The Great Ensemble Experiment 🗳️
For my first approach, I dreamed up something I was genuinely proud of: an inverted version of how you’d normally tier models in the cloud. The philosophy was elegant in its simplicity. Why pay for an expensive model on every page when cheap models might handle most of them? Here’s how the magic was supposed to work:
- Two 8B vision models vote: Qwen3-VL-8B and InternVL3-8B, both at 4-bit quantization, independently read each page.
- A judge reconciles: A dedicated judging step compares their outputs and scores the agreement.
- Escalation as a safety net: If agreement falls below a 0.85 threshold, the page escalates to a mighty 30B mixture-of-experts model.
On paper? A masterpiece of efficiency. In practice? Well, keep reading. 👀
That was the plan. And now, drumroll please, here’s what running it actually looked like:
Feast your eyes on that agreement column, because it tells a story of beautiful dreams meeting harsh reality. The plan was that most pages would gracefully clear the cheap path and stop right there. Instead? The two 8B models structured pages so wildly differently that mean agreement crash-landed at a heartbreaking 0.60, well under my 0.85 threshold, meaning 19 of 20 pages escalated anyway. The cheap path almost never terminated! I was effectively paying for the entire stack on nearly every single page, which is exactly the thing this whole design was engineered to avoid. The irony is almost poetic, isn’t it? And as if that weren’t enough, 4-bit quantization seemed to be actively sabotaging digit reading. Ouch. 💔
The scores against Claude told the very same sobering story. Behold, the ensemble on 20 pages:
| metric | ensemble vs Claude |
|---|---|
| numeric recall | 0.61 |
| table-cell recall | 0.17 |
| block-structure recall | 0.52 |
| text recall / precision | 0.73 / 0.96 |
| page-type match | 60% |
Let’s break down what these numbers are really telling us:
- Text coverage was fine: 0.73 recall is respectable territory.
- No hallucinations: A stellar 0.96 precision means the model wasn’t making things up, and that deserves genuine applause.
- But the numbers vanished: It missed roughly 40% of the numbers on the page. For a workshop manual, that’s devastating.
- Tables were decimated: A table-cell recall of 0.17 means tables barely survived at all.
- The table of contents tragedy: On the TOC page, it flattened the entire section-code index into 22 plain headings and lost every single code.
Here’s the bottom line, friends: for a workshop manual, where the numbers are the content, this isn’t a stumble. It’s a fail. Full stop.
Attempt Two: One Big Model, Twice the Determination 💪
Did I give up? Absolutely not! Instead, I threw the entire ensemble out the window and embraced the polar opposite philosophy. Instead of several small models bickering like committee members, why not recruit the single biggest, highest-precision model the 48 GB box can possibly hold, and empower it to check its own work?
Enter our new champion: Qwen3-VL-32B-Instruct at 8-bit, tipping the scales at approximately 36 GB and fitting into memory with barely any room to spare. Talk about living on the edge! Here’s the complete game plan:
- 8-bit instead of 4-bit: I made this choice specifically to slay the digit-reading dragon from attempt one.
- Higher resolution input: I fed it pages at 2048 px instead of 1536, so those tiny little numbers stay beautifully legible.
- Two passes for good measure: The second pass re-reads the page plus the first draft and is explicitly told to fill in anything missed.
- A hardened schema: Malformed blocks stopped blowing up the run. Resilience, baby!
The Easy Pages: A Triumph for the Ages 🏆
On the two front-matter pages I’d previously compared head-to-head with the ensemble, the solo model didn’t just improve, it absolutely soared on every single axis:
| metric (front-matter pages) | ensemble | solo |
|---|---|---|
| numeric recall | 0.61 | 1.00 |
| table-cell recall | 0.00 | 1.00 |
| block-structure recall | 0.66 | 0.90 |
| text recall | 0.75 | 0.99 |
| page-type match | 1 / 2 | 2 / 2 |
Perfect numeric recall. Perfect table cells. It heroically captured the table-of-contents section codes the ensemble had utterly destroyed, and it even recovered the cover title. On easy pages, ladies and gentlemen, this is nothing less than Claude-parity. What a game-changer! 🎉
The Hard Pages: Where Reality Checks In 😬
But hold on. Front matter is the easy stuff, and we both know it. The real manual is an entirely different beast: spec tables, exploded parts diagrams, wire-level schematics, and diagnostic flowcharts. So I added a page selector and courageously pointed the pipeline at 10 dense pages deep in the book, including a spec page with a jaw-dropping 316 numbers on it and an HVAC diagnostic table boasting 337.
A quick plot twist: two of the ten turned out to be unusable, because the local 881-page PDF and the Claude corpus drift apart around the foldout wiring pages, so those page numbers point at physically different pages with almost no text in common. Excluding those, here are the eight aligned dense pages, with the ensemble’s front-matter scores alongside for contrast:
| metric | solo (8 dense pages) | ensemble (front-matter) |
|---|---|---|
| numeric recall | 0.67 | 0.61 |
| table-cell recall | 0.56 | 0.17 |
| block-structure recall | 0.63 | 0.52 |
| text recall | 0.72 | 0.73 |
And here’s where it gets fascinating: performance depends enormously on the page:
- Spec tables are great. One parts-data page scored a magnificent 0.94 text, 0.95 numbers, and 0.96 table cells. The 316-number spec page hit 0.90 numeric recall. Chef’s kiss! 👨🍳
- Diagrams and dense diagnostics are not. An exploded parts diagram collapsed into a single sad block. A wiring page limped to 0.30 on numbers. And that 337-number HVAC table? It came back at a heartbreaking 0.17.
So what’s the verdict? The solo model triumphantly fixed the ensemble’s structural collapse (tables rocketed from 0.17 to 0.56) and nudged ahead on numbers. But, and this is the crucial reality check, that front-matter near-perfection does not carry over to dense content, where it still drops about a third of the numbers. Not Claude-parity on the hard pages. And which pages matter most? You guessed it: the hard ones.
The Second Pass: A Plot Twist Nobody Wanted 🎭
And now for the part I genuinely, truly did not see coming. Across all 12 solo pages, the verify pass added… zero blocks. Zero! Whatever small value-level corrections it may have made (and full transparency: I didn’t measure those), it recovered absolutely no missing content, all while doubling the processing time. As built, the two-pass design simply isn’t paying for itself. Sometimes the most valuable lesson is discovering what doesn’t work, right?
The Wall I Actually Hit: Time Itself ⏰
Here’s the beautiful thing about local inference: there’s no dollar cost. Here’s the not-so-beautiful thing: there’s wall-clock time, and my friends, it is nothing short of brutal:
| on the dense pages | value |
|---|---|
| avg per page (2 passes) | ~10.8 min |
| slowest single page (a wiring diagram) | ~24 min |
| 10 dense pages, total | ~107 min |
| front-matter pages (earlier) | ~2-3 min/page |
| peak RAM (no headroom to spare) | ~36 GB |
| extrapolated to the full 881-page manual | ~6 days |
The model sat at ~36 GB of RAM for the entire journey, with no out-of-memory crashes but absolutely no headroom either. A true tightrope walk! Now, extrapolate that dense-page rate to the whole 881-page manual and you arrive at roughly six days of continuous compute, or three days if you drop to a single pass. Six. Days.
That, dear reader, settles it for bulk work. Targeted pages? Fine. A small manual? Fine. An overnight batch? Absolutely fine. The whole book on one box? A resounding no.
What I Took Away From This Incredible Journey 🎓
By the end of this rollercoaster, several profound lessons had crystallized into wisdom:
- One big 8-bit model beats a 4-bit ensemble for this. Better numbers, far better tables, and a dramatically simpler pipeline. The clever agreement-based escalation, for all its theoretical elegance, simply collapsed in practice.
- Quantization precision matters for digits. Going from 4-bit to 8-bit was the single lever that moved numeric recall, which makes perfect sense when the whole point is reading torque values off a photocopy.
- Local matches Claude on easy pages and trails on hard ones. Dense wiring, parts diagrams, and big diagnostic tables are where the gap lives and breathes.
- Throughput is the constraint, not quality or money. Eleven minutes a page simply doesn’t scale to a full book on one box, no matter how much we wish it did.
- The verify pass, as designed, isn’t worth twice the time. It added nothing. Zero. Zilch.
Where I’d Take It Next: The Road Ahead 🛣️
Every great experiment plants the seeds of the next one, and this adventure is no exception. Here’s my roadmap for the future:
- Smarter verification: Make pass two cheaper or drop it by default, and only re-verify pages that look uncertain. That’s the original escalation idea reborn, except gated on the model’s own uncertainty rather than two models disagreeing.
- Chase the number gap: Attack the dense-page deficit with full-resolution input on spec pages, or bring in a proper document-OCR specialist once one is turnkey on MLX.
- Unleash the MoE: Try the 30B mixture-of-experts model at 8-bit as the primary, since it only activates ~3B parameters per token and should decode dramatically faster.
- Fix the alignment: Repair the page alignment between the local PDF and the Claude corpus before any larger comparison, because losing 2 of 10 deep pages to drift made them non-comparable.
Conclusion: The Journey Is the Destination 🌅
So, where does this leave us? For now, Claude stays in the pipeline, and it has earned that place. But let’s not lose sight of the bigger picture: the local approach is genuinely close on the easy 80%, and the on-device version costs nothing and never, ever touches the network. That’s not a consolation prize. That’s a foundation.
This experiment reminded me why I fell in love with tinkering in the first place. It’s not just about the metrics or the pipelines. It’s about asking bold questions, embracing spectacular failures, and inching ever closer to something wonderful. A 48 GB laptop reading a thirty-year-old workshop manual, one page at a time, with no cloud in sight? That’s the kind of future worth building toward. I’m not done poking at it, and honestly, I hope I never am.
Thank you, truly, for joining me on this deep dive. If you made it this far, you’re my kind of reader. Until next time: stay curious, keep experimenting, and never stop reading the manual. 🚗✨
A caveat on the numbers: these are automated structural and text-overlap scores against the shipped Claude data, which isn’t ground truth itself, so they undercount output that’s semantically right but structured differently. The samples are small. The scanned page, not the Claude reference, is the real arbiter.