Are AI takeoffs accurate? How to test one on a past bid before a live one
An AI takeoff is as accurate as a test on your own drawings shows, and no more. No standard we found sets an acceptable error for an automated takeoff line. A vendor's figure, for its part, was measured on its own drawings, with its own idea of a correct line. So test before any live bid, whoever the vendor. Run the tool blind on a bid your team has already priced and checked, on the set as issued, for the trades in scope. Write a tolerance per item type and unit before any output exists. Then compare every line with your reference and give each difference a class: missed item, wrong scale, wrong revision, double count, wrong assembly or unit, or out of scope. Check that each quantity names the sheet and revision it came from. Rerun the sheets an addendum changed, and log the hours, review included. Finally, decide go or no-go on rules written in advance, and use the tool on live bids under review.
- Published
- Updated
- Reading time
- 15 min
- Written by
- Jules Toussaint
This guide is a test protocol for estimators at general contractors, heavy civil and specialty contractors who price from drawings. You run it on any takeoff tool before the tool touches a live bid. It names no vendor's results and reviews no software.
Who wrote this, and why. Mirage Metrics deploys AI agents for construction estimating teams, takeoff among them. At OCBM, a Moroccan general contractor, every quantity carries a reference to the drawing it was measured on: 100% of quantities are traceable to a source drawing, and estimators review the takeoff line by line. At John Cockerill, piping takeoff takes 80 to 90% less time, with engineers reviewing the output instead of measuring. Mirage publishes no accuracy percentage, and no test run under this protocol has been published yet. This is the protocol to run on any tool, Mirage's included.
What goes wrong in an AI takeoff, and how each error shows
A takeoff can be wrong in a handful of ways, and each leaves a different mark when you set it against a checked takeoff. GAO's cost guide asks that a reliable estimate be "structured with sufficient detail to ensure that cost elements are neither omitted nor double-counted" (GAO-20-195G, p.31). Those two failures, a missed item and a double count, are also the two that can cancel each other out in a trade total. That is why the test compares lines, not totals. ConstructConnect, which sells an AI takeoff, wrote its own article on the question. It lists what still needs full review: complex or irregular geometry, scope boundaries, material specifications and changes, revision clouds and addenda, and low-quality scans (Maila Kim, August 2026). A test set without those cases only tests the easy part.
| Class | How it shows against your reference | Usual cause on a PDF set | How to confirm it |
|---|---|---|---|
| Missed item | A line of yours with no tool quantity, or a tool total short by whole items | A symbol the tool did not recognize; an item shown only in a schedule, a detail or an enlarged plan; a sheet left out of the run | Search the output for the item under another description before calling it missed |
| Wrong scale | On one view, lengths at a ratio of 2 or 0.5 and areas at 4 or 0.25, while counts agree | A sheet printed to fit the paper; a detail or enlarged plan drawn at another scale than its sheet; a view marked not to scale | Measure one dimensioned length on that view and read the scale on its title |
| Wrong revision | Differences gathered on the sheets an addendum changed; the line cites a superseded revision | An earlier issue of the sheet was read, or its quantities were kept after the revision | Compare the revision the line cites with your drawing register |
| Double count | A ratio of 2, or one typical floor too many; one quantity citing two sheets for the same area | The same area measured on a plan and on its enlarged plan; a typical floor counted again; overlapping zones | Open every sheet the line cites |
| Wrong assembly or unit | The right place with the wrong item, or a ratio of 9, 27 or about 10.76 | A wall type, slab thickness or pipe class read from the wrong legend entry; square feet against square yards, cubic feet against cubic yards, square meters against square feet | Check the item against the legend or schedule, then the unit |
| Out of scope | A tool quantity where your reference has none | Work by others or not in contract; another trade's item; an alternate measured as base bid | Check the scope sheet and the alternates on the bid form |
| Convention not given (not the tool's error) | A small, steady difference on every line of one type | Deductions for openings, laps and waste, rounding, or the contract's measurement rule, never given to the tool | Give the tool your written conventions and rerun |
| Error in your reference (not the tool's error) | The tool's quantity holds up and yours does not | A slip in the original takeoff | Re-measure the line by hand and correct the reference before scoring |
Read the ratio of the tool's quantity to yours before the variance, because the ratio often names the error. A line at 9 is a unit error until shown otherwise, since a square yard is 9 square feet and a cubic yard 27 cubic feet. A view where every length comes out at 2 and every area at 4 has one wrong scale, not dozens of errors, so it has one fix. A ratio of 0 is a missed item, unless the tool measured the item under another name.
Choose the reference bid, the trades and the conventions
The test is only as good as its reference. NIST's AI Risk Management Framework, quoting ISO/IEC TS 5723:2022, defines accuracy as the "closeness of results of observations, computations, or estimates to the true values or the values accepted as being true". It also asks that accuracy measurements be paired with "clearly defined and realistic test sets" that are "representative of conditions of expected use", with the test method documented (NIST AI 100-1, section 3.1). For a takeoff, the values accepted as true are your team's checked quantities, and the conditions of expected use are your drawings.
The bid
Take a bid your team priced recently, of the kind you bid most, whose takeoff was checked. The check can come from a second estimator, from quantities remeasured after award, or from final pay quantities on unit-price work. Use the set as issued for bid and keep its addenda aside for the revision test. Never use the as-built set. You choose the bid, not the vendor, and the vendor sees neither the set nor your quantities before the run. The set should hold what your bids hold: scanned sheets, details and enlarged plans at other scales, schedules, revision clouds, and a specification that changes an assembly. Sample sizes and the holdout rule for any document workflow are in the guide on evaluating a document automation vendor with a pilot.
The trades and item types
Pick the trades that carry most of your takeoff hours. For each one, list the item types it measures: counts, lengths, areas, volumes, weights. A tool line outside that list is out of scope, and a line of yours inside it with no tool quantity is missed. Then mark the lines your team would never let through unchecked, such as structural concrete or self-performed work. The scorecard counts critical lines outside tolerance on their own.
The conventions
Write down the conventions your reference was measured to, and give the tool the same list. A difference that comes from a convention the tool never had is a setup item, not an error, and the scorecard keeps it apart. Where the rules come from depends on the work.
| Work | Rules | What they fix |
|---|---|---|
| UK building works | RICS NRM 2, 2nd edition, effective 1 December 2021 | Work net as fixed in position; laps, joints and waste inside the net quantity; dimensions to the nearest 10 mm; whole units, a quantity under one given as one; tonnes to two decimal places; when voids are deducted |
| UK civil engineering | ICE, CESMM4 Revised, June 2019 | The bill's classes and rules; ICE describes it as the standard for bills of quantities in civil engineering, usable with NEC, FIDIC and ICC contracts |
| US public works on unit prices | The owner's standard specifications | How each pay item is measured. At WSDOT: areas from neat dimensions, with exceptions over 9 square feet deducted; lengths along the element; excavation by average end area or terrain model; a ton of 2,000 pounds |
| US private work | Your company's written conventions | Everything the drawings leave open. ASPE's Standard Estimating Practice, built on MasterFormat, sets out estimating practice |
Source: NRM 2, sections 1.3 and 3.2.1; ICE Publishing's description of CESMM4 Revised; WSDOT M 41-10 (2027), section 1-09.1; ASPE, 12th Edition Standard Estimating Practice.
In the UK, NRM 2 asks that a bill prepared to it say so (section 1.3). The US, by contrast, has no single method of measurement that bids are prepared to. On unit-price public work, the owner's specification decides. WSDOT's, for example, has every measurement made as its section 1-09.1 describes, "unless individual Specifications require otherwise". On private work, your own conventions decide. Either way, the tool needs them in writing. For the manual method your reference was measured with, see the quantity takeoff guide.
Set a tolerance per item type and unit, before any output
No standard we found sets an acceptable error for a takeoff line, manual or automated, so the tolerance is your team's call. Set it per item type and unit, and write down why. Do it before anyone sees the tool's output, because a tolerance written afterwards drifts toward the result. Three things shape it.
- Counts. A door frame is there or it is not. A count has no measuring error to tolerate: any difference is an item missed or added.
- Your own rounding. NRM 2 measures dimensions "to the nearest 10mm" and gives quantities "to the nearest whole number", except tonnes, "to two decimal places" (3.2.1). WSDOT carries each item to "the proper significant figure or fraction of units" (1-09.1). A tolerance tighter than the rounding your reference follows measures nothing.
- Which way the error costs you. On a lump sum, a quantity short of the real one is cost you carry, and a quantity over it is price you add to the bid. You may want a tighter limit on the side that costs you more. The scorecard takes one value per item type, either side, and accepts a value of its own for any line.
Then score line by line and report by trade and item type, never as one number. NIST notes that accuracy measurements "may include disaggregation of results for different data segments" (section 3.1). For a takeoff, the segments are trades and item types. A tool that counts fixtures well can still measure walls badly. A single percentage also depends on what counts as a line: four hundred light fixtures on one line weigh the same as a slab.
Run it blind, then compare every line
Give the tool the set as issued, the scope and the conventions, and nothing else. It sees neither your quantities nor your trade totals. Record the tool's version and settings, and freeze them for the scored run. Whoever will run the tool on live bids runs it for the test, under the conditions of a live bid. That is your estimator, or the vendor's team if the vendor operates it.
Then match the lines, pairing each tool line with your line for the same item and location. A tool line with no match is out of scope or a double count. A line of yours with no match is missed. For each pair, the ratio is the tool's quantity divided by yours, and the variance is that ratio less one. The line is either within tolerance or not, which the scorecard records as a 1 or a 0. Give every line outside tolerance a class from the table above. Keep the two differences that are not the tool's errors apart: correct your reference, give the tool the missing convention, and score again.
The spreadsheet holds the whole test. It sets tolerances by item type and takes up to 300 lines, each with your quantity, the tool's, the sheet and revision each cites, the ratio, the variance and a 1 or 0 for within tolerance. Error classes and the reviewer's source check are picked from lists, and hours are logged. A summary reports by set, by trade and by error class, and six go/no-go rules are computed from thresholds you type. Every quantity, tolerance and threshold is left empty.
Trace every quantity to its sheet and revision
A quantity you cannot trace is a quantity you will measure again. GAO calls an estimate well documented when it "can easily be repeated or updated and can be traced to original sources through auditing". Such an estimate is described so that "a cost analyst unfamiliar with the program could understand what was done and replicate it" (GAO-20-195G, pp.31 to 32). For a takeoff, the original source is a sheet, at a revision, and a place on it.
Ask for that reference on every line, then check it. Open the cited sheet for every line outside tolerance and for a random sample of the lines within it, and record whether the quantity is there. A line that cites nothing fails whatever its value, because on a live bid nobody will know whether to trust it. A line that cites a superseded revision is a wrong revision even when its quantity happens to match. This is the bar Mirage's published takeoff deployment holds: at OCBM, every quantity carries a reference to the drawing it was measured on, and none is without one (OCBM case study).
Test a revision the way bids receive them
Bids change late, so the test should include a revision. AIA's standard Instructions to Bidders has addenda issued "no later than four days prior to the date for receipt of Bids". The exception is an addendum withdrawing the request for bids or postponing that date. Each bidder "shall acknowledge their receipt in the Bid" (A701-2018, sections 3.4.3 and 3.4.4). A takeoff that handles a revision badly therefore fails in the last days before bids are due, which are the days that matter most.
After the first run, give the tool the addendum, or the next issue of the sheets it changed, and check four things:
- It finds the changed sheets and the clouded areas on them.
- It updates the lines those changes affect and leaves the others as they were.
- Every updated line cites the new revision, and no line still cites a superseded one.
- The updated quantities are within tolerance of your own revised takeoff.
Score those lines as the addendum set, apart from the first run. Comparing two issues of a set is a workflow of its own, described on our page on plan revision comparison.
Measure the time, review included
Log three numbers. The first is the hours spent preparing the documents, scope and conventions. The second is the hours running the tool, and the third the hours reviewing and correcting its output. Set them against the hours your team logged on the reference takeoff for the same trades and sheets. If nobody logged them, time a fresh measure of a few sheets by hand rather than rebuild the figure from memory. A takeoff produced in minutes and checked over two days has not saved two days. The guide on automating construction estimating explains how to build a baseline over several bids, and what to measure once the tool is live.
What to do with a failure
A failed line says as much about the tool's setup as about the tool. Sort the failures before deciding anything.
- Systematic or scattered. Errors of one class on one kind of sheet, every enlarged plan or every scanned detail, tend to share a cause and a fix. Misses scattered across clean sheets do not.
- Wrong scale. Check how the scale was set. Bluebeam's instructions, for one, allow a preset scale only when no scaling occurred while the PDF was created, and otherwise calibrate on a known length. One page can also carry viewports at different scales (Bluebeam Technical Support). A scale read once per sheet is wrong on every view drawn at another.
- Wrong assembly or unit. Give the tool the legends, the schedules and your unit conventions, then rerun.
- Wrong revision. A tool that cannot tell two issues of a sheet apart is a no-go for any bid with addenda, whatever else it does well.
- Missed items. Look for a pattern: a symbol, an item shown only in a schedule, a kind of sheet. A pattern becomes a review rule for live bids. No pattern means a wider review sample.
After any fix, score again on a second past bid. The first one has been seen, and a tool tuned on it will look better there than on the next bid.
Decide go or no-go, then move to live bids under review
NIST's framework puts the decision plainly: "A determination is made as to whether the AI system achieves its intended purposes and stated objectives and whether its development or deployment should proceed" (MANAGE 1.1). Write the rules before the run, with your own thresholds. The scorecard computes six:
- the share of compared lines within tolerance, at least your threshold, overall and per trade;
- critical lines outside tolerance, at most your threshold;
- lines without a cited sheet and revision, at most your threshold;
- sources the reviewer rejected, at most your threshold;
- addendum lines within tolerance, at least your threshold;
- hours with the tool, review included, as a share of the reference hours, at most your threshold.
One failed rule is a no-go. The answer can differ by trade: a tool can pass on finishes and fail on concrete, and go live on finishes alone.
Live bids, under review
Go means live bids with a named reviewer, not without one. On the first live bids the reviewer checks every line the tool flags, every critical line and a sample of the others, and keeps logging error classes in the same scorecard. The sample shrinks as lines hold over several bids, trade by trade, and grows again with a new trade, a new kind of sheet or a new version of the tool. Write down who signs the quantities of each bid before the first one. In Mirage's deployments each workflow has a named approver, and every agent action leaves an audit record (approvals and audit).
When not to use an AI takeoff on a bid
An AI takeoff does not belong on every bid, tested or not. Keep it off in these six cases.
- You have no checked reference. A past bid whose takeoff nobody checked can only show that two unchecked numbers differ.
- The bid is due this week and the tool is untested. Testing and bidding at once puts the risk on the bid.
- The tool cannot show where a quantity came from. Without a sheet and a revision per line, the review becomes a second takeoff, and nothing is saved.
- The drawings are unlike the ones you tested. A new trade, a new kind of sheet such as hand-marked scans, or another unit system needs a test of its own.
- The estimate is priced per unit of area. At concept stage there is little to measure, and a takeoff's accuracy does not move a price per square foot.
- Nobody can review. A tool that passed its test still needs a reviewer on live bids. Without one, keep it off them.
Questions readers ask
AI takeoffs are as accurate as a test on your own drawings shows, and no more. No standard we found sets an acceptable error for an automated takeoff line, and a vendor's figure comes from its own test set. To find out, run the tool blind on a bid your team has already priced and checked, and compare every line with your quantities. Then judge the result against tolerances and rules you wrote before the run.
The tolerance for an AI takeoff is your own to set, per item type and unit, in writing before any output exists. None of the standards cited in this guide sets one for a takeoff line. A count has no measuring error to tolerate, so any difference is an item missed or added. For lengths, areas and volumes, a tolerance tighter than the rounding your reference follows measures nothing. NRM 2, for example, rounds dimensions to the nearest 10 mm and quantities to whole units.
The six kinds of error to look for in an AI takeoff are missed items, wrong scale, wrong revision, double counts, wrong assembly or unit, and quantities outside the bid's scope. Two other differences are not the tool's errors: a convention it was never given, and an error in your own reference. The ratio of the tool's quantity to yours often names the class: 2 for a double count, 4 for an area at a scale off by two, 9 for square feet against square yards.
To test a takeoff tool on an addendum, give it the sheets the addendum changed after the first run, then check what it does with them. It should find them and their clouded areas, update the lines they affect and leave the others alone. Every updated line should cite the new revision. Score those lines apart from the first run, against your own revised takeoff.
The vendor should not choose the test project: you choose the past bid yourself. Keep your quantities from the vendor until its output is in, and freeze the tool's version and settings for the scored run. If the vendor operates the tool, it works under the same conditions as it would on a live bid. After any fix, score again on a second past bid, because the first one has been seen.
Sources and scope
Checked on . All sources read on 10 November 2026. NRM 2, the WSDOT specifications, NIST AI 100-1 and GAO-20-195G were read in the publishers' PDFs; A701-2018 in a copy issued for a public bid; CESMM4 Revised is described from ICE Publishing's book description, and no CESMM rule is stated. ConstructConnect's article is a vendor's view and is cited as one. OCBM's and John Cockerill's facts come from their case studies on this site.
- NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023 (PDF): section 3.1, p.14: accuracy as closeness to the values accepted as true (quoting ISO/IEC TS 5723:2022); false positive and false negative rates; realistic test sets representative of conditions of expected use, with the test method documented; results disaggregated by data segment. MANAGE 1.1, p.32: whether deployment should proceed
- U.S. Government Accountability Office, Cost Estimating and Assessment Guide, GAO-20-195G, March 2020 (PDF): chapter 3, p.31: comprehensive estimates detailed enough that cost elements are neither omitted nor double-counted; well-documented estimates traced to original sources through auditing. p.32: described so that an analyst unfamiliar with the program could replicate it
- RICS, NRM 2: Detailed measurement for building works, 2nd edition UK, October 2021, effective 1 December 2021, reissued October 2022 as practice information (PDF): 1.3: a bill prepared under NRM 2 says so; 1.4: rules of measurement for bills of quantities and quantified schedules of works; 3.2.1: work net as fixed in position, laps, joints and waste inside the net quantity, dimensions to the nearest 10 mm, whole units with a quantity under one given as one, tonnes to two decimal places, voids
- ICE Publishing, CESMM4 Revised: Civil Engineering Standard Method of Measurement, June 2019 (publisher's description, as listed by Wildy & Sons): the standard for preparing bills of quantities in civil engineering for over 40 years; usable with NEC, FIDIC and ICC contracts
- Washington State DOT, Standard Specifications for Road, Bridge, and Municipal Construction M 41-10 (2027), Division 1 (PDF): 1-09.1 Measurement of Quantities: measurements as the section describes unless an individual specification requires otherwise; each item carried to the proper significant figure; areas from neat dimensions, exceptions over 9 square feet deducted; linear foot along the element; excavation by average end area or terrain model; a ton of 2,000 pounds
- American Society of Professional Estimators, 12th Edition Standard Estimating Practice (web page): 2025 edition in four volumes, built on CSI MasterFormat
- AIA Document A701-2018, Instructions to Bidders, a copy issued in February 2024 for a public bid (PDF): 3.4.3: addenda issued no later than four days before the date for receipt of bids, except one withdrawing the request for bids or postponing that date; 3.4.4: each bidder makes sure it has every addendum and acknowledges them in the bid
- Bluebeam Technical Support, Set the page scale on your drawing: a preset scale only when no scaling occurred during PDF creation; calibration on a known length; viewports at different scales on one page
- Maila Kim, ConstructConnect, Are AI Takeoffs Accurate in 2026? What Estimators Gain and Still Need to Review, August 18, 2026: a vendor's view: standard walls, areas, lines and repeated symbols on clean plans as the fast cases; complex geometry, scope boundaries, material specifications and changes, revision clouds and addenda, low-quality scans as needing full review
- Mirage Metrics, OCBM case study: every quantity carries a reference to the drawing it was measured on, 100% traceable to a source drawing; estimators review line by line
- Mirage Metrics, John Cockerill case study: 80 to 90% less time on piping takeoff, engineers reviewing the output instead of measuring
Related on this site
- AI takeoff and estimating: Mirage's takeoff workflow and the software question
- How to automate construction estimating: which work package first, the data and the measures
- Document automation pilot: holdout sets and sample sizes, across sectors
- Quantity takeoff from PDF drawings: the manual method behind the reference
- Plan revision comparison: what changed between two issues of a set
Show us the trades your estimators measure by hand
Leave your work email, then pick a time for a 30-minute call on those trades and the past bid you could test on. You leave with a written plan: how the test would run, what it would measure and what stays with your team. No commitment.