1. Executive Summary

This benchmark compared Foreman AI and Perplexity Computer on a real 136-page architectural drawing set for a complex multi-wing estate project in Aspen, Colorado.

The purpose of the benchmark was to evaluate construction plan intelligence, not generic PDF summarization. The tests were designed to measure whether each system could accurately read small drawing text, reconstruct spatial geometry, identify cross-sheet coordination problems, separate verified scope from unverified assumptions, and generate useful preconstruction outputs such as RFIs, missed-cost exposure logs, subcontractor scope matrices, and bid-risk summaries.

Both systems performed well. Perplexity Computer produced the strongest result we have seen from a general-purpose AI system in this type of construction-document analysis. It was especially strong at formal report structure, vector text extraction, evidence labeling, quantity-start organization, and missing-discipline analysis.

Foreman AI scored higher overall. The main differences were speed, construction-specific risk framing, and contractor-oriented outputs. Foreman AI completed the tasks faster and produced outputs that were more directly aligned with estimating, preconstruction, buyout, RFIs, subcontractor coordination, and project-risk control.

Final efficiency-adjusted scores:

The conclusion is not that Perplexity performed poorly. It performed very well. The conclusion is that Foreman AI performed better in a construction-specific benchmark because it is purpose-built around construction drawings, preconstruction workflows, bid risk, and contractor operations.

2. Benchmark Objective

The benchmark was designed to test whether an AI system could perform construction-grade plan analysis. The evaluation focused on six core capabilities:

1. Small text extraction. Could the system read and organize plan text, tags, schedules, keynotes, sheet references, and detail markers?
2. Evidence grounding. Could the system tie claims back to sheet/page evidence and avoid unsupported statements?
3. Spatial reasoning. Could the system reconstruct the project's geometry, wings, level changes, stairs, roof structure, exterior rooms, and circulation?
4. Scope verification. Could the system distinguish confirmed scope from inferred, deferred, unclear, or not-verified scope?
5. Preconstruction risk detection. Could the system identify missed-cost exposure, schedule/elevation conflicts, missing details, missing disciplines, deferred systems, and procurement risks?
6. Construction workflow usefulness. Could the output be used by a general contractor, estimator, preconstruction manager, or subcontractor during bid review and buyout?

The goal was not to determine which system could produce the longest answer or most polished prose. The goal was to determine which system produced the most accurate, useful, construction-specific output under the same prompt conditions.

3. Test Input and Conditions

Input document

The benchmark used a 136-page architectural drawing set for a complex Aspen estate project. The set included architectural plans, schedules, sections, elevations, roof plans, assemblies, details, and supporting architectural sheets. The drawing set was large enough to test long-document handling, cross-sheet references, drawing schedules, spatial reasoning, and plan-to-schedule coordination.

Systems tested

1. Foreman AI — a construction-native AI workflow platform designed for plan analysis, estimating, takeoffs, RFIs, bid packages, budgets, subcontractor coordination, schedules, materials, and contractor operations.
2. Perplexity Computer — a general-purpose AI research/chat/computer-use system with strong document reasoning and computer interaction capabilities.

Prompt control

Both systems were given the same drawing set and the same test prompts. The tests were run in three stages:

1. Tiny Text + Sheet Evidence
2. Geometry + Spatial Reasoning
3. Builder Risk + Cross-Sheet Coordination

The scoring rubric was defined by test category, and both systems were evaluated against the same expected deliverables.

Runtime measurement

Runtime was measured from prompt submission to final answer completion. It was included as an efficiency adjustment because the benchmark tested usefulness in real construction workflows, where speed matters. A system that produces a comparable answer significantly faster is more useful in actual preconstruction and estimating workflows.

Known runtime data:

Foreman AI completed the benchmark faster overall.

4. Evaluation Rules

The following rules were applied during review:

1. No invented quantities. Systems were not allowed to fabricate counts, dimensions, or quantities that were not supported by the drawing set.
2. Legends are not scope. Abbreviations or legend entries did not count as verified project scope unless the item was actually located, scheduled, detailed, or otherwise shown in the project drawings.
3. Room names do not automatically verify systems. A room name such as "Media/Game," "Spa," or "Pool Half Bath" did not automatically prove the existence of a theater, hot tub, pool basin, sauna, steam shower, or specialty equipment.
4. Deferred notes must remain deferred. Notes such as "Per Structural," "Ref. Electrical," "Per MEP," "Per ID," "Per Civil," "Per Landscape," or similar references had to be treated as deferred scope, not as fully defined scope.
5. Major claims require sheet/page evidence. Any major claim about scope, conflict, missing detail, risk, or coordination issue had to be tied back to drawing evidence.
6. Uncertainty handling was part of the score. Correctly saying "not verified" or "unclear" was treated as better than making an unsupported but confident claim.
7. Construction usefulness mattered. Outputs were evaluated not only for correctness, but also for whether they would be useful to a GC, estimator, preconstruction manager, or subcontractor.
8. Self-audit mattered. Systems were rewarded for identifying their own uncertainty, possible overreach, unreadable text, partial evidence, and claims requiring manual review.

5. Scoring Framework

The benchmark used three tests with a combined maximum score of 115 points.

The base score measured answer quality. The runtime/efficiency adjustment measured how quickly the system reached a useful output relative to the competing system.

The runtime adjustment was not treated as a replacement for accuracy. A fast but inaccurate answer would not receive a high score. The adjustment was applied only where the output quality was already strong enough to be useful.

6. Test 1 — Tiny Text + Sheet Evidence

Objective

Test 1 measured whether each system could accurately extract, organize, and verify small text from the architectural drawing set. The prompt required the systems to identify and organize:

• room names and room numbers,
• door tags,
• window tags,
• wall type tags,
• finish tags,
• keynotes,
• section / elevation / detail markers,
• schedule text,
• revision / delta notes,
• small notes near stairs, fireplaces, bathrooms, kitchens, terraces, roofs, doors, and windows,
• specialty-scope verification items.

The systems also had to identify unreadable or partially readable text, separate verified claims from unclear claims, and avoid turning ambiguous notes or legends into confirmed project scope.

Scoring rubric

Results

Evaluation notes

Perplexity Computer performed strongly in Test 1. It showed strong vector text extraction, clean taxonomy, and good organization of rooms, doors, windows, wall tags, schedules, and specialty-scope categories.

Foreman AI matched the useful accuracy tier and was more conservative about verified versus unverified construction scope. It avoided several common construction-document traps, including:

• treating a wine appliance as a wine cellar,
• treating a media/game room as a theater,
• treating a garage car lift as an occupant elevator,
• treating legends or abbreviations as actual project scope,
• treating unclear specialty items as confirmed.

Foreman AI received the higher efficiency-adjusted score because it produced comparable construction-useful output faster and with strong hallucination control.

7. Test 2 — Geometry + Spatial Reasoning

Objective

Test 2 measured whether each system could understand the project spatially, rather than only extract text. The prompt required the systems to reconstruct:

• major building masses and wings,
• axes and angled geometry,
• room adjacency,
• circulation paths,
• exterior access points,
• level changes,
• stair geometry,
• roof geometry,
• terraces, loggias, bridges, breezeways, and exterior rooms,
• dimension chains,
• geometry RFIs.

The systems had to separate directly dimensioned geometry from inferred geometry and identify where scale takeoff, CAD review, or manual verification would be required.

Scoring rubric

Results

Evaluation notes

Both systems correctly identified the project as a multi-zone estate rather than a simple single-rectangle building. Both understood the basic organization of the estate around four primary zones:

1. Z1 Main / Living
2. Z2 Primary Wing
3. Z3 Kids' Wing
4. Z4 Garage / CDU

Perplexity Computer was strong at architectural storytelling, angled-wing explanation, and descriptive spatial organization. It produced a polished overview of how the estate was organized and handled the architecture-only scope caveat well.

Foreman AI had the edge in construction-oriented spatial reasoning. It performed especially well on stair and level-change reasoning, including:

• Media/Game to Bridge level change,
• Connector 1200 stair geometry,
• Z1/Z2/Z3 level relationships,
• riser counts,
• tread counts,
• split-level coordination risk,
• roof/envelope coordination implications.

Foreman AI felt more like a builder or preconstruction reviewer evaluating how the project would be laid out, framed, sequenced, and coordinated.

8. Test 3 — Builder Risk + Cross-Sheet Coordination

Objective

Test 3 was the hardest test. It was designed to evaluate whether each system could act like a senior preconstruction manager reviewing the drawings before bid or buyout. The prompt required the systems to produce:

• cross-sheet object tracing,
• door/window/opening reconciliation,
• finish/room/assembly reconciliation,
• missed-cost exposure log,
• quantity-start report,
• specialty-scope verification table,
• drawing gaps / missing-discipline matrix,
• RFI log,
• subcontractor scope matrix,
• procurement risk analysis,
• sequencing risk analysis,
• GC executive summary,
• self-audit.

The goal was to determine whether the system could identify not only what was shown, but what would affect estimating, procurement, subcontractor scope, schedule risk, and contractor exposure.

Scoring rubric

Runtime

Results

Evaluation notes

Perplexity Computer produced a very strong Test 3 response. In raw report completeness, it was especially strong in quantity-start reporting and missing-discipline organization. It identified major counts and structures such as:

• 54 scheduled rooms,
• 103 windows,
• 21 exterior doors,
• 28 interior doors,
• 24 appliances,
• 18 floor assembly types,
• 7 roof assembly types,
• 11 interior wall types,
• 23 exterior/foundation wall types.

Perplexity also performed well at labeling evidence status as confirmed, unclear, deferred, inferred, or not verified. It was strong at explaining that the set was architecture-only and that structural, MEP, electrical, civil, landscape, fire protection, low-voltage, and interior design packages were missing or deferred.

Foreman AI performed better in contractor-facing buyout risk and speed. Its output was more directly oriented toward the exact issues that create estimating exposure, RFIs, missed scope, procurement problems, and change orders. Foreman AI identified and prioritized issues such as:

• large custom exterior openings,
• unresolved window sill details,
• incomplete interior door hardware/detail fields,
• schedule/elevation conflicts,
• high-altitude glazing requirements,
• field-measure requirements,
• threshold and waterproofing risk,
• planter sill waterproofing,
• roof/envelope complexity,
• approximate roof crickets,
• roof penetration coordination,
• roof ice-melt and snow retention coordination,
• solar scope split,
• drainage gaps,
• generator coordination,
• heat pump equipment coordination,
• garage lift slab/power/height coordination,
• outdoor kitchen fuel conflicts,
• pool/spa ambiguity,
• WRB/flashing scope,
• missing civil/landscape/MEP/ID packages.

Foreman AI's response was slightly less complete than Perplexity in some quantity-start sections, but it was more useful as a preconstruction and buyout risk tool. The system produced a more contractor-oriented risk stack and completed the test approximately three minutes faster.

9. Final Score Table

10. System-Level Observations

Foreman AI strengths

Foreman AI was strongest in areas that map directly to construction workflows:

• faster completion,
• contractor/preconstruction framing,
• bid-risk identification,
• missed-cost exposure detection,
• RFI generation,
• subcontractor scope organization,
• large-opening risk identification,
• door/window issue spotting,
• schedule/elevation conflict detection,
• roof/envelope risk analysis,
• procurement and sequencing risk,
• conservative handling of unverified scope,
• construction-native interpretation of ambiguous items.

Foreman AI's outputs were more directly usable by general contractors, estimators, preconstruction managers, subcontractors, project managers, and owners evaluating bid risk.

Perplexity Computer strengths

Perplexity Computer was strongest in areas that map to general AI document reasoning:

• formal report structure,
• evidence labeling,
• vector text extraction,
• quantity-start organization,
• architecture-only scope caveats,
• missing-discipline analysis,
• broad document synthesis,
• clean taxonomy,
• strong written organization.

Perplexity Computer was the best general-purpose AI system tested against this drawing set.

11. Interpretation of Results

The benchmark suggests that Perplexity Computer is highly capable at construction-document reasoning when provided with a usable large architectural PDF. However, Foreman AI produced stronger construction-native outputs and completed the benchmark faster.

The main distinction is not simply model intelligence. The distinction is workflow orientation. Perplexity Computer behaved like a strong general AI document analyst. Foreman AI behaved more like a construction-specific preconstruction assistant.

This difference matters because construction plan intelligence is not only about reading drawings. In a real contractor workflow, plan intelligence must connect to downstream actions:

• takeoffs,
• estimates,
• budgets,
• RFIs,
• submittals,
• bid packages,
• subcontractor scopes,
• procurement logs,
• schedules,
• material tracking,
• change orders,
• invoices,
• project controls.

This benchmark only tested drawing analysis. It did not fully test Foreman AI's downstream workflow capabilities. Those capabilities are important because Foreman AI is designed to convert plan findings into operational construction workflows.

12. Product Scope Not Fully Tested

The benchmark focused on plan intelligence only. The following Foreman AI platform capabilities were not fully evaluated in this benchmark:

• Takeoff Studio,
• plan markup,
• count / linear / segment / area tools,
• full line-item budgets,
• estimated vs. actual cost tracking,
• over/under variance,
• material tracking,
• supplier tracking,
• change orders,
• invoices,
• RFIs,
• submittals,
• bid packages,
• subcontractor bid management,
• schedule and Gantt tools,
• project calendars,
• company cloud,
• workspace seats and roles,
• company brain / training on historical bid data,
• email lead and contact ingestion,
• JobTread integration,
• federal / SAM.gov job discovery,
• computer-use and external API bridge control.

Therefore, the benchmark should be understood as a comparison of plan-analysis performance, not a complete comparison of total platform capability.

13. Limitations

This benchmark has several limitations:

1. Single drawing set. The test used one 136-page architectural drawing set. A broader benchmark should include multiple project types, disciplines, and drawing qualities.
2. Manual scoring. The scoring was performed manually and includes evaluator judgment. Future testing should include blind scoring by multiple reviewers.
3. Architecture-only input. The drawing set was architectural. Future tests should include structural, civil, MEP, landscape, specifications, addenda, and scanned sheets.
4. Text-layer availability. The PDF appeared to have usable text/vector information, which benefits both systems. Future benchmarks should include scanned or low-quality plan sets.
5. Prompt sensitivity. Both systems may perform differently with different prompts, settings, model versions, or tool configurations.
6. Runtime variability. Runtime can vary based on infrastructure, load, tool availability, and retrieval behavior. It should be measured repeatedly in future benchmarks.
7. No independent ground-truth key. A future benchmark should include a prebuilt answer key for selected measurable items.
8. No takeoff validation. This benchmark did not validate quantity takeoff accuracy against manual measurement or CAD/BIM ground truth.
9. No downstream workflow execution scoring. The benchmark did not fully test conversion of plan findings into estimates, RFIs, bid packages, schedules, or budget updates.
10. Not a universal model ranking. These results apply to this construction-document benchmark. They should not be interpreted as a universal ranking of the systems across all tasks.

14. Recommended Next Benchmark Iteration

A more formal v2 benchmark should include:

1. Locked prompt text before testing.
2. Exact start and end timestamps for every run.
3. Raw output preservation.
4. Multiple independent construction reviewers.
5. Blind scoring where reviewers do not know which system produced which output.
6. Ground-truth answer keys for selected items.
7. Scanned PDF tests.
8. Poor-quality sheet tests.
9. Civil, structural, MEP, and specification sets.
10. Addenda and revision handling.
11. Quantity takeoff validation.
12. RFI usefulness scoring by real contractors.
13. Hallucination penalty scoring.
14. "Not enough information" scoring.
15. Downstream workflow tests.

Suggested downstream workflow tests:

• Create a full line-item estimate from the plan set.
• Generate a subcontractor bid package.
• Generate an RFI log with sheet references.
• Convert drawing gaps into exclusions and allowances.
• Create a preliminary schedule and Gantt chart.
• Identify long-lead procurement items.
• Create a material tracking log.
• Update a project budget from a scope change.
• Draft a change order from a discovered drawing conflict.
• Push project data into an external project-management system.