Production intelligence

Quality and learning statistics

Updated Oct 23, 2026, 8:17 AM

Accuracy is measured through production decisions

The primary quality measure is production acceptance—not the model's own confidence. A true geometric precision score will require labeled expected layouts for direct piece-by-piece comparison.

Approved references 39 11 homepage approval records · 2 approved without placement

Needs production review 2 Oldest is 81 days

AI plan acceptance 81.8% 9 approved of 11 reviewed

First-pass interpretation 70.7% 29 approved without interpretation edits

Reference pipeline

From synced ticket to trusted example

14.4% of tickets are approved references

1 270 Synced tickets

2 43 Interpreted

3 2 Needs review

4 39 Retrievable references

Interpretation coverage 15.9%

Verified AcroSketch mapping 82.6%

Average approval time 2.8 days From stored plan to approval

Planning quality

AI-generated plan decisions

11 reviewed generations

9 Approved

2 Needs changes

Production acceptance 81.8%

Passed stock-fit validation 100%

Only reviewed OpenAI-generated layouts are counted. Built-in plans and exact approved references are excluded.

Interpretation quality

Ticket review outcomes

41 approved interpretations

Approved without interpretation correction 70.7%

Required interpretation correction 31%

AI-reported extraction confidence 85.3% Diagnostic only—not a measured accuracy score.

Six-month activity

Approvals and planning decisions

Ticket references AI plan approved Needs changes

May 0

Jun 0

Jul 2

Aug 19

Sep 29

Oct 2

Production feedback

Why AI plans need changes

Production ready2 decisions

100%

Model monitoring

Acceptance by model

ModelReviewedAccepted

gpt-5.6-sol 11 81.8%

Metric definitions

How to read this page

AI plan acceptance

Approved AI generations divided by all AI generations reviewed by production.

First-pass interpretation

Approved ticket interpretations with no recorded material, dimension, or note correction.

Approved reference

A retrievable ticket plan with verified positioned pieces. Interpretation alone is insufficient.

Accuracy limitation

Acceptance measures usefulness. Exact precision requires an expected layout and geometric comparison.