From Diagnostic Code to Repair Guidance

The Engagement

All In On Data partnered with a US-based automotive diagnostics manufacturer to assess whether generative AI could power the next generation of their consumer-facing diagnostic product line. The client's OBD scanners produce industry-standard diagnostic trouble codes, but converting a raw code into clear, trustworthy, step-by-step repair guidance — calibrated to the user's mechanical experience level — remained a significant unsolved product problem.

The engagement paired a structured AI Discovery phase with an executed Proof of Concept and ran over approximately twelve weeks. Discovery inventoried candidate AI use cases across the client's product roadmap, plotted each on a benefit-versus-attainability matrix, and identified the technical proof points that would need to be confirmed before any production commitment. Those proof points defined the POC scope.

The Approach

The Proof of Concept focused on the most consequential question: can a language-model system reliably synthesize vehicle-specific diagnostic and repair instructions from heterogeneous public content? We designed and built a multi-stage agentic pipeline that retrieves relevant material from sources including video transcripts and archived technical documentation, extracts jargon and key terms per source, resolves synonymy across sources, and synthesizes a single cited instruction set adjusted to the user's stated experience level. The system was deployed to a working demo environment with an embedded feedback mechanism, so client stakeholders could mark up generated content and inform iterative refinement.

Findings and Path Forward

Structured testing confirmed the core proof points: useful novice, intermediate, and advanced diagnostic and repair instructions could be generated reliably enough to support productization. The work also surfaced the constraints — generation latency, output variability across runs, and occasional loss of small-but-important details — that would have to be addressed in a production build.

Our final recommendations laid out a three-phase development plan: an agentic data architecture on AWS suited to the breadth of source formats expected; the fine-tuning of a smaller open-weight model to achieve foundation-model-level performance at materially lower inference cost and latency; and a structured feedback program combining AI self-critique, community input, and internal human-in-the-loop evaluation. We additionally recommended building a dedicated benchmark of roughly 150–200 client-validated request/instruction pairs to anchor performance measurement as the platform matures.

The engagement delivered both a working artifact and a defensible roadmap — a concrete demonstration that the capability is achievable, paired with the architectural and data-strategy decisions needed to take it from prototype to production.