Why the convincing POC never reaches production
The scenario is common: an AI prototype reaches 92% accuracy on test data, management gives the green light for production, and 6 months later the system is still in the testing phase or has never worked correctly under real conditions.
The recurring causes:
- The model was trained and validated on non-representative data. In industrial vision: images collected under laboratory conditions, without the lighting, temperature and vibration variations of the factory floor. In predictive maintenance: manually filtered data, failure periods that are too "clean".
- IT integration was not designed from the start. The OPC-UA connection to the PLC, the API to the MES, the bridge to the CMMS: all of it arrives at the end of the project as an afterthought. Discovering the network, industrial security and protocol-version constraints happens too late.
- There is no monitoring. A model deployed without monitoring of its performance metrics drifts silently. The distribution of production data changes (new supplier, material change, tooling wear), the model keeps working without a technical error, but its predictions gradually become wrong.
- The POC success criterion was the wrong one. Accuracy on test data is not the production criterion. The right criteria are: detection rate / false positive rate on the real production flow, latency under real conditions, availability over 30 days, operational IT integration.
Defining a viable MVP: the non-negotiable criteria
An MVP of an industrial AI system is not a model with good test metrics. It is a system that can run in production for 30 days without manual intervention and whose performance is measured under real conditions.
The non-negotiable criteria of a production MVP:
- Representative training data: collected under real conditions, across the effective range of variation (materials, ambient conditions, different shifts). No demonstration data.
- Operational IT integration: the model's decision is transmitted to the right system (PLC, CMMS, ERP) without manual intervention. No intermediate file export.
- Active performance metric monitoring: at least one performance metric measured automatically in production (conformity rate, number of alerts, false positive rate). With an alert if the metric leaves its nominal range.
- Documented edge-case handling: what happens when the model cannot decide (confidence score too low)? What happens if the AI system is unavailable? These scenarios must be handled before production, not after.
The minimal MLOps infrastructure to get started
You do not need a complete MLOps stack for a first deployment. But certain building blocks are essential from the start: adding them after the fact costs 3 to 5 times more.
Model versioning: mandatory from day 1
MLflow or DVC, a few lines of configuration. Every model in production must have a version identifier, the associated validation metrics, and versioned training data. Without this, when the model has to be retrained 6 months later, you start from scratch.
Monitoring the input distribution
Data drift is the main enemy of models in industrial production. A change of raw-material supplier, tooling wear, a modified machine setting: all changes that make the input data distribution drift without anyone flagging it explicitly.
Evidently (open-source) lets you monitor input distribution drift with minimal configuration. Even a simple email alert when the distribution of your main feature moves beyond 2 sigma from its training value is infinitely better than nothing.
Documented retraining pipeline
Not necessarily automated: but documented and testable. When the quality manager tells you the model is making more errors, you must be able to retrain the model on updated data in less than a day, not in 2 weeks.
The minimal pipeline: a script to collect the new labeled data + a reproducible training script + non-regression tests + deployment in a single script. You do not need Airflow for that: a well-documented Makefile is enough to start.
The adoption factor: often more limiting than the technical side
A technically high-performing AI system that is not adopted by operators or maintenance technicians is a project failure. The adoption factors to address from the design phase:
- The operator interface must be in the usual tool. Alerts in Teams or in SAP PM: not in a new web interface that no one checks. The path from alert to action must be as short as possible.
- The model must be explainable about its errors. When the system generates a false positive (an operator is convinced the part is good), you must be able to show what triggered the decision. Heatmaps and per-zone scores are essential to maintain trust.
- Operator feedback must feed the model. An operator who invalidates a system decision must be able to do so easily: and that correction must enter the retraining dataset. Otherwise the model does not improve and operators learn to ignore it.
Studio23
Have a validated POC to move into production?
Tell us about the state of the project, the integration constraints and the required SLAs. We analyze what it takes to reach production within 48 hours.