If you promise a 99.9% uptime SLA for your platform, you are committing to your service being available for all but about 43 minutes per month. Customers generally accept this metric because it feels incredibly close to 100%, a tolerable risk for a non-critical system.
Now, what if you had to tell those same customers that your product is only accurate 89%, 72%, or even 53% of the time? How confident would they be in your solution then?
These aren't imaginary figures. They are the accuracy ratings of various major AI models on complex, real-world tasks like generating code and long-context reasoning. We have entered an era where we are building products on a foundation that we would consider unacceptably unreliable in any other area of engineering.
The Two Worlds: Deterministic vs. Probabilistic
Traditional engineering is a practice rooted in logic, understanding, and problem-solving with tangible and reproducible results. When you ask a database for a user's record, you expect to get that exact record every single time. The system is deterministic.
Modern generative AI, however, is probabilistic. When you ask it a question, it is not retrieving a known answer; it is predicting the most likely sequence of words that should form a good answer based on its training data. This is an incredibly powerful tool for creativity and summarization, but it is not a system of truth.
When "Good Enough" Has Real Consequences
We are continuously seeing a trend to build products that are "good enough" using AI. They look impressive in demos, but under pressure, they can reveal themselves to be illusions based on imaginary sources of data. And companies are starting to pay the price.
A now-famous legal case involved an airline's customer service chatbot that confidently invented a bereavement fare policy that did not exist. When the airline refused to honor the AI's promise, a tribunal forced them to, ruling that the airline was responsible for the information provided by all of its agents, including the digital ones. The chatbot wasn't just wrong; its error created a tangible financial liability.
The Leadership Mandate: Navigating the Hype
This is the new challenge for technology leadership. The "easy" solutions that AI offers can sound too good to be true, and often they are. A leader's job is to remain cautious but open, and to rely on experts with the skills to recognize what is truly productive versus what is a convincing but high-risk simulation.
We must have the discipline to ask the hard questions:
- For this specific task, is a probabilistic answer acceptable, or do we require a deterministic one?
- What are the real-world consequences if the AI is confidently wrong?
- What human-in-the-loop processes and guardrails do we need to build around this technology to mitigate that risk?
Engineering is more than vibes. Building a durable, trustworthy product in the age of AI requires a deeper level of scrutiny and a return to the first principles of reliability and user trust.