When Not to Use an LLM: The Case for Deterministic Logic and Rules Engines

Outline

• Introduction: The LLM Hype Cycle and Production Reality
• The Fundamental Challenge: LLM Non-Determinism
• When Determinism is Non-Negotiable: Core Product Logic
• Rules Engines: A Proven Alternative for Structured Decisions
• The Gap Between Agent Demos and Production Systems
• Measuring Success: Beyond LLM Output Quality
• Strategic Application: Knowing When to Build, Buy, or Skip the LLM
• Conclusion: Thoughtful AI Integration for Product Engineers

Large Language Models (LLMs) have transformed what is possible in software, from content generation to complex reasoning tasks. However, as product engineers, our focus must remain on shipping reliable, maintainable, and cost-effective solutions. The enthusiasm for LLMs sometimes overshadows their inherent limitations, particularly when deterministic outcomes are critical.

The core challenge with LLMs in many production scenarios is their non-deterministic nature. Given the same prompt, an LLM might produce slightly different outputs each time. This variability stems from their probabilistic generation process, which is excellent for creativity but problematic for tasks requiring exact, repeatable results. For instance, if a system needs to apply a specific discount based on defined criteria, a probabilistic model introduces unacceptable risk.

Tap to enlarge

Consider scenarios within a product platform where exactness is paramount. Payment processing, user permissions, inventory management, or compliance checks cannot tolerate ambiguity. If a user's access to a feature depends on a specific subscription tier, an LLM deciding this could lead to incorrect authorisations or billing errors. These are areas where a clear, auditable decision path is essential. For more on building robust platforms, see our insights on [product platforms in Manchester](/product-platforms-manchester).

For these critical, deterministic tasks, rules engines offer a robust and proven alternative. A rules engine executes a set of predefined rules against input data, always yielding the same output for the same input. This predictability is invaluable for business logic, fraud detection, or workflow automation. Tools like Drools or AWS Step Functions with conditional logic provide transparent, auditable, and performant ways to manage complex decision trees without the overhead or uncertainty of an LLM. For example, a system determining shipping eligibility based on location and item weight is far better served by explicit rules.

The concept of 'LLM agents' has gained significant traction in demos, showcasing models that can plan, execute tools, and self-correct. While impressive, the leap from a proof-of-concept to a production-grade agent is substantial. Current agent frameworks often struggle with reliability, error handling, and grounding in real-world constraints. Their performance can degrade unexpectedly with slight changes in environment or task, making them difficult to trust in high-stakes applications. Production systems require consistent performance and defined failure modes, which current agent architectures often lack. This is a key area where careful evaluation is needed, as discussed on our [applied AI page](/applied-ai-manchester).

Tap to enlarge

When evaluating any AI component, especially LLMs, it is crucial to move beyond subjective assessments. For deterministic tasks, the evaluation is straightforward: does it always produce the correct output? For LLM-powered components, metrics like task success rate, grounding accuracy, and refusal rates become important. It is also vital to measure latency and cost per successful task. OpenAI's Evals framework provides a structured approach to testing LLM performance against specific criteria, moving beyond anecdotal observation to quantifiable results [1]. Similarly, Google's Responsible AI Toolkit offers tools for understanding model behaviour and identifying potential issues [2].

Before integrating an LLM, product engineers should ask: Is this task inherently probabilistic or deterministic? Does it require creativity or strict adherence to rules? What are the consequences of an incorrect or non-deterministic output? Often, a hybrid approach works best, using LLMs for creative or ambiguous tasks like content summarisation, and rules engines for critical, structured decisions. Understanding these trade-offs is central to building effective and reliable systems. This strategic thinking is part of what we offer when we [work with clients](/work-with-me).

In conclusion, while LLMs are powerful, they are not a silver bullet. Thoughtful product engineering involves selecting the right tool for the job. For tasks demanding precision, auditability, and predictable performance, traditional deterministic logic and rules engines remain superior. As LLM technology evolves, particularly in agentic capabilities, rigorous evaluation and a clear understanding of their limitations will be key to successful and responsible integration into production platforms. Always measure, always evaluate, and always prioritise system reliability.

References

• OpenAI Evals: A framework for evaluating LLMs and LLM systems.
• Challenges in Deploying LLM-based Agents: A research paper discussing practical challenges and limitations of LLM agents.