Why precision and governance are essential in Agentic AI
At the end of the 1990s, the founders of Google tried to sell their algorithm to Yahoo for a comparatively modest sum (according to various accounts, between $1 million and $ 2 million). The offer was refused and labeled a “marginal improvement” on the incumbent technology. Today, that moment stands as a pivotal milestone — a reminder that what seemed like a “marginal improvement” ultimately reshaped the future of search as we know it.
The technology race to deploy agentic AI systems could lead us to repeat the mistake: focusing on ever larger models or on API‑connectivity, and losing sight of what can truly move the needle — contextual precision, system governance and the ability to operate effectively in real‐world environments. That’s where LLMOps comes in.
What is LLMOps and why does it matter?
LLMOps spans the full lifecycle of an agentic AI solution in enterprise settings:
- Development & validation: design of use cases, pilot programs, evaluation and iteration.
- Deployment & operations: implementation in production environments, monitoring and alert management.
- Measurement & continuous improvement: analysis of metrics, prompt adjustment, redefinition of objectives and cost optimization.
Five Strategic Levers for Effective Context Management
Organizations must master five key dimensions to implement context management effectively:
- Knowledge base: GenAI PoCs typically move fast because they leverage a foundational model from the start, eliminating the need to build core capabilities from scratch. Incorporating business‐specific information, however, requires data modeling in vector or graph structures — namely Retrieval‑Augmented Generation (RAG). Structuring, chunking, embeddings and repository design are indispensable when handling large volumes of data.
- Security & reliability: Hallucinations or off-target responses that breach predefined tolerance thresholds can compromise an entire deployment. It is essential to implement guardrails based on internal policies, operational limits, traceability and regulatory compliance.
- Business rules: A recurring question often arises when presenting agentic use cases: where exactly are the business rules embedded? They often reside in the prompts. That’s why templates for prompts and prompt management techniques are essential — treating prompts as a strategic asset: versioning, auditability, adaptability to continuous learning.
- Memory: In the early GenAI use cases — typically assistants — memory management was limited to the short term. In the agentic world, where multiple agents engage in a journey, long‑term memory management is required to ensure functional continuity.
- Intelligence: Choosing the right LLM for each domain — whether by considering SLMs, exploring VideoLMs, applying fine-tuning, or leveraging open or private AI models — is essential to finding the optimal fit for your business context.
What role do low‑code and no‑code platforms play?
In my view, low‑code and no‑code platforms have an increasingly valuable role in this ecosystem.
They are ideal for horizontal use cases like productivity assistants or task automation. However, when scaling into more complex environments, it becomes necessary to ensure:
- Data robustness
- Knowledge governance
- Security
- Precision in critical processes
Strategic recommendation
The future of agentic AI will not be decided by whoever connects a model to an API the fastest — but by those who master the discipline of LLMOps. My recommendation for CIOs, CDOs and digital transformation leaders:
- Approach LLMOps as an emerging organizational discipline.
- Build specialized teams.
- Apply proprietary methodologies and accelerators.
Only then is it possible to develop an agent that aligns with a well-defined business case, undergoes rigorous validation, is deployed with control, runs reliably in production, and is assessed using meaningful impact metrics.
Conclusion
Overlooking a so-called “marginal improvement” can cost your organization the opportunity to lead a global transformation. The true advantage lies in mastering context — not merely connecting models.
Learn more about NTT DATA and Agentic AI.