Are AI agents just glorified chatbots? The next-gen evolution of robotic process automation (RPA)? Tools everyone in your company will build?
In a recent Dataiku “Let’s Talk Agents” session, Mythbusters: AI Agents Edition, Emma Irwin, Director of Sales Engineering at Dataiku, and guest speaker Rowan Curran, Principal Analyst at Forrester, debunked the top myths around AI agents and explored their real value in enterprise applications. Here's a look at what they had to say.
Myth #1: AI Agents Are Just Chatbots
Right out of the gate, Emma Irwin tackled the misconception that AI agents are merely chatbots. According to Emma, it's easy to understand why people compare the two. Chatbots are familiar. You ask a question, they give an answer.
But AI agents go far beyond that, with the ability to execute multi-step processes, make decisions, reference previous steps, and interact with tools or systems.
Rowan Curran expanded on this: “AI agents don’t equal chatbots in the same way that early Amazon.com didn’t equal a graphical point-and-click interface. The interface might be the same, but the power behind it is vastly different.”
He noted that modern AI agents can augment workflows and trigger actions across systems, often without a user-facing interface at all.
It's not just about that chat interface. There's different modalities of interacting with AI agents — maybe it’s a notification, maybe it’s a data visualization. It doesn’t have to be a chat.
— Rowan Curran, Principal Analyst at Forrester
Myth #2: AI Agents Are Just Next-Gen RPA
While agents and RPA both drive automation, conflating the two limits our understanding of what AI agents can do.
RPA systems are about automating structured, rule-based tasks that imitate human clicks or actions. They need strict instructions and are best for high-volume, repetitive processes — especially with legacy systems.
In contrast, agentic AI introduces decision-making capabilities. Agents can dynamically evaluate options, pull in external context, and even interact with APIs, predictive models, or web searches.
Rowan pointed out that agentic AI combines deterministic logic with foundation models, enabling flexibility and adaptability: “It opens up a broader set of capabilities for automation workflows,” he said. But he cautioned that governance and reliability challenges still remain.
An RPA bot might copy and paste data. An AI agent could decide which data to use, how to use it, and where to send it — all autonomously.
— Emma Irwin, Director of Sales Engineering at Dataiku
Myth #3: AI Agents Will Solve Every Problem
Could a real estate agent be replaced by an AI agent? That question sparked a deeper discussion around when AI agents are appropriate — and when they’re not.
Emma recounted a Dataiku workshop where attendees brainstormed tasks real estate agents handle: scheduling, pricing, negotiating, listing homes. Some of these are great candidates for AI augmentation. Others, not so much.
Just because something can be done by an agent doesn’t mean it should. There’s still a lot of power in traditional ML or deterministic systems for structured tasks.
An agent is only as powerful as the tools it has access to. Often, a traditional ML model or even a simple automation can do the job just fine.
— Emma Irwin, Director of Sales Engineering at Dataiku
Rowan added that excitement from the developer and hobbyist community can mislead companies into thinking agents are the solution to everything. “We haven’t yet translated many of these cool experiments into enterprise-ready, robust applications,” he said.
Myth #4: Everyone in Your Organization Will Build Agents
No-code and low-code platforms are making AI development more accessible, but Rowan cautioned against the idea that everyone should — or will — be building agents.
It's a governance nightmare. What we need are cross-functional teams: app developers, data scientists, and business users working together. Everyone doesn’t need to build, but everyone should benefit.
— Rowan Curran, Principal Analyst at Forrester
Emma emphasized the importance of accountability. If something goes wrong with an agent, who’s responsible? Who fixes it? Who monitors its performance? She noted that widespread adoption hinges on trust, usability, and proper enablement — not mass deployment of DIY agents. According to Emma, people may build agents in their free time, but that doesn’t mean they’re ready for enterprise use.
Myth #5: AI Agents Are Where Every Company Should Start With AI
Not so fast. While AI agents are exciting, Emma recommended a crawl-walk-run approach.
Agents can be a good mental model to uncover business problems, but they’re not always the end solution. Sometimes, structured analytics or traditional automation will serve better.
Rowan agreed that trying to jump straight to agents is like jumping to advanced computer vision in 2015. Most companies still need to get their data house in order first.
He pointed out that organizations can also adopt prebuilt agentic solutions focused on tightly scoped domains like customer support or sales, which often provide a faster path to ROI than trying to build from scratch.
Myth #6: Measuring the Cost and ROI of Agents Is Easy
Spoiler: It’s not.
There are high upfront costs, unclear success metrics, and challenges with tracking usage and impact across teams, Emma explained. Even attributing value to a specific agent becomes tricky once it’s embedded across systems and processes.
Rowan discussed the technical complexity of optimizing agent cost: “You need to know how your model architecture affects performance and compute. Even slight changes in prompts can impact token usage and cost.”
Ultimately, AI agents deliver value in two main ways: efficiency gains and revenue generation. However, capturing that value requires deliberate tracking, governance, and iteration.
Real-World Use Cases
To ground the conversation, Emma and Rowan shared real examples of AI agents in action:
- Technical Support Agents: One Dataiku customer is deploying an agent that interacts with users through ServiceNow. It assesses tickets, replies when possible, escalates complex issues, and flags major incidents — bringing intelligence to an already automated process.
- Software Architecture Assistants: Rowan highlighted agents helping developers draft design documents. “They’re tightly constrained and focused on knowledge retrieval, not complex back-end actions — but they still deliver huge value,” he said.
- HR Automation: Some companies are using agents in HR to handle PTO requests through natural language. The agent understands intent, then acts within a deterministic framework to ensure compliance and accuracy.
Final Thoughts: A Tool, Not a Silver Bullet
In closing, Rowan reminded attendees that AI agents are “a tool in the toolbox, not a hammer for infinite problems.” As with any technology, success comes from thoughtful application, solid foundations, and strong governance.
Emma echoed that sentiment: “What matters isn’t who builds the agents — it’s who uses them, and whether they trust them.”