A Field Guide to Agentic AI: Service Edition
Welcome to the second study of that most fascinating specimen: the autonomous agent in its natural enterprise habitat. I’m your guide, John J.C. Cosgrove, and today we venture into particularly intriguing territory – the customer service ecosystem, where these remarkable creatures begin to reshape the very landscape they inhabit.
The Thank You Moment
In the vast expanse of enterprise technology, few subjects have captured attention quite like service design. Having spent years in the field – quite literally, from the trenches of outbound call centers to the rarified air of service strategy – I’ve developed what one might call an anthropologist’s fascination with this particular corner of our digital world. Though unlike most anthropologists, I’ve had the distinct pleasure of being both observer and specimen.
What’s truly remarkable about the service professional species is their defining characteristic – an unwavering drive to make a difference. In my years of observation, I’ve noted this trait consistently across subspecies, from front-line support agent to the seasoned service architect. It’s not about metrics or KPIs (though these creatures are certainly measured by such things) – it’s about that moment of validation, that simple “thank you” that signals a successful interaction.
For service professionals, it’s all about the “thank you.”
It’s a deeply human motivator, and throughout these posts, you’ll find that I believe deep human empathy is the key not just to a successful agentic deployment, but to a safe one. So we’re going to pay attention to “thank you”.
But first… for something completely different…
Chatting with a Goldfish
Every time you engage with an LLM – be it Anthropic’s offerings or OpenAI’s latest – you essentially start fresh. They’re goldfish. Brilliant, sophisticated, incredibly well-read goldfish – but goldfish nonetheless. The system retains nothing from your previous interactions. This isn’t a bug – it’s a deliberate design choice that makes these systems scalable and secure. Though it makes one wonder about the existential implications of being brilliant but perpetually amnesiac.
What you’re experiencing in modern chat interfaces is like a sophisticated stage play. That chat history you see? It’s not the LLM’s memory – it’s our sleight of hand. With each interaction, we gather your conversational history and carefully package it with your latest input. This combined bundle of context is what we feed to our amnesiac but brilliant friend.
This is why we call it the “context window” – space we have before things start falling off the edge, like trying to fit an entire library onto a shelf. And whatever we put in there is the context that the LLM will use to make decisions. When chatbots seem to forget things, it’s not because they’ve grown bored of the conversation – it’s because we’ve had to remove older books to make room for new ones, like a librarian with a particularly aggressive approach to management.
The RAG Revolution
What we’re doing is taking that principle of context injection beyond conversation history. We’re giving specimens access to vast knowledge repositories – documentation, databases, customer records – anything relevant to their current task. You’ve heard of the thought experiment about giving a thousand monkeys typewriters to write Shakespeare? Well, instead we’re giving billions Wikipedia access to help Grandad reset Netflix.
Field Note: I’m not sure what the ethics committee would think was the worst of those two experiments…
This idea is Retrieval Augmented Generation (RAG) – the natural evolution of that genius idea to make chatbots. Every operational LLM chatbot uses RAG – yes really, come fight me. If we can pump conversation history into every iteration with the LLM, then we can pump anything relevant to the interaction into every iteration. More context, more detail, more associations means better attention paid to things and THAT means better outcomes (for reasons we’ll explore in other blogs).
Think of it as upgrading from a clever parrot that mimics human speech, to a research assistant who digs through archives, runs experiments, and returns insights you hadn’t considered. The former is impressive but limited, like a calculator that only adds. The latter represents a fundamental shift – it’s the difference between having someone who reads cookbooks and someone who actually cooks.
Field Note: I’ve witnessed countless “eureka” moments when developers grasp these implications. While it might seem like your bubble is bursting, it’s essential to understand what’s happening when you use an LLM. I’ve said repeatedly: telling me you’ve used ChatGPT so you understand enterprise AI is like telling me you’ve used Word, so now you’re an Azure Engineer… don’t be that person…
It’s The Service, Stupid
This is where service environments become fascinating as a habitat for these emerging creatures we call agents. Consider the pattern of a service interaction:
- Understanding customer needs (retrieval from conversation)
- Looking up information (retrieval from knowledge bases)
- Planning next steps (planning and reasoning)
- Taking action (tool usage)
- Confirming results (communication)
This is what Agentforce is designed to do, with a crucial difference: it handles this loop autonomously, making decisions about what to retrieve and which tools to use based on the context.
Like any great service member, Agentforce isn’t following scripts. It uses the React pattern – a way of thinking that mirrors how humans solve problems:
- Observe: What’s the situation?
- Think: What are options?
- Act: Take appropriate action
- Reflect: Did it work?
This is where the “agent” in Agentforce comes from. It’s not about connecting an LLM to tools and data – it’s about creating a system that reasons about using capabilities effectively.
Field Note: Lost? Wondering where this React-reasoning stuff comes from? Fear not reader, you simply arrived in the wrong thread! Check the first blog for a primer. Or wing it – most people in gen AI are winging it anyway…
And what does RAG do for a service agent? It does THE THING from The Matrix movies. Suddenly, mid call it knows Kung-Fu.
If I had that chip in my brain when I was on the phones in a call center, I would have paid any price.
And that’s why service centers and Customer Service Management are the tip of the spear for generative AI. It’s not because it’s obvious to automate, nor because it’s the path to economic returns. Rather, it’s because excellent customer service organisations worldwide share something fundamental.
Yes, they have usual traits – an obsession with customer experience, defined processes, and metrics that matter. But the secret sauce, at the foundation, is knowledge. Organisations that achieve scalable excellence in customer experience have realised their superpower lies in how they manage knowledge.
Every service center, regardless of modality – voice, chat, or interaction – faces a core challenge: how to equip people with knowledge they need to solve problems, handle crises, resolve issues. The tools and techniques they’ve developed to capture, organise, and deploy this knowledge aren’t nice-to-have features – they’re survival mechanisms.
This is why service centers are our perfect proving ground. They have what RAG systems need: curated, battle-tested repositories of knowledge. They’ve spent decades perfecting the art of capturing not just what worked, but why it worked, and under what circumstances it might work again.
When we connect knowledge repositories to agentic systems, something extraordinary happens. We’re not creating another automation tool – we’re building something that understands and leverages the wisdom of thousands of service interactions.
What Benioff Saw?
These systems thrive on knowledge repositories – that’s what retrieval augmented generation is about. Salesforce got ahead of this, providing ways to connect our LLM layer inside Agentforce to knowledge sources, not just Salesforce’s own.
I believe Salesforce leads the competition in providing a simple way to do enterprise RAG. Their solution enables trusted, secure retrieval augmented generation on any source through Prompt Builder – a tool you’ll hear about often.
The Prompt Builder is integral to Agentforce deployment. It gives us a way to construct prompts by dragging retrieval components like building a mail merge, working with Flow and Data Cloud. It is my favourite enterprise tool.
Field Note: The author becomes animated discussing Prompt Builder, drawing diagrams that make sense to no one. Several whiteboards have filed for restraining orders.
What makes this tool fascinating is how it constructs prompts by drag-dropping retrieval concepts. Imagine the difference between explaining your life story to each person you meet, versus having a curated introduction that adapts. The agent doesn’t need twenty questions to understand you – it knows, and knows quickly and that knowledge becomes woven into responses, like someone who’s read your autobiography but politely omits the embarrassing parts.
When people call RAG a ‘hack’, they mean we ‘googled’ it. We use tricks from Google Search to take what humans type, run semantic search against knowledge, and things we get back – while not perfect – are similar to retraining the model with embedded information.
Very neat.
Field Note: For technical boffins, yes I know about cosine similarity scores versus collapsing logits on GPT decoder passes, but this explanation works and I’m keeping it. Get off my lawn!
RAG allows us to ground LLM responses – a word you’ll hear with Agentforce, which has a grounding layer to help. The LLM biases toward responding with stuff we gave it because that’s how the context window works. We had a way to boost relevance and trustworthiness of LLM responses; we just needed trusted knowledge.
Having worked with Salesforce for a decade, I’m proud of the product teams and grateful for my time there. I’ll embarrass one – Reinier Von Leuken, who took vector databases – technology powering semantic search – and made it production-ready in Data Cloud that you deploy with clicks. It’s battle-tested in production today. It WASN’T ON THE PLAN at the start of 2024. If artists ship Renier, your team are Rembrandts <3.
This matters because without it, organised knowledge wouldn’t have a home. Now we have the fastest path to get enterprise RAG into production, securely.
And that makes Agentforce special – not another GenAI solution, it exists on Salesforce because they solved hard problems: security, scalability, multi-tenancy, distribution. These weren’t new problems for GenAI – they were prerequisites Salesforce had. Even with that foundation, they innovated at speed to add vector search. You can boot Google-style search inside your org for RAG – that’s why Agentforce isn’t going to change service – it’s already here.
Thinking in Loops
Let’s go human. What motivates people in service isn’t a number. It’s not KPIs. It’s not leaderboards, though they’re fond of those. It’s knowing they made a difference – hearing that customer say “thank you.” Sometimes we know the issue isn’t resolved how we’d like, but genuine gratitude makes it worthwhile.
That’s a bespoke outcome. If you like hashtags: it’s a “#hyper-personalised_experiential_inflection_point”*.
Field Note: No, it isn’t.
We learn through interactions, patterns of resolution, but no resolutions are identical. Because each person, time, and context differs.
That makes service automation hard: combinations. The combinatorial complexity (use at parties!) is vast when you see it emerge, and traditional automation staggers. Under those permutations, paths, exceptions, complaints – we never get enough code coverage. We try our best, but humans throw wrenches.
Not so with agentic flow. An agent gets a simple state. It can evaluate if jobs are done. If not, it keeps trying to understand why. It asks questions. It checks your happiness. “Did you get that email? Let me check. I see the issue – I’ll update it. Now? Different email? No problem. Please confirm receipt. Got it? Sorry about that. Welcome!”
That iterative conversation, adapting to the environment – that’s hard with automation. It’s trivial for Agentforce. I coach people: “don’t overthink it”. The agent is smarter than you expect.
Keeping loops simple,keeping success criteria simple, helps us use the power in agentic patterns. Trusting it to evaluate and try solutions is the way to leverage this tech.
It’s tempting to map every process flow (sometimes that’s right), but I advise caution because that’s old tech thinking. To really understand what this new tech could do, we’re going to have to challenge ourselves with some pretty incredible new ideas for what automation could look like.
Pretty Incredible New Idea
That’s what I’ll close with. If this blog claims we’ll revolutionise service centers, I want to put meat behind that statement.
No hype. No hashtag bingo. I want to speak to people who’ve been on those phones, who know both sides of service. Sure, I’ll claim zero wait time and stick to it – because it will happen.
But let’s go further.
Instead of just describing SLA breaches, containment breaches, rules for escalation and rules for bypassing – all of those are still going to be helpful – but if we really think about how best practice agentic systems simply evaluate the environment and how their actions have changed the state AND then use that to decide when the job is done… Well, what if we achieve a world where in any service center, there’s really only ONE standard of excellence for our agentic teams, our agentic workforce?
What if the definition of done is “thank you”?
Imagine what could be possible if we could trust these little machines, infused with a deep understanding of our empathy as humans, our need as customers and our commitment as businesses to not just race to hit an SLA or escalate out if they can’t, but instead fight with the tenacity of your best call center operative because damn it they want to make this right.
What if our little agents were driven to hear a human say “thank you” because that was the ultimate sign of a fulfilling outcome? Is that so crazy? How can it be?
That’s what it is for us.
Imagine if service centers equipped to support mental health, disadvantaged youth, domestic violence victims and more never gave up because of exhausted staff and restricted budgets, but instead advocated tirelessly to achieve the right outcomes. We don’t need to water down our expectations – while we might miss sometimes and escalation to humans will always matter, we’re letting our fears hold us back. We’re so worried about people talking to “soulless machines” instead of humans that we’re missing the real opportunity: there’s ALWAYS A HUMAN… on the other side of the call, who needs help.
Getting them that help is more important than debating the nature of how it’s delivered.
There’s enormous potential to deliver urgent help to those who need it most – the weak, sick, vulnerable, and scared. We can augment our best service people, connecting them with those who truly need human interaction, while providing immediate assistance to others through responsive automated systems. This allows us to deliver the right type of help, at the right time, to everyone who needs it.
The agents we’re already building are NOT cold and heartless, not because they’re sentient or alive, but because they run in a DEEPLY HUMAN CONTEXT – we fill their context window with the knowledge, skills, processes and tools that we as humans have demanded for each other. And OUR empathy, woven through all of that, shines through them.
I’m really genuine about what I’ve said, and I hope you can read that in my words. I am not an apocalypse doomer on what this is going to do for service across this planet. I’ve been from a very young age on the front line of the realities of service, and I don’t need to tell you all – it’s not a walk in the park. It’s not a bed of daisies. In fact, it’s one of the hardest, most thankless, most frustrating jobs you can do.
So why do people keep doing it? It’s not just because of the money. There are people who do it because of that drive to know they made a difference. That’s the service industry I love. That’s the service industry I’m proud to have been a part of and to have helped my entire life. And that, for me, is the first tangible example of how we’re going to use this technology – not to create some dystopian future where we only get help from machines, but to use machines to get more help. Because I think we can all agree: every aspect of our society, in every industry and every walk of life, could do with that right now.
And perhaps that’s the most remarkable observation of all – that in our quest to create more intelligent machines, we might just end up building a more empathetic world. Though I suspect the machines might still need help understanding why humans insist on trying to solve problems by turning them off and on again. I wonder how all the IT help desk agents will feel about that?
See you in the next deep dive.
-JC
