In-Ear Insights: Setting up Agentic AI For Success Part 2, Employee Handbooks

In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss the critical transition from vague prompting to structured management for agentic AI. You’ll learn how to apply human management principles like employee handbooks and standard operating procedures to your AI workflows. You’ll discover why the 5P framework prevents your agents from wasting thousands of tokens and expensive compute power on guesswork. You’ll master the art of defining the “definition of done” to ensure your agents deliver perfect results every single time. You’ll understand why your current delegation style might hinder both your human team and your machine agents.

00:00 – Introduction
03:15 – Why agentic AI fails without structure
07:45 – The importance of defining the “definition of done”
12:20 – Managing humans vs. managing machines
16:45 – Applying the 5P framework to your prompts
22:10 – Call to action

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!
• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn: In this week’s In-Ear Insights, part two of our series on how we work well with agentic AI. In part one, we talked about decomposing job descriptions and being very specific about here is what you do and here is how you do it. But at a company, you do not just have a job description; you also have things like an employee handbook, and that can be for the company as a whole. And then, if it is a well-run team like the teams that Katie runs, you probably also have a book of standard operating procedures for that team for the specific roles.

And yet, when people prompt agentic AI systems like OpenClaw, Noose, Hermes Agent, Paperclip, Claude Code, or Claude Cowork, they do not provide any of this. They just say, “Hey, go do the thing,” and then wonder why it can never replicate its results. It is random every single time. So, Katie, when you are setting up a team for success in terms of an employee handbook and a bible of standard operating procedures, what is in it? What do you do as a great manager to set up a team for success?

Katie Robbert: The first thing I do is set expectations. I recall, Chris, when you hired me onto the agency to be the manager of your marketing technology team, one of the first things I did was I sent a note to the entire team, including you, HR, and your boss, saying these are my expectations of you as a team. Those expectations included things like open communication, honest feedback, collaboration, and conflict resolution. I said my expectations of you as a team are mutual respect. I want to earn your respect, and in turn, I want you to earn my respect. Here is the pathway forward.

What I realized is that those expectations had never been set for the team. They were all kind of wondering what the heck is this? Who is this person coming in making these demands of me? Because from day one when the team was formed, and I was not there for that, there were no expectations of success, professionalism, or quality of work. I had to treat the team like it was day one. The first few months with the team were really rocky because I was introducing a structure that did not previously exist. I was the outsider coming in saying this is what needs to happen in order for this team to be successful. It was really all in the spirit of making the team more successful and making everyone on the team more successful. At the time, it was seen as something very combative and something to feel defensive about.

If I had to do it over again, I would do it the exact same way because that team was struggling for any kind of consistency and any kind of structure. To your question about what I would do first, that is what I would do. I would establish structure, consistency, and feedback loops, because these are humans who are getting paid to do a job. In order to ensure the job is getting done, you have to have a way to say this is how the job is supposed to be done. This is my expectation of the job. This is how you know you are successful at the job. Those are basic tenets of any employment, and so many companies fail at those three basic things.

That is where I would start. If we had an employee handbook and standard operating procedures, I would say this is the job, this is how the job is to be done, and this is my expectation of success. But then, as you bring more people in, those things evolve and they change. That is the other thing that companies fail to notice is that when you bring in someone with different experience or a different cultural background, they are going to have done things a different way and they are going to bring in a new perspective. You need this to be a living, breathing thing, not a static “here is my SOP.” That is how it is only ever going to get done, which means it does not evolve with the people, it does not evolve as processes change, and it does not evolve as platforms change.

There are a few layers in there. To recap: one, you have to set expectations about what the job is, how it gets done, and how it is successful. Then, you have to have open conversations about what the standard operating procedure looks like today with what we know and what we have. But let’s evaluate it as people, process, and platform change—maybe quarterly, maybe yearly—so that everybody can learn and grow but also stay aligned consistently.

Christopher S. Penn: That is literally the recipe for success with agentic AI. You should have a document of first principles; it is almost your ethics. This is what is inbounds, and this is what is out of bounds. We have talked in the past about the 12 first principles that I use with agents, things like never defer necessary work. There is no such thing as putting it off for later or “out of scope.” You see a problem, you fix it in the moment. Stop trying to reinvent the wheel. If any solution exists, look for it first before you go reinvent the wheel because it is a waste of everybody’s time.

The second thing you mentioned is templates. Here is how to do the task, and this is the container that it goes in. You will save yourself hours and thousands of API calls and tokens if you provide the machine with the templates. This is how to do the task; fill in this thing. Third is the definition of done. If you are writing code, 100% test coverage is mandatory. Not 96%, not “close enough.” 100% test coverage is expected, and 100% passing is expected. That is the definition of done. Until you hit those numbers, you are not done.

Today’s agent systems, when you give them those details, can say, “Am I done?” and if you say “98%,” they respond, “Nope, not done,” and they will keep on trying. Xiaomi just released its new model, Mimo M 2.5, which has a 72-hour, several-thousand-tool-calling run horizon, which means that it can keep working on a task for up to 3 days on its own if you give it a definition of done. Otherwise, it just says, “I am done” as fast as possible. These lessons that you are talking about, Katie, that are important for humans are essential for machines.

Katie Robbert: Think about basic things we do in everyday life. Let’s say you are driving to the grocery store and you need to get some milk. I drive to the grocery store—am I done? No, I do not have the milk yet. I go into the store and I walk down the milk aisle—am I done? No, I still do not have the milk yet. I pick up the milk in my hand—am I done? I don’t know, how did I define done? Do I have to pay for the milk? Do I have to bring it home? If I just said, “Go to the store and get some milk,” the large language model would say, “Okay, I have the milk in my hand; I am done.”

You, as the human, inherently know you still have to pay for the milk, put it in your car, bring it all the way home, and put it in your refrigerator. Those four extra steps, if not explicitly stated, make “go to the store, get the milk, pay for it, bring it home, and put it in the fridge” very different from “go to the store and get milk.” We as humans can infer what you mean, but you do not want the large language models to make guesses. You told me to get the milk, I got the milk, I am done. You didn’t tell me otherwise. It is a simple, silly example, but it highlights your point about being specific about what “done” means; otherwise, it is going to give up too early or keep spinning and trying and burning all of your usage. You might find yourself with a bill for $1,000 when you were just supposed to get milk, and you ask, “Why do I have a whole dairy farm?”

Christopher S. Penn: This cow, where did this cow come from?

Katie Robbert: Well, you didn’t say how to get the milk.

Christopher S. Penn: That is a fair point. “Go to the store and get some milk” could lead to me going to a farm and buying a cow.

Katie Robbert: I went all the way across the continental United States, found the world’s best milk, and now you have to reimburse me for airfare, hotel, expenses, and shipping the milk back to you.

Christopher S. Penn: Exactly. If you provide insufficient specificity like “go to the store and get some milk” to a probabilistic tool, that is not enough information. You have to say, “Go to this store at this location, get this brand of milk, pay no more than this price. If the milk is $8, don’t buy it. Check out, put it in the car, take it home, put it in the refrigerator.” That is the process. It sounds so stupid, but if you think about it with robots like the Unitree H2, you would have to tell the robot every single step because it does not understand the world.

Katie Robbert: It has not had that experience before. Going to get milk sounds so basic and human, but imagine trying to describe that to someone who has never done it before. That is essentially what you are doing. Oh, and by the way, I need lactose-free milk. Make sure it is lactose-free and that it is 32 ounces, not 16 ounces, because I am baking with it. Those are details you want to ensure are clear because otherwise, you will keep sending that person back to the store over and over again until they get it right. That is when you start to burn up those tokens and cost yourself a lot of money, headache, and time.

We were talking about this in the last episode. We figured out how to give those details to the machines, but we still struggle to give them to humans. I think part of that is because we just assume humans can infer the rest of it, and then we get frustrated when they can’t. I remember when I first became a manager, a decade prior to meeting you, Chris. I was a brand new, shiny manager thinking I knew everything. I suddenly had two direct reports and was told, “Here is all the stuff you guys need to get done this week; delegate it however you see fit.” I said to my direct report, “Here are the five things I need you to do; let me know when it’s done.” They gave me back the five things, and I was unhappy with how they were done. They said, “Well, you never told me how you wanted it done; you just said to do it.”

It took me a long time to figure out how to have that specificity. Giving templates, standard operating procedures, and examples makes everybody happier because I am getting it done the way I expect, and there is a lot less guesswork on their part as to how it is supposed to get done and what “done” looks like. We are repeating ourselves, but we have to because people aren’t doing this.

Christopher S. Penn: The worst thing I have ever seen in a job description—which is in almost every job description and if you handed it to AI could cause World War III—is “and all other duties as appropriate.” I need you to make me a billion dollars, and in the job description, you give it something amorphous like “strategy advisor” and “all other duties as appropriate.” To make a billion dollars, the easiest way would be to incite a war and then start selling arms. You wake up one day and find out you started a war.

Katie Robbert: You are just casually starting a war.

Christopher S. Penn: So, in a standard operating procedure, what belongs in a good SOP?

Katie Robbert: Shocker: the 5Ps. If you want to learn more about the 5P framework, you can go to TrustInsights.ai/5P-framework. The 5Ps are purpose, people, process, platform, and performance. You start with your purpose: why are you doing this in the first place? What is the goal of this task, and what is it meant to accomplish? If it is to create a media list, that is the task, but why? What is it for? Start with that.

Then, the people. You are part of the people, but do you need to talk to your manager, subject matter experts, or clients? Are those people involved in making the media list? Then, the process. This is a good place for a template to exist. A media list might include names, emails, specific places to get information, and a certain cadence. It should take about this long and look like this.

Then, your platforms. What are the tools you are using? We have a subscription to this site, or we want to look at these specific competitors. Be specific about how you are compiling this information—is it in a sheet, a presentation, or a document? And then, your performance. Did you accomplish the task? Did you create a media list that someone can take and do something with, or do they have to refine it because you didn’t do it to satisfaction? Your performance is directly correlated to your purpose: why are we doing this, and did we do it?

Christopher S. Penn: When you hand off SOPs to agentic AI systems, purpose and performance are the two most important pieces. Purpose is essential because, as you said, if you specify the downstream usage of the output, you get phenomenally better results when using deep research tools. If you say, “I want you to build this for the purpose of pitching these reporters,” and you omit contact information or process, the smartest models will infer you probably need to add that.

Katie Robbert: That is only if you are using the smartest models. Think about large language models as someone who takes instructions very literally; do not assume they will infer information you missed. If you say the purpose is to contact reporters but you leave out “email” in the process, have you specified how to contact them? Are you writing a letter, calling them, DMing them on social media, or emailing them? You have not said, and you are either creating extra work for the model to research, or it is just going to be omitted altogether. If you do not look at an SOP and think, “Wow, that is a lot of detail,” it is probably not detailed enough for a large language model.

Christopher S. Penn: This is a prompt I put together to build a local newspaper. Up front, you have core objectives—what is your purpose? Every day you are going to do this, and here is what this means. I said, “You have to be truthful,” but we had to spell out what “truthful” means because “truthful” by itself is a confusing term. We never make stuff up; you only summarize what is there. Here is the second thing: these are the things that belong in it, and these are things that do not. That is the purpose section. Then the people: who is the reader, and who are the stakeholders we draw information from? Then the process: here is the exact process you are going to follow, down to specific code and templates. That is the level of depth agentic systems need, and it would also be a heck of a blueprint for a human team to follow.

Katie Robbert: If you are okay giving that level of detail to a machine, why can’t you also give it to a human? We focus so much on what we need to do with machines that I want to encourage people to step back and ask if they are doing this with their human teams, too. Not doing so breeds resentment toward the machines. People think, “You told the machine exactly what you wanted; why can’t you tell me? I could have completed that task.” Those are the things we as managers need to question. Maybe you haven’t given your human team enough information to complete the task successfully. We are the problem because we didn’t delegate effectively; we just said, “Go do the thing,” and then we are mad when it didn’t come back correctly.

Christopher S. Penn: One thing we teach in our workshops is that one magic sentence that belongs in every prompt—and to your point, Katie, every task delegated to a human—which is: “Ask me questions until you have enough information to succeed at the task.” Tell people they should be asking questions; don’t just nod, say you will do it, and suffer in silence. Ask me questions, because I have forgotten something.

Katie Robbert: You should factor in who is in charge of whether or not this belongs with AI or humans. There was a time when, if the team came to you with questions, it annoyed you. But if a machine asks questions, it doesn’t annoy you. Is this task something that belongs with a human who can be set up for success, or is it something that should go to AI because a human will never be set up for success due to who is in charge? I am only picking on you because you are here, but there are a lot of bad managers who say, “My door is always open,” but do not actually welcome questions.

Christopher S. Penn: That is a topic for another show. I am personally more comfortable with a machine than a human because I know the machine is “stupid,” has no feelings, and doesn’t understand. I don’t have to have empathy for it, so there are no consequences for me saying, “You have catastrophically failed; you should feel bad about yourself and try again.”

Katie Robbert: That is funny because people ask me, “What is it like to work with Chris?” and I am serious—you have never spoken to me that way. When I have asked you a question, you have always responded. I am a stubborn human and am willing to sit there and wait until I get the answer I need. It takes a certain kind of chemistry to make professional relationships work. Maybe AI is more appropriate sometimes only because the people you have on your team are better individual contributors, and they can get more done if they have access to a team of agentic employees.

Christopher S. Penn: It is interesting that the things that make me not a great manager of humans might make me a great manager of machines. Either you micromanage—which machines love and people don’t—or you do the “seagull” thing: fly in, squawk, and leave. That is how you handle agentic AI. You make a bunch of noise, give it a bunch of stuff, and walk away. AI agents like that work style. They are tuned to go off for 72 hours and not bother anyone. Those are terrible things to do as a human manager, but a set of machines thinks, “Great, we can work with this.”

Katie Robbert: Provided—big asterisk—that you gave it the appropriate instructions in the first place, which goes back to structuring things with the 5P framework. We have data to prove that structuring prompts with the 5P framework is more effective than not. We have real numbers on this, and agentic AI is just a bunch of prompts. If you structure your instructions with the 5P framework, you will be more successful. If you are not suited to be a people manager, there is still a path. We have always struggled with the idea that the only way to have upward mobility is to manage a human team, which often falls apart if the person isn’t meant for it. Now, there is another path.

Christopher S. Penn: Yes, you can manage a non-human team. Employee handbooks are equivalent to those principles for agentic AI: this is who we are, this is what we do, and this is what is in or out of bounds. Pair that with standard operating procedures for how to get things done, and include the job descriptions from our previous episode, and you have the trifecta of what you need to make agentic AI work well for you. If you have thoughts about how you are using these tools with AI, share your findings in our free Slack group at TrustInsights.ai/analytics-for-marketers, where over 4,700 other marketers are asking and answering questions every day. Find us at Trust Insights AI/podcast in all the places podcasts are served. Thanks for tuning in; we will talk to you on the next one.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.