Watch on YouTube · 2026-08-04
- 00:00 Can AI Agents Run Your Business Alone?
- 01:15 How to Know What to Automate With AI
- 05:13 Why Your First AI Automation Won't Work
- 08:23 Why AI Automation Still Needs Human Oversight
- 11:10 Why You Shouldn't Automate Everything At Once
- LevelUp Labs
- The Nuanced Perspective (newsletter)
- LevelUp Labs education
- Awesome Generative AI Guide
- My courses on Maven
Auto-generated captions from YouTube, lightly cleaned.
You've probably seen those videos where someone automates their entire company with AI agents over just a weekend and each of them seem to have the exact same promise. Set it up, let the agents run everything and just walk away. They'll pretty much do everything for you. And a lot of people watching this are convinced that it's really that simple except it isn't. I've spent the last 10 years building and leading AI applications for companies of all sizes, first as a tech lead at Amazon and now through my own startup. And here's what I can tell you from experience. You can set up agents to take on large workflows and automate work that would previously take too many hours, have multiple handoffs and require a lot of extra resources.
But these videos make it look like you can just flip a switch and it all runs on itself. So people start building with the wrong expectations. And I have watched this happen over and over again across companies, teams and people. So today, I'm going to be taking a flipped approach and instead of teaching you how to automate just everything, I'm going to walk you through the four most common mistakes people and companies make while automating with AI so that you can save hundreds of hours and stay away from the hype and also build something that stands the test of time. Let's get into it. The first mistake is handing the wrong things to AI.
Here are three questions that tell you exactly what's worth automating and what isn't. Now, there is a consistent pattern with every new technology. We overestimate what it does in the short run and underestimate what it can do in the long run. Here's something I see pretty constantly, right? A founder gets super excited, spends about three weekends building an agent that writes an update for their investors every month. It's a pretty impressive build and demos beautifully. But the problem is that the founder sends an investor update just once in a month and each of that has to be really precise. One wrong number, one off note and it damages the relationship with investors.
So they end up rewriting whatever the AI did anyway. Weeks of work all to save themselves nothing. Meanwhile, the same founder is personally sorting 200 customer support tickets every week. Now, that is the work an automation would literally crush. It runs constantly, it's cheap to get wrong, and it's the same process pretty much every time. But, sorting support tickets doesn't make it look like a fancy demo, so they never automate it even though it was costing them hours every week. There's an old line often credited to Bill Gates, which reads something like this: Automation applied to an efficient operation magnifies the efficiency, and applied to an inefficient one, it magnifies the inefficiency.
So, automate a rare or broken workflow, and all you built is a faster way to be wrong. So, think of it this way. Every workflow automation is like making a bet, and you can score the bet to decide if you should build it with just three simple questions. One, how often does this workflow run? Two, what does a single mistake in this workflow cost you? And three, how often do the steps in this workflow change? Remember the investor update example? It loses in all three. It runs once a month, the stakes are too high to allow for an error, and the content pretty much shifts every time.
The support ticket example though pretty much wins in all three of these questions. It runs constantly, getting something wrong costs you much lesser, and the steps are nearly identical each time. So, the answer is pretty obvious once you actually score it. But, I want you to also think of one more thing apart from these questions before you go there and start automating. You also want to be thinking about how these automations are very specific to your voice, the way you write to a customer who's upset, or the way you frame a hard piece of feedback that no template could ever capture. And it also shouldn't because for that work, the effort is the message, and people can tell instantly when the effort is replaced by something super generic.
So, you pretty much have two choices. Either codify your style, give the AI real examples and phrasing so that it genuinely sounds like you, or do it by hand until it gets there. Both of these are good solutions. The only bad solution is to simply automate everything and hope that the output looks and sounds like you. So, I want you to list about 15 most repeated workflows for you on a daily basis and score each one of them on the three questions that we discussed. Anything that's scores well on all the three, it's good for automation. For everything else, leave it alone. Flag the work that is specific to your voice and decide whether to codify for the AI to do it or for you to do it manually.
You can also brainstorm with AI systems like Claude to help you understand what your most repetitive workflows are. Here's a prompt you can use. Here are the workflows I run in my business. For each one, score it from one to five on how often it happens, what it costs if it goes wrong, and how often the steps change. Add the scores and rank what's safest and most valuable to automate first. Now, you want to start at the top of that ranking and make your first build the most boring and high-volume task you have at hand. Now, once you know which workflows you want to automate, there's still a right and a wrong way to start.
And most people get it wrong in pretty much the same way, which brings me to mistake number two, which is assuming that your first AI automation is going to work seamlessly and you don't have to do anything about it. And then, when it doesn't, scrapping it and completely starting over. Here's what you should do instead. Say you scored your workflows and you picked the right one to build and you launched your first support ticket automation like we were discussing about. And you realize that it's correct 80% of the times and it gets things wrong about 20% of the times. And one specific use case where it goes wrong is that the AI keeps misreading refund requests as general questions.
A customer is probably saying that they want their money back, and if it gets filed as a general question instead, nobody acts on it. And those refunds are probably sitting on unanswered for days. And by the time you notice, you've already lost trust from these customers. And what I've noticed is that the team's first instinct is to often scrap it, start over, or start complaining that AI is not working for them. So, that's weeks and weeks of work when the fix would have literally taken about 20 minutes of work per week. When in reality, a few weeks of small corrections to the automation and calibrating it would have made it perform much better instead of starting over again.
So, this is exactly what you want to keep in mind. Your automation will not start working seamlessly on day one, and you want to account for it. You don't hand over the keys to an automation on day one. Instead, you do it gradually like climbing rungs on a ladder, and it only moves up when your automation has earned trust from you. There are pretty much three sequential rungs it has to pass. The first rung is consult. It suggests and you decide. Once it's proven itself in real work, it moves up to the second rung, which is augment, where it drafts the whole thing and you approve before anything goes out.
Only once that is solid, do you go to the third rung, which is automate, allowing it to act on its own. And that climb is powered by what you log. Manufacturing has a word for this kind of incremental improvement, it's called Kaizen. Small corrections that compound over time beat one heroic rebuild every time. The first version of your automation is never going to be a finished destination. It's pretty much the starting point, and you need to make sure that it gets better over time. Before you give up on your automation or switch something or completely start over, just set up a simple log, a record of every time the automation runs and what it specifically did.
Those logs are your improvement flywheel. Each week, look at the 10 times it got it wrong, find the pattern, and hand it those exact decisions as examples of what to do instead. Like always, you can pretty much brainstorm with an agent to understand how your workflows are performing. Here's a prompt to run that weekly pass. Here are the cases my workflow got wrong this week, each with what should have happened. Group them by the underlying mistake and suggest changes that fixes these use cases. The automation gets better, your trust in it grows, and it earns its way from consult to augment to automate. Most people don't log their automations, which is why they don't understand why they're performing bad, and there's literally no flywheel.
And every change they're making is nothing more than a guess. Now to the next problem. People also start assuming that once a fine-tuned automation is set up and it's running, they can pretty much walk away from it. They can't. So, the next mistake is thinking you can fully step away. No business workflow exists right now that can improve itself with literally no one watching, at least in the beginning. What actually changes over time is how much of your attention it needs, and it gets lesser and lesser as you calibrate your automations better. I'll give you a real example to back this up, right? Imagine you have a bookkeeping automation that has been running clean for months.
It's the safe one, so you stop checking it, you stop logging it, you don't care about it because it runs fine end-to-end. But then suddenly one day, a vendor has changed their invoice format, and the automation starts filing a whole category of expenses in the wrong place. One miscategorized entry at a time, and this keeps happening for weeks. And then in tax season, you find that tens of thousands of dollars are placed in the wrong category, and you're left with weeks of errors to untangle and a large bill from your accountant. Engineers call this the irony of automation, which is the more of a system you hand to a machine, the more the whole thing depends on somebody catching what becomes too too for it to handle.
It's the same thing that keeps planes in the air. The autopilot flies almost for the entire flight, but the pilots stay up front for decisions that need human judgment. Think of an unexpected storm, an emergency, a difficult landing, or whatever it might be. The human has responsibility of making these high-stakes decisions. On my own team, every person manages set of automations the way they'd manage direct reports, checking their work, correcting the misses, and keeping each one on track. And for anything shared across the team, we have one rule. Anyone can propose a change, but one named person signs off on it. They own that entire automation.
Your involvement can slowly hit zero, but you still want to keep an eye on it. What changes over time is how much of involvement each automation needs, and the more it proves itself, the less it needs from a human. So, whenever you're setting up automations, remember to give it these four things. An owner, which is one named person who's accountable for it. A review cadence that's allowed to stretch as it proves out. A tripwire, which essentially is a condition that pulls the owner back in the moment something goes outside the lines. For instance, a refund of a certain amount, an email from a very important customer, a number that suddenly spikes, or whatever outlier happens.
And finally, a hard floor. For anything legal or anything touching money or customers, a human has to sign off, no matter how well an automation is performing. For anything that's shared across people, clearly document who can propose changes and who is that single person allowed to commit them. And mistake number four is what I'd like to call the everything, everywhere, all at once. Once people start to understand what's possible with AI, they try to automate the whole company all at the same time. When it falls apart, which it will, they blame the technology. Every AI automation needs someone watching it. If you launch five of them at once, you'll spend the next month firefighting mistakes across all of them.
And you'll probably scrap the whole thing before it ever works. Here's exactly how this goes, because I've seen it on repeat. A founder sees what's possible with an automation, gets excited, and decides to go all in over the course of a weekend. They connect an AI agent to everything, their inbox, their Slack, their invoicing, five or six automations live by Monday morning. And Monday, the support agent is confidently quoting a refund policy, except that the policy actually stopped being used a year ago, so it's using outdated information. On Tuesday, the lead follow-up agent emails the same prospect three times because it never saw the CRM was already updated.
On Wednesday, two of them are double booking the same calendar slot. By Friday, the founder has switched off half of their automations, gone back to doing everything by hand, and concluded that AI just doesn't work for a business like this. But if you really think of it, none of this was an AI problem. Instead, it's pretty much like they hired five new employees on the same day, supervise none of them, and somehow they were shocked when mistakes started to happen. Every automation you turn on is almost like a standing commitment. It's a feedback loop you have to tend to as long as it runs. If I had to put it in a simple and short principle, it would be to stop starting and start finishing.
Your real constraint isn't how many automations you can build, it's how many you can keep healthy at once. So even before you start automating just everything, decide what's ready to leave alone and come back to later, and also decide the ones that are right for automation, just like we spoke about in the first part of this video. When the automations you've currently set up run consistently on real work with only minor oversight from you, and have reached their calibration phase, and the list of things it's getting wrong is reducing over time, that's when you want to increase the number of automations and not just do everything everywhere all at once.
So remember that the tools are going to keep changing, and the automation you build today might not be the one you build in the next 2 years. AI is moving at such a fast pace, but what stays constant is this. The more powerful the AI gets, the more valuable the human directing it becomes. There's a pattern in economics that says when a kind of work gets cheaper and easier, the demand for it goes up. So, automation won't empty your plate. It fills it with bigger and more impactful work to do. Think of it almost like the kind of work that needs a level of judgement and critical thinking only a human who understands the business can do.
So, the companies winning won't be the ones with the most number of automations. They'll be the ones with the sharpest people who are running them, calibrating them, and understanding what the outliers are. If you want to go deeper on building this for your own business, that's exactly what we do at Level Up Labs. Please reach out to us. The link is in the description below. And if this was useful, please subscribe. I'll keep bringing you the stuff that actually works in practice, not just claims of automating my entire company with AI, because we all know that's not how it works.