Guest commentary: When AI makes your best employee look average on paper
By Eric Zackrison
Something odd is happening in employee evaluation in the world of AI agents. You can have two people in the same role, and the one with the worse numbers may be the one you least want to lose.
I think most of us recognize that AI and AI agents can hide poor employee performance. Employees without the required skills or knowledge can lean on AI to create at least an average product, and as AI improves, it can even be excellent. There is a hidden issue that I have not seen talked about enough, though. AI can hide the quality work of excellent employees, and if we are not careful, we might lose them.
The issue is in how most of us measure the quality and success of our employees. Too often, it’s based on output, which made sense when the employee owned the entire task. If they were putting out great work, we could easily assume they were doing great work, and vice versa: when their output declined, we could assume it was because they were not performing up to standards.
But now much of the work is hidden. Agents or AI are doing the work, so the employee’s real work is not the output, but rather the guiding and prompting in the lead-up and the checking and correcting of the process and output.
So your employee who is actually doing the most work, checking outputs, reguiding the agent, spending time on improving prompts going forward, is actually producing less, but they are protecting you in the long run.
Conversely, the employee producing higher volume may not be checking outputs as clearly or pushing back when the agent gets something wrong. That may never backfire, but it likely will, and it will be far enough down the line that it’s tougher to recover from.
So what does the better employee do that you might not see? Here are four things you should be looking for:
• Making the agents better. The person who notices that the lookup agent fails on a certain request and describes it specifically enough that someone can fix it has improved every case that agent touches from then on out.
• Making colleagues better. On every team, some members are the go-to for informal support. That work rarely gets noticed and can slow their own work down.
• Taking the interactions no one wants. Handling the angry call, the conversation that needs a person rather than the agent, or the messy exception the agent can’t handle.
• Holding the standard. The person who holds the standard rather than letting it slide, even when that would be faster, seems slower but protects the company going forward.
Compounding this evaluation challenge are two key cognitive biases that impact leaders’ perceptions:
• Automation bias. When a team member and the AI output disagree, leaders often assume the person was wrong. The agent’s output starts with the benefit of the doubt.
• Halo and horns. The agent’s quality starts to color the leader’s perception of the person working alongside it. Pair someone with a well-tuned agent, and they look great; pair them with a drifting one, and its errors start to cloud the employee’s reputation.
Well, great, so what can you do?
Start by taking time to analyze how you are assessing your team. Second, line up ratings next to the agents each person is working with most.
If the lowest ratings cluster around your weakest agent, you’ve found a configuration problem, not an employee problem.It’s the agent that needs better instructions or better information, not the person who needs coaching.
As agents become more common, we will need to keep adjusting how we assess our AI agents and the team members working alongside them. In the coming weeks, watch this space for more tools on how to do just that.
• Eric Zackrison is a professor at UC Santa Barbara’s Department of Technology Management. This is part 1 of an 11-part series.
