I feel guilty using AI at work.
I get that this does not make much sense. I am not using it to do all of my thinking for me. I mostly started using it for things like organizing information and proofreading.
To make myself feel better about using AI, though, I started asking a different question: If I am going to use AI, how can I use it to make what I produce better than what I could reasonably produce on my own?
Performance reviews are a good example.
At first, I would give the LLM the notes from me and the employee and have it organize them. Then I would use those notes to write a blurb at the end.
That saved me some time, but it wasn’t doing much to improve the actual process.
Now I am trying to use AI to set a higher standard for how I write reviews.
I give it the notes from the employee and me, but I also give it detailed explanations of our behaviors and how HR defines each level of engineer. I have it organize the notes, analyze the evidence, weigh it against those expectations, and provide feedback.
It can consider more information at once than I can and apply that information more consistently across engineers.
Before, I had to remember all of the details of our behaviors, the expectations for each engineering level, everything the employee accomplished, and everything I had observed during the year. That is just too much information to reliably hold in my head at once.
There is also a consistency problem.
Imagine I write one employee’s review first thing in the morning when I am fresh and another late in the day when I am tired. Even if I am trying to be fair, those reviews are probably going to be a little different.
The LLM doesn’t have that problem.
It can also find patterns across the information that I might miss. More importantly, it can hold all of the relevant context in front of it while evaluating the evidence.
That doesn’t mean I accept whatever it produces.
When it completes the analysis, I review it thoroughly for mistakes. I check its assertions against what I know about the employee, and I make sure its ratings align with how I would actually rate that person.
There are still problems. Our LLM tends to have lower expectations than our organization does, and it embellishes more than I would. Those are things I have to correct for.
But the result is better than what I could have reasonably produced on my own.
And that is the part I think matters.
The employee gets a review that more clearly explains what they do well, where they need to improve, and why they received the rating they did. I get there in less time while considering more information and applying the standards more consistently.
My next step is to create another revision of this process that reduces the problems I have found so far and gives employees even more useful information in the summary.
After that, I want to develop a tool that employees—and I—can use throughout the year to review the results and evidence we have collected. Instead of trying to reconstruct an entire year when review time comes around, we can continuously improve the quality of the information going into the review.
If you have access to an LLM for your work, don’t only ask what simple tasks it can do for you.
Ask what your work would look like if you could consistently do it at a much higher standard without spending significantly more time doing it.