LLMs and Putting Out Fires
As these things typically do, implementing a new tool became my fire to fight this time.
I'm pretty interested in building things, fast, often very flawed, and even more often for myself. I was forced into the AI race via my day job, and I needed to adapt to that quickly. The initial scope was something that really ate at me, ethically. I value people over tools, and challenge over tedium. It wasn't very challenging from a technical aspect and that is probably why I was so standoffish initially after being given the mandate.
My first task was supposed to automate ticketing for a support desk. I know support work. I started my career at the help desk and it is also a place I can't seem to escape as an IT guy. The work is always dictated by the most important fire to put out in the general day to day, and as I have grown that is only proving more important especially as I focus on maintaining the stack of a production-focused facility or two. As these things typically do, implementing a new tool became my fire to fight this time.
The main mantra I always approached work with is relatively simple. Standardize, then optimize. The same basic idea is presented in Atomic Habits and maybe that's why I like the book so much. One percent increments of improvement, built on a standard, means the returns are visible after a few months of gnawing on a problem. This is how you build something sustainable.
"You have to standardize before you optimize." — James Clear, Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones (2016)
Metrics rule the day when tracking progress like this, as the small improvements inevitably add up, whether it's with training, documentation, or just consistency. So, the AI question.
Large language models are not instant personal second brains, they aren't any dark magic, they're tools built to both surface information and scaffold projects from my perspective. That's not really the point though. I was told to use it, and I generally don't like that type of heavy-handed move when I am working on anything. I like to pick what tools I use, and how I use them. Here's the rub. Regardless of tool, the results are what matter in the day to day. When the LLM was dropped in, it became a game of optimization before standardization, and this made no sense.
How do we apply this tool to the work that needs to be done? Here's what I've been working on. First figure out where it can save time and not clip quality. For me, this was around both documentation and timely responses I was terrible at writing. The documentation serves as the foundation for the other, I always think through a workflow before writing out a process or building something to enable my crew. That serves as the groundwork to improve on in the increments that Atomic Habits also summarizes in a better way than I can:
"Habits are the compound interest of self-improvement. The same way that money multiplies through compound interest, the effects of your habits multiply as you repeat them." — James Clear, Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones (2016)
I started with a quick look at what documentation I was missing. Building a help desk isn't a difficult thing, the framework already exists with ITSM, the tools are mature and I have the Cadillac of them to work with, Jira Service Management, and I have some smart folks working with me. Fortunately, I am also fully enabled to use tools like Claude and ChatGPT to get pieces moving. The goal is to build something that
doesn't depend on the latter two to function.
The natural course here is to understand the process, and digest it with the tools I have in play already. I don't think LLM technology is able to replace the creativity needed to deal with support, but I do think I can apply it to kill some of the monotony.
Next up is how we define removing the pain as the requirement as well as the result. Getting people help fast is always the first metric I try to tackle, so why not make a machine do it? That was the most natural fit to get the pressure off of the typical metrics for the folks on the ground.
We got that working by ingesting tickets into Jira first, and then using the native Rovo to generate a response with troubleshooting tips. While it took us a while to recognize that the LLM shouldn't be doing the intake and creation of the tickets, we did see immediate returns with the response as soon as it was in. Knowing when to use the tool still proves more important than just having it, that's the skill we need to appreciate.
Sometimes you need to help people reboot, and not just tell them. This allows the team to actually focus on resolving issues that take a lot more interaction to solve. The skill and personality remain the craft my team can bring. A ticketing system only records the problem; people fix it. So, what's next? Training the team on how to leverage the tool to make all of our lives a little easier. Probably categorization, something no one enjoys doing while running to put the next fire out.
