Sometimes it takes me twice as long to finish a task with coding agents as it would have taken me by hand. By the time I notice something is wrong, they have produced a perfectly plausible pull request for the wrong problem.
In some cases, the trouble began with the ticket. An alarm pointed to the wrong metric, or the description had drifted away from what the system actually did. I was new to the codebase and skimmed the ticket because I was keen to get started. The agents knew even less than I did, but that did not stop them from filling in the gaps and writing code.
There is a deserved lesson here about how I use these tools. But my surrounding system also made it too easy to turn an unverified description into an implementation. Nobody had reproduced the problem or checked the premise against the current system. The first serious investigation happened after the pull request existed. When attention is scarce, spending it reviewing the wrong thing that late is wasteful.
Stuffing the Junk Drawer Link to heading
Most tickets are still static records of a request. You fill out a form, hand it over to some future person, and hope you included enough detail. That made sense when the tracker was mostly a queue, but now the tracker can be somewhat sentient and leverage tools. It can look at the deployment history, fetch a trace, attempt a reproduction, and ask a question or two. The ticket does not have to be a flawless, one-time instruction. It can be a place where the reporter, the system, and whoever is doing the work improve their understanding together.
It’s tempting to give an agent the ticket and tell it to do everything. Along with many others (and I mean many), I have tried enough versions of that to understand how convincing the wrong result can appear. A common frustrating failure mode occurs when an agent reads an unclear ticket, makes a test for its version, adjusts the code to pass it, and says it’s done. The test and patch originated from the same guess, and so the passing test only shows the agent was consistent. It does not show that it solved the reporter’s problem.
This situation calls for a talking ticket. Before a guess becomes a code change, it could help establish what actually happened and what’s needed. For debugging an issue, this can be an executable reproduction, an existing failing test, a trace tied to the reported request, or confirmation from someone who knows the intended behaviour. It will give the agent something outside its own interpretation to answer to. Another agent or two repeating the same premise can’t play that role.
Just to warn you, do not insist on a long, detailed report or a massive amount of text. In one study of software repair agents, executable reproductions, likely file locations, and targeted fixes were associated with better outcomes. Another study also found value in localization and suggested fixes. In both cases, longer reports were linked to worse outcomes. Controlling length seems to be valuable, though with these being two small-scoped observational studies, I wouldn’t yet go around hacking existing ticket templates. Besides, often times an issue reporter knows what feature they need or what bug affected them, but rarely do they know exactly which file to touch or how to exactly reproduce their problem outside of their own environment. If we’re lucky, there may exist systems that have the code, logs, and deployment history, but still making the reporter collect all of that is an odd division of labour.
It also makes little sense to dump every remotely matching trace into the ticket as help. Someone still has to decide which request matters, what was observed, and what remains an inference. Otherwise, the ticket becomes an expensive way to make everyone read logs. And no, an agent can’t decide on its own which parts matter because too much info hurts its ability to pattern match substantially. As with all software before it, there are limits, and pretending otherwise is silly. To reiterate, it’s not only the ticket description length that matters but also its total length across all comments and messages.
What Should the Ticket Do Instead? Link to heading
Imagine a ticket replying to a bug report like this:
I couldn’t reproduce this from the report. I found two requests near the supplied timestamp, but neither failed. Which account or request ID should I follow? I haven’t inferred a root cause or started a patch.
The point is the small next step that’s presented with grace. The ticket says what it tried, separates observations from conjectures, and asks one amenable question. Something a reporter can supply, or at least respond to, so the loop keeps going until it reaches a stopping point. A domain expert might recognize the issue and short-circuit the investigation, but that’s optional. If a reproduction succeeds, the ticket can attach what it observed and move the work forward. If it can’t make progress, it should say so and stop.
I don’t want a ticket that becomes the yappy fella nobody wants to sit next to. It should speak only when presenting decent evidence or a blocking question, not whenever an agent has another thought. It should use the access it already has to fetch relevant information, within the same permissions that the people doing the work would have. And it should keep its actions auditable. This came from a trace; this came from the reporter; this is the agent’s current hypothesis and so on. If every investigation begins by fetching the same three things, that process should become a tool or script. The agent picks the next uncertain lead to chase, whereas the predictable work unfolds predictably. This is a version of gradual automation where we learn the routine, then take the routine out of the guessing loop. It is incredibly useful.
The same principle applies beyond bugs. An on-call handoff could check whether the alert still fires before sending someone down yesterday’s path. A runbook could fetch the current state before proposing its next step. These things can remain useful records while also helping people question stale assumptions.
How far any of them should advance depends on what an error would cost and how well the result can be checked. A small, reversible change with a reliable verifier needs less intervention than a production change with no clear way to tell whether it helped. For uncertain work, I’d want a person to agree on the diagnosis and scope before the agent starts implementing. Give that person the relevant evidence, including what the automation could not establish, so they can correct it while the mistake is still cheap.
A ticket that stops to ask one useful question may look less impressive than a ticket that opens a pull request. It might also save me from reviewing another beautifully written fix for a problem I never had.