What happens when you turn chaotic bug reports into standardised developer tasks automatically? Andriy Mudryk set out to answer this question by experimenting with an AI agent built on Rovo Chat, a tool within the Atlassian ecosystem.
In this conversation, Andriy describes how these AI agents work, explains the technical implementation, and shares the design principles and lessons learned from building a production AI agent.
Background & experience:
With extensive experience in cloud-native architecture and enterprise transformation, Andrii designs scalable solutions and pioneers the integration of Generative AI and autonomous agents across the software delivery lifecycle.
1. Let's talk about the ecosystem and tools. What is Rovo Chat and how does it work?
Andriy Mudryk: Rovo Chat is a tool within the Atlassian ecosystem designed to help users find knowledge and information within the system. It's a conversational interface where users can ask questions or describe tasks in plain text and get answers — similar to ChatGPT, but with important limitations.
The output depends directly on the question or task you pose. A key feature is that Rovo Chat only accesses data within the Atlassian ecosystem. It doesn't reach beyond the system and only works with content stored there.
Access control is also critical. The chat operates under strict security rules: users only see information they have permission to access. If you ask about data you don't have rights to, the chat and any agents built on it won't provide that information.
This proper separation of access rights works particularly well in corporate environments and for enterprise clients.
2. What's the difference between Rovo Chat and an AI agent built on top of it?
AM: Rovo Chat is a universal interface for interacting with the artificial intelligence model. It's a way to "talk" with the system: ask a question, describe a task, and get an answer. It's universal and flexible but not specialised for any specific process.
AI agents built on top of Rovo are different. They’re fully fledged process participants. Each agent has a clearly defined role, specific task, behavioural rules, knowledge sources they can or can't access, and a response format.
As a result, an agent produces not just a one-off answer, but a stable, predictable output in a fixed format that can be reused within a team or process.
Just like with Rovo Chat, access rights remain paramount. An agent never exceeds its granted permissions. If access to certain data is restricted, the agent can't work with it or respond to requests outside that scope.
A particular strength of agents is intelligence automation. They can be used in automated scenarios, but everything is strictly limited by access rules and administrator settings. Even with many agents running complex automations, they can't access anything beyond what's allowed by access policies.
Each agent exists in its own clearly defined context and operates only within it.
3. Tell us about your experiment. What problem were you trying to solve?
AM: As part of an assessment for one of our projects, the team approached us with a request to optimise their current process. The project delivers products to a single client company, which engages an external ethical hacking community to test those products and submit vulnerability reports. Our task is to analyse these reports and properly address the identified issues.
The main challenge was that these reports almost always came in arbitrary, unstructured formats. Often, it was more like stream-of-consciousness writing, with lots of personal narrative and no clear structure or template. Worst of all, important technical details often appeared not in the report itself, but later in comment threads.
Each hacker described problems in their own way, and transforming these reports into a standardised format was quite labour-intensive.
Basic classification was also a significant time drain: it wasn't always obvious which product the problem affected, whether it impacted other components, and where exactly it needed to be logged.
In summary, the biggest time drain was "translating" hacker reports from freeform, descriptive language into clear, structured tasks that the development team could act on.
4. How did you design the agent? What was its role, and what constraints did you set?
AM: Like any agent, it started with a clearly defined role. In our case, the agent had a dual role. It was essentially an Application Security Architect/System Analyst at the intersection of architecture and business analysis. Its job was to transform an unstructured problem description into clear, formalised requirements that could form the basis of a developer task.
The agent's main task was to take a raw, unstructured bug or vulnerability report and generate a standardised developer task from it. We wanted the team to start treating the agent's output as a reliable standard that's convenient to work with.
We set several hard constraints for the agent:
- Prohibited from describing attack details
- Prohibited from suggesting any scripts, automations, or technical advice on implementing fixes
- Prohibited from using the advice and tools mentioned in hackers' reports
The agent was only to formulate the task. The solution remained with the developer.
We made it clear that input data would always be unstructured. The agent's job was to clean it up, separate facts from narrative, and bring it to a clearly defined structure.
As output, the agent always had to generate an artefact with a clear structure that included:
- Problem statement — description of the problem
- Affected area — which part of the system is affected
- Root cause — the underlying issue
- Risks/Impact — risks and impact on the system or business
- Scope of fix
- Attack surface — boundaries of the problem
- System guarantees
- Acceptance criteria — clear criteria to verify that the problem is actually solved
This format was clear enough for a developer: there's context, there's a problem, there are risks, and there are quality criteria to check the result.
We also very clearly defined tone requirements. This had to be dry, engineering, technical language. The agent was forbidden from making assumptions, inventing details not in the input data, or using emotional or subjective narrative.
If information was missing, the agent had to explicitly state this and formulate a list of clarifying questions. Its key task was to completely remove emotion and subjectivity from the report, leaving only a structured, logical description of the problem that's easy for a developer to work with.
5. What processes are best suited for creating an agent?
AM: Agents work best for processes with repeatable tasks or stable business objectives. They're especially effective when input data is unstructured — like emails, comments, meeting notes — but the output needs to be standardised.
A typical scenario: we receive "raw" or messy context as input, but expect a clearly defined output in a uniform format. This could be a developer task, a Confluence page, or any other standard deliverable the team uses regularly.
Agents also excel with tasks that have clear quality criteria: the result either meets expectations or it doesn't. This makes it easy to verify completion and maintain consistent quality. Agents are particularly valuable where the cost of error is high and discipline, standards, and quality control are important — scenarios commonly found in data science workflows, software development, and compliance-driven processes.
That said, it's worth understanding where agents aren't effective. They're poorly suited for one-off or creative tasks without clear quality criteria. There's also no point in creating an agent for very simple tasks that are faster to do manually.
Another important limitation is context. An agent always operates within its available data sources. If a task requires information from different systems or with different access levels, there's no guarantee the agent will have the full picture, and the quality of results will suffer.
6. What risks can arise when we automate routine tasks using agents?
AM: That's a really important question. One key risk is treating the agent as a silver bullet. There's that classic trap: when you have a hammer, everything looks like a nail. The same can happen with agents.
The first risk is a shift in responsibility. The team starts treating the agent's output as gospel: they stop verifying results, analysing them, and thinking critically. People start to feel the agent is "always right," and responsibility gradually shifts from people to the automated tool.
In reality, an agent should simplify routine work, but not remove ownership. If a team stops verifying the agent’s logic, the quality of our output could drop, which ultimately risks degrading the customer experience. We must ensure that automation serves to enhance the final product, not just to move faster at the expense of the user's needs.
The second risk is delegating tasks at the wrong scale. An agent works well with clearly formulated, focused problems. It can help solve a specific task, but you shouldn't expect systemic thinking from it.
For example, in our experiment, we focused on bugs and vulnerabilities in software. The logical step is to "patch" a specific hole. The agent can handle that. But higher-level questions, whether this problem is part of a broader pattern, whether it recurs systemically, that's the team's responsibility, not the agent's. The agent doesn't think at that level of abstraction.
It's also risky to try making an agent "think like a human." We can ask it to generalise, systematise, and think more broadly, and technically, it will respond with something. But the question is whether that's the right approach. Today, an agent should be viewed as a tool with a clear focus, not as a full replacement for human thinking.
We're not yet at the level where we can create truly human-like agents. So the healthiest approach is to use an agent where it's strong: in repeatable scenarios, standardisation, and automating routine steps, while leaving strategic thinking and systemic decisions to humans.
7. What were the results? Did the agent deliver what you expected?
AM: Yes. As output, we wanted a task — an artefact that's a specific, understandable task for developers, which we can hand off for work. The agent successfully delivered that. We had a clear class of problems we were solving, and we configured the agent with context specific to security vulnerabilities and bug reports.
But the full story of how it performed in practice, what worked well, and what challenges we encountered — that will be part of the next interview. Stay tuned.
8. Does this approach work outside the Atlassian ecosystem?
AM: Absolutely. The concept translates well to other platforms. In fact, understanding this approach is valuable for anyone interested in AI agents in general.
The key isn't the specific tool. In our case, it's the Atlassian ecosystem, but the real power lies in discipline and process. We define roles, set boundaries, specify the expected results, and determine the format of the outputs we need.
When you combine these elements, you can apply this approach just as effectively with other tools.
So if you have messy input data and want standardised output, this approach will work for you.
Related insights
The breadth of knowledge and understanding that ELEKS has within its walls allows us to leverage that expertise to make superior deliverables for our customers. When you work with ELEKS, you are working with the top 1% of the aptitude and engineering excellence of the whole country.
Right from the start, we really liked ELEKS’ commitment and engagement. They came to us with their best people to try to understand our context, our business idea, and developed the first prototype with us. They were very professional and very customer oriented. I think, without ELEKS it probably would not have been possible to have such a successful product in such a short period of time.
ELEKS has been involved in the development of a number of our consumer-facing websites and mobile applications that allow our customers to easily track their shipments, get the information they need as well as stay in touch with us. We’ve appreciated the level of ELEKS’ expertise, responsiveness and attention to details.
