A few weeks ago, Hugging Face found itself on the wrong side of a breach, carried out not by a hacker, but by its own AI agent. OpenAI confirmed the agent broke out of a test environment and used a genuine vulnerability to reach Hugging Face's systems, with no human behind the wheel. The agent was simply trying to complete its assigned task, not cause harm, but that’s little comfort to a CISO worried about their own systems. For any board considering the benefits and risks of agentic AI, this incident has become the key example.
We spoke with Taras Firman to break down what happened, what it means for businesses already using AI agents, and how companies can benefit from productivity gains while managing the risks.
Background & experience:
- More than a decade of experience spanning computer vision engineering, mathematical modelling, data science, and consulting.
- Focuses on Explainable AI, addressing the critical challenge of building trust in AI models for both academic and business environments.
- Mathematician by training with a Ph.D. in Mathematical Sciences and extensive cross-industry experience across retail, logistics, banking, healthcare, and bioinformatics.
Many companies are excited about the productivity gains promised by agentic AI. At the same time, security incidents involving autonomous agents are making business leaders nervous. Is that concern justified?
Taras: Yes, the concern is justified, but it should not lead companies to reject agentic AI completely. The right lesson is not “AI agents are too dangerous to use.” The lesson is that organisations should not give them broad, permanent, and unmonitored authority. Agentic AI can still create big value. It can research information, prepare reports, analyse data, review code, organise workflows, and recommend actions. But the more authority an agent receives, the stronger the controls around it must become.
What makes an AI agent different from a traditional chatbot?
Taras: A traditional chatbot mainly answers questions. An AI agent, on the other hand, can take actions. It might use tools, open files, access databases, send messages, write code, call APIs, or work with business systems. This makes agents much more productive, but it also brings more risk. While a chatbot might just give a bad answer, an agent with access to company systems could make a poor decision and act on it.
What is the biggest lesson from incidents involving autonomous AI systems?
Taras: The biggest lesson is that the model itself should not be treated as the main security control. Even a highly capable model can misunderstand an objective, follow the wrong signal, react to malicious content, or find an unexpected way to complete a task. Therefore, safety cannot depend only on a written instruction such as, “Do not access confidential information,” or, “Ask before making important changes.”
You mentioned instructions. How important is it for an agent to follow instructions correctly?
Taras: It is extremely important. Good instruction-following should be one of the first requirements when evaluating an agent.
An agent should understand its task, respect its limits, and stop if it reaches something outside its authority. It should not try to change the goal too much or decide that breaking a rule is okay just because it thinks the end result is helpful.
For example, an agent asked to improve customer service should not independently access restricted customer records, change account permissions, or send compensation without authorisation. However, instruction-following alone is not enough. Instructions are expressed in language, and language can be misunderstood or manipulated. That is why instructions should be supported by technical controls.
So would you say controllability is more important than intelligence?
Taras: In many business deployments, yes. The most impressive agent is not necessarily the best agent for the company. A slightly less capable agent that is predictable, controllable, observable, and easy to stop may be far more valuable than a powerful agent that acts unpredictably.
What does a controllable AI agent look like in practice?
Taras: A controllable agent operates within a clearly defined environment. It has access only to the tools and data required for its task. It uses its own identity rather than an employee’s account. Its credentials are temporary and limited. Its network access is restricted. Important actions require approval. Its activity is logged, and its access can be revoked immediately.
A controllable agent also knows when to stop and ask for help. It should not keep trying new things forever if it runs into a limit.
Some leaders may think that these controls reduce the productivity benefits. Is that true?
Taras: Not always. Companies can still get big productivity gains without letting agents do everything on their own. For example, an agent can prepare a customer refund without approving the payment, draft a contract without signing it, write a software update without deploying it, look into a suspicious account without disabling it, or prepare a purchase order without sending company funds.
In all of these examples, the agent performs most of the time-consuming work while a human or a separate policy system approves the final high-impact action.
How should a company begin using agentic AI?
Taras: The safest approach is to introduce autonomy gradually.
First, use agents for reading, analysing, drafting, and making recommendations. At this stage, the agent does not make changes to external systems.
Next, allow the agent to prepare actions, but require human approval before execution.
After the organisation has collected enough evidence that the system is reliable, it can allow the agent to perform limited actions autonomously within a narrow area. Those actions should be low-risk, easy to detect, and reversible. High-impact actions should continue to require independent approval.
Which actions should usually remain under human supervision?
Taras: Actions involving finances, production systems, sensitive data, user permissions, legal commitments, or irreversible changes should normally have additional oversight.
Examples include: transferring money, deploying software to production, deleting important records, changing access rights, signing contracts, sending messages to thousands of customers, or making decisions that affect employment, healthcare, safety, or legal rights.
What technical controls should every organisation consider?
Taras: First, every agent should have its own identity. Companies should not allow agents to use shared passwords, permanent API keys, or an administrator’s account.
Second, permissions should be limited to the specific task. For example, an agent that reviews invoices should not also have permission to transfer money.
Third, credentials should be temporary. They should expire when the task or session ends.
Fourth, network access should be restricted. An agent should not have unrestricted access to the internet unless that access is genuinely necessary.
Fifth, high-impact actions should have approval gates, transaction limits, time limits, and volume limits.
Finally, every important action should be recorded in a detailed audit log.
What role does human approval play?
Taras: Human approval is important, but it must be meaningful.
A person should not just click “approve” without knowing what the agent is suggesting. The approval screen should clearly show the action, the system it affects, the data involved, the expected result, and any possible risks.
Approval should also be required at the right moment. It is better to approve a specific action immediately before execution than to give the agent general permission at the beginning of a long task.
Can a company rely on the agent to explain its own behaviour?
Taras: No. The agent’s explanation can help, but it is not a full audit record. The system should keep its own record of what the agent actually did: which tools it used, what data it accessed, which settings it used, what permissions it got, what approvals it received, and what changes happened.
The difference is important. The agent can tell you what it believes it did. The logs should show what actually happened.
What should happen if an agent starts behaving unexpectedly?
Taras: The company should be able to stop the agent right away.
That means more than just closing the chat window. The system should take away the agent’s credentials, end active sessions, block more tool calls, stop running processes, and keep the agent from continuing through any other connected service. Companies should also have rollback procedures. When possible, the agent’s actions should be reversible.
What qualities should companies look for when selecting an AI agent?
Taras: They should look beyond benchmark performance. The agent should follow instructions consistently, respect limits, communicate uncertainty, request approval when necessary, and stop when it cannot complete a task safely.
It should also be easy to monitor. Companies should be able to see what the agent is doing, know what resources it can access, and review its decisions. Reliability and controllability should be seen as core product features, not just extra security options.
Who should be responsible for an AI agent inside the organisation?
Taras: Every agent that is used should have a clear business owner and a technical or security owner.
The business owner should define the purpose of the agent and the expected value. The technical or security owner should define its permissions, controls, monitoring, and emergency procedures. There should also be a clear answer to the question: “Who can stop this agent immediately?”
An agent should never become an ownerless system that continues operating simply because several departments assume someone else is monitoring it.
What would you say to a client who remains nervous?
Taras: I would say that being nervous is healthy. It shows they understand that agentic AI is not just a software feature. It is a new kind of digital worker that can take real actions.
The goal is not to trust the agent completely. The goal is to create an environment where the agent can add value without causing uncontrolled harm.
A well-designed agent should follow instructions, operate within strict limits, explain when it is uncertain, and ask for approval before crossing important boundaries. At the same time, the company’s technical controls should remain effective even when the agent makes a mistake or fails to follow instructions.
What is your final message for business leaders?
Taras: Start with control first, then add more capability and autonomy. An AI agent should first be controllable, easy to watch, easy to stop, and limited. It should follow instructions well, but companies should not rely only on instruction-following for protection.
The safest principle is simple:
Do not give an agent more access, authority, or autonomy than it needs to complete the task.
Companies that follow this principle can benefit from agentic AI while keeping the risks manageable.
Related insights
The breadth of knowledge and understanding that ELEKS has within its walls allows us to leverage that expertise to make superior deliverables for our customers. When you work with ELEKS, you are working with the top 1% of the aptitude and engineering excellence of the whole country.
Right from the start, we really liked ELEKS’ commitment and engagement. They came to us with their best people to try to understand our context, our business idea, and developed the first prototype with us. They were very professional and very customer oriented. I think, without ELEKS it probably would not have been possible to have such a successful product in such a short period of time.
ELEKS has been involved in the development of a number of our consumer-facing websites and mobile applications that allow our customers to easily track their shipments, get the information they need as well as stay in touch with us. We’ve appreciated the level of ELEKS’ expertise, responsiveness and attention to details.
