We've spent decades building controls into ERP systems because we don't entirely trust humans.
That may sound cynical, but it is one of the foundations of enterprise technology.
We separate duties. We restrict access. We impose approval thresholds. We record transactions. We review privileged accounts. We prevent the person creating a supplier from also approving the payment. We design controls around the uncomfortable reality that people make mistakes, misunderstand instructions, exceed their authority and, occasionally, commit fraud.
Now we are preparing to introduce a new kind of user into those same systems.
One that doesn't sleep, doesn't take lunch and can potentially execute thousands of actions before somebody notices the first one was wrong.
The AI agent.
SAP calls its destination the Autonomous Enterprise. Its vision extends across finance, supply chain, procurement, HR and customer experience, with assistants orchestrating potentially hundreds of specialised agents. Other enterprise software vendors are travelling in much the same direction, even if they use different terminology.
There is plenty to be excited about.
There is also a question I think we are in danger of skipping.
How much authority are we actually prepared to give these things?
Because automation and autonomy are not the same thing.
Automation executes rules. Autonomy exercises authority.
And that distinction changes almost everything.
When does assistance become authority?
Imagine an AI agent working in procurement.
It analyses purchasing history, supplier performance, inventory and expected demand and recommends raising a purchase order.
Useful.
Now allow it to create the purchase order automatically.
We've moved a little further.
Allow it to select the supplier.
Further again.
Allow it to approve the order within a financial threshold.
Now it isn't merely helping someone perform a business process. The organisation has delegated financial authority to software.
Allow another agent to respond to that order by changing inventory allocation, adjusting a production schedule and arranging logistics, and something more interesting has happened.
We no longer have an AI feature.
We have a chain of autonomous business decisions.
This is why asking an ERP vendor what its AI agents can do isn't enough.
The more important question is what we are prepared to authorise them to do.
Those two questions sound similar. They are not.
Your next privileged user might not be human
Enterprise security already has a language for much of this.
Least privilege. Explicit authorisation. Segregation of duties. Identity lifecycle management. Logging. Monitoring. Revocation.
We should apply exactly the same thinking to AI agents.
An agent operating inside an ERP system should have an identity. Its permissions should be explicit. Its access should be limited to what it genuinely requires. Its actions should be attributable to that identity rather than disappearing into a generic system account.
And its authority should have boundaries.
Consider an agent authorised to approve purchase orders below €10,000.
That sounds reasonably controlled.
But can it approve one €9,000 order?
Ten?
Ten thousand?
A human with a €10,000 approval threshold is naturally constrained by the speed at which a human can work. Software isn't.
The traditional financial threshold therefore becomes insufficient. We may also need cumulative limits, transaction limits, velocity controls and behavioural thresholds.
An agent might legitimately approve a €7,000 transaction.
Approving 400 of them in six minutes should probably cause something to ask what on earth is happening.
This is where simply applying existing human access controls to non-human actors may not be enough.
Machine-speed authority requires machine-speed controls.
Does segregation of duties still work when both users are agents?
This becomes more complicated as agents begin interacting with other agents.
Suppose one agent identifies a purchasing requirement and creates an order. A second agent evaluates it and approves the expenditure.
On paper, we have separation.
But have we really achieved segregation of duties?
What if both agents operate within the same orchestration platform? What if they rely on the same model? What if they consume the same corrupted data? What if one agent's output becomes trusted context for the next?
We could end up recreating the appearance of traditional controls without preserving the independence those controls were designed to provide.
That matters because autonomous systems will inevitably become interconnected.
A forecasting agent predicts increased demand.
An inventory agent concludes that stock will be insufficient.
A procurement agent places additional orders.
A finance agent approves expenditure.
A production agent changes the manufacturing schedule.
A logistics agent reserves additional transport capacity.
Each individual action may be entirely rational based on the information available to that agent.
But what happens if the original forecast was wrong?
The problem is no longer one incorrect AI output.
It is the propagation of authority.
One bad decision has become trusted input for another autonomous decision, which triggers another, and another.
Humans can create exactly the same chain reaction, of course.
The difference is speed.
A dysfunctional management meeting can cause an extraordinary amount of damage, but at least it normally takes an hour.
Who is accountable when the agent was authorised to act?
This is where the conversation becomes uncomfortable.
If an AI system recommends an action and a human approves it, accountability is relatively familiar.
The human made the decision.
But what happens when the organisation has deliberately authorised an agent to make that decision?
Saying that “the AI did it” isn't an accountability model.
Nor is pointing at the software vendor.
If a company delegates authority to an autonomous system, somebody within that organisation still owns the consequences of that delegation.
That makes explainability and auditability much more than technical features.
I want to know what the agent did.
I want to know what information it used.
I want to know what authority it possessed at the time.
I want to know which other agents or systems influenced its decision.
I want to know what happened downstream as a consequence.
And, critically, I want to know who authorised the agent to possess that authority in the first place.
We already perform access reviews for privileged human accounts.
Perhaps the autonomous enterprise will require something similar for digital workers: periodic reviews not merely of what an agent can access, but of what it is permitted to decide.
The autonomous enterprise needs a kill switch
There is another question I would ask any vendor proposing autonomous agents inside a system of record.
How do I stop them?
Not eventually.
Not after opening a support ticket.
Not after somebody has found the correct administrator.
Now.
If an agent begins behaving incorrectly, the organisation needs to be able to revoke its authority immediately.
And that revocation becomes more difficult when agents operate across systems.
Disabling an ERP agent may achieve very little if it has already instructed another agent in a procurement platform, triggered a workflow in a manufacturing system and called an external logistics service through an API.
Which raises another awkward question.
What exactly does rollback mean in an autonomous enterprise?
Software developers are accustomed to rollback.
Business transactions aren't always so accommodating.
You can reverse a database change.
You cannot necessarily reverse a purchase order after a supplier has started manufacturing something.
You cannot simply undo a logistics booking after a truck has left.
You cannot necessarily restore yesterday's production schedule after physical materials have already moved through a factory.
Once autonomous agents cross the boundary between information and the physical or commercial world, reversibility becomes a business problem rather than a technical feature.
The objective therefore cannot simply be recovery.
It must also be containment.
Autonomy should be earned
None of this is an argument against AI agents.
Quite the opposite.
There are enormous opportunities to remove repetitive work, accelerate decisions and coordinate complex enterprise processes far more efficiently than humans can today.
But I wouldn't begin by asking how quickly we can reach autonomy.
I'd ask how an agent earns it.
Start with observation.
Let the agent analyse what is happening without changing anything.
Then allow recommendations.
Measure them.
How often are they correct? When are they wrong? What happens in unusual circumstances? How frequently do humans override them?
If the evidence supports it, allow limited execution.
Measure again.
Then expand authority gradually where the business value and reliability justify doing so.
Observe → Recommend → Act → Authorise → Orchestrate.
Those stages should not collapse into one another simply because a vendor's technology technically allows them to.
The ability to automate something is not the same as having sufficient evidence to delegate authority over it.
And different processes should reach different levels.
An agent might be trusted to resolve routine inventory discrepancies autonomously while a €20 million capital commitment remains resolutely human.
That's not a failure to achieve the autonomous enterprise.
That's governance.
The Pragmatic View
The autonomous enterprise probably is part of the future.
But I suspect the organisations that benefit most from it will not be those that pursue maximum autonomy.
They will be those that understand precisely where autonomy creates value and where human authority still matters.
CIOs should therefore treat AI agents much as CISOs treat privileged access.
Give them identities.
Apply least privilege.
Separate duties where genuine independence is required.
Limit their financial and operational blast radius.
Monitor their behaviour.
Record their decisions.
Understand their dependencies.
Test how failures propagate.
And make their authority immediately revocable.
Most importantly, don't grant autonomy simply because the technology supports it.
Make the agent earn it.
We have spent decades deciding which humans can be trusted with the keys to our ERP systems.
The next challenge is deciding when we are prepared to hand those keys to something that isn't human at all.
