Insight / 2026
Governing AI Agents in the Enterprise
New York AI Group™ | Insights | AI Governance | 2026
Artificial intelligence is beginning to move from systems that primarily generate information toward systems capable of performing actions.
That transition has significant implications for enterprise governance.
Generative AI largely introduced organizations to systems that could draft documents, summarize information, answer questions, write code, and assist employees. AI agents extend that model. Depending on their design and authority, agents can retrieve information, interact with enterprise applications, call software tools and APIs, execute sequences of tasks, coordinate workflows, and operate with varying degrees of autonomy.
The OECD’s 2026 examination of the agentic AI landscape reflects the growing importance of these systems, identifying autonomy, goal-directed behavior, interaction with external environments, and the capacity to execute actions among characteristics frequently associated with AI agents.
For enterprises, this creates a fundamental governance distinction.
When AI generates an answer, organizations primarily govern an output.
When AI performs an action, organizations must govern authority.
That shift changes the control model.
The central question is no longer simply whether an AI system is accurate, responsible, or safe. Enterprises must increasingly determine what an AI agent is permitted to do, what resources it can access, when humans must intervene, and who remains accountable for its actions.
The enterprise is moving from copilots to agents
The first generation of enterprise generative AI largely operated as an assistant.
A user initiated an interaction.
The AI generated an output.
A person generally decided what happened next.
Agentic systems can change this sequence.
An enterprise agent may receive an objective and then determine some combination of the steps required to achieve it. Depending on the implementation, an agent could interact with databases, enterprise applications, APIs, documents, software development environments, workflow systems, or other agents.
Consider a traditional enterprise AI assistant tasked with helping an employee prepare a customer report.
The assistant may retrieve information and generate a draft.
An agentic system could potentially go further:
retrieve the relevant customer information → analyze activity → prepare the report → initiate an internal workflow → request approval → update an enterprise system.
The value proposition is considerably larger.
So is the governance challenge.
The more authority an AI system receives, the more closely its governance begins to resemble the governance traditionally applied to employees, applications, privileged accounts, automated processes, and other operational actors.
An AI agent should be treated as an operational actor
One of the most useful conceptual shifts for enterprise leaders is to stop thinking about an AI agent purely as a model.
An agent is better understood as a system operating around a model.
It can include:
- one or more AI models;
- instructions and policies;
- memory or stored context;
- enterprise data access;
- software tools;
- APIs;
- identity credentials;
- workflow logic;
- monitoring;
- human approval mechanisms; and
- external services.
Risk therefore does not arise solely from the underlying foundation model.
It emerges from the combination of the model and the environment in which the agent operates.
A highly capable model with no access to enterprise systems may have relatively limited operational authority.
A less sophisticated model with permission to modify financial records, interact with customers, or trigger critical workflows can create substantially greater institutional exposure.
Governance should therefore focus not only on model capability, but on system authority.
The four questions every enterprise should answer
Before deploying an AI agent into a material business process, organizations should be able to answer four basic questions.
What can the agent see?
This concerns access.
What documents, databases, customer information, internal systems, APIs, or other information sources can the agent retrieve?
What can the agent do?
This concerns authority.
Can it only generate recommendations, or can it send communications, modify records, execute software functions, initiate transactions, or trigger downstream workflows?
Under what conditions can it act?
This concerns controls.
Are actions limited by thresholds, business rules, authorization levels, human approval, time restrictions, or other policies?
Who remains accountable?
This concerns institutional responsibility.
Every material AI agent should have an identifiable business owner and technical owner, with escalation and accountability mechanisms established before deployment.
These questions appear simple.
In practice, they form the foundation of an enterprise agent governance architecture.
Identity becomes fundamental
Humans have identities inside enterprise technology environments.
Applications have identities.
Services have identities.
Increasingly, agents should as well.
An AI agent interacting with enterprise infrastructure should not operate through unidentified or unnecessarily broad credentials.
Organizations need to know:
which agent performed an action;
which resources it accessed;
which authority it was exercising;
who authorized that authority;
and what happened as a result.
This leads toward an important enterprise architecture principle:
Every material AI agent should have a distinct identity, an accountable owner, defined permissions, and an auditable activity history.
Identity creates the foundation upon which authorization and accountability can operate.
Without it, an enterprise may know that an automated process performed an action but have insufficient visibility into which agent initiated it or under what authorization.
Apply least privilege to AI agents
Cybersecurity has long relied on the principle of least privilege: users and systems should receive only the access required to perform their legitimate functions.
The same principle is particularly important for agents.
An agent designed to analyze invoices should not automatically receive the ability to approve payments.
An agent summarizing customer information should not necessarily be able to modify customer records.
An internal research agent may require access to information repositories without requiring access to administrative systems.
The objective should be to reduce the potential consequences of incorrect, manipulated, or unexpected behavior.
CISA’s guidance concerning secure AI integration emphasizes the importance of security controls as AI becomes capable of decision-making and autonomous action in operational environments.
This implies that enterprise agent governance should increasingly connect with existing disciplines such as:
identity and access management;
privileged-access management;
segregation of duties;
application security;
data-loss prevention;
and zero-trust architecture.
Agent governance should not become an isolated technology program.
It should extend existing enterprise control systems into a new category of computational actor.
Separate recommendation authority from execution authority
One of the most important governance distinctions is the difference between an agent being permitted to recommend an action and being permitted to execute it.
Consider four progressively stronger levels of authority.
Level 1 — Information
The agent retrieves, organizes, or summarizes information.
A person performs the actual business action.
Level 2 — Recommendation
The agent proposes an action but cannot execute it.
A human decision-maker remains responsible for approval.
Level 3 — Supervised execution
The agent prepares or initiates an action but requires human approval before a consequential step occurs.
Level 4 — Autonomous execution
The agent is permitted to complete defined actions without individual human approval.
These levels should not be treated identically.
As autonomy increases, organizations should generally increase the rigor of authorization, testing, monitoring, and escalation.
This is proportional governance applied to agentic systems.
Human oversight must be designed, not declared
Many AI governance programs rely upon the phrase “human in the loop.”
That phrase alone is insufficient.
Human oversight must be operationally meaningful.
An organization should determine:
Who is responsible for reviewing an action?
At what point does review occur?
What information does the reviewer receive?
How much time is available for intervention?
Can the reviewer stop or reverse the action?
What happens if the reviewer disagrees with the agent?
What happens if the human fails to respond?
When should the system automatically escalate?
A nominal approval process in which employees routinely click “approve” without adequate information provides little meaningful control.
Human oversight therefore needs to be engineered into the workflow itself.
Establish transaction and authority limits
Financial institutions have long understood that authority should be bounded.
Employees can have spending limits.
Trading systems can have position limits.
Corporate cards have transaction limits.
Approvals can require multiple levels of authorization.
The same concept can apply to AI agents.
Rather than answering the binary question—
“Can this agent perform this action?”
organizations can define:
“Under what conditions and within what limits can this agent perform this action?”
An organization might establish thresholds related to:
financial value;
customer impact;
data sensitivity;
volume;
risk classification;
external communication;
or operational significance.
The purpose is not to eliminate autonomy.
It is to constrain autonomy within a clearly defined operational envelope.
Maintain segregation of duties
AI agents may eventually participate in workflows traditionally involving multiple people or systems.
That creates a temptation to consolidate activities.
But efficiency should not automatically override established control principles.
An agent that creates a transaction should not necessarily approve the same transaction.
An agent generating a compliance assessment should not necessarily provide final approval for the system being assessed.
A development agent that changes production code should not automatically become the entity responsible for independent validation.
Segregation of duties exists because organizations recognize that independence can reduce fraud, error, conflicts, and control failures.
The principle remains relevant when some participants are artificial rather than human.
Governance requires an agent inventory
Enterprises increasingly need inventories of their AI systems.
Agentic AI makes this even more important.
A useful agent inventory might record:
- agent name and identifier;
- business purpose;
- business owner;
- technical owner;
- model or models used;
- deployment environment;
- connected systems;
- accessible data;
- available tools;
- authorization level;
- autonomy level;
- human approval requirements;
- risk classification;
- monitoring requirements;
- version;
- approval status;
- dependencies; and
- lifecycle status.
The inventory creates institutional visibility.
Without it, agent deployment can become fragmented across business units, technology platforms, SaaS applications, and development environments.
Eventually, an enterprise should be capable of answering a straightforward executive question:
How many AI agents are operating within our organization, and what are they authorized to do?
If leadership cannot answer that question, agent governance is not yet mature.
Govern tools, not only models
One of the distinguishing features of agentic systems is their ability to use tools.
A model might have access to:
email;
databases;
file repositories;
code execution environments;
enterprise applications;
browsers;
APIs;
or workflow systems.
These capabilities determine much of the agent’s operational power.
Governance therefore needs to consider the tool layer.
Organizations should understand:
Which tools are approved?
Which agents can invoke them?
What parameters can be supplied?
What data can move between systems?
What actions require additional authorization?
How are tool calls recorded?
Can a tool be disabled centrally?
An agent connected to powerful tools is fundamentally different from an AI system that only generates text.
Data governance remains central
Agents frequently require greater access to organizational information than conventional standalone AI assistants.
They may need contextual information from multiple enterprise systems to complete objectives.
This expands data-governance requirements.
Access should remain aligned with legitimate business purpose.
Sensitive information should remain appropriately protected.
Agents should not inadvertently create pathways around existing permissions simply because an AI system is acting on behalf of a user.
CISA has emphasized that securing AI data is important to the integrity and trustworthiness of AI outcomes, including protection across the AI lifecycle.
This means agentic AI should inherit existing enterprise controls around:
data classification;
access management;
retention;
privacy;
security;
and information governance.
Agents require comprehensive logging
The greater the autonomy, the greater the importance of observability.
Organizations need more than a record of the agent’s final answer.
For material agents, it may be necessary to understand the broader activity sequence:
what objective was received;
what information was retrieved;
which tools were invoked;
what decisions were made;
what actions were attempted;
which actions were approved;
which systems were modified;
what errors occurred;
and what outcome resulted.
Logging serves several purposes simultaneously.
It enables:
security investigations;
operational troubleshooting;
risk monitoring;
performance measurement;
auditability;
and accountability.
GAO’s AI Accountability Framework places continuing monitoring alongside governance, data, and performance as a core element of responsible AI oversight.
For agentic systems, this becomes particularly consequential because monitoring increasingly concerns behavior rather than merely output quality.
Monitor behavior, not just accuracy
Traditional machine-learning systems are frequently monitored for performance measures such as accuracy or drift.
Agents require a wider perspective.
Organizations may need to monitor:
task completion
Does the agent achieve its intended objective?
tool behavior
Which systems and functions does it invoke?
authorization behavior
Does it attempt actions outside its expected authority?
risk indicators
Are prohibited or unusual actions occurring?
human intervention
How frequently are actions overridden?
economic performance
Does automation create measurable value?
security events
Are there anomalous access patterns?
failure modes
How does the agent behave when systems are unavailable or information is ambiguous?
This introduces the concept of behavioral assurance.
The enterprise is not simply evaluating what the model knows.
It is evaluating how a software actor behaves inside an operational environment.
Establish a kill mechanism
Every consequential automated system should have mechanisms for containment.
For AI agents, organizations should establish the ability to suspend access or stop execution when abnormal behavior is detected.
This may include disabling:
agent credentials;
tool access;
API access;
individual workflows;
particular actions;
or the entire agent.
The critical principle is operational control.
An enterprise should never design an autonomous system that it cannot reliably stop.
This becomes more important as agents become interconnected.
If multiple agents coordinate workflows, containment should be possible at both the individual-agent and system level.
Change management becomes agent governance
AI agents are not necessarily static.
Models can change.
System prompts can change.
Tools can change.
Permissions can change.
Connected APIs can change.
Retrieval sources can change.
Agent objectives can change.
Any of these modifications can materially alter behavior.
An agent that passed an evaluation six months ago may no longer behave identically after its surrounding architecture changes.
This is why AI governance should be lifecycle-oriented.
The NIST AI Risk Management Framework was explicitly designed to integrate risk management throughout AI system design, development, deployment, use, and evaluation. Its Generative AI Profile extends this approach to risks associated with generative systems.
Enterprises therefore need criteria specifying which changes require:
retesting;
risk reassessment;
security review;
approval;
or temporary suspension.
Third-party agents create additional risk
Not all enterprise agents will be developed internally.
Increasingly, software vendors are embedding agentic capabilities directly into enterprise applications.
This creates a governance challenge.
A company may acquire autonomous capabilities indirectly through platforms it already uses.
Vendor risk management therefore needs to evolve.
Organizations should understand:
what the vendor’s agent can access;
how permissions are established;
what models support the service;
what data is transmitted;
how actions are logged;
whether agent functionality can be disabled;
how model or product changes are communicated;
and what contractual accountability exists.
Traditional SaaS due diligence may not fully capture these questions.
Agentic capability should increasingly become a specific component of technology procurement and vendor assessment.
Multi-agent systems increase complexity
A further development is the emergence of systems in which multiple agents interact.
One agent may conduct research.
Another may evaluate information.
Another may prepare an action.
Another may execute the action.
This architecture can create specialization, but it also complicates accountability.
If a system fails, which agent was responsible?
Which system’s information influenced the final action?
Can one compromised or incorrect agent influence another?
How are permissions inherited across an agent chain?
How is the complete workflow reconstructed?
These are not merely technical questions.
They represent emerging enterprise governance questions.
Multi-agent systems will likely require end-to-end observability rather than separate logs viewed in isolation.
Accountability must remain human and institutional
As AI systems become more autonomous, organizations may be tempted to describe decisions as having been “made by the AI.”
That framing creates an accountability problem.
AI agents do not eliminate organizational responsibility.
An institution decides to deploy the agent.
An institution determines its purpose.
An institution gives it access.
An institution establishes—or fails to establish—controls.
An institution benefits from its operation.
OECD AI principles explicitly place accountability on AI actors according to their roles and emphasize traceability and responsible functioning of AI systems.
The appropriate enterprise model is therefore not:
the agent is accountable.
It is:
the organization is accountable for the authority it delegates to the agent.
A practical Enterprise Agent Governance Model
Organizations can organize agent governance around eight interconnected controls.
1. Inventory
Know which agents exist and why.
2. Identity
Give material agents identifiable enterprise identities.
3. Authority
Explicitly define permitted actions.
4. Access
Limit data and systems according to legitimate need.
5. Human oversight
Establish meaningful approval and intervention mechanisms where appropriate.
6. Evidence
Maintain records supporting testing, authorization, and deployment.
7. Monitoring
Observe agent behavior, system actions, security events, and performance.
8. Lifecycle control
Reassess agents as models, tools, permissions, data, or objectives change.
These controls should operate together.
An inventory without monitoring provides visibility but little assurance.
Monitoring without identity makes accountability difficult.
Identity without bounded authorization leaves excessive authority.
Authorization without change management can become obsolete.
Agent governance therefore functions as a system of controls, not a checklist.
A risk-based approach to agent autonomy
Not every AI agent requires the same governance.
An internal agent that organizes public research is materially different from an agent participating in a financial transaction.
A useful conceptual model is:
Low autonomy / low consequence
AI assists with information processing while humans remain responsible for action.
Governance can generally remain lightweight.
Moderate autonomy / limited consequence
AI executes routine actions within well-defined boundaries.
Monitoring and authorization become more important.
High autonomy / material consequence
AI can initiate actions affecting customers, operations, sensitive information, financial activity, or regulated processes.
Strong controls, testing, evidence, human intervention mechanisms, and continuing monitoring become increasingly important.
Critical autonomy
AI participates in mission-critical or high-impact processes with significant execution authority.
These systems should receive correspondingly rigorous governance and security treatment.
The precise categories will differ by enterprise.
The principle should remain consistent:
Governance intensity should increase with both autonomy and potential consequence.
Boards and executives need visibility into agentic AI
AI agent governance cannot remain entirely a developer concern.
As agentic systems acquire greater authority, they create issues relating to:
operational risk;
cybersecurity;
regulatory obligations;
financial controls;
customer outcomes;
workforce design;
and corporate accountability.
Boards do not need to oversee every agent.
They should, however, understand the organization’s material exposure to autonomous AI.
Executive reporting could eventually address questions such as:
How many material agents are operating?
Where are they deployed?
What is their maximum permitted authority?
Which processes involve autonomous execution?
What incidents have occurred?
How frequently do humans override agents?
How is agent performance measured?
Which critical systems do agents access?
This turns agent governance into part of enterprise risk management rather than a specialized AI initiative.
The strategic opportunity
Governance should not be viewed solely as a constraint on agentic AI.
Done well, it can become an enabler.
Enterprises are unlikely to grant significant authority to systems they cannot understand, monitor, constrain, or stop.
Strong governance can establish the institutional confidence required to move agents into increasingly consequential workflows.
The relationship may therefore become:
greater assurance → greater permissible autonomy → greater potential enterprise value.
This is particularly important in regulated industries, where the largest productivity opportunities may exist within processes that also carry substantial risk.
Governance enables organizations to establish clear boundaries within which automation can expand.
From AI governance to digital authority governance
The emergence of AI agents ultimately reveals something larger about enterprise technology.
Organizations are beginning to delegate limited forms of institutional authority to software systems capable of interpreting information and selecting actions dynamically.
That is fundamentally different from conventional automation.
Traditional software executes instructions developers explicitly defined.
Agentic systems can introduce greater discretion into how objectives are achieved.
Consequently, enterprises may eventually need a broader discipline that can be described as digital authority governance.
It asks:
Who—or what—has authority inside the enterprise?
What is that authority?
Who granted it?
Within what boundaries can it operate?
How is its use observed?
And how can that authority be withdrawn?
AI agents make these questions increasingly urgent.
Conclusion
Agentic AI represents a significant evolution in enterprise computing.
The technology is moving from systems that predominantly assist human decision-making toward systems that can increasingly participate in execution.
This transition has the potential to reshape productivity, workflows, software architecture, and organizational design.
But autonomy requires institutional discipline.
Enterprises should know what agents exist.
Agents should have identities.
Authority should be explicit.
Access should be limited.
Consequential actions should have appropriate controls.
Behavior should be observable.
Changes should trigger reassessment.
And accountability should remain clearly assigned to people and institutions.
The objective is not to eliminate agent autonomy.
It is to make autonomy deliberate, bounded, observable, and accountable.
The enterprises that establish these capabilities early will be better positioned to move beyond experimenting with agents and begin deploying them confidently within real institutional environments.
About New York AI Group™
New York AI Group™ is an artificial intelligence company focused on enterprise AI, research, technology, governance, and infrastructure. Based in New York, the firm works at the intersection of emerging AI technologies and institutional enterprise requirements, developing research and solutions that help organizations navigate the transition toward increasingly intelligent, responsible, and AI-enabled operations.