Research / 2026
Enterprise AI Operating Model: Strategy, Governance & Infrastructure
New York AI Group™ Research | 2026
Enterprise AI Research Series
Executive Summary
Artificial intelligence is moving beyond the boundaries of individual applications and becoming part of the operating infrastructure of the modern enterprise.
The challenge for organizations is therefore changing. Early enterprise AI programs were largely organized around experimentation: identifying use cases, acquiring models, deploying copilots, and demonstrating technical feasibility. The next phase requires something materially more difficult—creating an operating model capable of coordinating AI strategy, technology architecture, governance, data, security, workforce adoption, and business execution across the institution.
This paper defines the Enterprise AI Operating Model as the system of decision rights, organizational structures, technology capabilities, governance mechanisms, and operational processes through which an enterprise selects, deploys, controls, measures, and scales artificial intelligence.
The central conclusion is straightforward:
Enterprise AI will not scale sustainably through technology deployment alone. It requires an institutional operating system.
Three interconnected foundations sit at the center of that system:
Strategy determines where and why AI should be deployed.
Infrastructure determines how AI capabilities are technically delivered and integrated.
Governance determines under what conditions AI can operate and remain accountable.
These foundations must operate together.
A strong AI strategy without adequate infrastructure remains largely aspirational. Advanced infrastructure without governance can create uncontrolled operational exposure. Governance without strategy can become a compliance function disconnected from economic value.
The emerging enterprise AI model therefore requires organizations to integrate all three into a single management architecture.
This direction aligns with established institutional approaches to AI risk management. The U.S. National Institute of Standards and Technology’s AI Risk Management Framework organizes AI risk activities around Govern, Map, Measure, and Manage, emphasizing that governance should operate across the AI lifecycle rather than as a final approval activity.
The U.S. Government Accountability Office similarly organizes AI accountability around governance, data, performance, and monitoring, reinforcing the importance of connecting organizational accountability with technical operations.
The enterprise implication is significant: AI governance is increasingly becoming part of enterprise architecture itself.
1. Enterprise AI is becoming an operating-model question
Organizations rarely struggle to identify potential AI applications.
The harder problem is scaling them.
An enterprise may have dozens—or eventually hundreds—of AI initiatives spanning customer service, finance, software development, marketing, cybersecurity, analytics, operations, legal functions, research, supply chains, and employee productivity.
Without an operating model, these initiatives often develop independently.
Business units acquire different models.
Technology teams construct separate architectures.
Risk teams introduce controls after systems have already been designed.
Procurement manages AI vendors through conventional software processes.
Security teams discover new AI integrations after deployment.
Executives receive fragmented measures of AI investment and value.
This produces what might be described as AI fragmentation.
The problem is not necessarily that individual systems are poorly designed. The problem is that the enterprise lacks a common mechanism for coordinating them.
Traditional technology governance provides part of the solution, but AI introduces additional considerations.
AI systems may change behavior as models, prompts, data, retrieval environments and surrounding applications change. Outputs can be probabilistic rather than deterministic. Foundation models can support multiple downstream applications. Third-party models may change independently of the enterprise. Generative systems can produce new content, while agentic systems can increasingly perform actions.
These characteristics require an operating model capable of governing continuous systems rather than static deployments.
2. Defining the Enterprise AI Operating Model
An Enterprise AI Operating Model should answer seven fundamental questions.
Purpose
What business outcomes should AI support?
Authority
Who can approve, deploy, modify or retire AI systems?
Architecture
What technological foundations should those systems use?
Risk
How should AI applications be classified and controlled?
Operations
How are AI systems monitored after deployment?
Accountability
Who remains responsible for outcomes?
Measurement
How does the organization determine whether AI is creating value?
Taken together, these questions establish the institutional framework around the technology.
A mature model can be represented as:
Enterprise Strategy
↓
AI Portfolio & Investment Priorities
↓
Architecture & Infrastructure
↓
Data & Model Services
↓
AI Applications / Agents
↓
Governance & Control Environment
↓
Business Operations
↓
Monitoring, Evidence & Performance
↓
Executive Oversight
The important characteristic is not the precise structure.
It is the feedback loop.
Enterprise AI should not operate as a linear technology implementation process. Performance, risk information, operational incidents, model changes, user feedback and business outcomes should continuously influence subsequent decisions.
3. Pillar I — Strategy
Enterprise AI strategy begins with a distinction that is frequently overlooked:
AI strategy is not an inventory of AI projects.
A strategy establishes choices.
Organizations have finite capital, technical capacity, data resources, management attention, risk tolerance and workforce readiness. Attempting to apply AI everywhere simultaneously can produce extensive experimentation without corresponding economic value.
An effective enterprise AI strategy therefore determines where intelligence produces differentiated value.
This requires examining processes through several lenses:
Economic significance — What is the potential effect on revenue, cost, productivity or capital efficiency?
Decision intensity — Does the process involve substantial analysis, judgment or information processing?
Data availability — Does the enterprise possess data capable of supporting the application?
Technical feasibility — Can models perform the required task at acceptable performance levels?
Operational integration — Can the technology realistically be integrated into the underlying workflow?
Risk materiality — What could happen if the system performs incorrectly?
Scalability — Can the capability extend across functions, markets or customer segments?
These criteria shift AI planning away from demonstrations toward portfolio management.
4. Build capabilities, not isolated use cases
A common early-stage approach is to identify hundreds of potential AI use cases.
That exercise can be helpful, but it can also obscure an important architectural reality.
Multiple use cases frequently depend upon the same underlying capability.
Document intelligence may support legal, compliance, finance and procurement.
Enterprise search may support almost every knowledge-intensive function.
Prediction services may support operations, fraud, sales and financial planning.
Agent orchestration may eventually support multiple business processes.
Therefore, an advanced enterprise AI strategy should move from:
use-case portfolio
toward:
capability portfolio.
Consider five reusable capabilities:
- Enterprise knowledge retrieval
- Document intelligence
- Predictive intelligence
- Generative assistance
- Agentic workflow automation
Instead of constructing independent technology stacks for 50 individual applications, organizations can establish common platforms upon which numerous applications operate.
This produces architectural leverage.
It also makes governance substantially easier.
5. AI portfolio governance
Once opportunities have been identified, organizations require a systematic investment mechanism.
A useful AI portfolio can distinguish between three categories.
Run
AI used to improve existing operations.
Examples include employee assistance, service automation, document processing and software development support.
Transform
AI used to redesign major workflows or operating models.
Examples include automated claims workflows, AI-enabled financial analysis or intelligent supply-chain planning.
Reinvent
AI used to create products, services or business models that were previously impractical.
This distinction matters because the investment characteristics differ substantially.
A productivity assistant and an autonomous business process should not pass through identical investment or governance processes.
The enterprise AI operating model should therefore connect investment tier to risk tier.
High strategic significance does not inherently mean high risk, and high risk does not inherently mean high financial value.
Both dimensions must be evaluated.
6. Pillar II — Governance
The governance component determines how the enterprise establishes accountability for AI.
NIST defines governance as a cross-cutting component of AI risk management. Within the AI RMF, the Govern function establishes policies, processes, procedures and organizational structures that support risk management throughout the lifecycle.
This is a useful distinction.
Governance should not begin when an application is ready for production.
It begins when an AI opportunity is proposed.
A mature enterprise governance lifecycle could therefore follow:
Identification → Classification → Assessment → Control Design → Validation → Approval → Deployment → Monitoring → Change Management → Retirement
Every material enterprise AI system should have a traceable position within this lifecycle.
7. Establish an enterprise AI inventory
The foundation of governance is visibility.
An enterprise cannot govern systems it does not know exist.
The AI inventory should therefore become a core institutional record.
Depending on organizational requirements, it can include:
- system name;
- business owner;
- technical owner;
- business purpose;
- model provider;
- model/version;
- deployment environment;
- data categories;
- affected stakeholders;
- integration points;
- decision impact;
- autonomy level;
- risk classification;
- approval status;
- monitoring requirements;
- control evidence;
- vendor dependencies; and
- lifecycle status.
The inventory becomes more than a catalog.
It becomes the control plane for enterprise AI governance.
Executives can understand exposure.
Security teams can identify model connections.
Risk teams can monitor high-impact systems.
Procurement can identify vendor concentration.
Technology teams can identify architectural duplication.
Audit teams can access evidence.
This creates institutional visibility around an increasingly distributed technology environment.
8. Risk-tier AI rather than governing everything equally
Not every AI system requires the same level of oversight.
An employee summarizing a publicly available document does not create the same organizational exposure as an AI system involved in lending, healthcare decisions or autonomous financial transactions.
Governance therefore benefits from risk classification.
A simplified enterprise framework could include:
Tier 1 — Limited Risk
Low-impact productivity and internal assistance.
Controls emphasize data handling, acceptable use and basic monitoring.
Tier 2 — Moderate Risk
AI materially influences workflows but operates with meaningful human oversight.
Additional testing, documentation and performance monitoring become appropriate.
Tier 3 — High Risk
AI influences consequential decisions, regulated processes or sensitive information.
Independent review, extensive evidence and ongoing monitoring may be required.
Tier 4 — Critical / Autonomous
AI systems possess substantial operational authority or affect mission-critical processes.
These systems may require enhanced cybersecurity, continuous monitoring, human intervention mechanisms and senior-level approval.
The precise framework should depend on the organization’s industry, jurisdiction, risk tolerance and applicable obligations.
The underlying principle is proportional governance.
OECD guidance similarly emphasizes lifecycle-oriented accountability and risk management rather than assuming identical treatment for every AI system.
9. Decision rights
The most important governance question may be surprisingly simple:
Who is allowed to make which decisions?
An effective AI operating model establishes clear decision rights.
The board typically provides oversight of material enterprise risks and strategic direction.
Executive leadership establishes AI priorities and institutional risk appetite.
Business leaders own business outcomes.
Technology organizations own architectural integrity and operational technology.
Security organizations address cyber risk.
Legal and compliance functions interpret applicable obligations.
Data teams establish data controls.
Risk teams provide independent challenge where appropriate.
Internal audit may provide assurance regarding whether the governance system operates as intended.
The purpose is not to create a committee for every AI application.
It is to prevent accountability gaps.
Every material system should ultimately have an identifiable business owner and accountable executive.
10. Evidence becomes the foundation of assurance
Governance increasingly requires evidence.
A statement that an AI system is “responsible” provides relatively little institutional value.
An organization should be able to demonstrate why the system was approved.
Evidence might include:
- documented intended use;
- risk assessment;
- architecture documentation;
- data evaluation;
- performance testing;
- security assessment;
- human-oversight design;
- evaluation results;
- model/vendor documentation;
- approvals;
- monitoring metrics; and
- incident records.
This changes governance from policy administration to assurance infrastructure.
GAO’s accountability framework similarly emphasizes documentation, performance and continuing monitoring as important components of responsible AI oversight.
11. Pillar III — Infrastructure
Enterprise AI infrastructure is substantially broader than model access.
Organizations frequently begin AI programs by asking:
Which model should we use?
That may be the wrong architectural starting point.
Models are only one component of a much larger system.
A scalable enterprise AI environment increasingly includes at least eight technical layers.
1. Compute
Cloud or on-premise environments supporting inference, training and processing.
2. Model layer
Foundation models, specialized models, machine-learning models and embedding systems.
3. Data layer
Enterprise data stores, documents, knowledge bases and streaming sources.
4. AI platform services
Model gateways, model registries, evaluation services, retrieval infrastructure and orchestration.
5. Integration
APIs, event systems, application connectors and workflow platforms.
6. Security and identity
Authentication, authorization, secrets, access controls and data protection.
7. Observability
Logs, telemetry, performance metrics, model evaluation and cost visibility.
8. Applications and agents
The AI systems ultimately interacting with employees, customers and enterprise processes.
Governance sits across every layer.
12. The model gateway becomes an important architectural control
Large enterprises are unlikely to depend permanently on a single AI model.
Different models may be appropriate for different workloads based upon:
quality, cost, latency, privacy, deployment requirements, modality and risk.
This makes a centralized model-access layer increasingly useful.
Instead of hundreds of applications individually connecting directly to external models, an enterprise can route model requests through a controlled platform.
Such architecture can potentially provide:
authentication
Who can access which models?
routing
Which model should handle a particular workload?
logging
What model interactions occurred?
policy enforcement
Which requests or information categories are permitted?
cost management
Which business units are consuming model resources?
resilience
Can workloads move between providers?
model governance
Which models are approved?
This creates a technical enforcement mechanism for governance decisions.
The conceptual boundary between infrastructure and governance begins to disappear.
13. Data remains the strategic foundation
AI capabilities ultimately depend on information.
Publicly available models may become increasingly commoditized, but proprietary organizational knowledge remains differentiated.
This creates a strategic shift.
Competitive advantage may increasingly come not from possessing a model, but from the enterprise’s ability to combine models with:
- proprietary data;
- institutional knowledge;
- transaction history;
- operational information;
- customer context; and
- specialized domain expertise.
But more data does not inherently produce better AI.
Data must be relevant, sufficiently reliable, appropriately authorized and accessible under applicable controls.
GAO’s accountability framework explicitly treats data as one of its four fundamental AI accountability principles, emphasizing quality, reliability and representativeness.
Enterprise AI therefore increases rather than decreases the importance of data governance.
14. Retrieval becomes part of enterprise architecture
One of the most important architectural patterns for enterprise generative AI is retrieval.
Rather than expecting a foundation model to possess organizational knowledge, systems can retrieve relevant information from enterprise repositories and provide it as contextual material for model processing.
This creates significant opportunities.
But it also introduces new governance questions.
Who should be permitted to retrieve a particular document?
Should the AI inherit the user’s access permissions?
What happens when source documents conflict?
How is information freshness established?
How can outputs be traced back to source material?
Can sensitive information cross organizational boundaries?
Therefore, retrieval architecture should be integrated with identity, entitlement, metadata and data-governance systems.
15. Security must evolve for AI systems
Enterprise AI expands the technology attack surface.
Traditional cybersecurity remains essential, but AI introduces additional considerations involving:
model interactions, prompts, retrieval systems, third-party models, agent permissions, training data and generated outputs.
NIST’s Generative AI Profile was developed specifically to help organizations identify and manage risks that may be distinctive to generative AI systems while applying the broader AI RMF.
The key architectural principle is that AI should not become an exception to enterprise security.
It should inherit and extend established security disciplines:
identity
least privilege
segmentation
secure software development
data classification
logging
incident response
vendor risk
change management
As AI systems become more autonomous, these controls become increasingly important.
16. Agentic AI changes enterprise architecture
Generative AI primarily expanded what computers could produce.
Agentic AI could expand what software is allowed to do.
An agent may potentially:
retrieve data;
call an API;
create documents;
interact with applications;
initiate workflows;
coordinate other agents;
or execute multi-step processes.
That changes the architecture problem substantially.
Traditional software applications receive permissions.
AI agents may increasingly require delegated authority.
Organizations will therefore need mechanisms for establishing:
- agent identities;
- permitted actions;
- access boundaries;
- transaction limits;
- human-approval thresholds;
- audit trails;
- termination controls; and
- escalation processes.
An important future enterprise principle may therefore be:
Every autonomous AI agent should have an identity, an owner, a defined authority boundary and a complete activity record.
This will connect agentic AI directly with identity governance and cybersecurity architecture.
17. Human oversight should be engineered
“Human in the loop” is frequently used as a governance phrase.
It is not sufficient.
Human oversight must be operationally defined.
Organizations should determine:
Who intervenes?
Under what conditions?
What information does the reviewer receive?
Can the reviewer reverse the AI decision?
How quickly must intervention occur?
What happens when the human and AI disagree?
A human approval that exists only as a checkbox can provide little meaningful oversight.
Effective human control must be integrated into workflow architecture.
18. The AI lifecycle becomes continuous
Traditional enterprise software frequently follows a relatively recognizable sequence:
design → build → test → deploy → maintain
AI systems introduce a more continuous lifecycle.
discover → classify → design → evaluate → approve → deploy → observe → reassess → modify → retire
Why?
Because the surrounding system can change.
Models change.
Data changes.
User behavior changes.
Business conditions change.
Threats change.
Regulation changes.
And the purpose for which a system is being used may gradually expand.
NIST’s framework is deliberately lifecycle-oriented and describes risk management as an ongoing process rather than a single certification event.
Enterprise AI governance therefore needs continuous assurance rather than one-time approval.
19. Monitoring is a strategic capability
Once an AI system enters production, organizations should monitor more than uptime.
A mature monitoring environment can include:
Technical performance
Latency, availability and errors.
Model performance
Quality, accuracy or task-specific evaluation.
Risk indicators
Policy violations, inappropriate outputs or unusual behavior.
Data indicators
Changes in data distribution, source availability or data quality.
Security indicators
Unexpected access, misuse or anomalous agent behavior.
Economic performance
Cost, utilization, productivity and business value.
Human outcomes
Overrides, complaints, escalation and user feedback.
GAO similarly identifies monitoring as one of the four core components of AI accountability and emphasizes ensuring that systems remain reliable and relevant over time.
Monitoring is therefore not simply an engineering function.
It is an executive-management capability.
20. Measuring enterprise AI value
AI programs can become vulnerable to a familiar technology-management problem: extensive activity with limited economic measurement.
Number of pilots is not a value metric.
Number of employees with AI access is not necessarily a value metric.
Number of models deployed is not a value metric.
Value should connect technology adoption with business outcomes.
Depending upon the use case, organizations might measure:
Productivity
- cycle-time reduction;
- employee hours redirected;
- throughput.
Financial
- revenue improvement;
- operating cost reduction;
- avoided losses.
Customer
- response time;
- service quality;
- conversion.
Risk
- incidents;
- control effectiveness;
- error reduction.
Technology
- development velocity;
- system reliability;
- model cost.
Ultimately, the strongest enterprise AI portfolios will likely be those where technical, risk and financial metrics can be viewed together.
21. Organizational structure
There is no universal organizational chart for enterprise AI.
However, effective operating models generally require coordination between centralized and decentralized capabilities.
A fully centralized model can create consistency but become disconnected from business expertise.
A fully decentralized model can move quickly but produce duplication and inconsistent controls.
A federated structure often provides a useful alternative.
Enterprise AI function
Establishes common strategy, architecture, platforms, standards and measurement.
Business AI teams
Develop applications grounded in specific business requirements.
AI governance / risk function
Defines governance standards and provides oversight proportional to risk.
Shared platform engineering
Provides models, retrieval, evaluation, integration and observability capabilities.
Security and data functions
Provide controls that remain integrated with existing enterprise systems.
The objective is to centralize common infrastructure and institutional standards while allowing business capabilities to remain close to domain expertise.
22. The role of the board and executive leadership
Enterprise AI increasingly warrants senior-level oversight because it affects multiple dimensions simultaneously:
technology;
capital allocation;
cybersecurity;
workforce;
operational risk;
regulatory exposure;
and competitive strategy.
Boards do not need to approve individual prompts or models.
They do need sufficient visibility to understand whether management has established an adequate system of oversight.
Important board-level questions include:
Where is AI materially affecting the enterprise?
What risks are considered material?
Who owns those risks?
How does management know controls are operating?
What is the economic return from AI investments?
What would constitute a major AI incident?
How would management detect it?
The board’s role is therefore not technical administration.
It is institutional accountability.
23. A target Enterprise AI Operating Model
A mature target state can be represented as five interconnected layers:
Layer 1 — Enterprise Direction
Board oversight
Executive strategy
Risk appetite
Investment priorities
↓
Layer 2 — AI Portfolio
Use cases
Business cases
Prioritization
Risk classification
Funding
↓
Layer 3 — AI Platform
Models
Data
Retrieval
Integration
Agents
Infrastructure
↓
Layer 4 — Control Environment
Identity
Cybersecurity
Governance
Testing
Evidence
Human oversight
↓
Layer 5 — Operations
Deployment
Monitoring
Incident management
Performance
Value measurement
Continuous improvement
The strength of the operating model comes from the connection between layers.
Risk findings can change portfolio priorities.
Monitoring can trigger reassessment.
Architecture can enforce governance rules.
Business performance can influence investment.
Executive strategy can determine acceptable autonomy.
This is what makes an operating model different from an organizational chart.
24. Enterprise maturity model
New York AI Group proposes five broad maturity stages.
Level 1 — Experimental
AI activity occurs primarily through isolated pilots.
Infrastructure and governance are fragmented.
Level 2 — Coordinated
The enterprise introduces approved models, initial policies and common platforms.
Level 3 — Integrated
AI is connected with enterprise data, applications and workflows.
Formal risk classification and monitoring become established.
Level 4 — Governed
AI inventory, controls, evidence, validation, monitoring and accountability are institutionalized.
Level 5 — AI-Native
AI becomes deeply embedded across operations, with intelligent systems operating within defined enterprise control boundaries.
The objective should not necessarily be to reach Level 5 everywhere.
Different functions will mature at different speeds.
The objective is to create the institutional infrastructure capable of supporting whichever level is appropriate.
25. What enterprises should build now
For organizations moving from experimentation toward scale, eight capabilities warrant immediate attention:
1. Enterprise AI strategy
Establish a limited number of strategic priorities tied to measurable business outcomes.
2. AI inventory
Create institutional visibility into AI systems and their owners.
3. Risk classification
Establish proportional governance based on potential impact.
4. Shared AI platform
Reduce duplication through common model, retrieval, integration and evaluation capabilities.
5. Data architecture
Connect AI with reliable enterprise information under appropriate authorization.
6. AI security architecture
Apply identity, access, logging, cybersecurity and vendor controls to AI systems.
7. Continuous monitoring
Measure technical behavior, risk and economic performance.
8. Executive accountability
Make AI ownership explicit across business and technology leadership.
None operates independently.
Together, they form the beginnings of an enterprise AI operating model.
26. The convergence of strategy, governance and infrastructure
The most significant change may ultimately be organizational rather than technological.
Historically, strategy, technology and governance were frequently treated as separate disciplines.
Strategy determined what an organization wanted to accomplish.
Technology implemented systems.
Risk functions evaluated those systems.
AI increasingly compresses these decisions.
Model choice can affect risk.
Data architecture can determine business capability.
Governance requirements can determine system design.
Infrastructure choices can create strategic dependencies.
Agent permissions can become corporate authority decisions.
The disciplines therefore converge.
This is why the enterprise AI operating model should not belong exclusively to the CIO, CTO, chief data officer, risk function or business leadership.
It is inherently cross-enterprise.
27. Outlook
The enterprise AI market is likely to move through a predictable transition.
The first competitive advantage came from access to AI.
The next came from adoption.
The emerging advantage will come from integration.
Eventually, the durable advantage may come from institutional capability.
When AI tools become broadly available, simply possessing them provides limited differentiation.
Differentiation moves toward the organization’s ability to combine:
proprietary information + domain expertise + scalable infrastructure + governance + organizational execution.
This is the enterprise layer competitors cannot easily reproduce.
Conclusion
Artificial intelligence is becoming infrastructure.
But infrastructure alone is insufficient.
For AI to become a durable enterprise capability, organizations need an operating model connecting economic purpose with technological capability and institutional accountability.
That model rests on three foundations:
Strategy determines where intelligence creates value.
Infrastructure makes intelligence operational.
Governance establishes the boundaries within which intelligence can be trusted to operate.
The organizations that integrate these capabilities will be better positioned to move beyond fragmented experimentation.
They will be able to establish something more consequential:
an enterprise capable of systematically deploying, governing and improving artificial intelligence at scale.
That is the transition from adopting AI to operating with AI.
And it may become one of the defining institutional transformations of the decade.
About New York AI Group™
New York AI Group™ is an artificial intelligence company focused on enterprise AI, research, technology, governance, and infrastructure. Based in New York, the firm works at the intersection of emerging AI technologies and institutional enterprise requirements, developing research and solutions that help organizations navigate the transition toward increasingly intelligent, responsible, and AI-enabled operations.