Introduction
Electricity and water utilities already use statistical and machine-learning tools for demand forecasting, equipment health, leakage detection, energy optimisation, work prioritisation and customer operations. The decision boundary is now moving closer to physical control. An algorithm may rank corrective switching actions, propose pump schedules, adjust chemical dosing targets, coordinate distributed resources, identify an unstable operating state or recommend emergency load relief. In some architectures the output can enter supervisory control with limited human delay.
The resulting question is not whether the model is accurate on average. A utility needs evidence that the complete system behaves acceptably in the operating conditions that matter, including rare and abnormal conditions. That evidence must cover sensors, communications, training data, model, optimisation objective, integration code, deterministic controls, operator interface, procedures, cybersecurity, maintenance and change. A high test score cannot compensate for a missing safe state, an unclear handover or an untested failure mode.
The central conclusion is that safety-critical utility AI should be governed through an assurance case. The organisation states the safety claims it relies on, the argument connecting controls to those claims, and the evidence supporting the argument. Authority expands only when evidence expands. The AI remains inside a defined operating envelope, while independent protection and accountable people retain the ability to constrain, reject or stop its influence.
1 Define the controlled decision
The assurance process begins with the physical decision. “Use AI for grid operations” is too broad. A testable use case names the asset, action, cadence, consequence, data, controller, operator and fallback. Examples include recommending a day-ahead pump schedule within approved reservoir and pressure limits; adjusting a battery set point inside a deterministic protection envelope; or ranking remedial switching options for an operator who remains responsible for execution.
The decision definition should record who owns the plant, who owns the model, who is authorised to act, how quickly a person can intervene and which protection systems remain independent. It should identify affected customers, environmental constraints and regulatory obligations. The same model can carry different assurance obligations depending on whether it informs planning, advises an operator or actuates equipment.
Scope discipline prevents a pilot from acquiring authority through convenience. An advisory model connected to a dashboard can become a de facto controller when operators routinely accept its top recommendation. The assurance case therefore addresses actual operational reliance as well as formal system architecture.
2 Classify authority before assessing performance
Three authority classes are useful. Advisory AI supplies information or ranked options; a competent person checks the evidence and makes the decision. Constrained-control AI can influence a process inside deterministic limits; independent logic rejects commands outside the envelope. Safety-related AI performs or materially shapes a function whose failure can cause harm or widespread disruption; it requires the strongest assurance and may be unsuitable when evidence cannot support the claim.
Classification should consider credible consequence, time to intervene, reversibility, coupling and scale. A poor maintenance ranking may increase cost over months. An incorrect chemical-dosing command can affect water quality within minutes. An unstable dispatch action can propagate through an interconnected system. The highest credible consequence governs the design unless independent controls demonstrably prevent it.
Table 1. Utility AI authority classes and minimum assurance conditions
| Authority class | Typical role | Physical influence | Minimum controls | Permitted reliance |
|---|---|---|---|---|
| Analytical | Forecasting, anomaly triage, work ranking | Indirect and delayed | Data lineage, validation, user review, monitoring | Planning and prioritisation |
| Advisory | Recommended dispatch, switching or treatment response | Human executes recommendation | Decision evidence, trained operator, independent checks, recorded acceptance | Operator decision support |
| Constrained control | Set-point optimisation inside approved limits | Direct within deterministic envelope | Hard limits, independent protection, safe fallback, real-time monitoring | Automated operation inside validated domain |
| Safety-related | Function materially protecting people, assets or service continuity | Direct and consequential | Functional-safety lifecycle, diverse protection, independent assessment, regulator acceptance where required | Only within explicitly approved safety claim |
| Prohibited or deferred | Use lacks evidence, safe state or effective supervision | Consequence cannot be bounded | Redesign, reduce authority or retain deterministic control | No operational reliance |
The classification is an author-developed decision aid. Utilities should adapt it to applicable law, engineering standards and operating rules.
3 Build the hazard model around the physical system
Hazard analysis should start from unacceptable physical outcomes. For electricity, outcomes can include loss of stability, equipment damage, uncontrolled islanding, protection miscoordination, unsafe restoration or cascading service interruption. For water, outcomes can include unsafe quality, loss of pressure, contamination pathway, overflow, equipment damage or failure to maintain fire-flow and public-health requirements.
The analysis then asks how AI could contribute. Failure can arise from wrong input, missing input, incorrect objective, extrapolation, latency, software defect, cyber manipulation, operator confusion or unexpected interaction with another controller. The model can be individually “correct” while the system action is unsafe because the plant state, constraint or timing has changed.
A structured hazard log should link each hazard to initiating conditions, affected safeguards, detection, consequence, owner and required evidence. The log remains a living configuration item. Operational experience, incidents, near misses, model changes and plant modifications update it.
4 Establish the validated operating envelope
An AI system should have a declared region of validity. The envelope covers plant configuration, sensor ranges, data quality, weather, demand, asset condition, network topology, operating mode, latency and human staffing. A model tested on normal summer demand does not automatically support black-start, storm restoration, drought restriction or abnormal water-quality conditions.
The envelope needs machine-readable limits where possible. Runtime monitors compare current conditions with validated conditions. When the system leaves the envelope, authority should reduce automatically. A constrained controller may revert to advisory mode, freeze at a safe set point or hand control to deterministic logic. The transition itself requires testing.
Envelope design is also a commercial discipline. It exposes whether the proposed benefit depends on conditions that occur rarely or whether the model can support a material share of operations. It prevents a vendor demonstration from being interpreted as general operational fitness.
5 Use an assurance case rather than a compliance list
A compliance checklist can confirm that documents and processes exist. An assurance case explains why the operator believes the system is acceptably safe for a defined use. The top claim might state that the AI-assisted pump-control function cannot command operation outside approved hydraulic and water-quality constraints during the declared operating envelope. Supporting claims address data validity, model behaviour, command constraints, independent protection, human supervision, cybersecurity and change control.
Each claim needs an argument and evidence. Evidence can include requirements, design analysis, test results, simulation, hardware-in-the-loop trials, independent review, operator exercises, configuration records and monitored operational data. Weak evidence remains visible. A claim is not treated as proved because a policy uses reassuring language.
The assurance case also records defeaters: conditions that would undermine the claim. Examples include sensor drift beyond detection, communication latency above a threshold, an unassessed model update or a plant configuration absent from testing. Defeaters create stop rules and monitoring requirements.
6 Design the Utility AI Assurance Case
The proposed framework has eight linked claims. Purpose states the exact operational decision. Hazard identifies unacceptable outcomes and causal paths. Envelope defines validated conditions. Architecture bounds AI authority and preserves independent protection. Evidence tests data, model, integration and people. Operation monitors health, drift and incidents. Change controls updates and revalidation. Accountability records approval, residual risk and withdrawal authority.
The framework is author-developed. Authority expands only when the associated safety claims have current evidence.
7 Separate AI from deterministic protection
Independent protection is central to a credible architecture. A model that optimises performance should not be the only mechanism preventing unsafe voltage, pressure, temperature, dose, speed or flow. Deterministic interlocks, protection relays, safety instrumented functions and equipment limits should remain outside the AI decision path where practicable.
Independence requires more than separate code. Common sensors, communications, power supplies, configuration tools and personnel can create common-cause failure. The assurance case should identify shared dependencies and show how diversity, redundancy or procedural control addresses them. A fallback that uses the same corrupted input may provide little protection.
The design should also prevent the AI from modifying its own safety constraints. Optimisation objectives and hard limits belong to different governance layers. Changes to the constraint layer receive engineering approval and testing independent of model tuning.
8 Define a safe state and graceful degradation
A safe state is context dependent. Stopping a pump can protect equipment yet undermine pressure or firefighting capacity. Opening a breaker can isolate a fault yet destabilise an island. The design therefore defines safe states by operating mode, time horizon and system consequence.
Graceful degradation can be more appropriate than immediate shutdown. The system may reduce optimisation authority, use conservative set points, limit rate of change, require operator confirmation or transfer to a conventional controller. The transition should be deterministic, observable and rehearsed.
Recovery needs the same attention as fallback. The procedure identifies who diagnoses the issue, what evidence supports return to service, which configuration is restored and how accumulated operational changes are reconciled. Automatic recovery can be unsafe when the initiating condition is unknown.
9 Control data as a safety input
Training and runtime data are part of the safety argument. A dataset should have provenance, time coverage, operating-state coverage, known exclusions, quality rules and approval. Labels and derived features need documented methods. Synthetic data can extend testing, while its assumptions and limitations remain explicit.
Runtime data require plausibility checks, freshness limits, unit validation, time synchronisation and sensor-health monitoring. Cross-checks against independent measurements can reveal drift or compromise. Missing values should trigger a defined response rather than silent imputation in high-consequence use.
Utility data also carry security and confidentiality obligations. Grid topology, protection settings, water-system layouts and operational histories can be sensitive. Access, retention, supplier use and model-training terms should be controlled throughout the lifecycle.
10 Test the model at the boundary
Average-case accuracy is weak evidence for a safety-related function. Testing should concentrate on boundaries, transitions and rare combinations. Cases include high and low demand, abnormal topology, sensor disagreement, communications delay, equipment outage, extreme weather, maintenance mode, start-up, shutdown and restoration.
The test set should include conditions outside the envelope. The expected response may be abstention, reduced authority or fallback. A system that remains confident outside its knowledge can be more dangerous than one with lower average accuracy and reliable abstention.
Evaluation should be stratified by consequence. Missing an ordinary anomaly and missing an incipient instability are not equivalent. Metrics should report false negatives, false positives, calibration, latency and worst-case performance by hazard class.
11 Validate the integrated control loop
Model testing is only one layer. The integrated system includes data acquisition, preprocessing, model, optimiser, command path, plant interface, protection, display and operator. Errors can be introduced by unit conversion, sign convention, stale topology, time alignment, software integration or control saturation.
Simulation and digital twins can test operating scenarios before plant connection. Hardware-in-the-loop testing adds real controllers, communications and timing. Shadow mode allows the AI to produce recommendations while existing control remains authoritative. A controlled pilot can then apply limited authority within a narrow envelope.
Each stage has entry evidence and stop conditions. Progression reflects evidence from the complete loop rather than model maturity alone.
12 Use scenario coverage with explicit limits
Complete enumeration is impossible for complex utility systems. The assurance case therefore uses structured scenario selection. Hazards, operating modes, weather, asset states, communication failures, cyber events and human responses form the dimensions. Pairwise and risk-based combinations can improve coverage, while the highest-consequence interactions receive direct tests.
Coverage should be reported honestly. A large number of simulations does not prove relevance. The organisation records which hazard paths were exercised, which assumptions constrain the result and which regions remain untested. Independent reviewers should be able to reproduce critical cases.
Scenario libraries become controlled assets. Incident and near-miss evidence adds new cases. Plant changes and model updates trigger a review of affected scenarios.
13 Evaluate adversarial and cyber conditions
An AI system connected to operational technology introduces attack paths involving data poisoning, model substitution, adversarial inputs, credential theft, insecure interfaces, supply-chain compromise and manipulation of monitoring. Conventional OT threats remain relevant. The design should assume that an adversary may understand the model and exploit predictable transitions.
Threat modelling links attack paths to physical consequences. Tests should include spoofed measurements, replayed data, corrupted configuration, loss of communications, malicious files, privilege escalation and unavailable external services. The organisation records which controls detect, contain and recover from each path.
Security monitoring should observe model and plant behaviour without creating new remote-control risk. Logs need integrity, time synchronisation and retention suitable for incident reconstruction. Emergency access should be tightly controlled and rehearsed.
14 Assure third-party models and components
Utility AI often combines vendor models, open-source libraries, cloud services, edge devices and system integrators. The operator remains responsible for operational acceptance. Procurement should require architecture disclosure, model and data lineage, update policy, vulnerability handling, testing evidence, subcontractors, hosting, service continuity, export and secure deletion.
The supplier should notify the operator before material changes. Silent changes to a hosted model can invalidate testing. The contract should define version pinning, rollback, incident cooperation and access to evidence. The operator should retain enough information to reproduce material decisions and migrate away from the supplier.
Proprietary restrictions can limit assurance. When the operator cannot inspect necessary evidence, authority should be reduced or compensating controls strengthened. Commercial convenience cannot substitute for a supported safety claim.
15 Allocate human authority deliberately
Human oversight needs a defined task, adequate time and usable information. An operator cannot meaningfully supervise a fast automated action that cannot be understood before execution. In that case the safety argument must rest on architecture and independent protection rather than a nominal human-in-the-loop label.
For advisory use, the interface should show recommendation, supporting evidence, uncertainty, constraints, alternatives and expected consequence. It should make rejection easy and avoid pressure toward automatic acceptance. Operators should understand known failure modes and the conditions that require fallback.
Workload and automation bias should be tested. A model can reduce routine burden while increasing the difficulty of rare interventions. Training and drills should include incorrect confident recommendations, partial data, conflicting alarms and degraded communications.
16 Define decision latency and intervention time
Safety depends on time. The assurance case should distinguish sensing delay, communication delay, inference time, optimisation time, command execution, plant response and human intervention. Each hazard has a maximum tolerable response time.
Latency distributions matter more than an average. Tail delay can coincide with stressed networks or extreme events. The system should detect stale outputs and prevent execution after the relevant decision window. Time stamps, clocks and ordering become safety controls.
Where a person cannot intervene in time, the architecture should constrain the automated action and rely on independent protection. Where a person can intervene, drills should establish realistic response rather than assumed response.
17 Create evidence for explainability and traceability
Explainability should serve an operational purpose. An operator may need the variables driving a recommendation, the violated or active constraints, the relevant scenario and the reason the model abstained. An engineer may need feature behaviour, sensitivity and failure analysis. A regulator or investigator may need configuration, data, version and decision history.
A generic feature-importance chart may not answer these needs. The assurance case defines explanation requirements for each role and decision. Traceability links requirements to design, tests, model version, data version, deployed configuration and operating record.
The system should preserve the recommendation and the evidence available at the time. Later data should not rewrite the historical decision record.
18 Control model uncertainty and abstention
Model uncertainty has several sources: measurement noise, limited training coverage, stochastic behaviour, changing plant conditions and ambiguity between plausible states. A single confidence score can obscure these differences. The operator needs actionable uncertainty.
Abstention rules should correspond to hazards and fallback. The model can decline when inputs are missing, the state lies outside the envelope, models disagree, calibration deteriorates or monitoring detects a cyber anomaly. An abstention should create a clear operational path rather than an unexplained blank output.
The rate and context of abstention are monitored. A rising rate can signal drift, equipment change or data-quality failure. An unusually low rate can also be suspicious if the model continues to answer unfamiliar cases.
19 Monitor drift across data, model and system
Utility conditions change through asset ageing, network expansion, seasonal patterns, customer behaviour, climate, operating policy and sensor replacement. Data drift is only one dimension. Concept drift changes the relationship between observations and outcomes. System drift changes the plant or control architecture. Governance drift changes how people rely on the output.
Monitoring should cover input distributions, missingness, sensor health, performance against delayed ground truth, intervention, fallback, overrides, incidents and near misses. Thresholds should be tied to action. A dashboard without authority to reduce use creates weak control.
Monitoring also checks benefits. A safety case does not establish economic value. The operator separately measures verified improvement after review, control cost and operational burden.
20 Govern every material change
Models, prompts, data pipelines, libraries, hardware, sensors, control logic and plant configuration all change. The organisation needs a materiality rule. A change is material when it can affect a safety claim, operating envelope, performance, cybersecurity, human interaction or recovery.
Material changes trigger impact analysis, regression testing, scenario reruns, documentation and approval. Emergency patches receive controlled retrospective review. Versioning should allow rollback to a known configuration. Parallel deployment can compare behaviour before promotion.
Continuous learning deserves particular caution. A model that changes during operation can move beyond approved evidence. Safety-related use may require frozen versions and controlled offline retraining unless the assurance case specifically supports online adaptation.
21 Establish independent assessment
Independence strengthens challenge. The assessor should have sufficient technical, operational and safety competence and should not be solely accountable for delivering the model. The degree of independence should reflect consequence and regulatory context.
Assessment reviews claims, hazard analysis, architecture, evidence, unresolved limitations, supplier dependence, operator readiness and change control. It should sample raw evidence and reproduce critical tests. It should record disagreements and conditions rather than produce a generic approval statement.
Regulatory approval, certification or conformity assessment may be required depending on jurisdiction and use. The operator should map obligations early. A technical assurance case can organise evidence while it does not replace legal compliance.
22 Align with functional-safety practice
Functional-safety methods provide useful discipline: hazard-based requirements, safety integrity, independence, lifecycle control, verification, validation, competence and management of change. AI introduces challenges because behaviour is learned from data and can be difficult to specify exhaustively.
ISO/IEC TR 5469:2024 identifies contexts in which AI may sit inside a safety-related function, be constrained by non-AI safety functions or assist development of safety-related functions. The practical implication is architectural choice. A utility can often gain value by keeping AI in optimisation and diagnosis while deterministic functions enforce safety.
Where AI enters a safety-related function, the assurance case should explain how data-driven behaviour satisfies the required safety claims. Unsupported equivalence between a model metric and a safety-integrity target should be rejected.
23 Apply the framework to electricity operations
Electricity use cases include forecasting, state estimation, congestion management, voltage control, remedial action, distributed-resource coordination, outage restoration and asset protection. The grid is interconnected, dynamic and time sensitive. Local actions can create system effects.
The assurance envelope should include topology, generation mix, inverter behaviour, protection state, load, weather, communication and market conditions. Testing should include contingencies, oscillations, islanding, low-inertia conditions, restoration and erroneous telemetry. Independent protection and operator authority remain explicit.
The US Department of Energy has identified planning, permitting, operations and reliability, and resilience as areas where AI can support grid management. DOE also emphasises risk-aware guidance and testbeds. The assurance framework converts those high-level needs into evidence for a bounded operational decision.
24 Apply the framework to water operations
Water and wastewater use cases include demand forecasting, leak detection, pump scheduling, pressure management, dosing support, treatment optimisation, overflow prediction and maintenance. Public-health consequences and ageing operational technology make architecture and data quality central.
The envelope should include source-water condition, treatment train, reservoir level, network pressure, flow, weather, chemical availability, sensor health and laboratory confirmation. Controls should distinguish optimisation targets from regulatory and safety limits. Independent alarms, interlocks and manual procedures remain available.
EPA and WaterISAC guidance highlights the importance of OT inventories, access control, incident planning and safeguarding operational information. AI assurance should build on these controls rather than create a separate governance island.
25 Map the European regulatory boundary
The European Union Artificial Intelligence Act classifies certain AI systems used as safety components in the management and operation of critical digital infrastructure and the supply of water, gas, heating and electricity as high-risk. The classification and obligations depend on intended purpose and the applicable provisions.
High-risk obligations include risk management, data governance, technical documentation, records, transparency, human oversight, accuracy, robustness and cybersecurity. The assurance case can organise evidence across these areas. Legal analysis should confirm whether a specific use is covered, the responsible economic operator and the applicable timetable.
Regulatory classification should occur before procurement and architecture are fixed. A late conclusion that the system is high-risk can affect data, documentation, supplier contracts, conformity work and deployment schedule.
26 Build a claims and evidence register
The operating team needs a concise register connecting each claim to evidence, owner, status, expiry and defeater. Evidence should have a version and location. A test report without the tested configuration provides weak support. An approved claim can expire when the plant or model changes.
Table 2. Minimum claims and evidence register
| Safety claim | Evidence examples | Independent challenge | Runtime monitor | Withdrawal trigger |
|---|---|---|---|---|
| Inputs are valid for the decision | Sensor calibration, data-quality tests, time alignment, lineage | Data and instrumentation review | Freshness, plausibility, disagreement | Missing, stale or implausible critical input |
| Model remains inside validated envelope | Coverage analysis, boundary tests, scenario library | Independent model evaluation | Envelope and drift monitor | State outside validated region |
| Commands cannot breach hard constraints | Architecture review, formal checks, integration tests | Control and protection review | Constraint and interlock status | Failed or bypassed protection |
| Human supervision is effective | Interface tests, competence records, drills | Human-factors review | Overrides, response time, workload | Intervention no longer timely or reliable |
| Cyber compromise is detected and contained | Threat model, penetration test, access review, recovery exercise | Independent security assessment | Identity, integrity and anomaly telemetry | Suspected compromise or untrusted configuration |
| Change remains controlled | Version record, regression suite, approval | Configuration audit | Deployed-version attestation | Unapproved model, data or software change |
Evidence depth should reflect consequence, authority and jurisdiction. The entries show structure, not universal acceptance criteria.
27 Set stage gates for deployment
The first stage defines purpose, hazards, authority and owners. The second prepares data, architecture, fallback and evidence requirements. The third validates model and integrated system in simulation. The fourth runs shadow mode. The fifth permits a constrained pilot. The sixth expands authority only after monitored evidence and independent review.
Each gate should have stop conditions. Missing hazard ownership, weak boundary performance, unavailable fallback, excessive operator burden, unresolved cyber exposure or inability to reproduce a result should pause progression. Commercial deadlines should not silently waive a safety claim.
Table 3. Stage-gated assurance and deployment roadmap
| Gate | Core work | Required evidence | Authority after approval | Stop condition |
|---|---|---|---|---|
| Define | Decision, hazard, consequence, owner | Approved scope and hazard register | Research only | No accountable risk owner |
| Prepare | Data, envelope, architecture, fallback | Traceable requirements and test plan | Offline development | Safe state or independent protection absent |
| Validate | Boundary, scenario, cyber and integration tests | Reproducible results against criteria | Shadow operation | Material scenario fails or cannot be tested |
| Pilot | Limited assets and operating envelope | Operator drills, monitoring, incident readiness | Constrained operational influence | Unsafe behaviour, weak supervision or excessive fallback |
| Assure | Independent assessment and residual-risk decision | Claims, evidence, limitations and approvals | Approved bounded use | Material claim lacks evidence |
| Operate | Monitor, learn, maintain and review | Current assurance case and change record | Continued use inside envelope | Drift, incident or unapproved change |
The roadmap is author-developed and should be adapted to the utility's safety-management system and regulatory obligations.
28 Quantify assurance effort without invented precision
Boards and financiers need a view of cost, schedule and residual risk. The assurance plan should identify effort for data preparation, simulation, hardware-in-the-loop testing, cybersecurity, independent review, operator training, monitoring and lifecycle maintenance. Benefits should be measured separately through verified operational outcomes.
Scenario models can compare authority choices. A narrow advisory deployment may deliver a smaller benefit with lower evidence cost and faster approval. Constrained control may deliver more value while requiring independent protection and stronger testing. Safety-related control may be uneconomic when evidence and liability exceed the incremental benefit.
Values are hypothetical author-developed index points. They do not represent observed cost, risk reduction or utility performance.
29 Connect assurance to financing and transactions
AI assurance can affect valuation, financing, insurance, warranties, covenants and integration. An acquirer should understand which operating gains depend on AI, whether the evidence transfers, which supplier rights survive change of control and what cost is required to maintain the assurance case. A lender may focus on service continuity, control authority, cyber exposure and fallback.
Diligence should request the hazard register, architecture, model and data versions, test evidence, incidents, overrides, supplier terms, regulatory analysis and management-of-change history. Claimed savings should be reconciled to operating records after assurance costs. Unproven automation benefits should remain scenario assumptions.
Transaction documents can allocate responsibility for undisclosed incidents, unsupported performance claims, licence continuity and remediation. The integration plan should preserve approved configurations and operator competence during ownership change.
30 Create a ninety-day implementation plan
Days one to fifteen should select one bounded use case and define its decision, hazards, authority and owners. Days sixteen to thirty should map architecture, data, independent protection and fallback. Days thirty-one to fifty should build the scenario library, acceptance criteria and claims register. Days fifty-one to seventy should test in simulation and hardware-in-the-loop where appropriate. Days seventy-one to ninety should run shadow mode, operator exercises and an independent readiness review.
The ninety-day decision is whether to begin a constrained pilot, narrow the use or stop. The pack should include evidence, unresolved limitations, monitored indicators, supplier obligations, expected assurance cost and withdrawal authority.
31 Monitor value and safety together
Safety performance and commercial value should share a governance cadence while remaining distinct measures. Safety indicators can include envelope breaches, failed constraints, fallback, overrides, incidents, near misses, test failures and overdue evidence. Value indicators can include verified energy reduction, loss reduction, avoided downtime, forecast improvement or maintenance productivity.
A benefit should be reported after the cost of review, monitoring, data, integration and fallback. A model that saves energy while generating frequent operator intervention may have weak net value. A safe system that delivers no material operational improvement may remain unsuitable for scale.
Values are hypothetical author-developed assumptions. They illustrate a gate design and do not predict approval, safety or financial performance.
32 Recognise common failure modes
Common failures include treating historical accuracy as safety evidence, testing only the model, omitting abnormal states, relying on a person who cannot intervene in time, sharing dependencies between AI and fallback, accepting vendor opacity, allowing silent updates, or defining a safe state without testing its system consequence.
Governance also fails when claims, versions and decisions cannot be traced, a risk score triggers no action, or operator sign-off lacks tested training and workload. Authority without a current assurance claim is the strongest warning signal.
33 Limitations
This paper does not establish that a particular model, supplier, control system or use case is safe, secure, compliant or commercially attractive. Fitness depends on the asset, architecture, operating conditions, evidence, people, jurisdiction and consequence. Qualified professionals should interpret the cited standards and guidance.
AI output is not an engineering, safety, cybersecurity, legal, regulatory, accounting or investment conclusion. The framework does not replace the utility safety-management system, functional-safety lifecycle, regulator, system operator, engineer or accountable executive. The figures and tables use hypothetical author-developed assumptions; they do not represent measured cost, accuracy, probability, risk reduction, performance or expected return.
No finite test programme covers every future condition. Where a safety claim lacks support, the organisation should reduce AI authority, strengthen independent protection, redesign the use case or defer deployment.
Conclusion
Artificial intelligence can support faster and more consistent utility decisions. Physical influence requires an assurance method connecting each material claim to evidence, architecture and accountable authority. The Utility AI Assurance Case defines the decision, identifies hazards, fixes the operating envelope, separates optimisation from independent protection, tests the control loop, makes uncertainty visible and governs change. It supports a practical sequence from offline analysis to shadow mode, constrained pilot and bounded operation. Authority grows with evidence and can be withdrawn when evidence weakens, giving operators, boards, regulators, insurers, lenders and investors a common record of permitted use and stop conditions.
References
- National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework 1.0, 2023. https://doi.org/10.6028/NIST.AI.100-1
- National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, 2024. https://doi.org/10.6028/NIST.AI.600-1
- National Institute of Standards and Technology, Concept Note: Trustworthy AI in Critical Infrastructure Profile, 2026. https://www.nist.gov/system/files/documents/2026/04/08/Draft%20Concept%20Note_%20Development%20of%20the%20NIST%20AI%20RMF%20Trustworthy%20Use%20of%20AI%20in%20Critical%20Infrastructure%20Profile.pdf
- National Institute of Standards and Technology, AI RMF Playbook. https://airc.nist.gov/airmf-resources/playbook/
- National Institute of Standards and Technology, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, NIST AI 100-2e2025, 2025. https://doi.org/10.6028/NIST.AI.100-2e2025
- National Institute of Standards and Technology, Guide to Operational Technology Security, SP 800-82 Revision 3, 2023. https://doi.org/10.6028/NIST.SP.800-82r3
- International Organization for Standardization, ISO/IEC TR 5469:2024 Artificial intelligence - Functional safety and AI systems. https://www.iso.org/standard/81283.html
- International Electrotechnical Commission, IEC 61508 Functional safety of electrical, electronic and programmable electronic safety-related systems. https://www.iec.ch/functional-safety
- International Organization for Standardization, ISO/IEC 42001:2023 Artificial intelligence management systems. https://www.iso.org/standard/81230.html
- International Organization for Standardization, ISO/IEC 23894:2023 Artificial intelligence risk management. https://www.iso.org/standard/77304.html
- International Organization for Standardization, ISO/IEC 5338:2023 AI system life cycle processes. https://www.iso.org/standard/81118.html
- International Electrotechnical Commission, IEC 62443 Industrial communication networks - Network and system security. https://www.iec.ch/blog/understanding-iec-62443
- European Parliament and Council, Regulation (EU) 2024/1689, Artificial Intelligence Act. https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng
- European Parliament and Council, Directive (EU) 2022/2555, NIS 2 Directive. https://eur-lex.europa.eu/eli/dir/2022/2555/oj
- European Parliament and Council, Directive (EU) 2022/2557 on the resilience of critical entities. https://eur-lex.europa.eu/eli/dir/2022/2557/oj
- US Department of Energy, AI for Energy: Opportunities for a Modern Grid and Clean Energy Economy, 2024. https://www.energy.gov/sites/default/files/2024-04/AI%20EO%20Report%20Section%205.2g%28i%29_043024.pdf
- US Department of Energy, Potential Benefits and Risks of Artificial Intelligence Use in Critical Energy Infrastructure, 2024. https://www.energy.gov/ceser/articles/doe-delivers-initial-risk-assessment-artificial-intelligence-critical-energy
- US Department of Energy, Mjolnir AI Testbed, 2026. https://www.energy.gov/ceser/articles/ceser-and-lawrence-livermore-national-laboratory-launch-ai-testbed-strengthen-energy
- US Department of Energy, Grid Cybersecurity and Communications. https://www.energy.gov/oe/grid-cybersecurity-and-communications
- Cybersecurity and Infrastructure Security Agency, Mitigating Artificial Intelligence Risk: Safety and Security Guidelines for Critical Infrastructure Owners and Operators, 2024. https://www.cisa.gov/resources-tools/resources/mitigating-artificial-intelligence-risk-safety-and-security-guidelines-critical-infrastructure
- Cybersecurity and Infrastructure Security Agency and UK National Cyber Security Centre, Guidelines for Secure AI System Development, 2023. https://www.cisa.gov/news-events/alerts/2023/11/26/cisa-and-uk-ncsc-unveil-joint-guidelines-secure-ai-system-development
- UK National Cyber Security Centre, Guidelines for Secure AI System Development, 2023. https://www.ncsc.gov.uk/collection/guidelines-secure-ai-system-development
- UK Government, AI Cyber Security Code of Practice, 2025. https://www.gov.uk/government/publications/ai-cyber-security-code-of-practice
- UK National Cyber Security Centre, Machine Learning Security Principles. https://www.ncsc.gov.uk/collection/machine-learning-security-principles
- UK National Cyber Security Centre, Cyber Assessment Framework. https://www.ncsc.gov.uk/collection/cyber-assessment-framework
- US Environmental Protection Agency, Guidance on Improving Cybersecurity at Drinking Water and Wastewater Systems, 2024. https://www.epa.gov/system/files/documents/2024-08/epa-guidance-on-improving-cybersecurity-at-drinking-water-and-wastewater-systems-1.pdf
- US Environmental Protection Agency, Securing the Future of Water: Addressing Cyber Threats Today, 2025. https://www.epa.gov/system/files/documents/2025-08/water-cybersecurity-recommendations_water-sector-cybersecurity-task-force_apr25-072525.pdf
- US Environmental Protection Agency, Assessing If a Water and Wastewater System Has Operational Technology. https://nepis.epa.gov/Exe/ZyPURL.cgi?Dockey=P1019U0W.txt
- WaterISAC, Cybersecurity Fundamentals for Water and Wastewater Utilities, updated 2025. https://www.waterisac.org/fundamentals
- WaterISAC and US Environmental Protection Agency, National Security Information Sharing Bulletin Q3 2026. https://www.waterisac.org/tlpclear-waterisac-epa-national-security-information-sharing-bulletin-q3-2026
- Institute of Electrical and Electronics Engineers, IEEE 7000-2021 Model Process for Addressing Ethical Concerns during System Design. https://standards.ieee.org/ieee/7000/6781/
- Institute of Electrical and Electronics Engineers, IEEE 1547-2018 Interconnection and Interoperability of Distributed Energy Resources. https://standards.ieee.org/ieee/1547/5915/
- North American Electric Reliability Corporation, Reliability Standards. https://www.nerc.com/pa/Stand/Pages/Default.aspx
- Federal Energy Regulatory Commission, Reliability Standards for the Bulk-Power System. https://www.ferc.gov/industries-data/electric/electric-reliability/reliability-standards
- International Atomic Energy Agency, Computer Security for Nuclear Security, NSS No. 42-G, 2021. https://www.iaea.org/publications/13629/computer-security-for-nuclear-security
- International Atomic Energy Agency, Artificial Intelligence for Accelerating Nuclear Applications, Science and Technology, 2022. https://www.iaea.org/publications/15198/artificial-intelligence-for-accelerating-nuclear-applications-science-and-technology
- International Organization for Standardization, ISO 31000:2018 Risk management guidelines. https://www.iso.org/iso-31000-risk-management.html
- International Organization for Standardization, ISO 55001:2024 Asset management systems. https://www.iso.org/standard/83054.html
- International Society of Automation, ISA/IEC 62443 Series of Standards. https://www.isa.org/standards-and-publications/isa-standards/isa-iec-62443-series-of-standards
- MITRE, ATLAS Adversarial Threat Landscape for Artificial-Intelligence Systems. https://atlas.mitre.org/
- Open Web Application Security Project, Machine Learning Security Top 10. https://owasp.org/www-project-machine-learning-security-top-10/
- Organisation for Economic Co-operation and Development, Recommendation of the Council on Artificial Intelligence, amended 2024. https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449
- US Government Accountability Office, Artificial Intelligence: An Accountability Framework for Federal Agencies and Other Entities, 2021. https://www.gao.gov/products/gao-21-519sp
- UK Government Office for Science, Future Risks of Frontier AI, 2023. https://www.gov.uk/government/publications/future-risks-of-frontier-ai
- International Energy Agency, Digitalisation and Energy, 2017. https://www.iea.org/reports/digitalisation-and-energy
- International Energy Agency, Electricity Grids and Secure Energy Transitions, 2023. https://www.iea.org/reports/electricity-grids-and-secure-energy-transitions
- World Bank, Utility of the Future: Taking Water and Sanitation Utilities Beyond the Next Level, 2021. https://openknowledge.worldbank.org/entities/publication/7d493bf6-6705-53e4-90d8-e41e6f94c36b
- World Bank, Digital Water: Operational Digital Tools for Water Utilities. https://www.worldbank.org/en/topic/water/publication/digital-water
About the Author
Chennakeshav (CK) is a corporate finance and investment banking executive with 25+ years of global experience in deal origination, structuring and execution across M&A, growth capital and corporate strategy. He has led value-creation mandates for founders, corporates and funds — bridging the boardroom view to hands-on execution and close.
His career spans Morgan Stanley, HSBC, Lloyds Banking Group, EWEC, ADQ portfolio companies and Emirates Growth Fund, across TMT, real estate, fintech, deeptech, cleantech, infrastructure and energy. He has partnered with C-suite leaders, private equity and venture funds, sovereign wealth funds and family offices to finance complex fund raises and scale-up ventures, and has led M&A due diligence, post-merger integration and business-transformation initiatives to create value.
At Matchpoint Partners he is Managing Partner, leading the firm's corporate finance, M&A and capital-raising practice. He holds an MBA from London Business School, an engineering degree from VTU and a Master of Laws (LLM, in progress) from UCL London.
An active start-up mentor, CK mentors at Techstars, DIFC FinTech Hive, Startup Grind, Founder Institute and IN5, serves as Entrepreneur Mentor in Residence (EMiR) at London Business School, and judges the Entrepreneurship World Cup.
