The systems are online.
Can you still trust them?
I am not reassured by a green dashboard. It tells me a system is running. A bad actor would ask a different question: can the organisation still trust what the system tells it?
The opportunity I see is quieter than a blackout. If confidence in an important automated workflow can be undermined, the organisation may stop itself while it tries to determine which decisions are reliable. That interruption is the centre of my scenario.
01 / WHAT THE EVIDENCE SAYS
AI-assisted intrusion is not just a thought experiment.
In a report published 13 November 2025, Anthropic described a campaign detected that September involving roughly 30 targets and a small number of successful intrusions. This is the vendor’s account, not AIAF’s independent forensic verification. Human operators remained involved, and the system also produced false claims and hallucinated credentials.
Read Anthropic’s report →Defensive readiness changes the outcome.
The UK NCSC’s assessment to 2027 expects AI to improve existing intrusion operations. Its warning about greater vulnerability of critical systems is conditional on mitigations lagging or remaining unchanged. It is not a statement that a particular disaster will happen.
Read the NCSC assessment →These are background sources, not breaking news. Neither establishes the specific cascade below or proves that attackers are currently planning it.
02 / THE BAD ACTOR’S OPPORTUNITY · ANALYSIS
Exploit the organisation’s trust—not necessarily its machinery.
Imagine a large organisation using AI-assisted systems to interpret supplier information and recommend business actions. Staff depend on a trusted information source. From an adversarial perspective, the attractive weakness is not simply the software. It is the organisation’s inability to operate confidently when that source becomes unreliable.
The opportunity: turn confidence in a shared workflow into a single point of disruption. If nobody can quickly prove which outputs remain sound, precaution becomes expensive.
03 / THE POSSIBLE MOVE · HIGH-LEVEL SCENARIO
Create doubt where consequential decisions are made.
A bad actor could seek to make information feeding an AI-assisted workflow unreliable, inconsistent or difficult to verify. The objective would be to leave defenders uncertain about which recommendations can still be used. Management may pause the affected process while teams reconcile records.
If several organisations depend on the same service, that uncertainty could spread beyond one company. Disruption could arise from precautionary shutdowns and manual recovery—not from an AI directly controlling physical machinery.
Defensive boundary: this describes the strategic objective and consequence, not a method for gaining access, selecting a target or executing an intrusion.
04 / WHY IT COULD WORK
Authority can grow faster than verification.
The conditions an attacker would want
- One information source influences many consequential decisions.
- Automation holds broader access than its documented role requires.
- Teams cannot independently reconstruct why a recommendation was made.
- The manual fallback exists on paper but has not been rehearsed.
Who could bear the cost?
Patients waiting on administrative services. Businesses waiting on deliveries or payments. Workers reconciling records under pressure. The harm depends on the workflow’s authority and whether usable alternatives exist.
05 / WHAT MAY HAPPEN
The organisation may interrupt itself to remain safe.
The immediate consequence may be suspended approvals, delayed services, manual verification and leadership uncertainty. The more organisations sharing the affected dependency, the harder it becomes to determine whether each discrepancy is local, systemic or malicious.
That is the leverage: even incomplete interference may impose a large recovery burden when trust cannot be restored quickly. This remains a hypothetical stress-test. I have no evidence that anyone is pursuing this particular scenario.
06 / WARNING SIGNS
Investigate patterns—not isolated anomalies.
Changes without provenance
Consequential recommendations or records change without a traceable source, accountable approval or reproducible explanation.
Authority outside the role
An automated system attempts or performs actions beyond its documented permissions or normal operating pattern.
Shared discrepancies
Multiple teams or organisations using the same supplier report unexplained inconsistencies around the same period.
Fallback failure
Teams discover that independent records, manual procedures or responsible decision-makers are unavailable when automation is paused.
Corroborate these signals. None alone proves an attack, and ordinary faults or poor data can create similar symptoms.
07 / WHAT YOU SHOULD DO NOW
Reduce the value of the opportunity.
If I had one question for your leadership team, it would be this: if you lose trust in an automated workflow tomorrow, who can stop it—and what keeps working? Put that question through an exercise, with named owners and a record of what failed.
A realistic 30-day exercise
- Name one important AI-assisted workflow and its accountable owner.
- Map the trusted information it depends on and the decisions it can influence.
- Identify which actions require independent human approval.
- Rehearse stopping the workflow and operating safely without it.
- Verify that records can be reconciled against independent evidence.
What could defeat this scenario?
- Narrow permissions and separation of safety-critical systems.
- Reliable data provenance and independent approval for consequential changes.
- Tested fallback processes that preserve essential services.
- Fast isolation of one supplier or workflow without stopping the whole organisation.
I want to see these controls demonstrated—not merely listed in a policy.
Responsibility belongs where authority sits. Leadership owns resourcing and continuity; security and operational teams investigate together. Employees should not be expected to solve systemic failures through personal vigilance alone.
08 / LIMITS & UNCERTAINTY
Thinking like an attacker is not evidence of an attack.
I would use the next 3–12 months, through 16 September 2027, as a preparedness window. That is not a forecast deadline. I assign no numerical probability and make no claim that this precise scenario has occurred. A problem is not automatically an attack, and an attack is not automatically AI-enabled.
The evidence supports concern about AI-assisted intrusion. It does not establish the likelihood of the cross-industry cascade I have described. Independent incident reports showing what AI actually did, which permissions mattered and whether safeguards worked would change my assessment. Evidence that organisations can isolate failures and keep operating would weaken my case for widespread disruption.
09 / IF NOTHING IS DONE · FICTIONAL EXAMPLE
Monday morning at an ordinary home-supplies company.
This company is fictional. The example combines ordinary business dependencies to show the scenario from beginning to end. It is not a report of a real incident and deliberately omits intrusion instructions.
Harbour Home Supplies is a regional company with six shops, a warehouse and 140 employees. Its AI-assisted operations system reads supplier updates, forecasts stock, recommends purchase orders and prioritises invoices for payment. Managers usually approve the system’s daily batch rather than checking every underlying record.
THE MISSING DEFENCESThe system relies on one supplier-data channel. Changes are not independently verified. Its permissions are broader than necessary. The company has never rehearsed running the warehouse without it, and its backup records depend on the same information.
The trusted information becomes unreliable.
A bad actor interferes with the supplier information feeding the workflow. The system continues operating normally from the employees’ point of view. No warning appears because the data arrives through a channel the company already trusts.
The automation spreads the error.
Stock forecasts change. Purchase recommendations favour products the warehouse does not need, while genuine shortages appear less urgent. Several supplier invoices are assigned unusual priorities. Managers approve the batch because every item still looks like a normal system recommendation.
Reality and the system stop matching.
Shop staff report unavailable products that the system lists as plentiful. The warehouse receives unexpected deliveries. Finance sees payment priorities that do not fit existing agreements. Each team initially treats its discrepancy as an isolated operational problem.
The company loses confidence.
Management can no longer determine which recommendations were valid. It suspends automated purchasing and invoice processing. Because no tested fallback exists, warehouse releases slow, supplier payments are held and shop replenishment becomes manual.
The disruption becomes a business crisis.
Customer orders are delayed. Suppliers demand clarification. Cash is tied up in unnecessary stock. Employees work overtime comparing emails, invoices and warehouse records. The company cannot restore yesterday’s trusted state because its backup reflects the same unreliable source.
The attacker did not need to switch the company off.
The company interrupted itself because it could not trust its own decisions. Revenue, supplier confidence and staff time were lost before anyone could determine which events were malicious, erroneous or legitimate.
The weakness was not simply “AI”. It was authority without independent verification, one trusted dependency, excessive permissions and an untested manual fallback. Any one of the controls described above could have reduced the reach or duration of the disruption.
No new ledger forecast is issued here. Our separately recorded predictions retain their original terms.
Explore the Forecast Ledger →