Logo
Home
Archive
Tags
About
Search
Sign in
Get The Context
Logo
  • Home
  • Posts
  • Who checks the system that checks everything?

Governance

Ecosystems

Trust

Who checks the system that checks everything?

When a trusted system becomes a not-so-trusted system

Who checks the system that checks everything?
David Hawks
David Hawks

21 October 2025

Share


The Australian Signals Directorate's Cyber Security Centre recently issued an advisory covering an incident affecting F5 systems and multiple high-severity vulnerabilities.

Another vulnerability affects a network management tool? Not so fast...the challenge is that F5 products often sit in network-management and application-delivery paths. The advisory therefore created an awkward problem for customers: a supplier whose technology helps control and inspect other systems was itself asking organisations to apply fixes and review its incident guidance.

The advisory did not establish that every customer environment was compromised. It did, however, expose a dependency that I think needs some more attention. When a trusted platform becomes not-so-trusted, how can a customer start to establish what's happening and who to trust?

‍

The Control Layer is Still a Dependency

The ASD advisory directed organisations to F5's incident article and October security notification, and recommended fixed versions or engineering hotfixes for affected products. That remediation came first.

The architectural question outlasts the patch. A security platform may provide the telemetry, identity controls and enforcement through which an organisation judges other systems. If that platform is under investigation, a second dashboard does not necessarily provide an independent view. It may draw from the same appliance, provider or telemetry pipeline.

Independence is therefore a property of the failure path, not the number of screens in the operations centre. A separate signal earns its place when it can disagree for a reason: different data, different credentials, different infrastructure or a customer-held record that remains available when the primary system does not.

Enterprise Platforms Are Giving Agents More to Do

This week, several major software vendors announced that AI agents are now inside their core business applications.

On 13 October, Salesforce introduced Agentforce 360 as part of its “Agentic Enterprise” direction. On 15 October, Oracle announced agents within Fusion Cloud Applications for work across finance, human resources, supply chain, sales, marketing and service.

These were vendor announcements.

They did not prove adoption, safety or business returns. They did make the intended shift explicit: enterprise platforms were moving from holding information and supporting decisions towards completing more work inside the applications where records, money and operational processes already live.

An agent may update a record, trigger another workflow or restrict access before a person sees the result.

That changes what evidence a customer needs to keep. A report can be checked before a person acts on it. An agent may update a record, trigger another workflow or restrict access before a person sees the result. The useful record is no longer only what the platform concluded, but what the agent received, which rule applied and what changed.

A provider log may be part of that evidence. If it is the only evidence, the customer inherits the same circularity exposed by the F5 incident: the system supplies both the action and the proof that the action was sound.

A Score Acquired the Weight of an Accusation

On 20 October, ABC News reported that more than a dozen Australian universities were using AI-detection tools amid disputes about incorrect allegations of academic misconduct.

The reporting included students who said original work had been flagged and described the financial, academic and personal cost of challenging the result. The universities that responded said they did not use AI detection as the sole evidence of misconduct. Australia's higher-education regulator, TEQSA, told the ABC that the tools were allowed but not recommended on their own.

That nuance is important. The story does not establish a universal error rate for AI detectors, and university assessment is not enterprise security. It shows what happens when a score moves from an indicator to a process with real impact.

A score is an output. A finding is a judgement supported by evidence that a person can inspect, an owner who can defend it and a route through which contrary evidence can change the result. Adding a human reviewer is poor protection if the reviewer sees only the score and has no practical way to test the basis of the findings.

Contestability has to exist before the accusation, transaction block, or the "access denied message" appears. An appeal process created after the output has already acquired institutional authority is a repair mechanism, not an independent check.

Final Perspective

I'd focus on the point where a platform's output acquires authority over a consequential decision. A second dashboard drawing from the same appliance would not convince me that F5's state had been checked independently. Nor would a nominal human review convince me if the reviewer could see only an AI-detection score and had no evidence with which to disagree.

What matters is a record or signal from a different failure path, available to someone who can pause or overturn the result. As enterprise agents gain the ability to change records and trigger workflows, that standard becomes more valuable: the system performing the action should not also be the only witness to whether the action was authorised.

Recent
What your AI access allows
The pain of revoking exposed credentials
A hotel-search became executable Python
Model finds signs before human diagnosis

Spotlight

Read next

Stay in the Loop

Get The Context

Regular briefings on developments in AI and security, and what they mean for our trust in technology.


What it takes to trust AI

Pages

Home
Archive
Tags
About
Contact

Tools

Login
Update Password
Reset Password

Mechanism first Consequence second No noise
Security in Context Practical Analysis Trust in AI
© 2026 Synoptikon.
beehiivPowered by beehiiv