TroutTrout
Back to Blog
Water and WastewaterOT SecuritySecure Remote AccessMFAThreat Intelligence

A Water Utility Caught CISA's Red Team in Two Minutes. The OT Jump Server Still Fell.

Trout Team12 min read

The short version

CISA published AA26-237A on August 25, 2026. It covers two red team assessments run at the same time, both requested by the organizations involved. Organization A is in the Government Services and Facilities Sector. Organization B is a water and wastewater utility.

Most of the coverage will land on the contrast: Organization A never noticed the red team, Organization B caught them fast. Both are true, and Organization B earned the credit. Its SOC isolated three compromised workstations in 10, 2, and 20 minutes and cut off the red team's command and control.

What happened after that is the part utilities should read. Even with that SOC, the red team took over the domain, got onto the jump server in the OT DMZ, and finished the assessment able to read every user's email from the public internet.

Look at which controls did what. The ones that worked were alerting, egress filtering and host isolation. All three caught the red team after a step had already succeeded. The ones that failed were about who is allowed to do what: a service account with rights over a domain controller, a password sitting in cleartext in an XML file, a machine account quota of 1,000, and a jump server that opened to a password borrowed from another service.

Detection was not the weak point at this utility. Access was.

What happened at the water utility

Getting in, and a SOC that worked

The red team collected employee email addresses from public websites (T1589.002) and sent spearphishing emails. Three people clicked (T1204), which gave the team three workstations.

Each payload set off a medium-severity alert with a dull name: "An executable file loaded an unexpected DLL file." SOC staff looked at all three and isolated the workstations within 10, 2, and 20 minutes.

That is the most useful detail in the advisory, and it has nothing to do with tooling. The alert that caught a federal red team was the kind most SOCs mute because it fires all day. It worked because someone read it and could pull a machine off the network without asking permission first. CISA's lessons-learned section makes the same point from the other side: untuned tools bury real alerts, and defenders without authority cannot act on the ones they see.

One thing got through before the isolation. From one workstation, the red team ran LDAP queries and pulled the whole domain: users, groups, computers, GPOs and the relationships between them. Twenty minutes was enough to get the map.

Assume breach, then the whole domain

With the foothold gone, the assessment switched to an assume breach model. Utility staff who knew about the test ran a red team payload on a chosen internal host, under a standard user account with no admin rights.

From there, still without exploiting anything:

  1. The machine account quota was 1,000 for every domain user. The team created a machine account with a name that looked legitimate, which gave them a domain account and a password they controlled. They used it to run tools from their own Linux box, proxied through the internal host (T1090.001), which put them outside the reach of the endpoint agent.
  2. A cleartext password in an XML file. The Active Directory data showed no obvious escalation path. Enumerating SCCM distribution points turned up an XML file with the cleartext password for a domain service account (T1552.001).
  3. That service account had AllExtendedRights over a domain controller, plus control over almost 1,000 other accounts. The team used a resource-based constrained delegation attack to get DCSync (T1003.006) and started dumping credentials with krbtgt, which is Golden Tickets for the whole domain.

Three findings, all of them about permissions and stored secrets. None of them is something monitoring would catch, which is why a good SOC did not.

The path into OT

The red team went looking for the OT network. They found an IT workstation with saved RDP files, one named ics-[redacted]-org. The filename told them which user had access to OT.

That workstation could reach a bastion host. CISA defines the term in the advisory: a jump box or jump server, "a specialized, highly secured system... that serves as the sole access point between a network segment (such as an internal IT network) and a protected internal network (like an OT environment)."

They logged into it using FTP credentials over SSH (T1021.004).

The only door into a water utility's control network opened to a password reused from a file transfer service, on a different protocol, with no second factor. CISA's next sentence: "At this point, they had visibility over the OT network."

What happened next is the good news, and it is worth naming which control did the work:

  • They dropped payloads on the host and ran them. The callback never reached them, because the host blocked outbound internet connections. Egress filtering, doing its job.
  • Running the payloads set off an alert, and SOC staff quarantined the host.

Both are real controls and both mattered. Both happened after the login. Nothing challenged the credential at the door.

The cloud, and the one control that held

The utility ran hybrid identity, with credentials syncing between on-premises AD and Entra ID. The red team used ADConnectDump to pull cleartext credentials for the on-premises MSOL account and the Entra Sync_ account. That account had no MFA. Its login triggered an automated Microsoft alert, and SOC staff blocked it.

They then abused Seamless SSO: DCSync a target user, use Rubeus to request SSO service tickets, proxy the traffic through the utility's own network so it looked like it came from a trusted IP, and open the Azure portal as any synced user without ever knowing a password.

Here CISA records the only hard stop in the assessment:

"If a user was set up to use MFA, then Entra ID would prompt the red team for a second factor during the sign-in process. Therefore, the red team was only able to log into any Entra ID account that did not have MFA enabled, which seemed limited to service accounts."

MFA held. Every account that had it stayed out of reach. The gap was the accounts nobody thinks of as accounts, which is the problem service accounts and industrial devices always present: the identities that cannot type a code end up without a second factor, and attackers know it.

The team eventually found an Entra application that could read, write and send mail for every user in the tenant. It was owned by an AD-synced account that had been disabled in AD. They re-enabled it, dumped its credentials, authenticated with Kerberos tickets and added a client secret to the application. From then on they could pull any user's email from the public internet.

The two lists

ControlWorked or failedType
Medium-severity DLL alert, triaged in 2 to 20 minutesWorkedDetection
Defenders allowed to isolate hosts without escalationWorkedResponse
Egress filtering on the OT DMZ bastion hostWorkedContainment
Entra risky-user and Graph API threshold alertsWorkedDetection
MFA on user accountsWorkedAccess
Machine account quota set to 1,000FailedAccess
Service account with AllExtendedRights over a DCFailedAccess
Cleartext service account password in an SCCM XML fileFailedSecrets
Standard users in privileged AD groups (found, not used)FailedAccess
Saved RDP file naming the OT jump hostFailedAccess
Bastion host login with reused FTP credentials, no MFAFailedAccess
No MFA on service accountsFailedAccess
Disabled AD account still owning a high-privilege Entra appFailedAccess

Five controls worked. Only one of them, MFA, prevented anything. The other four caught activity that had already succeeded. Everything else on the list is an access problem.

For a utility deciding where the next budget goes, that is the whole finding. Organization B already has what water utilities are usually told to buy: attentive monitoring and defenders who can act. It still ended the week with an attacker in the OT DMZ and a way to read every mailbox.

Why the jump server did not help

A bastion host is the standard answer to OT remote access, and it sits in nearly every architecture diagram that has thought about the problem at all.

It concentrates traffic through one place. That has real value, and it is why the egress filtering worked here: there was one host to configure and someone had configured it.

What it does not do is check who is connecting, limit what they can reach once they are through, or leave a record of what happened. All three gaps showed up:

  • Identity. The login worked with a password minted for a different service, and there was no second factor at the one boundary where a second factor matters most.
  • Scope. Getting on the host meant visibility over the OT network, not access to one device for one job. One password reached everything behind the jump box.
  • Evidence. What CISA describes is pieced together from red team notes and alerts. A recorded session would have shown exactly what was touched.

The vendor and third-party access problem has the same shape, and it is why remote access is still the biggest attack vector in OT. The mistake is treating a shared password on a jump server as if it were an access control.

There is a cheaper lesson in the RDP file. The red team never scanned for the OT network. A filename told them where it was. Saved connection files, bookmarks and mapped drives on IT workstations quietly document the route from IT into OT, and no segmentation project ever puts them in scope.

What to do this week

Cheapest first.

  1. Search IT workstations for saved RDP files, PuTTY sessions and VPN profiles that name OT hosts. This is an afternoon, not a project, and it will show you your real IT-to-OT paths faster than a network diagram will.
  2. Check your machine account quota. If it is not zero and you have no specific reason, set it to zero. Organization B's was 1,000. The Windows default is 10. Nobody chose either number.
  3. List your service accounts and check which have MFA. CISA found that the accounts without it were the ones the red team could still use. Where an account genuinely cannot carry a second factor, put one in front of the device instead.
  4. Grep your configuration management for secrets. The escalation here came from a cleartext password in an XML file on an SCCM distribution point. Check SCCM, Ansible, deployment shares and file servers.
  5. Audit extended permissions on domain controllers, AllExtendedRights first. One over-permissioned service account turned a standard user into full domain compromise.
  6. Review who owns your Entra applications, especially applications owned by accounts that are disabled or dormant in AD. That is how this assessment ended.
  7. Confirm your OT jump host blocks outbound internet. This is the control that actually stopped OT progression. Verify it rather than assume it.
  8. Then fix the door. Everything above hardens what sits behind the bastion host. None of it changes the fact that getting to OT took one reused password.

Where Access Gate fits, and where it does not

Start with what this advisory does not support. Access Gate would not have stopped the phishing, fixed the machine account quota, removed the password from the XML file, or prevented the domain compromise. Those are Active Directory and credential hygiene problems and they need that work. The utility's SOC did its job and does not need replacing.

The narrow claim is about one step: the login to the bastion host, and everything it opened up.

Access Gate is deployed beside the network the site already has rather than cut into it, so day one carries no production risk, and from there you steer the flows you want protected through it, one asset at a time. On the IT-to-OT boundary in this scenario, three things change:

  • A password on its own is not enough. Access is tied to a verified identity and challenged with a second factor at the gateway, including for accounts and devices that cannot carry MFA themselves. MFA was the only control here that stopped the red team rather than catching them, and it needs to sit at the boundary that matters, not only in the cloud tenant.
  • A session reaches one destination. Instead of visibility over the OT network, an authorized session is scoped to the asset and protocol the job needs. The reused FTP password would have reached one thing, if anything.
  • The session is recorded as it happens, which is also the compliance evidence an EPA sanitary survey or a state OT rule will ask for. For utilities working through New York's DOH Part 5 and DEC Part 6 requirements, proof of who reached what is a deliverable.

The limits are real. This does nothing against someone already inside an authorized session, it does not judge what an approved engineer chooses to do, and it does not replace the SOC that caught this in two minutes. What it removes is the case where one password, reused from an unrelated service, is all that stands between an office network and a water utility's controls.

Most small utilities do not have Organization B's SOC. SCADA remote access at a small water utility usually means nobody is reading alerts at 2 a.m. At those sites the access control is not one layer among several. It is the layer.


Source: CISA Cybersecurity Advisory AA26-237A, "A Tale of Two SOCs: Insights From Two Red Team Assessments" (August 25, 2026). CISA does not name either organization beyond its sector. All timings, techniques and quoted text are from the advisory.

FAQ

Frequently Asked Questions

What is CISA advisory AA26-237A?
"A Tale of Two SOCs: Insights From Two Red Team Assessments" is a Cybersecurity Advisory CISA published on August 25, 2026. It covers two CISA red team assessments run at the same time, both by request: one at a Government Services and Facilities Sector organization (Organization A) and one at a Water and Wastewater Systems Sector organization (Organization B). The red team used similar methods in both places and took over the domain in both. Organization A never noticed. Organization B caught and contained the first compromise within minutes.
How did CISA's red team reach the OT network at the water utility?
In three steps, without a single exploit. They found an IT workstation holding saved Remote Desktop files, one of them named "ics-[redacted]-org", which told them the user had remote access to OT. That workstation could reach a bastion host, the jump server acting as the only way into the OT environment. They logged into that bastion host over SSH using FTP credentials (MITRE ATT&CK T1021.004). CISA says that at that point they had visibility over the OT network.
What stopped the red team from going further into OT?
Two things, both after they were already on the host. The bastion host blocked outbound internet connections, so the callback from the payloads they dropped never reached them. Running those payloads then set off an alert, and SOC staff isolated the host. Nothing stopped the login itself.
Did the water utility do well or badly in this assessment?
Well on detection and response, and CISA says so. Each of the three initial payload executions raised only a medium-severity alert, and SOC staff isolated the affected workstations within 10, 2, and 20 minutes, cutting off the red team's command and control. Staff also caught and blocked suspicious cloud activity. That pushed the red team into an assume breach model. What is left over are findings about permissions and credentials, not monitoring.
What single control actually blocked CISA's red team?
Multifactor authentication. After abusing Seamless SSO to authenticate to Entra ID with Kerberos tickets, the red team could only log into accounts without MFA, which CISA says "seemed limited to service accounts." Every other control in the advisory caught the red team after a step had already worked. MFA was the one that stopped a step from working at all.
Is a jump server enough to secure OT remote access?
A jump server concentrates traffic through one place, which is useful. On its own it does not check who is connecting beyond a password, does not limit what they can reach once they are through, and does not leave a usable record of the session. In this assessment it was entered with a password from another service, with no second factor, and it gave immediate visibility over the OT network. To make it a real control you need every session tied to a verified identity, a second factor at the boundary, access scoped to one destination and protocol, and a recording.

Other Articles

ICS AdvisoriesOT Security

The ASE2000 Test Set: When IEC 60870-5-104 TLS Does Not Check the Certificate

CISA published ICSA-26-239-04 on August 27, 2026 for the Applied Systems Engineering ASE2000 V2 Communications Test Set. One flaw is an old log4net bug that came in with a bundled library. The other is an IEC 60870-5-104 TLS client that does not properly check certificate errors, which means the encrypted channel you turned on for compliance can be terminated by someone else. CISA lists the affected sectors as Chemical, Critical Manufacturing, Energy, and Water and Wastewater.

ICS AdvisoriesOT Security

Serial-to-IP Device Servers: Four CISA Advisories in Three Days

Between August 25 and August 27, 2026, CISA published four advisories against the same narrow device class: Update A to ICSA-26-069-02 for the Lantronix EDS lines, ICSA-26-237-06 for the Ebyte NE2-D11, then ICSA-26-239-05 for the Ebyte NA111-M and ICSA-26-239-01 for the Xiiaozet LK100W. One device class, three days, and a component that almost never appears in an OT asset inventory as a networked device.

ModbusOPC-UA

Real-Time PLC Data Streaming OPC-UA Modbus and Modern Integration Patterns

Getting live data off a PLC sounds simple until you have to do it without slowing the controller or opening a hole in the network. How OPC-UA and Modbus compare for real-time streaming.