A Framework, Not a Sales Pitch
Evaluating OT security vendors is harder than evaluating IT security vendors. The OT market is fragmented, terminology is inconsistent, and many products solve only part of the problem. A monitoring tool is not a segmentation tool. A firewall is not a zero-trust platform. A cloud dashboard is not an on-premise solution.
This post provides a structured scoring framework you can adapt to your environment. Use it as-is for RFI responses, or customize the weights based on your priorities.
The 8 Criteria That Matter
1. Deployment Model
Where does the product run? Cloud-only, on-premise, or hybrid?
For OT environments, on-premise deployment is often non-negotiable. Air-gapped networks cannot reach cloud consoles. Regulated environments may prohibit sending network telemetry off-site. Ask whether the product functions fully without any internet connectivity.
2. Enforcement Capability
Does the product monitor, enforce, or both?
Monitoring means passive observation — the product sees traffic and generates alerts. Enforcement means active control — the product blocks unauthorized connections, segments zones, and enforces access policies. Many OT security products only monitor. They require separate firewalls or NAC appliances to act on their findings. We explore this gap in detail in our TCO comparison of appliance vs cloud OT security solutions.
3. Protocol Awareness
Does the product understand industrial protocols natively?
OT networks run Modbus TCP, EtherNet/IP, PROFINET, OPC UA, DNP3, BACnet, and dozens of vendor-specific protocols. A product that treats all of these as generic TCP traffic cannot make intelligent security decisions. Ask for the specific protocol list and what level of parsing is supported (header only vs. deep packet inspection).
4. Air-Gap Support
Can the product operate in a fully isolated network with zero internet access?
This means no cloud license validation, no cloud-hosted management console, no automatic update dependency. The product must store all data locally, run its management interface locally, and accept manual updates.
5. Compliance Coverage
Does the product map its controls to specific compliance frameworks relevant to your industry?
- CMMC 2.0 — Defense supply chain
- NIS2 — EU critical infrastructure
- IEC 62443 — Industrial automation security
- NIST 800-82 — Guide to OT security
- ISO 27001 — Information security management
Ask for a controls mapping document. If the vendor can't produce one, their compliance claims are marketing, not engineering.
6. Pricing Model
How does the pricing scale? Per device, per user, per site, per appliance?
Per-device or per-asset pricing punishes network growth. Every new PLC or sensor increases your security cost. Per-appliance or per-site pricing is predictable — you know the cost regardless of how many devices sit behind the appliance.
7. OT Expertise
Is the vendor an OT-first company or an IT security company that added OT support?
This matters for protocol support depth, understanding of OT operational constraints (change windows, uptime requirements, legacy device handling), and the quality of their professional services team. Ask how many of their engineers have operated or maintained industrial control systems.
8. Time to Value
How long from purchase order to enforcing security policies on the network?
Some platforms take 3-6 months to fully deploy. Others are operational in hours. Ask for the deployment timeline for a site comparable to yours, and whether that includes both monitoring and enforcement — not just the monitoring phase.
Scoring Table
Use this table to score each vendor during your evaluation. Rate each criterion 1-5 (1 = poor, 5 = excellent). Multiply by the weight for your environment type.
| Criterion | What "Good" Looks Like | Manufacturing Weight | Utilities Weight | Defense Weight |
|---|---|---|---|---|
| Deployment Model | Fully on-premise, no cloud dependency | 4 | 5 | 5 |
| Enforcement Capability | Monitors AND enforces in same product | 5 | 5 | 5 |
| Protocol Awareness | Native parsing of Modbus, EtherNet/IP, OPC UA, PROFINET at minimum | 5 | 4 | 3 |
| Air-Gap Support | Full functionality with zero internet | 3 | 4 | 5 |
| Compliance Coverage | Published controls mapping for your framework | 3 | 4 | 5 |
| Pricing Model | Per-appliance or per-site, not per-device | 4 | 3 | 3 |
| OT Expertise | OT-first company, engineers with ICS experience | 4 | 5 | 4 |
| Time to Value | Operational (monitoring + enforcement) within 1 week | 4 | 3 | 3 |
| Max: 160 | Max: 165 | Max: 165 |
How to read the weights: A weight of 5 means this criterion is critical for that environment type. A weight of 3 means it matters but is not the deciding factor. Multiply each vendor's score (1-5) by the weight, sum the results, and compare.
Red Flags During Evaluation
Watch for these during vendor demos and RFI responses:
- Cloud-only with no on-premise option. If your network is air-gapped or regulated, this is a disqualifier. No exceptions.
- Requires agents on OT endpoints. Any vendor that needs software installed on PLCs, HMIs, or embedded devices does not understand OT environments. Our analysis of proxy-based security for OT explains why proxies succeed where agents fail.
- No industrial protocol support. If the product treats Modbus the same as HTTP, it cannot provide meaningful OT security.
- Pricing that scales with device count. This creates a perverse incentive to undercount assets, which defeats the purpose of network security.
- Vague compliance claims without documentation. "We support CMMC" means nothing without a control-by-control mapping.
- Enforcement requires third-party products. If you need to buy firewalls or NAC from another vendor to enforce policies, the platform is incomplete.
- No reference customers in your industry. OT security is domain-specific. A vendor with 50 enterprise IT deployments and zero manufacturing sites is unproven in your environment.
RFI Template
Send this to every vendor on your shortlist:
- Describe your deployment model. Can the product operate fully on-premise with no internet connectivity?
- Does the product enforce network policies (block, segment, quarantine), or does it require third-party enforcement infrastructure?
- List every industrial protocol your product parses natively, including the depth of parsing (header vs. payload).
- Provide a controls mapping document for [your compliance framework].
- Describe your pricing model. What is the 3-year total cost for a site with [X] OT devices?
- Provide 3 reference customers in [your industry] who are using the product in enforcement mode.
- What is the typical deployment timeline from purchase order to active enforcement?
- What ongoing management does the product require? Express in FTE hours per month.
Score the responses using the table above. The vendor with the highest weighted score for your environment type goes on the shortlist for a proof-of-concept.
Procurement decisions in OT security should be driven by technical fit and total cost, not by brand recognition or analyst quadrant placement. Use a structured framework, score consistently, and make vendors prove their claims with specifics.
