Storage area networks quietly carry the weight of an entire data center – databases, virtual machine images, backups, all riding on fibre channel or iSCSI fabric that most teams only think about when it breaks. Monitoring storage area networks means tracking latency, throughput, and error counts across the fabric so degradation shows up on a dashboard instead of as a slow application ticket. This article breaks down which SAN metrics actually matter, how to interpret them, and where teams typically get burned.
Why SAN problems are hard to diagnose without monitoring
A SAN sits between the application layer and physical disks, so when something goes wrong the symptoms show up everywhere except the SAN itself. A database team sees slow queries. A VMware admin sees datastore latency warnings. A backup job times out. Nobody immediately blames the storage fabric because it “was fine yesterday” and nothing was changed.
This is the classic SAN troubleshooting trap: the fault is several layers removed from where it’s noticed. A single degraded HBA port, a saturated ISL (inter-switch link), or a multipathing failover that silently dropped one path can all cause the same symptom – everything gets a little slower, intermittently, under load. Without fabric-level metrics, teams end up chasing the wrong layer for hours.
Latency: the metric that matters most
Latency is the single best early-warning signal for SAN health. Healthy block storage should return I/O in well under 10ms for most workloads, and spinning-disk-backed arrays doing sequential work might sit in the 5-15ms range depending on configuration. All-flash arrays should be sub-millisecond to low single digits.
What to actually watch:
Read and write latency separately – they often diverge. A write-heavy latency spike points at cache exhaustion or a RAID rebuild in progress; a read spike points at cache misses or a hot LUN.
Per-LUN and per-path latency – aggregate array-level latency hides the fact that one specific volume or one specific path is the problem. A single overloaded LUN can look invisible in an averaged number.
Latency spikes correlated with backup windows – this is one of the most common recurring incidents. Nightly backup jobs hammer the same LUNs that production databases use, and latency triples for two hours every night. It’s not a fault, it’s contention, but it needs the same visibility as a fault or capacity planning decisions get made on bad data.
The myth worth busting here: many teams assume that if the array vendor’s dashboard shows green, the SAN is fine. Array-side health checks mostly cover hardware faults – failed drives, degraded RAID groups, power supply issues. They rarely surface path-level latency creeping upward over weeks as workloads grow, which is exactly the kind of slow degradation that causes the worst outages because nobody notices until it crosses a threshold all at once.
Throughput and queue depth
Throughput (MB/s or IOPS) tells you how much work is moving, but on its own it’s a vanity metric. The number that actually predicts trouble is queue depth relative to throughput. If IOPS is flat but queue depth is climbing, requests are backing up somewhere in the path – often at the HBA, the switch port, or the array controller – and latency is about to follow.
A practical rule of thumb: track throughput per initiator and per target port, not just array-wide totals. Fibre channel fabrics commonly run 16Gb or 32Gb links; if a single ISL between switches is consistently running near saturation while individual host ports sit idle, that’s a fabric design problem, not a host problem, and no amount of host-side tuning will fix it.
Establishing what normal throughput and queue depth look like for each workload is what turns raw numbers into something actionable – a sudden 40% throughput drop only means something if there’s a baseline to compare it against. This is worth pairing with a baseline-driven approach to anomaly detection rather than relying on static thresholds that don’t fit every LUN’s normal behavior.
Errors: the signal teams ignore until it’s too late
Fibre channel and iSCSI both report error counters that are frequently overlooked because they don’t cause immediate outages on their own. That’s exactly why they matter.
Key counters to track:
CRC errors – almost always indicate a physical layer problem: a dirty or damaged SFP, a bad cable, or a failing transceiver. A slowly climbing CRC count on one port is a strong predictor of a future link failure.
Link resets and loss-of-sync events – these point to marginal physical connections or SFP compatibility issues, and they tend to cluster around temperature changes or vibration, which is why they’re often intermittent and hard to reproduce.
SCSI aborts and timeouts – these usually mean something downstream (array, switch, or path) failed to respond in time, and multipathing software silently retried on another path. If multipathing is doing its job, the application never notices – but the retries are a leading indicator of a path that’s about to fail completely.
A common mistake is treating a redundant, multipathed SAN as self-healing and therefore not worth monitoring closely. Multipathing hides failures from applications, not from operations. A fabric can be running on half its paths for weeks with zero user-facing impact, until the second path fails and there’s a full outage with no warning, because nobody was watching the error counters that would have shown the first path degrading.
Building practical SAN monitoring
A workable approach combines a few layers:
Poll switch and HBA statistics via SNMP for port-level errors, link state, and utilization – this is the layer most SAN outages actually originate from. For teams already collecting SNMP data from network gear, extending that to fabric switches and HBAs reuses existing tooling instead of standing up a separate system.
Collect host-side latency and queue depth from the OS or hypervisor, since that’s the number closest to what applications actually experience. This overlaps significantly with general disk I/O bottleneck detection work, and the same alerting logic applies whether the backing storage is local or SAN-attached.
Set alert thresholds based on sustained trends, not single readings – a one-second latency spike during a backup window is normal; the same latency sustained for ten minutes during business hours is not.
Frequently asked questions
What’s a normal latency threshold to alert on for SAN storage?
There’s no universal number, since it depends on the array and workload, but a common practical starting point is alerting when read or write latency sustains above 20ms for spinning-disk arrays or above 5ms for all-flash arrays for more than a few minutes. Tune this against your own baseline rather than a generic industry figure.
Can SAN issues cause problems without any downtime?
Yes, and this is the most common real-world pattern. Degraded paths, rising CRC errors, or growing queue depth typically show up as slow performance and application timeouts long before there’s an actual outage, which is why error and latency trending catches problems weeks before a hard failure would.
Is array vendor monitoring enough on its own?
It covers hardware faults well but usually misses fabric-level and host-side path degradation. Combining vendor tools with independent switch, HBA, and host metrics gives a fuller picture of what applications are actually experiencing.
Latency, throughput, and error counters each tell a different part of the SAN story, and none of them alone is reliable enough to catch every failure mode. Treating SAN visibility as a combination of fabric-level SNMP data, host-side latency, and trend-based alerting turns storage from a black box teams only investigate during an incident into infrastructure that’s monitored the same way the rest of the stack is.