# bms-1 (94.23.26.113) — Netdata health override for netdata.dbengine_global_errors # Deployed to: /etc/netdata/health.d/dbengine.conf (user health.d overrides the stock # /usr/lib/netdata/conf.d/health.d/dbengine.conf at the FILE level — so this file must # contain ALL THREE alarms, not just the tuned one, or FS/IO paging would be lost.) # Issue: radieu/p24-infra#4242 # # WHY: the stock 10min_dbengine_global_flushing_errors alarm counts transient in-flight page # flushes that miss their window during Netdata's own dbengine datafile rotation (~every 17 min # on bms-1). Those spikes (~410–823 errors/10min observed) tripped CRITICAL and paged a human on # every rotation cycle, even though I/O errors and FS errors stayed 0 and the disk/RAID were # healthy (self-recovering, no data loss). # # FIX (issue option 2): keep CRITICAL paging on the two dimensions that mean REAL trouble # (I/O errors, FS errors — copied verbatim from stock), and demote the flushing-errors dimension # to a high-threshold, non-paging (to: silent) informational WARNING. A genuinely failing disk # surfaces first as I/O errors, which still page CRITICAL. # # Netdata v1.19.0 on bms-1 has no `netdatacli`; reload health with: killall -USR2 netdata alarm: 10min_dbengine_global_fs_errors on: netdata.dbengine_global_errors os: linux freebsd macos hosts: * lookup: sum -10m unaligned of FS errors units: errors every: 10s crit: $this > 0 delay: down 15m multiplier 1.5 max 1h info: number of File-System errors dbengine came across the last 10 minutes (too many open files, wrong permissions etc) to: sysadmin alarm: 10min_dbengine_global_io_errors on: netdata.dbengine_global_errors os: linux freebsd macos hosts: * lookup: sum -10m unaligned of I/O errors units: errors every: 10s crit: $this > 0 delay: down 1h multiplier 1.5 max 3h info: number of IO errors dbengine came across the last 10 minutes (CRC errors, out of space, bad disk etc) to: sysadmin alarm: 10min_dbengine_global_flushing_errors on: netdata.dbengine_global_errors os: linux freebsd macos hosts: * lookup: sum -10m unaligned of flushing errors units: errors every: 10s warn: $this > 3000 delay: down 1h multiplier 1.5 max 3h info: dbengine failed to fully flush pages to disk in the last 10 minutes. On bms-1 short bursts during dbengine datafile rotation (~every 17 min, ~410-823 errors observed) are EXPECTED and self-recover with no data loss. Tuned per issue #4242 to WARN-only (never CRITICAL) with to:silent, tripping the dashboard badge only above 3000 (well clear of the rotation baseline). Real disk trouble surfaces first as I/O errors — see 10min_dbengine_global_io_errors, which still pages CRITICAL. to: silent