10.1 Failed Services
10.2 Filesystems Above Threshold (80%) .
10.3 Inodes Above Threshold
10.4 Memory Pressure Анализ PSI (Pressure Stall Information).
10.5 Swap Usage and Activity (swap-in/swap-out)
10.6 System Load – compare with Load Average
10.7 Disk and Filesystem Errors (I/O, EXT4, XFS, Btrfs, ZFS, NVMe and etc.)
10.8 Network Errors (Link Down, Carrier Loss, NETDEV WATCHDOG, TX Timeout and etc.).
10.9 SMART Health
10.10 ZFS Health (degraded-pools, ro/rw errors ,scrub)
10.11 mdadm RAID Health (degraded, recovery, reshape and etc.)
10.12 Time Synchronization (NTP).
10.13 Health Summary (Overall Health).
Why Base64?
Base64 encoding allows complex multi-line Bash scripts to be safely delivered as a single command, avoiding issues with line breaks, quoting, and macro parsing while preserving the original script intact. Use: printf %s <base64_data> | base64 -d | bash
printf %s 'IyEvdXNyL2Jpbi9lbnYgYmFzaAoKIyA9PT09PT09PT09PT09PT09PT09PSAxMC4gSEVBTFRIIENIRUNLID09PT09PT09PT09PT09PT09PT09CgplY2hvICI9PT09PT09PT09PT09PT09PT09PSAxMC4gSEVBTFRIIENIRUNLID09PT09PT09PT09PT09PT09PT09IgoKcHJpbnRmICJcbiIKCk9WRVJBTExfU1RBVFVTPSJPSyIKCiMgLS0tIDEwLjEgRkFJTEVEIFNFUlZJQ0VTIC0tLQoKZWNobyAiMTAuMSBGQUlMRUQgU0VSVklDRVMiCgpGQUlMRURfU0VSVklDRVNfQ09VTlQ9JCgKICAgIHN5c3RlbWN0bCAtLWZhaWxlZCBcCiAgICAgICAgLS10eXBlPXNlcnZpY2UgXAogICAgICAgIC0tbm8tbGVnZW5kIFwKICAgICAgICAtLXBsYWluIFwKICAgICAgICAyPi9kZXYvbnVsbCB8IFwKICAgICAgICBhd2sgJ05GIHtjb3VudCsrfSBFTkQge3ByaW50IGNvdW50KzB9JwopCgppZiBbICIke0ZBSUxFRF9TRVJWSUNFU19DT1VOVDotMH0iIC1ndCAwIF07IHRoZW4KICAgIGVjaG8gIkNSSVRJQ0FMOiAkRkFJTEVEX1NFUlZJQ0VTX0NPVU5UIGZhaWxlZCBzZXJ2aWNlKHMpIGRldGVjdGVkIgoKICAgIHN5c3RlbWN0bCAtLWZhaWxlZCBcCiAgICAgICAgLS10eXBlPXNlcnZpY2UgXAogICAgICAgIC0tbm8tcGFnZXIgXAogICAgICAgIDI+L2Rldi9udWxsCgogICAgT1ZFUkFMTF9TVEFUVVM9IkNSSVRJQ0FMIgplbHNlCiAgICBlY2hvICJPSzogTm8gZmFpbGVkIHN5c3RlbWQgc2VydmljZXMgZGV0ZWN0ZWQiCmZpCgpwcmludGYgIlxuJS4wcyIgezEuLjN9CgojIC0tLSAxMC4yIEZJTEVTWVNURU1TIEFCT1ZFIFRIUkVTSE9MRCAtLS0KCmVjaG8gIjEwLjIgRklMRVNZU1RFTVMgQUJPVkUgVEhSRVNIT0xEIgoKREZfT1VUUFVUPSQodGltZW91dCA4IGRmIC1oVCAyPi9kZXYvbnVsbCkKCmlmIFsgLW4gIiRERl9PVVRQVVQiIF07IHRoZW4KICAgIEZTX09WRVJfTElNSVQ9JCgKICAgICAgICBwcmludGYgIiVzXG4iICIkREZfT1VUUFVUIiB8IFwKICAgICAgICBhd2sgJwogICAgICAgICAgICBOUiA+IDEgewogICAgICAgICAgICAgICAgdXNlPSQ2CiAgICAgICAgICAgICAgICBnc3ViKC8lLywgIiIsIHVzZSkKICAgICAgICAgICAgICAgIGlmICh1c2UgfiAvXlswLTldKyQvICYmIHVzZSA+PSA4MCkgewogICAgICAgICAgICAgICAgICAgIHByaW50CiAgICAgICAgICAgICAgICB9CiAgICAgICAgICAgIH0KICAgICAgICAnCiAgICApCgogICAgaWYgWyAtbiAiJEZTX09WRVJfTElNSVQiIF07IHRoZW4KICAgICAgICBlY2hvICJXQVJOSU5HOiBGaWxlc3lzdGVtcyBhdCBvciBhYm92ZSA4MCUgdXNhZ2U6IgogICAgICAgIHByaW50ZiAiJXNcbiIgIiRERl9PVVRQVVQiIHwgaGVhZCAtMQogICAgICAgIHByaW50ZiAiJXNcbiIgIiRGU19PVkVSX0xJTUlUIgoKICAgICAgICBbICIkT1ZFUkFMTF9TVEFUVVMiID0gIk9LIiBdICYmIE9WRVJBTExfU1RBVFVTPSJXQVJOSU5HIgogICAgZWxzZQogICAgICAgIGVjaG8gIk9LOiBBbGwgZmlsZXN5c3RlbXMgYXJlIGJlbG93IDgwJSB1c2FnZSIKICAgIGZpCmVsc2UKICAgIEZTX09WRVJfTElNSVQ9IiIKICAgIGVjaG8gIldBUk5JTkc6IEZpbGVzeXN0ZW0gdXNhZ2UgcXVlcnkgdW5hdmFpbGFibGUgb3IgdGltZWQgb3V0IgoKICAgIFsgIiRPVkVSQUxMX1NUQVRVUyIgPSAiT0siIF0gJiYgT1ZFUkFMTF9TVEFUVVM9IldBUk5JTkciCmZpCgpwcmludGYgIlxuJS4wcyIgezEuLjN9CgojIC0tLSAxMC4zIElOT0RFUyBBQk9WRSBUSFJFU0hPTEQgLS0tCgplY2hvICIxMC4zIElOT0RFUyBBQk9WRSBUSFJFU0hPTEQiCgpERl9JTk9ERV9PVVRQVVQ9JCh0aW1lb3V0IDggZGYgLWloIDI+L2Rldi9udWxsKQoKaWYgWyAtbiAiJERGX0lOT0RFX09VVFBVVCIgXTsgdGhlbgogICAgSU5PREVfT1ZFUl9MSU1JVD0kKAogICAgICAgIHByaW50ZiAiJXNcbiIgIiRERl9JTk9ERV9PVVRQVVQiIHwgXAogICAgICAgIGF3ayAnCiAgICAgICAgICAgIE5SID4gMSB7CiAgICAgICAgICAgICAgICB1c2U9JDUKICAgICAgICAgICAgICAgIGdzdWIoLyUvLCAiIiwgdXNlKQogICAgICAgICAgICAgICAgaWYgKHVzZSB+IC9eWzAtOV0rJC8gJiYgdXNlID49IDgwKSB7CiAgICAgICAgICAgICAgICAgICAgcHJpbnQKICAgICAgICAgICAgICAgIH0KICAgICAgICAgICAgfQogICAgICAgICcKICAgICkKCiAgICBpZiBbIC1uICIkSU5PREVfT1ZFUl9MSU1JVCIgXTsgdGhlbgogICAgICAgIGVjaG8gIldBUk5JTkc6IEZpbGVzeXN0ZW1zIGF0IG9yIGFib3ZlIDgwJSBpbm9kZSB1c2FnZToiCiAgICAgICAgcHJpbnRmICIlc1xuIiAiJERGX0lOT0RFX09VVFBVVCIgfCBoZWFkIC0xCiAgICAgICAgcHJpbnRmICIlc1xuIiAiJElOT0RFX09WRVJfTElNSVQiCgogICAgICAgIFsgIiRPVkVSQUxMX1NUQVRVUyIgPSAiT0siIF0gJiYgT1ZFUkFMTF9TVEFUVVM9IldBUk5JTkciCiAgICBlbHNlCiAgICAgICAgZWNobyAiT0s6IEFsbCBmaWxlc3lzdGVtcyBhcmUgYmVsb3cgODAlIGlub2RlIHVzYWdlIgogICAgZmkKZWxzZQogICAgSU5PREVfT1ZFUl9MSU1JVD0iIgogICAgZWNobyAiV0FSTklORzogSW5vZGUgdXNhZ2UgcXVlcnkgdW5hdmFpbGFibGUgb3IgdGltZWQgb3V0IgoKICAgIFsgIiRPVkVSQUxMX1NUQVRVUyIgPSAiT0siIF0gJiYgT1ZFUkFMTF9TVEFUVVM9IldBUk5JTkciCmZpCgpwcmludGYgIlxuJS4wcyIgezEuLjN9CgojIC0tLSAxMC40IE1FTU9SWSBQUkVTU1VSRSAtLS0KCmVjaG8gIjEwLjQgTUVNT1JZIFBSRVNTVVJFIgoKTUVNX1BSRVNTX0FWRz0iIgoKaWYgWyAtZiAvcHJvYy9wcmVzc3VyZS9tZW1vcnkgXTsgdGhlbgogICAgTUVNX1BSRVNTX0FWRz0kKAogICAgICAgIGF3ayAnCiAgICAgICAgICAgICQxID09ICJzb21lIiB7CiAgICAgICAgICAgICAgICBmb3IgKGk9MTsgaTw9TkY7IGkrKykgewogICAgICAgICAgICAgICAgICAgIGlmICgkaSB+IC9eYXZnMTA9LykgewogICAgICAgICAgICAgICAgICAgICAgICBzcGxpdCgkaSwgdmFsdWUsICI9IikKICAgICAgICAgICAgICAgICAgICAgICAgcHJpbnQgdmFsdWVbMl0KICAgICAgICAgICAgICAgICAgICAgICAgZXhpdAogICAgICAgICAgICAgICAgICAgIH0KICAgICAgICAgICAgICAgIH0KICAgICAgICAgICAgfQogICAgICAgICcgL3Byb2MvcHJlc3N1cmUvbWVtb3J5CiAgICApCgogICAgaWYgYXdrIC12IHZhbHVlPSIke01FTV9QUkVTU19BVkc6LTB9IiBcCiAgICAgICAgJ0JFR0lOIHtleGl0ICEodmFsdWUgPiAxMC4wKX0nCiAgICB0aGVuCiAgICAgICAgZWNobyAiV0FSTklORzogTWVtb3J5IFBTSSBhdmcxMCBpcyBoaWdoOiAke01FTV9QUkVTU19BVkc6LXVua25vd259IgogICAgICAgIGNhdCAvcHJvYy9wcmVzc3VyZS9tZW1vcnkKCiAgICAgICAgWyAiJE9WRVJBTExfU1RBVFVTIiA9ICJPSyIgXSAmJiBPVkVSQUxMX1NUQVRVUz0iV0FSTklORyIKICAgIGVsc2UKICAgICAgICBlY2hvICJPSzogTWVtb3J5IFBTSSBhdmcxMCBpcyB3aXRoaW4gdGhyZXNob2xkOiAke01FTV9QUkVTU19BVkc6LTB9IgogICAgZmkKZWxzZQogICAgZWNobyAiSU5GTzogTWVtb3J5IFBTSSBpcyBub3QgYXZhaWxhYmxlIG9uIHRoaXMga2VybmVsIgpmaQoKcHJpbnRmICJcbiUuMHMiIHsxLi4zfQoKIyAtLS0gMTAuNSBTV0FQIFVTQUdFIEFORCBBQ1RJVklUWSAtLS0KCmVjaG8gIjEwLjUgU1dBUCBVU0FHRSBBTkQgQUNUSVZJVFkiCgpTV0FQX1RPVEFMX01FVFJJQz0kKAogICAgZnJlZSAtYiAyPi9kZXYvbnVsbCB8IFwKICAgIGF3ayAnL15Td2FwOi8ge3ByaW50ICQyfScKKQoKU1dBUF9VU0VEX01FVFJJQz0kKAogICAgZnJlZSAtYiAyPi9kZXYvbnVsbCB8IFwKICAgIGF3ayAnL15Td2FwOi8ge3ByaW50ICQzfScKKQoKU1dBUF9QQ1RfTUVUUklDPTAKU1dBUF9BQ1RJVklUWT0wCgppZiBbICIke1NXQVBfVE9UQUxfTUVUUklDOi0wfSIgLWd0IDAgXTsgdGhlbgogICAgU1dBUF9QQ1RfTUVUUklDPSQoCiAgICAgICAgYXdrIFwKICAgICAgICAgICAgLXYgdXNlZD0iJHtTV0FQX1VTRURfTUVUUklDOi0wfSIgXAogICAgICAgICAgICAtdiB0b3RhbD0iJHtTV0FQX1RPVEFMX01FVFJJQzotMX0iIFwKICAgICAgICAgICAgJ0JFR0lOIHtwcmludGYgIiUuMGYiLCAodXNlZC90b3RhbCkqMTAwfScKICAgICkKCiAgICBTV0FQX0FDVElWSVRZPSQoCiAgICAgICAgdm1zdGF0IDEgMiAyPi9kZXYvbnVsbCB8IFwKICAgICAgICB0YWlsIC0xIHwgXAogICAgICAgIGF3ayAne3ByaW50ICQ3KyQ4fScKICAgICkKCiAgICBlY2hvICJTd2FwIHVzYWdlICAgIDogJHtTV0FQX1BDVF9NRVRSSUN9JSIKICAgIGVjaG8gIlN3YXAgYWN0aXZpdHkgOiAke1NXQVBfQUNUSVZJVFk6LTB9IHBhZ2VzL3MgY29tYmluZWQgc2krc28iCgogICAgaWYgWyAiJHtTV0FQX0FDVElWSVRZOi0wfSIgLWd0IDAgXTsgdGhlbgogICAgICAgIGVjaG8gIldBUk5JTkc6IEFjdGl2ZSBzd2FwLWluIG9yIHN3YXAtb3V0IGFjdGl2aXR5IGRldGVjdGVkIgogICAgICAgIGZyZWUgLWgKCiAgICAgICAgWyAiJE9WRVJBTExfU1RBVFVTIiA9ICJPSyIgXSAmJiBPVkVSQUxMX1NUQVRVUz0iV0FSTklORyIKICAgIGVsaWYgWyAiJHtTV0FQX1BDVF9NRVRSSUM6LTB9IiAtZ3QgNTAgXTsgdGhlbgogICAgICAgIGVjaG8gIklORk86IFN3YXAgdXNhZ2UgaXMgYWJvdmUgNTAlLCBidXQgbm8gY3VycmVudCBzd2FwIGFjdGl2aXR5IHdhcyBkZXRlY3RlZCIKICAgIGVsc2UKICAgICAgICBlY2hvICJPSzogU3dhcCB1c2FnZSBhbmQgYWN0aXZpdHkgYXJlIHdpdGhpbiBleHBlY3RlZCBsaW1pdHMiCiAgICBmaQplbHNlCiAgICBlY2hvICJJTkZPOiBTd2FwIGlzIG5vdCBjb25maWd1cmVkIgpmaQoKcHJpbnRmICJcbiUuMHMiIHsxLi4zfQoKIyAtLS0gMTAuNiBTWVNURU0gTE9BRCAtLS0KCmVjaG8gIjEwLjYgU1lTVEVNIExPQUQiCgpyZWFkIC1yIExPQURfT05FX01JTiBMT0FEX0ZJVkVfTUlOIExPQURfRklGVEVFTl9NSU4gXyA8IC9wcm9jL2xvYWRhdmcKCkNQVV9DT1JFU19DT1VOVD0kKAogICAgbnByb2MgMj4vZGV2L251bGwgfHwgXAogICAgZ3JlcCAtYyAnXnByb2Nlc3NvcicgL3Byb2MvY3B1aW5mbyAyPi9kZXYvbnVsbCB8fCBcCiAgICBlY2hvIDEKKQoKTE9BRF9QRVJfQ09SRT0kKAogICAgYXdrIFwKICAgICAgICAtdiBsb2FkPSIke0xPQURfT05FX01JTjotMH0iIFwKICAgICAgICAtdiBjb3Jlcz0iJHtDUFVfQ09SRVNfQ09VTlQ6LTF9IiBcCiAgICAgICAgJ0JFR0lOIHsKICAgICAgICAgICAgaWYgKGNvcmVzIDwgMSkgewogICAgICAgICAgICAgICAgY29yZXM9MQogICAgICAgICAgICB9CiAgICAgICAgICAgIHByaW50ZiAiJS4yZiIsIGxvYWQvY29yZXMKICAgICAgICB9JwopCgplY2hvICJMb2FkIGF2ZXJhZ2UgOiAke0xPQURfT05FX01JTjotMH0gJHtMT0FEX0ZJVkVfTUlOOi0wfSAke0xPQURfRklGVEVFTl9NSU46LTB9IgplY2hvICJDUFUgY29yZXMgICAgOiAke0NQVV9DT1JFU19DT1VOVDotMX0iCmVjaG8gIkxvYWQvY29yZSAgICA6ICRMT0FEX1BFUl9DT1JFIgoKaWYgYXdrIC12IHZhbHVlPSIkTE9BRF9QRVJfQ09SRSIgXAogICAgJ0JFR0lOIHtleGl0ICEodmFsdWUgPj0gMS4wKX0nCnRoZW4KICAgIGVjaG8gIkNSSVRJQ0FMOiBPbmUtbWludXRlIGxvYWQgaXMgYXQgb3IgYWJvdmUgMS4wMCBwZXIgQ1BVIGNvcmUiCiAgICB1cHRpbWUKCiAgICBPVkVSQUxMX1NUQVRVUz0iQ1JJVElDQUwiCmVsaWYgYXdrIC12IHZhbHVlPSIkTE9BRF9QRVJfQ09SRSIgXAogICAgJ0JFR0lOIHtleGl0ICEodmFsdWUgPj0gMC43MCl9Jwp0aGVuCiAgICBlY2hvICJXQVJOSU5HOiBPbmUtbWludXRlIGxvYWQgaXMgYXQgb3IgYWJvdmUgMC43MCBwZXIgQ1BVIGNvcmUiCiAgICB1cHRpbWUKCiAgICBbICIkT1ZFUkFMTF9TVEFUVVMiID0gIk9LIiBdICYmIE9WRVJBTExfU1RBVFVTPSJXQVJOSU5HIgplbHNlCiAgICBlY2hvICJPSzogTm9ybWFsaXplZCBsb2FkIGlzIHdpdGhpbiB0aHJlc2hvbGQiCmZpCgpwcmludGYgIlxuJS4wcyIgezEuLjN9CgojIC0tLSAxMC43IERJU0sgQU5EIEZJTEVTWVNURU0gRVJST1JTIC0tLQoKZWNobyAiMTAuNyBESVNLIEFORCBGSUxFU1lTVEVNIEVSUk9SUyIKCmlmIGNvbW1hbmQgLXYgam91cm5hbGN0bCA+L2Rldi9udWxsIDI+JjE7IHRoZW4KICAgIERJU0tfRVJSX0hVTlQ9JCgKICAgICAgICBqb3VybmFsY3RsIFwKICAgICAgICAgICAgLWsgXAogICAgICAgICAgICAtYiAwIFwKICAgICAgICAgICAgLS1uby1wYWdlciBcCiAgICAgICAgICAgIDI+L2Rldi9udWxsIHwgXAogICAgICAgIGdyZXAgLUVpIFwKICAgICAgICAgICAgJ0kvTyBlcnJvcnxidWZmZXIgSS9PfGJsa191cGRhdGVfcmVxdWVzdHxjcml0aWNhbCBtZWRpdW0gZXJyb3J8dW5jb3JyZWN0YWJsZXxFWFQ0LWZzIGVycm9yfFhGUy4qZXJyb3J8QlRSRlMuKmVycm9yfFpGUy4qZXJyb3J8bnZtZS4qZXJyb3J8YmFkIHNlY3RvcnxyZWFkLW9ubHkgZmlsZSBzeXN0ZW0nIHwgXAogICAgICAgIHRhaWwgLTE1CiAgICApCmVsc2UKICAgIERJU0tfRVJSX0hVTlQ9JCgKICAgICAgICBkbWVzZyAyPi9kZXYvbnVsbCB8IFwKICAgICAgICBncmVwIC1FaSBcCiAgICAgICAgICAgICdJL08gZXJyb3J8YnVmZmVyIEkvT3xibGtfdXBkYXRlX3JlcXVlc3R8Y3JpdGljYWwgbWVkaXVtIGVycm9yfHVuY29ycmVjdGFibGV8RVhUNC1mcyBlcnJvcnxYRlMuKmVycm9yfEJUUkZTLiplcnJvcnxaRlMuKmVycm9yfG52bWUuKmVycm9yfGJhZCBzZWN0b3J8cmVhZC1vbmx5IGZpbGUgc3lzdGVtJyB8IFwKICAgICAgICB0YWlsIC0xNQogICAgKQpmaQoKaWYgWyAtbiAiJERJU0tfRVJSX0hVTlQiIF07IHRoZW4KICAgIGVjaG8gIkNSSVRJQ0FMOiBEaXNrIG9yIGZpbGVzeXN0ZW0gZXJyb3JzIGRldGVjdGVkOiIKICAgIHByaW50ZiAiJXNcbiIgIiRESVNLX0VSUl9IVU5UIgoKICAgIE9WRVJBTExfU1RBVFVTPSJDUklUSUNBTCIKZWxzZQogICAgZWNobyAiT0s6IE5vIGRpc2sgb3IgZmlsZXN5c3RlbSBlcnJvcnMgZGV0ZWN0ZWQgaW4gdGhlIGN1cnJlbnQgYm9vdCIKZmkKCnByaW50ZiAiXG4lLjBzIiB7MS4uM30KCiMgLS0tIDEwLjggTkVUV09SSyBFUlJPUlMgLS0tCgplY2hvICIxMC44IE5FVFdPUksgRVJST1JTIgoKaWYgY29tbWFuZCAtdiBqb3VybmFsY3RsID4vZGV2L251bGwgMj4mMTsgdGhlbgogICAgTkVUX0VSUl9IVU5UPSQoCiAgICAgICAgam91cm5hbGN0bCBcCiAgICAgICAgICAgIC1rIFwKICAgICAgICAgICAgLWIgMCBcCiAgICAgICAgICAgIC0tbm8tcGFnZXIgXAogICAgICAgICAgICAyPi9kZXYvbnVsbCB8IFwKICAgICAgICBncmVwIC1FaSBcCiAgICAgICAgICAgICdsaW5rIGlzIGRvd258bGluayBkb3dufGxvc3QgY2FycmllcnxjYXJyaWVyIGxvc3R8TkVUREVWIFdBVENIRE9HfHRyYW5zbWl0IHF1ZXVlIHRpbWVkIG91dHx0eCB0aW1lb3V0fHJlc2V0IGFkYXB0ZXJ8ZmlybXdhcmUgZmFpbGVkJyB8IFwKICAgICAgICB0YWlsIC0xNQogICAgKQplbHNlCiAgICBORVRfRVJSX0hVTlQ9JCgKICAgICAgICBkbWVzZyAyPi9kZXYvbnVsbCB8IFwKICAgICAgICBncmVwIC1FaSBcCiAgICAgICAgICAgICdsaW5rIGlzIGRvd258bGluayBkb3dufGxvc3QgY2FycmllcnxjYXJyaWVyIGxvc3R8TkVUREVWIFdBVENIRE9HfHRyYW5zbWl0IHF1ZXVlIHRpbWVkIG91dHx0eCB0aW1lb3V0fHJlc2V0IGFkYXB0ZXJ8ZmlybXdhcmUgZmFpbGVkJyB8IFwKICAgICAgICB0YWlsIC0xNQogICAgKQpmaQoKaWYgWyAtbiAiJE5FVF9FUlJfSFVOVCIgXTsgdGhlbgogICAgZWNobyAiV0FSTklORzogTmV0d29yayBsaW5rIG9yIGFkYXB0ZXIgZXJyb3JzIGRldGVjdGVkOiIKICAgIHByaW50ZiAiJXNcbiIgIiRORVRfRVJSX0hVTlQiCgogICAgWyAiJE9WRVJBTExfU1RBVFVTIiA9ICJPSyIgXSAmJiBPVkVSQUxMX1NUQVRVUz0iV0FSTklORyIKZWxzZQogICAgZWNobyAiT0s6IE5vIG5ldHdvcmsgbGluayBvciBhZGFwdGVyIGVycm9ycyBkZXRlY3RlZCBpbiB0aGUgY3VycmVudCBib290IgpmaQoKcHJpbnRmICJcbiUuMHMiIHsxLi4zfQoKIyAtLS0gMTAuOSBTTUFSVCBIRUFMVEggLS0tCgplY2hvICIxMC45IFNNQVJUIEhFQUxUSCIKClNNQVJUX0VSUl9GTEFHPTAKU01BUlRfVU5LTk9XTl9GTEFHPTAKU01BUlRfREVWSUNFX0NPVU5UPTAKCmlmIGNvbW1hbmQgLXYgc21hcnRjdGwgPi9kZXYvbnVsbCAyPiYxOyB0aGVuCiAgICB3aGlsZSBJRlM9IHJlYWQgLXIgREVWX0lEOyBkbwogICAgICAgIFsgLXogIiRERVZfSUQiIF0gJiYgY29udGludWUKCiAgICAgICAgU01BUlRfREVWSUNFX0NPVU5UPSQoKFNNQVJUX0RFVklDRV9DT1VOVCArIDEpKQoKICAgICAgICBTTUFSVF9PVVRQVVQ9JCgKICAgICAgICAgICAgc21hcnRjdGwgLUggIiRERVZfSUQiIDI+L2Rldi9udWxsCiAgICAgICAgKQoKICAgICAgICBpZiBwcmludGYgIiVzXG4iICIkU01BUlRfT1VUUFVUIiB8IFwKICAgICAgICAgICAgZ3JlcCAtRXFpICdQQVNTRUR8U01BUlQgSGVhbHRoIFN0YXR1czogT0t8b3ZlcmFsbC1oZWFsdGguKlBBU1NFRCcKICAgICAgICB0aGVuCiAgICAgICAgICAgIGVjaG8gIk9LOiBTTUFSVCBoZWFsdGggcGFzc2VkIG9uICRERVZfSUQiCiAgICAgICAgZWxpZiBwcmludGYgIiVzXG4iICIkU01BUlRfT1VUUFVUIiB8IFwKICAgICAgICAgICAgZ3JlcCAtRXFpICdGQUlMRUR8QkFEfENSSVRJQ0FMJwogICAgICAgIHRoZW4KICAgICAgICAgICAgZWNobyAiQ1JJVElDQUw6IFNNQVJUIGhlYWx0aCBmYWlsdXJlIHJlcG9ydGVkIG9uICRERVZfSUQiCiAgICAgICAgICAgIHByaW50ZiAiJXNcbiIgIiRTTUFSVF9PVVRQVVQiCgogICAgICAgICAgICBTTUFSVF9FUlJfRkxBRz0xCiAgICAgICAgICAgIE9WRVJBTExfU1RBVFVTPSJDUklUSUNBTCIKICAgICAgICBlbHNlCiAgICAgICAgICAgIGVjaG8gIklORk86IFNNQVJUIGhlYWx0aCBjb3VsZCBub3QgYmUgZGV0ZXJtaW5lZCBmb3IgJERFVl9JRCIKCiAgICAgICAgICAgIFNNQVJUX1VOS05PV05fRkxBRz0xCiAgICAgICAgZmkKICAgIGRvbmUgPCA8KAogICAgICAgIHNtYXJ0Y3RsIC0tc2NhbiAyPi9kZXYvbnVsbCB8IFwKICAgICAgICBhd2sgJ3twcmludCAkMX0nCiAgICApCgogICAgaWYgWyAiJFNNQVJUX0RFVklDRV9DT1VOVCIgLWVxIDAgXTsgdGhlbgogICAgICAgIGVjaG8gIklORk86IE5vIFNNQVJULWNhcGFibGUgZGV2aWNlcyBkaXNjb3ZlcmVkIGJ5IHNtYXJ0Y3RsIC0tc2NhbiIKICAgIGVsaWYgWyAiJFNNQVJUX0VSUl9GTEFHIiAtZXEgMCBdICYmIFsgIiRTTUFSVF9VTktOT1dOX0ZMQUciIC1lcSAwIF07IHRoZW4KICAgICAgICBlY2hvICJPSzogQWxsIGRpc2NvdmVyZWQgU01BUlQgZGV2aWNlcyByZXBvcnRlZCBoZWFsdGh5IgogICAgZmkKZWxzZQogICAgZWNobyAiSU5GTzogc21hcnRjdGwgaXMgbm90IGluc3RhbGxlZCIKZmkKCnByaW50ZiAiXG4lLjBzIiB7MS4uM30KCiMgLS0tIDEwLjEwIFpGUyBIRUFMVEggLS0tCgplY2hvICIxMC4xMCBaRlMgSEVBTFRIIgoKWkZTX0hFQUxUSF9TVEFUVVM9Ik5PVCBJTlNUQUxMRUQiCgppZiBjb21tYW5kIC12IHpwb29sID4vZGV2L251bGwgMj4mMTsgdGhlbgogICAgWlBPT0xfRkFVTFRfTUFQPSQoCiAgICAgICAgenBvb2wgc3RhdHVzIC14IDI+L2Rldi9udWxsCiAgICApCgogICAgaWYgWyAteiAiJFpQT09MX0ZBVUxUX01BUCIgXTsgdGhlbgogICAgICAgIGVjaG8gIklORk86IE5vIFpGUyBwb29scyBkZXRlY3RlZCIKICAgICAgICBaRlNfSEVBTFRIX1NUQVRVUz0iTk8gUE9PTFMiCiAgICBlbGlmIHByaW50ZiAiJXNcbiIgIiRaUE9PTF9GQVVMVF9NQVAiIHwgXAogICAgICAgIGdyZXAgLXFpICdhbGwgcG9vbHMgYXJlIGhlYWx0aHknCiAgICB0aGVuCiAgICAgICAgZWNobyAiT0s6IEFsbCBaRlMgcG9vbHMgYXJlIGhlYWx0aHkiCiAgICAgICAgWkZTX0hFQUxUSF9TVEFUVVM9Ik9LIgogICAgZWxzZQogICAgICAgIGVjaG8gIkNSSVRJQ0FMOiBaRlMgcG9vbCBpc3N1ZSBkZXRlY3RlZDoiCiAgICAgICAgcHJpbnRmICIlc1xuIiAiJFpQT09MX0ZBVUxUX01BUCIKCiAgICAgICAgWkZTX0hFQUxUSF9TVEFUVVM9IkNSSVRJQ0FMIgogICAgICAgIE9WRVJBTExfU1RBVFVTPSJDUklUSUNBTCIKICAgIGZpCmVsc2UKICAgIGVjaG8gIklORk86IFpGUyB0b29scyBhcmUgbm90IGluc3RhbGxlZCIKZmkKCnByaW50ZiAiXG4lLjBzIiB7MS4uM30KCiMgLS0tIDEwLjExIE1EQURNIFJBSUQgSEVBTFRIIC0tLQoKZWNobyAiMTAuMTEgTURBRE0gUkFJRCBIRUFMVEgiCgpNREFETV9IRUFMVEhfU1RBVFVTPSJOT1QgQ09ORklHVVJFRCIKCmlmIFsgLXIgL3Byb2MvbWRzdGF0IF0gJiYgZ3JlcCAtcSAnXm1kWzAtOV0nIC9wcm9jL21kc3RhdDsgdGhlbgogICAgTURBRE1fSEVBTFRIX1NUQVRVUz0iT0siCgogICAgTURBRE1fUFJPQkxFTVM9JCgKICAgICAgICBncmVwIC1FIFwKICAgICAgICAgICAgJ2luYWN0aXZlfHJlY292ZXJ5fHJlc3luY3xyZXNoYXBlfGNoZWNrfFxbW1VfXSpfW1VfXSpcXScgXAogICAgICAgICAgICAvcHJvYy9tZHN0YXQgXAogICAgICAgICAgICAyPi9kZXYvbnVsbAogICAgKQoKICAgIGlmIFsgLW4gIiRNREFETV9QUk9CTEVNUyIgXTsgdGhlbgogICAgICAgIGVjaG8gIldBUk5JTkc6IG1kYWRtIGFycmF5IGlzIGRlZ3JhZGVkLCBpbmFjdGl2ZSwgb3IgcnVubmluZyBtYWludGVuYW5jZToiCiAgICAgICAgY2F0IC9wcm9jL21kc3RhdAoKICAgICAgICBNREFETV9IRUFMVEhfU1RBVFVTPSJXQVJOSU5HIgoKICAgICAgICBbICIkT1ZFUkFMTF9TVEFUVVMiID0gIk9LIiBdICYmIE9WRVJBTExfU1RBVFVTPSJXQVJOSU5HIgogICAgZWxzZQogICAgICAgIGVjaG8gIk9LOiBBY3RpdmUgbWRhZG0gYXJyYXlzIGFwcGVhciBoZWFsdGh5IgogICAgZmkKZWxzZQogICAgZWNobyAiSU5GTzogTm8gYWN0aXZlIG1kYWRtIGFycmF5cyBkZXRlY3RlZCIKZmkKCnByaW50ZiAiXG4lLjBzIiB7MS4uM30KCiMgLS0tIDEwLjEyIFRJTUUgU1lOQ0hST05JWkFUSU9OIC0tLQoKZWNobyAiMTAuMTIgVElNRSBTWU5DSFJPTklaQVRJT04iCgpUSU1FX1NZTkNfU1RBVFVTPSJVTktOT1dOIgoKaWYgY29tbWFuZCAtdiB0aW1lZGF0ZWN0bCA+L2Rldi9udWxsIDI+JjE7IHRoZW4KICAgIFRJTUVfU1lOQ19TVEFUVVM9JCgKICAgICAgICB0aW1lZGF0ZWN0bCBzaG93IFwKICAgICAgICAgICAgLXAgTlRQU3luY2hyb25pemVkIFwKICAgICAgICAgICAgLS12YWx1ZSBcCiAgICAgICAgICAgIDI+L2Rldi9udWxsCiAgICApCgogICAgaWYgWyAiJFRJTUVfU1lOQ19TVEFUVVMiID0gInllcyIgXTsgdGhlbgogICAgICAgIGVjaG8gIk9LOiBTeXN0ZW0gY2xvY2sgaXMgc3luY2hyb25pemVkIgogICAgICAgIFRJTUVfU1lOQ19TVEFUVVM9Ik9LIgogICAgZWxzZQogICAgICAgIGVjaG8gIldBUk5JTkc6IFN5c3RlbSBjbG9jayBpcyBub3Qgc3luY2hyb25pemVkIgogICAgICAgIHRpbWVkYXRlY3RsIHN0YXR1cyAyPi9kZXYvbnVsbAoKICAgICAgICBUSU1FX1NZTkNfU1RBVFVTPSJXQVJOSU5HIgoKICAgICAgICBbICIkT1ZFUkFMTF9TVEFUVVMiID0gIk9LIiBdICYmIE9WRVJBTExfU1RBVFVTPSJXQVJOSU5HIgogICAgZmkKZWxzZQogICAgZWNobyAiSU5GTzogdGltZWRhdGVjdGwgaXMgbm90IGF2YWlsYWJsZSIKZmkKCnByaW50ZiAiXG4lLjBzIiB7MS4uM30KCiMgLS0tIDEwLjEzIFNVTU1BUlkgLS0tCgplY2hvICIxMC4xMyBTVU1NQVJZIgoKZWNobyAiLS0tIENvcmUgR2xvYmFsIEluZnJhc3RydWN0dXJlIEhlYWx0aCBNYXRyaXggLS0tIgoKcHJpbnRmICJQbGF0Zm9ybSBOb2RlIEluc3RhbmNlLi4uJXNcbiIgXAogICAgIiQoaG9zdG5hbWUgMj4vZGV2L251bGwgfHwgZWNobyB1bnJlc29sdmVkKSIKCmlmIFsgIiR7RkFJTEVEX1NFUlZJQ0VTX0NPVU5UOi0wfSIgLWVxIDAgXTsgdGhlbgogICAgZWNobyAiU3lzdGVtZCBTZXJ2aWNlcy4uLi4uLi4uLi5PSyIKZWxzZQogICAgZWNobyAiU3lzdGVtZCBTZXJ2aWNlcy4uLi4uLi4uLi5DUklUSUNBTCAoJEZBSUxFRF9TRVJWSUNFU19DT1VOVCBmYWlsZWQpIgpmaQoKaWYgWyAteiAiJEZTX09WRVJfTElNSVQiIF07IHRoZW4KICAgIGVjaG8gIkZpbGVzeXN0ZW0gQ2FwYWNpdHkuLi4uLi4uT0siCmVsc2UKICAgIGVjaG8gIkZpbGVzeXN0ZW0gQ2FwYWNpdHkuLi4uLi4uV0FSTklORyIKZmkKCmlmIFsgLXogIiRJTk9ERV9PVkVSX0xJTUlUIiBdOyB0aGVuCiAgICBlY2hvICJJbm9kZSBDYXBhY2l0eS4uLi4uLi4uLi4uLk9LIgplbHNlCiAgICBlY2hvICJJbm9kZSBDYXBhY2l0eS4uLi4uLi4uLi4uLldBUk5JTkciCmZpCgppZiBbIC16ICIkRElTS19FUlJfSFVOVCIgXTsgdGhlbgogICAgZWNobyAiS2VybmVsIERpc2sgSGVhbHRoLi4uLi4uLi5PSyIKZWxzZQogICAgZWNobyAiS2VybmVsIERpc2sgSGVhbHRoLi4uLi4uLi5DUklUSUNBTCIKZmkKCmlmIFsgLXogIiRORVRfRVJSX0hVTlQiIF07IHRoZW4KICAgIGVjaG8gIk5ldHdvcmsgTGluayBIZWFsdGguLi4uLi4uT0siCmVsc2UKICAgIGVjaG8gIk5ldHdvcmsgTGluayBIZWFsdGguLi4uLi4uV0FSTklORyIKZmkKCmVjaG8gIlpGUyBIZWFsdGguLi4uLi4uLi4uLi4uLi4uJFpGU19IRUFMVEhfU1RBVFVTIgplY2hvICJtZGFkbSBIZWFsdGguLi4uLi4uLi4uLi4uLiRNREFETV9IRUFMVEhfU1RBVFVTIgplY2hvICJUaW1lIFN5bmNocm9uaXphdGlvbi4uLi4uLiRUSU1FX1NZTkNfU1RBVFVTIgplY2hvICJOb3JtYWxpemVkIExvYWQvQ29yZS4uLi4uLiRMT0FEX1BFUl9DT1JFIgplY2hvICJPdmVyYWxsIEhlYWx0aC4uLi4uLi4uLi4uLiRPVkVSQUxMX1NUQVRVUyIK' | base64 -d | bash
# ==================== 10. HEALTH CHECK ====================
echo "==================== 10. HEALTH CHECK ===================="
printf "\n"
OVERALL_STATUS="OK"
# --- 10.1 FAILED SERVICES ---
echo "10.1 FAILED SERVICES"
FAILED_SERVICES_COUNT=$(
systemctl --failed \
--type=service \
--no-legend \
--plain \
2>/dev/null | \
awk 'NF {count++} END {print count+0}'
)
if [ "${FAILED_SERVICES_COUNT:-0}" -gt 0 ]; then
echo "CRITICAL: $FAILED_SERVICES_COUNT failed service(s) detected"
systemctl --failed \
--type=service \
--no-pager \
2>/dev/null
OVERALL_STATUS="CRITICAL"
else
echo "OK: No failed systemd services detected"
fi
printf "\n%.0s" {1..3}
# --- 10.2 FILESYSTEMS ABOVE THRESHOLD ---
echo "10.2 FILESYSTEMS ABOVE THRESHOLD"
DF_OUTPUT=$(timeout 8 df -hT 2>/dev/null)
if [ -n "$DF_OUTPUT" ]; then
FS_OVER_LIMIT=$(
printf "%s\n" "$DF_OUTPUT" | \
awk '
NR > 1 {
use=$6
gsub(/%/, "", use)
if (use ~ /^[0-9]+$/ && use >= 80) {
print
}
}
'
)
if [ -n "$FS_OVER_LIMIT" ]; then
echo "WARNING: Filesystems at or above 80% usage:"
printf "%s\n" "$DF_OUTPUT" | head -1
printf "%s\n" "$FS_OVER_LIMIT"
[ "$OVERALL_STATUS" = "OK" ] && OVERALL_STATUS="WARNING"
else
echo "OK: All filesystems are below 80% usage"
fi
else
FS_OVER_LIMIT=""
echo "WARNING: Filesystem usage query unavailable or timed out"
[ "$OVERALL_STATUS" = "OK" ] && OVERALL_STATUS="WARNING"
fi
printf "\n%.0s" {1..3}
# --- 10.3 INODES ABOVE THRESHOLD ---
echo "10.3 INODES ABOVE THRESHOLD"
DF_INODE_OUTPUT=$(timeout 8 df -ih 2>/dev/null)
if [ -n "$DF_INODE_OUTPUT" ]; then
INODE_OVER_LIMIT=$(
printf "%s\n" "$DF_INODE_OUTPUT" | \
awk '
NR > 1 {
use=$5
gsub(/%/, "", use)
if (use ~ /^[0-9]+$/ && use >= 80) {
print
}
}
'
)
if [ -n "$INODE_OVER_LIMIT" ]; then
echo "WARNING: Filesystems at or above 80% inode usage:"
printf "%s\n" "$DF_INODE_OUTPUT" | head -1
printf "%s\n" "$INODE_OVER_LIMIT"
[ "$OVERALL_STATUS" = "OK" ] && OVERALL_STATUS="WARNING"
else
echo "OK: All filesystems are below 80% inode usage"
fi
else
INODE_OVER_LIMIT=""
echo "WARNING: Inode usage query unavailable or timed out"
[ "$OVERALL_STATUS" = "OK" ] && OVERALL_STATUS="WARNING"
fi
printf "\n%.0s" {1..3}
# --- 10.4 MEMORY PRESSURE ---
echo "10.4 MEMORY PRESSURE"
MEM_PRESS_AVG=""
if [ -f /proc/pressure/memory ]; then
MEM_PRESS_AVG=$(
awk '
$1 == "some" {
for (i=1; i<=NF; i++) {
if ($i ~ /^avg10=/) {
split($i, value, "=")
print value[2]
exit
}
}
}
' /proc/pressure/memory
)
if awk -v value="${MEM_PRESS_AVG:-0}" \
'BEGIN {exit !(value > 10.0)}'
then
echo "WARNING: Memory PSI avg10 is high: ${MEM_PRESS_AVG:-unknown}"
cat /proc/pressure/memory
[ "$OVERALL_STATUS" = "OK" ] && OVERALL_STATUS="WARNING"
else
echo "OK: Memory PSI avg10 is within threshold: ${MEM_PRESS_AVG:-0}"
fi
else
echo "INFO: Memory PSI is not available on this kernel"
fi
printf "\n%.0s" {1..3}
# --- 10.5 SWAP USAGE AND ACTIVITY ---
echo "10.5 SWAP USAGE AND ACTIVITY"
SWAP_TOTAL_METRIC=$(
free -b 2>/dev/null | \
awk '/^Swap:/ {print $2}'
)
SWAP_USED_METRIC=$(
free -b 2>/dev/null | \
awk '/^Swap:/ {print $3}'
)
SWAP_PCT_METRIC=0
SWAP_ACTIVITY=0
if [ "${SWAP_TOTAL_METRIC:-0}" -gt 0 ]; then
SWAP_PCT_METRIC=$(
awk \
-v used="${SWAP_USED_METRIC:-0}" \
-v total="${SWAP_TOTAL_METRIC:-1}" \
'BEGIN {printf "%.0f", (used/total)*100}'
)
SWAP_ACTIVITY=$(
vmstat 1 2 2>/dev/null | \
tail -1 | \
awk '{print $7+$8}'
)
echo "Swap usage : ${SWAP_PCT_METRIC}%"
echo "Swap activity : ${SWAP_ACTIVITY:-0} pages/s combined si+so"
if [ "${SWAP_ACTIVITY:-0}" -gt 0 ]; then
echo "WARNING: Active swap-in or swap-out activity detected"
free -h
[ "$OVERALL_STATUS" = "OK" ] && OVERALL_STATUS="WARNING"
elif [ "${SWAP_PCT_METRIC:-0}" -gt 50 ]; then
echo "INFO: Swap usage is above 50%, but no current swap activity was detected"
else
echo "OK: Swap usage and activity are within expected limits"
fi
else
echo "INFO: Swap is not configured"
fi
printf "\n%.0s" {1..3}
# --- 10.6 SYSTEM LOAD ---
echo "10.6 SYSTEM LOAD"
read -r LOAD_ONE_MIN LOAD_FIVE_MIN LOAD_FIFTEEN_MIN _ < /proc/loadavg
CPU_CORES_COUNT=$(
nproc 2>/dev/null || \
grep -c '^processor' /proc/cpuinfo 2>/dev/null || \
echo 1
)
LOAD_PER_CORE=$(
awk \
-v load="${LOAD_ONE_MIN:-0}" \
-v cores="${CPU_CORES_COUNT:-1}" \
'BEGIN {
if (cores < 1) {
cores=1
}
printf "%.2f", load/cores
}'
)
echo "Load average : ${LOAD_ONE_MIN:-0} ${LOAD_FIVE_MIN:-0} ${LOAD_FIFTEEN_MIN:-0}"
echo "CPU cores : ${CPU_CORES_COUNT:-1}"
echo "Load/core : $LOAD_PER_CORE"
if awk -v value="$LOAD_PER_CORE" \
'BEGIN {exit !(value >= 1.0)}'
then
echo "CRITICAL: One-minute load is at or above 1.00 per CPU core"
uptime
OVERALL_STATUS="CRITICAL"
elif awk -v value="$LOAD_PER_CORE" \
'BEGIN {exit !(value >= 0.70)}'
then
echo "WARNING: One-minute load is at or above 0.70 per CPU core"
uptime
[ "$OVERALL_STATUS" = "OK" ] && OVERALL_STATUS="WARNING"
else
echo "OK: Normalized load is within threshold"
fi
printf "\n%.0s" {1..3}
# --- 10.7 DISK AND FILESYSTEM ERRORS ---
echo "10.7 DISK AND FILESYSTEM ERRORS"
if command -v journalctl >/dev/null 2>&1; then
DISK_ERR_HUNT=$(
journalctl \
-k \
-b 0 \
--no-pager \
2>/dev/null | \
grep -Ei \
'I/O error|buffer I/O|blk_update_request|critical medium error|uncorrectable|EXT4-fs error|XFS.*error|BTRFS.*error|ZFS.*error|nvme.*error|bad sector|read-only file system' | \
tail -15
)
else
DISK_ERR_HUNT=$(
dmesg 2>/dev/null | \
grep -Ei \
'I/O error|buffer I/O|blk_update_request|critical medium error|uncorrectable|EXT4-fs error|XFS.*error|BTRFS.*error|ZFS.*error|nvme.*error|bad sector|read-only file system' | \
tail -15
)
fi
if [ -n "$DISK_ERR_HUNT" ]; then
echo "CRITICAL: Disk or filesystem errors detected:"
printf "%s\n" "$DISK_ERR_HUNT"
OVERALL_STATUS="CRITICAL"
else
echo "OK: No disk or filesystem errors detected in the current boot"
fi
printf "\n%.0s" {1..3}
# --- 10.8 NETWORK ERRORS ---
echo "10.8 NETWORK ERRORS"
if command -v journalctl >/dev/null 2>&1; then
NET_ERR_HUNT=$(
journalctl \
-k \
-b 0 \
--no-pager \
2>/dev/null | \
grep -Ei \
'link is down|link down|lost carrier|carrier lost|NETDEV WATCHDOG|transmit queue timed out|tx timeout|reset adapter|firmware failed' | \
tail -15
)
else
NET_ERR_HUNT=$(
dmesg 2>/dev/null | \
grep -Ei \
'link is down|link down|lost carrier|carrier lost|NETDEV WATCHDOG|transmit queue timed out|tx timeout|reset adapter|firmware failed' | \
tail -15
)
fi
if [ -n "$NET_ERR_HUNT" ]; then
echo "WARNING: Network link or adapter errors detected:"
printf "%s\n" "$NET_ERR_HUNT"
[ "$OVERALL_STATUS" = "OK" ] && OVERALL_STATUS="WARNING"
else
echo "OK: No network link or adapter errors detected in the current boot"
fi
printf "\n%.0s" {1..3}
# --- 10.9 SMART HEALTH ---
echo "10.9 SMART HEALTH"
SMART_ERR_FLAG=0
SMART_UNKNOWN_FLAG=0
SMART_DEVICE_COUNT=0
if command -v smartctl >/dev/null 2>&1; then
while IFS= read -r DEV_ID; do
[ -z "$DEV_ID" ] && continue
SMART_DEVICE_COUNT=$((SMART_DEVICE_COUNT + 1))
SMART_OUTPUT=$(
smartctl -H "$DEV_ID" 2>/dev/null
)
if printf "%s\n" "$SMART_OUTPUT" | \
grep -Eqi 'PASSED|SMART Health Status: OK|overall-health.*PASSED'
then
echo "OK: SMART health passed on $DEV_ID"
elif printf "%s\n" "$SMART_OUTPUT" | \
grep -Eqi 'FAILED|BAD|CRITICAL'
then
echo "CRITICAL: SMART health failure reported on $DEV_ID"
printf "%s\n" "$SMART_OUTPUT"
SMART_ERR_FLAG=1
OVERALL_STATUS="CRITICAL"
else
echo "INFO: SMART health could not be determined for $DEV_ID"
SMART_UNKNOWN_FLAG=1
fi
done < <(
smartctl --scan 2>/dev/null | \
awk '{print $1}'
)
if [ "$SMART_DEVICE_COUNT" -eq 0 ]; then
echo "INFO: No SMART-capable devices discovered by smartctl --scan"
elif [ "$SMART_ERR_FLAG" -eq 0 ] && [ "$SMART_UNKNOWN_FLAG" -eq 0 ]; then
echo "OK: All discovered SMART devices reported healthy"
fi
else
echo "INFO: smartctl is not installed"
fi
printf "\n%.0s" {1..3}
# --- 10.10 ZFS HEALTH ---
echo "10.10 ZFS HEALTH"
ZFS_HEALTH_STATUS="NOT INSTALLED"
if command -v zpool >/dev/null 2>&1; then
ZPOOL_FAULT_MAP=$(
zpool status -x 2>/dev/null
)
if [ -z "$ZPOOL_FAULT_MAP" ]; then
echo "INFO: No ZFS pools detected"
ZFS_HEALTH_STATUS="NO POOLS"
elif printf "%s\n" "$ZPOOL_FAULT_MAP" | \
grep -qi 'all pools are healthy'
then
echo "OK: All ZFS pools are healthy"
ZFS_HEALTH_STATUS="OK"
else
echo "CRITICAL: ZFS pool issue detected:"
printf "%s\n" "$ZPOOL_FAULT_MAP"
ZFS_HEALTH_STATUS="CRITICAL"
OVERALL_STATUS="CRITICAL"
fi
else
echo "INFO: ZFS tools are not installed"
fi
printf "\n%.0s" {1..3}
# --- 10.11 MDADM RAID HEALTH ---
echo "10.11 MDADM RAID HEALTH"
MDADM_HEALTH_STATUS="NOT CONFIGURED"
if [ -r /proc/mdstat ] && grep -q '^md[0-9]' /proc/mdstat; then
MDADM_HEALTH_STATUS="OK"
MDADM_PROBLEMS=$(
grep -E \
'inactive|recovery|resync|reshape|check|\[[U_]*_[U_]*\]' \
/proc/mdstat \
2>/dev/null
)
if [ -n "$MDADM_PROBLEMS" ]; then
echo "WARNING: mdadm array is degraded, inactive, or running maintenance:"
cat /proc/mdstat
MDADM_HEALTH_STATUS="WARNING"
[ "$OVERALL_STATUS" = "OK" ] && OVERALL_STATUS="WARNING"
else
echo "OK: Active mdadm arrays appear healthy"
fi
else
echo "INFO: No active mdadm arrays detected"
fi
printf "\n%.0s" {1..3}
# --- 10.12 TIME SYNCHRONIZATION ---
echo "10.12 TIME SYNCHRONIZATION"
TIME_SYNC_STATUS="UNKNOWN"
if command -v timedatectl >/dev/null 2>&1; then
TIME_SYNC_STATUS=$(
timedatectl show \
-p NTPSynchronized \
--value \
2>/dev/null
)
if [ "$TIME_SYNC_STATUS" = "yes" ]; then
echo "OK: System clock is synchronized"
TIME_SYNC_STATUS="OK"
else
echo "WARNING: System clock is not synchronized"
timedatectl status 2>/dev/null
TIME_SYNC_STATUS="WARNING"
[ "$OVERALL_STATUS" = "OK" ] && OVERALL_STATUS="WARNING"
fi
else
echo "INFO: timedatectl is not available"
fi
printf "\n%.0s" {1..3}
# --- 10.13 SUMMARY ---
echo "10.13 SUMMARY"
echo "--- Core Global Infrastructure Health Matrix ---"
printf "Platform Node Instance...%s\n" \
"$(hostname 2>/dev/null || echo unresolved)"
if [ "${FAILED_SERVICES_COUNT:-0}" -eq 0 ]; then
echo "Systemd Services..........OK"
else
echo "Systemd Services..........CRITICAL ($FAILED_SERVICES_COUNT failed)"
fi
if [ -z "$FS_OVER_LIMIT" ]; then
echo "Filesystem Capacity.......OK"
else
echo "Filesystem Capacity.......WARNING"
fi
if [ -z "$INODE_OVER_LIMIT" ]; then
echo "Inode Capacity............OK"
else
echo "Inode Capacity............WARNING"
fi
if [ -z "$DISK_ERR_HUNT" ]; then
echo "Kernel Disk Health........OK"
else
echo "Kernel Disk Health........CRITICAL"
fi
if [ -z "$NET_ERR_HUNT" ]; then
echo "Network Link Health.......OK"
else
echo "Network Link Health.......WARNING"
fi
echo "ZFS Health................$ZFS_HEALTH_STATUS"
echo "mdadm Health..............$MDADM_HEALTH_STATUS"
echo "Time Synchronization......$TIME_SYNC_STATUS"
echo "Normalized Load/Core......$LOAD_PER_CORE"
echo "Overall Health............$OVERALL_STATUS"
