Server Monitoring Scripts for Sysadmins

Server Monitoring Scripts for Sysadmins

Photo by Digital Buggu on Pexels

Server Monitoring Scripts for Sysadmins

Every system administrator knows that proactive monitoring is the difference between a minor hiccup and a major outage. While full-scale monitoring solutions like Nagios, Zabbix, and Prometheus exist, they can be overkill for small to medium infrastructure. Lightweight custom monitoring scripts fill this gap perfectly — they are simple to deploy, easy to customize, and consume minimal resources on the systems they monitor. This article presents a suite of practical monitoring scripts that every sysadmin should have in their arsenal.

Why Build Your Own Monitoring?

Off-the-shelf monitoring platforms are powerful but come with substantial complexity overhead. They require dedicated servers or containers to run, database backends to store metrics, and significant configuration time to set up alerts and dashboards. Custom scripts, by contrast, can be deployed on any machine with SSH access, require no dependencies beyond standard Unix tools that are already installed, and can be tailored to your exact infrastructure requirements without any unnecessary overhead.

  • Minimal resource footprint — no agents, databases, or heavy runtimes needed
  • Full control over alerting logic, thresholds, and escalation policies
  • Easy integration with existing incident management tools like Slack, PagerDuty, or Telegram
  • Portable across any Unix-like operating system including Ubuntu, CentOS, Debian, and FreeBSD
  • Can be extended with custom metrics unique to your application’s domain
  • No vendor lock-in — you own the code and can modify it freely

1. CPU and Memory Health Monitor

This script tracks CPU load, memory usage, and swap consumption. It outputs a summary suitable for logging and triggers alerts when thresholds are exceeded. It uses standard Unix tools — top, free, and awk — so it works on any Linux distribution without additional packages.

#!/bin/bash
# cpu-mem-monitor.sh
THRESHOLD_CPU=80
THRESHOLD_MEM=90

CPU_USAGE=$(top -bn1 | grep "Cpu(s)" | awk '{print $2}' | cut -d. -f1)
MEM_USAGE=$(free | grep Mem | awk '{print int($3/$2 * 100)}')

if [ "$CPU_USAGE" -gt "$THRESHOLD_CPU" ]; then
  echo "ALERT: CPU at ${CPU_USAGE}% (threshold: ${THRESHOLD_CPU}%)"
fi

if [ "$MEM_USAGE" -gt "$THRESHOLD_MEM" ]; then
  echo "ALERT: Memory at ${MEM_USAGE}% (threshold: ${THRESHOLD_MEM}%)"
fi

echo "CPU: ${CPU_USAGE}% | MEM: ${MEM_USAGE}%"

Run this script every five minutes via cron to maintain a continuous health record. Redirect output to a log file and use a log watcher to trigger alerts on any line containing ALERT.

2. Disk Space Usage Reporter

Running out of disk space is one of the most common yet completely preventable server failures. This script checks each mounted partition, reports usage percentages, and warns when any partition exceeds a configurable threshold. It also reports inode usage, which is an often-overlooked cause of disk-full errors even when block usage appears low.

Features

  • Scans all mounted partitions except pseudo-filesystems like /proc, /sys, and /dev
  • Reports inode usage in addition to block usage to catch inode exhaustion early
  • Identifies the top 10 largest directories for quick cleanup reference
  • Supports email, Slack, and webhook alerting with customizable message templates

3. Network Connectivity Tester

Network issues are notoriously hard to diagnose because they can be intermittent and affect different services differently. This script checks connectivity to critical endpoints — your database server, API gateway, DNS resolvers, and external services like GitHub or AWS S3. It measures round-trip latency and reports any packet loss, helping you distinguish between a complete outage and degraded performance.

4. Service Status Checker

Ensure critical services are running and accepting connections at all times. This script checks systemd services for their active status, listens on expected TCP ports using /dev/tcp or nc, and verifies HTTP endpoints return the expected status codes. It can also attempt to restart failed services automatically, with a configurable maximum restart count to prevent infinite restart loops.

#!/bin/bash
# service-check.sh
SERVICES=("nginx" "postgresql" "redis-server" "docker")

for svc in "${SERVICES[@]}"; do
  if systemctl is-active --quiet "$svc"; then
    echo "OK: $svc is running"
  else
    echo "FAIL: $svc is stopped"
    systemctl start "$svc"
    echo "Attempted to restart $svc"
  fi
done

Enhance this script with exponential backoff — if a service fails to restart, wait longer before trying again to avoid hammering a failing process.

5. SSL Certificate Monitor

Expired SSL certificates cause immediate and visible outages that erode customer trust and can take hours to recover from if you do not have automated renewal. This script checks all your domains daily and alerts you when certificates are within 14 days of expiration. It supports Let’s Encrypt auto-renewal integration via certbot and sends a pre-emptive warning before the automatic renewal window closes.

6. Log File Growth Analyzer

Unmonitored log files can silently consume gigabytes of disk space, especially after an application enters a verbose logging mode during debugging. This script tracks log file growth rates, identifies files growing abnormally fast compared to their historical baseline, and suggests log rotation adjustments before disk space becomes critical.

7. Temperature and Hardware Health

For physical servers in colocation or on-premise data centers, hardware health is critical and cannot be ignored. This script reads CPU temperature from lm-sensors, checks disk SMART status for impending failures, and reports fan speeds and voltage levels. Early detection of hardware issues can prevent catastrophic data loss.

Centralizing Alerts

Individual scripts are useful on their own, but centralized alerting makes them powerful. Configure all monitoring scripts to send alerts through a common pipeline so you have a single source of truth for all incidents:

  • Email: Use mail or sendmail for traditional email alerts to operations teams
  • Slack/Discord: Post to a dedicated monitoring channel via webhook for instant visibility
  • PagerDuty/Opsgenie: Integrate with professional incident management platforms for on-call rotation
  • Telegram: Send instant messages to a private group for small teams who want a lightweight solution

Setting Up a Monitoring Dashboard

Combine script outputs with a lightweight dashboard tool like Netdata or Cockpit for visual monitoring. Your scripts feed data points that the dashboard can chart over time, giving you both real-time alerts and historical trend analysis. This combination gives you the simplicity of shell scripts with the visualization capabilities of a dedicated monitoring platform.

Scheduling with Cron

Run monitoring scripts at appropriate intervals depending on the criticality of each metric:

# Every 5 minutes for critical checks
*/5 * * * * /opt/scripts/cpu-mem-monitor.sh >> /var/log/monitor.log 2>&1

# Every hour for disk and log checks
0 * * * * /opt/scripts/disk-check.sh >> /var/log/monitor.log 2>&1

# Daily at 6 AM for SSL checks
0 6 * * * /opt/scripts/ssl-check.sh >> /var/log/monitor.log 2>&1

Always redirect both stdout and stderr to a log file so you can review the full execution history when troubleshooting.

Best Practices for Monitoring Scripts

  • Always log output with timestamps for post-mortem analysis and trend identification
  • Set different severity levels (INFO, WARNING, CRITICAL) so alerts can be prioritized appropriately
  • Avoid alert fatigue — only trigger alerts on actionable conditions that require human intervention
  • Use exponential backoff for repeated alerts to avoid notification storms during extended incidents
  • Test alert delivery paths regularly by proactively triggering test alerts
  • Keep scripts idempotent — running them multiple times should not cause side effects

Download the Complete Monitoring Suite

All monitoring scripts featured in this article are available as a free download on freefile.store. The package includes CPU and memory health monitoring, disk space reporter with inode tracking, network connectivity tester, service status checker with auto-restart, SSL certificate expiry monitor, log growth analyzer, and hardware health checker. Each script includes configuration templates, a centralized alerting wrapper, and a cron setup guide. Deploy them on your servers today and gain the peace of mind that comes with knowing your infrastructure is being watched.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *