Back to Blog

Netwatch: A Lightweight Internet Outage Monitor for OPNsense That Actually Sends Alerts

February 25, 2026 Justin 5 min read
OPNsense Networking Monitoring FreeBSD

When your internet goes down, you want to know when it happened, how long it lasted, and what the network looked like at the moment of failure. OPNsense can detect outages, but it can't reliably tell you about them. So I wrote a shell script called Netwatch that does.

The Problem

Monit sends alerts via SMTP. If the internet is down, the email never sends. When it comes back, Monit doesn't retry—the alert is just lost. Postfix should queue mail for later delivery, but on FreeBSD it was unreliable—mail would sit in the queue and never flush. dpinger logs state changes but doesn't email, doesn't collect diagnostics, and doesn't tell you anything useful.

Every tool either can't send alerts during an outage, or doesn't try to. If you only find out about downtime by checking logs manually, you don't have monitoring—you have a diary.

What Netwatch Does

Netwatch is a shell script that runs as a daemon on OPNsense. It pings 9.9.9.9 and 1.1.1.1 every second. When both go down, it collects diagnostics and sends an alert. The key difference from everything else: it handles its own email queue.

If the email can't send (because the internet is down), it writes it to disk. Every loop iteration, it checks the queue and tries to flush it. The moment connectivity returns, the queued alerts go out. Nothing is lost. No Postfix, no sendmail—just curl with SMTP/STARTTLS.

Alert types:

  • [DOWN] — All targets unreachable. Includes WAN IP, default gateway, DHCP lease, recent syslog events, and a traceroute.
  • [RECOVERY] — Internet restored. Includes post-recovery diagnostics and total outage duration.
  • [WAN IP] — Public IP changed (useful for DHCP connections). Includes old/new IP and lease info.

A separate keepalive script runs via cron every minute to restart the daemon if it dies. Survives reboots, OPNsense updates, and unexpected crashes.

Install It

You need SSH access to your OPNsense box and SMTP credentials (any provider that supports STARTTLS on port 587).

One-liner install: 

curl -fsSL https://internetsecure.org/scripts/netwatch/install.sh | sh

This creates the directories, downloads both scripts, sets permissions, adds the cron keepalive, and configures log rotation. Then edit your config: 

vi /usr/local/etc/netwatch/netwatch.sh

Change the first few lines: 

SMTP_HOST="smtp.example.com"   # Your SMTP relay
SMTP_PORT="587"                # STARTTLS port
SMTP_USER="[email protected]"   # SMTP login
SMTP_PASS="your-password"     # SMTP password or app password
MAIL_FROM="[email protected]"   # From address
MAIL_TO="[email protected]"     # Where alerts go
WAN_IF="em0"                  # Your WAN interface name

Start it and verify: 

/bin/sh /usr/local/etc/netwatch/netwatch_keepalive.sh
tail -f /var/log/netwatch.log

You should see netwatch started and WAN IP initialized. That's it—it's running. Cron will keep it alive from here.

Manual Install

If you prefer not to pipe curl into sh: 

# Create directories
mkdir -p /usr/local/etc/netwatch /var/netwatch/queue /var/netwatch/state

# Download the scripts
curl -fsSL https://internetsecure.org/scripts/netwatch/netwatch.sh \
  -o /usr/local/etc/netwatch/netwatch.sh
curl -fsSL https://internetsecure.org/scripts/netwatch/netwatch_keepalive.sh \
  -o /usr/local/etc/netwatch/netwatch_keepalive.sh
chmod +x /usr/local/etc/netwatch/*.sh

# Add cron keepalive + log rotation
echo "* * * * * root /bin/sh /usr/local/etc/netwatch/netwatch_keepalive.sh" >> /etc/crontab
mkdir -p /usr/local/etc/newsyslog.conf.d
echo "/var/log/netwatch.log 644 7 1024 * JC" > /usr/local/etc/newsyslog.conf.d/netwatch.conf

# Edit config, then start
vi /usr/local/etc/netwatch/netwatch.sh
/bin/sh /usr/local/etc/netwatch/netwatch_keepalive.sh

What It Installs

/usr/local/etc/netwatch/
  netwatch.sh              # Main daemon
  netwatch_keepalive.sh    # Cron watchdog

/usr/local/etc/newsyslog.conf.d/
  netwatch.conf            # Log rotation (7 compressed archives, 1MB each)

/var/netwatch/queue/       # Emails waiting to send
/var/netwatch/state/       # Target states, WAN IP, outage flag
/var/log/netwatch.log      # All log output (rotated automatically)

Gotchas

  • Don't paste script contents into the terminal. OPNsense defaults to tcsh, which breaks heredocs and printf. Use the installer or SCP.
  • WAN interface defaults to em0. Check yours with ifconfig or Interfaces → Assignments in the web UI.
  • FreeBSD ping uses -t for timeout. If adapting for Linux, change to -W.
  • OPNsense upgrades may reset cron. Add the job via the web UI (System → Settings → Cron) if you want it to survive upgrades.
  • Gmail needs an App Password. Your regular password won't work with SMTP.

Internet goes down at 2am, comes back at 2:47am. You wake up to two emails: one with a traceroute showing where the break was, one confirming recovery with the duration. No dashboards. No logs to grep. That's the whole point.