Pixel Pete, the CanaryHub canaryCanaryHub

How to Monitor a Cron Job (a Dead Man's Switch in 5 Minutes)

How to Monitor a Cron Job (a Dead Man's Switch in 5 Minutes)

Quick answer

Add one curl call to the end of every cron job that pings a unique monitoring URL on success. Configure the monitor with the job's expected interval plus a grace period; if the ping doesn't land in time, you get alerted. That's a dead man's switch — silence is the failure signal, not an error code.

Cron will run your job at 3am forever. It will also stay completely silent forever if that job starts failing — a bad deploy breaks the script, a dependency's API starts rejecting requests, a disk fills up — and cron won't tell you. It ran, from cron's point of view. Whether the script actually did anything useful isn't cron's problem.

This is why "the backup script has been failing for six weeks" is a genre of incident report. Nobody was watching for absence.

Why cron jobs fail silently

A web request that 500s shows up in your logs, your APM, maybe a Slack alert from your error tracker. A cron job that silently no-ops, exits early, or hangs produces none of that by default:

  • The exit code lies. A script can hit an early return after a partial failure and still exit 0.
  • Nobody's watching the logs. Cron output usually goes to /var/mail or /dev/null, not somewhere a human looks.
  • The failure mode is often "didn't run at all." The container never started, the deploy broke the crontab, the host rebooted and the scheduler didn't come back. There's no error to catch because there's no execution to fail.

Traditional monitoring watches for bad signals — errors, non-zero exit codes, exceptions. A dead man's switch watches for a missing good signal instead, which is the only thing that works when the failure mode is silence.

The pattern: ping on success, alert on absence

The setup is the same regardless of what platform you use to receive the pings:

  1. Create a monitor with a unique URL and an expected interval (e.g. "should receive a ping every 60 minutes").
  2. At the very end of your cron job — after the real work has succeeded — call that URL.
  3. If the monitor doesn't see a ping within the interval plus a grace period, it alerts you.
#!/usr/bin/env bash
set -euo pipefail

/usr/local/bin/backup.sh

# Only reached if the line above succeeded. This is the whole integration.
curl -fsS -m 10 --retry 3 "https://api.canaryhub.ai/ping/YOUR_MONITOR_ID" > /dev/null

That's the entire pattern. The ping call sits after your real logic, guarded by set -e (or an explicit exit-code check in other languages), so a failure earlier in the script means the ping never fires — which is exactly the signal you want.

Where to put the ping in longer pipelines

For multi-step jobs, ping once at the true end, not after step one:

set -euo pipefail
fetch_data
transform_data
upload_results
curl -fsS -m 10 "https://api.canaryhub.ai/ping/YOUR_MONITOR_ID" > /dev/null

If transform_data throws, set -e stops the script before the ping line runs, so the monitor goes quiet and you get alerted — even though there's no "error" for a log-scraper to find.

Handling long-running or flaky jobs

Two adjustments cover almost every edge case:

  • Grace period. Set it above the job's worst normal-case duration, not its average — see the FAQ below for why padding it too generously defeats the point.
  • Start/finish pings for long jobs. Some setups support a "start" ping and a "finish" ping so you can catch a job that starts but never finishes (stuck, deadlocked, killed by an OOM killer) rather than one that never starts at all.

What "good" monitoring coverage looks like

A dead man's switch on a cron job is cheap insurance for a specific, common failure class. It pairs with — doesn't replace — the monitoring you already have:

  • Uptime checks on the endpoints your users hit directly.
  • Error tracking for exceptions your app actually throws.
  • Dead man's switches for anything that's supposed to happen periodically without a request initiating it: cron jobs, queue workers, scheduled reports, backups, cert renewals.

If a scheduled task matters enough that its silent failure would cost you a support ticket, a data gap, or a lapsed backup, it's worth the one line of curl.

Frequently asked questions

What is a dead man's switch, exactly?
It's a monitor that expects to hear from you on a schedule. As long as the pings keep arriving on time, it stays quiet. The moment a ping is late or missing, it assumes something died and fires an alert — the same principle as a train conductor's pedal that has to stay pressed.
Do I need to change my cron job's script?
One line. Append a curl (or wget) call to a unique monitoring URL at the very end of the script, after everything else has succeeded. No SDK, no new dependency, no restructuring.
What if my job legitimately runs long sometimes?
Set the monitor's grace period a bit above your job's worst normal-case runtime, not its average. You want slack for a slow day, not for a stuck job — if you're padding it more than 2-3x average, that's a sign the job itself needs a timeout.