How to Run a Python Cron Job (and Catch It When It Stops Running)

Four patterns cover almost every Python cron job: the OS crontab calling a script directly, the schedule library running a loop inside a long-lived process, APScheduler for proper cron-style triggers with more control, or Celery Beat if a Celery worker is already part of your stack. Pick based on how long the process needs to stay alive and whether the schedule should live in a file or in code. All four have the same blind spot: they can tell you a job ran, or that it raised an exception, but not when it stopped running at all. That third case is the one that costs people the most sleep.

What's the best way to schedule a job in Python?

Use the OS crontab when the job is a short script and once-a-minute resolution is fine. Point a crontab line at a script and let the OS own the schedule entirely:

# /etc/cron.d/nightly-report
0 3 * * * deploy /home/deploy/venv/bin/python /home/deploy/app/nightly_report.py >> /var/log/nightly-report.log 2>&1

This is the simplest option because there's no scheduler state living anywhere. The script either exits 0 or it doesn't.

Use the schedule library for a lightweight, in-process timer when a long-running process is already part of the plan. It reads close to English and needs nothing beyond pip install schedule:

import schedule
import time

def nightly_report():
    # job logic
    ...

schedule.every().day.at("03:00").do(nightly_report)

while True:
    schedule.run_pending()
    time.sleep(1)

The catch: schedule keeps all of its state in memory. Restart the process and every schedule is gone until the loop starts back up, and nothing about the library itself notices or recovers a run it missed while it was down.

Use APScheduler when you want real cron syntax, more trigger types (cron, interval, or a one-off date), or a scheduler that persists its jobs to a database and survives a restart:

from apscheduler.schedulers.blocking import BlockingScheduler

scheduler = BlockingScheduler()

@scheduler.scheduled_job("cron", hour=3, minute=0)
def nightly_report():
    # job logic
    ...

scheduler.start()

BlockingScheduler takes over the thread it's started on, which is fine for a dedicated scheduler process. If the schedule needs to run alongside other work in the same process, apscheduler.schedulers.background.BackgroundScheduler runs it on its own thread instead. Point APScheduler at a database-backed job store (SQLAlchemy, Redis, MongoDB, and others are supported) and it can pick back up jobs it missed while the process was offline, which plain schedule cannot do.

Use Celery Beat if a Celery worker is already running in your stack. Beat sends a task onto the existing Celery queue on a schedule, instead of running the job in the scheduler process itself, which means you inherit Celery's retries and worker pool for free:

from celery import Celery
from celery.schedules import crontab

app = Celery("tasks", broker="redis://localhost:6379/0")

app.conf.beat_schedule = {
    "nightly-report": {
        "task": "tasks.nightly_report",
        "schedule": crontab(hour=3, minute=0),
    },
}

@app.task
def nightly_report():
    # job logic
    ...

Don't reach for Celery just to run one cron job; the broker and worker process are real infrastructure to run and monitor on their own. It earns its place when you already have it for background task processing and want scheduled jobs to share that same queue.

Use python-crontab not as a scheduler itself, but to create or edit crontab entries from a deploy script instead of hand-editing the file. It wraps the same underlying OS crontab, so none of the environment quirks below go away.

Why does my Python cron job fail silently under cron but work fine when I run it manually?

Because cron runs your script in a far smaller environment than your terminal does, and the three most common gotchas all produce no error message at all:

• Wrong Python interpreter. Your terminal's python resolves through a shell profile and an activated virtualenv. Cron has none of that, so a bare python script.py line often runs the system Python, missing every package your script depends on, which fails with an import error buried in a log file nobody's tailing. Call the virtualenv's interpreter by its absolute path instead: /home/deploy/venv/bin/python.
• Relative paths. A script that does open("config.json") works when you run it from your project directory by hand. Cron doesn't start your script from that directory, so the same relative path resolves to nothing and raises FileNotFoundError. Use absolute paths, or have the script compute paths relative to its own file location with os.path.dirname(os.path.abspath(__file__)).
• Missing environment variables. Cron does not source your .bashrc or .profile, so anything exported there (API keys, a custom PATH, locale settings) simply isn't present when the job runs. Set what the job needs directly in the crontab line, or have the script load a .env file with an absolute path.

Each of these fails the same way from the outside: the crontab entry fired, cron considers that a success, and the script died before doing anything useful. Redirect both stdout and stderr to a log file (as in the crontab line above) so at least the error is somewhere, but a log file nobody checks is only slightly better than silence.

How do I find out when a Python cron job stops running entirely?

Add a heartbeat: a ping sent after the job's real work succeeds, to a monitor that expects to hear from you on a schedule and alerts the moment it doesn't. It watches for absence instead of failure, which is the only thing that catches a job that never ran at all, whether because the host didn't reboot cleanly, a deploy overwrote the crontab entry, or the scheduler process crashed before logging anything.

import logging
import requests

def nightly_report():
    try:
        generate_nightly_report()
    except Exception:
        logging.exception("nightly report failed")
        return  # don't ping on failure; let the silence be the signal here too

    try:
        requests.post("https://ingest.canaryhub.ai/ping/YOUR_TOKEN", timeout=5)
    except requests.RequestException:
        logging.warning("heartbeat ping failed")

The whole integration is that one requests.post call at the true end of the function, guarded so an exception skips it. This pattern drops into any of the four scheduling approaches above unchanged, whether it's called from a crontab script, a schedule job, an APScheduler trigger, or a Celery task. CanaryHub's heartbeat monitors work on the same GET|POST https://ingest.canaryhub.ai/ping/<token> endpoint regardless of what's calling it; the token in the URL is the auth, so there's nothing else to configure. Set the monitor's expected interval to match the job's schedule plus a grace period, and it goes from quiet to alerting the first time a ping is late, without waiting for a second missed run.

This pairs with, rather than replaces, the error handling and log redirection above. Logs and an error tracker tell you why a run failed. A heartbeat tells you a run didn't happen in the first place, which is exactly the case nothing else here can see.

A working baseline

Pick a scheduling approach based on how long the process needs to live (OS crontab for a short script, schedule or APScheduler for a long-running process, Celery Beat if Celery is already there), fix the three environment gotchas that only show up outside your terminal, and ping a heartbeat monitor from the one line at the end of the job that only runs on success. That combination catches all three ways a Python cron job actually goes wrong: it raises an exception, it dies quietly from an environment mismatch, or it stops running altogether and leaves nothing behind to explain why.