Server reboot notification

The server rebooted at night — and you only found out in the morning, when users complained about “dropped” sessions. An unexpected reboot almost always means something serious: a kernel panic, a watchdog, a power outage, or someone’s reboot over SSH. A notification at boot time saves hours of investigation.

What we’ll do

We’ll create a oneshot systemd unit that runs once on every boot (WantedBy=multi-user.target). The script:

• collects uptime, the last boot time, and the reason;
• distinguishes a planned reboot from an unexpected one via a marker file: before a normal reboot/shutdown we drop a marker, and at boot we check for it;
• on an unexpected reboot it sends a high-priority message, on a planned one — a normal “server is back”.

Script

Save as /usr/local/bin/notifly-reboot-check:

#!/usr/bin/env bash
set -eu
set -a; source /etc/notifly.env; set +a

HOST=$(hostname -s)
MARKER=/var/lib/notifly-reboot/planned

# Last boot time and uptime in human-readable form
BOOT_TIME=$(uptime -s)
UPTIME_H=$(uptime -p)

# Last lines about previous shutdown reasons — panic/power loss are often visible
LAST_REBOOT=$(last -x -n 5 reboot shutdown 2>/dev/null | head -5 || true)

if [ -f "$MARKER" ]; then
    # Marker is present — shutdown was graceful
    rm -f "$MARKER"
    /usr/local/bin/notifly-send \
        "🔄 $HOST перезагрузился (планово)" \
        "Загрузка: $BOOT_TIME
Аптайм: $UPTIME_H" 4
else
    # No marker — the reboot was unexpected
    DMESG_ERR=$(journalctl -k -b -1 -p err --no-pager 2>/dev/null | tail -5 || true)
    /usr/local/bin/notifly-send \
        "⚠️ $HOST перезагрузился НЕОЖИДАННО" \
        "Загрузка: $BOOT_TIME
Аптайм: $UPTIME_H

Последние ребуты:
$LAST_REBOOT

Ошибки прошлого сеанса:
$DMESG_ERR" 9
fi

Make it executable:

sudo chmod +x /usr/local/bin/notifly-reboot-check

Planned-reboot marker

Before a graceful shutdown we need to drop a marker. The easiest way is a separate unit that fires when the system stops:

/etc/systemd/system/notifly-reboot-marker.service:

[Unit]
Description=Mark planned reboot for Notifly
DefaultDependencies=no
Before=shutdown.target reboot.target halt.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/mkdir -p /var/lib/notifly-reboot
ExecStop=/bin/touch /var/lib/notifly-reboot/planned

[Install]
WantedBy=multi-user.target

The unit stays “up” the whole time the system runs, and the marker is created by ExecStop — that is, exactly at the moment of a graceful systemctl reboot/shutdown. On a sudden power loss ExecStop won’t run, there will be no marker — and we’ll know the reboot was unexpected.

Run on boot

/etc/systemd/system/notifly-reboot.service:

[Unit]
Description=Notifly reboot notification
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/notifly-reboot-check

[Install]
WantedBy=multi-user.target

Enable both units:

sudo systemctl daemon-reload
sudo systemctl enable --now notifly-reboot-marker.service
sudo systemctl enable notifly-reboot.service

Test manually:

sudo systemctl start notifly-reboot.service

Windows: PowerShell + Task Scheduler

On Windows the boot moment is visible in the System Event Log: Event ID 6005 (“The Event log service was started” — a normal start), 6008 (“The previous system shutdown was unexpected”) and 6009 (the OS version at boot). Let’s subscribe to these events. Uses the common function Send-Notifly from the sysadmin template/index.

C:\scripts\Notifly-Reboot-Check.ps1

. C:\scripts\Notifly.ps1

$Host = $env:COMPUTERNAME
$uptime = (Get-Date) - (Get-CimInstance Win32_OperatingSystem).LastBootUpTime
$boot   = (Get-CimInstance Win32_OperatingSystem).LastBootUpTime

# Was the previous shutdown unexpected? (Event 6008 in System)
$unexpected = Get-WinEvent -FilterHashtable @{LogName='System'; Id=6008} -MaxEvents 1 -ErrorAction SilentlyContinue
$isUnexpected = $unexpected -and $unexpected.TimeCreated -gt (Get-Date).AddMinutes(-15)

if ($isUnexpected) {
    Send-Notifly -Title "⚠️ $Host перезагрузился НЕОЖИДАННО" `
                 -Message "Загрузка: $boot`nАптайм: $([int]$uptime.TotalMinutes) мин.`nПредыдущее выключение было аварийным (Event 6008)." `
                 -Priority 9
} else {
    Send-Notifly -Title "🔄 $Host перезагрузился (планово)" `
                 -Message "Загрузка: $boot`nАптайм: $([int]$uptime.TotalMinutes) мин." `
                 -Priority 4
}

Register a task at system startup:

$Trigger = New-ScheduledTaskTrigger -AtStartup
$Trigger.Delay = "PT1M"   # wait a minute after boot
$Action  = New-ScheduledTaskAction -Execute "powershell.exe" `
    -Argument "-NoProfile -ExecutionPolicy Bypass -File C:\scripts\Notifly-Reboot-Check.ps1"
$Princ   = New-ScheduledTaskPrincipal -UserId "SYSTEM" -LogonType ServiceAccount -RunLevel Highest
Register-ScheduledTask -TaskName "Notifly Reboot Check" `
    -Action $Action -Trigger $Trigger -Principal $Princ -Description "Notifly: уведомление о перезагрузке"

Benefits

• Zero silent reboots. Any restart is immediately visible on your phone.
• Planned or not — clear right away. The marker file separates your own reboots from crashes, so you don’t get used to ignoring the green messages.
• Context in your pocket. The message includes the boot time, uptime, and errors from the previous session — often enough to understand the cause.

What to improve next

• Attach the full output of journalctl -k -b -1 -p warning via the extras field to analyze the panic right in the app.
• Link with the heartbeat check: if the server never sends a sign of life after a reboot, it didn’t come back up.
• Count the reboot rate: three unexpected reboots in an hour is already a hardware problem — worth raising the priority to 10.