RESILIENT RPA · SYNTHETIC MONITOR · IDEMPOTENT

The hard part isn't the click. It's getting the cron to fire.

Logs into a portal with a real browser, does one thing, and reports what it saw. What's interesting isn't the Selenium — it's the three architectures it took to get a "simple cron" to fire reliably every day, for months, with nobody watching.

3 archs
GitHub Actions → Cloudflare Worker → own mini-PC
idem potent
reads the state, never assumes it: double trigger = no-op
10 min
jitter window so it never hits the target at a fixed instant
6 statuses
every run ends in one: CLOCKED · ALREADY · OUT_OF_WINDOW · NOT_APPLICABLE · MISMATCH · ERROR
Why it exists

Two needs, the same shape of problem.

SYNTHETIC MONITOR

A 200 doesn't tell you the portal works

Drives a real headless browser, does a real login, and captures a screenshot as evidence — so you find out the auth flow broke before your users do. That's how it caught an upstream portal going down intermittently at certain hours.

RELIABLE RPA

An action that can't be skipped

Some recurring web task has to happen within a time window, unattended, and can't silently be skipped. The runner reads the portal's current state and acts idempotently: it never repeats something already done.

The architecture's story

Three architectures to make it fire.

Each fix exposed the next weakest link: trigger → network identity → host.

1

GitHub Actions schedule (the naive one)

A cron: in a workflow. But GitHub's scheduled events are best-effort: under load they lag 1–1.5 h or get dropped. A monitor that silently skips days is worse than no monitor.

❌ DROP/DRIFT
2

Cloudflare Worker as the real clock

Timekeeping moves to a Worker (cron in ~4–7 s, no queue) that calls workflow_dispatch. GitHub's schedule becomes a redundant backup. New failure: the portal timed out logins from datacenter IPs.

❌ IP THROTTLE
3

Self-hosted on a 24/7 mini-PC

Runs under systemd timers on an owned machine. Residential IP → no login throttling. Full control of the clock, the browser, and the retries. ~20 W, months unattended.

✓ RELIABLE
Decision path

Four of these five outcomes are the bot being right.

A scheduled action that runs unattended has one job it must never get wrong: doing the thing twice. This one never decides from memory — it reads what the portal is currently offering and derives what to do from that. Pick a scenario and watch it decide.

scheduler external · may fire late day + window Mon-Fri · 06-10 / 16:30-18:30 headless login 3 attempts · 15s backoff the portal offers one button classify_button() start | finish | other OUT_OF_WINDOW no browser is opened ERROR after 3 attempts CLOCKED the only path that clicks ALREADY shift already open · no click NOT_APPLICABLE no open shift to close · no click MISMATCH unknown button · exits 2
Idempotency

Read the state, never assume it.

What makes redundant triggers safe: if two schedulers fire, the second one sees the action already done, reports ALREADY, and clicks nothing.

🔁

Idempotent

ALREADY / NOT_APPLICABLE states: safe against duplicate/redundant triggers and manual intervention.

⏱️

Jitter

RandomizedDelaySec=600 spreads each run across a 10-minute window: systemd delays by a random 0..600s, always forward, never earlier. The target sees naturally-paced traffic, not a synchronized spike. If there's a minimal gap between two runs, size the nominal so the worst-case jitter still meets it.

🪟

Time windows + holidays

Refuses to act outside sane hours even if mis-triggered. Weekend and holiday gating in two layers: workflow + code.

📄

Nothing ends silently

Every run writes STATUS|HOUR|MESSAGE — even an uncaught exception is trapped to leave one. Retries + backoff (3 attempts, 15s apart), and a screenshot on every outcome that opens a browser. The caller reads the status line and decides whether to shout.

Extension · team briefing

The same clock that clocks in, now alerts and informs.

Riding on the reliable scheduler, a second run sends the team chat the clock-in reminder — and enriches it with the day's menu, weather with forecast, and live travel times to the office. The interesting engineering isn't fetching the data: it's enriching without becoming fragile.

🍽️

Menu of the day

Lesson: scrape the marker, not the list. The portal flags the default dish and the vegetarian one with an icon; the first parser ignored them and dumped all 17 orderable items. The fix read that marker on the node it had been skipping.

🌡️

Weather with forecast

Lesson: the weather upstream returned 503 right on the hour — the peak — and the block dropped out. Retries with backoff instead of skipping: temp + humidity + forecast with icon.

🚗

Door-to-door traffic

Live-traffic times from several zones — inbound in the morning, outbound in the afternoon. The free tier caps QPS: sequential calls with retry. Color by with/without-traffic ratio (🟢🟡🔴).

🧾

Graceful degradation

Each block is optional and independent: if the menu, weather, or traffic block fails, the reminder still goes out. Per-block logging to a daily file + classified errors (401/403/429/5xx/timeout).

briefing.txt
🕘 Reminder: clock in

🍽️ Menu of the day: Pizza
🥗 Vegetarian: Veggie pizza

🌡️ 12° (humidity 58%)
☁️ forecast: cloudy, low 1° / high 15°

🚗 Arrival (with traffic):
   • North zone → 27 min 🟢
   • Downtown   → 47 min 🟡

Same principle as the rest of the project: unattended = observable. A per-block checkpoint (CK-BLOQUES) says what came through and what got skipped; the detail of every external call goes into a daily log for debugging.

The reminder never depends on all three upstreams working. If all three fail, the bare clock-in reminder still goes out — degrade, don't break.
Monitors that lie

Green, all the way through the outage.

The synthetic login monitor in this project exists because of a lesson learned three separate times on other systems of mine: a check does not usually miss an outage by failing to run. It runs, on time, and reports green — because it is measuring something adjacent to the thing that broke.

01

The endpoint that answered the wrong question

A health endpoint returned ready: true, state: CONNECTED for an account that had received nothing for hours. Both fields were true — the process was up and the socket was open. The only signal was an absent field three levels into the JSON. And the first hypothesis, a corrupted session, was wrong: the authorisation had been revoked upstream and the backup already contained the revoked state. The restore was faithful. It faithfully restored a broken thing.

02

The part that works hides the part that doesn't

A watchdog looked for "any recent work" in a log. Audio downloads were dead for a full day while text kept arriving and matching the same pattern, so it saw traffic and stayed green. It was right about the component and blind to the function. Aggregating every kind of work into one check is what makes this invisible.

03

The probe that would also have been green

The replacement probe downloaded the file directly — without ever performing the one step that had broken. It would have passed, cheerfully, for the entire duration of the outage it was written in response to. Caught on a re-read before it shipped. The version that runs walks the production path, including the step that fails, and reports which route it came in through: if it ever succeeds on the fallback, it says so while still green.

A synthetic check never observed failing is not a check. It is decoration with a green light on it. Testing one means breaking the thing on purpose and watching the alarm arrive — and then testing the branch you did not touch, because half of a careful fix is proving you left the working part alone.
The TL;DR

The lessons, on one screen.

lessons.md
# 1. A cron you don't control is a suggestion.
if reliability_matters: own_the_clock()

# 2. Each fix reveals the next weakest link
trigger -> network_identity -> host

# 3. Idempotency is what makes redundancy safe
read_state()   # don't assume

# 4. Unattended = observable
screenshots + structured_result

The right place for a small, private, always-on job is usually a box you own — not someone else's CI. Both schedulers ship included (GitHub Actions and systemd): the job logic is identical, the scheduler is swappable.

Python 3.11+Selenium 4 headlesssystemd timersCloudflare Workerzoneinfo TZscreenshots