Cron jobs

Two files define the scheduled jobs, one per profile, and which one an entry belongs in follows from what runs it. For the story of what these jobs achieve together, see Proactive autonomy; for the mechanism, What fires the schedule.

agents/chat/defaults/cron/jobs.json is the Planning Agent’s roster, and the only store the gateway’s own ticker advances — unless the experimental platformFrontDoor flag has re-homed the gateway, which moves the ticked store to the Platform Agent’s roster and stops this one. Every entry on it is a no_agent script job — a plain subprocess, no model prompted — because that profile’s toolsets are stripped to mcp-router, kanban and memory and it could not run an audit if asked. Five jobs ship: profile-cron-tick, the every-minute dispatcher that ticks every named profile with work due; the hourly cluster-agent-reconcile sweep that keeps Cluster Agent profiles aligned with the live fleet; oobe, which runs first-run onboarding as stages (delivery of the inventory report, the inventory scan, and fleet-wide-cost-analysis, compliance-audit, obtainability-audit and stockout-prevention once, one after another, starting as soon as the scan settles) and removes itself once the report was claimed a few minutes ago and its audits stage is done; and the old bootstrap-inventory-scan and bootstrap-inventory-delivery entries, which ship disabled.

agents/platform/cron/jobs.json is the Platform Agent’s roster, and carries the nine governance watchdogs plus the no_agent entries that run no model: github-repo-watcher, the daily recap described below, the findings-queue nudge described below, the kanban workspace collector, kanban-board-health, which names any kanban card blocked past a day in chat, with the commands to move it, chat-delivery-watch, which notices when scheduled reports stop reaching chat and says so on a GitHub ledger issue and in Cloud Logging rather than in chat, stall-watch, which sweeps the namespaces of every running or reconciling cluster that has a Cluster Agent every half hour for controllers that stopped reconciling and raises an alert per namespace with a new stall, at most three a tick (the cluster’s Cluster Agent triages it in the alert’s thread, and the watch posts one line of its own when the stall clears, and one when a sweep stops at its budget, saying what it covered and where the next tick resumes; spec.harness.stallWatch tunes which namespaces and kinds it reads and how old a stall must be before it alerts), upgrade-readiness-watch, which produces the upgrade readiness report when a cluster’s release channel offers a version it is below, sa-role-watch, which says in chat when the agent’s Google service account holds a role beyond the read-only permission set in the management project or a project in spec.scope, when the role appears, once a day from 12:00 UTC while it is held, and when it goes, and feedback-prompt, described below. profile-cron-tick is what makes it live: each watchdog is a real cron run in its own process, with that profile’s persona, toolsets, skills, model and max_turns. No id may appear on both rosters — two rosters both carrying one is that audit running twice per schedule, concurrently with itself.

eod-event-watcher-daily-report is one of those: a script that renders the k8s-event-watcher recap straight from the session ledger and delivers its stdout verbatim. It sits here despite needing none of the profile’s toolsets because what it reads belongs to the Platform Agent: the session database. Scripts are shared rather than per-profile (the entrypoint links profiles/platform/scripts at the shared directory), so script resolves here exactly as it does on the Planning Agent’s roster.

findings-morning-nudge names findings from the findings queue in the home chat channel, paced so the first days bring a short list rather than the whole queue. It runs hourly and says nothing on most runs. New critical findings are added from 12:00 UTC, at most two a UTC day; non-critical ones from 16:00 UTC, at most three a UTC day, and only while no critical is waiting for a decision, for 12:00 UTC, or for the next day’s allowance. While any non-critical finding it named is still undecided, it adds nothing new of any severity until that finding is dismissed, snoozed or accepted. Once a UTC day, on its first run at or after 12:00 UTC, it repeats up to FINDINGS_DAILY_CRITICALS of the undecided critical findings it named before, with their ids. It also names any non-critical finding that is holding new findings back, with its id and those three decisions, so that hold is never silent for more than a day. While a critical finding it named is undecided, non-critical findings wait, and that wait is not called out separately. One check failing on several objects in one cluster counts as one finding, and a decision on its id applies to every one of those objects still undecided; the count printed beside the id is how many objects that is. It includes objects the last complete sweep did not report, which are not listed by name; the message says how many of them there are. Findings you ask for yourself are never counted, and dismissing or snoozing a finding gives no allowance back the same day. The critical findings the first inventory report names count as added on the UTC day it is delivered, against that day’s allowance, and are repeated from the next day. It adds nothing until the first inventory report’s delivery is claimed, or until 24 hours after onboarding filed its sweep if the report has still not been delivered by then. A run that cannot read the queue fails, which is reported in chat, at most once a UTC day.

These environment variables, set through the PlatformAgent CR’s spec.deployment.env (the chart’s platformAgent.deployment.env), change the limits:

Variable	Default	Meaning
FINDINGS_FIRST_REPORT_CRITICALS	2	critical findings the first inventory report names
FINDINGS_DAILY_CRITICALS	2	new critical findings added per UTC day, and how many are repeated
FINDINGS_NONCRITICAL_MAX	3	new non-critical findings added per UTC day
FINDINGS_NONCRITICAL_AFTER_HOUR	16	the UTC hour (0–23) from which non-critical findings are added

FINDINGS_FIRST_REPORT_CRITICALS is read once, when onboarding files the card that writes the first report, so a later change does not reach that report. 0 adds none of that kind, and for the first report it means the report names none; it does not lift the limit, unlike the alert ceilings’ ALERT_DAILY_LIMIT_*. A value that is not a whole number, is negative, or (for the hour) is outside 0–23 is ignored and the default applies; the warning goes to the run’s stderr, which the scheduler keeps nowhere for a run that succeeds. On an install managed by install.sh, the installer regenerates terraform.tfvars from install.env on every install.sh and upgrade.sh run and install.env has no key for these, so a value added there (in extra_helm_values) is removed by the next run, and a value set only on the Helm release may be reverted with it: set it again after each upgrade. enabled: false on the roster entry turns the nudge off.

feedback-prompt is the one entry that asks the operator something instead of reporting on the fleet. A week after its first tick it posts one message in the install’s home chat channel: how kube-agents has been working out, the public feedback form’s short link, the form’s disclosure that a submission becomes a public issue, and that a reply in the thread reaches the agent. It is delivered once per install, guarded by two marker files in the Platform Agent’s profile home on the data volume, .feedback_prompt_armed (the first tick, which starts the clock) and .feedback_prompt_sent (the claim, taken before the message is printed), so a pod restart or an upgrade never re-sends it; only an uninstall and reinstall does. It is at most once: a post the scheduler records as undelivered (the relay or the chat platform was down that day, or no chat platform was bound yet) is not retried, because the delivery record cannot tell a post that landed from one that did not, and a repeat would risk posting twice; removing .feedback_prompt_sent sends it again on the next tick. Two environment variables, set through the chart value platformAgent.deployment.env (on an install.sh install, a TF_VAR_extra_helm_values line in install.env, as Operator settings shows; on a hand-driven Terraform apply, extra_helm_values in terraform.tfvars), are its per-install configuration. A hand edit of the CR’s spec.deployment.env can be lost at the next chart apply. The two variables are FEEDBACK_PROMPT_ENABLED (default true; false turns it off, and an install that turns it back on later still gets exactly one) and FEEDBACK_PROMPT_DELAY (default 7d; <n>d, <n>h or <n>m; the job runs daily, so the message lands on the first 13:00 UTC tick at or after the delay, and a value that does not parse fails the run, which is reported in chat like any other script failure). The form URL is not configurable. enabled: false on the roster entry is the fleet-wide switch, as for every other job.

The shipping jobs

Generated from agents/chat/defaults/cron/jobs.json and agents/platform/cron/jobs.json. Retired ids are omitted: an id on its way out ships switched off for a release before it is deleted, and a disabled entry on the Platform Agent’s roster is left out of this table rather than listed as a job an operator could reach. See The retired jobs.

ID	Profile	Schedule	Cadence	Enabled	Runs
profile-cron-tick	Planning Agent	* * * * *	—	yes	profile_cron_tick.py
cluster-agent-reconcile	Planning Agent	11 * * * *	Hourly at :11	yes	cluster_agent_reconcile.py
bootstrap-inventory-scan	Planning Agent	* * * * *	—	no	bootstrap_scan_gate.py
bootstrap-inventory-delivery	Planning Agent	* * * * *	—	no	bootstrap_delivery.py
oobe	Planning Agent	* * * * *	—	yes	oobe.py
compliance-audit	Platform Agent	20 6 * * *	Daily 06:20	yes	Run the daily fleet security and RBAC posture audit. Read the SOP at ’/opt/data/profiles/platform/governanc…
obtainability-audit	Platform Agent	50 6 * * *	Daily 06:50	yes	Run the daily workload reliability audit. Read the SOP at ’/opt/data/profiles/platform/governance/obtainabi…
security-patch-orchestrator	Platform Agent	20 7 * * 1	Weekly, Monday 07:20	yes	Run the weekly GKE upgrade and patch readiness audit. Read the SOP at ’/opt/data/profiles/platform/governan…
fleet-wide-cost-analysis	Platform Agent	50 7 * * 1	Weekly, Monday 07:50	yes	Run the weekly fleet waste audit. Read the SOP at ’/opt/data/profiles/platform/governance/fleet_wide_cost_a…
fleet-consistency-drift	Platform Agent	20 8 * * 1	Weekly, Monday 08:20	yes	Run the weekly fleet consistency drift audit. Read the SOP at ’/opt/data/profiles/platform/governance/fleet…
ai-security-audit	Platform Agent	50 8 * * *	Daily 08:50	yes	Run the daily AI workload security audit. Read the SOP at ’/opt/data/profiles/platform/governance/ai_securi…
stockout-prevention	Platform Agent	20 9 * * *	Daily 09:20	yes	Run the daily fleet stockout prevention and capacity audit. Read the SOP at ’/opt/data/profiles/platform/go…
gcp-networking-fabric-audit	Platform Agent	0 8 * * *	Daily 08:00	yes	Run the daily GCP networking fabric and VPC IPAM audit. Read the SOP at ’/opt/data/profiles/platform/govern…
gce-compute-fleet-audit	Platform Agent	45 7 * * *	Daily 07:45	yes	Run the daily GCE Compute Engine and MIG fleet audit. Read the SOP at ’/opt/data/profiles/platform/governan…
findings-morning-nudge	Platform Agent	0 * * * *	Hourly	yes	findings_nudge.py
github-repo-watcher	Platform Agent	*/10 * * * *	Every 10 minutes	yes	github_scan_gate.py
kanban-workspace-gc	Platform Agent	40 5 * * *	Daily 05:40	yes	kanban_workspace_gc.py
kanban-board-health	Platform Agent	35 12 * * *	Daily 12:35	yes	kanban_board_health.py
feedback-prompt	Platform Agent	0 13 * * *	Daily 13:00	yes	feedback_prompt.py
eod-event-watcher-daily-report	Platform Agent	0 21 * * 1-5	Weekdays 21:00	yes	eod_report_generator.py
stall-watch	Platform Agent	*/30 * * * *	Every 30 minutes	yes	stall_watch.py
upgrade-readiness-watch	Platform Agent	10 10 * * *	Daily 10:10	yes	upgrade_readiness_watch.py
sa-role-watch	Platform Agent	25 * * * *	Hourly at :25	yes	sa_role_watch.py
chat-delivery-watch	Platform Agent	*/30 * * * *	Every 30 minutes	yes	chat_delivery_watch.py

Job schema

Both rosters use one schema. A governance watchdog:

{
  "id": "compliance-audit",
  "name": "Security & RBAC Posture Audit",
  "schedule": {
    "kind": "cron",
    "expr": "20 6 * * *",
    "display": "20 6 * * *"
  },
  "prompt": "Run the daily fleet security and RBAC posture audit. Read the SOP at '/opt/data/profiles/platform/governance/compliance_audit_sop.md' — all 443 lines of it, before you run anything. Its sixteen checks are section 2, lines 90-372, so a read that stops early skips almost the entire audit and reports a clean fleet it never looked at. Section 2 opens by running the collector (`/opt/data/profiles/platform/skills/fleet-audit/scripts/collect.py compliance-audit`) in the foreground with the terminal tool's timeout: 600 (the foreground maximum), --workspace set to the workspace start returned and --out /opt/data/scratch/manifest_compliance-audit.json: run it before evaluating any check by hand, read the manifest the way section 2 says, and pass the same file to finish as --manifest-file (if the collector wrote none, section 2 says what to pass instead). Then execute it exactly, using the fleet-audit skill to open and close the audit run.",
  "skills": [
    "fleet-audit"
  ],
  "risk": "low",
  "enabled": true,
  "deliver": "chat"
}

Field	Type	Purpose
id	string	Stable identifier used in observability and enable/disable ops. It survives renames — obtainability-audit is now the Workload Reliability Audit.
name	string	Human-readable name for logs. For the audits it is also the ledger issue title, via the AUDITS map in fleet-audit’s audit_report.py.
risk	string	Declared risk tier ("low" or "high"). For agentic jobs, "high" enforces a read-only policy (cron_command_policy_block); "low" retains the configured mode. Runtime-created jobs default to "low", legacy volume entries backfill to "low", and unannotated in-flight dispatches default fail-closed to "high". For no_agent: true jobs, "high" is an input-threat label: repository events for github-repo-watcher, cluster status and event text for stall-watch, which its alerts leave out and the Cluster Agent reads again.
schedule.kind	string	"cron" on every entry. "interval" is supported but unused: Hermes re-anchors an interval job to when the last run finished, and the gateway ticker sleeps a fixed 60 seconds after each tick returns, so a 1-minute interval fires every two.
schedule.expr	string	Standard 5-field cron expression, evaluated in the pod’s time zone (UTC unless overridden).
schedule.display	string	Display form (usually equal to expr).
prompt	string	What the run is asked to do, copied verbatim into the turn. Governance jobs name their SOP relative to the profile home — governance/<sop>.md. It lives here and nowhere else.
skills	array of string	The skills the run needs. The scheduler force-loads each one’s text ahead of the first turn, rather than leaving the load to the model’s discretion. The audit watchdogs use fleet-audit. A no_agent job prompts no model, so the field is ignored there.
no_agent	bool	Set on every job on the Planning Agent’s roster and on the Platform Agent’s non-watchdog entries, github-repo-watcher, eod-event-watcher-daily-report, findings-morning-nudge, kanban-workspace-gc, kanban-board-health, chat-delivery-watch, stall-watch, sa-role-watch and feedback-prompt: the tick is a subprocess, not an LLM turn. The governance watchdogs omit it.
script	string	For a no_agent job, the script to run, resolved in that profile’s scripts/. The scheduler runs it with no arguments.
enabled	bool	Set false to disable without deleting the entry. See Disabling a watchdog — a deleted entry is not removed from a cluster that already has it.
deliver	string	Where the run’s outcome goes. "all" sends it to the configured target; "chat" posts it into a thread where a reply reaches the Planning Agent with the report attached; "local" resolves to no target at all and drops it. Every enabled job on the Platform Agent’s roster uses "chat" or "all", so a job that has stopped working is visible rather than indistinguishable from a quiet fleet, with one exception: chat-delivery-watch uses "local" because its product is a GitHub issue and a log line, and its purpose is to work when chat does not.

Editing

Adding or editing a job is a one-file change — see Adding a watchdog.

Edit jobs.json, then redeploy the agent image at the revision carrying the change:

./upgrade.sh --upgrade-mode=harness --image-tag=<SEMVER_TAG_OR_FULL_COMMIT_SHA>

The change is picked up on the next pod restart.

How an edit reaches an existing pod

This part is about the Planning Agent’s file, agents/chat/defaults/cron/jobs.json — the one the start-up reconcile below acts on. The Platform Agent’s roster, which the rest of this page documents, reaches its profile by a separate path: profile_scaffold.py scaffolds it, and merge_cron_store merges it under the same per-key rule.

$HERMES_HOME/cron/jobs.json lives on the agent’s persistent volume, and the scheduler writes last_run back into it on every tick — so the volume’s copy is always newer than the image’s, and the entrypoint’s update-only defaults copy never overwrites it. Simply overwriting the file is not an option either: it would reset every last_run (making all jobs look due at once), discard the chat binding first-run onboarding writes, and reinstate the two onboarding jobs that finishing onboarding deliberately removes, and oobe on an install that finished onboarding before it shipped.

So docker-entrypoint.sh runs cron_jobs_sync.py before the scheduler starts, merging the image’s declarations into the volume’s file by job id. The merge is per key, not per field-list: the image wins every key it ships, and every key it does not ship is left as the volume had it.

• A job’s definition — name, schedule, prompt, skills, script, no_agent, and enabled — is shipped, so it tracks the image. Shipping enabled: false is therefore the fleet-wide off switch described in Disabling a watchdog; the cost is that a job disabled by hand on a live pod is switched back on by the next image roll.
• The scheduler’s own state — last_run and whatever else Hermes records per job — is shipped by nothing, so it survives untouched. Stating the rule this way rather than as a list of runtime-owned names is deliberate: a list has to be complete to be correct, and it stops being complete the day upstream records a new field.
• deliver is the single exception, because a runtime hook owns it: onboarding rewrites it to origin on the delivery job, and taking the image’s value back would send the report nowhere.
• A job the image does not declare is never deleted; it may have been added by hand. Retirement therefore does not propagate to existing volumes — drop an id only once every live cluster has merged its disabled form.
• A job this script has installed before and that is now absent was removed on purpose, so it is not reinstalled. A ledger at $HERMES_HOME/.cron_jobs_installed records which ids those are.

The same start-up reconcile applies to the specialist profiles’ skills/ directories, which are wholly image-owned and replaced from the baked template on every start — see Skills.