Skip to content

In-cluster agent

The agent runs the same evaluation as scan, in the cluster, on a timer, and keeps the answer in a ClusterReadiness object: kubectl get ucr shows it, alerts and GitOps health checks read it, and nothing outside the cluster is needed.

Install

From v0.2.0 the chart is published as a signed OCI artifact:

helm install upgradescope oci://ghcr.io/abd-ulbasit/charts/upgradescope \
  --version 0.3.0 -n upgradescope --create-namespace

To run unreleased changes, install from a clone: helm install upgradescope deploy/chart -n upgradescope --create-namespace, with --set image.repository=…,image.tag=… naming an image you built from the clone (make docker-build) and pushed or loaded into the cluster: a clone's chart defaults to the image of the release it was cut for. Either way the release installs the agent Deployment, its ServiceAccount and read-only ClusterRole, and the ClusterReadiness CRD. Install lists the options, Helm values every value.

Read the result

kubectl get ucr

The columns come from the CRD: TARGET (the first target), SCORE, READY (the verdict: ready, blocked or unknown), BLOCKERS (the first target's blocker count), LASTEVALUATED and AGE. For example:

NAME      TARGET   SCORE   READY     BLOCKERS   LASTEVALUATED   AGE
cluster   1.37     75      blocked   2          4m              2d

Every column, and the Ready condition, read the first target. With several targets, the AllTargetsReady condition covers all of them: False when any target is blocked (its message names them, such as 1.38 blocked (3 blockers)), Unknown when none is blocked but one was not assessed, True only when every target is ready. Read it when the plan has more than one hop:

kubectl get ucr cluster -o jsonpath='{.status.conditions[?(@.type=="AllTargetsReady")].message}'

kubectl get ucr cluster -o yaml has the full status: per target the score, verdict, counts by severity and category, the top 20 findings, and what was not assessed. The complete list of findings is in upgradescope scan and on the server.

The agent ticks once at startup and then every agent.interval (10m by default, 1m at least) with ±10% jitter on the later ticks; the first is immediate, so the status appears as soon as possible. Its pod is Ready once the first tick has written the status, so helm install --wait waits for a first answer. A push to a server does not hold that back: with the in-chart server (server.enabled=true), which may not be listening yet at the first tick, the agent retries the snapshot between ticks (after 10s, doubling to every 2 minutes), so the first snapshot usually reaches the server within seconds of it starting to listen, at most about 2.5 minutes after, not at the next tick (Fleet server). To block a script or pipeline on readiness:

kubectl wait clusterreadiness/cluster --for=condition=Ready --timeout=15m

Ready is the first target's verdict. For a plan of several targets, wait on AllTargetsReady instead (--for=condition=AllTargetsReady), so a blocked later target does not pass.

The object is named by agent.crName (--cr-name), which must be an RFC 1123 subdomain: dot-separated labels of lowercase letters, digits and -, each starting and ending with a letter or digit, at most 253 bytes. The agent refuses to start under any other name, and the chart's schema refuses it at helm install. Changing it creates a new object and leaves the old one at its last verdict, because the agent's role has no delete permission and Helm does not own the object. Delete the old one yourself:

kubectl delete ucr <old-name>

The status is only as fresh as the agent's last successful write. An agent that can no longer write it (its role narrowed, for example) leaves the last verdict and Ready condition in place, but marks the object: the annotation upgradescope.basit.engineer/status-error holds the time of the failed write and its reason, and the next successful write removes it. A tick that could not read the object's spec, or set its spec.targets, writes no status and marks the object the same way: it does not know which targets the spec asks for. Collection never takes the whole tick: part of the tick deadline is kept for the status write and this marker (observability).

kubectl get clusterreadiness cluster -o jsonpath='{.metadata.annotations.upgradescope\.basit\.engineer/status-error}'

Treat a verdict on an object carrying it as stale. If the role lost patch on the object as well, nothing can be marked: the agent's /readyz, the upgradescope_agent_last_success_timestamp_seconds metric and the chart's UpgradescopeAgentNotTicking alert report the failure. Once at least 3 ticks in a row have failed and the last success is older than the alert's threshold, the agent also stops exporting its verdict, score, findings and capability gauges (see Observability).

Choose the targets

With no targets, the agent evaluates the next minor above the cluster's version. To evaluate others, set them in the chart (the agent then resets spec.targets to this list on every tick):

helm upgrade upgradescope deploy/chart -n upgradescope --reset-then-reuse-values \
  --set-json 'agent.targets=["1.37","1.38"]'

or leave agent.targets empty and set them on the object, for example from Git (GitOps):

kubectl patch ucr cluster --type merge -p '{"spec":{"targets":["1.37","1.38"]}}'

An edit to spec.targets or spec.ignore takes effect at the agent's next tick, not at once: the agent reads the spec only inside a tick, so the old verdict stays until then, up to agent.interval plus its 10% jitter (about 11 minutes with the 10m default; the interval may be set up to 1h). status.observedGeneration is the signal: while it is lower than metadata.generation, the status does not yet reflect your edit. To wait for it (here for generation 5; read the current one with kubectl get ucr cluster -o jsonpath='{.metadata.generation}'):

kubectl wait ucr/cluster --for=jsonpath='{.status.observedGeneration}'=5 --timeout=15m

To shorten the lag, lower agent.interval (1m at least).

A target past the knowledge base's horizon reads unknown, with a kb-coverage gap: the knowledge base cannot know what that release removes. The default target becomes such a target once the cluster runs the horizon minor itself, until a release with a newer knowledge base is installed.

Accept known findings

spec.ignore takes the same rules as the CLI's .upgradescope.yaml, with a required reason and an optional expiry; suppressed findings are counted in the status but leave the score and verdict. Like a target edit, a new rule applies at the next tick; see Choose the targets for how to wait for it. Suppressions and baselines has the format.

What the agent can touch

It reads with get and list only, never watch, and writes only its own ClusterReadiness object, that object's status, and (with agent.manageCRD=true, the default) the clusterreadinesses.upgradescope.basit.engineer CRD. Reading Helm releases needs get/list on Secrets and ConfigMaps cluster-wide, which RBAC cannot narrow; rbac.helmSecrets=false removes that grant at the cost of Helm findings. Security model and RBAC lists every rule.

Next