Changelog¶
All notable changes to upgradescope are documented here.
The format is based on Keep a Changelog, and the project follows Semantic Versioning. Before 1.0, a minor release may change flags, API responses or the CRD schema. Any such change is listed under Changed.
The embedded knowledge base (API lifecycle data and the add-on registry) ships inside the binary and the image. Knowledge-base updates therefore reach you only through a new release.
Unreleased¶
Project¶
- upgradescope is in maintenance mode: the weekly data refresh, fixes and Kubernetes 1.38 data continue; the v0.4.0 features are on hold until users ask for them (ROADMAP.md, #426). krew-index declined the plugin (#75).
Added¶
- A reproducible survey of the 50 most-starred Helm charts on Artifact Hub
against Kubernetes 1.37, run with the v0.3.0 release: the method, the
data (
results.json,results.csv) and hand checks are indocs/research/helm-chart-survey-2026-10/, andhack/chart-survey.shre-runs it (#412).
Changed¶
- The Action examples in the README and the docs, and the README's verification example, pin v0.3.0.
0.3.0 - 2026-10-12¶
A release about add-on data: whether the add-on versions you run are past end of life or will run on your next Kubernetes, with every range cited and two sources of false blockers removed. The README now leads with that, and the project has governance files and a public roadmap (#426).
Added¶
- Seven add-ons in the registry, which now holds 34: Crossplane, Linkerd, the AWS Load Balancer Controller, the AWS EFS CSI driver, the AWS EBS CSI driver, Cluster Autoscaler and Longhorn (#394).
- Linkerd, the AWS Load Balancer Controller, the AWS EFS CSI driver and Longhorn carry Kubernetes ranges read from their own pages, and Istio gains 19 rows from istio.io's supported-releases table. 15 of the 34 entries now carry a range.
- Crossplane's end-of-life dates come from its own release table, because endoflife.date does not track it. Longhorn's are synced from endoflife.date.
- The add-ons whose projects publish no Kubernetes bound (Argo CD, Flux, Gatekeeper, Traefik, CoreDNS, AKS app routing, kube-state-metrics, Cluster Autoscaler and the AWS EBS CSI driver) say so in their entries, with the page that was checked and the date.
MAINTAINERS.md,GOVERNANCE.md(with a contributor ladder),ADOPTERS.mdandROADMAP.md, linked from the README and CONTRIBUTING (#409).- A
# eol-sync: ignore-k8s-rangeline in a registry entry keeps endoflife.date's end-of-life dates but drops its Kubernetes range, for products that publish only a tested-with table (registry/CONTRIBUTING.md).
Changed¶
- The README leads with add-on end of life and compatibility, custom
resources at CRD versions no longer served, stored Helm release manifests,
and pull-request gating with
--files. The long material moved to the docs site (3,171 words to 1,181), and the README names kube-upgrade-check for the checks upgradescope does not do yet (#404). - The README's install table lists the chart on Artifact Hub and the Action on GitHub Marketplace, and the Action examples in the README and the docs pin a release tag (#75).
- The krew manifest has no usage lines, its short description is "Find what blocks your next upgrade", and its caveats list the PersistentVolume and StorageClass reads that a scan needs, as the krew-index review asked (#75).
- The Renovate preset lists every chart in the registry.
- The Fluent Bit entry records that endoflife.date's dates disagree with Fluent Bit's own SECURITY.md (5.0 ends on 2026-11-06 at endoflife.date and on 2026-11-30 upstream). The synced dates are unchanged until the upstream data is corrected (#408).
Fixed¶
- KEDA and Kyverno no longer raise
chart-incompatblockers. Their Kubernetes ranges came from endoflife.date, but both projects describe them as tested-with tables with no guarantee outside them, and the registry does not treat a tested-with table as a bound. KEDA 2.20.1 on Kubernetes 1.36, for example, was reported as blocked (#394).
0.2.2 - 2026-10-11¶
A patch release so the GitHub Action can be listed on GitHub Marketplace, which refused v0.2.1's description as too long. The chart is now on Artifact Hub as a Verified Publisher. Nothing in the binary, the image or the chart's templates changed.
Fixed¶
- The Action's
descriptionis 118 characters. GitHub Marketplace refuses one of 125 or more, and it readsaction.ymlat the release's tag, so v0.2.1's 247 characters kept it off the Marketplace.hack/action_test.shfails a description at or over the limit (#75).
Changed¶
- The chart's Artifact Hub metadata (
packaging/artifacthub/artifacthub-repo.yml) carries the repository's ID. Each release pushes it to the chart'sartifacthub.iotag, and with it Artifact Hub shows upgradescope as a Verified Publisher (#75). - The GitHub release notes open with a link to
CHANGELOG.mdat the release tag. The CHANGELOG is the reviewed record and lists every breaking change; the notes' Breaking changes group lists only commits markedtype!:(#390). - The
pr-lintcheck (hack/check-breaking.sh) also refuses a PR title or commit subject whose type is uppercase or mixed-case (Feat:,Chore!:,FIX(scope):), with or without aBREAKING CHANGE:footer, because the release notes group lowercase types only (#390). mainrequires thebreaking-changecheck as well asci-ok, so a PR with aBREAKING CHANGE:footer but notype!:subject cannot merge, and every release tag (vX.Y.Z[-pre]) is protected from deletion and moves while a floating major tag such asv1stays movable. CI checks that the rulesets applied on GitHub still equal.github/rulesets/(hack/rulesets-live.sh) (#391).
0.2.1 - 2026-10-11¶
A patch release after v0.2.0: the agent delivers its first snapshot within
about a minute on a combined install, and the GitHub Action refuses a
latest that moved back to a release from before provenance.
Added¶
make claims-server(hack/claims-server.sh), run by CI's repo-checks job, checks two server claims over HTTP on the built binary, with no cluster. A name first registered without aclusterIdis bound by the nextclusterIdpushed (202, 202, history 75 then 100, then 409, one adoption log line, no token in the log) (#368). A push still uploading when its cluster is renamed or deleted answers 409 and stores nothing with a per-cluster token, and registers the name again (202) with the shared token. The order is forced withExpect: 100-continueand a held body, never left to timing (#369).
Changed¶
- CI's claims-trackers step no longer runs when
release.ymlcallsci.yml: a release depends on the tagged code, never on the live state of an issue. Pull requests and pushes to main still run it. The v0.2.0 release run first failed on it, when #384 closed two trackers whose last acceptance item was not yet done (#368, #369).
Fixed¶
- Agent: a snapshot whose push failed at a tick is retried between ticks,
after 10 s, then 20 s, 40 s and 80 s, then every 2 minutes, each within
20%. It stops when the snapshot is delivered, refused for good, or replaced
by the next tick's. On a combined install (
server.enabled=true), whose server is not yet listening at the agent's first tick, the first snapshot now arrives within about a minute instead of one interval (10 minutes) later. Each retry logsbuffered snapshot pushed(INFO) orbuffered snapshot push failed(WARN), and a failed retry counts inupgradescope_agent_push_errors_total. The tick log'sconsecutiveFailurescounts the failure the line reports: it is 1 after the first failed push, where it used to read 0. The kind e2e requires the first snapshot before the agent's second tick (#386). - The Homebrew tap's test job installs the formula as the tap's default branch holds it. Before, it tested the run's own commit, which on the first run predates the formula the update job pushes (#75).
Security¶
- The GitHub Action fails when
latestresolves to v0.1.0 or v0.1.1, the releases from before provenance, from every ref: a branch or a commit SHA as well as a release tag. The error names the tag and the step never falls back togo install. Before, only an action at a release tag from v0.2.0-rc.2 on refused it.verify-provenance: falseand an explicitversion: v0.1.1are unchanged (#127).
0.2.0 - 2026-10-11¶
The first release built, signed and published by CI. It changes a lot of scanning behaviour and gating, so read Changed before you upgrade a CI gate.
Added¶
scanand the agent send aUser-Agentofupgradescope/<version> (<os>/<arch>)on every request to the Kubernetes API server, with the versionupgradescope versionreports (the agent's is its image's build), so an audit log tells releases apart. client-go's default, which every build sent before, readupgradescope/v0.0.0 (...) kubernetes/$Format; an audit-log filter or alert that matches that string must match theupgradescope/prefix instead.docs/operations/security-model-and-rbac.mdsays how to pick upgradescope's requests out of the audit log (#379).- CI:
hack/claims-trackers.sh, a step of the repo-checks job, asks GitHub about every issue a claims-ledger row names as its tracker (the#Nafternot true yet,not automated (yet),not auditedorunder repairin its Proven-by cell) and fails when one is not open, so a fix that closes a tracker cannot leave the ledger citing it. Without a token or withoutghit skips with a notice (CLAIMS_TRACKERS_REQUIRE=1makes the skip a failure); it is not part of the offlinemake claims-check. The repo-checks job now reads issues (issues: read), and the release workflow'scijob, which callsci.yml, grants it, without which GitHub would refuse to start the next release run (#366). - The knowledge base carries 21 hand-written migration notes, each cited to an
https page (
internal/kb/data/migrations.json, whichtools/gen-kband the weekly refresh never write), for the removed or deprecated kinds whose replacement is not a rename of theapiVersionor does not exist. A finding for such a kind ends its remediation with the note, after the generated hint and a;, or is the note alone when there is no hint, and the note's citations follow the deprecation guide's, in the table'sfix:line, the Markdown cell,--output jsonand SARIF alike.PodSecurityPolicy(policy/v1beta1andextensions/v1beta1) names Pod Security Admission and cites the migration page, which says there is no manifest to convert.Ingress(extensions/v1beta1andnetworking.k8s.io/v1beta1),CustomResourceDefinition,ValidatingWebhookConfiguration,MutatingWebhookConfiguration,CertificateSigningRequest,Event,PodDisruptionBudgetandEndpointSlicename the schema changes the deprecation guide lists (for an Ingress,spec.backendbecomingspec.defaultBackend,serviceNameandservicePortbecomingservice.nameandservice.port, and a requiredpathType). The 10 removed alpha kinds with no successor (AuditSink, ClusterCIDR, the dynamic resource allocation kinds and PodPreset) point at the changelog of the release that removed them and claim no replacement. A Helm release's stored-manifest finding, and a manifest at an API version the target does not serve yet, carry no note (#330). -ois the shorthand for--outputonscanandversion, as inkubectlandhelm(#339).- Kubernetes compatibility ranges, as
compatrows that each cite the upstream page they were read from on 2026-10-10, for cert-manager (release lines 1.10 to 1.21, a minimum and a maximum each), Karpenter (1.0 to 1.14, a maximum each, derived from the matrix upstream publishes, which lists the oldest Karpenter that runs each Kubernetes minor; the entry says it is derived), Calico (3.25 to 3.32, a minimum), Cilium (1.13 and 1.14, a minimum), Metrics Server (0.3.x, 1.8 to 1.21, and a minimum for 0.6 and later), Velero (1.14 to 1.18, a minimum) and Prometheus Operator (a minimum, one row for 0.84.0 and later and one for earlier releases). That is 13 of the registry's 27 add-ons with ranges, where 6 had them. The rows sit apart from thecyclesblock, sotools/eol-syncand the weekly refresh never rewrite them (two tests run a write-mode sync and check that the rows come back byte for byte), andregistry/CONTRIBUTING.mdsays how to add them to a synced entry. A range is encoded only as far as upstream states it, and an entry without one says why in a comment: Argo CD, Cilium 1.15 and later, and Calico's maximum are tested-with tables, which are not support ranges; kube-state-metrics publishes the client-go version it is built with and CoreDNS the version kubeadm installs, neither a range; Flux, Traefik and Gatekeeper publish no matrix. A release line the registry gains later (for example cert-manager 1.22) needs its row added by hand, and until then it is not judged. What the ranges do to a verdict is under Changed (#341, #343). - Image matchers for the managed-distribution and mirror paths of CoreDNS,
Metrics Server and kube-state-metrics, which
unrecognizedImagesused to list: AKS (mcr.microsoft.com/oss/kubernetes/<name>andmcr.microsoft.com/oss/v2/kubernetes/<name>), EKS Distro (eks-distro/coredns/corednsandeks-distro/kubernetes-sigs/metrics-server), GKE (gcr.io/gke-release/<name>andgke.gcr.io/<name>, for Metrics Server and kube-state-metrics), Rancher's verbatimrancher/mirrored-*copies (also of Cilium, Calico and Prometheus Operator, for RKE2 and k3s) and Bitnami's rebuilds (bitnami/andbitnamilegacy/, for Metrics Server, kube-state-metrics and Prometheus Operator, andbitnamilegacy/external-dns). Each path was read from the registry that serves it, except thebitnami/ones, whose repositories have been empty since 2025 and which are kept for pinned references and mirrors. The AKS and GKE builds of Cilium and Calico stay unrecognized, as do Rancher'shardened-*rebuilds: they are provider builds under the provider's own support policy, which the registry has no entry for, and upstream's release-line dates must not judge them (#342). - Chart value
rbac.helmSecretsNamespaces(default[]) and agent flag--helm-namespaces(repeatable or comma separated;$UPGRADESCOPE_HELM_NAMESPACES) limit Helm release detection to the namespaces they list. The chart then renders no cluster-wide rule on Secrets or ConfigMaps. Each listed namespace gets one Role (get and list onsecretsandconfigmaps) and one RoleBinding to the agent's ServiceAccount, and the agent is passed--helm-namespacesand lists theowner=helmobjects, metadata only, in those namespaces and nowhere else. Thehelmcapability is reported partial, saying that releases in other namespaces were not assessed. A Role still reads every Secret in its namespace, and kube-system holds sensitive ones, so the gain is partial; a listed namespace must exist before the install, and names must be DNS-1123 labels, each once (the flag and the values schema both refuse others). The default is unchanged: an empty list keeps the cluster-wide rule (#344). - The server Deployment has a
startupProbeon/healthz(server.startupProbe.periodSeconds, default5, andserver.startupProbe.failureThreshold, default60, both integers of at least 1). The server migrates its database before it opens its port, and the liveness probe, which gives up after three failed checks 15 seconds apart, could restart a server that was still migrating; liveness now starts once the startup probe passes, which gives a migration 5 minutes by default. RaisefailureThresholdfor a large database or slow storage (#345). server.persistencegainsexistingClaim(mount a PVC you manage; the chart renders none),accessModes(default[ReadWriteOnce]; the schema accepts the four Kubernetes access modes, each once),annotationsfor the chart's PVC, andretain, which annotates ithelm.sh/resource-policy: keepso thathelm uninstallleaves it and the history in it (#346).clusters list,clusters delete,clusters renameandmcp --server-urlaccept--server-ca-file, a PEM bundle of CA certificates trusted on top of the system roots, for a server behind a private CA, as the agent's flag of the same name is. It needs anhttpsserver URL, and a missing file or a file with no certificate stops the command. Verification is never skipped. On Linux and other Unix systems,$SSL_CERT_FILEand$SSL_CERT_DIRalso change the system roots, for the whole process; Go ignores them on macOS and Windows, where--server-ca-fileis the way (#347, #372).- In-tree volume plugins that Kubernetes removed, or moved behind a CSI
driver, are now
volume-pluginfindings, keyedvolume-plugin/<plugin>, from a cited, hand-written dataset of 16 plugins (internal/kb/data/volumeplugins.json): each minor was checked against the upstream changelog's GA section and the merged pull request, and each class taken from thek8s.io/apifield comments. Removed with no migration path:scaleIO(1.22),flocker,quobyteandstorageos(1.25),glusterfs(1.26),cephfsandrbd(1.31) andgitRepo(disabled by default from 1.33, counted as removed from there). Each is a blocker at or after that minor, a warning the minor before it, and info earlier. Moved to a CSI driver:cinder(1.26),awsElasticBlockStoreandazureDisk(1.27),gcePersistentDisk(1.28),vsphereVolume(1.29),azureFile(1.30) andportworxVolume(1.36). Each is a warning at or after that minor, naming the driver the volumes then need, and info earlier. None is ever a blocker, because a cluster with the driver installed is fine, and upgradescope does not check whether it is.flexVolume(deprecated in 1.23) is info.scan --filesandPOST /api/v1/gateread the pod templates of Deployments, DaemonSets, StatefulSets, ReplicaSets, Jobs and CronJobs, the volumes of Pods, and PersistentVolume manifests, each located by file and line. A livescanand the agent count the pods they already list, per plugin and namespace, with no extra API request for them (between full pod passes, the pods outside kube-system are counted as the last full pass read them, as the add-ons are), and list PersistentVolumes and StorageClasses, so a claim's in-tree volume is found live too (#362, see Fixed and Changed).photonPersistentDiskis not in the dataset, because its removal minor was not confirmed. The new optionalvolumescapability and the inventory fieldvolumePluginscarry the evidence, andvolume-pluginis a new category inapi/report.schema.jsonand the OpenAPI schema. An inventory from a collector that predates it (v0.1.x, the v0.2.0 release candidates, or a files inventory an older CLI saved) shows avolumesgap, "not reported by the collector", never a clean result; the gap is optional and changes no verdict, and agents and servers can be upgraded in either order. What this does to verdicts and to the knowledge base version is under Changed, and the checks on a pushedvolumePluginsunder Security; the concept page isdocs/concepts/volume-plugins.md(#351). - The dashboard's Fleet page can be searched, filtered and sorted, and has a
summary strip. A case-insensitive name search, a quick filter (Blocked,
Unknown, Stale, Has blockers, Ready, n/a, No stored evaluation) and a sort
(worst score first, the default, then name, then last seen with the oldest
first) narrow the matrix, which says "N of M clusters", keeps its header row
and cluster column in view, draws at most 100 rows and adds 100 more on
Show more. The strip counts the clusters for one target (the one most
clusters can still upgrade to, unless you pick another of the targets the
matrix shows) in six buckets: ready, blocked, unknown, stale, n/a and no
stored evaluation. Each cluster is in exactly one, so the buckets add up to
the clusters you can see, an
unknownverdict is never counted as ready, and clicking a bucket filters the matrix to it. The search, filter, sort and target live in the URL hash (#/?q=prod&filter=blocked&sort=name&target=1.37), so a reload or a pasted link restores them (#352, #353). - Every dashboard view has a Refresh button and an "updated HH:MM:SS" stamp, keeps its data on screen while a refetch runs (a failed refresh keeps it too and says so), and refetches when a hidden tab becomes visible and what it shows is more than 60 seconds old. The views still do not poll. The empty fleet has the button as well, so the first pushed snapshot shows up without a reload (#354).
- The Cluster view keeps its findings filters (
category,severity,teamandq) in the URL hash next totarget, and every finding has a#link whose address is that finding (#/cluster/3?target=1.37&finding=eol-addon%2Fingress-nginx): opening it scrolls to the finding and highlights it (#355). - A guide to the dashboard,
docs/guides/dashboard.md: its routes and URL parameters, entering a read token (kept insessionStorageunless Remember on this device is ticked), the banner a team-scoped token shows, and serving it under a path prefix behind a proxy (open the URL with the trailing slash, let the proxy strip the prefix, and allow the proxy's Host), with nginx and Ingress examples that CI does not run.docs/troubleshooting.mdhas an entry for a blank dashboard or 404 assets behind a proxy (#356). agent --pod-pass-every(default3) and--pod-pass-max-age(default1h; chartagent.podPassEveryandagent.podPassMaxAge) set how often the agent lists every pod outside kube-system; what that does to results is under Changed.addOnEvidenceAgeSeconds(optional; in the inventory, the report andapi/report.schema.json, andClusterReadiness.status) is the age of the pod pass the add-on evidence comes from. It is also the gaugeupgradescope_addon_evidence_age_seconds, in the tick's log line and on the dashboard's Cluster view, and it is left out of the snapshot hash, so a reusing tick pushes nothing new. The agent warns at start when--pod-pass-max-ageis at or below(--pod-pass-every - 1) x 1.1 x --interval, the least that reusing a pass on every tick between two full passes needs, and names the remedies: raise the max age or lower--pod-pass-every(#228).serve --require-read-credentialnever opens the read API: a read with no credential, or with a bearer nothing knows, gets401even when the database holds no read token, so a lost or restored database cannot reopen it.serverefuses it together with--allow-anonymous-read, and warns at startup, ending with the mint command (upgradescope tokens create --read --teams '*'), while no read credential exists yet. The chart passes it by default (see Changed) (#295).serveand the agent re-read the files behind their--*-fileflags (the token files,--slack-webhook-file,--webhook-fileand--webhook-secret-file) when they change, at most every 5 s, and swap the value atomically; the three server tokens reload as one group, so two of them can never end up equal. An empty, unreadable, oversized or colliding new file keeps the old value, and the log names the file, never its content; a read or admin token file that empties or disappears never opens the read API. A value from a flag or an environment variable is still read once, as are--db-url-fileand themcptoken files. Newserve --optional-secret-filetakes a comma-separated list ofingest-token,slack-webhook,webhookandwebhook-secret(each also needs its--<name>-file) whose file may be missing at start, as a key absent from a Secret you manage is: removing a required ingest-token file keeps the old token, and removing the optional one leaves per-cluster tokens only (#255).agent --pprof-addr HOST:PORTserves Go's profiler on its own listener. It is off by default and refused on any address but loopback; reach it in a pod withkubectl port-forward("Profiling the agent" indocs/observability.md) (#247).- Retention is observable:
upgradescope_retention_prune_failures_total{store},upgradescope_retention_last_success_timestamp_seconds(set only after a complete prune) andupgradescope_retention_rows_deleted_total{table}; none exists with--retention=0, and/readyzis unaffected. The chart's PrometheusRule addsUpgradescopeRetentionStale, which fires when there has been no successful prune in 2 days, or when a process that has completed no prune counts a failure, so a server that restarts more often than that and fails its startup prune every time is caught too (#263, #296). - The knowledge base gives 33 more removed APIs their GA replacement, cited from the successor's changelog and marked as defaulted in the generator; a removal past the knowledge base's horizon reads "(projected)" in live and Helm-manifest findings alike, and a target past it notes that its version knowledge is projected (#266).
- Registry data synced with endoflife.date: Calico 3.33 (3.31 ended 2026-10-01), Istio 1.31 on Kubernetes up to 1.37, and EKS 1.37 (standard support to 2027-12-01, extended to 2028-12-01) (#277).
GET /api/v1/fleet/teamscarries each team'sverdict: the worst of its verdicts in the clusters evaluated,blockedoverunknownoverready. A team with a clean score can still beblockedby a blocker no team owns, orunknownbecause a required check did not run, so read the verdict, not the score. The response also has a requiredexcludedarray, one{name, clusterId, reason}for each cluster left out of the rollup, withreasonno-snapshot,too-largeorunreadable(missingstill lists the same names). Score history points (ScorePoint,GET /api/v1/clusters/{id}/history) gain an optionalverdict, absent from servers that predate it. The Teams page and the score sparkline colour by verdict, so a blocked or unknown evaluation is never drawn green, and the score badge is red for any blocked verdict (#243).- A finding of an API usage check carries
callers(optional, inapi/report.schema.jsonand the server's responses): theapiserver_requested_deprecated_apisrows folded into it, in row order, each withgroup,version,resource,subresource,key,severity,titleanddetailof thedeprecated-api-in-usefinding it is on its own (#236). - Chart: with
server.replicasabove 1 the chart renders a PodDisruptionBudget for the server (server.podDisruptionBudget,enabled: true;minAvailable: 1, ormaxUnavailable, which replaces it; the render fails when both are empty) and spreads the server pods across nodes with a softkubernetes.io/hostnametopology spread (server.defaultTopologySpread: true; your ownserver.topologySpreadConstraintsreplace it). With one replica no budget is rendered, sinceminAvailable: 1would block every node drain (#242). serve --allowed-host <name>(repeatable or comma separated;$UPGRADESCOPE_ALLOWED_HOSTS): a host name or IP that requests may name in their Host header, at any port. A port, scheme or wildcard is refused at startup. Chart:server.allowedHosts(default[]) adds names to the ones the chart always passes (the server Service's DNS names andserver.ingress.host); a port or scheme fails the render. Chart:clusterDomain(defaultcluster.local; a DNS name, a trailing dot is dropped) is the domain of the server Service's fully qualified name, which the chart passes to--allowed-hostand names in the cert-manager Certificate; the Certificate always named.svc.cluster.local, so set it when your cluster uses another domain, or a client using the full name gets421and a certificate without that name. What the option is for is under Changed (#240).- Team-scoped read tokens:
upgradescope tokens create --read --teams <team>(repeatable, a team name taken as written;--teams '*'for the whole fleet),tokens list --readandtokens revoke --read --id <n>. A read token is printed once; the database keeps its sha256 hash and first 8 characters, and the server looks it up on every request, so one minted or revoked takes effect without a restart. A team-scoped token reads only the clusters one of its teams owns a namespace in (as the cluster's current evaluations attribute namespaces, after--team-map), and of those only its teams' findings and suppressed findings (one spanning teams cut to theirs) and team scores; the cluster's score, verdict and counts stay the whole cluster's. Any other cluster answers404, as an unknown one does, and is left out of/clusters,/fleetand/fleet/teams./metricsanswers403to a scoped credential. A scopedPOST /api/v1/gate?cluster=judges the pull request on the scope's share of the cluster. The dashboard says which teams it shows. A scoped answer names its teams inX-Upgradescope-Teamsand is sentCache-Control: private, no-store. Seedocs/operations/auth.md(#72). serve --trust-team-header <header>, which needs--trusted-proxy-cidr <cidr>, scopes a read from a TCP peer in those ranges (neverX-Forwarded-For) to the comma-separated, percent-encoded teams the header lists;*there is never the whole fleet. It is safe only when nothing but the proxy can reach the server from those addresses (on loopback, mindkubectl port-forwardand mesh sidecars) and the proxy strips client-supplied copies of the header:deploy/examples/oauth2-proxy/runs oauth2-proxy v7.15.5 as such a proxy, in a sidecar. The chart has no values for this mode yet (#72).upgradescope mcp, a read-only Model Context Protocol server for AI assistants. It speaks MCP on stdio;--http ADDRserves streamable HTTP athttp://ADDR/mcpinstead (a bare port binds 127.0.0.1, an address that is not loopback needs--allow-remote, and a request from another origin in a browser is refused). Without--http-token($UPGRADESCOPE_MCP_HTTP_TOKEN,--http-token-file) the HTTP endpoint has no authentication; with it, a request without the token gets 401. Tools:scanreads the cluster--kubeconfigand--contextname (the current context is fixed when the server starts), and its only input is 1 to 4targets;list_findingsandget_reportread the latest scan, areport_file(at most 8 MiB), aninventory_file(at most 20 MiB, refused where ingest would refuse it) or, with--server-url, a fleetcluster;registry_lookupsearches the embedded registry; with--server-url,fleet_summaryreturns the fleet matrix, using the read token from--read-token-file,$UPGRADESCOPE_READ_TOKENor--read-token. Results followapi/report.schema.json. No.upgradescope.yamlis read;upgradescope.dev/ignoreannotations still apply. A refusedreport_fileorinventory_fileis answered with the field and the rule broken, quoting nothing of the file except an unknowncollectorSchemanumber. The binary grows by about 2 MiB (2.00 MiB on linux/amd64, stripped, measured at67f30be) (#76).- A
support-lifecyclefinding for clusters on EKS, GKE and AKS, from provider calendars inregistry/data/providers/(EKS and AKS synced from endoflife.date, GKE curated from Google's release schedule): a warning when standard support for the cluster's minor ends within 90 days, and a blocker from the day it ends, through extended support and after, keyedsupport-lifecycle/<provider>/<minor>and judged whatever the target. Where a list price is cited (EKS, and GKE's Extended channel) it states what extended support adds per cluster per year,(extended − standard) × 8760cluster-hours ($4,380 at $0.60 against $0.10), with the price's as-of date; AKS gets dates only. On GKE and AKS, where extended support is opt-in, the wording is conditional. The inventory gainsprovider(eks,gke,aksorother), set from server-version suffixes and managed node-pool labels; aproviderIDscheme alone never claims a provider. Ingest accepts aproviderthis build does not know, but refuses (422) one that is not a label value: longer than 63 bytes, with characters outside letters, digits,-,_and., or not starting and ending with a letter or digit.scanprints aSupport:line, the JSON report and the server's report carrysupport, and theClusterReadinessstatus gainssupportPhase,extendedSupportFrom,extendedSupportEnds,annualCostDelta,currency,priceAsOf,annualCostNoteandextendedSupportCondition(#77). - Add-ons that Argo CD and Flux deploy are found from the charts those tools
declare: each Argo CD Application source that sets
chart, for Applications whose destination is this cluster, and each Flux HelmRelease'sspec.chart.specorchartRefto an OCIRepository (the newest ofv2,v2beta2andv2beta1served). A chart the registry knows is an install in the namespace it deploys into (add-onsource: gitopswhen nothing stronger found it), so an Ingress NGINX that Argo CD renders withhelm templateis aneol-addonblocker without a Helm release. The inventory lists the references asgitopsCharts. Reading them needs the new chart valuesrbac.gitops.argocd(get, list on Applications) andrbac.gitops.flux(get, list on HelmReleases, get on OCIRepositories), bothfalseby default. Ingest refuses (422) agitopsChartsentry whosenameis not an RFC 1123 subdomain, or whosenamespaceortargetis not a namespace name. See the GitOps guide for what is and is not read (#70). agent --server-ca-file: a PEM bundle of CA certificates the agent trusts for pushes, on top of the system roots, for a server behind a private CA; read at startup. It needs anhttps--server-url, and a missing file, a file with no certificate, or a certificate that does not parse stops the agent at startup. Chart:agent.serverCA.configMaporagent.serverCA.secret, withagent.serverCA.key(defaultca.crt), mounts the bundle read-only and passes it as--server-ca-file; set, it replacesserver.tls.caKeyfor the in-chart server. The render fails when both sources are set, the key is empty, nothing is pushed, or the push URL is nothttps(#65).serve --tls-cert-filere-reads the certificate and key when either file changes, checked at most once a second on a new handshake, so a renewed certificate (cert-manager rewriting a mounted Secret) needs no restart. A pair that does not load is logged and the previous certificate stays in service. TLS 1.2 is the minimum. The README's quickstart (the Fleet paragraph) and Exposing the server to remote agents in the fleet guide say how to serve over TLS, and that plainhttpsends the ingest token and the inventories in the clear (#65).POST /api/v1/gate?allow-incomplete=true, asscan --allow-incomplete: anunknownverdict with no finding at thefail-onthreshold answers- A blocker, a warning under
fail-on=warning, and a target that is not an upgrade of the cluster still answer 422, and the verdict header and body still sayunknown. Onlytrueorfalse, given once, is accepted; any other value is a 422. Without the parameter nothing changes (#177). - Registry entries for Kubernetes Dashboard (archived 2026-01-21), Promtail
(end of life 2026-03-02), Grafana Agent (2025-11-01) and Weave Net
(archived 2024-06-20), each retired as a whole and so an
eol-addonblocker at any version, and for Karpenter, Gatekeeper and Fluent Bit, synced from endoflife.date: 27 add-ons in all. Bitnami'sbitnamilegacy/nginx-ingress-controlleris matched as Ingress NGINX (#49). --registry-dir <file-or-directory>onscan,agentandserveloads more registry entries (<id>.yaml, in the schema ofregistry/CONTRIBUTING.md), validated like the embedded ones, citations included. An invalid file, a path with no*.yamlentry, or an entry that claims an image or chart another entry claims stops the command at start, naming the file or both entries. An entry with an embeddedidreplaces it. The entries are part of the knowledge base version label.servejudges stored inventories and/gatemanifests against its own registry, so give it the agents' entries;GET /api/v1/registrylists the embedded registry only. An image matcher of two or more segments matches as a path suffix on whole segments; a one-segment matcher is that repository exactly, and"*/name"matches the name under any registry prefix (only for distinctive names; only etcd uses it, so etcd is still found behind a mirror). Chart:agent.extraRegistrymaps<id>.yamlfile names to entries, rendered into a ConfigMap mounted as the volumeextra-registry(a name to avoid inagent.extraVolumes); the server has no such value, so useserver.extraArgsandserver.extraVolumes(#49).- Example policies over
ClusterReadinessinexamples/: a Kyverno ClusterPolicy and a Gatekeeper template and constraint that report a workload annotatedupgradescope.dev/upgrade-target: "<minor>"when the object (clusterby default) hasready: falsefor that minor. They only audit or warn by default, and allow the request whenlastEvaluatedis more than an hour old or missing, or when there is no object or no entry for the minor; each ships a read-only reader ClusterRole. A static Renovate preset (examples/renovate/upgradescope.json) groups minor and patch bumps of the registry's charts, holds major bumps for approval and prioritizes the end-of-life ones. CI checks them without a cluster, with pinned, checksum-verified Kyverno CLI and gator and Renovate's validator; live admission is not tested. The Acting on readiness guide describes them (#79). - The inventory lists kube-proxy once per node it runs on, with the node
(
controlPlane[].node, fromspec.nodeName), and aversion-skew/kube-proxy-kubelet/<node>warning names a node whose kube-proxy is more than 3 minors (2 for a kube-proxy older than 1.25) older or newer than its kubelet, citing the version skew policy. It does not depend on the target. A kube-proxy with no node, or on a node that is not listed, is not paired. Ingest refuses (422) acontrolPlane[].nodethat is not an RFC 1123 subdomain (#148). make bench-agent KUBECONFIG=<lab kubeconfig>measures the agent's tick on a disposable lab cluster it fills with 2,000 KWOK nodes (2,001 with the control plane), about 14,000 pods and 1,000 Helm release Secrets (requests by verb and resource, bytes, wall time, CPU, peak heap and RSS); it refuses a kubeconfig taken only from the environment and a cluster with more than 3 real nodes.make bench-ingestmeasuresserveingesting 200 clusters with three targets each, on SQLite and on a throwaway Postgres. The results, hardware and what is simulated are indocs/operations/scale.md: there, all 200 pushing at once, oneservetook 56 new snapshots a second on SQLite and 40 on Postgres 17, on a dual-core i3-7100U that also ran the database (#71).- Each team in a report and on the dashboard has a
verdict(blocked,unknownorready). A team isblockedby a blocker of its own or by one no team is attributed, which cannot be ruled out as its own; otherwiseunknownwhen the report has any required gap, which may hide a blocker of any team; otherwiseready. Another team's blocker leaves itready(#196). - Inventories carry
collectorSchema, the collector's version of the inventory contract. An inventory that carries it is judged as current; one without it is judged as a v0.1.x inventory unless itsagentVersionis a semantic version at or after 0.2.0-0, sodev,unknownand an empty version count as v0.1.x (#194). - The agent marks its
ClusterReadinesswith theupgradescope.dev/status-errorannotation while the status cannot be written, and clears it on the next successful write (#199). - Documentation site (MkDocs Material, published to GitHub Pages) with getting-started guides, concepts, operations pages, and CLI, Helm values, CRD, REST API, metrics and configuration references generated from the code and checked for freshness in CI.
- Published contracts:
api/openapi.yaml(OpenAPI 3.1 for every server route, tested against the real handlers),api/report.schema.json(the JSON report) andapi/webhook.schema.json(webhook payloads), plus a compatibility policy (docs/compatibility-policy.md) that says which changes a version may make. Each release attaches the three as assets (report.schema.json,webhook.schema.json,openapi.yaml), listed in its signedchecksums.txt(#60). - Deleted built-in APIs are in the knowledge base.
tools/gen-kbreads everyk8s.io/apirelease since v0.17 and records each type a later release dropped, removed at the release that stopped serving it. These include the DRA v1alpha1 to v1alpha3 kinds, ClusterCIDR, ServiceCIDR and IPAddress v1alpha1, LeaseCandidate v1alpha1, and older deletions such asbatch/v2alpha1CronJob andsettingsPodPreset. Manifests and callers that use them now block instead of passing silently (#124). - An
unknown-apiinfo finding: an object or API call at a version of a built-in group that the knowledge base does not know. CRD groups are not affected. - A readiness verdict:
ready,blockedorunknown. The verdict is shown in every output format, in theClusterReadinessstatus and as itsREADYcolumn.unknownmeans that no blocker was found but a required check could not run, so a blocker may have been missed. - Suppression with an auditable reason. Ignore rules in
.upgradescope.yaml(scan --config), theupgradescope.dev/ignoreandupgradescope.dev/ignore-reasonobject annotations, andClusterReadinessspec.ignoreaccept findings. Rules can match by key or category and by namespace, name or file globs, and can expire. - Baselines:
scan --baselinefails the gate only on findings that are new since an earlier JSON report, and--write-baselinewrites one. scan --output markdown, for CI step summaries.- The table output lists the affected objects, the fix and its citations.
The JSON report carries
schemaVersion,toolVersionand, for--filesscans,filesBase. - Helm charts are judged by their
kubeVersionconstraint and by the APIs in the stored release manifests. Releases stored by Helm's configmaps driver are read too. - Node container runtimes (containerd) are judged against the registry.
A runtime the registry does not cover (cri-o, docker) is an
addon-no-data/<runtime>info finding naming its nodes, instead of nothing (#169). - Add-on end of life is judged per release line (Istio 1.24, not "Istio"),
keyed on the installed app version. Registry schema v2 adds
tools/eol-sync-generated cycles, path-suffix image matchers and HelmappVersionmatching. - Add-ons are also detected from
helm.sh/chartandapp.kubernetes.io/*pod labels (the version fromapp.kubernetes.io/version) and from anIngressClasswhose controller isk8s.io/ingress-nginx, so Ingress NGINX installs that run mirrored or rebuilt images are found (#18). scan --files, and/gatewithout?cluster=, detect add-ons from the container and init-container images and labels of workload pod templates (Pod, Deployment, DaemonSet, StatefulSet, ReplicaSet, Job, CronJob), so a CI gate catches an end-of-life add-on in a pull request (#47). Images injected at admission time are not in the manifests./api/v1/gate?cluster=merges the add-ons and CRDs in the posted manifests into the cluster's stored inventory, as it merged their API usage. An add-on install replaces the cluster's install in its namespace; a posted CRD replaces the cluster's (keeping itsstatus.storedVersions), and posted custom resources are judged against the merged CRDs. An end-of-life add-on the manifests deploy, and a custom resource they write at a version the CRDs do not serve, are the manifests' findings (source: manifest), also when the cluster already has the same (#150)./api/v1/gateapplies suppression asscandoes, with the same code: theupgradescope.dev/ignoreannotations of the posted (and, with?cluster=, the stored) objects, and the ignore rules of a.upgradescope.yamlsent URL-encoded in the newconfigquery parameter (at most 32 KiB; an invalid config is a 422). Suppressed findings count toward neither the verdict norfail-on; the JSON answer lists them insuppressedwithsuppressedCountand names expired rules and annotations without a reason inwarnings; SARIF leaves them out and counts them in one note (#375, under Changed), JUnit skips them and Code Quality leaves them out. The gate has no baseline input: with?cluster=the cluster is its baseline (#44).- The server's report-shaped responses, a cluster's report and the gate's
JSON answer, lead with
schemaVersionandtoolVersionlikescan --output json(#60). - The report lists the image repositories no add-on matcher recognised
(
unrecognizedImages, at most 200, andunrecognizedImagesOmitted) in JSON, the table, Markdown and the dashboard's cluster view (not in SARIF). They are a gap in add-on detection, not findings, and change neither score nor verdict. - The knowledge base covers Kubernetes 1.37 (
k8s.io/apiv0.37.1), and the weekly refresh regenerates it without hand edits. - Agent:
--targetsreconcilesspec.targets;--manage-crdcontrols whether the agent touches the CRD;/healthz,/readyzand Prometheus/metrics; aReadycondition,observedGenerationand aLastEvaluatedcolumn onClusterReadiness. - Server: Prometheus
/metricsand/readyz; HTTPS with--tls-cert-fileand--tls-key-file;/api/v1/gatetakesfail-on, returns a verdict header and accepts a cluster baseline; every secret flag has an environment-variable and a-fileform;tokens list,tokens revoke --id, andclusters delete. - Chart: a values schema, production settings (resources, probes, security
contexts that work on OpenShift, pull secrets, scheduling), a
ServiceMonitor, a PrometheusRule and a Grafana dashboard. The ingest
token is generated, and every secret reaches the pods through
secretKeyRef. - GitHub Action: a root
action.ymlfor the Marketplace. Inputs are passed through the environment, the install is checksum-verified, and the Action writes outputs (verdict, score, counts, report paths) and a step summary. It takesconfig,baselineandwrite-baselineinputs, andallow-incomplete(scan --allow-incomplete), so a gate whose target is past the knowledge base's horizon can fail on findings alone instead of on theunknownverdict (#130). upgradescope version(and--version) prints the commit, the build date, the Go version, the knowledge base version and horizon, and the registry date.--output jsonprints the same as JSON.- Every command's
--helphas examples. A mistyped flag points to--help. - Packaging: deb, rpm and apk packages. Bash, zsh, fish and PowerShell
completions and man pages in every archive and package. A Homebrew tap
(
brew install abd-ulbasit/tap/upgradescope), a krew plugin manifest, and Artifact Hub metadata for the chart. LICENSE,NOTICEand a generatedTHIRD_PARTY_NOTICESfile (Go modules and the dashboard's npm packages) ship in every archive and package, and in the image under/licenses/. A CI license check fails on a dependency with an unknown or disallowed license.NOTICEandregistry/DATA-LICENSE.mdgive endoflife.date and the Kubernetes sources their attribution. The repository adds CONTRIBUTING, SECURITY, an architecture guide and an observability guide.--request-timeoutforscanandagent(default 30s; 0 = no per-request limit): client-go gives up on a single API request after it.- A live scan's report names the kube context and the API server it read:
JSON
kubeContextandapiServer(scheme, host and port only, never credentials), aContext:line in the table header, andContext ... · API server ...in the markdown summary. - CRD versions are judged (#48). A new
crdscapability (the inventory'scrdsfield) records each CustomResourceDefinition's versions (served, storage, deprecated and the deprecation warning) and itsstatus.storedVersions. A newcrd-versionfinding category has three keys: unserved: a blocker for custom resources at a version the CRD does not serve (or, in files mode, does not list). It can make the verdictblocked.deprecated: a warning, which quotes the CRD'sdeprecationWarning, when custom resources use a version the CRD deprecates. It is info when nothing uses that version.stored-unserved: a warning whenstatus.storedVersionslists a version the CRD no longer serves. Its remediation describes the storage version migration.
Live scans read CRDs through the apiextensions client. They list custom
resources metadata-only, at a served version that is not deprecated, and
judge them by managedFields authorship. --files scans take the served
versions from the CRD manifests in the files. A custom resource whose CRD
is not in the files makes crds partial. The agent's chart role can read
CRDs but no custom resources, so the agent reports crds as partial for a
CRD with a deprecated or unserved version. These findings concern the
add-on's CRDs, whatever the Kubernetes target.
- scan --output junit: one test suite per finding category and one test
case per finding. The outcomes follow the gate (--fail-on,
--allow-incomplete, --baseline): a finding that fails the gate is a
failure, one below the threshold passes, a suppressed or
baseline-unchanged one is skipped with its reason, and a required check
that could not run is an error, so an unknown verdict fails the test
report too (#73).
- scan --output gitlab-codequality: a GitLab Code Quality report for the
merge request widget. Severity maps blocker to critical, warning to
minor and info to info; each entry is anchored to the object's file and
line, and fingerprints hash the finding key and object identity (not
the line), so they are stable across runs. Findings without a file are
anchored to the virtual path upgradescope/<key>; suppressed findings
are left out (#73).
- /api/v1/gate?format=junit|gitlab-codequality, with the same status
code and verdict header as json and sarif (#73).
- CI templates for GitLab CI (ci/gitlab/upgradescope.gitlab-ci.yml, Code
Quality and JUnit reports), Jenkins (ci/jenkins/Jenkinsfile, the
junit step) and Azure Pipelines (ci/azure/azure-pipelines.yml,
PublishTestResults@2). Each installs a pinned, checksum-verified
release and keeps the report when the gate fails (#73).
- scan --plan: an upgrade plan for a multi-minor upgrade. The control
plane is upgraded one minor at a time, so --plan judges the cluster at
every minor from the one it runs (the oldest kube-apiserver) to
--target, and lists each finding in full at the first upgrade it
affects, with the upgrade where its severity changes (a warning at 1.33
that blocks at 1.34 shows both). The table and markdown outputs show a
per-upgrade section before the findings; the JSON report gains hops[],
an addition within schemaVersion 1 that the server does not serve.
With --files, --from gives the minor the cluster runs now. Ignore
rules and annotations apply to every upgrade. The verdict, score and
--fail-on gate stay those of --target. SARIF, JUnit and Code Quality
have no place for upgrade steps, so --plan with those outputs is an
error. Hops come
from the knowledge base's upgrade steps, one minor each by default; a
step that skips minors needs a citation, and none ship (#78).
- Kubernetes 1.24 to 1.28, which kind has no node images for, are tested
against a real kube-apiserver and etcd (envtest) in CI: GA-only objects
give no removed-API finding or blocker, a second scan of an unchanged
cluster is identical, and an object written through a beta API the
minor still serves blocks at its removal minor. Only the live collector
and engine are tested there (no nodes, agent or chart); the tested range
is on the compatibility page. make envtest runs it locally (#135, #69).
- Chart: HTTPS on the server's own port (server.tls), from an existing
certificate Secret or a cert-manager Certificate issued into
<fullname>-server-https. The in-chart agent then pushes over HTTPS and
trusts the Secret's CA (server.tls.caKey, empty for a publicly trusted
certificate); probes and the ServiceMonitor use HTTPS, and the chart's
Ingress is told the backend speaks HTTPS.
- GOMEMLIMIT is set when unset: the chart sets 90% of each container's
memory limit, and serve and agent take 90% of the cgroup's memory
limit (v2 or v1) outside the chart. An explicit value wins.
- Chart: server.sharedIngestToken=false drops the shared, any-cluster
ingest token, so only per-cluster tokens can push.
Changed¶
- Snapshot hashes ignore the order of a CRD's
spec.versionsand of a capability's skipped list: the server's duplicate check and the agent's snapshot hash sort both first, on copies, and the stored inventory keeps spec order. After the upgrade, a cluster whose inventory has a CRD's versions out of name order, or an unsorted skipped list, stores its first push or agent tick as one new snapshot with one history point; a later push that only reorders them is a200duplicate. A report no longer depends on either order: thecrd-versionfinding's detail lists the versions a CRD serves sorted,(it serves v1, v1alpha2)where it read(it serves v1alpha2, v1), andnotAssessed[].skippedis sorted. That sentence is the only change in report text; verdicts, scores, keys, severities and the SARIF and Code Quality fingerprints are unchanged (#367). - A push with a per-cluster token whose cluster is renamed or deleted at any
point after it was authenticated (while its body uploads, while it waits
for its ingest slot, or after its lookup) answers
409("renamed or deleted") and stores nothing. Before the lookup it answered401("revoked"), though the token was not revoked, so a script or alert that reads that401as a revoked token now sees a409. A revoked token still answers401. A push with the shared ingest token names its cluster only in its body, so for it the guarantee starts at the server's cluster lookup: a delete or rename before that lets the push register the name afresh, as the operations guide says (#369). - A live
scanor agent whose PersistentVolume or StorageClass list is refused or fails, on any page, has a requiredvolumesgap whenever the knowledge base has an in-tree plugin removed at or before the target that the unread list could show (a PersistentVolume source for the PersistentVolume list, an in-tree provisioner for the StorageClass list), which is any target from 1.22. Such a cluster readsunknown, notready:scanand the gate exit2unless--allow-incomplete, and theClusterReadinessReadycondition isUnknown. Before, anrbdPersistentVolume or akubernetes.io/rbdStorageClass behind the refused list readREADY, exit0. In practice this is an agent with its own RBAC (rbac.create=false) that lacks thegetandlistrules onpersistentvolumesandstorageclasses(#362, below), or a transient list error; the chart's RBAC grants both. A plugin that moved to CSI never makes the gap required, and a skipped pod list stays covered byaddons(#374). - SARIF (
scan --output sarif,/api/v1/gate?format=sarifand the Action'ssarif-file) no longer has a result for a finding that an ignore rule or annotation accepted. GitHub code scanning ignores a result'ssuppressionsand opened an alert for each located accepted finding, where the guide said they showed as dismissed. One run-level note counts them instead, and the JSON report, the step summary and JUnit still list them with their reasons. This replaces "SARIF marks them suppressed" from the gate's suppression (#44, under Added). A finding unchanged against a baseline is still a result and still opens an alert, and the guide says so. CI gains a dispatched check, off the default branch, that the suppressed fixture's SARIF leaves no open code-scanning alert (#375). - Notifications: an
unknownpass past the knowledge base's horizon is a notification baseline (see Fixed), which changes three things. A knowledge-base update that first decides a cluster's default target announces the blockers the update adds even when the cluster has no decided evaluation of a lower target (a cluster first seen on the newest minor); that first decided evaluation used to be silent. Such anunknownpass sendseol-approaching. Retention keeps each target's newest baseline, so a decided evaluation under a newerunknownbaseline ages out after--retention. Migration 0011 (SQLite and Postgres) addsevaluations.baselineand replaces the partial indexidx_evaluations_decidedwithidx_evaluations_baseline. On Postgres the first start builds the index inside the migration's transaction, which blocks writes toevaluations(the old replicas' pushes included) until it is built: on a large database, upgrade in a quiet window, or add the column and build the index first withCREATE INDEX CONCURRENTLY, asdocs/operations/upgrade.mdshows, and the migration finds both. Anunknownevaluation stored before 0011 is not a baseline (#380). - The auditor CSV export (
GET /api/v1/clusters/{id}/export?format=csv) has a new row type,suppressedin the severity column, one per finding an object annotation accepted (see Fixed), with the severity, source, reason and objects in its detail, and the summary row's detail ends with, N suppressedwhen N is above 0. A parser that expects only thesummary,not-assessed,blocker,warningandinforows must skip or read it. An export with nothing accepted is unchanged (#381). - The
READYcolumn ofkubectl get ucris the first target's verdict, the stringready,blockedorunknown(.status.targets[0].verdict). In v0.1.1 it was the boolean.status.targets[0].ready,trueorfalse, so a script that matches the column againsttruemust matchready, or read.status.targets[0].ready, which the status still carries. The Added entry for the readiness verdict introduced the column without saying it replaced the boolean (#366). scan --fileson a directory that holds an unrendered Helm chart (aChart.yamlwith templates that contain{{ }}) no longer readsREADY100/100 behind a warning for every template. It prints one hint per chart (warning: <dir> looks like an unrendered Helm chart (Chart.yaml found, N templates contain {{ }}); render it first: helm template NAME <dir> --output-dir rendered), and theapi-usagecheck is a required gap (N Helm template files under <dir> were not read; render the chart with helm template and scan that), because a template can carry any API version, and one unread template is enough. The verdict isunknown, so under the default--fail-on blockerthe scan exits2where it exited0;--allow-incompleteaccepts it. Check a CI gate that scans a chart directory before you upgrade, and render the chart first. Templates outside any chart directory are counted on one line and change nothing; a file that fails to decode and is not a template is still warned about one by one; a scan that finds no manifest and saw such templates names Helm in its error (exit1). The server's gate is unchanged (#335).- The compatibility ranges (see Added) can turn a scan or a CI gate red.
An install whose release line has a published maximum below the target is a
chart-incompatblocker, and so is one that needs a newer Kubernetes than the target (a minimum, which only a target below it can trip). At target 1.37 that makes every cert-manager release line the registry knows (1.10 to 1.21, for example "cert-manager 1.21.0 supports Kubernetes up to 1.36"), Karpenter up to 1.14 and Metrics Server 0.3.x blockers, because none of those lines lists 1.37 as supported. Run the gate against the new target before you upgrade (#341, #343). - A scan, an agent evaluation or a gate that finds a removed in-tree volume
plugin in use is now blocked where it passed: a Deployment with a
glusterfsvolume at target 1.26 or later,rbdorcephfsat 1.31 or later,gitRepoat 1.33 or later, and a warning the minor before. A scan of manifests can exit2for it. A plugin that moved to CSI only warns. Ignore rules, ignore annotations and baselines work onvolume-plugin/<plugin>as on any other key (#351). - Chart: the agent's ClusterRole gains
getandlistonpersistentvolumes(core) andstorageclasses(storage.k8s.io), with nowatchand no writes, for the in-tree volume plugin check (see Fixed). Each collection makes two more paged lists, one of each (more pages on a cluster with over 500 PersistentVolumes or StorageClasses). If you manage the agent's RBAC yourself (rbac.create=false), add both rules: an agent without them reportsvolumespartial, with a reason such aslist persistentvolumes: <error>; PersistentVolumes were not checked, and checks the pods only. A live volume-plugin finding that names PersistentVolumes or StorageClasses reads "2 pods and 1 PersistentVolume or StorageClass name it" and counts those pods inobjectsOmitted, which was 0 on every live finding before (#362). - Table and Markdown reports end with a scope line, "Field-level removals
other than in-tree volume plugins are not checked by this version." (the
Markdown report prefixes it with Scope.), so a
READYwith a score of 100 is not read as covering a removed field or annotation inside an API that is still served: the seccomp alpha annotations,kubernetes.io/ingress.class,Service.spec.externalIPsandbeta.kubernetes.io/osare not checked. Volume plugins are the one field-level removal this version does check (see Added). The line changes no score, verdict, exit code or JSON (#331, #351). - The knowledge base version changes. The
lifecycledigest inkbVersion(shown byupgradescope version) now covers the migration notes and the in-tree volume plugin dataset as well as the lifecycle entries, which also changed (see Fixed), and theregistrydigest changes with the new compatibility ranges and image matchers, so a build with this change reports akbVersionunlike every earlier build's, and an edit to a note or a plugin changes it from now on. An agent that runs a different build from its server therefore has itsapi-usage,helmandaddonschecks marked partial and readsunknown, neverready, until it runs the server's version (see the entry on knowledge base skew, #268): upgrade the agents and the server together. A script that compareskbVersionwith a pinned string needs the new value (#330, #351). ClusterReadinessgains a condition and a printer column.AllTargetsReadycovers every target of the plan, whereReadyis unchanged and is still the first target's verdict:False(reasonBlocked) when any target is blocked, with a message that names it, such as1.38 blocked (3 blockers);Unknown(reasonNotAssessed) when none is blocked but one was not assessed or no target was evaluated;Trueonly when every target is ready. Withspec.targets: ["1.37","1.38"], a ready 1.37 and a blocked 1.38 leaveReadyTrue, so wait onAllTargetsReadyfor a plan of several targets. TheBlockerscolumn (BLOCKERSinkubectl get ucr) is the first target's blocker count,.status.targets[0].blockers. The agent brings the CRD up to date at start unlessagent.manageCRD=false; otherwise applydeploy/chart/crds/clusterreadinesses.upgradescope.basit.engineer.yaml, since Helm does not upgrade CRDs. The Argo CD health check in the GitOps guide now reportsDegradedwhenReadyisTrueandAllTargetsReadyis not (an agent that predates this change, v0.1.x or a v0.2.0 release candidate, writes noAllTargetsReady, and the check then readsReadyalone), and the Flux example waits onAllTargetsReady, so with such an agent it stays in progress until you change it back toReady(#349).clusters list,clusters deleteandclusters renamename the server with--server-url, asagentandmcpdo.--serveris a hidden, deprecated alias: it takes the same value, prints cobra's deprecation notice, and, like--server-url, cannot be combined with--db,--db-urlor--db-url-fileor with the other of the two. Existing scripts keep working; change them to--server-url. The error for a redirect and the409that a push gets when its cluster name is registered to another cluster UID, whose remedy isupgradescope clusters delete <name> --server-url <this server>, name the new flag (#348).serve --targetscan be given more than once as well as comma separated:--targets 1.37 --targets 1.38is--targets 1.37,1.38. Before, a later--targetssilently replaced the earlier one. The limit of 4 distinct minors counts them all, so a chart release that setsserver.targetsand also puts a--targetsinserver.extraArgsnow evaluates both lists, andserverefuses to start when together they pass 4 (#348).Evaluatebuilds the namespace-to-team map once per check, not once per finding, Helm release and CRD, so its cost grows linearly with the number of namespaces: with as many namespaces as Helm releases and CRDs, it took 1.20 seconds at 5,000 namespaces and 18.6 seconds at 20,000, and takes 26 ms and 104 ms (go test ./internal/engine -bench EvaluateLargeInventoryon an Apple M1 Pro under a load average of about 7; one run each, so an upper bound). Results and goldens are byte-identical (#333).- Findings for a manifest object with no
metadata.namespace(files mode and the gate) say "no namespace set" where they said "namespace unset", and "cluster-scoped or no namespace set" where a list mixes live and manifest objects. Keys, suppressions and baselines are unaffected, and a baseline an earlier version wrote still matches by key (#334). scan --filesandPOST /api/v1/gatenow fail with a blocker for a manifest at an API version the target does not serve yet ("not served until 1.X"), so a manifest that passed a CI gate at an older target can now exit2: run the gate against the new target before you upgrade. Its key ends in/unserved; see the entry on that key (#300). A namespace-scoped ignore rule no longer accepts an add-on install that has no namespace in files mode, nor a cluster-scoped IngressClass; use a rule withoutnamespace. Such a finding carries the new optional report fieldunnamespaced(api/report.schema.jsonand the OpenAPI schema) (#237, #266).- Breaking: the
ClusterReadinessCRD group is nowupgradescope.basit.engineer, notupgradescope.dev(a domain the project does not own), and every annotation key moved with it:upgradescope.basit.engineer/ignore,/ignore-reason,/status-errorand, in the example Kyverno and Gatekeeper policies,/upgrade-target. The kind,v1alpha1, the short nameucrand the object name are unchanged. Entries in this file written before the move still show the old names. Nothing migrates in place, because the new CRD is a different resource. Beforehelm upgrade, apply it:kubectl apply -f deploy/chart/crds/clusterreadinesses.upgradescope.basit.engineer.yamlof the release you upgrade to (Helm never installs CRDs on upgrade, and the chart's agent may not create them). Then carry overspec.targetsandspec.ignore, rewrite your annotations, point Argo CD and Flux health checks (the Argo CD key is nowresource.customizations.health.upgradescope.basit.engineer_ClusterReadiness), policies and RBAC at the new group, and delete the old CRD withkubectl delete crd clusterreadinesses.upgradescope.dev; "The API group moved" indocs/operations/upgrade.mdhas each step. Without the new CRD the chart's agent exits at start with an error that names the exactkubectl applycommand (withagent.manageCRD=falseit stays up and every tick fails with it, in/readyzand the log), and the install notes print the command after the upgrade. The scanner reads the oldupgradescope.dev/ignoreandignore-reasonannotations until v0.3.0, prefers the new key when an object has both, writes only the new ones, and warns once, naming the first five objects, that the old keys are deprecated. The agent never deletes the old CRD: it logs one warning with the delete command (#68, #221). - Breaking for chart users: with
server.enabledand noserver.readToken, the chart now passesserve --require-read-credential, so the read API, the dashboard's data,/api/v1/gate,/metricsand the MCP fleet reads answer401until you mint a read token (the install notes print the command). Before, they were open until the first token was minted, and a lost PVC, an emptyDir (persistence.enabled=false), a restored backup or a fresh--db-urlreopened them; a team token the new store did not know even got fleet scope. To keep an open read API, setserver.allowAnonymousRead=trueexplicitly. A release that setserver.ingress.allowAnonymousRead=trueonly to use minted tokens (the earlier advice) stays open after a database loss until you drop it.metrics.serviceMonitorwith noserver.readTokenfails the render unlessserver.allowAnonymousRead=true(orserver.ingress.allowAnonymousRead=true), and so does aserver.extraArgsentry that contradicts the mode the chart renders (--allow-anonymous-reador--require-read-credential, either way). On any read API that is open, a bearer that matches no known credential now gets401instead of fleet scope (leave a CIREAD_TOKENplaceholder empty), and an anonymous request (no bearer) whose Host is notlocalhost, a loopback address, the address it arrived on, a chart-named Service name or an--allowed-hostgets421, on every listener, not only a loopback one: a NodePort or LoadBalancer IP needsserver.allowedHosts(--allowed-host). Minting or revoking a read token takes effect within about a second, with no restart (#295, #309). - A suppression or baseline for an API the target does not serve yet (the
"not served until 1.X" blocker) needs the key
removed-api/<group>/<version>/<kind>/unserved. That blocker keeps the categoryremoved-apibut no longer shares the bare keyremoved-api/<api>with the removal blocker and the next-minor removal warning, so a rule or baseline taken from a--target 1.30report can no longer hide the--target 1.37removal blocker. A rule on the bare key matches nothing for that finding, and the scan warns once and names the new key.ReadBaselinedrops the entries an older binary wrote for a not-yet-served finding, so that blocker resurfaces once as new against an old baseline. SARIF rule ids, JUnit test names and GitLab Code Quality check names and fingerprints change for those findings (#300). - An add-on image pinned by digest, tagged
:latestor tagged with a version that cannot be parsed now takes theapp.kubernetes.io/versionof the pods that run it, under the existing label trust rule (the name label, or the chart label when there is no name label; provider builds are never claimed through labels), and a parsed image tag still wins over a label. The oldest version across the pods holds, in either pod order. A scan that passed because such an image read as "version unknown" can now report aneol-addonblocker and exit 2: a digest-pinned cert-manager labelled v1.12.3 is a blocker once that line is past its end of life, so check a CI gate before you upgrade. A pod's labels are writable by anyone with pod create in the namespace, so a digest-only image's version can be forged there; the damage stays in that namespace, and the oldest version wins. The reused pod pass carries the label-derived version too. Theaddon-no-datadetail names only what the detector consulted: for an inventory from an agent that predates this rule (nocollectorSchema: v0.1.x and v0.2.0-rc.1/rc.2) it says the agent did not read the pod's version label, and claims no Helm app version where that agent reported the chart version in its place (#301). - By default the agent lists every pod outside kube-system only on every
third tick (
--pod-pass-every=3) or when the last full pass is an hour old (--pod-pass-max-age), and applies the registry to that pass's images and labels in between. It reads kube-system pods, Helm releases, GitOps resources and IngressClasses on every tick, and lists every pod on the next tick after a failed or partial pass, a version step that did not list kube-system, a pass at the max age, or a change in the Helm releases and GitOps chart references that name a registry add-on. An add-on installed some other way, an image changed in place (kubectl set image) or a GitOps upgrade through a version constraint that leaves the resource untouched can therefore be reported as it was for up to two ticks (20 minutes at the default interval);--pod-pass-every=1lists every pod on every tick, as before. Measured against 2,001 KWOK nodes, about 14,000 pods and 1,000 Helm releases: a tick that reuses the pass makes 20 requests (27.8 MiB read, 2.4 s, 1.8 CPU-seconds) and one that lists every pod 31 (50.4 MiB, 4.4 s, 3.4 CPU-seconds), so a cycle of 3 averages 23.7 requests (computed from the medians): the target of under 25 requests is met on average, not on a tick that lists every pod (#228). - Decoding a Helm release's manifest parses each document once, not twice:
the decode CPU of the lab's 1,000 release payloads fell 28.8% (9.175 s to
6.530 s), and that of the largest 50 by 30.0%, with the same findings and
the same heap (16.1 to 55.3 MiB against the 64 MiB bound). A document with
an anchor, alias, tag or merge key, a non-string or duplicate key, or a
U+0085, U+2028 or U+2029 line break is parsed the old way, so the result
stays byte-identical to kubectl's decoder.
docs/operations/scale.mdhas the measurement and the code that was measured (#285). - Flux OCIRepositories are listed, not fetched one by one: one paged list of
the namespace the
chartRefs point into, or one cluster-wide list, per-namespace when that is forbidden, and a GET only where a list is. A steady tick at 1,000 Argo CD Applications and 1,000 HelmReleases went from 540 to 50 GitOps requests (81 in all) with all 2,000 charts resolved. The chart'srbac.gitops.fluxgrantslisttoo; a custom role withgetonly still works, on GETs. Releases decoded before a partial Helm step's deadline stay in the cache, so the next tick fetches and decodes only the unread ones. They are the largest, since they sort last, which is why that tick costs more CPU: a profile on the lab at 200m puts 97% of its extra CPU in decoding the 51 unread releases (#248, #247). - Chart: the token and webhook Secrets are mounted as files (no
subPath, read-only, only the keys in use) and passed with the--*-fileflags, not as environment variables, so rotating a value, or the contents of anexistingSecret, takes effect within the kubelet's Secret sync (about a minute) with no restart. This replaces thekubectl rollout restartthe #242 entry asks for; a value fromextraArgsorextraEnvis read once and still needs it. The upgrade rolls the pods once. A--*-tokenflag inserver.extraArgsoragent.extraArgsbeside a chart-supplied secret file now fails the render, and a same-namedUPGRADESCOPE_*_TOKENinextraEnvno longer overrides the chart's file. Withserver.replicasabove 1, replicas may disagree on a rotated token for up to the sync window (#255). - A cluster whose agent collected with a different lifecycle dataset or
add-on registry than the server's (the digests in
kbVersion) has its api-usage, helm and addons checks marked partial, so it readsunknown, neverready, in reports, what-ifs and the CI gate, until the agent runs the server's version; a v0.1.x agent, which sends nokbVersion, is covered too. Expect clusters to readunknown(not ready) in reports and the CI gate until the agents run the server's version; that move sends no notification. Give the server theagent.extraRegistryentries too (--registry-dir). Ingest refuses (422) anagentVersionover 128 bytes, akbVersionover 512 bytes, or either when it is not printable ASCII (#268). - A target below the knowledge base's oldest covered minor (1.16) is refused
by
scan --target, the agent's--targetsandspec.targets,serve --targetsand the gate'starget, which now answers400instead of422; the error says thattarget: 1.30in YAML is the number 1.3. The Action'stargetmust match^[0-9]+\.[0-9]+$(v1.36and1.36.2were accepted before) (#237, #266). - The support-lifecycle finding's key is now
support-lifecycle/<provider>/<minor>/<phase>, withending,extendedorended, so a suppression or baseline of one phase no longer hides a worse later one. A rule or baseline with the old key, without the phase, matches nothing, andscanwarns and names the key to write: write it with the phase you accept (a category rule still takes every phase). Until then the finding is back, a blocker in theextendedandendedphases; against an old baseline it is new, so it fails the gate when it is a blocker, and the server's next evaluation of a cluster inextendedorendedsends one new-blocker notification. "The support-lifecycle key names its phase" indocs/operations/upgrade.mdhas the steps (#266). - RKE2 clusters get an
eol-approachingwarning, and Flux installed from manifests, RKE1 ingress (-rancherNbuilds ofrancher/nginx-ingress-controller) and Kubernetes Dashboard v1 become blockers, which can turn a CI gate red (see Fixed). A--registry-diroragent.extraRegistrycopy ofrke2-ingress-nginx.yamltaken from v0.2.0-rc.x, or an extra entry that relied on a tag pattern to share a repository with an embedded entry, now stopsscan,agentandserveat start with a claim conflict that says to copy the current file again (#265). - MCP: every result and every error of every tool (
scan,get_report,list_findings,registry_lookup,fleet_summary) has two text blocks, a notice that text from the cluster, a report file, a manifest, the registry or a fleet server is data and not instructions, then the JSON: a client that readcontent[0]as JSON must readstructuredContentor the second block. Every outside string and object key over 2 KiB is cut ("…(cut by upgradescope mcp)"), soget_reportcan differ from the file it read, and_meta["upgradescope.basit.engineer/clusterSupplied"]carries the marker. With no current kube context at start,scanis off for the life of the server, including in a pod with no kubeconfig, wheremcpused to fall back to the service account; a later context switch never moves scans (#264). - Server database: migration 0010 (SQLite and Postgres) adds an index on
evaluations(created_at, id). SQLite builds it before the server answers/healthz, reading every evaluation row once, report overflow included (187 to 197 ms for 20,000 evaluations of 30 KB, about 627 MiB, on an Apple M1 Pro; not measured on a larger database). On Postgres it is a plainCREATE INDEX IF NOT EXISTS idx_evaluations_created_at, which blocks writes toevaluationswhile it builds: upgrade in a quiet window, or create it first withCREATE INDEX CONCURRENTLY(docs/operations/upgrade.md) (#297). - A multi-line error from a library (yaml's
unmarshal errors:list, for one) now prints on one line with a literal\n; only anerrors.Joinseparator stays a newline (#245). - Chart: a release that already runs
server.replicasabove 1 (which needs Postgres) gets the PodDisruptionBudget and the soft spread of the server pods across nodes onhelm upgrade, as both are on by default, so a node drain no longer takes the last server pod. To keep the release as it was, setserver.podDisruptionBudget.enabled=falseandserver.defaultTopologySpread=false(#242). serveanswers421to a request whose Host it does not answer for, before any route, credential, team header or scope is looked at, when it listens on a loopback address or--trust-team-headeris set (a routable listener without a trusted header answers any Host, except while its read API is open: see the entry on the chart closing the read API). It answers forlocalhost, a loopback address, the address the request arrived on (probes and scrapes of the pod IP), the--listenhost (not0.0.0.0or::) and every--allowed-host; the port is never compared. This closes DNS rebinding: a page on a name an attacker controls, resolved to the loopback address, could read an open loopback read API, or a server in header mode throughkubectl port-forward. If clients reach such a server under another name (an/etc/hostsalias, a LoadBalancer IP, a Gateway host), add it with--allowed-hostorserver.allowedHosts. In header mode, a proxy that forwards the client's Host (oauth2-proxy's default, and the earlier example manifest's) now gets421for every browser request: set--pass-host-header=false, as the example now does, or add that host with--allowed-host. The chart always passes--allowed-host(the Service's DNS names andserver.ingress.host), so an older server image with this chart stops on an unknown flag: upgrade the image with the chart. Refusals are counted inupgradescope_http_requests_total{route="host-refused",code="421"}and logged at most once a minute (#240).serverefuses to start unless--slack-webhookand--webhookare absolutehttp(s)URLs with a host, naming the flag and never the value; before, a bad URL started and then dropped every notification.clusters list,clusters deleteandclusters renamewith--serverno longer follow a redirect: any 3xx is an error naming its status andLocation, so an http to https301or302cannot turn a delete or rename into a GET reported as done, andclusters listbehind such a redirect, which used to work, now fails. Before you upgrade, fix or remove a webhook URL that is not an absolutehttp(s)URL with a host, and pass the URL the redirect names (for examplehttps://) as--server(#240).serverefuses to start when--read-tokenequals--ingest-token(every agent's push token would read the whole fleet). Every command's secret read from its$UPGRADESCOPE_*variable (serve's tokens, webhook URLs, webhook secret and--db-url, the agent's server token,clusters,mcp,tokens --db-url) is trimmed of surrounding whitespace, as one read from--<name>-filealready was, and a variable that holds only whitespace is now an error. The trusted team header trims only space and tab from each team name it lists, no other whitespace (#250).unattributed, the key of the findings no team owns, is now(unattributed)in theteamsofscan -o json, the report,GET /api/v1/clusters/{id}/teams, the gate response,GET /api/v1/fleet/teamsand the HTML export (schemaVersionstays 1), and in thescantable and the dashboard's team filter (team=(unattributed)in the hash), because a team calledunattributedshared its row. That team is now its own row, andserve --team-maprefuses a team named(unattributed). Update anything that reads the old key (#243).- The dashboard's extensionless paths of more than one segment, or ending in
/(/cluster/3,/teams/,/index.html/), answer302to the root with the route in the hash, as a relativeLocationthat keeps a proxy's path prefix, where they used to serve a page whose assets did not load; a single segment such as/teamsstill serves the dashboard, as before. A path with a file extension that is not a file stays a JSON404;/assets/(which listed the asset files) and any other path under it that is not a file are now a JSON404too (#243). - Chart: the agent's CPU limit is now
1CPU (agent.resources.limits.cpu), not200m; the request stays50m, so scheduling is unchanged. A ResourceQuota onlimits.cpuwithout room for 800m more per agent pod, or a LimitRange whose CPU maximum is below 1, will refuse the pod: give the quota the room or setagent.resources.limits.cputo what fits. Measured as a pod against 2,001 KWOK nodes and 1,001 Helm releases (docs/operations/scale.md): at 200m the first tick took 73.4 s and its Helm step gave up at its deadline with 132 releases unread, and a steady tick took 21 s, throttled in 58 to 80% of its periods; at 500m the first tick took 47.9 s and at 1 CPU 35.2 s, both complete, and a steady tick 6.9 s at 1 CPU. Peak RSS was 66 to 71 MiB at every limit. The new Sizing the agent section of the install guide has the rest (#229). - The agent keeps a reserve of each tick for what follows collection: the
ClusterReadiness status write, the
upgradescope.dev/status-errormarker and the push, each on its own slice of it, so a slow collector step no longer leaves the object stale and unmarked. The reserve is 30 s, or half the tick deadline (--interval/ 2, at most 5 minutes) when that is under a minute; collection gets the rest. At the default--interval 10mcollection gets 4 min 30 s (was 5 min); at the 1m minimum it gets 15 s (was 30 s). The computed ceiling of the Helm step at 1 CPU and the default interval is now about 1,900 releases, about 1,600 withrbac.gitops.*(arithmetic, not measured): give a larger cluster more CPU or a longeragent.interval(#238). - The agent refuses at start what could never work, so it fails fast instead
of failing every tick:
--cluster-name ""given explicitly (leave the flag out to name the cluster by its UID);--force-sync-everyof 0 or below; a--server-urlthat is not anhttporhttpsURL with a host; and, with--server-url, a push token with whitespace or a control character inside it (surrounding whitespace is trimmed).--force-sync-everyat or below--intervalforce-syncs every tick. An agent given one of these crash-loops withinvalid --<flag>in its log: checkagent.extraArgsand the token Secret before you upgrade. The chart schema now rejects anagent.serverUrlwithout a host, and is stricter than the agent about non-ASCII host names and whitespace in the path, so ahelm upgradewith an unusual value can fail the schema (#238). - A deprecated-API caller is graded by the knowledge base's removal release
when the knowledge base has one for that group, version and resource, else
by the metric's
removed_releaselabel. A caller the label alone would call a warning is now a blocker when the knowledge base has the API removed at the target (and the reverse, when the knowledge base removes it later than the label says), so a gate can turn red or green on the same cluster. When they differ, the finding names the knowledge base as the source. A remediation names only a replacement the target serves; otherwise it says "no replacement Kubernetes X serves is known", naming the release a later replacement is served from, for live and Helm manifest findings alike (#236, #237). - Folding caller rows into findings is indexed: an evaluation of 16,000 usage entries and 16,000 caller rows takes about 30 ms (27.8 to 31.7 ms measured), where it took 12.9 to 18.4 s (#236).
- Server database: migration 0009 (SQLite and Postgres) adds indexes for
"newest evaluation by id", a
carries_holdcolumn, and backfillsserver_versionfrom stored inventories, on the first start. Retention now also keeps each cluster's newest decided evaluation for every target the server evaluates for it (and the three minors below its default target), and its snapshot, however old, so a cluster can keep one older snapshot and evaluation per target in use. With a notification sink configured, a push whose notification baseline another writer replaced on each of its three attempts answers503withRetry-After: 10and stores nothing; the agent retries. AGET /api/v1/fleet/teams?target=rollup still computing after 20 s answers503withRetry-After, keeping what it computed for the retry. A what-if is kept per cluster snapshot (or stored evaluation), target, knowledge base, team map and UTC day, so a later request computes only what changed and a what-if'sevaluatedAtis when it was computed, not the request time (#241). - Chart: upgrade with
helm upgrade --reset-then-reuse-values(Helm 3.14 or later), as the docs now say everywhere, not--reuse-values, which pins the new chart to the old release's image digest and can fail the render when the chart adds a value. The chart never restarted pods when a secret changed, and still does not, because no pod annotation carries a function of a secret value (readable by everyone who can get Deployments): after you change a token, a webhook URL or the contents of a Secret you named, runkubectl rollout restarton the server and agent Deployments (the install notes now print the command; the chart's token and webhook Secrets no longer need it, see #255). Every key of the chart's Secrets is now written underdata, so from this version on, removing a value removes its key; the one exception is a value you remove in the same upgrade from an earlier chart, which keeps an inert key in the Secret (docs/operations/upgrade.mdhas the one-timekubectl patch) (#242). - Chart:
server.staleAfterdefaults to empty, which followsagent.interval: the larger of 2h and three intervals (2h for any interval up to 40m); the release candidates defaulted to2h. A value you set at or belowagent.intervalfails the render, and the install notes warn when one would flap.metrics.prometheusRule.clusterStaleAfterSecondsdefaults to0(follow the server's threshold; the release candidates defaulted to7200); one at or below the interval fails the render (#242). - Chart: the server's container port is its own value,
server.containerPort(default8080), soserver.service.portmay be80or443. Before, the pod listened onserver.service.port: if you set that to something other than 8080, the pod now listens on 8080 (the Service still answers on your port), so a NetworkPolicy, sidecar or pod port-forward of your own that names the old port must follow it, or setserver.containerPortto the old value (1024 or above). Service names are cut to 63 characters for a long release name, and everything that names a Service follows the cut (other resource names are unchanged).networkPolicy.enabledwithserver.ingress.enabledormetrics.serviceMonitor.enabledfails the render whilenetworkPolicy.serverIngressFromis empty: list the ingress controller's or Prometheus's pods there, which the policy would otherwise cut off.server.tmp.sizeLimitbounds the new SQLite temp volume (#242). - A steady agent tick on a cluster of 2,001 nodes, about 14,000 pods and
1,000 Helm releases makes 31 API requests, down from 53, reading the same
50 MiB with 3.0 CPU-seconds (
docs/operations/scale.md). The pod and node lists size each page after the first by the largest object of the one before, as many as fit 8 MiB encoded, at least 500 and at most 1,000. API discovery is kept between ticks (asked again when the server version or the CRDs change, or after an hour, which costs 4 more requests), and a tick reads its ClusterReadiness once and writes the status over what it read. The larger pages raised the benchmark's peak heap from 29.3 to 39.2 MiB and its peak RSS from 57.1 to 65.0 MiB, and a page of 1,000 pods of about 70 KiB each, after small ones, would take the agent past itsGOMEMLIMITat the chart's 256Mi limit (computed): raiseagent.resources.limits.memoryon a cluster with pods that large. The target of under 25 requests is not met (#228; met on average by the pod pass reuse described above). - The first tick, and every one-shot
scan, fetches Helm releases on 8 workers and still decodes one at a time, in order, so results and the release cache are unchanged; memory stays bounded at one release's decode plus up to 7 fetched payloads (about 21 MiB more, computed). At 60 ms of added round-trip time the whole first tick took 30.4 s and read all 1,000 releases, where the code before it stopped at the Helm step's deadline after 67.6 s with 231 unread. Under a CPU quota, where decoding is the bound, it is not expected to help (#226). - Minting the first read token closes an open read API: a server run without
--read-token(on loopback, or with--allow-anonymous-read) answers401from then on to a request without a valid credential, as it does once--trust-team-headeris set. Revoking the last read token does not open it again.--read-tokenstill reads the whole fleet and answers exactly as before. A--team-mapteam named*is read as the team(*), with a warning at startup, since*is the whole-fleet scope (#72). - Server database: migration 0008 (SQLite and Postgres) adds the
read_tokenstable andevaluations.teams. The first start re-evaluates every stored evaluation once; until its pass reaches a cluster, that cluster's evaluations readoutdated: trueand no team-scoped read sees it.servenow opens, and so migrates, the database before it refuses an open read API on an address that is not loopback. A server rolled back past it ignores read tokens, so a read API closed only by minted tokens is open again (#72). - A managed cluster (EKS, GKE, AKS) whose minor is past the provider's
standard support now has a
support-lifecycleblocker, so its verdict isblockedand a--fail-on blockergate fails; within 90 days of the end it has a warning, which lowers the score and fails--fail-on warning. Upgrade, or accept it with an ignore rule (key: support-lifecycle/eks/1.34, orcategory: support-lifecycle). The server reports support only for agents that sendprovider: v0.1.x agents and v0.2.0's release candidates send none. Withagent.manageCRD=false, apply the newdeploy/chart/crds/before the agent starts, or the new status fields are pruned. The knowledge base version label changes (#77). - A cluster where a node's kube-proxy is more than 3 minors (2 for a
kube-proxy older than 1.25) older or newer than that node's kubelet now
has a
version-skew/kube-proxy-kubelet/<node>warning, which lowers the score and fails--fail-on warning. It appears only for inventories that record kube-proxy's node, from agents and CLIs of this release. Bring the kube-proxy DaemonSet's version within the policy, or accept it with an ignore rule (key: version-skew/kube-proxy-kubelet/<node>) (#148). - The
helmcapability is partial, namingargocdorfluxinskipped, when a GitOps tool deploys charts it cannot read releases for: on a cluster with no Helm release that serves the tool's CRD or has workloads with its tracking metadata; for every chart read from an Argo CD Application, whether or not the cluster has Helm releases (helm templateleaves none, sokubeVersionand stored-manifest checks did not run); when the tool's list is forbidden; when a HelmReleasechartRefdoes not resolve to a chart; and, for both tools, when API discovery fails. A Flux whose HelmRelease list is served and empty adds no gap.helmis optional, so this alone never makes the verdictunknown, but a partialhelmholds add-on and Helm-release findings in the notification baseline instead of resolving them. Without the grant for the tool it runs (rbac.gitops.argocdorrbac.gitops.flux), a cluster that serves either CRD reportshelmpartial, since the list is forbidden; this includes upgrading the chart on a Flux cluster whose releases were read in full before. Withrbac.gitops.flux, the Flux gap clears once its Helm releases are read and its chartRefs resolve. On Argo CD,helmstays partial, grant or not, whenever an Application deploys a chart or the cluster has no Helm release (#70). - Clusters that run Kubernetes Dashboard, Promtail, Grafana Agent or Weave
Net, or the
bitnamilegacyIngress NGINX image, now have aneol-addonblocker, so their verdict isblockedand--fail-on blockergates fail. Karpenter, Gatekeeper and Fluent Bit installs, whose images used to be listed inunrecognizedImages, are judged by release line and can raiseeol-addonoreol-approachingfindings. Replace the add-on, or accept the finding with an ignore rule (key: eol-addon/promtail). The knowledge base version label changes (#49). - An empty Node list is no longer read as clean: the
versionscapability is partial, with the reasonno nodes listed: kubelet skew and node runtimes not assessedandnodesinskipped. On a live cluster that is a required gap, so a cluster with no listed node (a control-plane-only test cluster, say) readsunknowninstead ofready, as it does when the Node list is forbidden; gate it with--allow-incompleteif that is expected. The server does not flag an empty Node list itself, so clusters whose agents predate this read as before until the agent is upgraded (#174). - Notifications: a deprecated-call blocker missing from a scrape of an
apiserver that has been up for less than 24 hours is held in the baseline
instead of resolved, so an apiserver restart, which empties
apiserver_requested_deprecated_apis, no longer sendsbecame-readyor a laternew-blockerfor a caller that has not called again yet. The agent reports the scraped apiserver's start time asapiServerStartTime(process_start_time_seconds), which is not part of the snapshot's identity, so moves between HA apiservers store no new snapshot. The hold ends on the first scrape at least 24 hours after that apiserver started, measured on the agent'scollectedAt: within 24 hours plus--force-sync-every(default 1h) and one--intervalof the restart. A caller of an API the scraped server version no longer serves is resolved at once. A start time before 2014 or more than 10 minutes aftercollectedAtis ignored. Agents without the field behave as described in the #189 entry below. A fix is held too, and a client that calls less often than daily is resolved when the hold ends (#204). unknown-apicovers every API group the knowledge base generator's scheme registers (k8s.io/apiplus the apiextensions and apiregistration schemes), not only groups with lifecycle entries:--filesscans and/gatemanifests ofinternal.apiserver.k8s.ioStorageVersion andimagepolicy.k8s.ioImageReview objects now give anunknown-apiinfo finding, which changes neither score nor verdict. CRD and aggregated API groups stay silent. Live scans read only the APIs the knowledge base flags, so they are unaffected. The knowledge base version label changes (#172).- The agent keeps what it decoded from each Helm release, keyed by the
storage object's namespace, name, UID and resourceVersion, so a tick
fetches only releases that are new or changed; a one-shot
scankeeps nothing. Measured indocs/operations/scale.mdon a kind lab with 2,000 KWOK nodes (2,001 with the control plane), about 14,000 pods and 1,000 Helm releases, before thekube-systemchange below: a steady tick went from 1,061 to 61 API requests, 90 to 68 MiB read (4.3 MiB on the wire), 33 to 5.6 s and 23.5 to 3.6 CPU-seconds. The first tick after a start still reads every release (1,061 requests, 36 s there). At 200m, the chart's default until the CPU limit moved to 1 CPU (see above), such a first tick reached the Helm step's deadline and left 132 of 1,001 releases for the next tick (#71, #229). - Each
kube-systempod is read once per tick: the add-ons take thekube-systempods' images and labels from the control-plane version check's read and list the other namespaces with the field selectormetadata.namespace!=kube-system; about 4,000 pods at 2,000 nodes are no longer listed twice. When the version check fails before it has read them, the add-ons list every pod as before, and a server or proxy that rejects the selector with 400 is asked again without it. The RBAC is unchanged (#227). - The Helm manifest memory test runs only in
make test-heapand CI's test-heap job (UPGRADESCOPE_HEAP=1), no longer in a plaingo test ./..., where a busy machine inflated its reading. The quoted memory figures are re-measured on a GitHub-hostedubuntu-latestrunner: the worst Helm releases peak at 32 to 46 MiB of live heap against the 64 MiB bound (CI run 37160469086); the/fleetof the widest gaps grows the heap 11.4 MiB against a 20 MiB bound (CI run 37152753746); and for 2000 clusters with 200-byte names, 100/metricsclients that never read peak the heap at 125 MiB above idle, both fleet slots and the held responses included (CI run 37160469086). No bound changed (#212, #213). - Notifications: a blocker or EOL warning that came from a capability the
current pass did not assess (unavailable, or partial over it; required or
optional) is carried forward in the baseline. It is not resolved in
that pass and not new when the capability returns, and
became-readywaits while one is carried. A capability that a finding was already seen without cannot hide it later, so a steady gap such asrbac.helmSecrets=falsestill resolves and re-alerts. For an agent that does not reportapiServerStartTime(v0.1.x, v0.2.0's release candidates), an apiserver restart emptiesapiserver_requested_deprecated_apis, so a deprecated caller that has not called since is resolved, which can sendbecame-ready, and is announced again when it calls; newer agents get the hold described above (#189, #204). - A cluster inventory that does not report a required capability
(
api-usage,versions, andaddonswhen the knowledge base lists add-ons) has a required gap, so it readsunknownunless it holds a blocker: an emptycapabilitiesmap no longer readsready. Ingest refuses (422) an inventory whosesourceis set to anything butcluster, or whosecollectorSchemathis server does not know. In an inventory withoutcollectorSchemafrom an agent reporting 0.2.0-0 or later (v0.2.0-rc.1 and v0.2.0-rc.2), a usage count that names no object is not judged: api-usage is partial over that API, a required gap when the target removes it, so the verdict isunknown, notblocked(#194). - A team's
readyis true only when its verdict isready: a required gap in the report, or a blocker no team is attributed, now makes it false, where it used to be true whenever the team had no blocker of its own. The CLI's team table prints the verdict instead ofready yes/no(#196). spec.targetsof aClusterReadinesstakes at most 8 entries (schemamaxItems). An agent reading an older object with more evaluates the first 8 distinct valid targets, in spec order, and counts the rest in onestatus.notAssessednote that names the first left out. Given more than 8 distinct minors in--targets, the agent refuses to start. Finding titles in the status are clipped to 512 bytes and remediation to 1024, as stored (JSON escapes counted). Past 3 hops, the version-skew upgrade-path title, in every output, shows only the start, the first step and the end of the path (1.29 → 1.30 → … → 1.36). The worst-case status is about 273 KiB (#191).- The agent refuses at startup
--interval 0, which used to mean the 10m default (minimum 1m, as for any other value below it), and a--cr-namethat is not an RFC 1123 subdomain, which used to fail every tick (#192). The chart's values schema refuses the same names, and more than 8agent.targets, at install. - After at least 3 failed ticks in a row, and once the last success is
older than twice the interval plus 12 minutes, the agent stops exporting
its verdict, score, finding and capability gauges, so
UpgradescopeUpgradeBlockedandUpgradescopeVerdictUnknownresolve whileUpgradescopeAgentNotTickingfires: alert on that one too (#199). - The Action, with
versionunset, runs its own ref's release: the tag at a release tag ref (@vX.Y.Z,@vX.Y.Z-rc.N), or at a full commit SHA the release tag that points at that commit (git ls-remote --tags). At a commit SHA no release tag points at, or when the lookup fails, it runs the latest release and warns; at any other ref (a branch,v0) it runs the latest release. Withversion: latestat a release tag ref, it warns when latest is older than that ref. Only a ref of this repository counts: inside a wrapping composite action the default stays latest. It used to run GitHub's latest release, so@v0.2.0-rc.2ran v0.1.1 (#197). --db: a path containing a%XXescape opens the file of that literal name; it used to open the percent-decoded name. Rename a database created under the decoded name before upgrading. A path containing?or#, which rc.2 refused, now opens that exact file, and a NUL byte is refused (#195).- An object of a flagged kind that neither
managedFieldsnor the last-applied annotation attributes to a writer is an info finding,authorship unknown, which never changes the verdict or score; it used to be missed. Inventories record such objects inapiAuthorshipUnknown(#199). - The hand-written
supplement.jsonis gone: the four entries it held (autoscaling HPA v2beta1 and v2beta2, both PodSecurityPolicy versions) were already in the generated dataset, which won on overlap. The lifecycle dataset is now the generated file alone, and the facts the generator adds by hand carry citations (#166). - Breaking for webhook receivers: the generic webhook now sends one
JSON body per cluster and evaluation pass (
schemaVersion1, lowercase keys:deliveryId,type,timestamp,cluster,targets,changes,omitted) instead of one PascalCase event (Cluster,Target,Kind,Title,Detail) per change and target. With--webhook-secretthe body is signed (X-Upgradescope-Signature, HMAC-SHA256). Delivery is at least once: deduplicate ondeliveryId. Update receivers; seedocs/reference/webhook.mdandapi/webhook.schema.json. - Breaking for
serveoutside the chart:serve --listendefaults to127.0.0.1:8080(v0.1.1::8080). A binary ordocker run … servethat other hosts, a Service or a port-forward must reach now binds loopback and is unreachable: pass--listen :8080. Open read access off loopback is refused, so also pass--read-token(or--read-token-file), or--allow-anonymous-readto accept open reads. The Helm chart already passes--listen, so chart users are unaffected. - Breaking:
tokens revoke <cluster>no longer revokes every active token of the cluster. It exits 1 unless you pass--id <id>(one token, fromtokens listortokens create) or--all. Scripts that revoked a cluster's tokens by name alone must add--all; rotation can now usetokens create <cluster>thentokens revoke <cluster> --id <old id>. - Breaking for SARIF consumers: each result's
ruleId, and each rule intool.driver.rules, is the finding key (for exampleremoved-api/extensions/v1beta1/Ingress) instead of the category (removed-api), and there is one rule per key. GitHub code scanning accepts the new output. Update any other SARIF filter, dashboard or suppression list that matches a categoryruleId, such asremoved-apioreol-addon. - The JSON report (
scan --output json,--write-baseline) is a versioned contract:schemaVersion1 starts after v0.1.1, withtoolVersionandverdict(ready,blocked,unknown).readystays and is now true only for the verdictready, where v0.1.1 meant "no blocker" (see the fail-closed gate below).kbVersionhas a new format. Readers should ignore unknown fields; seeapi/report.schema.json. serve --retentiondefaults to90d: the server now prunes evaluations and snapshots older than that at startup and then daily, keeping each cluster's latest snapshot and its evaluations. v0.1.1 kept everything. Pass--retention 0to keep all history.serve --ingest-tokenis optional (v0.1.1 required it). Without it the server accepts pushes only with per-cluster tokens fromtokens create. A shared token may push as any cluster, so prefer per-cluster tokens.- A Slack or webhook sink that answers 429 or 503 with
Retry-Afteris left alone for that delay (capped at an hour): the sink is not called for that message or any other queued for it, and the held messages keep their attempts. A queued notification is given up once it has been queued for 8 hours, so a receiver limited for good cannot hold a growing backlog. The hold is kept in memory, so a restart forgets it, and messages still queued after a server outage longer than 8 hours are dropped unsent. -
Inventories from collectors that predate this release report
crdsas not assessed. This includes the reports the server re-evaluates for existing v0.1 agents, and saved--filesinventories. The gap is not required, so the verdict and the score are unchanged. -
Chart: the agent can read the resources of the newly known deleted APIs (for example
auditsinks,clustercidrs,podpresetsand the DRA alpha kinds), so a cluster that still serves them is checked. - Chart: the agent's ClusterRole can list
ingressclasses(networking.k8s.io) for add-on detection; a test pins the grant. --filesreports no longer listaddonsundernotAssessed: add-ons are assessed from the manifests.- The gate fails closed. Under
--fail-on blocker|warning(the default), a scan with verdictunknownexits 2. A required check that could not run givesunknown: for example, RBAC denied, the cluster was unreadable, or the target is beyond the knowledge base. v0.1.1 reported such scans asready: trueand exited 0. Pass--allow-incompleteto gate on findings alone. - The server gate fails closed too.
POST /api/v1/gatedefaults tofail-on=blocker: a blocker or anunknownverdict answers 422 with the full report body, andfail-on=neverkeeps 200. v0.1.1 always answered 200. Usecurl --fail-with-bodyto fail the CI step and keep the SARIF;?path=gives SARIF results file locations, and with?cluster=only what the manifests introduce counts (#120). - A live scan whose control-plane skew could not be judged where upstream
would have told the version now gives
unknownand exits 2 (it gavereadyand exited 0); the agent'sClusterReadinessstatus and the server's reports readunknowntoo. That is a component pod inkube-system(labelledcomponent=ork8s-app=, or named after the component) on an upstream-named image under a digest or a tag that is not a version (kube-proxy@sha256:…,kube-scheduler:latest), or a kube-apiserver, kube-controller-manager or kube-scheduler pod whose version is not read. Pin the image to a version tag, or pass--allow-incomplete. A kube-proxy pod on a vendor image of another name (Oracle OKE'soke-public-kube-proxy) does not change the verdict. A component image tagged 0.x is never read as Kubernetes 0.x: a kube-scheduler is read as a scheduler-plugins build, any other is unreadable (#169). - A scan of a cluster that could not be read at all exits 1, and so does a
--filesscan that found no Kubernetes objects. Neither reports 100/100 any more. - Deprecated-API detection asks who writes the deprecated API
(
managedFields), not whether the apiserver still serves it. This removes the false removed-API blockers on clean clusters (#3). One removed API is scored once, with its caller evidence attached. - SARIF results carry file and line locations, so GitHub code scanning accepts them. Findings without a file are reported as notifications.
- Kubernetes versions with a vendor suffix (EKS, GKE, k3s, RKE2, OpenShift) are parsed. The skew policy now enforces the kube-proxy and the kubelet upper bound after the upgrade.
- The knowledge base version label is derived from the embedded data:
k8s.io/api vX; lifecycle <digest>; registry <digest>. - Server: open read access is refused unless the address
serveactually binds is loopback or you pass--allow-anonymous-read. Connections have read, write and idle timeouts. Snapshot and gate bodies are capped (--max-snapshot-bytes,--max-gate-bytes). Responses carry CSP and other security headers. A cluster name is bound to the first non-empty cluster ID pushed under it (a name first registered with none is bound by the first ID pushed after it, #368), and a push with another ID, or none, is then refused with 409. Stored verdicts are re-evaluated, and reads come from the latest snapshot. Ingest and notifications are written atomically through an outbox. - Chart: the agent's ClusterRole names every rule; it has no wildcards.
The chart no longer renders the
ClusterReadinessobject, because the agent owns it. The image defaults to the chart'sappVersion, and the published chart pins it by digest (image.digest). go install …@vX.Y.Zbinaries report their real version and serve the dashboard, because the built dashboard is committed.go.modrequires Go 1.26.9.- Release notes keep breaking changes in housekeeping commits
(
chore!:,docs!:). - A
--targetthat is not an upgrade of the cluster (a downgrade, the same minor, or a typo) gives verdictunknownwith a requiredtargetgap. The gate exits 2 even with--allow-incomplete. The JSON report has a newserverVersionfield, and the table header shows it. - Add-on findings describe each install. Every namespace that runs an add-on is its own instance. Findings are grouped by release line, name only the affected namespaces and teams, and list at most 10 installs ("and N more"). A newer Helm release in one namespace no longer hides an older end-of-life install in another.
- A finding lists at most 100 affected namespaces, in
namespacesand in its evidence sentence, sorted, and counts the rest in the newnamespacesOmitted, asobjectsandobjectsOmitteddo;teamsstill names the teams of every affected namespace. A namespace-scoped ignore rule suppresses such a finding only when it lists every namespace (#121). - Version-skew finding keys:
version-skew/<component>is split intoversion-skew/<component>-newerandversion-skew/<component>-behind, and the upgrade-path finding isversion-skew/upgrade-path. Baselines and notifications see these findings as new once. - A baseline marks a finding
unchangedonly if its severity has not risen since the baseline. - Deprecations that take effect after the target are titled as future deprecations, with "(projected)" beyond the knowledge base's horizon.
- Server:
GET /api/v1/clusters/{id}and itsreport,findings,teams,historyandexport, andGET /api/v1/fleet/teams, run one at a time and can answer503withRetry-Afterafter waiting 30s for their turn; retry them as the agent retries pushes./api/v1/clusters,/api/v1/fleetand/metricsread no snapshot inventory and no stored report; a/metricsscrape waits at most 5s for its turn. When the responses those reads hold for slow clients fill their budget, a response is sent while the read holds its turn, and a client that has not taken it within 20s is disconnected (#121). - Server:
/api/v1/fleet?targets=takes at most 16 distinct minors, and/api/v1/clusters/{id}/history?limit=at most 1000; more is422. A snapshot push that is not valid UTF-8 is422(#121). - Server: JSON responses and stored reports write
<,>,&, U+2028 and U+2029 as themselves, not as\uescapes (#121). - Server:
POST /api/v1/snapshotsanswers422, naming the field and the rule, for an inventory no collector writes, before anything is stored: aclusterNamethat is not an RFC 1123 subdomain of at most 253 bytes (#37); a namespace that is not an RFC 1123 label, an object name over 253 bytes or with/or%, a node or Helm release name that is not an RFC 1123 subdomain, a team label that is not a label value; a string over 16 KiB (a capability reason over 64 KiB, an object's field manager over the apiserver's 128 bytes or not printable), more than 100 objects in an API usage entry, a group/version/kind listed twice in one list, more than 200 unrecognized images or more than 32 capabilities. The free text a collector copies whole from the cluster is cut to those limits, by the agent and at ingest, not refused: a capability's reason (one failure per resource the agent could not read) and skipped entries, and theupgradescope.dev/ignoreandignore-reasonannotations. v0.1 agents' inventories are within the rest, but not necessarily their cluster names:tokens create, cluster rename (the CLI's too) and the agent's--cluster-name(checked at startup, with--team-label) take only RFC 1123 subdomains now, and v0.1 took any name (#121, #37). - Upgrade note: a cluster a v0.1 server registered under a name that
is not an RFC 1123 subdomain (uppercase,
_or spaces:Prod-EU,prod_eu) gets422on every push once the server is upgraded, and its per-cluster tokens are bound to that name. For each such cluster (upgradescope clusters list), runupgradescope clusters rename <old> <new>, which keeps its history and re-binds its tokens, then set the agent's--cluster-name(chartagent.clusterName) to the new name. See the operations guide's cluster lifecycle (#37). - Server:
/api/v1/fleetwithout?targets=opens at most 16 columns, as?targets=takes at most 16: the minors with the most clusters to fill them, the older on a tie. The rest are counted in a newtargetsOmitted, which the dashboard shows; ask for them with?targets=(#121). - Server: a report is at most
--max-snapshot-bytes. A push whose report for a target would be larger is413and stores nothing; a what-if report is413(the fleet teams rollup lists such a cluster asmissing);/gate?cluster=is413; the re-evaluation pass keeps what is stored. An export larger than the limit is413, saying to read the JSON report (#121). - Server: the evaluation summaries that
/clusters,/fleetand a cluster's detail carry list each gap's capability cut to 64 bytes, its reason to 256 and at most 3 skipped entries of 128 bytes, with a newskippedOmittedcount;/clustersand/fleetlist at most 1 KiB of gaps per evaluation, the required ones first, and count the rest in a newnotAssessedOmitted. The report keeps every gap whole (#121). - The kubelet skew findings name at most 100 nodes, and count the rest ("and N more"); their titles count every node.
- Server:
POST /api/v1/gateanswers413for a stream over its node budget, which a realistic kubectl YAML stream reaches at about 4.4 MiB (v0.1 decoded streams up to 20 MiB), or a document whose YAML aliases expand past 4 MiB, or a stream they expand past--max-gate-bytes; split such streams. It answers413too when its answer could be larger than--max-gate-bytes(it lists every object the findings name, with?path=; with?cluster=, the cluster's findings count too), and422for a?path=over 512 bytes or not UTF-8. A UTF-16 stream gets422. A/gateor snapshot body that does not arrive within the 60s read timeout gets408, which the agent retries (#121, #100). - Server database: migration 0007 (SQLite and Postgres) adds
evaluations.not_assessedand fills it from every stored report on the first start, so that start takes longer on a large database. A server rolled back after it still runs, but summaries do not follow what it writes: an evaluation it creates shows nonotAssessed, and one it refreshes keeps thenotAssessedit had, until a newer server writes that evaluation again. - Chart: the server's memory limit is 1Gi (was 512Mi), and its
GOMEMLIMIT~921MiB: the worst case of one/gaterequest, one push, one read, two reads of a 500-cluster fleet, the responses held for their clients and the re-evaluation pass, measured on SQLite with fourserver.targetsand notifications configured, is ~865 MiB (a limit below about 962Mi does not fit it). Each extra target adds about one report of up to--max-snapshot-bytesto every push and one to the re-evaluation pass (18-33 MiB each at the default 20 MiB): ~54 MiB of that sum per extra target, so withoutserver.targetsit is about 650 MiB, which 768Mi holds. serve --targetsand the chart'sserver.targetstake at most 4 distinct minors; more is refused at startup (and by the chart's schema) with a message saying why. The server's memory bounds are measured at that many (#121).
Fixed¶
- The dashboard's Registry page no longer shows a stale stored status. An
entry's badge is now derived from its dates and today's UTC date (a date on
or before today has passed, as in the scan's own rule): a passed
eol_datewith an extended date still ahead reads "extended support only · until", and a passed eol_dateotherwise reads "eol ·". The split line is in the past tense once a date has passed. RKE2 Ingress NGINX read "supported · 2026-03-31" seven months after that date. Scans and verdicts were never affected, since they use the dates. The retention page now lists verdictamong the fieldsGET /historyreturns (#383). - Postgres:
serve,tokensandclustersno longer fail every start with "context deadline exceeded" after 10 s when a migration runs longer, or when another replica holds the migration lock, which a starting replica now waits on as documented. Only the connection check at open is still bounded, at 10 s; as on SQLite, the startupProbe bounds the migrations. SIGINT or SIGTERM during the migrations stopsservepromptly with an "interrupted" error, and the interrupted migration is not recorded, so the next start runs it again. The migration session setsclient_connection_check_interval=2s(PostgreSQL 14 and later), so a pod killed mid-migration does not leave a statement such as migration 0010'sCREATE INDEXrunning on the server with its locks and the migration lock (#364). /api/v1/gate?cluster=blames a pull request's in-tree volume plugin on the manifests. A PR that added a PersistentVolume withspec.rbd, or a pod template with an inlinegitRepovolume, to a cluster that already used the plugin answered200 ready, with an empty SARIF, a JUnit "no findings" and an empty Code Quality report; it is now422 blockedwithsource: manifest, as without?cluster=. The finding stays whole (the cluster's objects and the PR's),clusterVerdictis unchanged, and a PR that adds none still passes. The PR's volume-plugin objects carry?path=, with or without?cluster=, so SARIF, JUnit and Code Quality report them at the PR's file, where an object with no file was dropped as "no file location". A cluster whose stored row already listed 100 objects no longer pushes the PR's object out of the finding, SARIF or Code Quality: the merge keeps every ref, and the listing is cut after suppression. A team-scoped gate's share is under Security (#365).- A cluster name first registered by a push with no
clusterId(an agent that could not readkube-system) is bound by the firstclusterIdpushed after it, from whichever cluster sends it, and that push continues the name's history, as before; the docs now say so in place of "two clusters cannot share a name". The server logs it (server: cluster name adopted its first clusterId: cluster=... clusterId=... snapshotId=...), and the adopting push sends nobecame-readynotification, since the history it continues may be another cluster's. Two pushers that both send noclusterIdshare the name: give each agent its own--cluster-name(#368). - Documentation:
--server-ca-file's help and every page that names$SSL_CERT_FILEor$SSL_CERT_DIRsay that the two change the system roots on Linux and other Unix systems only; Go ignores them on macOS and Windows, where--server-ca-fileis the way. A test fails on a page, this changelog included, that names either variable without Linux, macOS and Windows (#372). - Ingest refuses (
422, naming the field) a negativecount,objectsOmittedor namespace count in every API usage list,apiUsage,apiAuthorshipUnknown, a CRD'susageand a Helm release'smanifestApis, as it did forvolumePluginsalone; a count of 0 is accepted. Such a push was stored and read "(-5 objects)", and a negative Helm manifest row was dropped by the engine with no finding, so ahelm upgradeblocker vanished and the score read 100. The agent repairs its inventory before it pushes: it drops the bad entry (a bad Helm manifest row drops its release, a bad CRD usage row its CRD), names an API usage entry inapi-usage's skipped list and a release inhelm's, and marks the capability partial. An agent that sends a negative count now gets422where it got202; no released agent writes one. The #351 and #361 entries took this check as given for every list; it held for volume plugins only (#373). go test ./deploy/chart/, and somake test, skips the chart tests on a machine withouthelminstead of failing the package: the read-closure test calledt.Skipon the parent test from a subtest.make hack-testruns the helm-calling packages withhelmhidden fromPATHand fails unless they pass with skips (#376).- Documentation: the documented
cosign verifyof the image and of the chart (docs/operations/install.md, the chart README and the release notes' Verify block) states the minimum cosign, 3.x or 2.6.3 or later: the signatures are cosign 3 bundles, and cosign 2.6.2 and older exit10withno signatures found.cosign verify-blobofchecksums.txtworks from 2.4.3. The release workflow's verify job runs the documented commands with cosign v2.6.3 as well as the default, andmake claims-checkfails a section of those pages that showscosign verifywithout the minimum (#377). - CI: kb-refresh opens or updates its failure issue when one of its jobs
times out or is cancelled: a job that hits
timeout-minutesconcludescancelled, which the report did not match, so a timed-out stage, run or verify job opened none. A person cancelling the whole run stays quiet. Its two pull-request jobs and the report job have a 15-minute timeout, and CONTRIBUTING.md documents the forced-diff dry run, the one way to exercise the pull-request path on demand (#378). upgradescope tokens create --readhonours$UPGRADESCOPE_DB_URLand--db-url-file. It ignored both, opened the default./upgradescope.db, printed a token that the server answered401to, and inside the chart's distroless pod failed with a read-only filesystem error, which broke the first step of the chart's NOTES for every Postgres hub withoutserver.readToken. Everytokenssubcommand resolves and opens the database the same way now (#382).- The server's stored and read-time evaluations apply the objects'
upgradescope.basit.engineer/ignoreannotations (withignore-reason), asscan, the agent and/gatealready did. The stored verdict and score, fleet cells, history, notifications (annotating the last blocker sendsbecame-ready; an object that arrives already annotated sends nonew-blocker),GET /api/v1/clusters/{id}/report(suppressed, sourceannotation) and the dashboard's suppressed count now agree with theClusterReadinessstatus and/gate?cluster=. The agent'sspec.ignorestill never reaches the server. Auditor exports list the accepted findings: the CSV has onesuppressedrow per accepted finding and a suppressed count in the summary row (under Changed), and the HTML export has a suppressed section. Evaluations stored before the upgrade pick this up at their next re-evaluation: the next changed push, a knowledge-base change, or the first pass after UTC midnight (#381). - Notifications on a cluster on the newest minor, whose default target is
past the knowledge base's horizon and so reads
unknownwhen clean: its first blocker (an EOL add-on, say) is now anew-blocker, and a blocker that anunknownpass saw resolved is announced again when it returns. Both were silent, because anunknownpass was never a baseline. A pass whose only required gaps hide no finding (kb-coverage,target) saw everything a blocker comes from, so it is now compared like a decided one and is the next pass's baseline, and the blockers it resolved keep abecame-readypending for the firstreadypass. A pass that isunknownbehind a required gap that hides findings (a collector failure, knowledge-base skew) still sends nothing and is not a baseline (#380). - Documentation:
go installfrom the module proxy builds a binary whoseupgradescope versionprintscommit: unknownandbuilt: unknown, asdocs/operations/install.mdnow says (it said the commit was reported). The README's verify block setsVERSIONand downloads the files it checks, so it runs as pasted. The authorship note says what the public history shows. The README, SECURITY.md, the chart'srbac.createdocs and the security page say that the agent'screateonclusterreadinessescannot be limited by name, so it can create a ClusterReadiness under any name, and that it never deletes one. The claims ledger names no closed issue as a tracker, and the rows round 3 passed moved into the area tables (#366). - A remediation never sends a manifest to a less mature API than the one it
uses (alpha before beta before GA, from the version name). A
flowcontrol.apiserver.k8s.io/v1manifest checked at target 1.18 was told to writev1alpha1; now the finding says to upgrade the cluster to the minor that first servesv1. When the GA replacement is not served at the target yet, the finding says which version is right until then, and names the GA one with the minor that first serves it ("no replacement Kubernetes 1.18 serves is known; batch/v1 CronJob is served from 1.21; batch/v1beta1 is the right version for Kubernetes 1.18 until then"). A manifest at an API version the target does not serve yet is offered another version only if that one is at least as mature; otherwise the finding says "Kubernetes 1.33 serves no version of DeviceClass as mature as v1 that the knowledge base knows: upgrade the cluster to Kubernetes 1.34 before applying it" (#332). - The introduced minors of six pre-GA APIs are corrected from the Kubernetes
changelogs, each cited in
tools/gen-kb/fixups.go:batch/v2alpha1CronJob is 1.5,scheduling.k8s.io/v1alpha1PriorityClass 1.8,settings.k8s.io/v1alpha1PodPreset 1.6,auditregistration.k8s.io/v1alpha1AuditSink 1.13 anddiscovery.k8s.io/v1alpha1EndpointSlice 1.16. These five had been clamped to 1.17, the oldest release the generator reads, so a target of 1.16 reported them as "not served until 1.17".discovery.k8s.io/v1beta1EndpointSlice was tagged 1.16 and is served from 1.17, so a manifest at that version is now "not served until 1.17" at target 1.16 (#332). scanwithout--targetsays what to pass:--target is required: the Kubernetes minor to upgrade to, e.g. --target 1.37 (the newest this build knows; see upgradescope version). The example, in this message, in the--targethelp and in the command's examples, is the knowledge base's horizon rather than a fixed minor. The exit code stays1(#336).- A live
scanwith no kubeconfig names where it looked and what to do instead, in place of client-go's hint aboutKUBERNETES_MASTER:no kubeconfig found (looked at <paths>); point to one with --kubeconfig <file> or $KUBECONFIG, pick a cluster in it with --context <name>, or scan rendered manifests without a cluster: upgradescope scan --files <dir> --target 1.37. A kubeconfig that has contexts but nocurrent-context, with no--context, says so and lists up to five of them:the kubeconfig (<paths>) sets no current-context; pick one of its contexts with --context <name> (a, b), or set one with kubectl config use-context <name>. Both exit1(#337). - A mistyped subcommand prints cobra's suggestion on separate lines (the
message, a blank line,
Did you mean this?and the commands), where it ran together with a literal\n, and does so when flags follow the typo (scann --target 1.37 --files .,scann -o json) as well. The typed name is still escaped (#338). - The values schema refuses a
server.persistence.sizethat is not a Kubernetes quantity (size=lots, where it needs1Gior500Mi) at render time, where the API server used to refuse the PVC when it was created (#346). - Documentation:
docs/getting-started/cli.mdexplainsUNRECOGNIZED IMAGES(images that no add-on in the registry claims: information about add-on detection, never a finding, with no effect on the score or the verdict) and how to scan a chart, and the FAQ has an entry on unrecognized images (#340). - Documentation: an edit to the
spec.targetsorspec.ignoreof aClusterReadinesstakes effect at the agent's next tick, not at once: the old verdict stays until then, up toagent.intervalplus its 10% jitter (about 11 minutes at the default 10m). Whilestatus.observedGenerationis lower thanmetadata.generation, the status does not yet reflect the edit; the in-cluster and GitOps guides givekubectl wait ucr/cluster --for=jsonpath='{.status.observedGeneration}'=<generation>to wait for it, and say that a sync wait or a Kustomizationtimeoutmust allow for the lag (#350). - Documentation: the CI-gate workflow in the README,
docs/getting-started/ci-gate.mdandaction/README.mdguards the SARIF upload withgithub.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository. GitHub gives a fork'spull_requestrun a read-only token, soupload-sariffailed with "Resource not accessible by integration" after the gate had run, and a pull request from a fork turned the job red for a reason that was not the gate. The job summary and the step annotations are the fork's report. Do not switch the trigger topull_request_targetto get the upload back: it runs with a write token and the base repository's secrets on files the pull request controls. Dependabot's pull requests have a read-only token too, though the head repository is yours; add&& github.actor != 'dependabot[bot]'if the gate runs on them.hack/docs-sarif-guard_test.shevaluates the condition in all three copies, and the claim AC-04 now says what the guarded snippet provides, and that the fork behaviour is read from GitHub's token rules, not from a fork run (#357). - The CI gate (
POST /api/v1/gate, with and without?cluster=) adds the posted manifests' counts to the cluster's with a saturating add. A cluster whose stored API-usage or volume-plugin count was at the integer maximum no longer wraps negative when the gate adds a manifest, which dropped the cluster's own finding from the proposed state and could change the verdict. The decrement for a replaced object no longer goes below zero (#361). Ingest refused a negative stored count involumePluginsonly, not in the API usage lists, until #373 (see above). - A live
scanand the agent read PersistentVolumes and StorageClasses for in-tree volume plugins. Anrbd(or other in-tree) PersistentVolume behind a claim, counted under its bound claim's namespace, or a StorageClass whose provisioner is an in-tree one such askubernetes.io/rbdorkubernetes.io/aws-ebs, is avolume-plugin/<plugin>finding that names the object (rbdblocks at 1.31). The knowledge base adds a cited in-tree provisioner to 13 plugins, part of the knowledge base version (cephfshas no provisioner upstream and is not mapped), and StorageClass manifests are checked byscan --filesand the gate too. The finding's text covers pods that mount a claim bound to such a PersistentVolume and the claims such a StorageClass provisions. Accepting every named PersistentVolume or StorageClass (an ignore annotation, or aname:rule) leaves the finding standing for the pods that name the plugin inline, which are counted as omitted; a rule with no object selectors still takes the whole finding. The gate counts the cluster's pods the same way beside a pull request's objects, so one annotated object in the pull request does not hide the cluster's blocker inclusterVerdict. The chart's new RBAC is under Changed (#362). - A Helm release whose newest revision failed is judged by its newest
deployed revision, else its newest superseded one, which is what
helm upgradewould diff against, not the failed one; a release with only failed revisions is a named partial gap onhelm, where it used to be dropped silently (#239). - When API discovery fails, the scanner's own deprecated-API LISTs are no
longer read as another client's calls, which could turn a ready cluster
red: those rows are withheld for that pass and
deprecated-callsis partial with the reason. The architecture and API-usage pages now match the collector: there is no Endpoints LIST, four managers are trusted (with the kube-scheduler caveat), and an object no writer can be attributed to is "authorship unknown", not "undetected" (#239). - Registry false greens: RKE2's ingress entry claims only its
-hardenedNbuilds, so RKE1's and early RKE2'srancher/nginx-ingress-controller-rancherNbuilds are found as Ingress NGINX (end of life) instead of reading ready; Flux is found from its v2 controller images (source-, kustomize-, helm-, notification- and image-controllers and source-watcher, mapped to the Flux line) and as Flux v1 (fluxcd/flux,weaveworks/flux), not only from its chart; Kubernetes Dashboard v1.x per-arch images are matched (end of life); and RKE2 Ingress NGINX carries split support, community end 2026-03-31 and SUSE Rancher Prime LTS to 2027-11-30, on the registry page as in the report (#265). - Argo CD Applications whose source is a native OCI chart (an
oci://repoURLand nochartfield) are read, as a chart named after the repository's last path element attargetRevision, for single- and multi-source Applications; Flux OCIRepository versions follow Flux's precedence, digest over semver over tag, and a semver constraint is reported as a range, never as an exact version (#267). - The agent conforms its inventory before pushing: it leaves out, and names as a partial gap, the Helm releases, manifest objects, add-on installs and images the server would refuse, so an object with a hostile name can no longer stop a cluster's reports (#268).
storage.k8s.io/v1alpha1VolumeAttachment is dated removed in 1.23, when kube-apiserver stopped serving it (#237, #266).- A removed or deprecated alpha API with no replacement chain, one whose
kind has only a later pre-GA version, now remediates to the newest later
version the target serves, with its stability (
migrate to scheduling.k8s.io/v1beta1 Workload (beta; may need enabling)) and a projected removal marked; this covers 7 entries (Workload, PodGroup, LeaseCandidate, ResourceClass, PodSchedulingContext) (#302). - Retention deletes in batches of at most 5,000 rows a transaction, on
SQLite and Postgres (evaluations, then notification baselines, then
snapshots), so a large backlog drains over successive transactions and
runs, one failure cannot wedge retention, and a run cut short keeps what
it committed. On SQLite the drain reads the new
evaluations(created_at, id)index instead of walking every report under the write lock on every daily run, even with nothing to delete: a prune with nothing to delete took about 190 ms over 20,000 evaluations of 30 KB before and 0.25 to 0.6 ms after (#263, #297). - The release path and the Action: GitHub's Latest moves only after the
provenance attestation exists, so a failed attest step leaves it on the
previous release; a
gholder than 2.68, which lacks the flags the Action passes togh attestation verify, is no verifier, and the Action falls back to cosign or stops with "gh 2.68 or later (or cosign) is needed to verify provenance"; and a commit SHA pin installs the newest tag at that commit whose release is published, or fails with "is not published yet", never suggestingverify-provenance: false(#303, #304, #305). - Documentation:
CONTRIBUTING.mdanddocs/architecture.mdlist every subcommand and package that exists; the Renovate example uses--output json(there is no--format);other-ci.mddescribes both clean JUnit shapes (a clean live scan is one passing test,readiness/no findings; a clean files-mode scan is 0 passed and 3 skipped); copy-paste examples carry no dates that expire; and the sizing guidance for large pods says about 60 KiB (computed), not 70, as the point past which a page of 1,000 pods passes the agent'sGOMEMLIMITat the chart's 256Mi limit. Tests now fail when a documented flag does not exist or the package map drifts (#306, #307, #308, #269, #290). - Suppressing every listed object of a removed-API finding (an
upgradescope.dev/ignoreannotation, or a rule withnamespace,nameorfile) no longer hides the live callers folded into it, which could show a cluster with a blocker caller as ready. Each folded caller now stands as its owndeprecated-api-in-usefinding at its own severity, in severity order; only a rule with no object selectors, taking the object finding or the caller's own key or category, suppresses it. This holds forscan,POST /api/v1/gateandspec.ignore: at the gate a re-emitted caller issource: cluster, and a blocker one keepsclusterVerdictblocked, not the gate's ownverdict. A team-scoped read drops the cluster-wide callers from a finding it cuts to its teams. A Helm manifest's warning is no longer dropped when the live scan flags the same object as info, so the score and--fail-on warningare not lowered by more evidence (#236). - The in-chart server on SQLite lost a retention prune or a
clusters deleteof more than a few tens of MB todisk I/O error (6410): the root filesystem is read-only and SQLite had no temp directory. The chart now mounts an emptyDirsqlite-tmpat/tmp(server.tmp.sizeLimit, default1Gi) and setsSQLITE_TMPDIR; measured on a 60 MB backlog, a delete failed without it and worked with it. The volume is node ephemeral storage, and one that outgrows its limit evicts the pod. A volume of your own mounted at/tmp(server.extraVolumeMounts) is used instead; without one, aserver.extraVolumesentry namedsqlite-tmpfails the render. If prunes had been failing, the first one after the upgrade deletes the whole backlog at once: raiseserver.tmp.sizeLimitto the size of the database before you upgrade (#242). - A push's notifications are diffed against the baseline the push committed
against, so a push racing another replica's push or the background pass no
longer loses or repeats a
became-readyornew-blockernotification (the push is evaluated up to three times in all: see Changed). Each notification sink is delivered in queue order with its own timeout, up to four sinks at once, so one hung sink no longer delays the others. The background pass and the reads that need a snapshot's server version,/historyamong them, no longer decode a whole inventory to learn it. On SQLite, behind 4,000 history rows with 30 KB reports, the notification baseline is read in under 2 ms and 100 points of history in under 5 ms; the notification delta of 20,000 new callers against 20,000 carried findings takes 38 to 54 ms, where it took 46 s to about 1 min 32 s (#241). - The agent's tick no longer leaves the ClusterReadiness stale and unmarked
after a slow collector step (see Changed). A failed spec read, or a
failed
spec.targetspatch, writes no status for targets it guessed and stamps noobservedGeneration: it marks the objectupgradescope.dev/status-error, fails the tick, and the next tick reads the spec again; the patch path no longer drops earlier errors of the tick. A CRD check that failed or ran out its 30 s at start is retried on every later tick until it succeeds; meanwhile each tick line carriescrdErrorat WARN andstatus.notAssessedleads with a note, without failing the tick. Push retries wait a random time from 0 to the backoff step (aRetry-Afteris the least wait plus up to a quarter more), so agents answered with the same503do not retry together (#238). POST /api/v1/gatemade room for the pull request's objects under the 100-object listing cap before it evaluated, so a cluster object pushed out of the listing could not be accepted by its annotation or a?config=rule selecting it, and a pull request that posted many objects could turn the gate red, or lower the score, for findings it did not cause. The gate now evaluates and suppresses with every object, then cuts each listing to 100, counting the rest inobjectsOmitted(#72).- The agent no longer follows a redirect when it pushes: a 3xx answer is a
permanent failure that names the status and
Locationand says to use the final URL in--server-url. A 301, 302 or 303 used to turn the push into a body-less GET that could read as delivered.Retry-Afteron 429 and 503 is honoured, up to 1 minute (#190). - When the cluster-wide pod list fails, add-ons are still matched from Helm
releases and IngressClass controllers. The
addonscapability is then partial with the reason, and a required gap, so the verdict staysunknownunless a blocker is found; when no Helm release was read and the IngressClass list could not be read either,addonsis not assessed, as before (#199). Upgrade the server before the agent of a cluster whose agent role cannot list pods: an older server takes that partialaddonsfor an optional gap, so a cluster it read asunknowncan readready. - The Action sets
sarif-fileonly after a scan that completed (exit 0 or 2). After a scan error it used to point at an empty or partial file, so the documented upload step guarded onsarif-file != ''failed with Invalid SARIF (#197). - A what-if evaluation (a target with no stored evaluation, in the cluster
report, findings and teams endpoints and the
/api/v1/fleet/teamsrollup) is timestamped in UTC, as a stored one is; it used to carry the server's local offset (#196). - The rpm is byte-reproducible: its Build Host header is pinned to
upgradescope, and the reproducibility check scans the packages for the build machine's host name and build paths. Pre-release deb and rpm file names use.where nfpm puts~, so they match theirchecksums.txtentries as downloaded, and the release notes of a pre-release no longer offer the Homebrew tap (#198). - Switching
server.sharedIngestTokenoff removes theingestTokenkey from a chart Secret written by this version or later, which writes it underdata. A Secret written by v0.2.0-rc.2 or earlier keeps the key: remove it once with thekubectl patchthat theserver.sharedIngestTokencomment gives. Checked by rendering; not yet on a live API server (#200). - A control-plane or kube-proxy pod whose version upstream would have
told but cannot be read makes the verdict
unknown, notready: an upstream-named component image under a digest or a tag that is not a version (latest), or an unread kube-apiserver, kube-controller-manager or kube-scheduler pod. Its skew was not evaluated, and it may be the component past the policy. A kube-proxy pod on a vendor image of another name (Oracle OKE'soke-public-kube-proxy) is an optional, disclosedversionsgap, and the verdict is unaffected. The gate change this brings is under Changed (#169). - Notifications after a cluster upgrade: a blocker that the cluster's new
default target adds (for example
networking.k8s.io/v1beta1ServiceCIDR, removed in 1.37, once the cluster runs 1.36) is notified instead of being taken as a silent baseline. Blockers it already had at its previous default target are not announced again (#34). --output tableand--output markdownexit 1 when the report cannot be written, like JSON and SARIF.- Pre-GA kinds written only by control-plane components (for example
LeaseCandidate) are no longer counted as removed-API blockers (#108). - Istio's end-of-life date. Pre-1.0 ingress-nginx images, and the ingress-nginx builds of AKS and RKE2, are now detected. GKE and AKS builds of an add-on are no longer judged by upstream's lifecycle.
- Remediation follows the replacement chain to an API that is still served.
--filesskips non-manifest YAML and unrendered chart templates, expandsListobjects, and records object locations.- The agent evaluates a repeated target once, and a stop in the middle of a tick is no longer counted as a failure.
- A stalled or unresponsive API server no longer hangs
scan(one ran for more than 7 minutes). Each request is bounded by--request-timeout,/versionand discovery honour the scan's deadline, and each collector step has its own share of the 5-minute budget, so one stalled step degrades only its own capability (#94). - An add-on older than the oldest release line the registry tracks is end
of life when that line has ended: cert-manager 1.5, Istio 1.5, Calico
3.24, Cilium 1.12, Kyverno 1.7, Argo CD 0.12 and Flux 1.24 were only an
addon-no-datainfo and read READY 100/100. They are now aneol-addonblocker keyedeol-addon/<id>/below-<oldest line>(a warning for a node runtime), citing that line and the product's lifecycle pages. Versions between tracked lines or newer than the newest, and products without release lines, stayaddon-no-data(#165). - A Helm release no longer hides an older install of the same add-on in
its namespace. Its
appVersionnow stands only for image tags and labels on its own release line; an image or label version on another line is a second install there, judged at its own version. An istioctl canary runningistio/pilot:1.28.10beside anistiod1.31.1 release inistio-systemread clean; it now gets the ended 1.28 line's blocker. The manifest gate replaces every install of an add-on in a namespace its manifests deploy it to (#165). - The same inventory gives the same report whatever the order of its
deprecated-API caller rows (and, since #367, of a CRD's
spec.versionsand a capability's skipped list): caller rows are sorted before they are judged and folded into usage findings, whose evidence sentence listed subresources in the order the rows arrived (#165). - Knowledge base:
rbac.authorization.k8s.io/v1alpha1(ClusterRole, ClusterRoleBinding, Role, RoleBinding; removed in 1.23) andnode.k8s.io/v1alpha1RuntimeClass (removed in 1.24) now block at the release that stopped serving them; they scanned as ready with only anunknown-apiinfo. Kinds kube-apiserver never stored as resources, and no manifest can create (PodStatusResult,EphemeralContainers,ReplicationControllerDummy,JobTemplate, the removedScaleandDeploymentRollbackbodies, the v1beta1AdmissionReview,ConversionReviewandpolicyEviction, and theapidiscovery.k8s.iov2beta1APIGroupDiscovery) are no longer removed-API blockers.scheduling.k8s.io/v1alpha3types stayunknown-apiinfos (#166). kb.Loadrefuses a lifecycle dataset with fewer than 150 entries or 100 removals, with an error that says the embedded data is corrupt, instead of judging every API as served (#166).- Registry validation rejects a citation on a placeholder or local host
(
example.com,localhost, an IP address, or a private name such as.local,.lanor.internal, which an intranet page cited for a private add-on now trips) andstatus: unknownwith aneol_date, which printed an uncited end-of-life blocker; a test fails on any file inregistry/datathat is not*.yaml. The registry notes that endoflife.date puts Istio 1.29's end of life at 31 October 2026 where istio.io says 12 October (#166). - Control-plane and kube-proxy versions are read from per-architecture
images (
gke.gcr.io/kube-proxy-amd64,kube-scheduler-amd64), so GKE's kube-proxy is skew-checked, and from VMware TKG's tags, whose build suffix follows an underscore (v1.28.7_vmware.1). A component pod whose version cannot be read (a digest-only image, a tag such aslatest, or a labelled pod running an image of another name) makes theversionscapability partial, naming the components and the first pod, instead of being dropped silently. scheduler-plugins'kube-scheduleris read by the Kubernetes minor its tag is built on (v0.31.8asv1.31.8), and kOps' and Talos'k8s-app=<component>labels mark component pods (#169). - The containerd compat blocker (
chart-incompat/containerd/<line>) names the nodes that cannot run the target even when they all run one version, so blocker-only outputs (gate, JUnit, code quality) say where (#169). servestopped during startup (a signal while it opens the store) no longer leaves its notification and re-evaluation workers running on a closed store: a shutdown that came before or during the server's start did not always stop them.
Security¶
- A team-scoped
/api/v1/gate?cluster=no longer sees other teams' in-tree volume-plugin evidence. The share it evaluates keeps a cluster's volume-plugin counts in the scope's teams' namespaces only, and drops the cluster-scoped counts (unbound PersistentVolumes and StorageClasses) and the cluster's PersistentVolume and StorageClass names. Before, a volume-plugin finding a PR introduced (blamed on the manifests since #365, see Fixed) merged every team's evidence, so a team-scoped read token learned other teams'rbdpod, PersistentVolume and StorageClass counts from its count, title and detail, and the share's cluster verdict and score depended on other teams' volume plugins. A live PersistentVolume ref carries no namespace, so the share also drops the scope's own PersistentVolumes bound to its claims: theirupgradescope.basit.engineer/ignoreannotations cannot accept a finding in a team-scoped?cluster=gate, where before (while the share leaked) they could.docs/operations/auth.mdsays so, and that a scoped read of a finding whose namespaces are all the scope's keeps the cluster-scoped refs it names, such as the cluster's StorageClass names (#365). serve --trusted-proxy-cidrrefuses a set of ranges that together hold every address of IPv4 or of IPv6, however they are split across flags (0.0.0.0/1with128.0.0.0/1), not only a single/0; such a set let any client set the team header. A partial cover is still accepted, and keeping the ranges to the proxy remains the operator's job (#370).- For a webhook URL that holds an
@, a delivery error in the log and inoutbox.last_erroris cut to its reason (no such host, connection refused, i/o timeout, the TLS reason), and no longer names the DNS name or the dialedip:port, which the transport error carried past the withheld host (see the #240 entry).serve's refusal of a bad--slack-webhookor--webhookURL uses a fixed text per kind of failure and no longer quotes part of the value (#371). clusters list,clusters delete,clusters renameandmcp --server-urlwarn on stderr when a token would go over plainhttpto a host that is not loopback (localhost,127.0.0.1,::1), saying that the bearer token crosses the network unencrypted. The warning names the flag that carries the token (--read-tokenforclusters listandmcp,--admin-tokenforclusters deleteandclusters rename) and never the token. It is a warning, and the request is still sent, so scripts that work keep working; anhttpsURL, a loopback host and a call with no token print nothing.mcpwarned already;clustersdid not (#347).- A pushed inventory's
volumePluginsis validated like its API usage (namespace keys, object references and the cap on objects; the API usage lists refuse negative counts too only since #373). Ingest refuses (422), naming the field, apluginthat is not a field name (at most 64 ASCII letters and digits, starting with a letter, so control characters and long names fail), a plugin listed twice or more than 64 entries, a negativecount,objectsOmittedor namespace count, a namespace key or object reference that is not a Kubernetes name, and more than 100 objects in an entry. The agent repairs its own inventory before it pushes: it drops an entry with a bad plugin or count, and the bad objects and namespace keys of the others, names what it dropped, and marksvolumespartial, so one hostile pod cannot make every push of a cluster fail (#351). - Text that a manifest or the cluster controls (titles, details, object
names, files, namespaces, warnings, the
clustersandtokenslist commands,serve's--team-maplogs and the binary's own error text) is escaped in table, markdown and text output and on stderr: C0 and C1 controls, DEL, bidi controls and line separators print as visible escapes, so a manifest cannot move the cursor, rewrite earlier lines or reorder text in a reviewer's terminal. JSON, SARIF and JUnit stay byte-exact.ci-gate.md,action/README.mdand the suppressions guide now say that onpull_requestthe config, baseline and ignore annotations come from the PR's own tree, so a PR can suppress its own findings, and show how to take them from the base commit (#245). mcp --httphas header, read and idle timeouts (10 s, 60 s, 120 s) and a 64 KiB header cap, and no write timeout, so long scans still return. Idle sessions expire after 10 minutes, at most 100 are open (a new one past that gets503withRetry-After), a401closes the connection, and a call is cancelled when its client disconnects, so an abandoned scan frees the scan slot. MCP reads exactly the members the schema check judged: a report with a repeated member name, or one that differs only in case from a property the schema declares, is refused, and so is a pattern-constrained string (target,since, a hop'sfromandto,support.minor,annualCostDelta) over 2 KiB, which is no longer cut. A cut that would break the tool's output schema is a tool error, never a protocol error (#264, #298, #299).- The Action verifies the downloaded binary's build provenance for every
release that publishes it (
gh attestation verifyagainst this repository'srelease.ymlat the release's tag, on a GitHub-hosted runner, orcosign verify-blobofchecksums.txtwith the archive pinned to it by sha256): the new inputverify-provenanceistrueby default, andfalsechecks the checksum only, with a warning. With it on there is no unverified fallback: thego installfallback after a failed download is gone, and release candidates are verified too. No job that builds, signs or publishes a release restores a cache (a test fails if one does), only the highest stable release movesv0,:latestand GitHub Latest, the tag ruleset protects release tags only, so a major tag can move, pg-conformance images are pinned by digest, and CI no longer uploads the Action's fixture findings to code scanning frommain(#244). kb-refreshno longer builds its unreviewed bump in a run scheduled frommain: the scheduled workflow builds nothing and dispatcheskb-refresh-build.ymlon the throwaway branchbot/kb-refresh-build, whose jobs configure no cache (an entry written there lands in a cache scopemain's CI and tag runs never restore). The job that commits checks the dispatched run through the API (workflow path, branch, head SHA, event, success, repository) and the download against the digest a seal job computed, and only it can write the bot branch or open the PR.make check-toolchainnow also fails on a literalgo-version:in a workflow, and the build jobs read Go fromgo.mod, as every other job does (#273, #11).- Built with Go 1.26.9 and
golang.org/x/netv0.60.0 (go.modrequires Go 1.26.9, and the Dockerfile pins thegolang:1.26.9image by digest). This fixes thenet/httpand HTTP/2,crypto/tls,html/template,net/textprotoandmime/multipartadvisoriesgovulncheckreported against Go 1.26.8 andgolang.org/x/netv0.59.0;make vulnpasses with an empty allowlist (#259). - Slack and generic webhook URLs, which are secrets, never reach the
server's log or the outbox's
last_errorpastscheme://host/…, and the host is withheld too when the URL holds an@(#240); the dial error still named it until #371 (above). - A snapshot push authenticated before its token was revoked, or before its
cluster was deleted or renamed, no longer commits: both stores re-check
the token and the cluster inside the commit transaction. A revoked token
answers
401, a rename or delete409, with nothing stored and no duplicate cluster created (#240). That held from the server's cluster lookup on: a rename or delete before it answered a per-cluster push401and let a shared-token push register the name again. Since #369 a per-cluster push answers409at any point after authentication; for a shared-token push the guarantee still starts at the lookup (see Changed). - The oauth2-proxy example sets
--pass-host-header=false, so the upstream sees the Host the server's new check accepts. Thedocs/operations/auth.mddanger box now states that group naming in the identity provider is part of the trust boundary: a group name holding a comma or a%XXescape, or with a space or tab around it, reads as other teams (#240, #250). - Argo CD
repoURLs and Flux OCIRepository URLs are recorded in the inventory without userinfo, query string or fragment: everything between the scheme (or the start, where there is none) and the last@(before any trailing OCI digest) is dropped, whether or not the value parses as a URL (an OCI reference pinned by digest, such asghcr.io/acme/chart@sha256:…, keeps its digest), and a URL with a?or#before that@, or with a scheme and no host, is recorded empty. A token embedded in the URL path (a Cloudsmith entitlement URL) cannot be told from a path and is kept. The agent does not read repository credential Secrets. GitOps chart references are written by whoever can create an Application or HelmRelease, who can therefore raise a falseeol-addonfinding; the GitOps guide says so (#70). - The Action prefixes every line of the gate's stderr and of the Markdown
summary it echoes to the log (both carry the scanned tree's file names)
with
|, and writes a CR in them as%0Dand a##[as# #[; the JSON and Markdown passes' errors reach the log only escaped inside a::warning. It escapes%, CR and LF in every input it echoes. So a pull request's file names or manifests, or an input, cannot start a workflow command; v0.2.0-rc.2 relayed them as written. A check for a leading::would not be enough: the runner strips all leading Unicode whitespace (a form feed, a vertical tab, a no-break space) before it parses a line. The release-tag lookup runs from an empty directory of its own, never the workspace, with repository discovery stopped there and a stall limit, so files a fork places in the workspace cannot redirect it or run a credential helper (#197). - The SQLite database, and its
-waland-shmfiles, are created 0600 under exactly the name given: a%3F,%23or%00in--dbcould bypass the?/#guard and leave a 0644 file under another name next to an empty 0600 decoy (#195). - Release tags must point at a commit on
main(release.yml preflight), and a pull request whose description, or one of whose commits, has aBREAKING CHANGE:(orBREAKING-CHANGE:) footer without a lowercasetype!:subject (the PR title, for the description) fails thepr-lint / breaking-changecheck, which is not yet a required check onmain(#198). - Built with current dependencies. govulncheck finds 22
reachable vulnerabilities in the v0.1.1 binary and none in this build.
A daily workflow now scans the latest published release, not only
main. - Releases come from CI only. Archives, packages and images are
reproducible from the tag.
checksums.txtand every image (the index and each platform) carry keyless cosign signatures, and there are SBOMs, SLSA provenance and build attestations. A ruleset makes release tags immutable. - An OpenSSF Scorecard workflow publishes its results.
/api/v1/gateand snapshot ingest bound their memory by the input's YAML or JSON structure and by concurrency budgets, not only by bytes. Nodes are counted before anything is decoded. YAML aliases are charged at their expanded size, and that is measured inside the single evaluation slot. UTF-16 streams are refused. A body over a budget gets 413, and one too slow for the read timeout gets 408, before any 503 (#121, #100).- Reads of stored data are bounded too. Reads that load a cluster's
snapshot run in a read slot of their own, and only a what-if decodes
the whole inventory: one 370 KB push, from any ingest token, had made
10 concurrent reads of it grow the heap ~400 MiB. The cluster list, the
fleet matrix and
/metricsread evaluation summaries instead of whole reports (after one 17 MB push, 30 concurrent requests to them had grown the heap by up to 584 MiB), and run two at a time in fleet slots of their own. Every read's response and every/gateanswer is built in its slot and waits for its client in one budget of twice--max-snapshot-bytes; one that does not fit what is left gets 503 withRetry-After, and one larger than the whole budget is sent in its slot under a 20s write deadline. Unbounded, 20 clients that asked for a 17.5 MB report and never read it had held 366 MiB, and 100 that asked for a 2000-cluster fleet's/fleet251 MiB (#121). - A
/gateanswer is bounded before it is encoded: a 60 KB?path=had made a 1.2 MB stream answer 817 MB, YAML aliases across 136 documents 532 MB, and names of<72 MB of SARIF./fleet?targets=with 8,718 minors had held a fleet slot for over two minutes. Object names of<had made a push store reports six times its size (a 323 MiB ingest); stored reports and responses no longer escape them (#121). - With notifications configured (
--slack-webhook,--webhook), each evaluation of a target decided before decoded that earlier report whole as the baseline of what changed, and a cluster's later push loaded its previous inventory to compare hashes. At the report limit with four--targets, a later push grew the heap up to ~257 MiB and a re-evaluation that changed every finding up to ~265 MiB; the memory bounds had been measured on first pushes without notifications only. A push now reads the previous snapshot's head alone, a baseline only its findings' keys and severities, and one pass's changes are merged target by target, keeping a digest of those past the notification's cap: ~216 and ~198 MiB, and the bounds are measured that way (#121). - The CSV export guards every cell against formula injection, including
after leading white space. Anonymous read access is decided on the
resolved bind address. The
Bearerscheme is case-insensitive. The SQLite database and its WAL and SHM files are created 0600. The agent warns when it would push its token over plain HTTP to a host that is not loopback (#126). - One Helm release object can no longer exhaust the agent's memory. A 951 KB
release Secret whose gzip held 700 MiB took a scan to 1.93 GB and
OOM-killed the agent at its 256Mi limit; anyone who can create a Secret or
ConfigMap in one namespace could plant one, and a valid release built to
amplify parsing (a 127 KiB Secret of tiny ConfigMaps took 564 MiB) could
too. Each release payload is now decoded within 4 MiB stored and 16 MiB
decompressed (real releases decode to under 7 MiB), and its manifest is
parsed in runs of at most 1 MiB and 64Ki YAML nodes; a single document
over 2 MiB or 64Ki nodes is not parsed. The worst releases that fit the
bounds peak at 32 to 46 MiB of live heap against a 64 MiB bound, on a
GitHub-hosted runner (CI run 37160469086). A release over a bound is
skipped (
release payload too large) or recorded without that document, named on a partialhelmcapability, and the rest are still read (#168). - The security model documents that Helm release Secrets, pod images and
labels are tenant-controlled evidence: findings are only as trustworthy
as namespace write access. A forged release can raise a finding in its
namespace; its
appVersionstands only for pods on its own release line, so it cannot hide an older image on another line there, only a patch-level difference on the same line, an untagged image, or an image no matcher recognizes (#165).
0.1.1 - 2026-07-27¶
A packaging release. Scanning, scoring, the agent, the server and the knowledge base are unchanged since 0.1.0.
Fixed¶
- The GitHub Action can install from a release. 0.1.0 published archives as
upgradescope-v0.1.0-<os>-<arch>.tar.gz, but the Action requestedupgradescope_<os>_<arch>.tar.gz. Every run fell back togo installand built from source. Archive names no longer contain the version, so/releases/latest/download/<asset>resolves to a stable filename. The Action also now extracts the binary from the tarball. - The README examples target the knowledge-base horizon (Kubernetes 1.36,
maxKnownK8sfromk8s.io/apiv0.36.1) instead of 1.38. With 1.38, the headline command's only finding was akb-stalewarning about the tool itself.
Changed¶
- CI and
make lintrungo vetplus staticcheck pinned to v0.7.0, instead of golangci-lint or an unpinnedstaticcheck@latest, so an upstream linter release cannot fail an unrelated pull request.make lintruns the pinned version throughgo runand ignores anystaticcheckonPATH. - The release archives are
upgradescope_{linux,darwin}_{amd64,arm64}.tar.gz, published withchecksums.txt.
Removed¶
- The Go Report Card badge. goreportcard.com retired the service.
- The unused
.golangci.yml.
0.1.0 - 2026-06-11¶
First release.
Added¶
upgradescope scan: a point-in-time readiness scan of a live cluster (kubeconfig) or of rendered manifests (--files). It prints a table, JSON or SARIF 2.1.0. The CI exit codes are: 2 when findings reach--fail-on, 1 on an operational error.- Detection of:
- deprecated and removed API objects, probed at the deprecated group/version endpoints with paged, metadata-only lists;
- clients still calling deprecated APIs, read from the apiserver
apiserver_requested_deprecated_apismetric; - end-of-life and soon-to-be-EOL add-ons, matched by container image and Helm chart;
- add-on and Helm chart compatibility with the target Kubernetes version;
- version skew between kubelets, HA apiservers, controller-manager, scheduler and kube-proxy;
- a knowledge base too old for the cluster or the target (
kb-stale). - A deterministic readiness score,
max(0, 100 − min(75, 25×blockers) − min(20, 5×warnings)), withready = no blockers, and per-team scores from a namespace label. - Collectors that degrade independently. A collector that is denied or broken is reported as "not assessed (reason)" instead of failing the scan.
upgradescope agent: an in-cluster loop that maintains a cluster-scopedClusterReadinessCRD (upgradescope.dev/v1alpha1, short nameucr) and optionally pushes content-deduplicated inventory snapshots to a server.upgradescope serve: snapshot ingest, SQLite or Postgres storage, score history, what-if re-evaluation (?target=), fleet rollups, thePOST /api/v1/gateCI gate, auditor CSV and HTML exports, Slack and generic-webhook notifications on finding deltas, and an embedded React dashboard (fleet matrix, cluster drill-down, registry browser).upgradescope tokens create|revoke: per-cluster ingest tokens, stored hashed.- A Helm chart (
deploy/chart) for the agent and an optional server. - A composite GitHub Action (
action/) that runsupgradescope scan --filesover a directory of rendered manifests and emits SARIF for code-scanning annotations. - A knowledge base with an API-lifecycle dataset generated from upstream
k8s.io/apisource (tools/gen-kb) and an 18-entry add-on registry. Every claim in the registry requires an upstream citation, and 10 entries are synced with endoflife.date (tools/eol-sync). A weeklykb-refreshworkflow opens a pull request when either dataset changes.
Known limitations¶
- kubectl client skew is not evaluated, because client versions are visible only in apiserver audit logs.
- Managed control planes often block apiserver
/metrics. The deprecated-callers signal is then reported as not assessed. kind: Listitems are not expanded in--filesmode.