# kubectl plugin reference > Every flag, JSON field, exit status and permission of each kubectl napwake command, and the detail behind what it prints. Source: https://docs.napwake.dev/next/reference/cli/ [scan 39](https://docs.napwake.dev/next/reference/cli/#scan) * [`--namespace`](https://docs.napwake.dev/next/reference/cli/#scan-namespace) * [`--all-namespaces`](https://docs.napwake.dev/next/reference/cli/#scan-all-namespaces) * [`--exclude-namespaces`](https://docs.napwake.dev/next/reference/cli/#scan-exclude-namespaces) * [`--output`](https://docs.napwake.dev/next/reference/cli/#scan-output) * [`--limit`](https://docs.napwake.dev/next/reference/cli/#scan-limit) * [`--cpu-threshold`](https://docs.napwake.dev/next/reference/cli/#scan-cpu-threshold) * [`--cpu-price`](https://docs.napwake.dev/next/reference/cli/#scan-cpu-price) * [`--memory-price`](https://docs.napwake.dev/next/reference/cli/#scan-memory-price) * [`--window`](https://docs.napwake.dev/next/reference/cli/#scan-window) * [`--idle-after`](https://docs.napwake.dev/next/reference/cli/#scan-idle-after) * [`--prometheus-service`](https://docs.napwake.dev/next/reference/cli/#scan-prometheus-service) * [`--prometheus-url`](https://docs.napwake.dev/next/reference/cli/#scan-prometheus-url) * [`--prometheus-selector`](https://docs.napwake.dev/next/reference/cli/#scan-prometheus-selector) * [`--prometheus-header`](https://docs.napwake.dev/next/reference/cli/#scan-prometheus-header) * [`--prometheus-bearer-token-file`](https://docs.napwake.dev/next/reference/cli/#scan-prometheus-bearer-token-file) * [`--prometheus-ca-file`](https://docs.napwake.dev/next/reference/cli/#scan-prometheus-ca-file) * [`--prometheus-insecure-skip-verify`](https://docs.napwake.dev/next/reference/cli/#scan-prometheus-insecure-skip-verify) * [`--timeout`](https://docs.napwake.dev/next/reference/cli/#scan-timeout) * [`--html`](https://docs.napwake.dev/next/reference/cli/#scan-html) * [`--open`](https://docs.napwake.dev/next/reference/cli/#scan-open) * [`nodeAutoscaler`](https://docs.napwake.dev/next/reference/cli/#scan-json-nodeautoscaler) * [`wontPause`](https://docs.napwake.dev/next/reference/cli/#scan-json-wontpause) * [`namespaces`](https://docs.napwake.dev/next/reference/cli/#scan-json-namespaces) * [`history.prometheus`](https://docs.napwake.dev/next/reference/cli/#scan-json-history-prometheus) * [`history.hours`](https://docs.napwake.dev/next/reference/cli/#scan-json-history-hours) * [`history.others`](https://docs.napwake.dev/next/reference/cli/#scan-json-history-others) * [`replicasFromGit`](https://docs.napwake.dev/next/reference/cli/#scan-json-replicasfromgit) * [`gitOpsLeavesReplicas`](https://docs.napwake.dev/next/reference/cli/#scan-json-gitopsleavesreplicas) * [`gitOpsUnchecked`](https://docs.napwake.dev/next/reference/cli/#scan-json-gitopsunchecked) * [`platform`](https://docs.napwake.dev/next/reference/cli/#scan-json-platform) * [`totals.live`](https://docs.napwake.dev/next/reference/cli/#scan-json-totals-live) * [`totals.liveProtected`](https://docs.napwake.dev/next/reference/cli/#scan-json-totals-liveprotected) * [`totals.protectedFreedThisMonth`](https://docs.napwake.dev/next/reference/cli/#scan-json-totals-protectedfreedthismonth) * [`totals.protectedSavedThisMonth`](https://docs.napwake.dev/next/reference/cli/#scan-json-totals-protectedsavedthismonth) * [`onSpot`](https://docs.napwake.dev/next/reference/cli/#scan-json-onspot) * [`spotAtOnDemand`](https://docs.napwake.dev/next/reference/cli/#scan-json-spotatondemand) * [`nodePrices`](https://docs.napwake.dev/next/reference/cli/#scan-json-nodeprices) * [`incomplete`](https://docs.napwake.dev/next/reference/cli/#scan-json-incomplete) * [`workloads`](https://docs.napwake.dev/next/reference/cli/#scan-json-workloads) [forecast 15](https://docs.napwake.dev/next/reference/cli/#forecast) * [`--namespace`](https://docs.napwake.dev/next/reference/cli/#forecast-namespace) * [`--all-namespaces`](https://docs.napwake.dev/next/reference/cli/#forecast-all-namespaces) * [`--exclude-namespaces`](https://docs.napwake.dev/next/reference/cli/#forecast-exclude-namespaces) * [`--window`](https://docs.napwake.dev/next/reference/cli/#forecast-window) * [`--forecast`](https://docs.napwake.dev/next/reference/cli/#forecast-forecast) * [`--timezone`](https://docs.napwake.dev/next/reference/cli/#forecast-timezone) * [`--cpu-threshold`](https://docs.napwake.dev/next/reference/cli/#forecast-cpu-threshold) * [`--idle-after`](https://docs.napwake.dev/next/reference/cli/#forecast-idle-after) * [`--output`](https://docs.napwake.dev/next/reference/cli/#forecast-output) * [`--timeout`](https://docs.napwake.dev/next/reference/cli/#forecast-timeout) * [`operator`](https://docs.napwake.dev/next/reference/cli/#forecast-json-operator) * [`operator.stopped`](https://docs.napwake.dev/next/reference/cli/#forecast-json-operator-stopped) * [`forecast`](https://docs.napwake.dev/next/reference/cli/#forecast-json-forecast) * [`forecast.wakes`](https://docs.napwake.dev/next/reference/cli/#forecast-json-forecast-wakes) * [`noReplay`](https://docs.napwake.dev/next/reference/cli/#forecast-json-noreplay) [status 13](https://docs.napwake.dev/next/reference/cli/#status) * [`--namespace`](https://docs.napwake.dev/next/reference/cli/#status-namespace) * [`--all-namespaces`](https://docs.napwake.dev/next/reference/cli/#status-all-namespaces) * [`--output`](https://docs.napwake.dev/next/reference/cli/#status-output) * [`--since`](https://docs.napwake.dev/next/reference/cli/#status-since) * [`--html`](https://docs.napwake.dev/next/reference/cli/#status-html) * [`--open`](https://docs.napwake.dev/next/reference/cli/#status-open) * [`--timeout`](https://docs.napwake.dev/next/reference/cli/#status-timeout) * [`forecast`](https://docs.napwake.dev/next/reference/cli/#status-json-forecast) * [`forecastStartsActing`](https://docs.napwake.dev/next/reference/cli/#status-json-forecaststartsacting) * [`turnedAway`](https://docs.napwake.dev/next/reference/cli/#status-json-turnedaway) * [`wokenBy`](https://docs.napwake.dev/next/reference/cli/#status-json-wokenby) * [`kind`](https://docs.napwake.dev/next/reference/cli/#status-json-kind) * [`target`](https://docs.napwake.dev/next/reference/cli/#status-json-target) [nodes 2](https://docs.napwake.dev/next/reference/cli/#nodes) * [`--output`](https://docs.napwake.dev/next/reference/cli/#nodes-output) * [`--timeout`](https://docs.napwake.dev/next/reference/cli/#nodes-timeout) [pause 5](https://docs.napwake.dev/next/reference/cli/#pause) * [`--namespace`](https://docs.napwake.dev/next/reference/cli/#pause-namespace) * [`--wait`](https://docs.napwake.dev/next/reference/cli/#pause-wait) * [`--yes`](https://docs.napwake.dev/next/reference/cli/#pause-yes) * [`--clear-hold`](https://docs.napwake.dev/next/reference/cli/#pause-clear-hold) * [`--timeout`](https://docs.napwake.dev/next/reference/cli/#pause-timeout) [wake 4](https://docs.napwake.dev/next/reference/cli/#wake) * [`--namespace`](https://docs.napwake.dev/next/reference/cli/#wake-namespace) * [`--for`](https://docs.napwake.dev/next/reference/cli/#wake-for) * [`--wait`](https://docs.napwake.dev/next/reference/cli/#wake-wait) * [`--timeout`](https://docs.napwake.dev/next/reference/cli/#wake-timeout) [deps 2](https://docs.napwake.dev/next/reference/cli/#deps) * [`--namespace`](https://docs.napwake.dev/next/reference/cli/#deps-namespace) * [`--timeout`](https://docs.napwake.dev/next/reference/cli/#deps-timeout) [enable 4](https://docs.napwake.dev/next/reference/cli/#enable) * [`--namespace`](https://docs.napwake.dev/next/reference/cli/#enable-namespace) * [`--all`](https://docs.napwake.dev/next/reference/cli/#enable-all) * [`--force`](https://docs.napwake.dev/next/reference/cli/#enable-force) * [`--timeout`](https://docs.napwake.dev/next/reference/cli/#enable-timeout) ## Every command The request limit and the connection flags every command shares. [Namespaces and connections](https://docs.napwake.dev/next/getting-started/kubectl-plugin/#namespaces-and-connections) says which namespaces each command reads. Each request to the API server gives up after 30 seconds. So a cluster that stops answering ends a command with an error. `--request-timeout` changes the limit. Its `0` keeps 30 seconds, where kubectl’s means no limit. Every command takes kubectl’s connection flags: `--kubeconfig`, `--context`, `--cluster`, `--user`, `--as`, `--as-group`, `--as-uid`, `--token`, `--server`, `--certificate-authority`, `--client-certificate`, `--client-key`, `--insecure-skip-tls-verify`, `--tls-server-name`, `--proxy-url`, `--username`, `--password`, `--disable-compression` and `--request-timeout`. ## `scan` Its flags, with their defaults. [Scan a cluster](https://docs.napwake.dev/next/getting-started/kubectl-plugin/#scan-a-cluster) shows it in use. | Flag | Default | What it does | | ----------------------------------- | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | `--namespace` `-n` | all you can read | Namespace to scan. Repeat for several. | | `--all-namespaces` `-A` | | Every namespace, failing rather than falling back. Not with `-n`. | | `--exclude-namespaces` | `kube-system`, `kube-public`, `kube-node-lease` | Namespaces to skip | | `--output` `-o` | `table` | `table`, `json` or `yaml` | | `--limit` | `25` | Rows in the table, idle first, then paused, then active, each by what pausing could free | | `--cpu-threshold` | `10` | CPU use, as a percentage of requests from 0.1 to 100, such as `10` or `0.5`, above which a workload is active | | `--cpu-price` | `0.031` | Your price per vCPU-hour, in dollars, zero or more | | `--memory-price` | `0.004` | Your price per GiB-hour of memory, in dollars, zero or more | | `--window` | `7d` | Prometheus history to replay, such as `7d` or `36h`, up to `366d`. `0` judges from CPU now. | | `--idle-after` | `1h` | Time without activity that makes a workload idle, as Napwake’s `idleAfter` | | `--prometheus-service` | the one in the cluster with the most history | Prometheus Service in the cluster, as `namespace/name` or `namespace/name:port`, read through the API server | | `--prometheus-url` | the one in the cluster with the most history | Prometheus API to read, such as Thanos, Mimir, or Google Cloud Managed Service for Prometheus | | `--prometheus-selector` | | Label matchers added to every history query, such as `'cluster="prod"'`, for a Prometheus of several clusters | | `--prometheus-header` | | Header for `--prometheus-url`, as `"Name: value"`, such as `"X-Scope-OrgID: tenant"` for Mimir. Repeat for several. | | `--prometheus-bearer-token-file` | | File holding a bearer token for `--prometheus-url` | | `--prometheus-ca-file` | | PEM file of CA certificates to trust for `--prometheus-url`, besides the system’s | | `--prometheus-insecure-skip-verify` | `false` | Don’t verify `--prometheus-url`’s certificate | | `--timeout` | `5m` | How long the scan may take before it gives up | | `--html` | a temporary file | Save the web report to this file, to share | | `--open` | `true` | Open the web report, when the table is shown in a terminal | ### More on the flags `--limit 0` lists every workload. `--limit` also caps the dependency list, and JSON and YAML list everything. `--cpu-threshold` and `--idle-after` apply now and in the replay. Workloads Napwake manages use their own settings. JSON and YAML give each workload’s as `idleAfter`. `--prometheus-selector` values are double-quoted, as in PromQL. ### More on the first line and the columns JSON and YAML keep the full context name. Their `namespaces` counts every namespace read. `Your bill:`, under the first line, says what turns what pausing frees into savings. It’s from the node autoscaler [`nodes`](https://docs.napwake.dev/next/reference/cli/#nodes-autoscaler) finds (JSON `nodeAutoscaler`). | The node autoscaler | `Your bill:` says | | -------------------------------------------------- | ----------------------------------------------------------------------------------------- | | Karpenter, Cluster Autoscaler or EKS Auto Mode | It `drops only as` it `removes nodes` | | Karpenter | The same, with how its NodePools consolidate | | Cluster Autoscaler, hinted at only by a Deployment | The same, and that it may not be running | | GKE Autopilot, which bills what pods request | It `drops as pods stop` | | None found | It `won't drop until something removes nodes`, with the guide to setting an autoscaler up | | Your access doesn’t allow reading what would tell | It says so, and names what it couldn’t read | Before Napwake runs, the scan can’t know what of COULD FREE/MO would be saved. So it never calls it a saving. Each money and history cell has the one source the table gives. A live workload isn’t replayed, as there are no pods for Prometheus to record while it’s paused. Its pauses are what FREED THIS MONTH and SAVED count. A StatefulSet that deletes its PVCs when scaled down would lose its data to a pause. So Napwake refuses to pause it. It’s left out of what pausing could free and of the next steps. A protected workload is left out of what pausing could free and the next steps. The scan sees only the label, not the operator’s `protectedNamespaces` patterns. A workload scaled to zero by hand is counted apart in the headline. It counts toward nothing the scan says could be freed. ### How it decides a workload is idle It uses the signals Napwake uses to pause. | Workload | How it’s judged | | ----------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Managed by Napwake | By the activity clock Napwake keeps for it. Idle is no activity until the clock’s `pauseAt`. That’s `idleAfter` after the last activity, or longer after [pauses made in use](https://docs.napwake.dev/next/concepts/idle-detection/#paused-while-in-use). | | Managed, after pauses made in use | BECAUSE says so, such as `not idle yet: it woke 6m ago; after pauses made in use it waits 20 minutes, not 5 minutes` | | Managed, its activity clock | Combines every source configured, Prometheus queries too | | Managed, held awake | The scan says so, such as for its dependents | | Managed, its clock started without activity | BECAUSE says what started it | | Started by a wake | `not idle yet: it woke 3m ago` | | Started as Napwake took it on | `not idle yet: Napwake began managing it 3m ago` | | Started by a [gap in watching](https://docs.napwake.dev/next/concepts/idle-detection/#restarts-and-outages) | `not idle yet: Napwake wasn't watching until 3m ago` | | Any other, active | CPU above `--cpu-threshold` (10% by default) of what it requests | | Any other, active | A rollout within `--idle-after` (1h by default) | | Any other, active | A `napwake.dev/last-activity` annotation within `--idle-after` | | Any other, active | A `napwake.dev/active-until` in the future | | Any other, idle | None of those, with its CPU measured | | Unmanaged, without metrics-server | As on EKS by default. Judged from its CPU history, and BECAUSE says so. | | Its state then | Where the replay left the activity clock | | No CPU requests, or no metrics yet | Can’t be judged. It’s in the notes, not the table, and out of the headline’s totals. | ### What the notes list, and what the scan leaves out | In the notes | What it means | | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | Replicas set from Git | A GitOps tool last set the workload’s replicas, by the field managers the API server records | | Which tools | Argo CD, Flux, Fleet, kapp or Kluctl. Workloads are listed by tool. | | Why it matters | The tool would undo every pause until told, once, to leave replicas to Napwake. See [GitOps tools](https://docs.napwake.dev/next/guides/gitops/). | | Once the tool is told | The workload isn’t listed | | For Argo CD | The scan reads that from the Application and `argocd-cm`, as [`enable`](https://docs.napwake.dev/next/reference/cli/#enable) does | | Without access to them | The note says so, and lists the workload to check | | In JSON | The tool in `replicasFromGit`, `gitOpsLeavesReplicas` once told, and `gitOpsUnchecked` for why it couldn’t check | | In the next steps | Never suggested, as the tool would revert a label put on by hand too | | The cluster’s own infrastructure | Left out of the table, the totals and the next steps. A note counts it by namespace. | | Infrastructure in JSON and YAML | Listed in `platform`, each with what it is | | Left out | Found by | | ---------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | Kubernetes’ namespaces, never paused | `kube-*`, `openshift-*`, `cattle-*`, `gke-*` and `gmp-*` | | Napwake’s namespace, never paused | Its Deployment’s `app.kubernetes.io/name: napwake` label | | The operator’s and the doorman’s Deployments, never paused | `control-plane: controller-manager` or `control-plane: doorman`, in any namespace | | The same, also needed | `app.kubernetes.io/name: napwake`, or a `napwake` `helm.sh/chart` label | | What the scan guesses nobody means to pause | Well-known kinds, such as ingress controllers, cert-manager, Vault, Argo CD, Flux, service meshes, policy engines and monitoring | | The same, by namespace | `ingress-nginx`, `cert-manager`, `vault`, `argocd`, `flux-system`, `istio-system`, `linkerd`, `gatekeeper-system`, `kyverno`, `monitoring` and others | | The same, by label | `app.kubernetes.io/name`, `app.kubernetes.io/part-of`, `app`, `k8s-app` or `helm.sh/chart` | | Workloads labelled `napwake.dev/ignore=true` | They aren’t listed at all | Napwake never needs to wake what it never pauses, so apps’ dependencies on those aren’t listed. The guessed infrastructure is left out of the estimate only, and the operator has no such rule. Opted in, Napwake pauses it like any app, so dependencies on it are listed. There’s no guessing for a workload Napwake manages. Nor in a namespace labelled `napwake.dev/managed=true`, such as a preview with its own Vault. A namespace labelled `napwake.dev/allow-protected=true` is scanned like any other, apart from Napwake’s own Deployments. ### History from Prometheus With a Prometheus in the cluster, the scan replays each workload’s last week of CPU and rollouts. It runs Napwake’s activity clock over them, as if managing it with `--idle-after` and `--cpu-threshold`. | | How it works | | ---------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | Finding it | By the Services the Prometheus Operator (so kube-prometheus-stack) and the community Helm chart create | | Reaching it | Through the API server’s service proxy, so there’s no port-forward or URL to give | | Naming it | The `History:` line and the report’s subtitle (JSON `history.prometheus`) | | How much was replayed | On the same line and subtitle (JSON `history.hours`) | | Several | It reads the one with the most of the window’s CPU history, ranked as below | | How many | It tries five at most. The `History:` line lists the others (JSON `history.others`). | | `--prometheus-service namespace/name` | Reads that Service instead, found or not, through the API server | | Its port | Its API port, or the one you add, as in `monitoring/thanos-query:http` | | `--prometheus-url` | Reads one outside the cluster, such as Thanos or Mimir. See [A Prometheus elsewhere](https://docs.napwake.dev/next/reference/cli/#a-prometheus-elsewhere). | | Either, unreadable | The scan fails, rather than going on without history | | CPU | Per container, from cAdvisor’s `container_cpu_usage_seconds_total`, which every common setup scrapes | | Steps | Five minutes, coarser for windows over about 34 days | | Pods | Matched to their workload by name | | A Deployment’s pods | Those of deleted ReplicaSets too, named `--` | | A long Deployment name | If Kubernetes cut its pods’ names short, only current ReplicaSets’ pods are replayed | | Sidecars | Those added at pod creation are left out of activity, as for the activity clock | | An active step | A rollout, or the workload’s own CPU above the threshold of its running pods’ requests | | After `--idle-after` with none | It sleeps until the next active step, which is one wake | | CPU within 15 minutes of a pause, or within `--idle-after` if that’s shorter | Shows it was in use | | Each such pause in a row | Doubles the next wait, up to `--idle-after` plus an hour, as Napwake does after [a pause made in use](https://docs.napwake.dev/next/concepts/idle-detection/#paused-while-in-use) | | A pause that lasts past that window | Ends the longer wait | | A rollout | Wakes it, without counting either way | | What it can’t see | Requests and activity annotations, which aren’t in Prometheus. It sees only CPU and rollouts. | | So | A workload used with little CPU can look like it would sleep more than it would | | Requests and prices | Today’s | The active-step threshold is as the operator counts it. | Rank | Preferred | | ---- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | cAdvisor’s container CPU from the window’s start, then any CPU, then any that answers | | 2 | A monitoring namespace, in order `monitoring`, `prometheus`, `observability`, `kube-prometheus-stack`, `cattle-monitoring-system`, `openshift-monitoring` | | 3 | A usual name, in order `prometheus-operated`, `kube-prometheus-stack-prometheus`, `prometheus-server`, `prometheus`, `prometheus-k8s` | | 4 | The Prometheus Operator’s Service over the community chart’s | | 5 | The first by namespace and name | | Time and short history | What happens | | ------------------------------------------- | --------------------------------------------------------------------------------------------------------- | | History’s share | The first 60% of `--timeout`, so a slow Prometheus or Thanos can’t use up the scan | | A namespace’s reads | Its API reads come before its history query, sharing that time | | Once it’s spent | A namespace reached after that isn’t queried | | A slow query | It may take whatever is left, however long. `--request-timeout` doesn’t cut it short. | | Not read in time | Judged from CPU right now. The notes name these namespaces. | | The notes’ advice | A longer `--timeout` gives history more time, or `--window 0` skips it | | A query given up with time left | By Prometheus or the API server’s proxy. The notes name the namespaces, and suggest a shorter `--window`. | | Less history than the window | The replay covers what there is, and says how much, such as “Replaying the last 30 minutes” | | The same, in the notes | A note says it’s shorter than asked | | Under a day | It can’t show a day’s pattern. The headline says so, in the terminal and the report. | | The headline’s advice | Scan again after a few days | | No Prometheus, or no permission to reach it | The scan judges from CPU right now, and says why | A longer `--timeout` wouldn’t help with a query Prometheus gave up on by itself, which is why the advice differs. ### A Prometheus elsewhere `--prometheus-url` reads a Prometheus API your machine can reach, rather than one in the cluster. It needs no cluster permission at all. If it doesn’t answer like a Prometheus API, the scan fails rather than going on without history. | Store or need | What to do | | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------- | | A store of several clusters, such as Thanos or Mimir | Pass `--prometheus-selector` with the label that tells them apart | | The notes’ hint | They suggest it when they see series from more than one source | | Authentication | `--prometheus-header`, such as `X-Scope-OrgID` for a Mimir tenant, or `Authorization` | | Or | `--prometheus-bearer-token-file`. Not both for one header. | | Its certificate | `--prometheus-ca-file` and `--prometheus-insecure-skip-verify` | | A Prometheus in the cluster | These flags don’t apply. It’s reached through the API server, with your kubeconfig. | | Google Cloud Managed Service for Prometheus | GKE’s keeps its data in Cloud Monitoring, so the scan doesn’t find it | | Its collectors | In `gmp-system`, with nothing to query. The scan reads another Prometheus if the cluster has one. | | To read it | Give it an access token, and name your project and cluster, as below | | Its frontend, deployed in the cluster | `--prometheus-service NAMESPACE/frontend` reads through that, with the frontend’s credentials | | Either way | The replay’s CPU is there only if it collects cAdvisor’s metrics | | Turning those on | `kubeletScraping` in its OperatorConfig | | Amazon Managed Service for Prometheus | It needs SigV4 signing, which the scan doesn’t do. Point `--prometheus-url` at a signing proxy. | Name the cluster too, since the store holds every cluster in the project. ```bash kubectl napwake scan \ --prometheus-url https://monitoring.googleapis.com/v1/projects/PROJECT_ID/location/global/prometheus \ --prometheus-bearer-token-file <(gcloud auth print-access-token) \ --prometheus-selector 'cluster="CLUSTER_NAME"' ``` ### The headline’s counts | Count | What it counts | | ---------------------------------------------- | ----------------------------------------------------------------------------------- | | `pausing 14 live workloads` | Only live workloads Napwake pauses | | Its money | What they freed, and what any put back in dry-run freed while live | | With none live | It says the money came from workloads since put back in dry-run, if there are some | | Put back in dry-run | A workload keeps what it freed and saved while live | | Would-be pauses | Never add to what’s freed or saved | | Protected live workloads | A line of their own, with what they freed this month before protection, if anything | | `0 of 13 unmanaged workloads would have slept` | Unmanaged workloads replayed | | `Not counted:` | Listed workloads the replay count leaves out, and why, as the report’s summary does | | Not counted, won’t pause | `1 that won't pause (PVCs deleted on scale-down)` | | Not counted, the others | Those in protected namespaces, scaled to zero by hand, or without CPU history | The protected line reads like this. Its workloads are in a protected namespace, part of Napwake, or StatefulSets deleting PVCs on scale-down (`Protected=True`). ```plaintext 1 more live workload is protected, so Napwake never pauses it; it freed $12 this month before it was, and saved $5 of that. ``` JSON gives `totals.live` and `totals.liveProtected`. It also gives `totals.protectedFreedThisMonth` and `protectedSavedThisMonth`, which are part of `freedThisMonth` and `savedThisMonth`. ### Dependencies The scan lists the workloads each workload’s environment variables point at, literal or from ConfigMaps. ``` 3 dependencies found, and what Napwake does with each: preview-42 deployment/checkout-api -> statefulset/postgres DATABASE_URL declared; woken and held with it preview-42 deployment/checkout-api -> messaging/statefulset/nats NATS_URL not managed, so never paused preview-42 deployment/worker -> statefulset/postgres PGHOST woken with it once both are managed ``` | | How it reads them | | --------------- | --------------------------------------------------------------------------------------------------------------------------------- | | An address | It names a Service in a scanned namespace, in any form cluster DNS gives it | | Its forms | `postgres`, `postgres.preview-42`, `postgres.preview-42.svc.cluster.local`, or a headless Service’s pod, `postgres-0.postgres-hl` | | Where | In a URL, a `host:port`, or a list of either | | The workload | The one the Service’s selector picks | | Not an address | A bare word on its own, such as `MODE=postgres` | | Names | Variable names don’t matter. Service names come from the cluster, not a list the scan keeps. | | Shown | Only its scheme, host and port, such as `postgres://postgres:5432` | | Hidden | A user, password, path and query string, any of which could hold a credential | | In the terminal | The variable it was found in. Only JSON and YAML carry the address. | | Secrets | Never read, so addresses there aren’t found. The notes name workloads that take variables from Secrets. | | Code and files | Addresses built in code or read from files aren’t found either | When Napwake manages both, it wakes a dependency with the workload that needs it. It keeps it up while that workload is. So the first request doesn’t wait for it too. See [Dependencies](https://docs.napwake.dev/next/concepts/dependencies/). | Last column | When | | ------------------------------------- | ------------------------------------------------------------ | | `woken with it once both are managed` | Napwake doesn’t manage the workload yet | | `not managed, so never paused` | Napwake doesn’t manage the dependency, which stays up anyway | | `woken and held with it` | Learned from the workload’s environment, or a wake | | `declared; woken and held with it` | Declared with `napwake.dev/depends-on` | | `not learned yet` | Napwake manages it, but hasn’t learned it | Before Napwake manages any workload that needs them, every dependency would say the first. So the scan and the report say it once above the list, and leave the column out. ### Prices | Case | Price | | -------------------------------------- | -------------------------------------------------------------------------------------------------- | | Any workload | Its requests while running, sidecars included. See [Prices](https://docs.napwake.dev/next/concepts/cost-tracking/#prices). | | The rate | The list price of each pod’s node, by instance type and region | | No pod on a node | Where Napwake last saw it run, or the cluster’s most common node type | | A node with no list price | Assumed prices, AWS on-demand in us-east-1. The report says which. | | No access to list nodes | Every workload at those assumed prices | | `--cpu-price` and `--memory-price` | Yours, in place of list prices, for discounts, savings plans or negotiated rates | | Spot and preemptible nodes | Their type’s spot price. The report says how many workloads run on them. | | A spot node of a type not sold as spot | On-demand. The report names those types, and says they cost less than shown. | | In JSON, per workload | `onSpot` and `spotAtOnDemand` | | In JSON, per node type | `spot` and `atSpot`, in `nodePrices`. See [Spot nodes](https://docs.napwake.dev/next/concepts/cost-tracking/#spot-nodes). | | An hour asleep | Frees capacity. It’s money only when a node autoscaler removes it. | `kubectl napwake nodes` shows what keeps nodes from scaling down. The `Prices:` line at the end sums up what costs were priced at: | Priced at | The line gives | | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | | List prices | Their provider and region, the node type or how many there are, and how many workloads are on spot, priced at spot | | Your prices | `yours` | | Assumed prices, when no node has a list price | `assumed` | | Too mixed for that | Each in full | | Too mixed, such as | Some nodes priced and others not, the nodes unread, or one price yours and the other assumed | ### The web report It’s written for whoever you send it to. | Part | What it shows | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- | | Top | The cluster drawn as a hotel at night, one room per workload up to 35. Past that, the mix is rounded to 35 rooms, with the exact counts beside it | | On a scan | The idle workloads’ lit rooms go dark by themselves once, to show what Napwake would do. Nothing is paused | | Then | **Prices** and **Your bill**, as the terminal’s lines say them | | A share card | Your browser draws it, wide or square, with totals only. The cluster’s name is on it only if you tick it | | Buttons | **Copy summary**, three plain lines without the cluster’s name or the Prometheus source unless you tick it, and **Save as PDF** | | What it sends | Nothing, anywhere | | Headline figures | Two, below | | Fact one | How many workloads, in how many namespaces, with their cost once that’s left the headline | | Fact two | What’s idle right now | | Fact three, with history | How many unmanaged workloads would have slept, and listed ones the replay doesn’t count | | Fact three, without | What idle workloads cost an hour | | Fact four | How many Napwake manages | | Tables | A table by namespace, with a filter | | Workloads | Every one the scan could judge, in the terminal’s columns, sortable and with a filter | | Then | The dependencies found | | End | What the numbers mean, in plain words, and the next steps | | Headline figures | First | Beside it | | ---------------------------------- | ------------------------------------------------------------------------ | --------------------------------------------------- | | Before Napwake has paused anything | What pausing could free a month, as a share of what those workloads cost | What the workloads cost a month | | Once it has | What it freed this month, with what of it was saved | What pausing could free, while there’s something to | | A zero | Says why | | Before anything is paused, the first figure is measured for dry-run and estimated from history for unmanaged workloads. It says “over the same time” when history was replayed. Then it’s measured against what they cost over the history, with the pods they ran. Without either, it’s what’s idle now costs while running, the most pausing could free. A zero with under a day of history, short of the window, says that’s too little history yet. Scan again after a few days. | The file | | | ------------------ | --------------------------------------------------------------------------------- | | Loads | No scripts, fonts or images. Sorting and filtering are built in. | | Leaves out | Dependency addresses, as it’s meant to be passed around. JSON and YAML keep them. | | Hides | Any password in the history source’s URL | | Access | Readable only by you | | Where | Your temporary directory, unless you pass `--html FILE` | | Opens | Only when the table is shown in a terminal on a machine with a desktop | | No report | Piped output, `-o json`, CI and SSH, unless you pass `--html` | | SSH with a display | It opens when X forwarding gives a Linux machine a display | | `--open=false` | Keeps it from opening | ### JSON and YAML The JSON and YAML fields this page names, and what each holds. | Field | What it holds | | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `nodeAutoscaler` | What would turn freed capacity into savings, the `Your bill:` line | | `wontPause` | A StatefulSet whose `persistentVolumeClaimRetentionPolicy.whenScaled` is `Delete`. BECAUSE says `won't pause: PVCs deleted on scale-down`. | | `namespaces` | Every namespace read. The first line counts only the namespaces listed workloads are in. | | `history.prometheus` | The Prometheus read, named on the `History:` line and the report’s subtitle | | `history.hours` | How much was replayed | | `history.others` | The others found, which the `History:` line lists | | `replicasFromGit` | The GitOps tool that last set the workload’s replicas | | `gitOpsLeavesReplicas` | Once the tool is told to leave replicas to Napwake | | `gitOpsUnchecked` | Why it couldn’t check | | `platform` | The cluster’s own infrastructure, each with what it is | | `totals.live` | Live workloads Napwake pauses, as the headline counts them | | `totals.liveProtected` | Protected live workloads, which the headline gives a line of their own | | `totals.protectedFreedThisMonth` | What those freed this month before protection. Part of `freedThisMonth`. | | `totals.protectedSavedThisMonth` | What of that was saved. Part of `savedThisMonth`. | | `onSpot` | Per workload, on spot or preemptible nodes, priced at its type’s spot price | | `spotAtOnDemand` | Per workload, on a spot node of a type not sold as spot, priced on-demand | | `nodePrices` | Per node type, with `spot` and `atSpot`. See [Spot nodes](https://docs.napwake.dev/next/concepts/cost-tracking/#spot-nodes). | | `incomplete` | The namespaces, by why, as `missing`, `denied`, `failed` or `unscanned`. See [Exit status](https://docs.napwake.dev/next/reference/cli/#exit-status). | | `workloads` | `[]` when there are none | ### Exit status | Exit | When | | ----------------------------- | -------------------------------------------------------------------- | | `0` | It read everything your access allows | | `0`, with a `403` | A namespace or resource your access denies is explained in the notes | | `1`, after writing the report | It missed something it should have read | | `1`, causes | A timeout, throttling, a server error, or a failed Prometheus query | | `1`, named namespaces | A `403` or missing namespace among those you named with `-n` | | `1`, no report | Only a scan that ran out of `--timeout` having scanned none | A `403` changes the exit status only for a namespace you named with `-n`. On `1`, the notes say what was missed. The error says to check a namespace’s name, ask for access, or retry with a longer `--timeout`. Out of `--timeout`, the scan still reports the namespaces it scanned in time, and lists the rest as `unscanned`. | | | | ---------------------------------- | --------------------------------------------------------------------------------------- | | JSON and YAML | The namespaces in `incomplete`, by why, as `missing`, `denied`, `failed` or `unscanned` | | No workloads | JSON and YAML give `"workloads": []` | | Every namespace | It lists each kind cluster-wide, a page at a time | | So | Thousands of namespaces take a few dozen requests | | Listing allowed only per namespace | That kind is read namespace by namespace | ### Permissions | Your user needs | Where, or why | | ------------------------------------------------------------------------------------------- | ----------------------------------------- | | `list` on Deployments, StatefulSets, ReplicaSets, ControllerRevisions, Pods and pod metrics | The scanned namespaces | | `list` on Services and ManagedWorkloads | The scanned namespaces | | To read the namespaces, and the ConfigMaps workloads take variables from | The scanned namespaces | | `list` on namespaces | Unless you name them with `-n` | | `list` on nodes | To price workloads at their nodes | | What [`nodes`](https://docs.napwake.dev/next/reference/cli/#nodes-permissions) reads | To name the node autoscaler | | `get` on `services/proxy` for the Prometheus Service | To read history through the service proxy | | `get` on a Service you name with `--prometheus-service` | To find its port | The standard `view` role covers the namespaced reads, ManagedWorkloads too, as Napwake’s chart and manifests add them to it. It doesn’t cover nodes, which are cluster-wide, or `services/proxy`. Without a read, the scan carries on, and the notes say what it couldn’t see. Without `services/proxy`, it judges from CPU right now. It prints what an admin can run for that one Service and nothing else. ``` To replay history, an admin can let you read Prometheus, and nothing else, with: kubectl create role napwake-scan -n monitoring --verb=get --resource=services/proxy --resource-name=prometheus-operated:web kubectl create rolebinding napwake-scan -n monitoring --role=napwake-scan --user=jane@example.com Or pass --prometheus-url if Prometheus is reachable from your machine. ``` A Prometheus you name with `--prometheus-service` fails the scan instead, with the same commands in its error. `view` covers its `get`. All this is only when the API server refuses you the proxy. A `403` from behind it, such as a `kube-rbac-proxy` in front of Prometheus, shows as Prometheus’s own answer. A Role for `services/proxy` wouldn’t change it. ## `forecast` Its flags, with their defaults. [Replay the forecast](https://docs.napwake.dev/next/getting-started/kubectl-plugin/#replay-the-forecast) shows it in use. | Flag | Default | What it does | | ------------------------------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------- | | `--namespace` `-n` | all you can read | Namespace to replay. Repeat for several. | | `--all-namespaces` `-A` | `false` | Every namespace, failing rather than falling back | | `--exclude-namespaces` | system namespaces | Namespaces to skip | | `--window` | `28d` | History to read, from `2d` to `34d`. The operator learns up to four weeks. | | `--forecast` | `balanced` | How soon the forecast acts for unmanaged workloads. See [Forecast setting](https://docs.napwake.dev/next/concepts/forecasting/#forecast-setting). | | `--timezone` | the operator’s | The zone whose hours and weekdays the forecast learns, and times show in | | `--cpu-threshold` | `10` | CPU use, as a percentage of requests from 0.1 to 100, such as `10` or `0.5`, above which a workload is in use | | `--idle-after` | `1h` | How long without use before a pause | | `--output` `-o` | `table` | `table`, `json` or `yaml`. JSON and YAML give every workload’s days. | | `--timeout` | `5m` | How long the replay may take before it gives up | | `--prometheus-url` and the other `--prometheus-*` flags | | As for [`scan`](https://docs.napwake.dev/next/reference/cli/#a-prometheus-elsewhere) | `--forecast` is `eager`, `balanced` or `cautious`. `--timezone` defaults to the operator’s `timezone`, from `status.prediction.timeZone`, or `UTC` if you can read none. Managed workloads use their own `--cpu-threshold` and `--idle-after`. ### History, and what would wake a workload | | | | --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Prometheus | Read as [`scan`](https://docs.napwake.dev/next/reference/cli/#scan-history) reads it, with the same flags and [permissions](https://docs.napwake.dev/next/reference/cli/#permissions) | | No history | It fails | | Under two days | A day to learn and a day to replay. It says so, and when there will be enough. | | The same, managed | It says how to see the workload’s own forecast, which needs no history | | Left out | Scan’s notes on dependencies, GitOps tools and infrastructure, which are about what pausing could free | | A managed workload’s routing | What its operator found at its last pause, including Services served by other workloads’ pods | | An unmanaged workload’s routing | Judged by the ManagedWorkload that opting in would make | | Its `napwake.dev/wake` | From its own annotations, then its namespace’s, then the operator’s `--default-wake`, as opting in does. A value that can’t be read asks | | One that asks before waking | `people` in WAKES ON, where the doorman would route its Services, and otherwise `nothing` or `query`. Only people wake it in the replay, and the forecast learns its paused hours from whether they did. `noRequest` says `the doorman asks before waking it, and only a person's click does` | | `doorman.enabled` and the NEG forwarder | From the flags of the operator managing the namespace | | That operator | `forecast` finds its Deployment by labels, across the cluster | | Without access to Deployments there | It looks in the operator’s namespace, `napwake-system` by default | | Or | The namespace a ManagedWorkload’s `napwake.dev/operator` annotation names | | No operator flags | It assumes the defaults, the doorman on, the forwarder off, and `--default-wake=request` | | Then | Its notes say so, with why, and the access that reads it | | In JSON | `forecast.wakes`, with `request`, `query`, `people`, and `noRequest`, why a request wouldn’t wake it | ### Finding the workload, gaps, and exit status | Case | What it does | | -------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | NAME | Looked for in every namespace `forecast` reads, or those given with `-n` | | What it matches | A Deployment or StatefulSet, or a ManagedWorkload’s own name, for one written under another name | | Several matches | Such as `api` in two namespaces. `forecast` names them, and asks for `-n` or `kind/name`. | | No match | It says so | | No match, in a `-n` namespace | It names a workload labelled `napwake.dev/ignore=true`, which the forecast leaves out, as Napwake does | | No forecast recorded yet | It says so, and what to check | | Napwake leaves it alone | It says that instead, as `status` does in NEXT (JSON `operator.stopped`) | | Left alone means | `TargetAvailable=False` on its ManagedWorkload, a `napwake.dev/ignore=true` label, or gone | | No replay | A line says why, and what to do | | No replay, too little history | A day to learn and a day to replay, with when there will be enough | | No replay, no Prometheus | None to read, or one it couldn’t | | No replay, the workload | No CPU requests to measure use against, no CPU history of its pods, or scaled to zero outside Napwake | | A managed workload | It shows its own forecast whatever its history, and exits 0 | | An unmanaged one, nothing else to show | Exits 1 when there’s no Prometheus it can read | | An unmanaged one, too little history | It says so, and exits 0 | | Times | In the operator’s time zone, as the ManagedWorkload records it, unless `--timezone` says otherwise | | JSON and YAML | Each workload’s `operator`, `forecast` and `noReplay` | | `operator` | `learned` (`hours`, `fromHistory`, `hoursOfDay`, `hoursOfWeek`) and `busy`, each span’s `from`, `to`, `peak` | | Access | Reading the ManagedWorkload takes `list` on `managedworkloads`, which `view` has | | Without it | The workload shows as one Napwake doesn’t manage | ### JSON and YAML The JSON and YAML fields this page names, and what each holds. | Field | What it holds | | ------------------ | ------------------------------------------------------------------------------------------------------------ | | `operator` | `learned` (`hours`, `fromHistory`, `hoursOfDay`, `hoursOfWeek`) and `busy`, each span’s `from`, `to`, `peak` | | `operator.stopped` | Napwake leaves it alone, as `status` says in NEXT | | `forecast` | Each workload’s replay | | `forecast.wakes` | What would wake it, with `request`, `query`, `people`, and `noRequest`, why a request wouldn’t wake it | | `noReplay` | Why there’s no replay | ## `status` Its flags, with their defaults. [See what Napwake is doing](https://docs.napwake.dev/next/getting-started/kubectl-plugin/#see-what-napwake-is-doing) shows it in use. | Flag | Default | What it does | | ----------------------- | ---------------- | ------------------------------------------------------------------------------------- | | `--namespace` `-n` | all you can read | Namespace to show. Repeat for several. | | `--all-namespaces` `-A` | | Every namespace, failing rather than falling back. Not with `-n`. | | `--output` `-o` | `table` | `table`, `json` or `yaml` | | `--since` | `24h` | How far back to list pauses and wakes | | `--html` | | Also write the status as a web page to this file, or without one, to a temporary file | | `--open` | `true` | Open the `--html` page, when run in a terminal | | `--timeout` | `1m` | How long to wait for the cluster before giving up | ### Needs attention | Needs attention shows | Shown as | | --------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | | A GitOps tool undoing pauses | `GitOpsConflict` | | A dependency that’s missing, in a cycle, or in a namespace that doesn’t allow it | `DependencyNotAllowed` | | Two ManagedWorkloads for one target | | | A protected namespace | | | A target that’s gone | | | Metrics or Prometheus that can’t be read | | | A Prometheus query that has never matched a series | `PrometheusQueryEmpty` | | A workload whose requests can’t wake it while it’s paused | `WakeOnRequest` | | A resume that stopped with replicas not Ready | `ResumeIncomplete` | | A pause that stopped waiting with pods still running | `PauseIncomplete` | | A pause made while it was in use, until a pause holds, with what to change, such as a CPU threshold | `PausedWhileInUse` ([Paused while in use](https://docs.napwake.dev/next/concepts/idle-detection/#paused-while-in-use)) | | A pause or wake taking over 10 minutes, with what it waits for | `Stuck` | For an awake workload, `WakeOnRequest` is what its last pause found with its Services, to fix before the next. Its being awake (`NotPaused`) isn’t shown, nor the doorman being turned off. In `-o json`, each problem’s `target` names its workload as the table does, such as `deployment/worker`. ### The LAST ACTIVITY column | LAST ACTIVITY, none since its clock started | What started the clock | | --------------------------------------------- | ------------------------------------------------------------------------ | | `none since Napwake began managing it 3m ago` | Napwake taking it on | | `none since it woke 3m ago` | A wake | | `unobserved until 3m ago` | A [gap in watching](https://docs.napwake.dev/next/concepts/idle-detection/#restarts-and-outages) | ### The NEXT column | NEXT | What it means | | ---------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | | `pauses in 45m` | When it pauses | | `pauses now` | It’s `Idle`, with or without an `idlePolicy`, as after a pause request | | `would pause …` | The same, in dry-run | | `pauses now, early: the forecast expects quiet until Fri 20:00 UTC` | The forecast [brought the pause forward](https://docs.napwake.dev/next/concepts/forecasting/#pausing-early) | | `expected`, for `expects` | That time has passed, as for a dry-run would-be pause, which stays `Idle` until activity | | `pauses at 14:20 UTC (after GitOps conflict)` | A GitOps tool undid its last pause, so Napwake waits an hour. See [GitOps tools](https://docs.napwake.dev/next/guides/gitops/#what-napwake-does). | | `pauses soon` | Still running up to 7 minutes past the time shown, as status is written every few minutes | | `overdue to pause` | After that. Something the table can’t see holds it. | | `held awake while preview-7/worker uses it` | Not yet. Its dependents hold it, until they pause. | | `held awake for 2h (active-until)` | Not yet. `napwake.dev/active-until`, on the ManagedWorkload or the workload, as `wake --for` sets. | | `held awake by the forecast: busy from 13:00 UTC` | Not yet. The forecast expects demand that hour. | | `never pauses: no idlePolicy`, `never pauses: namespace protected` | Never | | `never pauses: part of Napwake` | Never. It’s the operator’s or the doorman’s own Deployment. | | `never pauses: PVCs deleted on scale-down` | Never. A StatefulSet with `persistentVolumeClaimRetentionPolicy.whenScaled: Delete`. | | `nothing: labelled napwake.dev/ignore`, `nothing: workload not found`, `nothing: another ManagedWorkload has it` | Never | | `won't pause: dependsOn cycle`, `won't pause: no CPU metrics`, `won't pause: Prometheus failing` | Never | | `wakes on a request or activity` | Paused, and what wakes it. Without the doorman, `wakes on activity`. | | `wakes on a click, a wake request or activity` | Paused, and its doorman [asks before waking](https://docs.napwake.dev/next/concepts/wake-on-request/#asking-before-waking) it | | `paused early: the forecast expected quiet until Fri 20:00 UTC; wakes on a request or activity` | The forecast [brought the pause forward](https://docs.napwake.dev/next/concepts/forecasting/#pausing-early), before `idleAfter` ran out | For `overdue to pause`, `kubectl describe managedworkload` says what holds it. ### The FORECAST column | FORECAST | What it means | | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | | `acting` | Its [forecast](https://docs.napwake.dev/next/concepts/forecasting/#when-it-acts) acts | | `learning`, with `acts from Oct 8` | When the operator expects it to start, if the coming days repeat the last week it saw | | `relearning` | Demand broke from what it had learned | | `not learning while paused:` and why | Paused, with neither the doorman nor a [Prometheus activity query](https://docs.napwake.dev/next/guides/prometheus-signals/#while-the-workload-is-paused) to see its demand | | Such as | `no Service routes requests to it`. `status.prediction.message` says how to fix it. | | `-` | The operator hasn’t recorded one yet | | In JSON | The reason, from `status.prediction.reason`, in `forecast`, and `forecastStartsActing` | ### Turned away while asleep Under the table, when a workload whose doorman asks before waking it turned requests away during its current or last pause: ```text Turned away while asleep, by a doorman that asks before waking: something automated may call these. preview-42/web 312 requests during the pause from 3h ago ``` A person shown the asleep page isn’t counted. So many mean something automated calls it, such as a monitor, a webhook or another service. `-o json` has them as `turnedAway`, from the ManagedWorkload’s `status.turnedAway` ([What it turns away](https://docs.napwake.dev/next/concepts/wake-on-request/#what-it-turns-away)). ### Recent pauses and wakes | Recent pauses and wakes | | | ------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------- | | A wake’s cause | A request on a Service, an activity annotation, a wake request, the forecast, or a scale-up from outside Napwake | | Activity annotations | Including a dependent waking | | Wake requests | From `kubectl napwake wake`, or a tool setting `napwake.dev/wake-requested`, by their `WakeRequested` event | | Requests the doorman closed without reaching the workload | Listed too | | A wake with no recorded cause | Listed by its `resumed` event alone, such as a hand-back when a namespace becomes protected | | How many | The table shows the latest 10, and `-o json` every one | | Each one’s `kind`, in `-o json` | `pause`, `wake` or `notServed` | | `wokenBy`, in `-o json` | On each wake. Each workload carries it too, for its latest wake until it pauses again | | `wokenBy` left out | When the cause wasn’t recorded, and once the events have expired | | A ManagedWorkload deleted and made again under the same name | Starts without one, since the old one’s events name its UID, not the new one’s | | How far back | The API server keeps events for an hour, unless its `--event-ttl` says otherwise | So on most clusters, an hour is what `status` can show. | `wokenBy` | What woke it | | ---------------- | -------------------------------------------------------------------------------------------------------- | | `request` | A request on one of its Services, through the doorman, including a person’s click on one that asks first | | `activity` | An activity annotation, a dependent waking, or a Prometheus activity query | | `wakeRequest` | A wake request, such as `kubectl napwake wake` | | `forecast` | `autoResume`, ahead of the demand the forecast expects | | `outside` | A scale-up from outside Napwake | | `pauseCancelled` | Whatever came as it paused, which the event’s message names | ### JSON and YAML The JSON and YAML fields this page names, and what each holds. | Field | What it holds | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | `forecast` | The reason, from `status.prediction.reason` | | `forecastStartsActing` | When the operator expects the forecast to start acting, as `acts from` shows | | `turnedAway` | For a workload that asks, `pausedAt` and `requests`, what the doorman turned away during that pause, from `status.turnedAway` | | `wokenBy` | What woke it, on each wake, and on each workload for its latest wake until it pauses again ([the values](https://docs.napwake.dev/next/reference/cli/#recent-pauses-and-wakes)) | | `kind` | Each pause or wake’s kind, `pause`, `wake` or `notServed` | | `target` | Each problem’s workload, as the table names it, such as `deployment/worker` | ### Permissions | Your user needs | Where, or why | Without it | | ------------------------------------------------------------------------------------- | --------------------------------------------------- | ---------------------------------------------- | | `list` on `managedworkloads` and `events` | Every namespace, or the `-n` ones | Without events, it shows the rest, and says so | | Karpenter’s NodePools, and nodes’ metadata | The node autoscaler | The autoscaler is unknown | | In `kube-system`, the `cluster-autoscaler-status` ConfigMap and Deployments’ metadata | The node autoscaler | The autoscaler is unknown | | `list` on `deployments` and `statefulsets`, metadata only | `napwake.dev/active-until` on the workload itself | It goes without | | `get` on `namespaces` | Telling a missing `-n` namespace, which is an error | It shows as one Napwake manages nothing in | The autoscaler is unknown when there isn’t enough to tell. ## `nodes` Its flags, with their defaults. [See why nodes don’t scale down](https://docs.napwake.dev/next/getting-started/kubectl-plugin/#see-why-nodes-dont-scale-down) shows it in use. | Flag | Default | What it does | | --------------- | ------- | ------------------------------------------------- | | `--output` `-o` | `table` | `table`, `json` or `yaml` | | `--timeout` | `1m` | How long to wait for the cluster before giving up | ### The BLOCKED BY column | On a node | Blocked by | | ----------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | A Karpenter NodePool owns | The node or a pod annotated `karpenter.sh/do-not-disrupt: "true"` | | The same | A pod whose PodDisruptionBudget allows no disruptions (`status.disruptionsAllowed` is 0) | | The same | A NodePool with `consolidationPolicy: WhenEmpty`, and anything but DaemonSet pods on the node | | The same | `consolidateAfter: Never` | | Any other, by Cluster Autoscaler’s rules | The node annotated `cluster-autoscaler.kubernetes.io/scale-down-disabled: "true"` | | The same | A pod annotated `cluster-autoscaler.kubernetes.io/safe-to-evict: "false"` | | The same | A pod with an `emptyDir` not in memory, or a `hostPath` volume | | The same, unless | It’s named in `cluster-autoscaler.kubernetes.io/safe-to-evict-local-volumes` | | The same | A pod no controller owns | | The same | A `kube-system` pod without a PodDisruptionBudget | | The same | The PodDisruptionBudget rule | | Any | DaemonSet and static pods never block | | Any | A pod annotated `safe-to-evict: "true"` blocks only by its PodDisruptionBudget, as with Cluster Autoscaler | | Local storage and `kube-system` pods | Block only while `--skip-nodes-with-local-storage` and `--skip-nodes-with-system-pods` are on | | Those flags | On by default upstream. AKS turns the first off. | | Not Karpenter’s, autoscaler unknown | Cluster Autoscaler’s rules | | No autoscaler found scales it | `not autoscaled` | | A Cluster Autoscaler node whose node group is at its minimum size | `node group at its minimum`. It stays, whatever runs on it, so no step would free it. Above the next steps, a line names its group and minimum instead of steps | | Where the sizes come from | Cluster Autoscaler’s `cluster-autoscaler-status` ConfigMap, in the YAML it writes since 1.30 or the text before, GKE’s included | | Which group a node is in | Matched as GKE, EKS managed node groups and AKS name them. Karpenter keeps no minimum | | Without access to the ConfigMap | A note says nodes at their minimum aren’t told apart | ### How it finds the node autoscaler | It shows | When it finds | | --------------------- | -------------------------------------------------------------------------------------------------------- | | Karpenter | The `nodepools.karpenter.sh` CRD, or nodes labelled `karpenter.sh/nodepool` | | Karpenter’s detail | Each NodePool’s `spec.disruption.consolidationPolicy` and `consolidateAfter` | | EKS Auto Mode | Nodes labelled `eks.amazonaws.com/compute-type: auto`. Its NodePools are Karpenter’s, read the same way. | | Cluster Autoscaler | The `cluster-autoscaler-status` ConfigMap it writes in `kube-system` | | Which | Self-managed, or run for you by GKE Standard and AKS | | Its detail | The cloud its nodes are on | | Without the ConfigMap | A `kube-system` Deployment ending in `cluster-autoscaler` is named, as a hint it may not be running | | GKE Autopilot | That ConfigMap, and nodes all named `gk3-`, as Autopilot names them. It scales nodes itself. | | `none found` | Nothing, with every place read | | Then | Freed capacity won’t come off the bill until something removes nodes | | `unknown` | Your access didn’t allow reading what would tell, which it names | | Never | It never says none when it couldn’t look | ### Permissions | Your user needs | Without it | | ------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- | | `list` on `nodes` | There’s no table | | `list` on `pods` and `poddisruptionbudgets` in every namespace | What each node runs, and the reasons from its pods, are left out | | `list` on `nodepools.karpenter.sh` | | | `get` on the `cluster-autoscaler-status` ConfigMap, and `list` on `deployments`, in `kube-system` | | | `list` on `managedworkloads` | | It reads what it can, and the notes say what it couldn’t. ## `pause` Its flags, with their defaults. [Pause a workload](https://docs.napwake.dev/next/getting-started/kubectl-plugin/#pause-a-workload) shows it in use. | Flag | Default | What it does | | ------------------ | ------------------------- | ----------------------------------------------------------------------------------------------- | | `--namespace` `-n` | your kubeconfig context’s | Namespace of the workload | | `--wait` | `true` | Wait until it’s Paused, or Napwake says why not | | `--yes` `-y` | `false` | Pause even if the forecast expects it busy within the hour, without asking | | `--clear-hold` | `false` | Clear a `napwake.dev/active-until` hold, on its ManagedWorkload or the workload, without asking | | `--timeout` | `5m` | How long to wait, cluster calls included. It starts again after you answer a question. | On the ManagedWorkload, `wake --for` sets that hold. With [`wake: ask`](https://docs.napwake.dev/next/concepts/wake-on-request/#asking-before-waking), its answer says a person’s click on the asleep page, or `kubectl napwake wake`, wakes it, not a request. On a ManagedWorkload Napwake hasn’t taken on yet, such as one created a moment ago, `pause` waits until Napwake has. A request already on it then isn’t acted on. ### When the forecast expects it to be busy When a confident forecast expects the workload busy within the hour, Napwake declines the pause and says when. It would likely be woken straight back, so `pause` asks. ``` preview-42/api wasn't paused: the forecast expects demand at 80% of requests in the hour from 09:00 UTC Pause it anyway? [y/N] ``` | Answer or case | What happens | | --------------------------- | ------------------------------------------------------------------- | | `y` | It asks again, overriding the forecast, and it pauses | | Anything else | It stays running, and `pause` exits 0 | | Taking your time | The question waits as long as you take | | `--timeout` | Counts only time on the cluster, and starts again after your answer | | `--yes` | Overrides the forecast from the start, without asking | | No terminal, as in a script | `pause` exits 1 with the reason, and says to pass `--yes` | ### When something holds it awake A `napwake.dev/active-until` annotation holds it awake, on the ManagedWorkload, as `wake --for` sets it, or on the workload. Napwake would refuse the pause, so `pause` asks first, naming which holds it. ``` preview-42/api wasn't paused: the napwake.dev/active-until annotation on its ManagedWorkload keeps it awake until 2026-10-17T20:00:00Z Clear it and pause now? [y/N] ``` | Answer or case | What happens | | ----------------------------------- | ---------------------------------------------------------------- | | `y` | It removes the annotation, from both if both hold it, and pauses | | Anything else | It stays running, and `pause` exits 0 | | `--clear-hold` | Clears it without asking | | No terminal, without `--clear-hold` | `pause` exits 1, and says to pass it | | `--yes` | Doesn’t clear a hold | So a nightly script that pauses with `--yes` doesn’t end a hold someone set for a demo. ### When a GitOps tool sets its replicas A GitOps tool may set the replicas from Git, without being told to leave them to Napwake. Then `pause` warns before asking, with the one-time fix. [GitOps tools](https://docs.napwake.dev/next/guides/gitops/#checking-the-fix-is-in) says how it tells. ``` Warning: Argo CD application argocd/shop sets preview-42/api's replicas from Git, as argocd-controller, so its next sync will likely scale it back up, undoing this pause. To have Argo CD leave replicas to Napwake, once: Add RespectIgnoreDifferences=true to the syncPolicy.syncOptions of Argo CD application argocd/shop, so syncs leave the replicas it ignores. See https://docs.napwake.dev/guides/gitops/#argo-cd ``` When it can’t tell, such as without access to Argo CD’s namespace, the warning says why. It says to make the fix if you haven’t. ### Permissions | Your user needs | For | Without it | | ----------------------------------------------------------------------------- | ----------------------------------------------- | ---------------------------------- | | `get`, `list` and `patch` on `managedworkloads` in the namespace | The pause | | | `get` on the workload | The GitOps warning, and the workload’s own hold | They’re left out | | `patch` on the workload | Clearing a hold on it | | | `get` on the `argocd-cm` ConfigMap | Telling whether Argo CD leaves its replicas | The warning says it couldn’t check | | `list` on `applications.argoproj.io` in every namespace, or `get` in `argocd` | The same | The same | The Argo CD reads are only for a workload Argo CD applies. ## `wake` Its flags, with their defaults. [Wake a workload](https://docs.napwake.dev/next/getting-started/kubectl-plugin/#wake-a-workload) shows it in use. | Flag | Default | What it does | | ------------------ | ------------------------- | ------------------------------------------------------------------------------------------ | | `--namespace` `-n` | your kubeconfig context’s | Namespace of the workload | | `--for` | | Also keep it awake this long, by setting `napwake.dev/active-until` on its ManagedWorkload | | `--wait` | `true` | Wait for Napwake’s answer to the request, then until it’s Running | | `--timeout` | `5m` | How long to wait, cluster calls included | With `--for`, a hold on the ManagedWorkload that ends later stays, and `wake` says so. `wake` sets a fresh `napwake.dev/wake-requested` value on the ManagedWorkload, as any tool can. So it wakes the workload whatever its wake setting. The `WakeRequest` condition says what came of it ([Wake now](https://docs.napwake.dev/next/guides/pause/#wake-now)). Another tool asking meanwhile replaces `wake`’s token with its own, and Napwake’s answer to that one stands for both. | When | `wake` | | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Napwake refuses the request, such as for a protected workload | Fails at once with Napwake’s reason, without waiting out `--timeout` | | Napwake doesn’t answer by `--timeout` | Fails with its phase, saying Napwake hasn’t taken the request up. That usually means its operator isn’t running | | The ManagedWorkload is deleted meanwhile | Fails, saying Napwake handed the workload back as it let go | | A slow start | `--timeout` covers the answer and `Running`. A request made as the workload pauses is answered at once, and calls the pause off. One made as it wakes is answered only once the wake finishes. For a start over 5 minutes, raise it, such as `--timeout 10m` | ### Permissions Your user needs `get`, `list` and `patch` on `managedworkloads` in the namespace. ## `deps` Its flags, with their defaults. [Show a workload’s dependencies](https://docs.napwake.dev/next/getting-started/kubectl-plugin/#show-a-workloads-dependencies) shows it in use. | Flag | Default | What it does | | ------------------ | ------------------------- | ------------------------------------------------- | | `--namespace` `-n` | your kubeconfig context’s | Namespace of the workload | | `--timeout` | `1m` | How long to wait for the cluster before giving up | ### Permissions It only reads ManagedWorkloads. Without access to them in every namespace, it reads the workload’s own namespace, and says dependents elsewhere aren’t shown. A dependency in another namespace then shows as `unknown (can't read )`. ## `enable` Its flags, with their defaults. [Enable a workload](https://docs.napwake.dev/next/getting-started/kubectl-plugin/#enable-a-workload) shows it in use. | Flag | Default | What it does | | ------------------ | ------------------------- | ----------------------------------------------------------------------- | | `--namespace` `-n` | your kubeconfig context’s | Namespace of the workload | | `--all` | `false` | Enable every workload in the namespace | | `--force` | `false` | Change the cluster even when a GitOps tool applies the setting from Git | | `--timeout` | `1m` | How long to wait for the cluster before giving up | ### When it changes nothing `enable` only ends dry-run for a workload Napwake manages. Otherwise it changes nothing, and exits non-zero with the reason. | Reason | What it means | | ------------------------------------------------ | ------------------------------------------------------------------------------------------- | | The namespace is protected | Labelled `napwake.dev/protected=true`, or a `kube-*` namespace | | Unless | It’s labelled `napwake.dev/allow-protected=true` | | The workload is left out | Labelled `napwake.dev/ignore=true` | | Or opted out | Its own `napwake.dev/managed` is anything but `"true"`, in a managed namespace | | No ManagedWorkload manages it | The operator isn’t running, doesn’t watch the namespace, or protects it by its own settings | | Right after you label it | Its ManagedWorkload may not exist yet. Run `enable` again a few seconds later. | | Its ManagedWorkload says Napwake never pauses it | `Protected=True`, a target labelled `napwake.dev/ignore`, or `DuplicateTarget=True` | | `Protected=True` | Such as by a `protectedNamespaces` pattern, or for Napwake’s own operator and doorman | | No ManagedWorkload CRD | It says Napwake isn’t installed | ### With `--all` `--all` ends dry-run for every workload in the namespace. | When | What it does | | --------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | | The namespace’s own `napwake.dev/dry-run`, or the cluster default, keeps any in dry-run | Sets the namespace’s annotation to `"false"` | | The same | Drops the workloads’ own annotations, so the namespace decides for all | | Each is in dry-run by its own annotation | Leaves the namespace alone, and sets each workload’s to `"false"` | | Either way | Sets `spec.dryRun: false` on ManagedWorkloads written by hand in the namespace | | Git owns the namespace | Leaves its annotation for you to change in Git, ending dry-run for workloads that take it | | The same | Workloads with their own annotation get `"false"` | | You can’t change the namespace, as with `edit` or `admin` bound in it | Leaves its annotation, and says so | | The same | Sets every workload’s own to `"false"`, which wins over it | | The same, with only the namespace’s to change | Exits non-zero | | Labelled workloads no ManagedWorkload manages | Left alone | | The same, after `enable` set the namespace to `"false"` | Their dry-run ends once Napwake picks them up, unless they have their own annotation | | The rest of those | Listed, and `enable` exits non-zero | | None of the namespace’s workloads picked up yet | Changes nothing, the namespace’s annotation included, and exits non-zero | | Why | The operator may not be running | | Workloads Napwake never pauses, by their ManagedWorkload | Left alone, and listed too | ### When a GitOps tool or Helm applies the setting A GitOps tool would put a changed setting straight back, so `enable` changes nothing it applies. The tool is Argo CD, Flux, Fleet, kapp or Kluctl. The setting is the `napwake.dev/dry-run` annotation on the workload or its namespace, or `spec.dryRun`. `enable` names the tool, prints the change to make in Git, and exits non-zero. ``` deployment/checkout-api is managed by Argo CD application shop, which would undo a change made here. In its manifest: set the annotation napwake.dev/dry-run: "false" Or pass --force to change the cluster anyway. ``` | | | | ------------------------------------------------- | --------------------------------------------------------------------------------------------- | | How it tells | From the field managers the API server records, as `pause` does for replicas | | So | How the tool tracks what it applies doesn’t matter | | With none recorded | By the tracking label or annotation Argo CD or Flux adds | | The tool applies the workload, not the annotation | Such as when Git doesn’t have it. The tool leaves a change to it alone. | | Set by Helm | It changes it, since Helm only applies a chart when you run it | | Then | It warns that the next `helm upgrade` puts the chart’s value back, so set it in the chart too | | Replicas set from Git | It warns as [`pause`](https://docs.napwake.dev/next/reference/cli/#pause) does, with the one-time fix, unless the tool is told | | When it warns | Whether it ends dry-run, finds it ended already, or leaves the change to Git | | With `--all` | Workloads a tool treats alike, such as one Argo CD application’s, share one warning | Ending dry-run lets Napwake pause the workload. The tool then scales it back up on the next sync after each pause. That’s why it warns. See [Ending dry-run](https://docs.napwake.dev/next/guides/gitops/#ending-dry-run). ### Permissions | Your user needs | Without it | | ---------------------------------------------------------------- | --------------------------------------------- | | `get` and `patch` on the workload, and `get` on its namespace | | | `list` on `managedworkloads` in the namespace | It goes by the annotations alone, and says so | | With `--all`, `list` on Deployments and StatefulSets | | | With `--all`, `patch` on the namespace, to end its dry-run there | Each workload gets its own annotation instead | | For the GitOps warning, what `pause` reads of Argo CD | | Going by the annotations alone, `enable` can’t check that Napwake manages the workload. Nor can it see a cluster-default dry-run. Next * [kubectl plugin →Each command in use, and what it prints.](https://docs.napwake.dev/next/getting-started/kubectl-plugin/) * [Troubleshooting →Common problems, and how to fix them.](https://docs.napwake.dev/next/operations/troubleshooting/)