# Install on Kubernetes

> **Sentinel v2 (beta):** This page documents Sentinel v2, a beta rewrite that may change before the stable release. It does not apply to Sentinel v1, the current stable version: see the [Sentinel v1 documentation](/docs/sentinel.md).

Run Sentinel on Kubernetes with a few plain manifests: the configuration in a ConfigMap, the secrets in a Secret, and a StatefulSet that gives each pod a stable name and its own data volume. This page starts with one replica on embedded SQLite, then extends it to several replicas with PostgreSQL and Redis.

For the license, secrets, database, and Redis requirements of a production install, see [Install Sentinel](/docs/sentinel/v2/install/).

<Callout type="note" title="Helm chart">

The ALTCHA Helm chart (`altcha-org/sentinel`) deploys Sentinel v1 and doesn't work with v2 yet: v2 reads a mounted configuration file and uses different ports and probes. Use the manifests on this page until a v2 chart is released.

## Prerequisites

- A Kubernetes cluster with a default StorageClass, and `kubectl` configured for it.
- A Sentinel license key. To get a trial, register at [cloud.altcha.org](https://cloud.altcha.org) and create a Sentinel V2 trial license in the **Licenses** section.

## Step 1 — Create the secret files

Keep each secret in its own file. The Secret you create from them is mounted at `/run/secrets` in the pod, and Sentinel reads each file with a `${file:/run/secrets/…}` reference in the configuration.

```bash
mkdir -p sentinel/secrets && cd sentinel
printf '%s' 'SENT-…' > secrets/sentinel_license_key   # your license key
openssl rand -base64 24 > secrets/root_password        # first root password
```

## Step 2 — Write `sentinel.toml`

Create `sentinel.toml` next to the `secrets` folder:

```toml
[server.admin_ui]
enabled = true   # stays on 127.0.0.1 in the pod; reach it with kubectl port-forward

[license]
key = "${file:/run/secrets/sentinel_license_key}"

[bootstrap]
root_password = "${file:/run/secrets/root_password}"   # applied on first boot only
```

Every other key keeps its default: the main API listens on port `3000`, data goes to `/home/nonroot/data` in the container, and the signing and encryption secrets are generated into the data volume on first start. The admin UI listens on the pod's loopback interface only, so nothing outside the pod reaches it; `kubectl port-forward` connects to it from inside the pod. All keys are listed in the [Configuration reference](/docs/sentinel/v2/reference/configuration/).

If Docker is available on your machine, validate the file before you deploy it, as shown in [Install with Docker](/docs/sentinel/v2/install/docker/#step-2--write-sentineltoml). Otherwise, an invalid file stops the pod at startup, and `kubectl logs` names the key and the reason:

```text
Caused by:
unknown field `enabeld`, expected one of `enabled`, `host`, `port`, …
in `server.admin_ui`
```

## Step 3 — Create the namespace, Secret, and ConfigMap

```bash
kubectl create namespace sentinel

kubectl -n sentinel create secret generic sentinel-secrets \
--from-file=secrets/ --dry-run=client -o yaml | kubectl apply -f -

kubectl -n sentinel create configmap sentinel-config \
--from-file=sentinel.toml --dry-run=client -o yaml | kubectl apply -f -
```

`--from-file=secrets/` adds every file in the folder under its file name. Piping through `kubectl apply` lets you run the same commands again after you change a file.

## Step 4 — Deploy Sentinel

Create `sentinel.yaml`:

```yaml
apiVersion: v1
kind: Service
metadata:
name: sentinel
namespace: sentinel
spec:
selector:
app.kubernetes.io/name: sentinel
ports:
- name: http
  port: 3000
  targetPort: http
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: sentinel
namespace: sentinel
spec:
serviceName: sentinel
replicas: 1
selector:
matchLabels:
  app.kubernetes.io/name: sentinel
template:
metadata:
  labels:
    app.kubernetes.io/name: sentinel
spec:
  automountServiceAccountToken: false   # Sentinel doesn't call the Kubernetes API; frees /run/secrets
  securityContext:
    runAsNonRoot: true
    runAsUser: 65532
    runAsGroup: 65532
    fsGroup: 65532
  containers:
    - name: sentinel
      image: ghcr.io/altcha-org/sentinel:2.0.0-beta.1
      ports:
        - name: http
          containerPort: 3000
        - name: admin
          containerPort: 3002
      volumeMounts:
        - name: config
          mountPath: /home/nonroot/sentinel.toml
          subPath: sentinel.toml
          readOnly: true
        - name: secrets
          mountPath: /run/secrets
          readOnly: true
        - name: data
          mountPath: /home/nonroot/data
      startupProbe:
        httpGet:
          path: /
          port: http
        periodSeconds: 5
        failureThreshold: 60
      livenessProbe:
        httpGet:
          path: /
          port: http
        periodSeconds: 10
        failureThreshold: 3
      readinessProbe:
        httpGet:
          path: /
          port: http
        periodSeconds: 10
      resources:
        requests:
          cpu: "1"
          memory: 512Mi
        limits:
          memory: 1Gi
  volumes:
    - name: config
      configMap:
        name: sentinel-config
    - name: secrets
      secret:
        secretName: sentinel-secrets
volumeClaimTemplates:
- metadata:
    name: data
  spec:
    accessModes: ["ReadWriteOnce"]
    resources:
      requests:
        storage: 5Gi
```

- **StatefulSet.** Each pod gets a stable name (`sentinel-0`, `sentinel-1`, …), which Kubernetes also sets as `HOSTNAME`. Sentinel uses it as the node ID of its [license instance slot](/docs/sentinel/v2/operations/clustering/#instance-slots), so a restarted pod takes its slot back at once. Each pod also gets its own data volume from `volumeClaimTemplates`.
- **`automountServiceAccountToken: false`.** Kubernetes mounts the service account token at `/var/run/secrets/kubernetes.io/serviceaccount`. In the image, `/var/run` is a link to `/run`, so with the Secret mounted read-only at `/run/secrets` the pod fails to start with `read-only file system`. Sentinel doesn't use the token.
- **`fsGroup: 65532`.** The container runs as `65532:65532` and needs write access to the data volume. `fsGroup` gives that group ownership of the volume when it's mounted. The Secret's files are mounted with mode `0644`, readable by the same user.
- **Probes.** All three call `GET /`, which answers `{"ok":true}` without touching the database or Redis, so a storage outage doesn't restart or unpublish every pod. The startup probe allows up to 5 minutes for the first start. Don't probe `GET /.health`; see [Liveness and readiness](/docs/sentinel/v2/operations/monitoring/#liveness-and-readiness-get-).
- **Configuration file.** The ConfigMap is mounted at the default path, so the container needs no arguments. Sentinel reads the file only at startup, and a `subPath` mount doesn't receive ConfigMap updates, so restart the pods after you change it.
- **Resources.** The requests match the evaluation size in [Sizing](/docs/sentinel/v2/install/#sizing). Raise them for production, and raise the memory limit when you enable the local similarity model or memory-hard proof of work.
- Use an exact `2.0.0-beta.x` tag; during the beta, `latest` still points to Sentinel v1.

Apply it and wait for the pod:

```bash
kubectl apply -f sentinel.yaml
kubectl -n sentinel rollout status statefulset/sentinel
kubectl -n sentinel logs sentinel-0
```

The log shows the license status (for example `licensed to "…"`), `created root user from bootstrap.root_password`, `web app listening addr=127.0.0.1:3002`, and `listening addr=0.0.0.0:3000`. If you didn't set `[bootstrap] root_password`, the log contains the generated root password instead.

## Step 5 — Sign in

Forward the admin UI port to your machine:

```bash
kubectl -n sentinel port-forward pod/sentinel-0 3002:3002
```

Open `http://localhost:3002` and sign in with the username `root` and the password from `secrets/root_password`.

> **Warning:**
>
> `[bootstrap] root_password` is applied only when the root user is created on first boot. Changing it later has no effect; change the password in the admin UI instead.

## Verify it works

Forward the main API in a second terminal and call its status endpoint:

```bash
kubectl -n sentinel port-forward svc/sentinel 3000:3000
curl -s http://localhost:3000/
```

```json
{"ok":true}
```

To check the configuration, storage, and network access from inside the pod, run `doctor` in it:

```bash
kubectl -n sentinel exec sentinel-0 -- /usr/local/bin/altcha-sentinel doctor
```

`doctor` prints one line per check and exits with `1` if any check fails. `security.root_mfa` warns until you enable TOTP or a passkey for root. See the [CLI reference](/docs/sentinel/v2/reference/cli/) for the options.

## Expose Sentinel

The Service is `ClusterIP`, so backends inside the cluster call `http://sentinel.sentinel.svc:3000`. To serve widgets on the internet, put the main API behind an Ingress that terminates TLS. This example uses ingress-nginx and a TLS Secret named `sentinel-tls`:

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: sentinel
namespace: sentinel
spec:
ingressClassName: nginx
tls:
- hosts: [sentinel.example.com]
  secretName: sentinel-tls
rules:
- host: sentinel.example.com
  http:
    paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: sentinel
            port:
              name: http
```

Then tell Sentinel about the proxy in `sentinel.toml`, update the ConfigMap as in Step 3, and restart the pods with `kubectl -n sentinel rollout restart statefulset/sentinel`:

```toml
[server]
trusted_proxies = ["10.42.0.0/16"]           # your cluster's pod network, where the ingress controller runs
base_url        = "https://sentinel.example.com"
```

- Without `trusted_proxies`, every request appears to come from the ingress controller, which breaks rate limits, penalties, and IP rules. The ingress controller itself must also see the client IP, for example with `externalTrafficPolicy: Local` on its Service or the PROXY protocol from your cloud load balancer.
- To check the result, run the doctor through the admin UI and look at `security.client_ip`; see [Listeners: Verify it works](/docs/sentinel/v2/configure/listeners/#verify-it-works).
- To publish only the widget endpoints, enable `[server.public_api]` (port `3003`), add the port to the Service, and point the Ingress at it. To publish the admin UI instead of port-forwarding, set its `host` to `"0.0.0.0"`, restrict it with `allowed_ips`, and give it its own `trusted_proxies` and `base_url`. See [Listeners & reverse proxy](/docs/sentinel/v2/configure/listeners/#choose-what-to-expose).

## Run several replicas with PostgreSQL and Redis

Several replicas keep Sentinel available while a pod restarts or a node drains. Every replica must share an external database, a Redis-compatible KV store, and the same secrets; see [Clustering & high availability](/docs/sentinel/v2/operations/clustering/#requirements). Use a managed PostgreSQL and Redis, or ones running in your cluster. Size Redis with [Redis or Valkey](/docs/sentinel/v2/install/#redis-or-valkey) and set `maxmemory-policy noeviction`.

<Callout type="note" title="License required">

PostgreSQL needs the <code>exotdb.backend.postgres</code> license feature, and the replica count is capped by the license's `instances` limit. See [Plans & Licensing](/docs/sentinel/v2/plans/).

1. **Add the secrets.** With an external database, Sentinel no longer generates its signing and encryption keys, and every replica must use the same values. Generate them next to the existing files, and add the database and Redis passwords:

```bash
openssl rand -hex 32 > secrets/jwt_secret
openssl rand -hex 32 > secrets/altcha_hmac_secret
openssl rand -hex 32 > secrets/encryption_key
printf '%s' '<database password>' > secrets/db_password
printf '%s' '<redis password>' > secrets/redis_password
```

Back up these files with the database: a different `encryption_key` can't read the API key and TOTP secrets stored in it. See [Server secrets](/docs/sentinel/v2/configure/secrets/).

2. **Point `sentinel.toml` at the database and Redis:**

```toml
[server.admin_ui]
enabled = true

[database]
url = "postgres://sentinel:${file:/run/secrets/db_password}@postgres.example.internal:5432/sentinel?sslmode=verify-full"

[redis]
nodes = ["redis://:${file:/run/secrets/redis_password}@redis.example.internal:6379"]

[secrets]
jwt_secret         = "${file:/run/secrets/jwt_secret}"
altcha_hmac_secret = "${file:/run/secrets/altcha_hmac_secret}"
encryption_key     = "${file:/run/secrets/encryption_key}"

[license]
key = "${file:/run/secrets/sentinel_license_key}"

[bootstrap]
root_password = "${file:/run/secrets/root_password}"
```

Use `rediss://` for Redis over TLS. A password with characters that aren't allowed in a URL must be percent-encoded. Each replica opens up to `[database] pool_max` connections (default `20`), so keep `pool_max` × replicas below the database's connection limit. See [Databases & KV store](/docs/sentinel/v2/configure/databases/).

3. **Update the Secret and ConfigMap** with the commands from [Step 3](#step-3--create-the-namespace-secret-and-configmap).

4. **Scale the StatefulSet.** Set `replicas: 2` (or more) in `sentinel.yaml`, then apply it and restart the pods so they read the new configuration:

```bash
kubectl apply -f sentinel.yaml
kubectl -n sentinel rollout restart statefulset/sentinel
kubectl -n sentinel rollout status statefulset/sentinel
```

The replicas migrate the schema one at a time under a database lock: the first logs `schema migrations complete applied=…`, and the others find the schema current and log `applied=0`. Each log shows `kv store ready pubsub=true`. To see all replicas, open **Cluster** in the admin UI, or see [List the replicas](/docs/sentinel/v2/operations/clustering/#list-the-replicas).

5. **Limit voluntary disruptions** so a node drain never stops every replica at once:

```yaml
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
 name: sentinel
 namespace: sentinel
spec:
 maxUnavailable: 1
 selector:
   matchLabels:
     app.kubernetes.io/name: sentinel
```

Spread the replicas across nodes or zones with `topologySpreadConstraints` in the pod template.

The replicas start with an empty PostgreSQL database; the SQLite database in the first pod's data volume stays untouched. To carry over accounts, keys, and security groups, follow [Restore into a fresh deployment](/docs/sentinel/v2/operations/snapshots/#restore-into-a-fresh-deployment): the restored API keys work only with the old `encryption_key`, which the single instance generated into `secrets.json` in its data volume.

Each replica keeps its own data volume for downloaded remote files and caches. Don't share one volume between replicas.

## Update Sentinel

Before an update, read the release notes and take a snapshot, as described in [Before you update](/docs/sentinel/v2/operations/updating/#before-you-update). Then change the image tag:

```bash
kubectl -n sentinel set image statefulset/sentinel sentinel=ghcr.io/altcha-org/sentinel:2.0.0-beta.2
kubectl -n sentinel rollout status statefulset/sentinel
```

Update the tag in `sentinel.yaml` too, so the next `kubectl apply` doesn't roll it back. The StatefulSet replaces one pod at a time, from the highest ordinal down, and waits for each new pod to pass its readiness probe. The first new pod migrates the schema, and the older pods keep working against it meanwhile. Each new pod reuses its predecessor's `HOSTNAME`, so it takes over the same license instance slot without a burst slot. With a single replica, Sentinel is unavailable for a few seconds while the pod restarts. Rolling updates and rollback are covered in [Updating Sentinel](/docs/sentinel/v2/operations/updating/).

## Next steps

- [Create a security group and an API key](/docs/sentinel/v2/get-started/), then embed the widget.
- [Licensing](/docs/sentinel/v2/configure/licensing/): offline license files and instance slots.
- [Monitoring & metrics](/docs/sentinel/v2/operations/monitoring/): scrape the Prometheus listener and alert on storage health.
- [Snapshots & backups](/docs/sentinel/v2/operations/snapshots/): back up the database before you go live.
- [Install Sentinel](/docs/sentinel/v2/install/): production requirements, and the [Production checklist](/docs/sentinel/v2/security/production-checklist/).
