Documentation / Operations

Deploy and operate wallet instances

Use the Deployer for normal changes. This guide is for reviewing a generated deployment bundle, manual recovery, and diagnosing an instance when the application is not healthy.

Start with the instance bundle

The Deployer writes a recoverable record for every environment.

Find the bundle at deployments/environments/<instance>/. It contains instance.yaml, rendered values.yaml, helm-upgrade.sh, setup-tenant.json, and per-operation snapshots.

Use the kube context recorded in helm-upgrade.sh. Do not depend on the active kubectl context.

Generated values can reference secrets. Do not paste API keys, registry credentials, or decoded Secret values into a ticket, chat, or commit.

Manual deploy or re-apply

Review the generated command, then use the same chart, values, namespace, and context as the approved plan.

cd /path/to/krumtap/deployments/environments/$INSTANCE
sed -n '1,160p' helm-upgrade.sh
./helm-upgrade.sh

First checks

These commands are read-only and are the right first response to a failed or degraded instance.

INSTANCE=<instance-name>
NAMESPACE="$INSTANCE"
RELEASE="$INSTANCE"
CONTEXT=<recorded-kube-context>

kubectl --context "$CONTEXT" get pods,svc,ingress,pvc -n "$NAMESPACE"
kubectl --context "$CONTEXT" get events -n "$NAMESPACE" --sort-by=.metadata.creationTimestamp
helm status "$RELEASE" --namespace "$NAMESPACE" --kube-context "$CONTEXT"
helm history "$RELEASE" --namespace "$NAMESPACE" --kube-context "$CONTEXT"

Inspect the failing component

Use pod describe and both current and previous container logs before restarting anything.

POD=<failing-pod-name>
kubectl --context "$CONTEXT" describe pod "$POD" -n "$NAMESPACE"
kubectl --context "$CONTEXT" logs "$POD" -n "$NAMESPACE" --all-containers --tail=250
kubectl --context "$CONTEXT" logs "$POD" -n "$NAMESPACE" --all-containers --previous --tail=250

app.kubernetes.io/name=walletapp.kubernetes.io/name=policy-nodesapp.kubernetes.io/name=keycloakapp.kubernetes.io/name=approver-webapp.kubernetes.io/name=frontend

Known failure paths

Fix the cause, then re-apply or restart the smallest affected part.

Image pull or platform failure: confirm the exact registry tag and node architecture. ARM64 k3d needs ARM64 or multi-arch images.

Helm hook timeout: inspect events and database/NATS setup-job logs; correct missing Secrets, PVCs, or image pulls before rerunning the generated script.

Event streaming incomplete: check webhook reachability and Aegis target quota. Redeploying does not bypass an exhausted target quota.

Wallet and policy-node mismatch: use the same MPA build by default. A policy node newer than the wallet CWPLib requires an upgrade or downgrade.

Recovery commands

These change running state. Read the comments and inspect Helm history first.

# Restart only after reviewing the workload logs.
kubectl --context "$CONTEXT" rollout restart deployment/<deployment-name> -n "$NAMESPACE"
kubectl --context "$CONTEXT" rollout status deployment/<deployment-name> -n "$NAMESPACE" --timeout=5m

# Inspect history before choosing a known-good revision.
helm rollback "$RELEASE" <known-good-revision> \
  --namespace "$NAMESPACE" --kube-context "$CONTEXT" --wait --timeout 10m

Local k3d and Colima

The local k3d cluster is ARM64; the Deployer must reach Colima’s Docker socket to prepare local images.

If the Deployer reports that /var/run/docker.sock is unavailable, restart its local dev process with this environment variable:

cd /Users/sprasmantas/Sites/gitlab/blockdaemon/sepior/krumtap/deployer
DOCKER_HOST="unix://$HOME/.colima/default/docker.sock" pnpm dev

Do not attempt to deploy AMD64-only Wallet or Policy Node images to an ARM64 k3d node. Select compatible tags first.

← Documentation·Open the wallet instance wizard