Merge pull request #15 from vish-mv/fluent-bit

Set the Fluent Bit toleration as a Helm value in the isolation-tier guides

Author: menakaj

docs/guides/isolation-tiers/gvisor.mdx

@@ -197,13 +197,48 @@ kubectl apply -f "${AMP_DIST}/deployments/k8s/gvisor-runtimeclass.yaml"
 # 2. Label and taint the node (find the name with: kubectl get nodes)
 kubectl label node <node-name> gvisor=true --overwrite
 kubectl taint node <node-name> gvisor=true:NoSchedule --overwrite
+```
+
+The label is what the RuntimeClass schedules onto; the taint keeps regular runc pods off the node.
+
+The taint also keeps the Fluent Bit log collector off the node, and without it agents there produce no runtime logs. Fluent Bit is a DaemonSet rendered by the `observability-logs-opensearch` Helm chart, so its toleration is a chart value. Check whether the release stores it:
+
+```bash
+helm get values observability-logs-opensearch -n openchoreo-observability-plane
+```
+
+Check the release values rather than the live DaemonSet: a toleration patched onto the DaemonSet (as earlier versions of the setup scripts did) shows up on the live object, but is lost when the DaemonSet is recreated.
+
+If `fluent-bit.tolerations` does not list `- operator: Exists` with no `key` or `effect`, set the value on the release:
+
+```bash
+helm upgrade observability-logs-opensearch \
+  oci://ghcr.io/openchoreo/helm-charts/observability-logs-opensearch \
+  --namespace openchoreo-observability-plane \
+  --version 0.5.3 \
+  --reuse-values \
+  --set "fluent-bit.tolerations[0].operator=Exists"
+```
+
+Pass the chart version that is already installed (the `CHART` column of `helm list -n openchoreo-observability-plane`): combining `--reuse-values` with a chart version change can carry over values the new chart does not expect.
+
+:::warning Keep the toleration on later upgrades
+The toleration is stored in the release's values, so it must be present on every upgrade that re-renders the DaemonSet. An upgrade of `observability-logs-opensearch` with `--reset-values`, or one that passes new `--set`/`-f` values without `--reuse-values`, drops it unless you pass `--set "fluent-bit.tolerations[0].operator=Exists"` again. Agents on the gVisor node then stop producing logs with no error.
+:::
+
+<details>
+<summary>Logs module not installed as a Helm release</summary>
 
-# 3. Let the Fluent Bit log DaemonSet tolerate the taint (so agent logs are collected)
+Only if `observability-logs-opensearch` is not a Helm release in your cluster, patch the DaemonSet directly:
+
+```bash
 kubectl patch daemonset fluent-bit -n openchoreo-observability-plane --type=json \
   -p='[{"op":"add","path":"/spec/template/spec/tolerations","value":[{"operator":"Exists"}]}]'
 ```
 
-The label is what the RuntimeClass schedules onto; the taint keeps regular runc pods off the node.
+This replaces the DaemonSet's whole tolerations list, and is lost whenever the DaemonSet is recreated (a reinstall of the logs module or a namespace teardown), so re-check the toleration after any such change.
+
+</details>
 
 :::tip Verify
 ```bash
@@ -325,7 +360,7 @@ The Fluent Bit DaemonSet must run on the tainted node. Check it covers the gViso
 kubectl get pods -n openchoreo-observability-plane -o wide | grep fluent-bit
 ```
 
-If no Fluent Bit pod is on the gVisor node, apply the toleration patch from Step 3.
+If no Fluent Bit pod is on the gVisor node, set the Fluent Bit toleration as described in [Step 3](#step-3-register-the-node-for-the-gvisor-tier).
 
 </details>
 

docs/guides/isolation-tiers/kata.mdx

@@ -45,7 +45,7 @@ Like gVisor, Kata uses a **dedicated node** labeled `kata=true` and tainted `kat
 | Virtualization | `/dev/kvm` present and functional (see warning above) |
 | Architecture | x86_64 |
 | OS | Ubuntu (or similar) with systemd + containerd — **not** Container-Optimized OS |
-| Access | kubectl access to the cluster (the installer runs from your machine, not on the node) |
+| Access | `kubectl` and `helm` access to the cluster (the installer runs from your machine, not on the node) |
 
 ## Set Up a Kata Node
 
@@ -75,7 +75,7 @@ If your platform lets you set taints at node-pool creation, apply `kata=true:NoS
 
 ### Step 2: Run the installer from a machine with kubectl access
 
-Unlike the gVisor installer (which runs on the node), the Kata installer runs from **any machine with kubectl access** and does everything through the Kubernetes API:
+Unlike the gVisor installer (which runs on the node), the Kata installer runs from **any machine with `kubectl` and `helm` access** and does everything through the Kubernetes API:
 
 ```bash
 KATA_NODES="<node-name>" bash "${AMP_DIST}/deployments/setup/install-kata.sh"
@@ -85,10 +85,10 @@ What it does:
 
 1. **Pre-flights `/dev/kvm`** on every target node (via a short-lived privileged pod) and aborts if any node lacks it.
 2. Labels the node(s) `kata=true`.
-3. Applies the upstream **kata-deploy** DaemonSet, with a nodeSelector and toleration injected **before** apply — so it only ever lands on the labeled node(s); your other nodes are never reconfigured.
-4. **Auto-wires k3s nodes**: k3s bundles its own containerd (configured only via `config.toml.tmpl`, read at startup), which upstream kata-deploy does not handle — the installer detects k3s nodes and patches the template + restarts the k3s agent for you.
-5. Applies the `kata-qemu` RuntimeClass and taints the node(s) `kata=true:NoSchedule`.
-6. Patches the Fluent Bit log DaemonSet to tolerate the taint, so agent logs are collected.
+3. Installs upstream **kata-deploy** from its Helm chart (kata-containers `KATA_VERSION`, default `4.2.0`), scoped to the labeled node(s) so your other nodes are never reconfigured. The chart wires the node's containerd itself on standard Kubernetes, k3s and RKE2; the installer detects which one from the node's kubelet version. Nodes set up by the earlier manifest-based installer are migrated on the first run.
+4. Applies the `kata-qemu` RuntimeClass and boots a test pod under it on every target node. The installer reports success only once that pod has run in its own VM, with a kernel different from the node's.
+5. Taints the node(s) `kata=true:NoSchedule`. Set `KATA_TAINT=false` to leave them schedulable for other pods.
+6. Lets the Fluent Bit log DaemonSet tolerate the taint, so agent logs are collected. It checks the toleration stored in your `observability-logs-opensearch` Helm release and, if it is missing, sets the chart value `fluent-bit.tolerations` (a `helm upgrade --reuse-values` at the release's installed chart version). It patches the DaemonSet only when that release cannot be upgraded with Helm, and then prints the `helm upgrade` command to make it durable, because a patch is lost whenever the DaemonSet is recreated.
 
 :::tip Verify
 ```bash
@@ -98,41 +98,10 @@ kubectl get node <node-name> --show-labels   # kata=true
 ```
 :::
 
-<details>
-<summary>RKE2 nodes (bundled containerd)</summary>
-
-Upstream kata-deploy assumes a standalone containerd at `/etc/containerd/config.toml`, which RKE2 nodes do not have. Two extra steps are needed on each RKE2 Kata node:
-
-**1. Before running the installer**, create an empty stub so the kata-deploy pod gets past its pre-flight check (it crash-loops with `cp: can't stat '/etc/containerd/config.toml'` otherwise):
-
-```bash
-sudo mkdir -p /etc/containerd && sudo touch /etc/containerd/config.toml
-```
-
-**2. After kata-deploy has installed the binaries into `/opt/kata`** (the node gets the `katacontainers.io/kata-runtime=true` label on success), wire the runtime through RKE2's containerd template — kata-deploy wrote its config to the stub, which RKE2 never reads:
-
-```bash
-sudo mkdir -p /usr/local/bin
-sudo ln -sf /opt/kata/bin/containerd-shim-kata-v2 /usr/local/bin/containerd-shim-kata-qemu-v2
-sudo tee /var/lib/rancher/rke2/agent/etc/containerd/config.toml.tmpl > /dev/null <<'EOF'
-{{ template "base" . }}
-
-[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata-qemu]
-  runtime_type = "io.containerd.kata-qemu.v2"
-  privileged_without_host_devices = true
-  pod_annotations = ["io.katacontainers.*"]
-  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.kata-qemu.options]
-    ConfigPath = "/opt/kata/share/defaults/kata-containers/configuration-qemu.toml"
-EOF
-sudo systemctl restart rke2-agent
-```
-
-:::warning Rewrite, never append
-The template must contain `{{ template "base" . }}` exactly once. If the file already exists (e.g. it also carries a gVisor or build-related block), merge the runtime sections into one file and rewrite it wholesale; appending a duplicate template line produces an invalid containerd config and the node's kubelet will not come back after the restart — recovery then requires SSH/console access, `kubectl` cannot reach the node.
+:::warning Keep the Fluent Bit toleration on later upgrades
+The toleration is stored in the `observability-logs-opensearch` release's values, so it must be present on every upgrade that re-renders the Fluent Bit DaemonSet. An upgrade with `--reset-values`, or one that passes new `--set`/`-f` values without `--reuse-values`, drops it unless you pass `--set "fluent-bit.tolerations[0].operator=Exists"` again. Agents on the Kata node then stop producing logs with no error.
 :::
 
-</details>
-
 ### Local development
 
 There is no macOS-local path for Kata (no nested virtualization, and k3d node containers cannot boot Kata VMs). On a KVM-capable **Linux** host running the repository's dev k3d cluster:
@@ -200,13 +169,10 @@ The RuntimeClass exists but containerd on the node has no `kata-qemu` runtime 
 ```bash
 kubectl -n kube-system rollout status daemonset/kata-deploy
 kubectl -n kube-system logs -l name=kata-deploy
-
-# on the node — confirm the runtime is in containerd's config:
-sudo grep -n kata-qemu /var/lib/rancher/k3s/agent/etc/containerd/config.toml   # k3s nodes
-grep -n kata-qemu /etc/containerd/config.toml                                  # other nodes
+kubectl get node <node-name> -L katacontainers.io/kata-runtime   # true once kata-deploy has wired the node
 ```
 
-On k3s nodes, re-running `install-kata.sh` re-applies the containerd template wiring.
+Re-running `install-kata.sh` is safe: it re-applies the kata-deploy chart and boots a test pod on every target node again, failing with the reason if Kata still cannot run there.
 
 </details>
 

package.json

@@ -127,6 +127,10 @@
     },
     "postcss-selector-not": {
       "postcss-selector-parser": "^7.1.4"
-    }
+    },
+    "tinypool": "^2.2.0",
+    "proxy-addr": "^2.0.8",
+    "compression": "^1.8.2",
+    "source-map-js": "^1.2.2"
   }
 }

versioned_docs/version-v1.0.0/guides/isolation-tiers/gvisor.mdx

@@ -115,13 +115,48 @@ kubectl apply -f "${AMP_DIST}/deployments/k8s/gvisor-runtimeclass.yaml"
 # 2. Label and taint the node (find the name with: kubectl get nodes)
 kubectl label node <node-name> gvisor=true --overwrite
 kubectl taint node <node-name> gvisor=true:NoSchedule --overwrite
+```
+
+The label is what the RuntimeClass schedules onto; the taint keeps regular runc pods off the node.
+
+The taint also keeps the Fluent Bit log collector off the node, and without it agents there produce no runtime logs. Fluent Bit is a DaemonSet rendered by the `observability-logs-opensearch` Helm chart, so its toleration is a chart value. Check whether the release stores it:
+
+```bash
+helm get values observability-logs-opensearch -n openchoreo-observability-plane
+```
+
+Check the release values rather than the live DaemonSet: a toleration patched onto the DaemonSet (as earlier versions of the setup scripts did) shows up on the live object, but is lost when the DaemonSet is recreated.
+
+If `fluent-bit.tolerations` does not list `- operator: Exists` with no `key` or `effect`, set the value on the release:
+
+```bash
+helm upgrade observability-logs-opensearch \
+  oci://ghcr.io/openchoreo/helm-charts/observability-logs-opensearch \
+  --namespace openchoreo-observability-plane \
+  --version 0.5.3 \
+  --reuse-values \
+  --set "fluent-bit.tolerations[0].operator=Exists"
+```
+
+Pass the chart version that is already installed (the `CHART` column of `helm list -n openchoreo-observability-plane`): combining `--reuse-values` with a chart version change can carry over values the new chart does not expect.
+
+:::warning Keep the toleration on later upgrades
+The toleration is stored in the release's values, so it must be present on every upgrade that re-renders the DaemonSet. An upgrade of `observability-logs-opensearch` with `--reset-values`, or one that passes new `--set`/`-f` values without `--reuse-values`, drops it unless you pass `--set "fluent-bit.tolerations[0].operator=Exists"` again. Agents on the gVisor node then stop producing logs with no error.
+:::
+
+<details>
+<summary>Logs module not installed as a Helm release</summary>
 
-# 3. Let the Fluent Bit log DaemonSet tolerate the taint (so agent logs are collected)
+Only if `observability-logs-opensearch` is not a Helm release in your cluster, patch the DaemonSet directly:
+
+```bash
 kubectl patch daemonset fluent-bit -n openchoreo-observability-plane --type=json \
   -p='[{"op":"add","path":"/spec/template/spec/tolerations","value":[{"operator":"Exists"}]}]'
 ```
 
-The label is what the RuntimeClass schedules onto; the taint keeps regular runc pods off the node.
+This replaces the DaemonSet's whole tolerations list, and is lost whenever the DaemonSet is recreated (a reinstall of the logs module or a namespace teardown), so re-check the toleration after any such change.
+
+</details>
 
 :::tip Verify
 ```bash
@@ -222,7 +257,7 @@ The Fluent Bit DaemonSet must run on the tainted node. Check it covers the gViso
 kubectl get pods -n openchoreo-observability-plane -o wide | grep fluent-bit
 ```
 
-If no Fluent Bit pod is on the gVisor node, apply the toleration patch from Step 3.
+If no Fluent Bit pod is on the gVisor node, set the Fluent Bit toleration as described in [Step 3](#step-3-register-the-node-for-the-gvisor-tier).
 
 </details>
 

versioned_docs/version-v1.0.0/guides/isolation-tiers/kata.mdx

@@ -93,6 +93,19 @@ What it does:
 5. Applies the `kata-qemu` RuntimeClass and taints the node(s) `kata=true:NoSchedule`.
 6. Patches the Fluent Bit log DaemonSet to tolerate the taint, so agent logs are collected.
 
+:::note Make the Fluent Bit toleration durable
+The v1.0.0 installer adds the toleration by patching the DaemonSet, and a patch is lost whenever the DaemonSet is recreated (a reinstall of the logs module or a namespace teardown). Fluent Bit is rendered by the `observability-logs-opensearch` Helm chart, so set the toleration as a chart value as well, passing the chart version that is already installed (the `CHART` column of `helm list -n openchoreo-observability-plane`):
+
+```bash
+helm upgrade observability-logs-opensearch \
+  oci://ghcr.io/openchoreo/helm-charts/observability-logs-opensearch \
+  --namespace openchoreo-observability-plane \
+  --version 0.5.3 \
+  --reuse-values \
+  --set "fluent-bit.tolerations[0].operator=Exists"
+```
+:::
+
 :::tip Verify
 ```bash
 kubectl get runtimeclass kata-qemu
@@ -101,6 +114,10 @@ kubectl get node <node-name> --show-labels   # kata=true
 ```
 :::
 
+:::warning Keep the Fluent Bit toleration on later upgrades
+The toleration is stored in the `observability-logs-opensearch` release's values, so it must be present on every upgrade that re-renders the Fluent Bit DaemonSet. An upgrade with `--reset-values`, or one that passes new `--set`/`-f` values without `--reuse-values`, drops it unless you pass `--set "fluent-bit.tolerations[0].operator=Exists"` again. Agents on the Kata node then stop producing logs with no error.
+:::
+
 <details>
 <summary>RKE2 nodes (bundled containerd)</summary>