LAB04b: Taint & Toleration

LAB 18: Taint & Toleration

1️ Taint worker node 2 2️ Try to schedule an nginx Pod using nodeSelector → it should fail 3️ Add toleration → Pod should run successfully

---

STEP 1 — Taint k8s-cluster-w2

kubectl taint nodes k8s-cluster-w2 env=prod:NoSchedule

Verify:

kubectl describe node k8s-cluster-w2 | grep Taints -A3

You should see:

Taints: env=prod:NoSchedule

---

STEP 2 — Try scheduling a Pod with nodeSelector

This Pod will fail because it does not have a toleration.

Run nginx pod targeting worker-2:

kubectl run test-nginx \
  --image=nginx \
  --restart=Never \
  --overrides='
  {
    "apiVersion": "v1",
    "spec": {
      "nodeSelector": {
        "kubernetes.io/hostname": "k8s-cluster-w2"
      }
    }
  }'

Alternatively, you can first check your node labels:

kubectl get nodes --show-labels

Most clusters use:

kubernetes.io/hostname=k8s-cluster-w2

---

STEP 3 — Verify pod is stuck in Pending

kubectl get pods -o wide

And check events:

kubectl describe pod test-nginx

You should see something like:

0/3 nodes are available: 1 node(s) had taint {env: prod}, that the pod didn't tolerate.

This is expected.

---

STEP 4 — Delete the failed Pod

kubectl delete pod test-nginx

---

STEP 5 — Recreate with toleration (should work)

Run with toleration included:

kubectl run test-nginx \
  --image=nginx \
  --restart=Never \
  --overrides='
{
  "apiVersion": "v1",
  "spec": {
    "nodeSelector": {
      "kubernetes.io/hostname": "k8s-cluster-w2"
    },
    "tolerations": [
      {
        "key": "env",
        "operator": "Equal",
        "value": "prod",
        "effect": "NoSchedule"
      }
    ]
  }
}'

---

STEP 6 — Confirm Pod is running

kubectl get pods -o wide

You should now see:

test-nginx   Running   k8s-cluster-w2

---

✔️ Summary of behavior

Pod	Toleration	Result
No toleration	❌	Fails (stays Pending)
With toleration	✅	Schedules on tainted node

To remove (untaint) the taint you added on k8s-cluster-w2, use the exact same taint key/value/effect with a trailing dash -.

---

Remove the taint env=prod:NoSchedule

kubectl taint nodes k8s-cluster-w2 env=prod:NoSchedule-

This deletes the taint from the node.

---

Verify taint is removed

kubectl describe node k8s-cluster-w2 | grep Taints -A3

Expected output:

Taints: <none>

or:

Taints:   (no value)

Also:

kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints

---

⚠️ If you don't remember the exact taint

List all taints:

kubectl describe node k8s-cluster-w2 | grep -A2 Taints

Then remove it using:

kubectl taint nodes k8s-cluster-w2 <key>=<value>:<effect>-

---

Need help checking if any pods become schedulable now?

Just tell me and I’ll guide you.