How To Run Kube-Bench In Platform9 Managed Kubernetes Clusters

Problem

How To Run Kube-Bench In Platform9 Managed Kubernetes Clusters?

Environment

• Platform9 Managed Kubernetes - 5.3

• K8s version 1.20

Procedure

1. Checkout the kube-bench repo~# git clone --single-branch --branch private/platform9-1.20.5 https://github.com/platform9/kube-bench.git

2. Switch to the kube-bench directory~# cd kube-bench

3. Create a kube-bench image and ensure it's available for cluster nodes. 3A. By pushing image to repository: a. Enter your docker hub username and password when prompted ~# docker login b. Build the image locally: ~# docker build -t kube-bench:latest .

Note: You will have to replace with the docker hub username if you want to push it later else you can use the following command: ~# docker build -t /kube-bench:latest .

c. Once the above command is successful, push the image to your dockerhub repo ~# docker push /kube-bench:latest

- To change the username - [https://docs.docker.com/engine/reference/commandline/push/](https://docs.docker.com/engine/reference/commandline/push/)
- To push image to a private registry replace <UserName> with localhost:5100

Note: Skip this step(step #3A) if you have no intention of pushing the image to your dockerhub repo. You may proceed with the below. OR 3B. Without pushing image to repository: a. Check the accurate image name with tag - https://docs.docker.com/engine/reference/commandline/save/ ~# docker images

b. Create a gzip tarball file of your image: ~# docker save /kube-bench:latest | gzip > kube-bench.tar.gz

c. Copy the image tarball to destination node using SCP. ~# scp kube-bench.tar.gz <user@destination:/path/to/user/dir>

d. Load this image on a different node, use this: ~# docker load -i kube-bench.tar.gz

e. Run docker images on the destination node and you should see the kube-bench image ~# docker images | grep "kube-bench"

1. Ensure the following files are present in kubebench/cfg/pmk-1.0/ kube-bench]# ls cfg/pmk-1.0/ config.yaml controlplane.yaml etcd.yaml master.yaml node.yaml policies.yaml

2. Running the benchmark test:

• Make sure the image is present on Master and Worker nodes before testing.

• Test individually on each Master and worker nodes.

a. For Master Node: Master]$ sudo docker run -t --network=host --rm --pid=host -v /etc:/etc:ro -v /var:/var:ro -v /run:/run:ro -v $HOME/kube-bench/cfg:/opt/kube-bench/cfg -v /usr/local/bin/kubectl:/usr/local/mount-from-host/bin/kubectl -v /etc/pf9/kube.d/kubeconfigs/admin.yaml:/.kube/admin.yaml -e KUBECONFIG=/.kube/admin.yaml /kube-bench:latest run --benchmark=pmk-1.0 --targets=master,etcd,controlplane,policies

b. For worker nodes: Worker]$ sudo docker run -t --network=host --rm --pid=host -v /etc:/etc:ro -v /var:/var:ro -v /run:/run:ro -v $HOME/kube-bench/cfg:/opt/kube-bench/cfg -v /usr/local/bin/kubectl:/usr/local/mount-from-host/bin/kubectl -v /etc/pf9/kube.d/kubeconfigs/admin.yaml:/.kube/admin.yaml -e KUBECONFIG=/.kube/admin.yaml /kube-bench:latest run --benchmark=pmk-1.0 --targets=node