Long gone are the days when developers worked with a single Kubernetes cluster. As the complexity of applications and infrastructure grows, so does the need for multiple Kubernetes clusters. Managing multiple clusters can be a daunting task, but with the right tools, life can be much easier.
In this article, we'll look at some of the best tools for managing multiple Kubernetes clusters.
1. kubie
Kubie is one of my favorite CLI tools, and most likely my top most used. It's a cross-platform tool that helps you switch between Kubernetes contexts a lot faster than kubectl config set-context.
By default, Kubie reads all kubeconfig in your ~/.kube folder, so you can store each cluster on a different file, and then switch between by using kubie ctx and selecting the desired context using fuzzy search and arrow keys.
From my experience, this is the fastest way to switch between contexts, and it's a must-have tool for anyone working with multiple clusters.
2. kubectl-foreach
kubectl-foreach is a kubectl plugin that allows you to run a command across multiple Kubernetes clusters. This is particularly useful when you need to run the same command on multiple clusters, for example, to get the status of a deployment or to delete a resource.
Example:
kubectl foreach /-prod/ minikube -- get pods -n kube-system --selector compute.twitter.com/app=coredns --no-headers
eu-prod | coredns-59bd9867bb-6rbx7 2/2 Running 0 78d
eu-prod | coredns-59bd9867bb-9xczh 2/2 Running 0 78d
eu-prod | coredns-59bd9867bb-fvn6t 2/2 Running 0 78d
minikube | No resources found in kube-system namespace.
useast-prod | coredns-6fd4bd9db4-7w9wv 2/2 Running 0 78d
useast-prod | coredns-6fd4bd9db4-9pk8n 2/2 Running 0 78d
useast-prod | coredns-6fd4bd9db4-xphr4 2/2 Running 0 78d
uswest-prod | coredns-6f987df9bc-6fgc2 2/2 Running 0 78d
uswest-prod | coredns-6f987df9bc-9gxvt 2/2 Running 0 78d
uswest-prod | coredns-6f987df9bc-d88jk 2/2 Running 0 78d
The command can get quite long, but it's definitely much faster than running the same command on each cluster individually, considering you have to switch contexts every time.
3. Aptakube
Aptakube is like if the Kubernetes Dashboard, Kubie and kubectl-foreach had a baby. It's a desktop-based GUI that allows you to connect to multiple Kubernetes clusters simultaneously, without having to switch contexts.
Aptakube is a fantastic tool if you need to monitor multiple clusters at the same time, especially if you like a more visual approach to Kubernetes than the CLI.
The screenshot below shows Aptakube connected to two different clusters, and you can see the resources from both clusters in the same window.
4. kcf
kcf is another kubectl plugin that's somewhat similar to kubectl-foreach, but is limited to a single command. kubectl fleet will connect to all your configured clusters and run a few commands on each of them to build a status overview of all clusters.
Example:
$ kubectl fleet
CLUSTER VERSION NODES NAMESPACES PROVIDER API
kind-kind-3 v1.16.3 1/1 4 kind https://127.0.0.1:32769
test-cluster-2 v1.16.2 1/1 4 minikube https://192.168.64.4:8443
kind-test2 v1.16.3 1/1 4 kind https://127.0.0.1:32768
minikube v1.16.2 1/1 4 minikube https://192.168.64.3:8443
gke_krew-release-bot-260708_us-central1-a_standard-cluster-1 v1.15.8-gke.3 3/3 4 GKE https://104.197.42.183
do-sfo2-k8s-1-16-6-do-0-sfo2-1581265844177 v1.16.6 3/3 4 Digital Ocean https://f048f314-4f77-47c2-9264-764da91d35e0.k8s.ondigitalocean.com
It's quite limited in terms of what you can do, but I love the simplicity of this tool. It's a great way to get a quick overview of all your clusters.
5. kubectl-config-doctor
kubectl-config-doctor is yet another kubectl plugin. This one is a bit different, as it helps you diagnose issues with your kubeconfig files. It will check for common issues such as:
- Duplicate contexts/clusters/users
- Missing contexts/clusters/users
- Incorrectly formatted kubeconfig files
- Unreachable proxy api
- Unreachable cluster api
- Invalid authentication methods
- Files that don't exist
- ... and more!
Example:
$ kubectl config-doctor
I find this plugin particularly useful when I'm setting up a new cluster, or when I'm helping my customers troubleshoot connectivity issues with their clusters. Sometimes the issue is as simple as a typo in the kubeconfig file, and this tool can help you identify it much faster than manually checking the file.
Conclusion
I hope you found this list of tools helpful. Do you have a favorite tool for working with multiple Kubernetes clusters? I'd love to hear about it, send me a tweet at @goenning.
Until next time! 👋
