Enabling the Calico CNI provider with Minikube on Hyper-V

(NOTE: Although this blog post is specifically about Minikube on Windows using the Hyper-V hypervisor, this may be useful for other host operating systems and hypervisors too. YMMV)

(NOTE 2: Since Minikube version 1.7.3 was released, this now works much more smoothly. I've left my instructions in place for versions prior to this for reference).

Minikube is a great tool for local testing of Kubernetes. For me it has replaced Docker for Windows and docker-machine (and while we're on the subject, you can still use local Docker commands against Minikube, by setting up your Docker env vars using minikube docker-env)

minikube start --memory 4096 --network-plugin=cni

This allows Minikube to start up with the CNI plugin enabled. Next is to install the latest Calico provider:

kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

Once all pods are running and passing their readiness checks, you're good to go!

Prior to Minikube 1.7.3

I did, however, come up against an issue recently when trying to test something locally - Network Policies. The default network plugin that Minikube uses is kubenet, and this is fine for most (perhaps the vast majority of) testing purposes. Kubenet does not allow the use of network policies though, and whilst I could test my work on a real cluster, I fancied seeing if I could get Minikube to use CNI so that I could test locally.

This turned out to be much more of a pain than I had expected, mostly due to my complete lack of experience with configuring CNI providers, and also because there was no simple howto guide as far as my Google-fu could tell. Most of my attempts to configure Calico, Flannel and Cilium fell flat with issues where pods could not resolve DNS or make any external connections, or would make the entire cluster inaccessible.

So, here is my list of incantations that finally got Calico working with Minikube on Hyper-V. I hope it works for you!

minikube start --memory 4096 `
               --network-plugin=cni `
               --extra-config=kubelet.network-plugin=cni `
               --extra-config=kubelet.pod-cidr=192.168.0.0/16 `
               --extra-config=controller-manager.allocate-node-cidrs=true `
               --extra-config=controller-manager.cluster-cidr=192.168.0.0/16 `
               --host-only-cidr=172.17.17.1/24

Note that this is using backticks as a line continuation indicator, since I'm using a Powershell console here. Adjust for your environment as appropriate. I also can't vouch for the portability of the network ranges being used here (my lack of knowledge in this area means I don't even know if the ranges chosen even matter).

This allows Minikube to start up with the CNI plugin enabled. Next is to install the Calico provider:

kubectl apply -f https://docs.projectcalico.org/v3.9/manifests/calico.yaml

We're not quite done yet. At this point, the calico-node pod ends up in a state where it isn't passing its readiness check:

NAMESPACE              NAME                                         READY   STATUS    RESTARTS   AGE
kube-system            calico-kube-controllers-6b9d4c8765-rgtts     1/1     Running   0          2m41s
kube-system            calico-node-88jcz                            0/1     Running   1          2m41s
kube-system            coredns-6955765f44-98z5j                     1/1     Running   0          25m

Upon pulling the logs, this reveals that it's due to something called RPF not being enabled or something like that.

int_dataplane.go 1037: Kernel's RPF check is set to 'loose'.  
This would allow endpoints to spoof their IP address.  
Calico requires net.ipv4.conf.all.rp_filter to be set to 0 or 1. 
If you require loose RPF and you are not concerned about spoofing, 
this check can be disabled by setting the IgnoreLooseRPF configuration 
parameter to 'true'.

I don't pretend to fully understand this part, but it's due to a kernel setting on the Minikube host. The check for this can be disabled by using the following magic:

kubectl -n kube-system set env daemonset/calico-node FELIX_IGNORELOOSERPF=true

And voila! The calico-node pod will reinitialise, and when it comes up you'll have (if you're as lucky as me) a fully CNI-enabled Minkube instance where Network Policies will work!