How to Skip TLS Verify for Internal Registry on Containered

If you like me are working at a company, probably you have to store your artifacts internally. Well, you don’t have to, but it is a best practice. You would be laying some of your system’s codes and patterns available on the internet and it is not the best way to store artifacts from either security or economic perspective — economics related to the constant package transfering through your internet link.

A little while ago I’ve updated my EKS cluster to version 1.23 and decided to start using containerd runtime already, since it will be mandatory on version 1.24.

At first, everything seemed to be working as it should but after I applied the AMI update to the staging environment, some analysts started reporting about pods that wouldn’t run.

Problem

Long story short, after some analysis, I got to the cause: pods on containerd runtime wasn’t able to get images from our internal registry due to our certificate not having a chain related to a Certification Authority (CA).

If there is an error on the node sites such as: “certificate signed by unknown authority”. That is the reason a pod isn’t able to run.

If you want to make sure that’s the reason your pod won’t run, you can simply add a -k at the end and it should work like a charm.

But what about our pod? For that we don’t have a way to use the “-k”. That’s the reason I’m writing, since I didn’t find any tutorial related.

Solution

So, in order to configure your containerd to skip TLS verification it’s a little trickier than in docker. For docker, you just need to add the “insecure-registry” information on the daemon.json and it’s done.

1. You have to ensure that the “certs.d” folder is configured in your /etc/containerd/config.toml file.

[plugins.”io.containerd.grpc.v1.cri”.registry] config_path = “/etc/containerd/certs.d:/etc/docker/certs.d”

2. If this folder isn’t present on your system, you have to create it.

# mkdir /etc/containerd/certs.d

3. Inside the certs.d folder, you have do create a folder named as your registry’s name

# mkdir /etc/containerd/certs.d/myregistry.local

4. Inside this folder, you create a file named “hosts.toml” and set the instructions like below

server = “https://myregistry.local”

[host.”https://myregistry.local”]   skip_verify = true

That’s all! After that, restart the containerd service to be sure that all have been set and try again.

Automating

Now if, as well as me, you got to this after you upgraded your EKS version and need to put this on your EKS node group user data, we can write a script like this:

# mkdir -p /etc/containerd/certs.d/myregistry.local

# cat > /etc/containerd/certs.d/myregistry.local/hosts.toml <<EOF
  server = “https://myregistry.local”

  [host.”https://myregistry.local”]     skip_verify = true
  EOF

Conclusion

EKS is going to replace the default runtime from docker to containerd in its 1.24 version and if I were you, I would get this replacement done as soon as possible to get rid of these little things that could cause you some headaches in the future.