In the recent past, organizations used to build Docker images outside the Kubernetes cluster. With more companies adopting the Kubernetes technology, the idea of continuous integration builds within the cluster has increased.
However, building and running containers within a cluster pose a security threat since the container has to access the file system of the worker node to connect with the docker daemon. In addition to that, the containers need to run in a privileged mode which exposes the node to innumerable security threats.
This problem has been solved by Google with the introduction of Kaniko. This tool helps one to build container images from a docker file without any access to the docker daemon. It executes all the commands within the Dockerfile completely in the userspace without allowing any access to the host file system.
Kaniko works in the following ways:
- It reads the specified Dockerfile, build context or a remote Docker registry
- Proceeds to extract the base image into the container filesystem
- Runs the commands in the Dockerfile individually.
- After every run, a snapshot of the userspace filesystem is taken
- After every snapshot, it appends only the changed image layers to the base image and updates the image metadata
- Then the image is pushed to the registry
The illustration below will help you understand how Kaniko works.
Before You Begin
This guide requires one to have a Kubernetes cluster with permissions to create, update, list, and delete a job, pods, services, and secrets. There are several guides to help you deploy a Kubernetes cluster.
- Install Kubernetes Cluster on Rocky Linux 8 with Kubeadm & CRI-O
- Install Kubernetes Cluster on Ubuntu using K3s
- Deploy Kubernetes Cluster on Linux With k0s
- Run Kubernetes on Debian with Minikube
- Install Kubernetes Cluster on Ubuntu with kubeadm
In this guide, we will build container images from Dockerfile using Kaniko in Kubernetes with Github and Docker Registry. So, you need the following:
- Github repository with a Dockerfile: we will use the repo URL as the path of the Dockerfile
- Docker hub account: to be able to authenticate and push the Docker image.
- Access to Kubernetes cluster: to be able to deploy the Kaniko pod and create the docker registry secret.
Step 1 – Create the Container Registry Secret
We will start off by setting up the container registry secret. This is useful when pushing the built image to the registry. There are several registries supported by Kaniko. These are:
- Docker Hub
- Google GCR
- Amazon ECR
- Azure Container Registry
- JFrog Container Registry/JFrog Artifactory
These registries are declared using the –destination flag in the manifest. In this guide, we will push the image to the Dockerhub registry.
I assume that you already have a Dockerhub account.
Now create the registry secret using the command:
kubectl create secret docker-registry dockercred \ --docker-server=https://index.docker.io/v1/ \ --docker-username=<dockerhub-username> \ --docker-password=<dockerhub-password>\ --docker-email=<dockerhub-email>
After this, you should have the secret deployed
$ kubectl get secrets NAME TYPE DATA AGE default-token-ncn6w kubernetes.io/service-account-token 3 3m48s dockercred kubernetes.io/dockerconfigjson 1 15s
Step 2 – Configure Kaniko Build Contexts
In the build context, the following types are supported by Kaniko:
- S3 Bucket
- Local Directory
- GCS Bucket
- Azure Blob Storage
- Local Tar
- Git Repository
- Standard Input
The build context represents your directory containing the Dockerfile to be used to build your image.
You can use any of the supported types by specifying them in the manifest using the –context flag. The prefix to be used are:
|Local Directory||dir://[path to a directory in the kaniko container]||dir:///workspace|
|Local Tar Gz||tar://[path to a .tar.gz in the kaniko container]||tar://path/to/context.tar.gz|
|GCS Bucket||gs://[bucket name]/[path to .tar.gz]||gs://kaniko-bucket/path/to/context.tar.gz|
|Azure Blob Storage||https://[account].[azureblobhostsuffix]/[container]/[path to .tar.gz]||https://myaccount.blob.core.windows.net/container/path/to/context.tar.gz|
|Git Repository||git://[repository url][#reference][#commit-id]||git://github.com/acme/myproject.git#refs/heads/mybranch#|
|S3 Bucket||s3://[bucket name]/[path to .tar.gz]||s3://kaniko-bucket/path/to/context.tar.gz|
In this guide, We will use a GitHub repository as our build context. So we need to configure it with the required Dockerfile.
For this guide, we will use a private Git, repository with a Dockerfile. The repo URL is:
The repository will have a Dockerfile with the below content
FROM ubuntu ENTRYPOINT ["/bin/bash", "-c", "echo hello"]
The repository will have the file as shown.
Step 3 – Create the Kaniko Pod Manifest
So we will create our manifest below.
Remember to replace the values appropriately.
apiVersion: v1 kind: Pod metadata: name: kaniko spec: containers: - name: kaniko image: gcr.io/kaniko-project/executor:latest args: - "--context=git://<Git API token>@github.com/computingforgeeks/kubernetes-kaniko.git#refs/heads/master" - "--destination=<dockerhub-username>/kaniko-demo-image:1.0" volumeMounts: - name: kaniko-secret mountPath: /kaniko/.docker restartPolicy: Never volumes: - name: kaniko-secret secret: secretName: dockercred items: - key: .dockerconfigjson path: config.json
In the above file, we have used the Git API token to authenticate the private git repository. This can be avoided if you are using a public repository. The –destination is the location we want to push the image. For example in my case, the location will be klinsmann1/kaniko-demo-image:1.0.
Step 4 – Run Kaniko in Kubernetes
With the manifest created as desired, run Kaniko with the command:
kubectl apply -f pod.yaml
Follow the image build and push process.
kubectl logs kaniko --follow
After this, I will have my image pushed to Dockerhub.
Step 5 – Pull and Test the Image
Now the image can be used to run containers on both Kubernetes and Docker.
Begin by installing Docker on your system. The below guide can help you achieve this:
Now run a container with the image using the command:
docker run -it <user-name>/<repo-name>
docker run -it klinsmann1/kaniko-demo-image:1.0
The image can also be pulled and tested on Kubernetes. Create a deployment file as shown.
$ vim deploy.yml apiVersion: apps/v1 kind: Deployment metadata: name: hello-world spec: selector: matchLabels: app: hello replicas: 1 template: metadata: labels: app: hello spec: containers: - name: hello-world image: klinsmann1/kaniko-demo-image:1.0
Apply the manifest:
kubectl apply -f deploy.yml
Check the status:
$ kubectl get pods NAME READY STATUS RESTARTS AGE hello-world-7f67c97454-xv8xs 0/1 Completed 2 (18s ago) 19s kaniko 0/1 Completed 0 32m
Check if the execution was successful.
$ kubectl logs hello-world-7f67c97454-xv8xs --follow hello
We have triumphantly walked through how to build container images from Dockerfile using Kaniko in Kubernetes. Now proceed and build your images and use them within your cluster.