In OpenShift 4.x environment, each container running will be limited to the default maximum PID value of 1024. If there is a need to run applications with more than 1024 processes within one single container, the OpenShift Container Platform Cluster operator is required to adjust the default maximum PID value to a higher number.
pids_limit is the maximum number of processes allowed in a container, and it can be viewed by running the command below in an OpenShift node:
$ sudo crio-status config | grep pid pids_limit = 1024
In OpenShift, it is NOT recommended to change the value directly by editing crio.conf file:
$ grep pids_limit /etc/crio/crio.conf pids_limit = 1024
It is recommended to follow the correct way that depends on the OCP version in use. I’m running OpenShift 4.4 and in this version,
ContainerRuntimeConfig custom resource was introduced. You can check KCS Article 5133191.
Create ContainerRuntimeConfig custom resource for configuring cri-o pidsLimit
cat <<EOF > custom-pidslimit.yaml apiVersion: machineconfiguration.openshift.io/v1 kind: ContainerRuntimeConfig metadata: name: custom-pidslimit spec: machineConfigPoolSelector: matchLabels: custom-crio: custom-pidslimit containerRuntimeConfig: pidsLimit: 4096 EOF
You can update the configuration file before applying:
$ oc create -f custom-pidslimit.yaml containerruntimeconfig.machineconfiguration.openshift.io/custom-pidslimit created
Verify the resource has been created
$ oc get ctrcfg NAME AGE custom-pidslimit 44s
Once the custom resource is created, we need to roll out the pidslimit changes to all the worker nodes in the cluster.
Let’s add custom-crio: custom-pidslimit under labels in the machineConfigPool config
$ oc edit machineconfigpool worker apiVersion: machineconfiguration.openshift.io/v1 kind: MachineConfigPool metadata: creationTimestamp: "2020-07-15T08:29:58Z" generation: 7 labels: custom-crio: custom-pidslimit #add this line
Check to ensure that a new 99-worker-XXX-containerruntime is created and that a new rendered worker is created:
$ oc get machineconfigs | grep containerruntime 99-worker-261cdd8d-c387-4f61-b1ce-b9ab2d025f09-containerruntime 601c2285f497bf7c73d84737b9977a0e697cb86a 2.2.0 93s
The changes should now be rolled out to each node in the worker pool via that new rendered-worker machine config.
You can verify by checking that the latest rendered-worker machine-config has been rolled out to the pools successfully:
$ oc get mcp NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE master rendered-master-238bb9ffd94d526621cba8ee876c3ac8 True False False 5 5 5 0 216d worker rendered-worker-6c236aa19af4d88fa0acdbc8f6ff53f3 False True True 10 0 0 7 216d
Once a worker node is rebooted you can login and confirm the current setting:
$ oc debug node/<workernode> sh-4.4# chroot /host sh-4.4# grep pids_limit /etc/crio/crio.conf
I hope this short guide was helpful in changing the default value of pids_limit in your OpenShift 4.x cluster.