Skip to main content

Unleash Your Worker Nodes on OpenShift 4.3 and GCP

With the OpenShift Container Platform (OCP) release 4.1 in June 2019, Red Hat introduced Infrastructure MachineSets. These sets allow you to host only infrastructure components, such as:

  • The default router
  • The container image registry
  • The cluster metrics collection, or monitoring service
  • Cluster aggregated logging

An Infrastructure MachineSet consists of Machine resources (kind: Machine). These Machine resources spin up new virtual machines in your cloud.

Specific Kubernetes labels can be applied to these machines to move one or more of the above mentioned infrastructure components to run on only those machines.

The kicker: The infrastructure nodes do not count towards the number of subscriptions that are required to run the environment!

Unleashing Worker Nodes

Worker nodes in the OCP cluster must be covered by subscriptions and their primary purpose is to run your application workloads.
To free resources from these worker nodes, which normally run the OCP infrastructure components, it is beneficial to move the infrastructure components to dedicated infrastructure nodes.

So let’s get started.

Creating an Infrastructure MachineSet for Production

For a production-ready deployment, it is recommended to deploy three MachineSets at minimum to run infrastructure components. The aggregated logging solution, i.e., ElasticSearch, requires three instances that run on different nodes. Since each MachineSet is assigned to one availability zone of the (public) cloud provider only, deploy three MachineSets at minimum.

For demonstration purposes, we will limit the scope to only one MachineSet in the next section.

Defining the MachineSet Custom Resource for the Google Cloud Platform

Once your OCP cluster is deployed to your Google Cloud Platform (GCP) project, you can create your first MachineSet to move infrastructure components. Sidenote: OCP 4.3 supports the installer provisioned infrastructure (IPI) installation method to pre-existing Virtual Private Clouds (VPC) and subnets. Choose the GCP region in which you deployed your OCP4 cluster. Then, select a GCP zone within that region to deploy the MachineSet.

Note: Double-check that the GCP zone actually exists. I tried to deploy to us-east1-a, which does not exist 😉 Unfortunately, no logs or events revealed this to me. Instead, a kind colleague showed me the light.

Please find the YAML-file machineset1.yaml defining the MachineSet below. Change the following values according to your environment:

  • Replace the string myclus-khb5h with your OCP cluster ID
  • Replace region with the region your OCP cluster is in
  • Replace zone with an (existing 😉 ) GCP zone
  • Replace projectID with your GCP project ID
  • Replace serviceAccounts with your service account
  • The name must be unique in your OCP cluster
kind: MachineSet
  labels: myclus-khb5h infra infra
  name: myclus-khb5h-w-a
  namespace: openshift-machine-api
  replicas: 1
    matchLabels: myclus-khb5h myclus-khb5h-w-a
      creationTimestamp: null
      labels: myclus-khb5h infra infra myclus-khb5h-w-a
          canIPForward: false
            name: gcp-cloud-credentials
          deletionProtection: false
          - autoDelete: true
            boot: true
            image: myclus-khb5h-rhcos-image
            labels: null
            sizeGb: 128
            type: pd-ssd
          kind: GCPMachineProviderSpec
          machineType: n1-standard-4
            creationTimestamp: null
          - network: myclus-khb5h-network
            subnetwork: myclus-khb5h-worker-subnet
          projectID: marek-ocp4-blog
          region: us-east1
          - email: [email protected]
          - myclus-khb5h-infra
            name: worker-user-data
          zone: us-east1-b

Now that the YAML-file is prepared, apply it to your cluster.

oc create -f machineset1.yaml created

You can check that the resource is starting to be created.

oc get machinesets -n openshift-machine-api
myclus-khb5h-infra-a   1         1                             7s
myclus-khb5h-w-b       1         1         1       1           19h
myclus-khb5h-w-c       1         1         1       1           19h
myclus-khb5h-w-d       1         1         1       1           19h

Further insight on the creation process can be gained with oc describe.
It is important to note the output should include the Status: and Events: section; if not, then there is likely an error with the YAML-file.

oc describe machine myclus-khb5h-infra-a -n openshift-machine-api
<output omitted>
    Type:        InternalIP
    Type:        InternalDNS
    Address:     myclus-khb5h-infra-a-69j7n.c.marek-ocp4-blog.internal
    Type:        InternalDNS
  Last Updated:  2020-04-28T14:46:43Z
  Phase:         Provisioned
  Provider Status:
      Last Probe Time:       2020-04-28T14:46:23Z
      Last Transition Time:  2020-04-28T14:46:23Z
      Message:               machine successfully created
      Reason:                MachineCreationSucceeded
      Status:                True
      Type:                  MachineCreated
    Instance Id:             myclus-khb5h-infra-a-69j7n
    Instance State:          RUNNING
      Creation Timestamp:  <nil>
  Type     Reason        Age                From           Message
  ----     ------        ----               ----           -------
  Warning  FailedCreate  14m                gcpcontroller  requeue in: 20s
  Warning  FailedUpdate  14m (x4 over 14m)  gcpcontroller  requeue in: 20s
  Normal   Update        14m (x3 over 14m)  gcpcontroller  Updated Machine myclus-khb5h-infra-a-69j7n

Now, the GCP console shows the new instance.

Moving the Container Image Registry

To free resources from the worker node, let’s move the container image registry to the newly created infrastructure node.

Since the image registry resource already exists, we will edit the existing config/cluster object and add the infra nodeSelector to move the registry to our new infrastructure node.

oc edit config/cluster

# Add these two lines to the spec: section
  nodeSelector: ""
# Save and exit the file edited

Watch the resources being moved:

watch -n 1 'oc get pods -n openshift-image-registry -o wide'

# Following output is edited for brevity
NAME                                            READY   STATUS    AGE   IP            NODE
cluster-image-registry-operator-9754995-rg2n5   2/2     Running   21h   myclus-khb5h-m-2.c.marek-ocp4-blog.internal
image-registry-75b4bd664f-rvrn5                 0/1     Pending   28s   <none>        <none>
image-registry-dd874db66-29hzp                  1/1     Running   21h    myclus-khb5h-w-c-d94s4.c.marek-ocp4-blog.internal

After a few moments, the original image-registry pod will be removed.

Getting Started

As you can see, after Infrastructure MachineSets have been created, existing OCP infrastructure components can be moved easily to the dedicated infrastructure nodes.

Next, try to move the cluster monitoring service, cluster aggregated logging, or the default router to your new infrastructure nodes.

You are now ready to apply this technique in your new and existing OCP4 clusters.

Interested in learning more about the OpenShift journey? //take the first step

Share this story

Arctiq Team

We service innovation.