Skip to main content

Curator Deployment in OpenShift Enterprise 3.1 and 3.2

If you have an OpenShift (Origin or Enterprise) environment, you have likely deployed the EFK (elasticsearch, fluentd, and kibana) stack to help operators and developers easily view log files. Referred to as the Aggregated Container Logs in OSE, this stack is very useful and should be deployed in most situations. But log data tends to grow rapidly, especially in development environments with a large amount of projects and containers being deployed in a CI/CD pipeline, and cleanup is necessary; enter curator.

Curator allows operators to define how long elasticsearch indices should be retained. On a defined daily schedule it will purge any qualified indices from elasticsearch. Unfortunately, curator has only been added into the OpenShift Origin version of software, and is not yet included in the OpenShift Enterprise deployments (as of 3.2). Fortunately, version 3.1.1 and 3.2 of the EFK images do include admin credentials that can be used to authenticate a manually deployed curator template.

A few notes about these steps:

  • Commands are run with a cluster-admin authorization
  • The project we are using for logging is called ‘logging’
  • The logging-es deployment configuration specifies version 3.1.1 or 3.2
  • We are setting the defaults with environment variables within the yaml file, which can be done outside of this file as well
  • We are using v1.2.0 of the origin-logging-curator image here https://hub.docker.com/r/openshift/origin-logging-curator
  1. Create the aggregated-logging-curator service account:

oc project logging oc create -f – <<API apiVersion: v1 kind: ServiceAccount metadata: name: aggregated-logging-curator secrets:

  • name: aggregated-logging-curator API
  1. Extract admin keys from elasticsearch deployment and create logging-curator secret, ensuring to replace the unique ID with your instance ID:

oc exec logging-es- cat /etc/elasticsearch/keys/admin-ca | tee es-admin-ca oc exec logging-es- cat /etc/elasticsearch/keys/admin-cert | tee es-admin-cert oc exec logging-es- cat /etc/elasticsearch/keys/admin-key | tee es-admin-key oc secret new logging-curator ca=es-admin-ca cert=es-admin-cert key=es-admin-key

  1. Create curator template named curator-template.yaml with the following content:

apiVersion: v1 kind: Template labels: component: curator logging-infra: curator provider: openshift metadata: annotations: description: Template for logging curator deployment. openshift.io/generated-by: OpenShiftNewApp tags: infrastructure labels: logging-infra: curator name: logging-curator-template objects:

  • apiVersion: v1 kind: DeploymentConfig metadata: labels: component: curator provider: openshift name: logging-curator spec: replicas: 1 selector: component: curator provider: openshift strategy: resources: {} rollingParams: intervalSeconds: 1 timeoutSeconds: 600 updatePeriodSeconds: 1 type: Recreate template: metadata: labels: component: curator provider: openshift name: curator spec: containers:
    • env:
    • name: certs secret: secretName: logging-curator triggers:
      • type: ConfigChange
      • imageChangeParams: automatic: true containerNames:
    • curator from: kind: ImageStreamTag name: logging-curator:${IMAGE_VERSION} type: ImageChange parameters:
  • description: The version tag of the image to use. name: IMAGE_VERSION value: v1.2.0
  • name: IMAGE_PREFIX value: docker.io/openshift/origin-
  1. Create and deploy the curator pod :

oc project logging oc create -f curator-template.yaml oc new-app logging-curator-template oc deploy logging-curator –latest

In order to customize the retention on a per-project basis, you can create a yaml file and pass it to the curator deployment configuration. An example of this file would be:

myapp-dev: delete: days: 1

myapp-qe: delete: weeks: 1

.operations: delete: weeks: 8

.defaults: delete: days: 30 runhour: 0 runminute: 0

Once created, modify the deployment config to include this file:

oc secrets new index-management settings=</path/to/your/yaml/file> oc volumes dc/logging-curator –add –type=secret –secret-name=index-management –mount-path=/etc/curator –name=index-management –overwrite oc deploy logging-curator –latest

If all is well, the logs of the curator pod should show something similar to the following:

logging-curator running [1] jobs No indices matched provided args: {‘regex’: None, ‘index’: (), ‘suffix’: None, ‘newer_than’: None, ‘closed_only’: False, ‘prefix’: None, ‘timeunit’: ‘days’, ‘timestring’: u’%Y.%m.%d’, ‘exclude’: (u’.searchguard‘, u’.kibana‘, u’.apiman*’), ‘older_than’: 30, ‘all_indices’: False} logging-curator run finish

Check here for more detail on the Origin logging deployment: https://github.com/openshift/origin-aggregated-logging

Check here for more detail on the OpenShift Enterprise logging deployment: https://docs.openshift.com/enterprise/3.2/install_config/aggregate_logging.html

Share this story

Arctiq Team

We service innovation.