Skip to main content

Paying Technical Debt Produces Returns (Technical Details)

Modernization journeys drive tangible results for your end-users.

Recently, Arctiq completed a project that netted some substantial performance improvements for one of our clients’ consumer-facing websites. These improvements not only saved an average of 1.5 seconds per page load, they also provided stability and enhanced visibility into overall systems performance during the recent Black Friday & Cyber Monday events.

* This graph is indicative of the results realized, not actual client data.

There are many possible reasons to take on a new project. I think the majority of new projects I’ve seen in my career can be simplified into two categories: A project is either in the pursuit of new functionality to support a business strategy, or the project is intended to address accrued technical debt. This client’s project fell into the latter category.

The client has several consumer-facing websites that are perceived to deliver a suboptimal user experience and when they investigated the concerns they found some issues that needed to be addressed. These systems were built with all the best practices of the day, but those days had passed and the current state warranted attention as a number of components were falling behind in terms of their currency.

In pursuit of improving their consumer experience and their overall risk posture, our client engaged us to assess some components of the application delivery platform. The primary objective was to identify the accrued technical debt so that we could plan a program to address it.

Our assessment of the existing environment was comprehensive. It included:

  • Current deployment automation
  • Looking at deltas between prod and non-prod
  • Investigating a bunch of infrastructure (workload placement and lifecycle, capacities, etc)
  • Looking at the network design (north-south & east-west, as well as looking into slow paths)
  • Security controls
  • Logging and telemetry
  • Failure mode analysis
  • Software currency

With this assessment we identified a series of achievable activities that could be undertaken to modernize the platform and address any deficiencies that may have been identified.

Arctiq subsequently formed a collaborative program with the client’s project sponsor and project lead and delivered the modernization program of activities identified in the assessment. The primary goals of the modernization program were to improve the consumer experience with their properties and to address the risk associated with the aging solution.

This comprehensive modernization of the platform applied the latest practices and took advantage of the capabilities of the cloud and software partners involved and we helped this client realize significant benefits from their investments.

Arctiq’s program delivered a few notable outcomes:

    • End-user experience improved
    • Software currency resolved
    • Latest versions of NGINX software delivered with a containerization strategy
    • Kubernetes platform will auto upgrade to stay current
    • Reliability improved
    • Google Cloud Kubernetes Engine adopted across multiple regions with automatic workload management and scaling
    • Platform upgrades are rolling and do not affect workload
    • Security of platform improved
    • Arctiq integrated Google Cloud Platform’s Cloud Armor to ensure that security policies are enforced at the edge of the network before a bad actor can affect the core services
    • Secrets management using HashiCorp Vault integrated into GitOps workflow and Kubernetes workload deployments to improve security posture
    • Logging and telemetry improved
    • Adopting Google Cloud’s Cloud Logging capabilities and modifying the NGINX logging to conform to Structured Logging
    • Adoption of Google Cloud’s Cloud Monitoring capabilities for enhanced visibility and observability of the metrics and telemetry of systems under management
    • Automation improvements
    • GitOps workflow created with GitHub repositories and Google Cloud’s Cloud Build to provide continuous deployments
    • Workflow was created to enable both the legacy and new solutions to coexist in parallel using a single repository as the source of truth for the customer to manage configuration. This avoided the need for the developers to maintain different repositories for NGINX configs and improve their experience and efficiency in working with the modernized system
    • HashiCorp Terraform was used in the Cloud Build workflows to create predictable and repeatable declarative deployments
    • Change & Risk Management

We managed the risk profile of changes by using a phased approach that took advantage of the traffic management features of Google Cloud Load Balancing through the use of a variety of Network Endpoint Groups and URL maps. Using these techniques, we kept traffic flowing to legacy systems while slowly phasing in new systems

The solution involved the integration of cloud and software components from F5, GitHub, Google Cloud, HashiCorp, and Red Hat.  There are several additional improvements already identified and queued as this client continues on their journey of modernization.

Arctiq is able to deliver modernization and this has paid significant dividends for our client.

An average of 1.5 seconds saved per page load can represent hundreds or thousands of hours of end-user time saved per day if you’re serving millions of views and in addition to improving the end-user experience it can also drive improved financial performance of the properties.  In addition, the client’s operations teams are now enabled with a modern software architecture that scales, upgrades made simpler, and they now have end-to-end observability using the advanced capabilities of the cloud platform.

The majority of the performance improvements that were visible to the end-users of the websites were primarily enabled via the optimization of flows using the Google Cloud Platform and its global Load Balancing capabilities and the Premium Network Tier. The target state architecture was designed to take advantage of all the affordances of the Google Cloud and there are more improvements to come. The software deployment is also designed to autoscale and will withstand sudden influxes of traffic while continuously delivering a great user experience.

The enhanced visibility and observability enabled by using Google Cloud Operations (formerly Stackdriver) has also provided the tools needed to immediately identify areas of interest when managing the systems.  This has enabled the operations staff to pinpoint actionable intelligence quickly and efficiently so they can effectively address the needs of the organization.

If you’ve got a suboptimal user experience within your existing systems, let’s talk.  Arctiq’s experience with advanced software and cloud-native patterns can help you modernize your platform and get the best possible outcomes from your investments.

Share this story

Michael Kennedy

Michael Kennedy is a Principal Solutions Architect at Arctiq.