IT & Test Environment Management Anti Patterns #2


By Sylvia Fronczak.  

Anti-patterns often arise when developers find creative or outside-the-box uses for a technology – or when they find workarounds for limitations that were specifically put in place by the creators of the technology to prevent misuse. Application container platforms like Docker have been the hot new thing for several years now, but they’re still fairly new concepts to many IT environment managers. And since they’ve been around for a while, there are a number of anti-patterns that environment managers have fallen into. In this post, we follow on from our previous article on Test Environment Anti Patterns and talk about some of the common anti-patterns of containers and consider why they exist.

1. Confusing Application Containers with System Containers or VMs

Some developers make the mistake of comparing application containers with small VMs. This may seem inconsequential, but it actually lays a dangerous frame of reference.


The comparison between containers and VMs leads to developers running multiple processes and services, or even operating systems within application containers. However, containers don’t work like VMs. They should run one process or service. System containers like LXD or OpenVZ explicitly allow developers to install operating systems inside the container. They operate much like virtual machines. However, VMs emulate hardware while system containers do not. What does this mean for us? If you’re looking to run Windows on your Linux, you’ll want a virtual machine. If you want to run multiple types of Linux-based operating systems on your server instead, then you can use system containers. Why is this an anti-pattern? Application containers isolate processes; they don’t virtualize operating systems. They encapsulate parts of your application ecosystem into standalone deployable parts. If I have a Docker container that includes my web services, database, and message bus, I’m unable to scale or deploy different components of my stack. And that results in blowing away one of the main features of containerization.


An application container should have only one service or process running in it. And that’s it.

2. Creating Tightly Coupled Dependencies

So now you’re ready to use application containers to deploy and run your application. You’ve properly segregated your services and functions into separate containers. What anti-pattern might you fall into next?


Coupling your containers to each other may seem innocent at first. It’s as simple as adding a depends-on or waits-for-it to your Docker configuration. However, this can result in long deployments, circular dependency, and reduced availability.


To get the most benefits from containerization, your containers should be independently runnable. This lets you destroy and create containers whenever necessary and lets you scale horizontally. But what does independently runnable mean? Does it mean that my one process can’t be dependent on anything else? That would be fairly difficult to create meaningful apps this way. It means that your container and the process inside should not be dependent on anything else when starting up. Your container should never wait for other containers or dependencies to be up or accessible upon startup. And again, this doesn’t mean that your application can’t depend on anything else. In fact, it should still throw errors if someone is using the container’s app and a critical dependency is down. This can be achieved in different ways. For one, don’t try to connect to another dependency on startup. Alternatively, consider using message buses to keep containers independently of each other.

3. Modifying Running Containers

Our next anti-pattern scolds those of us that like changing things on the fly.


Now that you have an independently running process in each of your containers, you may be tempted to update application configuration or even the application code inside.


However, a much better practice involves deploying a brand new version of the container. You should consider the container and the process running inside to be immutable. If you need to change a configuration, then change it in your Docker file or Helm chart and deploy the new version. If you want to change the application running, then deploy a new container with the new version of your code.

4. Including Unnecessary Services

Even if you follow the good patterns above, you may be tempted to add just a small service here or there to your containers.


Developers strive to make life easier for themselves. That includes adding tools and small services that are unrelated to the core functionality of a container. For example, a common anti-pattern here involves adding an SSH daemon to your container image. Not only does it encourage developers to change or tweak containers once they’re running, but they create another way for someone to try to gain access to your container. And if you’re not following the anti-patterns above, there should be no need for having an SSH daemon on your container anyway. The more services you include in your container, the more bloated it becomes. Also, those extra services provide additional attack vectors for hackers.


Keep your containers light and simple. Don’t add anything that isn’t necessary to what functionality your service provides.

5. Persisting Data Inside a Container or Image Registry

It may seem easy and convenient to store data inside our containers. But once again, that’s a big no-no.


As a reminder, containers are ephemeral. They spin up and come down fairly frequently. They should never be considered as stable or long-lasting. If you rely on data or logs that are available only in the container, then you’re going to have a bad time. Data doesn’t persist in containers by default. Additionally, you’re introducing another attack vector or security flaw. Anyone that gains access to the container or image registry may also gain access to your data.


Data shouldn’t be stored in containers. Instead, use Docker volumes or an external dependency to store your data and logs.

6. Relying on Backups and Restores of Containers

Our next anti-pattern might seem harmless at first. After all, we’ve all been trained to make sure everything is always backed up in case of emergencies. What’s wrong with that?


In and of themselves, backups are useful. However, some developers use them as a shortcut to deployments. They take backups of existing containers and restore them on new servers—circumventing their deployment pipeline.


There are some legitimate reasons why you may want to make a backup of your container. For example, in the event of a security breach in a container, you may want to back it up before destroying it for forensic analysis. However, restoring backups should not be part of your normal CI/CD or disaster recovery process. Instead, build and deploy a new version of your container and related services.

7. Giving Little Thought to Backward Compatibility

Whether your containers are directly dependent on other containers or dependencies like Kafka or RabbitMQ, you should always consider how changes will affect other systems.


The anti-pattern here involves relying on deployment order to coordinate upgrades and changes across containers and dependencies. One big example many of us are guilty of involves forgetting backward compatibility for database changes. If your web application relies on a database, it may be tempting to make changes in both and then coordinate deployments so that both are upgraded at the same time. Another step may involve a maintenance window. Here, services are taken down or made inaccessible. This goes against the high-availability promise of containerization.


When making changes to parts of your system, consider ways in which upgrading one or the other independently will work just fine. This may involve extra future deployments to remove unused code or columns, but it ensures more availability for your services. Give some thought to ways of enhancing and modifying systems so that other older versions of dependencies can still work—even if only temporarily.

8. Assuming Containers Will Fix Everything

I’d like to close out this list with an anti-pattern that’s less technical in nature. As lovers of technology, we’re often looking for the next great thing that will solve the limitations of our current platforms and tools. And this forward-looking experimentation is one of our strengths. However, we also need to be realistic about the limitations of the tools we use, like application containers. They were built to solve a specific set of problems. And if we try to circumvent that by creating workarounds, we’ll end up creating tomorrow’s anti-pattern.  
Author: Sylvia Fronczak.   This post was written by Sylvia Fronczak. Sylvia is a software developer that has worked in various industries with various software methodologies. She’s currently focused on design practices that the whole team can own, understand, and evolve over time.  

Relevant Articles

Test Environment Management – Modelling

30JANUARY, 2023 by Jane TemovTest Environment Management (TEM) is an essential process for ensuring the stability and consistency of the testing environment. It includes activities such as setting up the environment, monitoring and controlling the environment, and...

What is Deployment Planning?

15DECEMBER, 2022 by Jane TemovDeployment planning is the process of creating a plan for the successful deployment of a new software or system. It involves identifying the resources, tasks, and timeline needed to ensure that the deployment is successful. Deployment...

Why CICD & Test Environment Management Goes Hand-in-Hand

12DECEMBER, 2022 by Jane TemovWhy CICD & TEM Goes Hand-in-Hand Continuous Integration/Continuous Delivery (CICD) and Test Environment Management are two essential components of a successful software development process. CICD enables teams to deploy new code...

Enov8 Releases their Latest “Evaluation Edition”

08DECEMBER, 2022 by Enov8Enov8 is happy to announce the latest “evaluation”* edition is ready for consumption. *A complete Release & Environment Management product with a full license for 3 months. Our Release & Environment Management solution is designed to...

The Agile Release Train Explained

04DEC, 2022 by Jane TemovIf your organization is starting an agile transformation, you might be looking at it as an opportunity. Or perhaps you’re looking at it with some healthy skepticism.  Either is understandable—or even both at the same time. The opportunity...

Self-Healing Applications

02NOVEMBER, 2022 by Sylvia Froncza Original March 11 2019An IT and Test Environment Perspective Traditionally, test environments have been difficult to manage. For one, data exists in unpredictable or unknown states. Additionally, various applications and services...