Infrastructure as Code – DevOps best practices in 2021 and beyond!

What is Infrastructure as Code (IaC) and how does it gel with DevOps? What’s the best way of provisioning your IT infrastructure? And most importantly – how does IaC make our development scalable, easier, with greater speed, less risk, and reduced costs?

Last time we’ve discussed DevOps, we had our very own DevOps team put up with our writers’ team, as we grilled them on VMs, Containers and Kubernetes. So much so that we’ve been banned from their part of the office for an undisclosed number of months.

This however did not stop us with bothering them via emails, video chats, and passive aggressive notes in the office fridge.

Just like this, except with questions… in binary.

So, building upon what we’ve learned last time on basics of Virtual Machines and baby steps into Kubernetes deployment; we’re taking another leap into the best practices for DevOps. We’ll be discussing the process of automating the provisioning of Infrastructure, otherwise known as “Infrastructure as Code”.

DevOps summarized

But, before we discuss the main topic of the day, let’s get our lingo straight. In our current IT world, when it comes to DevOps, the first thing you must understand is that DevOps is simply put – a set of practices, culture changes and tools.

DevOps combines software development (Dev) and IT operations (Ops) in order to streamline development lifecycles, make them faster, cheaper and with a higher degree of quality.

It is not a department, a team, or a task, even though some might understandably think that way. Some companies even designate their team as a “DevOps” department (such as ours).

However, the core of the message states: DevOps is a set of methodologies, which constantly evolve and improve. One of these is the paradigm of CI/CD – continuous integration (CI) and continuous delivery/deployment (CD).

The core principle of CI/CD pipeline is automation, during – building, testing, deployment, and maintenance of applications during their development lifecycle.

In the past, various developers working on the same app and making changes to different parts of it, would often break each other’s code or run into unexpected errors.

The tradition method involved integrating various code updates into large batch files before rolling out a new version of the app, typically on the same day (Called “merge day”).

Now days, the paradigm of CI/CD is in effect, where the dev team, writes code and uploads it into a shared branch (usually Git). Once these changes are merged with the main branch, they pass automatic testing to ensure nothing has been broken.

That is the CI part, CD ensures that any changes to the app can get almost instant user feedback, if all the automated tests have been passed, and the automated push to production environment has been performed.

Why do we need “Infrastructure as Code”?

Speaking of automation, let’s get closer to our topic of the day.

Typically, a developer, before starting their work on the code for a piece of software it must first create their own dev environment.

Merely a decade ago, this same developer would create a support ticket to their system administrator to set up, configure, install DBs’ and deploy their own VM and Dev Environment on the company’s server. This accumulates costs, both in time and money.

Sadly, even today it’s not uncommon to run into the following scenario:

1. A new developer is introduced into the project and is given a lengthy readme file.
2. After hours of studying the file, half of procedures are outdated, or have the “ToDo” tag on them.
3. Installing the required software on the machine; updates/downgrades version etc.
4. Attempts to run the build, 20 minutes of waiting.
5. Build fails, half an hour of figuring out why.
6. Ask colleague – “Oh yeah, you need to install X and then do Y and Z to make it work”
7. Start from nr.3

Then after multiple attempts, the build miraculously works, it takes another few days before everything “irons out” and everyone in the team works in the same environment. And even then – it’s not guaranteed; not to mention smaller problems that inevitably arrive in such a scenario.

Often due to these issues we have another infamous problem:

“But it works on my machine!”

In-comes containerization, but for deploying your own server – K8s, VM, VPC, and everything else!

What is Infrastructure as Code?

Infrastructure as Code (IaC) is a concept; a means of setting up your infrastructure same as if it were an application, treating it as code. It means using a high-level descriptive coding language to automate the provisioning of your IT infrastructure.

Infrastructure provisioning is simply that what we’ve discussed earlier, it’s the process of setting up your infrastructure. The process of setting up your servers, configuring the networking, creating route tables, installing software, configuring said software, installing databases, and much more.

With IaC, you create configuration files, much like images we’ve discussed with docker, typically in YAML (but other configuration files can apply such as TOML, JSON or even XML). These configuration files hold all the necessary instructions and infrastructure specifications.

Deploying your infrastructure as code holds the benefits of modularity of components, version control, and ease of editing and distributing these configurations to your team.

Of course, because IaC is a concept of writing a set of instructions in a form of code which directs appropriate tools (that we’ll discuss later), clean code practices are paramount if you want to avoid unnecessary costs and headaches!

In such, Infrastructure as Code ensures that you provision exactly the same environment every time, on any machine for any user!

Infrastructure as Code has 3 main tasks:

• Infrastructure provisioning:

Spinning up fresh servers, configurating the network, creating load-balancers, everything typically related to almost bare-metal servers.

• Infrastructure configuration:

Installing any additional applications/software, databases or any dependencies, libraries, frameworks etc.

• Deployment of the application:
  This is our final goal, pushing our application version into some form of production. Running it, testing, and working on it!

However, as we’ve discussed in our Docker article, in essence two of the three tasks are achieved by employing a dedicated containerization tool. By employing docker for instance, and packaging all those libraries, bins, dependencies into an image, we avoid steps 2 and 3. We can directly run our application by installing a Docker runtime onto our servers during step 1!

Imperative vs Declarative Approach

Now that we know what IaC is and what it achieves, lets discuss the two main approaches to Infrastructure as Code. Procedural vs Function – How vs What.

The imperative (procedural) approach defines a specific order in which commands must be adhere to during provisioning. Think of setting up a server, you receive your configuration files and then must use a CLI to run the commands in a step-by-step order.

The disadvantage to the imperative approach is its relative lack of versatility. If you’re aiming at scalability of any sort, you must write custom scripts each time you desire to change something. And the deployment of these changes typically requires a full tear down of your infrastructure.

However, this is considered more user friendly for existing administrative staff and legacy systems.

Declarative (functional) approach, on the other hand, focuses primarily on the end/desired state of the final configuration. It holds information on what resources each state needs and is typically used by specific tools we’ll touch upon soon.

A declarative approach keeps track of the current state of all your objects, making it far easier for on-the-fly changes, restructuring, or scalability. The disadvantage, however, is primarily in human resources. It takes a very skilled system administrator to set up these files and maintain this system.

The declarative approach works much like a GPS would, you simply put down the address you wish to get to, and it does the rest. However, you’d need an expert to tell you how it does that. Imperative approach, on the other side, is more akin to personal experience, and it might not necessarily be the most optimal route either.

IaC tools favour one approach over the other, however many of them *can* be used in either approach you prefer.

Infrastructure as Code tools

Speaking of tools, let’s take a glance at the most popular (and for good reason) tools on the market.

HashiCorp – Terraform

Possibly one of the most known open-source-tool for IaC orchestration. Created in 2014 and initially developed for AWS, the tool has since evolved into its own beast. Based on the HCL (HashiCorp Configuration Language) it balances machine-readable-code and user-friendly language.

Terraform offers its own git repository and module registry, allowing multiple teams to effortlessly collaborate. It also supplies an extensive set of functions and string interpolations for complex scenarios without the need to use another programming language.

AWS – CloudFormation

CloudFormation is primarily an Infrastructure as a Service (IaaS) automation tool, though it can definitely be used as a IaC orchestration tool itself. It uses JSON or YAML as its main configuration formats, relying on the robust template-based AWS stack.

Simply put, with CloudFormation you can ignore the details on how exactly it stores your instructions. This is because its heavily reliant on templates, making it both flexible and not.

Flexible due to the sheer weight and experience of AWS and their size, but not flexible due to reliance on predefined templates. However, 99% of the time CloudFormation hits the mark on the nail, when it comes to supplying exactly what the doctor ordered.

Red Hat – Ansible

Red Hat’s Ansible is truly a beast when it comes to open-source provisioning tools and configuration management. Not so long ago, Linux dominated the server’s network landscape. And thus, Ansible was the “GoTo” infrastructure automation solution for many of these; this did not stop it however to evolve where now it provides support for Windows, IBM Oss, VMs, containers and much more.

Based on Python-YAML syntax it focuses on procedural style to code the desired state in a step-by-step manner. Ansible started off with a simple CLI, now however it has a very pleasing UI, making server infrastructure provisioning so much more accessible to inexperienced administrators.

Puppet Inc

Puppet, an enterprise-level configuration management tool, written in Ruby, focuses primarily on declarative approach to its provisioning.

Based on its own Domain Specific Language (DSL) for its code structure, it’s incredibly robust when it comes to managing Windows or Linux environments.

It is considered by many experts to be one of the most complex, yet powerful solutions on the market. This however doesn’t mean that Puppet is not intuitive. On the contrate, their user-friendly GUI makes it a breeze for even uninitiated users to monitor and manage their server infrastructure. With a big community, and customizable reports, Puppet certainly deserves its high praise among DevOps engineers!

Docker

It wouldn’t be right not to mention our favourite whale on this list! As we’ve mentioned before, Docker, while not a configuration management tool, it certainly can achieve the same, if not better results!

Being a containerization solution, it allows developers to create applications with all their dependencies packaged to run on any environment regardless of its specifications!

A single host can easily run multiple, highly configurable, environments for both unit testing and deployment. With its own CLI, Daemon and API for communication, you have all the tools required. Not to mention the ability for multiple teams to develop simultaneously in different languages, irrespective of their own Dev or the target environments.

You can easily spin up a container for every step during CI/CD pipeline and kill it once the job is done. Simply set up your server once, install the docker runtime, configure it, and you’re done for years to come.

Check out our Docker specific article!

IaC sees growth in adoption

In short, DevOps is maturing, and with the rise of new applications on a daily basis, the need to automate and cut down on costs when it comes to infrastructure provisioning is incredibly important.

You would think that we’re going over old ground, but that is simply not the case, the server management culture is still laden with poor practices, lack of consistency, or partial automation. This causes configuration drifts and undocumented changes, resulting it even higher degrees of headaches for everyone involved.

Even to this day, there’s a sharp lack of professionally trained and IaC certified administrators out there. This, however, does provide favourable grounds for smaller IaC solution start-ups.

Good IaC engineers and practices are essential!

That wraps up our small foray into the world of DevOps again, good thing too, because we’ve just received a complaint notice from HR. Something about nonwork hours harassment, 3 AM messages and our System administrators installing new locks on their doors, we better go and ask them about it!

But before you go, tell us in the comment section: What was the funniest instance of undocumented changes to your infrastructure or application’s code?

Stay classy business and tech nerds!