Suppose we have developers who are creating a new application. On one side, we have servers that are managed and configured to run this application. Making this available to the end-users is the responsibility of DevOps.
In this article, we will learn what tools you need to learn to become a DevOps engineer.
Basic of DevOps
First, you need to understand that there are two main parts when creating an application
1. Development part
- Developing of application
- Testing the application
2. Operations part
- Deployment of an application
- Maintained on a server
DevOps is a link between the two. Now, this is a bit too abstract so let’s dive into the details to really understand the DevOps tasks and which tools are needed to carry out these tasks.
Development process you should know.
It all starts with the application. The developer’s team will program an application with any technology stack, different programming languages, build tools, etc. They have a code repository to work on the code in a team. One of the most popular ones today is git. As a DevOps engineer, you will not be programming the application, but you need to understand the concepts of how developers work. Which git workflow they’re using, also how the application is configured to talk to other services or databases. As well as concepts of automated testing and so on. Here are the key points
- How developers work
- Which git workflow
- How the application is configured
- Concept of automated testing
Operating system & Linux basics
Now that application needs to be deployed on a server. So that eventually users can access it. That’s why we’re developing it, so we need some kind of an infrastructure on-premise servers or cloud servers. These servers need to be created and configured to run our application.
Again you as a DevOps engineer may be responsible for preparing the infrastructure to run the application. Since most of the servers where applications are running are Linux servers, you need knowledge of Linux. It would help if you were comfortable using a command-line interface. Because you will be doing most of the stuff on the server using a command-line interface. Hence, knowing basic Linux commands, installing different tools and software on servers, understanding Linux file system basics of how to administer a server, SSH into the server, etc.
- Linux Basics
- Comfortable using CLI
- Shell commands
- Linux file system
- Server management
Networking and Security
You also need to know the basics of networking and security. For example, to configure firewalls, secure the application, and open some ports to make the application accessible from outside, you need to understand these things.
- Firewall, Proxy Services
- Load Balancers
- Ports, IP, DNS Name Resolution
DevOps VS IT-operations
To draw a line between its operations and DevOps, you don’t have to have an advanced super operating system or networking and security skills. Also, be able to administer the servers from start to finish; there are own professions like network and system administrators security engineers. They really specialize in one of these areas. Your job is to understand the concepts so that you can prepare the server to run your application.
Your duty as DevOps here is
- Basic system admin works
- Basic networking management
- Basic security management
Nowadays, containers have become the new standard.
You need to understand the concepts of virtualization and containers generally. You should also know how to manage containerized applications on a server. One of the most popular container technologies today is docker. You should have knowledge of it.
Introduction to Automation & CI/CD
CI/CD is a must for all large-scale projects. Doing things manually is slower also a hassle to manage daily. CI/CD helps us to do things.
Continuous integration (CI) helps developers merge their code changes back to a shared branch, or “trunk,” more frequently — sometimes even daily.
Build Automation & CI/CD
First of all, when the feature or bug fix is done, we need to run the tests. Then package the application as an artifact like jar file or zip, etc. So that we can deploy it, that’s where build tools and package manager tools come in. Advanced companies are adopting containers as a new standard.
You need to know these things for CI/CD
- Build Tools and package manager
- Artifact repository
- Build automation
Docker & DevOps
You will probably be building Docker images from your application. This image must be saved somewhere right in an image repository. Docker artifact repository on nexus or docker-hub or any other will be used here. So you need to understand how to create and manage artifact repositories.
Of course, you don’t want to do any of this manually. You need one pipeline that does all of these in sequential steps. So it would help if you build automation. One of the most popular build automation tools is Jenkins.
You need to connect this pipeline with the git repository to get the code. This is part of the continuous integration process.
When code changes get deployed continuously on a deployment server, and there could be some additional steps. Sending notification to the team about the pipeline state or handling failed deployment is also possible here.
This flow represents the core of the ci/cd pipeline, and the CI/CD pipeline happens to be at the heart of the DevOps tasks and responsibilities. As a DevOps engineer, you should be able to configure the complete CI/CD pipeline for your application, and that pipeline should be continuous. That’s why the unofficial logo of DevOps is an infinite cycle because the application improvement is infinite. New features and bug fixes get added all the time that need to be deployed.
Now let’s go back to the infrastructure where our application is running. Nowadays, many companies are using virtual infrastructure on the cloud instead of creating and managing their physical infrastructure. These are some infrastructure as a service platform.
- Google cloud
- Microsoft azure
One apparent reason for that is to save costs by setting up your infrastructure. These platforms also manage a lot of stuff like
- Load balancer
- Security and many more
Things you can manage here using a UI
- Create your network configure
- Route tables
- All parts of your infrastructure
You need to learn them to manage infrastructure there, so if your applications run on AWS, you need to learn the AWS and its services. AWS is pretty complex. So many features
You don’t have to learn all the services that it offers. You need to know those concepts and services that you need.
Our application will run as a container because we’re building docker images. Containers need to be managed. For smaller applications, docker-compose or docker-swarm is enough to manage them. If you have a lot more containers, like big microservices, you need a more powerful container orchestration tool to do the job. Kubernetesmost is the most popular among all.
You need to understand how Kubernetes works and be able to administer and manage the cluster to deploy applications in it
when you have thousands of containers running in Kubernetes on hundreds of servers, how do you track the performance of your individual applications? How do you know in real-time if your users are experiencing any problems? One of your responsibilities as a DevOps engineer. You need to know a monitoring tool like
Infrastructure as Code
In your project, you will need development and testing or staging environments. Properly test your application before deploying it to production.
It would help if you had that same deployment environment multiple times, creating and maintaining. But that is not that easy because that is
So we don’t want to do it manually multiple times. We want to automate as much as possible. How do we automate this process?
- Creating the infrastructure
- Configuring it to run your application
- Deploying your application on
Configuring infrastructure can be done using a combination of two types of infrastructure as code tools
1. Infrastructure provisioning tool
2. Configuration management tool
As a DevOps engineer, you should know one of these tools to make your own work more efficient, so your system gives you these advantages.
- More efficient
- Easy to replicate
- Easy to recover
More automation probably need to write scripts. There are lots of benefits if you know scripting language. What you can do with a scripting language is
- System monitoring tasks
- Cron jobs
- Network management
Here is an article you can follow to know the basics of a cronjob.
How to Schedule Tasks with Cron Job
Beginners guide to cronjob
To be able to do that, you need to know a scripting language. Here are two options you can choose from.
1. Operating system specific scripting language
2. More demanded a more powerful and flexible language
Learn one of these languages without a doubt. Python is the most popular and demanded one in today’s DevOps space.
- Easy to learn
- Easy to read
- Very flexible
- So many libraries
Python has libraries for most of the database operating system tasks as well as for different cloud platforms.
With these automation tools and languages, you write all of this automation logic as code like
- configuring infrastructure
That’s why the name infrastructure as code. How do you manage your code just like the application code? You manage this also using version control like git. As a DevOps engineer, you also need to learn git.
At this point, you may be thinking, how many of these tools do I need to learn? Do I need to learn multiple tools in each category? Which ones should I learn? Because there are so many of them. Just focus on the following two points.
- One tool in each category one that’s the most popular
- Most widely used/Popular one
Because once you understand the concepts well. Building on that knowledge and using an alternative tool will be much easier.
Thank you for reading. Have a great day!