Day 15: Python Libraries for DevOps

Reading JSON and YAML in Python

Understanding how to read JSON and YAML files in Python is vital for DevOps engineers as it enables them to efficiently manage configurations, automate tasks, ensure smooth communication between systems, and implement infrastructure as code practices, all contributing to effective and reliable DevOps workflows.

JSON:

JSON stands for JavaScript Object Notation
It is a lightweight data interchange format.
It is easy for humans to read and write.
It is often used when data is sent from a server to a web page.

YAML:

YAML stands for YAML Ain't Markup Language.
YAML is a human-friendly data serialization language for all programming languages.
It is used for automation because of its flexibility and accessibility.

There are many libraries in Python but we will see the some of libraries that we use as a DevOps engineer in day-to-day tasks.

os:

The os library in Python provides functions for interacting with the operating system.

It allows us to perform various tasks such as navigating file systems, manipulating files and directories, executing commands, and obtaining information about the system environment.

With the os library, we can write platform-independent code that works seamlessly across different operating systems like Windows, Linux, and macOS.
sys:

The sys library in Python provides access to some variables and functions related to the Python interpreter and its environment.

It allows you to interact with the Python runtime, access command-line arguments, manipulate the Python path, and handle system-level operations such as exiting the interpreter.

The sys module is particularly useful for tasks like accessing command-line arguments passed to a script, interacting with the standard input/output streams, and controlling interpreter behavior.
JSON:

The json library in Python is used for working with JavaScript Object Notation (JSON) data.

It provides functions to parse JSON strings into Python objects and serialize Python objects into JSON strings.

This library is essential for tasks involving data exchange between different systems, configuration management, and web APIs.

With the json library, you can easily convert data between JSON format and Python objects, enabling seamless integration and communication between different components of your application or system.
YAML:

The yaml library in Python allows you to work with YAML (YAML Ain't Markup Language) data.

YAML is a human-readable data serialization format commonly used for configuration files, data exchange, and structured data representation.

With the yaml library, you can parse YAML strings or files into Python data structures and vice versa.

This library is invaluable for tasks involving configuration management, data processing, and interoperability between different systems or applications that use YAML as a data format.

Let's do some hands-on:

TASK1: Create a Dictionary in Python and write it to a JSON File.

import json

# Create a dictionary
cloud_service_provider = {
    "aws" : "ec2",
    "azure" : "VM",
    "gcp" : "compute engine"
}

with open("services.json", "w") as json_file:
    json.dump(cloud_service_provider, json_file)

with open("services.json", "w") as json_file:: This line opens a file named "services.json" in write mode. The with statement ensures that the file is automatically closed after the block of code is executed.

json.dump(cloud_service_provider, json_file): This line writes the contents of the cloud_service_provider dictionary to the JSON file specified by json_file. The json.dump() function takes two arguments: the dictionary to be written (cloud_service_provider) and the file object to write to (json_file).

Task 2: Read a JSON file services.json kept in this folder and print the service names of every cloud service provider.

#Read JSON File and Print Service Names
with open("services.json", "r") as json_file:
    data = json.load(json_file)

for provider, service in data.items():
    print(f"{provider}: {service}")

with open("services.json", "r") as json_file:: This line opens the "services.json" file in read mode. The with statement ensures that the file is automatically closed after the block of code is executed.

data = json.load(json_file): This line uses the json.load() function to parse the JSON data from the file json_file and load it into a Python dictionary named data.

for provider, service in data.items():: This line iterates over each key-value pair in the data dictionary using the items() method. The provider variable stores the key (cloud service provider), and the service variable stores the corresponding value (service name).

Output:-

TASK 3: Read the YAML file using Python, file services.yaml and read the contents to convert YAML to JSON

# Read YAML File and Convert to JSON
import yaml
import json
# Read the contents of the services.yaml file
with open('services.yaml', 'r') as yaml_file:
    yaml_data = yaml.safe_load(yaml_file)

# Convert the YAML data to JSON format
json_data = json.dumps(yaml_data, indent=4)
print("\n Converted YAML to JSON:")
print(json_data)

This block of code opens the "services.yaml" file in read mode using the open() function and a context manager (with statement). The yaml.safe_load() function is then used to parse the contents of the YAML file into a Python dictionary (yaml_data). The safe_load() function is used here to safely parse the YAML data, which helps prevent the execution of arbitrary code from malicious YAML input.

After parsing the YAML data into a Python dictionary (yaml_data), the json.dumps() function is used to convert the dictionary to JSON format. The indent=4 argument specifies that the JSON output should be formatted with an indentation of 4 spaces for better readability. This function returns a JSON-formatted string (json_data) representing the converted data.

Finally, the converted JSON data (json_data) is printed to the console. The first print() statement adds a newline character for formatting, and the second print() statement outputs the JSON data. This allows you to view the converted JSON data in the console output.

Summary

In this blog, we see the concept related to JSON and YAML. Both are very crucial to know as a DevOps engineer in Python helps DevOps teams keep their systems running efficiently and reliably. Apart from this we also saw libraries that are used mostly in day-to-day tasks in DevOps like OS and sys. At the end, we do some hands-on activity.