How to Parse JSON with Python
JSON STRING FORMAT
The closest text format in Python to JSON is the Python dictionary. It can be used to hold the same data types JSON holds which include strings, lists, null, numbers (int, float, and decimal) objects, and boolean values. However, it is still not JSON. JSON in Python is presented as a dictionary surrounded by (‘) as in the case of a string. Below is an example of what JSON looks like in Python programming.
user = {"full name": "Cambell Johnson", "phone": 8103568353, "address": "34 Yanga Boulevard, NY", "age": 35, "profession": "coder"}
print(type(user))
The above will print
Likewise, Javascript has its dictionary-like data type known as an object. This also isn’t supported by Python. However, both languages read and write JSON. For Python, all you need to do is to wrap a dictionary in (‘dictionary’) and you have converted it into JSON provided it does not contain any unsupported data type.
user = '{"full name": "Cambell Johnson", "phone": 8103568353, "address": "34 Yanga Boulevard, NY", "age": 35, "profession": "coder"}'
print(type(user))
When you run the code, instead of the
Parsing JSON in Python
From the above format, you can see that JSON appears as a string in Python. For you to parse it, you will need to convert it into the dictionary and you need the JSON module for that. This Python library is available in the Python standard library and as such, you do not need to install it before using it. All you have to do is to import it. Below is a code showing how to import and use JSON text.
import json
user = '{"full name": "Cambell Johnson", "phone": 8103568353, "address": "34 Yanga Boulevard, NY", "age": 35, "profession": "coder"}'
user_dict = json.loads(user)
print(type(user_dict))
As you can see above, I imported the JSON library using the “import json” command and then went ahead to use the loads method to convert the JSON to dictionary. That’s the keywords here (JSON to dictionary data type). When you convert JSON, what you do is convert it to dictionary data type which is the format you understand.
It is important you know that when you also send JSON (dictionary content to JSON) to the frontend, Javascript also needs to convert it to an object using json.parse. When it needs to send you data from the front end, it uses another method json.stringify to convert objects to JSON.
Now how do you parse out data after converting JSON to dictionary? You access the data points as an access data point in a dictionary. For example, let's say you want to access the age of the user. You access it using the key(age) as done below.
print(user_dict["age"]) The above will print out 35 on the console. Now, let's parse out all of the values. print(list(user_dict.values()))
As you can see, I use the user_dict.values() to parse and retrieve all of the values (minus the keys). However, this is not a regular Python list, and as such, no support for index. I wrapped that method in the list() function to convert it to a real Python list. You can also parse only the keys and return using the below method.
print(list(user_dict.keys()))
How to Edit JSON with Python
Why you can parse JSON and get the data in it, but you can’t edit the original JSON. However, you can edit the data you parse and convert it back into JSON. This is how you can edit JSON and send it back to a user. Using the same data we worked with earlier. Let's say we want to edit the age and change it to 40, we can do that by modifying the dictionary and then converting it to JSON using the json.dumps method. Below is how to do that.
import json
user = '{"full name": "Cambell Johnson", "phone": 8103568353, "address": "34 Yanga Boulevard, NY", "age": 35, "profession": "coder"}'
user_dict = json.loads(user)
user_dict["age"] = 40
user = json.dumps(user_dict)
print(user)
If you run the above, you will see the age has changed from 35 to 40. You will also notice you can no longer access the values. To find out why, just run the print(type(user)) code and you will see that it will display
How to Parse JSON File in Python
Most times when we talk of JSON, our mind goes to the data format we used in sending data back and from — from and from a server. It is the most popular for this kind of task. However, that is not its only task. Some use JSON as a data format. There are some web frameworks including ExpressJS store their configuration data in a JSON format in a file with the .json extension. If you have to deal with data in the .json file, all you have to do is load the file and read its content then parse it as done above. Let's say how to do that in practice below.
First, this is the content of the .json file I will use.
{
"name": "John",
"age": 25,
"city": "Sampleville",
"married": false,
"hobbies": ["reading", "traveling", "programming"],
"address": {
"street": "123 Main Street",
"city": "Sample City",
"postal_code": "12345"
}
}
Now take a look at how I read the file and parse it.
with open("json_example.json", "r") as f:
# Read the file contents
file_data = f.read()
json_data = json.loads(file_data)
print(json_data["name"])
As you can see above, I read the file content using the standard method of reading a file and stored the content which is in string form to the file_data variable. I then use the json.loads method to convert the JSON to a dictionary and then print the name on the console.
Common Errors Experienced While Parsing JSON with Python
From the above, there is no error experienced. This is because the data used above are test data either generated by me or downloaded from sample data sites. In a production environment where you interact with real data, you will experience some errors while parsing JSON. Below are some of the errors
- JSONDecodeError
When you see this error, it is because your JSON data is invalid. There are many reasons why Python will consider your code invalid and those, can’t decode it thereby throwing this error. The reason can range from serious issues to small issues such as a missing or extra comma. Make sure the JSON you want to parse follows the syntax of JSON else, this error will be thrown.
- TypeError
Let's say you parsed out a supposed number from JSON and try to add it or carry out a mathematical operator with it, however, even though it appears as a number, it is a string, you will get the TypeError. So, the TypeError occurs when there is a mismatch between the expected data type and the actual one. You will need to carry out some type of conversion to solve this problem.
- KeyError
If you try to parse data using a key that does not exist, you will get this error. This is common in situations where you have a list of objects in the JSON and most of them contain a particular key. However, as you parse and get to an object that does not contain the key, this error will be thrown. If you don’t want this error thrown, instead of using the object[“key”] approach, you should use the object.get(“key”, None).
- AttributeError
The AttributeError is thrown when you try to access data or parse data from JSON without converting it to a dictionary using the json.loads method. This error is thrown because JSON is treated as a string before conversion. So if you try to access data from it, you will get this error, suggesting the string does not contain such an attribute.
- NameError
Did you forget to import the JSON library? You will experience the NameError. Ordinarily, if you were using an IDE, this should be caught without even running the code. But should it fly under the radar and not be caught, you will experience this error. All you have to do is import JSON and you are good to go.