Retrieving Data via RESTful APIs

Retrieving Data via RESTful APIs

After this lesson you will be able to: understand how an API returns data, call an API with the requests library, and read JSON data into Python structures.

1. What is an API

An API (Application Programming Interface) is the doorway through which your program asks another service for data. A REST API uses the HTTP protocol: you send a request to an address (URL) and receive a response, usually in JSON format.

The two most important HTTP methods:

Method	Used to	Example
GET	Retrieve data without changing anything on the server	Get a list of users
POST	Send data to create something new	Create a post

2. Status codes

A response always comes with a code that tells you the outcome:

Code	Meaning
200	OK — success
401	Unauthorized — missing or invalid API key/credentials
404	Not Found — the address or resource does not exist
429	Too Many Requests — you are calling too fast and are being rate-limited
500	Internal Server Error — a server-side error

The 2xx group means success, 4xx means an error on the caller's side, and 5xx means an error on the server's side.

3. Calling an API with requests

import requests

resp = requests.get(
    "https://jsonplaceholder.typicode.com/users",
    params={"_limit": 3},   # query parameters, appended to the URL as ?_limit=3
    timeout=10,             # ALWAYS set a timeout
)
resp.raise_for_status()     # raise an error for 4xx/5xx codes
users = resp.json()         # convert JSON to a Python list/dict
print(resp.status_code)     # 200
print(users[0]["name"])

Two important points:

• timeout: requests does not set a time limit by itself, so without this parameter your program can hang forever when the server does not respond.
• params: let the library encode the parameters and attach them to the URL instead of concatenating strings yourself.

You can see the final URL without making a network call:

req = requests.Request("GET", "https://api.example.com/search",
                       params={"q": "python", "page": 2}).prepare()
print(req.url)    # https://api.example.com/search?q=python&page=2

4. JSON and Python structures

JSON has two main structures: an object {...} corresponds to a dict and an array [...] corresponds to a list. So you can access them with the familiar indexes and keys:

import json

text = '{"user": {"id": 7, "name": "An", "tags": ["ml", "py"]}, "active": true}'
data = json.loads(text)                 # JSON string → dict
print(type(data).__name__)              # dict
print(data["user"]["tags"][1])          # py
print(data["active"])                   # True   (JSON true → Python True)
print(json.dumps({"a": 1}))             # {"a": 1}   (Python → JSON string)

With requests, resp.json() does exactly what json.loads(resp.text) does.

5. Good habits when using APIs

• Read the API documentation: the address, parameters, rate limits, and whether an API key is needed. Never put a key in publicly shared code.
• Handle errors and set a timeout; slow down when you get a 429.
• Large datasets are usually split into pages (pagination): call the pages one by one using parameters such as page or _limit/_start.
• Turn the results into a table for analysis: pd.DataFrame(users).

Key Points

• GET to retrieve, POST to send; 2xx codes mean success, 4xx a caller-side error, 5xx a server error.
• Always set a timeout; use params instead of concatenating strings.
• resp.json() returns a dict/list; JSON object ↔ dict, JSON array ↔ list.