latest
Design and Development of a REST API in Python. Part 1
recipesAPI RESTflaskpython

Design and Development of a REST API in Python. Part 1

Oct 06, 2019 0 Comments
(Lee esta receta en español pulsando sobre la bandera  )

With this recipe we intend to introduce you to the wonderful world of Web APIs. Nowadays, consulting certain information from a remote server or the process of exchanging data between different servers is commonly done using Web APIs. Practically every Web apps and mobile applications that we use every day have e a Web API in their backend to implement their functionalities, from adding and saving products in the shopping cart in Amazon, to consult the map of your city. Web APIs are a fundamental mechanism to allow different software running on the same or different platforms to communicate with each other.

At the end of this recipe you will be able to:

  • Understand the concepts of Web API and Web Service.
  • Distinguish the main differences between a SOAP Web Services and REST Web Services.
  • Contrast the design principles of a Web API of type REST against the RPCs.
  • Know the advantages of Web programming using frameworks.
  • Design and develop your own API REST in Python.

Before going into depth in the design and development of your Web API we must make a brief introduction to what is and what consists of a Web Service.

Web Services are essentially connection points that allow the integration between applications hosted on platforms that may be different (or not), run different operating systems and be implemented in different programming languages. Web services communicate with each other through the WWW and require two parts:

  1. A server part that exposes a series of Web APIs.
  2. And a second client part that invokes/consumes requests from these Web APIs hosted on the server.

Web services are independent of the implementation in the client side, which can be a Web browser, a mobile application or a desktop application capable of making calls to a number of Web APIs. Web services can be classified according to the communication protocols used:

  • SOAP (Simple Object Access Protocol).
  • HTTP REST (Representation State Transfer).
SOAP & REST services
Service-Oriented Architecture (SOA). Main types.

SOAP services work over HTTP, FTP, POP3, TCP, messaging queues (JMS, MQ, etc.), although they are usually invoked over HTTP protocol. Another special feature of SOAP services is that they only allow the exchange of data using the XML metal language.

On the other hand, REST-type services transport data only via the HTTP protocol, but allow the use of various methods provided by HTTP for communication (GET, POST, PUT, PATCH, DELETE) and, at the same time, they use the native HTTP response codes (404, 200, 204, 409).

REST is more flexible than SOAP and allows us to transmit various types of data (not only XML). The type of data is defined by the HTTP Content-Type header, enabling exchanges of XML, JSON, binary data, plain text, and so on. Based on this, the formats of exchange most frequently used by (REST) Web services are:

  • JSON (Javascript Object Notation) -> Not available in SOAP services.
  • XML (eXtensible Markup Language).

In this recipe we will focus on REST Web APIs (HTTP), learning to develop a set of RESTful services (that respect the REST architecture) using the Python language. Web APIs such as REST (or REST APIs) have achieved great popularity in the development community becoming the predominant architecture for designing and implementing Web services.

Web services make up the basic operating core of Web applications. Web APIs are the mechanisms of integration between these Web services, being deployed on applications of the same or other servers / services and, in turn, they can act as clients consuming information from other Web APIs.

It is required that the implementation of Web services is scalable, at the same time that the services provide an adequate and reliable operation.

REST Architecture

The REST architecture is designed based on the HTTP protocol using resources (called URIs - Uniform Resource Identifiers) and a reduced set of verbs corresponding to HTTP methods (GET, POST, PUT, PATCH, DELETE). This approach differs significantly from the one adopted by other machine-to-machine communication mechanisms for remote code execution, such as RPCs (Remote Procedure Calls), which emphasize the use of a wide variety of operations/verbs. For example, an RPC-based application could define operations as follows to manage a set of users and their locations:

findUser(), getUser(), addUser(),removeUser(), 
updateUser(), getLocation(),addLocation(), 
removeLocation(), updateLocation(), listUsers(), 
listLocations(), findLocation()

An API REST, on the other hand, would define the following resource types:

user {}, location {}

Each particular resource has its own identifier (e.g., http://www.example.org/locations/es/madrid/getafe) which makes it easier to track that resource.

Client applications consume resources using standard HTTP methods, such as GET to download a copy of a particular resource. POST is usually utilized for side-effect actions such as sending a purchase order or adding new resources to a collection.

Using the same example, a user could be represented as the resource below, where the data for this particular case is formatted in XML.

<user>
   <name>Frida Cruz</name>
   <gender>female </gender>
   <location href="http://www.example.org/locations/es/madrid/getafe">
Getafe, Madrid, ES
   </location>
</user>

To update this user's location, a REST API client application could first download the previous XML record using GET. Then, it would modify the record to change the location and, finally, upload it to the server using the PUT method.

At this point, it should be noted that HTTP methods/verbs do not provide standard mechanisms for discovering resources, i.e. there is no LIST or FIND operation in HTTP similar to the list*() and find*() operations of the previous RPC example. Instead, REST APIs solve this problem by treating a collection of search results as another type of resource, which requires knowing the additional URLs to display or search for each particular resource type.

For example, a GET HTTP request over the URL http://www.example.org/locations/es/madrid/ could return a XML list with all possible locations in Madrid, while a GET request to the http://www.example.org/users?surname=Rodríguez URL could return a list with all XML users with surname "Rodríguez".

Creating an API REST

To deploy the API REST we are going to use a micro framework written in Python called Flask, conceived to facilitate the development of Web applications under the MVC (Model-View-Controller) pattern in a simple way (with a learning curve smaller than other major frameworks such as Django) and using a minimum number of modules/extensions/plugins (not full-stack).

In case you are a little clueless about what a framework is, what it is used for or what advantages it provides, we'll give you a few key points:

  • In the modern Web development different frameworks are used. They are tools that provide a working scheme (generally based on the MVC pattern) and a series of utilities and functions to facilitate and abstract low-level tasks during the development of dynamic Web pages.
  • These frameworks are associated to a programming language: Symphony (PHP), Ruby on Rails (Ruby), Django (Python) or in this particular case, Flask (Python).
  • The project associated to the Web application maintains a homogeneous structure (same elements, same files).
  • Facilitates collaboration between developers and designers.
  • Easy to find plugins and other libraries adapted to the framework used.
  • Particular advantages of using Flask:
    • It starts from a minimum development with the essential modules/extensions (not full-stack. New functionalities are added as needed).
    • Includes a development Web server (no extenal infrastructure with a Web server is needed for testing before deployment).
    • Debugger included and support for unit tests.
    • Efficient route management. Determine which route is requested by the client to execute the necessary code.
    • Native support for the use of secure cookies.
    • Open Source.
    • Abundant documentation.

After this brief walk to highlight the virtues of frameworks for Web development and after pointing out some of the positive aspects of Flask, finally, let's move to action...

We are going to design and implement a Web service for user management, which is undoubtedly a necessary requirement in the context of using any Web application or mobile client with cloud services.

  1. The first thing is to model the resource associated with a "user". Since this is a conceptual example, we will only consider the following attributes in this first part of this recipe:
    * id: Unique identifier for the user.
    * username: The user name utilized in the Web application, mobile app, etc.
    * email: The user's e-mail account for notification tasks.
    * status/active: For example, to know if the user is online/offline.
  2. The design of an API REST implies identifying the URI resources and the associated verbs (HTTP methods) that intervene to model the "user" resource. Actions such as: creating a new user, updating attributes of an existing user, getting information from a specific user, listing all users or deleting a specific user are required. Therefore, it is necessary to relate the previous actions with the HTTP methods (GET, POST, PUT, PATCH, DELETE), defining the CRUD operations (Create, Read, Update, Delete) for our "user" model. The following table illustrates this visually:

    URI HTTP method Action
    /v1/usuarios/ GET Get a list of all users
    /v1/usuarios/ POST Create a new user
    /v1/usuarios/1 GET Obtain information from an existing user whose
    identifier is 1
    /v1/usuarios/1 PUT/DELETE Update or remove the user whose identifier is 1

  3. To start with the implementation of the service we will first configure a virtual environment with the virtualenv tool. A virtual environment allows us to create an isolated working configuration with a specific Python interpreter (we can choose a specific version), along with a series of Python modules that we consider appropriate. You can have several different environments on the same machine, installing the necessary modules without affecting each other in each environment. Using a virtual environment avoids contaminating or filling with "rubbish" the global installation of Python and the conflicts of different versions of the same module between applications, as well as other problems related to permissions. We can install the virtualenv tool with the package manager in Linux distributions or more generally with the tool pip: 
    $ pip install virtualenv

    virtualenv saves each virtual environment in a directory with the name of that environment. To create it we execute:

    $ virtualenv VIRTUAL-ENVIRONMENT-DIRECTORY

    For example:

    $ virtualenv /home/gilmour/environmentVirtual1
  4. Once installed virtualenv, it is activated executing the activate script inside the bin directory of the virtual environment: 
    $ source /home/gilmour/environmentVirtual1/bin/activate

    (environmentVirtual1):~/environmentVirtual1/bin$

  5. Now we install Flask in the configured virtual environment ("environmentVirtual1") using pip: 
    (environmentVirtual1):~/environmentVirtual1$ pip install flask

    If you look at the installation log shown in the terminal you can see that Flask is installed together with Jinja2, a template engine written in Python which allows you to insert processed data (context) as default text inside the HTML files/templates that make up the View of the MVC pattern.

    Also installed is Werkzeug , a collection of various utilities for WSGI (Web Server Gateway Interface) applications based on which Flask runs. WSGI establishes a convention for Web servers about how to handle requests and redirect responses to Web applications and frameworks written in Python. Among other utilities, it includes a debugger, a session and cookies handler, a URL routing system, a utility to manage file uploads, tools to control cache options, so that the browser can be specified the caching policies to follow, both for client requests and server responses. Specifically, for caching, Werkzeug uses the parameters specified in the HTTP headers of cache-control and the HTTP Entity Tags (Etag).

    Werkzeug is independent of the template engine used, the adapter for the database, etc. It does not specify or force a certain way of handling customer requests and leaves this completely in the hands of the developer.

    Collecting flask
Downloading
https://files.pythonhosted.org/packages/7f/e7/08578774ed4536d3242b14dacb4696386634
607af824ea997202cd0edb4b/Flask-1.0.2-py2.py3-none-any.whl (91kB)
100% |████████████████████████████████| 92kB 3.7MB/s
Collecting Werkzeug>=0.14 (from flask)
Downloading
https://files.pythonhosted.org/packages/20/c4/12e3e56473e52375aa29c4764e70d1b8f3ef
a6682bef8d0aae04fe335243/Werkzeug-0.14.1-py2.py3-none-any.whl (322kB)
100% |████████████████████████████████| 327kB 13.1MB/s
Collecting click>=5.1 (from flask)
Downloading
https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a1
95a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl (81kB)
100% |████████████████████████████████| 81kB 25.0MB/s
Collecting Jinja2>=2.10 (from flask)
Downloading
https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4
d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl (126kB)
100% |████████████████████████████████| 133kB 12.0MB/s
Collecting itsdangerous>=0.24 (from flask)
Downloading
https://files.pythonhosted.org/packages/dc/b4/a60bcdba945c00f6d608d8975131ab3f25b2
2f2bcfe1dab221165194b2d4/itsdangerous-0.24.tar.gz (46kB)
100% |████████████████████████████████| 51kB 19.3MB/s
Collecting MarkupSafe>=0.23 (from Jinja2>=2.10->flask)
Downloading
https://files.pythonhosted.org/packages/4d/de/32d741db316d8fdb7680822dd37001ef7a44
8255de9699ab4bfcbdf4172b/MarkupSafe-1.0.tar.gz
Building wheels for collected packages: itsdangerous, MarkupSafe
Running setup.py bdist_wheel for itsdangerous ... done
Stored in directory:
/home/gilmour/.cache/pip/wheels/2c/4a/61/5599631c1554768c6290b08c02c72d731791037
4ca602ff1e5
Running setup.py bdist_wheel for MarkupSafe ... done
Stored in directory:
/home/gilmour/.cache/pip/wheels/33/56/20/ebe49a5c612fffe1c5a632146b16596f9e64676
768661e4e46

Successfully built itsdangerous MarkupSafe

Installing collected packages: Werkzeug, click, MarkupSafe, Jinja2, itsdangerous,
flask

Successfully installed Jinja2-2.10 MarkupSafe-1.0 Werkzeug-0.14.1 click-7.0 flask-1.0.2 itsdangerous-0.24
  6. With Flask installed and configured in our virtual environment we can implement our first small Web application in Flask that we will call applicacion.py (save the file in the root of the virtual environment directory). The code you have to include is the following: 
    from flask import Flask

app = Flask(__name__)

@app.route('/')
def index():
    return "Hello I'm your first Web Flask app"
    
if __name__ == '__main__':
    app.run(debug=True)
  7. When running python aplicacion.py you can see how the Flask server is deployed and it is listening on port 5000 (default):
    WARNING: Do not use the development server in a production environment.
Use a production WSGI server instead.
* Debug mode: on
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 439-951-258
  8. If you access the URL http://127.0.0.1:5000 you will see the result expected as a response to the HTTP GET request with the path('/'):


    SOAP & REST services
  9. All right, let's go further to implement the REST API. Let's start with the HTTP GET request to retrieve all users. The URI associated to that resource is /v1/users. We need to implement a route for the URI that will trigger the execution of the function getUsuarios(). This function will return all users in JSON format from a structure that is a combination of Lists and Dictionaries in Python. Our script is as follows: 
    from flask import Flask
from flask import jsonify

app = Flask(__name__)

usuarios = [
    {'id':1,
        'nombreUsuario': u'David Gilmour',
        'email': u'gilmour@pinkfloyd.com',
        'activo': True
    },
    {'id':2,
        'nombreUsuario': u'Richard Wright',
        'email': u'wright@pinkfloyd.com',
        'activo': False
    },
    {'id':3,
        'nombreUsuario': u'Roger Waters',
        'email': u'waters@pinkfloyd.com',
        'activo': True
    }
]

@app.route('/v1/usuarios/', methods=['GET'])
def getUsuarios(): 
    return jsonify({'usuarios': usuarios})

@app.route('/')
def index():
    return "Hola soy tu primera app Web Flask"
    
if __name__ == '__main__':
    app.run(debug=True)

    If we relaunch the application with python aplicacion.py it is now possible to make a HTTP GET request to /v1/users using the same browser as shown in the image below. We have just implemented a part of the API REST of our user management Web service.

    GetUsers()

    If we inspect the request with the developer tool of the browser we can see:

    • The Content-Type is application/JSON.
    • The server is Werkzeug (Flask is based on Werkzeug).
    • The GET request was correct so the code 200 is returned.
    • ...
    Respuesta 200 Ok
  10. We'll continue with another resource to get a user from his/her identifier (id). We add the following route to our script: 
    #...
from flask import abort
from flask import make_response

    @app.route('/v1/users/<int:id>/', methods=['GET'])
    
def getUsuario(id):
    

     for user in users:
       if user.get('id') == id:
           return jsonify({'users':user})
 abort(404)

    

    @app.errorhandler(404)
    
def not_found(error):
    
return make_response(jsonify({'error': 'Not found'}),404

  11. With the previously defined route we invoke the function getUsuario(id) passing as argument the identifier of the user from whom we want to obtain information. When we perform the HTTP GET request in the previous URI, internally, we iterate over the users until the one with the identifier we are looking for is located. If the user is found, the information is returned in JSON format. If not, the server sends a HTTP 404 response. If we invoke with the URL http://127.0.0.1:5000/v1/usuarios/4 we will obtain the HTTP 404 error response for the data of the example.

    User Not Found
  12. To be able to manage user registrations in a future client application (e.g.., Web application, mobile application, etc.) we require a new route to create users using the HTTP POST method. We add the following code in our script:
    #...
from flask import request
@app.route('/v1/users/', methods=['POST'])
def createUser():
    if not request.json or not 'email' in request.json:
        abort(404)
    id = users[-1].get('id') + 1
    username = request.json.get('username')
    email = request.json.get('email')
    active = False
    user = {'id': id, 'username': username, 'email': email, 'active': active}
    users.append(user)
    return jsonify({'user':user}),201
  13. For example, we can use the curl command in Linux to test the POST request passing the necessary information of the new user in JSON format. Below is an example.
    $ curl --header "Content-Type: application/json" --request POST --data
'{"username": "Nick Mason", "email": "mason@pinkfloyd.com"}'
http://127.0.0.1:5000/v1/usuarios/
    As you can see in the previously createUser() function code, the email attribute is mandatory and, by default, new users appear as "inactive".
  14. Let's now use the necessary resource to modify user's information by adding the following route in the application.py script:
    #...
@app.route('/v1/users/<int:id>/', methods=['PUT'])
def updateUser(id):
    user = [user for user in users if user['id'] == id]
    user[0]['username'] = request.json.get('username', user[0]['username'])
    user[0]['email'] = request.json.get('email', user[0]['email'])
    user[0]['active'] = request.json.get('active', user[0]['active'])
    return jsonify({'users':user[0]})
    If we execute the curl sentence we can see how the data of the user with id=3 is modified. 
    $ curl --header "Content-Type: application/json" --request PUT --data '{"username": "Roger Waters", "email": "rogerWaters@pinkfloyd.com"}'
http://127.0.0.1:5000/v1/usuarios/3/
  15. Finally, it is required the resource to delete a certain user through his/her identifier. Adding the following route in the application.py script:
    #...
@app.route('/v1/users/<int:id>/', methods=['DELETE'])
def deleteUser(id):
    user = [user for user in users if user['id'] == id]
    users.remove(user[0])
    return jsonify({}), 204 # No content
    If we execute the curl statement, a HTTP request to delete the user with id=3 is performed: 
    $ curl --request DELETE http://127.0.0.1:5000/v1/usuarios/3/

    16. We will continue working on REST APIs and Web services in future recipes.