Now that we have seen how to aggregate useful data into a database, we will want to share that data with the world. In today's lecture we are going to learn the basics of the Flask web framework, and we will see how we can use Flask to quickly put together a server application to serve up our data.
The first example program below shows how to use flask to set up a minimal web server.
from flask import Flask
app = Flask(__name__)
@app.route('/')
def index():
return "Hello, World!"
app.run(debug=True)
Two things are going on here. First, we are constructing a flask application object and telling it to start running. Secondly, we are setting up a single route.
The flask app implements a very simple web server. Since web servers operate by serving up content in response to requests containing URLs, we have to specify which URLs the server should respond to and what information should be returned for each such URL. A route is a fragment of a URL: when this is combined with the address of the server we get a complete URL that the server can respond to. In flask we set up a route by adding a Python decoration to a function, making that function a route handler function. The function should return some sort of content. In this first example the index function is the route handler for the route whose full URL is http://127.0.0.1:5000/
Running this program launches the flask web server. You can test that the application is working correctly by entering the URL http://127.0.0.1:5000/ into a browser.
In the main example today we are going to construct a flask web server that serves up data from a database. Specifically, we are going to serve up data from the solar power database we set up in an earlier lecture.
The next example program illustrates how to set up a database connection in a flask web app and serve up data from it.
from flask import Flask, g, jsonify
import sqlite3
import datetime
import time
def getDB():
db = getattr(g, 'database', None)
if db is None:
g.database = sqlite3.connect('solar.db')
db = g.database
return db
app = Flask(__name__)
@app.teardown_appcontext
def closeConnection(exception):
db = getattr(g, 'database', None)
if db is not None:
db.close()
@app.route('/')
def index():
return "Hello, World!"
@app.route('api/<day>')
def getDay(day):
start_date = datetime.datetime(2019, 4, int(day), 0, 0, 0)
start_time = int(time.mktime(start_date.timetuple()))
end_time = start_time + 3 * 24 * 60 * 60
c = getDB().cursor()
sql = "SELECT time, power, temp, cloud FROM power WHERE time>=? AND time <=?"
c.execute(sql, (start_time, end_time))
result = [{"time":t,"power":p,"temp":f,"cloud":c} for t,p,f,c in c.fetchall()]
return jsonify(result)
app.run(debug=True)
In preparation for working with the database we set up a getDB() function to open the connection to the database. Any route handler that needs to interact with the database will call this function to obtain a connection to the database. In the interest of efficiency, getDB() only opens the database connection once. Immediately after opening the connection, getDB() stashes a reference to the database connection in a special flask global application object, g. On subsequent calls to getDB() the code will simply fetch that reference back from the g object and return that reference to anyone who needs it.
A well-behaved application should also close any database connections that it has opened, so we also set up a second function, closeConnection(), that closes the database connection. The decorator on that function sets the function up as a teardown function that will be called automatically when the flask server shuts down.
To serve up data from the database I have added a second route handler function, getDay(). Since this function is designed to serve up data, its route begins with api: this is a widely used convention for setting up data access URLs. The route for getDay() also contains a route parameter, <day>, which is a placeholder for an integer day number that the client will provide in the URL. Any value that the client provides for this placeholder gets passed to the getDay() function as a parameter when the route handler function gets invoked.
The purpose of the getDay() function is to serve up power and weather data for an entire day. The database code in getDay() does a query against the database to fetch the necessary data. Since we are going to convert this data into JSON, we need to convert data for each row that we get back from the query into a format that is appropriate for conversion to JSON. The code above will take data from each row, which comes to us a tuple, and convert it to a dictionary. The conversion process that converts our data to JSON will then map each of those dictionaries into a JSON object. The final conversion to JSON is handled by the flask jsonify() function.
You can test this latest version of the code by running the program and entering the URL http://127.0.0.1:5000/api/8 into a browser.
For our final example today I am going to construct a more complete flask application. This final version will serve to demonstrate additional flask features such as serving up files, working with templates, and handling form requests.
Most applications that serve up data should also provide some documentation on how to use the application. One simple convention for how to handle this is to have the application serve up an HTML documentation page in response to the base URL http://127.0.0.1:5000/. This next version of the program will replace the simple "Hello, World!" response from the previous versions with a more complete documentation page.
Many web applications that serve up data also implement some sort of security mechanism to prevent unauthorized users from accessing the application. This version of the application will also implement an API key mechanism to secure the application. To give users a way to obtain an API key, we will also implement a simple form in the documentation page that users can use to request an API key.
Here now is the full source code for the final version of our flask application:
from flask import Flask, g, request, render_template, jsonify
import sqlite3
import datetime
import time
import secrets
def getDB():
db = getattr(g, 'database', None)
if db is None:
g.database = sqlite3.connect('solar.db')
db = g.database
return db
def makeKey(name):
db = getDB()
key = secrets.token_urlsafe(8)
c = db.cursor()
sql = "INSERT INTO users(name,apikey) VALUES(?,?)"
c.execute(sql,(name,key))
db.commit()
return key
def checkKey(key):
c = getDB().cursor()
sql = "SELECT name FROM users WHERE apikey='"+key+"'"
c.execute(sql)
name = c.fetchone()
if name is None:
return False
return True
app = Flask(__name__, static_folder='')
@app.teardown_appcontext
def closeConnection(exception):
db = getattr(g, 'database', None)
if db is not None:
db.close()
@app.route('/', methods=['GET'])
def index():
return app.send_static_file('index.html')
@app.route('/getKey', methods=['POST'])
def getKey():
name = request.form['name']
key = makeKey(name)
return render_template('key.html', key=key)
@app.route('/api/<key>/day/<day>', methods=['GET'])
def getDay(key, day):
if not checkKey(key):
return jsonify([])
start_date = datetime.datetime(2019, 4, int(day), 0, 0, 0)
start_time = int(time.mktime(start_date.timetuple()))
end_time = start_time + 24 * 60 * 60
c = getDB().cursor()
sql = "SELECT time, power, temp, cloud FROM power WHERE time>=? AND time <=?"
c.execute(sql, (start_time, end_time))
result = [{"time":t,"power":p,"temp":f,"cloud":c} for t,p,f,c in c.fetchall()]
return jsonify(result)
app.run(debug=True)
The route handler for the base route, index(), is now set up to return an entire page of HTML. We manage this by setting up a separate file, index.html in the project folder and having the route handler return that entire file by calling the flask send_static_file() function. In addition, when we set up the Flask object we pass in an additional parameter telling flask where the static files for our project are located. For this simple example it suffices to put the lone static file in the same folder as the other project files. More complex sites will want to set up a separate static folder in the project folder and place all of the application's static content there.
Here now is the HTML code for the index.html file:
<!DOCTYPE html>
<html>
<head>
<title>Solar Server</title>
</head>
<body>
<h4>Welcome</h4>
<p>Welcome to the Solar data server. Data for a particular day is
available via the URL</p>
<pre>
http://127.0.0.1:5000/api/<key>/day/<day>
</pre>
<p>where <key> is your API key and <day> is an integer in the range
from 5 to 30.</p>
<h4>Request API key</h4>
<p>Don't have an API key? You can use the form below to request one.</p>
<form action="getKey" method="post">
<label for="name">Enter your name:</label>
<input type="text" name="name" />
<input type="submit" />
</form>
</body>
</html>
One special feature of this code is an HTML form that users can use to sign up for an API key. The action attribute of the form will send requests from this form to the URL http://127.0.0.1:5000/getKey. Here is the route handler for that URL:
@app.route('/getKey', methods=['POST'])
def getKey():
name = request.form['name']
key = makeKey(name)
return render_template('key.html', key=key)
HTML form requests will send data back to the server in the form of key/value pairs. The form in the HTML page above uses a single parameter with the key of 'name'. To access the value that the user entered for this parameter the route handler accesses the flask request object's form data. This data is a dictionary containing all of the key/value pairs sent in by the form. Once we have the name the user has entered we pass it to a special makeKey() function that converts the name into an API key and stores that data in the application's database. The last thing the route handler has to do is to return an HTML page that shows the user their new API key. To handle this we use the flask template mechanism. To use this mechanism we set up an HTML page for the response and embed template expressions in the HTML for any content that we don't know ahead of time. We then serve up that template page by calling the flask render_template() function, passing that function values for any template expressions that appear in the page.
Here is the code for the template page key.html:
<!DOCTYPE html>
<html>
<head>
<title>API Key</title>
</head>
<body>
<p>Your API key is <b>{{key}}</b></p>
</body>
</html>
Now that we have implemented a system of API keys, we can modify the route handler for our data serving URL to look for and check API keys:
@app.route('/api/<key>/day/<day>', methods=['GET'])
def getDay(key, day):
if not checkKey(key):
return jsonify([])
start_date = datetime.datetime(2019, 4, int(day), 0, 0, 0)
start_time = int(time.mktime(start_date.timetuple()))
end_time = start_time + 24 * 60 * 60
c = getDB().cursor()
sql = "SELECT time, power, temp, cloud FROM power WHERE time>=? AND time <=?"
c.execute(sql, (start_time, end_time))
result = [{"time":t,"power":p,"temp":f,"cloud":c} for t,p,f,c in c.fetchall()]
return jsonify(result)
This code starts by confirming that the URL contains a valid API key. If it does not, the function simply returns some empty data.
Now that our server application is up and running, we can write a client program to fetch data from the server and do something with it.
Here is the code for a client program that replicates the plotting application from the last lecture. This time around the application requests the data from the server instead of reading it directly from a database:
import requests
import time
import datetime
import matplotlib.pyplot as plt
start = int(input("Enter a start day:"))
start_date = datetime.datetime(2019,4,start,0,0,0)
start_time = int(time.mktime(start_date.timetuple()))
results = []
for d in range(start,start+3):
response = requests.get('http://127.0.0.1:5000/api/KBBX4xvQ5xI/day/'+str(d))
results = results + response.json()
# Process the results
t = []
p = []
tp = []
cl = []
for entry in results:
t.append((entry['time']-start_time)/(60*60))
p.append(entry['power']/33.7)
tp.append(entry['temp'])
cl.append(entry['cloud']*100)
plt.plot(t,p,'r-',t,tp,'b-',t,cl,'y-')
plt.title('Starting day '+str(start))
plt.savefig('plot.png')
Here is an archive containing all of the files for today's lecture.