Ethan Swan • PyCon 2023 • Slides: eswan18.github.io/sklearn-api-deploy-slides
Take a pre-trained model and deploy it within a FastAPI app.
LogisticRegression
modelIntroduce yourself, your job, where you're from.
Hopefully you can find a buddy to chat with as we work through challenges together!
We'll move quickly through some lecture for each section, and then you'll work through a "to-do list" on your own to replicate what we saw in the lecture.
You can find lectures on the internet. The benefit of in-person tutorials is that I can come around and help you as you work.
eswan18.github.io/sklearn-api-deploy-slides
github.com/eswan18/sklearn-api-deploy
Section 1: eswan18.github.io/sklearn-api-deploy-slides/diffs/1.html
Section 2: eswan18.github.io/sklearn-api-deploy-slides/diffs/2.html
Section 3: eswan18.github.io/sklearn-api-deploy-slides/diffs/3.html
Section 4: eswan18.github.io/sklearn-api-deploy-slides/diffs/4.html
Before we get started...
app
), models (app/models
), and tests (tests
)__init__.py
files in app/
and app/models/
project
├── app
│ ├── __init__.py
│ └── models
│ └── __init__.py
└── tests
Then we'll download the sklearn model file and store it in app/models
.
iris_regression.pickle
-> https://github.com/eswan18/sklearn-api-deploy/blob/main/app-section-1/app/models/iris_regression.pickleproject
├── README.md
├── app
│ ├── __init__.py
│ └── models
│ ├── __init__.py
│ └── iris_regression.pickle
└── tests
We want to keep track of our project's dependencies and metadata, which we can do with two files in the base of the project:
setup.cfg
-> https://github.com/eswan18/sklearn-api-deploy/blob/main/app-section-1/setup.cfgpyproject.toml
-> https://github.com/eswan18/sklearn-api-deploy/blob/main/app-section-1/pyproject.tomlproject
├── app
│ ├── __init__.py
│ └── models
│ ├── __init__.py
│ └── iris_regression.pickle
├── pyproject.toml
├── setup.cfg
└── tests
setup.cfg
¶# setup.cfg
[metadata]
name = app # What name will we import our package under?
version = 0.1.0
[options]
package_dir = # Where is the source code for the "app" package?
app = app
install_requires = # What dependencies do we need?
anyio==3.6.2
attrs==22.2.0
... # omitted
[options.package_data]
app = models/* # Include non-Python files in app/models.
pyproject.toml
¶setuptools
is the tool that should build our package# pyproject.toml
[build-system]
requires = ["setuptools"]
project
├── README.md
├── app
│ ├── __init__.py
│ └── models
│ ├── __init__.py
│ └── iris_regression.pickle
├── pyproject.toml
├── setup.cfg
└── tests
Readmes are usually written in Markdown
*
, #
) have special meaning.md
You can write your own or use mine:
This repo contains an Iris prediction server. To start the application, run:
uvicorn app.main:app --host 0.0.0.0 --port 8000
If the API server is running at http://localhost:8000
, then the following should work in a local Python session:
>>> import requests
>>> response = requests.post(
... "http://localhost:8000/predict",
... json={
... "sepal_width": 1,
... "sepal_length": 1,
... "petal_length": 1,
... "petal_width": 1,
... },
... )
>>> response.status_code
200
>>> response.json()
{'flower_type': 0}
cd ~/path/to/project
(I can help with this)python3 -m venv venv
source venv/bin/activate
(Bash -- Mac/Linux).\venv\Scripts\activate
(Powershell -- Windows)pip install -e .
You can make sure it worked by starting up Python and trying to import FastAPI
(venv) $ python
>>> import fastapi
If that runs without error, we're good to go!
To exit that interactive Python session:
>>> exit()
app
, app/models
, tests
app/__init__.py
and app/models/__init__.py
app/models
foldersetup.cfg
and pyproject.toml
files in base of project folderREADME.md
file in base of project folderpython3 -m venv venv
source venv/bin/activate
on Mac/Linux.\venv\Scripts\activate
on Windowspip install -e .
Change summary: eswan18.github.io/sklearn-api-deploy-slides/diffs/1.html
"the API is running"
at localhost:8000/
A Web API is a bit like a function that you can call over the internet
You send a request and get back a response
Requests specify a method -- a special argument for what type of action to take
GET
-> fetch some dataPOST
-> submit some dataNetwork issues
Status codes
Routes
https://myweatherapi.com/chicago/temp
myweatherapi.com
is the "domain name"/chicago/temp
is the "route" or "path"A popular Python framework for building APIs
Simple
Easy Documentation
Great Performance
# app/main.py
from fastapi import FastAPI
app = FastAPI()
@app.get("/") # Listen for GET method requests at "/" route
def status():
"""Check that the API is working."""
return "the API is up and running!" # Return this as response
$ uvicorn app.main:app --host 0.0.0.0 --port 8000
INFO: Started server process [36347]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
Let's look at http://localhost:8000/
and http://localhost:8000/docs
.
We can also send requests to the API via curl
, a command line utility:
$ curl -X GET http://localhost:8000
"the API is up and running!"
Or even with Python and the httpx
library:
>>> import httpx
>>> response = httpx.get("http://localhost:8000")
>>> response
<Response [200 OK]>
>>> response.json()
'the API is up and running!'
Interactive testing is good at first, but automated tests are a better solution
We're going to use the very popular pytest
library for testing
tests/
directorytests/test_<thing>.py
assert
statements# tests/test_addition.py
def test_addition_of_2_and_2():
result = 2 + 2
assert result == 4
tests/conftest.py
In our case, we want to create an instance of our app to use in our tests:
# tests/conftest.py
import pytest
from fastapi.testclient import TestClient
from app.main import app
@pytest.fixture
def client() -> TestClient:
return TestClient(app)
Then we can use it in a test that checks the status endpoint:
# tests/test_app.py
from fastapi.testclient import TestClient
def test_status_endpoint(client: TestClient):
response = client.get("/")
# 200 is the standard code for "success"
assert response.status_code == 200
# response.json() turns the response into Python objects
payload = response.json()
assert payload == "the API is up and running!"
GET
endpoint for /
app/main.py
"the API is running"
when pingeduvicorn app.main:app
http://localhost:8000/
in the browserTestClient
tests/conftest.py
/
endpointtests/test_app.py
pytest
Change Summary: eswan18.github.io/sklearn-api-deploy-slides/diffs/2.html
/predict
endpoint that accepts an observation and returns a (dummy) predictionRight now, our status endpoint doesn't accept any inputs
@app.get("/") # Listen for GET method requests at "/" route
def status():
"""Check that the API is working."""
return "the API is up and running!" # Return this as response
We need to take inputs if we want to return a prediction, e.g.
def predict(observation):
"""Return a prediction for the given observation."""
prediction = mymodel.predict(observation)
return prediction
Pydantic is a library for defining data models
Observation
model¶We can use Pydantic to define a model of the data in an "observation":
from pydantic import BaseModel
class Observation(BaseModel):
sepal_length: float
sepal_width: float
petal_length: float
petal_width: float
Annotating the types of the fields allows Pydantic to validate the data.
from pydantic import BaseModel
class Observation(BaseModel):
sepal_length: float
sepal_width: float
petal_length: float
petal_width: float
>>> obs = Observation(sepal_length=1.4, sepal_width=2, petal_length=3.3, petal_width=4)
>>> obs
Observation(sepal_length=1.4, sepal_width=2.0, petal_length=3.3, petal_width=4.0)
>>> obs = Observation(sepal_length=1.4, sepal_width=2, petal_length=3.3, petal_width="abc")
ValidationError: 1 validation error for Observation
petal_width
value is not a valid float (type=type_error.float)
We only used float
here, but Pydantic supports most common Python types...
int
, float
, str
, bool
...as well as abstract types like...
Literal
, Union[X, Y]
, Optional[X]
Prediction
model¶We can use the Literal
type to specify that a prediction must be one of a few specific strings:
from typing import Literal
class Prediction(BaseModel):
flower_type: Literal["setosa", "versicolor", "virginica"]
Let's save both models, along with docstrings.
# app/pydantic_models.py
from typing import Literal
from pydantic import BaseModel
class Observation(BaseModel):
"""An observation of a flower's measurements."""
sepal_length: float
sepal_width: float
petal_length: float
petal_width: float
class Prediction(BaseModel):
"""A prediction of the species of a flower."""
flower_type: Literal["setosa", "versicolor", "virginica"]
predict
endpoint¶Now that we have models for observations and predictions, we can build a (fake) /predict
endpoint...
# app/main.py
from .pydantic_models import Observation, Prediction
@app.post("/predict", status_code=201)
def predict(obs: Observation) -> Prediction:
"""For now, just return a dummy prediction."""
return Prediction(flower_type="setosa")
Using POST
instead of GET
POST
is often used for "creating" something, and here we're creating a predictionSpecifying a 201
status code to be returned
200
is the default for FastAPI, and just means "OK"201
means "Created", which is more appropriate for a POST
requestUsing Python type hints to tell FastAPI what Pydantic models to use
We need to update our tests to exercise the new /predict
endpoint.
# tests/test_app.py
def test_predict(client: TestClient):
response = client.post(
"/predict",
# We pass observation data as a dictionary.
json={
"sepal_length": 5.1,
"sepal_width": 3.5,
"petal_length": 1.4,
"petal_width": 0.2,
},
)
assert response.status_code == 201
payload = response.json()
# For now, our fake endpoint always predicts "setosa".
assert payload["flower_type"] == "setosa"
Observation
Pydantic modelsepal_length
, sepal_width
, petal_length
, petal_width
Prediction
Pydantic modelflower_type
/predict
endpointhttp://localhost:8000/docs
tests/test_app.py
Change Summary: eswan18.github.io/sklearn-api-deploy-slides/diffs/3.html
/predict
endpoint to use our sklearn modelapp/models/iris_regression.pickle
importlib.resources.open_binary
to open it, and decode it with pickle
# app/main.py
...
import importlib
import pickle
from sklearn.linear_model import LogisticRegression
def load_model(model_name: str) -> LogisticRegression:
with importlib.resources.open_binary("app.models", model_name) as f:
model = pickle.load(f)
return model
@app.get("/")
def status():
"""Check that the API is working."""
return "the API is up and running!"
iris_regression_v2.pickle
# app/main.py
...
import importlib
import pickle
from sklearn.linear_model import LogisticRegression
def load_model(model_name: str) -> LogisticRegression:
with importlib.resources.open_binary("app.models", model_name) as f:
model = pickle.load(f)
return model
MODEL_NAME = "iris_regression.pickle"
model = load_model(MODEL_NAME)
app = FastAPI()
@app.get("/")
def status():
"""Check that the API is working."""
return "the API is up and running!"
We could run load_model()
within the endpoint where it's used -- why not?
load_model()
outside any endpoint loads the model just a single time for our whole applicationBefore writing a predict endpoint, let's look at how to make predictions with a model.
>>> model = load_model("iris_regression.pickle")
>>> model
LogisticRegression(max_iter=1000)
>>> import pandas as pd
>>> observation = pd.Series({
... "sepal length (cm)": 1,
... "sepal width (cm)": 2,
... "petal length (cm)": 3,
... "petal width (cm)": 4,
... })
>>> observation
sepal length (cm) 1
sepal width (cm) 2
petal length (cm) 3
petal width (cm) 4
dtype: int64
Predicting on a single observation gives an error
>>> observation
sepal length (cm) 1
sepal width (cm) 2
petal length (cm) 3
petal width (cm) 4
dtype: int64
>>> model.predict(observation)
/Users/eswan18/Develop/sklearn-api-deploy/.venv/lib/python3.10/site-packages/sklearn/base.py:439: UserWarning: X does not have valid feature names, but LogisticRegression was fitted with feature names
warnings.warn(
Traceback (most recent call last):
<...omitted...>
ValueError: Expected 2D array, got 1D array instead:
array=[1 2 3 4].
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.
The model expects a 2-dimensional array or a DataFrame but we gave a Series (1-dimensional)
Luckily, we can easily turn an observation into a one-row DataFrame...
>>> obs_df = pd.DataFrame([observation])
>>> obs_df
sepal length (cm) sepal width (cm) petal length (cm) petal width (cm)
0 1 2 3 4
>>> predictions = model.predict(obs_df)
>>> predictions
array([2])
We get back an array with a prediction for every row -- although we only passed one.
>>> predictions
array([2])
>>> obs_prediction = predictions[0]
>>> obs_prediction
2
We'll need a mapping to convert the class number (here 2
) back to a flower name.
>>> class_flower_mapping = {
... 0: 'setosa',
... 1: 'versicolor',
... 2: 'virginica',
... }
>>> predicted_flower = class_flower_mapping[obs_prediction]
>>> predicted_flower
'virginica'
/predict
code.¶We left last section with this "dummy" /predict
endpoint.
@app.post("/predict", status_code=201)
def predict(obs: Observation) -> Prediction:
"""For now, just return a dummy prediction."""
return Prediction(flower_type="setosa")
Now...
CLASS_FLOWER_MAPPING = {
0: 'setosa',
1: 'versicolor',
2: 'virginica',
}
@app.post("/predict", status_code=201)
def predict(obs: Observation) -> Prediction:
"""For now, just return a dummy prediction."""
# .predict() gives us an array, but it has only one element
prediction = model.predict(obs.as_dataframe())[0]
flower_type = CLASS_FLOWER_MAPPING[prediction]
pred = Prediction(flower_type=flower_type)
return pred
We're missing just one piece -- the implemention of Observation.as_dataframe()
.
Updating our Observation with .as_row()
and as_dataframe()
:
import pandas as pd
class Observation(BaseModel):
"""An observation of a flower's measurements."""
sepal_length: float
sepal_width: float
petal_length: float
petal_width: float
def as_dataframe(self) -> pd.DataFrame:
"""Convert this record to a DataFrame with one row."""
return pd.DataFrame([self.as_row()])
def as_row(self) -> pd.Series:
row = pd.Series({
"sepal length (cm)": self.sepal_length,
"sepal width (cm)": self.sepal_width,
"petal length (cm)": self.petal_length,
"petal width (cm)": self.petal_width,
})
return row
Let's try it out!
$ curl -X POST localhost:8000/predict \
-d '{"sepal_length": 1, "sepal_width": 2, "petal_length": 3, "petal_width": 4}' \
-H "Content-Type: application/json"
{"flower_type":"virginica"}
Last thing: we need to update our test for /predict
...
def test_predict(client: TestClient):
# Test an obs that should come back as setosa
response = client.post(
"/predict",
json={
"sepal_length": 5.1,
"sepal_width": 3.5,
"petal_length": 1.4,
"petal_width": 0.2,
},
)
assert response.status_code == 201
payload = response.json()
assert payload["flower_type"] == "setosa"
# Test an obs that should come back as versicolor
response = client.post(
"/predict",
json={
"sepal_length": 7.1,
"sepal_width": 3.5,
"petal_length": 3.0,
"petal_width": 0.8,
},
)
assert response.status_code == 201
payload = response.json()
assert payload["flower_type"] == "versicolor"
load_model()
functionapp/main.py
Observation.as_row()
method, returning a pandas.Series
Observation.as_dataframe()
method, returning a pandas.DataFrame
/predict
endpoint with the real modelhttp://localhost:8000/docs
/predict
endpoint[7.1, 3.5, 3.0, 0.8]
-> versicolor
/batch_predict
endpointdef batch_predict(observations: List[Observation]) -> List[Prediction]:
Change Summary: eswan18.github.io/sklearn-api-deploy-slides/diffs/4.html
Installing an application as a package is a very good idea
requirements.txt
A true dependency management tool is an improvement and that's what I do in my own projects.
I use Poetry which is pretty mature and supports most of my needs
There's also Pipenv which is similar but I haven't used as much.
Other options: