BentoML makes it easy to create Machine Learning services that are ready to deploy and scale.
Join our Slack community today!
Looking deploy your ML service quickly? Checkout BentoML Cloud for the easiest and fastest way to deploy your bento.
Unified Model Serving API
Standardized process for a frictionless transition to production
Scalable with powerful performance optimizations
Deploy anywhere in a DevOps-friendly way
Save your trained model with BentoML:
import bentoml
saved_model = bentoml.pytorch.save_model(
"demo_mnist", # model name in the local model store
model, # model instance being saved
)
print(f"Model saved: {saved_model}")
# Model saved: Model(tag="demo_mnist:3qee3zd7lc4avuqj", path="~/bentoml/models/demo_mnist/3qee3zd7lc4avuqj/")
Define a prediction service in a service.py
file:
import numpy as np
import bentoml
from bentoml.io import NumpyNdarray, Image
from PIL.Image import Image as PILImage
mnist_runner = bentoml.pytorch.get("demo_mnist:latest").to_runner()
svc = bentoml.Service("pytorch_mnist", runners=[mnist_runner])
@svc.api(input=Image(), output=NumpyNdarray(dtype="int64"))
def predict(input_img: PILImage):
img_arr = np.array(input_img)/255.0
input_arr = np.expand_dims(img_arr, 0).astype("float32")
output_tensor = mnist_runner.predict.run(input_arr)
return output_tensor.numpy()
Create a bentofile.yaml
build file for your ML service:
service: "service:svc"
include:
- "*.py"
python:
packages:
- numpy
- torch
- Pillow
Now, run the prediction service:
bentoml serve
Sent a prediction request:
curl -F '[email protected]/1.png' http://127.0.0.1:3000/predict_image
Build a Bento and generate a docker image:
$ bentoml build
Successfully built Bento(tag="pytorch_mnist:4mymorgurocxjuqj") at "~/bentoml/bentos/pytorch_mnist/4mymorgurocxjuqj/"
$ bentoml containerize pytorch_mnist:4mymorgurocxjuqj
Successfully built docker image "pytorch_mnist:4mymorgurocxjuqj"
$ docker run -p 3000:3000 pytorch_mnist:4mymorgurocxjuqj
Starting production BentoServer from "pytorch_mnist:4mymorgurocxjuqj" running on http://0.0.0.0:3000
For a more detailed user guide, check out the BentoML Tutorial.
There are many ways to contribute to the project:
#bentoml-contributors
channel in the community slack.Thanks to all of our amazing contributors!
BentoML collects usage data that helps our team to improve the product.
Only BentoML's internal API calls are being reported. We strip out as much potentially
sensitive information as possible, and we will never collect user code, model data, model names, or stack traces.
Here's the code for usage tracking.
You can opt-out of usage tracking by the --do-not-track
CLI option:
bentoml [command] --do-not-track
Or by setting environment variable BENTOML_DO_NOT_TRACK=True
:
export BENTOML_DO_NOT_TRACK=True