Using statically compiled dependencies whole package is shrunk to only
30MB. Previous approach to PyTorch network deployment on AWS Lambda (fastai) uses outdated PyTorch (
1.1.0) as dependency layer and requires AWS S3 to host your model. Now you can only use AWS Lambda and host your model as layer and PyTorch
masterand latest stable release are supported on a daily basis.
torchlambdahas at most
4commands and deployment is done via YAML settings. No need to modify your PyTorch code.
torchlambdafocuses solely on inference of PyTorch models on AWS Lambda.
aws-cli). You can also use your favorite third party tools (say saws, Terraform with AWS and MLFlow, PyTorch-Lightning to train your model).
torchlambdauses Amazon Linux 2 Docker images under the hood & allows you to use lambci/docker-lambda to test your deployment on
localhostbefore pushing deployment to the cloud (see Test Lambda deployment locally tutorial).
--flags(changing various properties of PyTorch and AWS dependencies themselves). You can also write your own C++ deployment code (generate template via
3000LOC (most being convenience wrapper creating this tool) make it easy to jump into source code and check what's going on under the hood.
Benchmarks can be seen in
BENCHMARKS.md file and are comprised of around ~30000 test cases.
Results are divided based on settings used, model type, payload, AWS Lambda timing etc. Below is an example of how inference performance changes due to higher resolution images and type of encoding:
Clearly the bigger image, the more important it is to use
base64 encoding. For all results and description click here.