Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Deep Learning Inference Open Source Projects

Open source projects categorized as Deep Learning Inference
Categories > Machine Learning > Deep Learning
Categories > Machine Learning > Inference
Yolov5 ⭐ 44,755
YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite
dependent packages 2total releases 3most recent commit 3 months agopypi ultralytics-yolov5} Downloads
Colossalai ⭐ 37,814
Making large AI models cheaper, faster and more accessible
most recent commit 21 hours ago
Deepspeed ⭐ 32,358
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
dependent packages 87total releases 79most recent commit 10 days agopypi deepspeed} Downloads
Mediapipe ⭐ 24,511
Cross-platform, customizable ML solutions for live and streaming media.
dependent packages 190total releases 38most recent commit 3 months agopypi mediapipe} Downloads
Ncnn ⭐ 19,097
ncnn is a high-performance neural network inference framework optimized for the mobile platform
dependent packages 1total releases 26most recent commit 11 days agopypi ncnn} Downloads
Yolov3 ⭐ 9,865
YOLOv3 in PyTorch > ONNX > CoreML > TFLite
most recent commit 3 months ago
Amazon Sagemaker Examples ⭐ 9,221
Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
most recent commit 3 months ago
Tensorrt ⭐ 8,908
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
dependent packages 5total releases 4most recent commit 2 months agopypi polygraphy} Downloads
Faster Whisper ⭐ 8,003
Faster Whisper transcription with CTranslate2
dependent packages 22total releases 12most recent commit a month agopypi faster-whisper} Downloads
Jetson Inference ⭐ 7,038
Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
most recent commit 3 months ago
Server ⭐ 6,981
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
total releases 64most recent commit a month ago
Text Generation Inference ⭐ 6,743
Large Language Model Text Generation Inference
dependent packages 5total releases 13most recent commit 3 months agopypi text-generation} Downloads
Serving ⭐ 6,045
A flexible, high-performance serving system for machine learning models
most recent commit 3 months ago
Nanodet ⭐ 5,416
NanoDet-Plus⚡Super fast and lightweight anchor-free object detection model. 🔥Only 980 KB(int8) / 1.8MB (fp16) and run 97FPS on cellphone🔥
most recent commit 3 months ago
Openvino ⭐ 5,316
OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
total releases 3most recent commit 3 months ago
Edward ⭐ 4,503
A probabilistic programming language in TensorFlow. Deep generative models, variational inference.
dependent packages 1total releases 28most recent commit 4 years agopypi Edward} Downloads
Tnn ⭐ 4,210
TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep learning inference framework for mobile、desktop and server. TNN is distinguished by several outstanding features, including its cross-platform capability, high performance, model compression and code pruning. Based on ncnn and Rapidnet, TNN further strengthens the support and performance optimization for mobile devices, and also draws on the advantages of good extensibility and high performance from existed open source efforts
most recent commit 3 months ago
Probability ⭐ 4,028
Probabilistic reasoning and statistical analysis in TensorFlow
dependent packages 316total releases 51most recent commit 3 months agopypi tensorflow-probability} Downloads
Serve ⭐ 3,844
Serve, optimize and scale PyTorch models in production
dependent packages 15total releases 22most recent commit 3 months agopypi torchserve} Downloads
Autogptq ⭐ 3,637
An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
most recent commit 18 days ago
3ddfa ⭐ 3,243
The PyTorch improved version of TPAMI 2017 paper: Face Alignment in Full Pose Range: A 3D Total Solution.
most recent commit 2 years ago
Ctranslate2 ⭐ 2,437
Fast inference engine for Transformer models
dependent packages 23total releases 103most recent commit 3 months agopypi ctranslate2} Downloads
Awesome System For Machine Learning ⭐ 2,244
A curated list of research in machine learning systems (MLSys). Paper notes are also provided.
most recent commit 4 months ago
The 10 Most Depended On Deep Learning Inference Open Source Projects
Probability ⭐ 4,028
Probabilistic reasoning and statistical analysis in TensorFlow
dependent packages 316total releases 51latest release November 20, 2023most recent commit 3 months agopypi tensorflow-probability} Downloads
Mediapipe ⭐ 24,511
Cross-platform, customizable ML solutions for live and streaming media.
dependent packages 190total releases 38latest release November 09, 2023most recent commit 3 months agopypi mediapipe} Downloads
Deepspeed ⭐ 32,358
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
dependent packages 87total releases 79latest release December 01, 2023most recent commit 10 days agopypi deepspeed} Downloads
Ctranslate2 ⭐ 2,437
Fast inference engine for Transformer models
dependent packages 23total releases 103latest release December 05, 2023most recent commit 3 months agopypi ctranslate2} Downloads
Faster Whisper ⭐ 8,003
Faster Whisper transcription with CTranslate2
dependent packages 22total releases 12latest release November 26, 2023most recent commit a month agopypi faster-whisper} Downloads
Serve ⭐ 3,844
Serve, optimize and scale PyTorch models in production
dependent packages 15total releases 22latest release October 12, 2023most recent commit 3 months agopypi torchserve} Downloads
Gotch ⭐ 444
Go binding for Pytorch C++ API (libtorch)
dependent packages 6total releases 72latest release October 30, 2023most recent commit 5 months ago
Text Generation Inference ⭐ 6,743
Large Language Model Text Generation Inference
dependent packages 5total releases 13latest release September 28, 2023most recent commit 3 months agopypi text-generation} Downloads
Serving ⭐ 834
A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）
dependent packages 4total releases 21latest release May 24, 2022most recent commit 7 months agopypi paddle-serving-client} Downloads
The 10 Latest Releases In Deep Learning Inference Open Source Projects
Pytriton ⭐ 611
PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.
dependent packages 2total releases 13latest release December 06, 2023most recent commit 3 months agopypi nvidia-pytriton} Downloads
Ctranslate2 ⭐ 2,437
Fast inference engine for Transformer models
dependent packages 23total releases 103latest release December 05, 2023most recent commit 3 months agopypi ctranslate2} Downloads
Deepspeed Mii ⭐ 1,437
MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
total releases 12latest release December 01, 2023most recent commit 3 months ago
Deepspeed ⭐ 32,358
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
dependent packages 87total releases 79latest release December 01, 2023most recent commit 10 days agopypi deepspeed} Downloads
Fastdeploy ⭐ 90
Deploy DL/ ML inference pipelines with minimal extra code.
total releases 56latest release November 29, 2023most recent commit 4 months ago
Faster Whisper ⭐ 8,003
Faster Whisper transcription with CTranslate2
dependent packages 22total releases 12latest release November 26, 2023most recent commit a month agopypi faster-whisper} Downloads
Probability ⭐ 4,028
Probabilistic reasoning and statistical analysis in TensorFlow
dependent packages 316total releases 51latest release November 20, 2023most recent commit 3 months agopypi tensorflow-probability} Downloads
Mediapipe ⭐ 24,511
Cross-platform, customizable ML solutions for live and streaming media.
dependent packages 190total releases 38latest release November 09, 2023most recent commit 3 months agopypi mediapipe} Downloads
Popular Machine Learning Categories
Deep Learning
Machine Learning
Pytorch
Tensorflow
Natural Language Processing
Neural Network
Neural
Computer Vision
Convolutional Neural Networks
Opencv
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.