Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for model serving
model-serving
x
67 search results found
Vllm
⭐
13,832
A high-throughput and memory-efficient inference and serving engine for LLMs
Bentoml
⭐
6,575
The most flexible way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Inference Graph/Pipelines, Compound AI systems, Multi-Modal, RAG as a Service, and more!
Deep Learning In Production
⭐
4,138
In this repository, I will share some useful notes and references about deploying deep learning-based models in production.
Fedml
⭐
3,946
FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, FEDML Nexus AI (https://nexus.fedml.ai) is the dedicated cloud service for generative AI
Kserve
⭐
3,239
Standardized Serverless ML Inference Platform on Kubernetes
Envd
⭐
1,869
🏕️ Reproducible development environment
Lightllm
⭐
1,417
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
Mlrun
⭐
1,177
Machine Learning automation and tracking
Hopsworks
⭐
1,041
Hopsworks - Data-Intensive AI platform with a Feature Store
Truss
⭐
776
The simplest way to serve AI/ML models in production
Yatai
⭐
748
Model Deployment at Scale on Kubernetes 🦄️
Lorax
⭐
719
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
Mosec
⭐
661
A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine
Model_server
⭐
618
A scalable inference server for models optimized with OpenVINO™
Pinferencia
⭐
473
Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
Stable Diffusion Deploy
⭐
373
Learn to serve Stable Diffusion models on cloud infrastructure at scale. This Lightning App shows load-balancing, orchestrating, pre-provisioning, dynamic batching, GPU-inference, micro-services working together via the Lightning Apps framework.
Fastapi Ml Skeleton
⭐
307
FastAPI Skeleton App to serve machine learning models production-ready.
Onediffusion
⭐
293
OneDiffusion: Run any Stable Diffusion models and fine-tuned weights with ease
Chitra
⭐
219
A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.
Kafka With Akka Streams Kafka Streams Tutorial
⭐
191
Code samples for the Lightbend tutorial on writing microservices with Akka Streams, Kafka Streams, and Kafka
Rtp Llm
⭐
157
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
Zoltar
⭐
140
Common library for serving TensorFlow, XGBoost and scikit-learn models in production.
Fate Serving
⭐
131
A scalable, high-performance serving system for federated learning models
Gallery
⭐
121
BentoML Example Projects 🎨
Clearml Serving
⭐
115
ClearML - Model-Serving Orchestration and Repository Solution
Fastdeploy
⭐
90
Deploy DL/ ML inference pipelines with minimal extra code.
Nbox
⭐
84
The official python package for NimbleBox. Exposes all APIs as CLIs and contains modules to make ML 🌸
Flink Jpmml
⭐
82
flink-jpmml is a fresh-made library for dynamic real time machine learning predictions built on top of PMML standard models and Apache Flink streaming engine
Monai Deploy App Sdk
⭐
74
MONAI Deploy App SDK offers a framework and associated tools to design, develop and verify AI-driven applications in the healthcare imaging domain.
Fasttext Serving
⭐
57
fastText model serving service
Model Serving Tutorial
⭐
53
Code and presentation for Strata Model Serving tutorial
Fdp Modelserver
⭐
45
An umbrella project for multiple implementations of model serving
Serving Pytorch Models
⭐
43
Serving PyTorch models with TorchServe 🔥
Serving Tensorflow Models
⭐
37
Serving TensorFlow models with TensorFlow Serving 📙
Stackn
⭐
36
A minimalistic and pluggable machine learning platform for Kubernetes.
Ml Workflow
⭐
35
A hands-on case study for demonstrating the stages involved in a machine learning project, from EDA to production.
Hexgen
⭐
30
Serving LLMs on heterogeneous decentralized clusters.
Clip Api Service
⭐
27
CLIP as a service - Embed image and sentences, object recognition, visual reasoning, image classification and reverse image search
Transformers Nlp Service
⭐
26
Online Inference API for NLP Transformer models - summarization, text classification, sentiment analysis and more
Mms
⭐
25
MXNet Model Serving
Ocr As A Service
⭐
24
Turn any OCR models into online inference API endpoint 🚀 🌖
Sdk Python
⭐
23
Python library for Modzy Machine Learning Operations (MLOps) Platform
Mlserve
⭐
23
mlserve turns your python models into RESTful API, serves web page with form generated to match your input data.
Surround
⭐
22
Surround is a framework for building AI driven microservices in Python, https://surround.readthedocs.io/en/latest/
Titus2
⭐
22
Titus 2 : Portable Format for Analytics (PFA) implementation for Python 3.4+
Kedro Mlflow Tutorial
⭐
21
A tutorial on how to use kedro-mlflow plugin (https://github.com/Galileo-Galilei/kedro-mlflow) to synchronize training and inference and serve kedro pipeline
Inferencedb
⭐
20
🚀 Stream inferences of real-time ML models in production to any data lake
Console
⭐
20
⛅ Versatile Data Pipeline (VDP) console website
Dai Deployment Templates
⭐
18
Production ready templates for deploying Driverless AI (DAI) scorers. https://h2oai.github.io/dai-deployment-templates/
Flink Modelserver
⭐
16
Generic Model Serving Implementation leveraging Flink
Drogon Torch Serve
⭐
14
Serve pytorch / torch models using Drogon
Kubeflow Recommender
⭐
14
Kubeflow example of machine learning/model serving
Kedro Serving
⭐
13
A kedro-plugin to serve Kedro Pipelines as API
Model_deployment
⭐
12
A collection of model deployment library and technique.
Hugging Face Raspberry Pi
⭐
10
Deploy, serve, and run a Hugging Face model on a Raspberry Pi with just a few lines of code
Diffusers Examples
⭐
10
API serving for your diffusers models
Fdp Beam Modelserver
⭐
9
Model serving using Beam
Tfserving Demos
⭐
9
TF Serving demos
Fraud Detection Model Serving
⭐
8
Online model serving with Fraud Detection model trained with XGBoost on IEEE-CIS dataset
Rhods Mnist
⭐
7
Data science pipelines and model serving using Red Hat OpenShift Data Science
Machine Learning Api
⭐
7
Hopsworks Machine Learning Api 🚀 Model management with a model registry and model serving
Pipelines Model Serving
⭐
6
Implementation of Model serving in pipelines
Ventu
⭐
6
Serving the deep learning models easily.
Flink Speculative Modelserver
⭐
6
Speculative model serving with Flink
Ray_vllm_inference
⭐
5
A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
Openvino Model Server Wrapper
⭐
5
Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code.
Fdp Speculative Model Serving
⭐
5
Experimental implementation of speculative model serving
1-67 of 67 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.