Awesome Open Source

Programming Languages

Search results for python model serving

model-serving x

36 search results found

Vllm ⭐ 13,832

A high-throughput and memory-efficient inference and serving engine for LLMs

Deep Learning In Production ⭐ 4,138

In this repository, I will share some useful notes and references about deploying deep learning-based models in production.

Fedml ⭐ 3,946

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, FEDML Nexus AI (https://nexus.fedml.ai) is the dedicated cloud service for generative AI

Lightllm ⭐ 1,417

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

Mlrun ⭐ 1,177

Machine Learning automation and tracking

Hopsworks ⭐ 1,041

Hopsworks - Data-Intensive AI platform with a Feature Store

The simplest way to serve AI/ML models in production

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

Pinferencia ⭐ 473

Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

Stable Diffusion Deploy ⭐ 373

Learn to serve Stable Diffusion models on cloud infrastructure at scale. This Lightning App shows load-balancing, orchestrating, pre-provisioning, dynamic batching, GPU-inference, micro-services working together via the Lightning Apps framework.

Fastapi Ml Skeleton ⭐ 307

FastAPI Skeleton App to serve machine learning models production-ready.

Onediffusion ⭐ 293

OneDiffusion: Run any Stable Diffusion models and fine-tuned weights with ease

A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.

Gallery ⭐ 121

BentoML Example Projects 🎨

Clearml Serving ⭐ 115

ClearML - Model-Serving Orchestration and Repository Solution

Fastdeploy ⭐ 90

Deploy DL/ ML inference pipelines with minimal extra code.

The official python package for NimbleBox. Exposes all APIs as CLIs and contains modules to make ML 🌸

Monai Deploy App Sdk ⭐ 74

MONAI Deploy App SDK offers a framework and associated tools to design, develop and verify AI-driven applications in the healthcare imaging domain.

A minimalistic and pluggable machine learning platform for Kubernetes.

Serving LLMs on heterogeneous decentralized clusters.

Transformers Nlp Service ⭐ 26

Online Inference API for NLP Transformer models - summarization, text classification, sentiment analysis and more

MXNet Model Serving

Ocr As A Service ⭐ 24

Turn any OCR models into online inference API endpoint 🚀 🌖

mlserve turns your python models into RESTful API, serves web page with form generated to match your input data.

Sdk Python ⭐ 23

Python library for Modzy Machine Learning Operations (MLOps) Platform

Titus 2 : Portable Format for Analytics (PFA) implementation for Python 3.4+

Surround ⭐ 22

Surround is a framework for building AI driven microservices in Python, https://surround.readthedocs.io/en/latest/

Kedro Mlflow Tutorial ⭐ 21

A tutorial on how to use kedro-mlflow plugin (https://github.com/Galileo-Galilei/kedro-mlflow) to synchronize training and inference and serve kedro pipeline

Inferencedb ⭐ 20

🚀 Stream inferences of real-time ML models in production to any data lake

Kedro Serving ⭐ 13

A kedro-plugin to serve Kedro Pipelines as API

Diffusers Examples ⭐ 10

API serving for your diffusers models

Tfserving Demos ⭐ 9

TF Serving demos

Machine Learning Api ⭐ 7

Hopsworks Machine Learning Api 🚀 Model management with a model registry and model serving

Serving the deep learning models easily.

Openvino Model Server Wrapper ⭐ 5

Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code.

Ray_vllm_inference ⭐ 5

A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.

Related Searches

Python Django (28,897)

Python Machine Learning (20,195)

Python Flask (17,643)

Python Docker (14,113)

Python Tensorflow (13,736)

Python Deep Learning (13,092)

Python Jupyter Notebook (12,976)

Python Algorithms (10,033)

Python Natural Language Processing (9,064)

Python Artificial Intelligence (8,580)

1-36 of 36 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.