Awesome Open Source

Programming Languages

Search results for pytorch llm serving

7 search results found

Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

Vllm ⭐ 13,832

A high-throughput and memory-efficient inference and serving engine for LLMs

Superduperdb ⭐ 3,924

🔮 SuperDuperDB: Bring AI to your database! Build, deploy and manage any AI application directly with your existing data infrastructure, without moving your data. Including streaming inference, scalable model training and vector search.

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

Llm Finetuning Large Language Models ⭐ 120

LLM (Large Language Model) FineTuning

Ray_vllm_inference ⭐ 5

A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.

Related Searches

Python Pytorch (15,943)

Deep Learning Pytorch (7,533)

Jupyter Notebook Pytorch (4,892)

Machine Learning Pytorch (2,934)

Dataset Pytorch (1,848)

Pytorch Convolutional Neural Networks (1,777)

Pytorch Neural Network (1,631)

Pytorch Natural Language Processing (1,408)

Pytorch Computer Vision (1,230)

Pytorch Neural (1,217)

1-7 of 7 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.