Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for llama model serving
llama
x
model-serving
x
4 search results found
Vllm
⭐
13,832
A high-throughput and memory-efficient inference and serving engine for LLMs
Lightllm
⭐
1,417
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
Lorax
⭐
719
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
Rtp Llm
⭐
157
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
Related Searches
Llm Llama (268)
Python Llama (83)
1-4 of 4 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.
