Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

Powerinfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
Stars
6,416
License
mit
Open Issues
59
Most Recent Commit
3 months ago
Programming Language
C
Categories
Repo
Alternatives To Powerinfer
Select To Compare
Openllm ⭐ 7,871
Operating LLMs in production
dependent packages 2total releases 132most recent commit 3 months agopypi openllm} Downloads
Text Generation Inference ⭐ 6,743
Large Language Model Text Generation Inference
dependent packages 5total releases 13most recent commit 3 months agopypi text-generation} Downloads
Responder ⭐ 3,587
A familiar HTTP Service Framework for Python.
dependent packages 10total releases 48most recent commit a month agopypi responder} Downloads
Falcon ⭐ 2,407
A high-performance web server for Ruby, supporting HTTP/1, HTTP/2 and TLS.
dependent packages 16total releases 126most recent commit 4 months agorubygems falcon} Downloads


Suggest An Alternative To PowerInfer
Popular Operations Categories
Deployment
Logger
Metrics
Monitoring
Monitor
Prometheus
Grafana
Error Reporting
Tracing
Systemd
Related Searches
Llm Large Language Models
Llm Llama
Large Language Models Llama
C Falcon
Llm Falcon
Falcon Llama
Llama Llm Inference
C Llm
C Llama
Large Language Models Llm Inference

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
C
Falcon
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.