Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for llm inference
llm-inference
x
71 search results found
Gpt4all
⭐
60,352
gpt4all: open-source LLM chatbots that you can run anywhere
Autogen
⭐
20,880
Enable Next-Gen Large Language Model Applications. Join our Discord: https://discord.gg/pAbnFJrkgZ
Openllm
⭐
7,871
Operating LLMs in production
Mistral Src
⭐
6,645
Reference implementation of Mistral AI 7B v0.1 model.
Powerinfer
⭐
6,416
High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
Openvino
⭐
5,979
OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
Superduperdb
⭐
3,924
🔮 SuperDuperDB: Bring AI to your database! Build, deploy and manage any AI application directly with your existing data infrastructure, without moving your data. Including streaming inference, scalable model training and vector search.
Llm Action
⭐
2,955
本项目旨在分享大模型相关技术原理以及实战经验。
Deepsparse
⭐
2,729
Sparsity-aware deep learning inference runtime for CPUs
Llama2 Webui
⭐
1,797
Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
Lmdeploy
⭐
1,762
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
Intel Extension For Transformers
⭐
1,712
⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡
Medusa
⭐
1,483
Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads
Ray Llm
⭐
972
RayLLM - LLMs on Ray
Lorax
⭐
719
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
Leancopilot
⭐
593
LLMs as Copilots for Theorem Proving in Lean
Llmflows
⭐
565
LLMFlows - Simple, Explicit and Transparent LLM Apps
Generativeaiexamples
⭐
458
Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
Sagify
⭐
422
LLMs and Machine Learning done easily
Llm Vm
⭐
401
irresponsible innovation. Try now at https://chat.dev/
Aquila2
⭐
376
The official repo of Aquila2 series proposed by BAAI, including pretrained & chat large language models.
Distributed Llama
⭐
292
Run LLMs on weak devices or make powerful devices even more powerful by distributing the workload and dividing the RAM usage.
Swiftinfer
⭐
277
Efficient AI Inference & Serving
Ray Educational Materials
⭐
232
This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.
Talkingheads
⭐
166
A Headless Chrome interface to communicate with Google Bard, HugginChat, OpenAI ChatGPT, and Pi
Runbooks
⭐
151
Finetune LLMs on K8s by using Runbooks
Bespoke_automata
⭐
146
Bespoke Automata is a GUI and deployment pipline for making complex AI agents locally and offline
Llmtuner
⭐
137
Tune LLM in few lines of code
Inferflow
⭐
135
Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
Llmunity
⭐
130
Integrate LLM models in Unity!
Ialacol
⭐
127
🪶 Lightweight OpenAI drop-in replacement for Kubernetes
Llm Finetuning Large Language Models
⭐
120
LLM (Large Language Model) FineTuning
Llm Api
⭐
109
Run any Large Language Model behind a unified API
Llm.swift
⭐
100
LLM.swift is a simple, and readable library which lets you locally interact with LLMs with ease for macOS, iOS, visionOS, watchOS, and tvOS.
Nos
⭐
89
⚡️ Nitro boost your AI infrastructure.
Hackbot
⭐
79
AI-powered cybersecurity chatbot designed to provide helpful and accurate answers to your cybersecurity-related queries and also do code analysis and scan analysis.
Neural Speed
⭐
77
An innovation library for efficient LLM inference via low-bit quantization and sparsity
Libre Chat
⭐
71
🦙 Free and Open Source Large Language Model (LLM) chatbot web UI and API. Self-hosted, offline capable and easy to setup. Powered by LangChain.
Prompt Highlighter
⭐
69
Prompt Highlighter: Interactive Control for Multi-Modal LLMs
Ecoassistant
⭐
69
EcoAssistant: using LLM assistant more affordably and accurately
Llm Powerhouse A Curated Guide For Large Language Models With Custom Training And Inferencing
⭐
61
LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.
Local Llm Function Calling
⭐
55
A tool for generating function arguments and choosing what function to call with local LLMs
Llama2.zig
⭐
53
Inference Llama 2 in one file of pure Zig
Aibitat
⭐
40
Multi-Agent Conversation Framework in TypeScript
Ht
⭐
26
ht - a shell command that answers your questions about shell commands
Friendli Client
⭐
23
Friendli: the fastest serving engine for generative AI such as LLMs
Spatten Llm
⭐
22
[HPCA'21] SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning
Companionllm
⭐
21
CompanionLLM - A framework to finetune LLMs to be your own sentient conversational companion
Local Llm
⭐
21
支持chatglm.cpp和llama_cpp的一键安装启动
Gpt 4 Enem
⭐
20
Code and data to evaluate LLMs on the ENEM, the main standardized Brazilian university admission exams.
Exa
⭐
17
Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and minimal learning curve.
Export_llama_to_onnx
⭐
14
export llama to onnx
Aoororachain
⭐
13
Aoororachain is Ruby chain tool to work with LLMs
Arcee Python
⭐
12
The Arcee client for executing domain-adpated language model routines
Llm Inference Solutions
⭐
12
A collection of all available inference solutions for the LLMs
Palmhill.blazorchat
⭐
11
PalmHill.BlazorChat is a chat application and API built with Blazor WebAssembly, SignalR, and WebAPI, featuring real-time LLM conversations, markdown support, customizable settings, and a responsive design. This project supports Llama2 models and was tested with Orca2.
Llm Minutes Of Meeting
⭐
11
🎤📄 An innovative tool that transforms audio or video files into text transcripts and generates concise meeting minutes. Stay organized and efficient in your meetings, and get ready for Phase 2 where we'll be open for contributions to enable real-time meeting transcription! 🚀
Llm Sharp
⭐
10
Language models in C#
Instruct Finetune Mistral
⭐
10
Fine-tune Mistral 7B to generate fashion style suggestions
Prompter.vim
⭐
9
vim as a perfect large language models prompts playground
Llm Vscode Inference Server
⭐
9
An endpoint server for efficiently serving quantized open-source LLMs for code.
Saycanpay
⭐
8
Official code release of AAAI 2024 paper SayCanPay.
Chitchat
⭐
7
Modal LLM LLama.cpp based model deployment as part of series of Model as a Service (MaaS)
Promptbook
⭐
7
Library to supercharge your use of large language models
Awesome Llm Productization
⭐
6
Awesome-LLM-Productization: a curated list of tools/tricks/news/regulations about AI and Large Language Model (LLM) productization
Tree Prompt
⭐
6
Tree prompting: easy-to-use scikit-learn interface for improved prompting.
Cttc
⭐
6
Analyze cyber threat research post from given URLs and get insights with the help of ChatGPT
Llms In Prod Workshop 2023
⭐
6
Deploy and Scale LLM-based applications
Anyscale Berkeley Ai Hackathon
⭐
5
Ray and Anyscale for UC Berkeley AI Hackathon!
Taskyon
⭐
5
Browser based Interface for Generative AI. Chat/Agent/Taskmanager Hybrid.
Chat_prompt_templates
⭐
5
Collection of Basic Prompt Templates for Various Chat LLMs (Chat LLM 的基础提示模板集合)
Related Searches
Python Llm Inference (53)
1-71 of 71 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.