|Project Name||Stars||Downloads||Repos Using This||Packages Using This||Most Recent Commit||Total Releases||Latest Release||Open Issues||License||Language|
|Tensorflow||179,113||327||78||11 hours ago||46||October 23, 2019||2,085||apache-2.0||C++|
|An Open Source Machine Learning Framework for Everyone|
|Transformers||116,182||64||2,452||11 hours ago||125||November 15, 2023||917||apache-2.0||Python|
|🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.|
|Pytorch||73,068||3,341||8,194||11 hours ago||39||November 15, 2023||13,168||other||Python|
|Tensors and Dynamic neural networks in Python with strong GPU acceleration|
|Cs Video Courses||61,557||19 days ago||2|
|List of Computer Science courses with video lectures.|
|Keras||59,873||688||17 hours ago||86||November 28, 2023||120||apache-2.0||Python|
|Deep Learning for humans|
|D2l Zh||51,074||1||1||8 days ago||51||August 18, 2023||59||apache-2.0||Python|
|Faceswap||47,636||8 days ago||19||gpl-3.0||Python|
|Deepfakes Software For All|
|Yolov5||43,580||2 days ago||8||September 21, 2021||204||agpl-3.0||Python|
|YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite|
|Deepfacelab||43,455||a month ago||547||gpl-3.0||Python|
|DeepFaceLab is the leading software for creating deepfakes.|
|Tensorflow Examples||42,312||a year ago||218||other||Jupyter Notebook|
|TensorFlow Tutorial and Examples for Beginners (support TF v1 & v2)|
DeepSpeed empowers ChatGPT-like model training with a single click, offering 15x speedup over SOTA RLHF systems with unprecedented cost reduction at all scales; learn how.
***DeepSpeed enables world's most powerful language models like MT-530B and BLOOM***. It is an easy-to-use deep learning optimization software suite that powers unprecedented scale and speed for both training and inference. With DeepSpeed you can:
DeepSpeed offers a confluence of system innovations, that has made large scale DL training effective, and efficient, greatly improved ease of use, and redefined the DL training landscape in terms of scale that is possible. These innovations such as ZeRO, 3D-Parallelism, DeepSpeed-MoE, ZeRO-Infinity, etc. fall under the training pillar. Learn more: DeepSpeed-Training
DeepSpeed brings together innovations in parallelism technology such as tensor, pipeline, expert and ZeRO-parallelism, and combines them with high performance custom inference kernels, communication optimizations and heterogeneous memory technologies to enable inference at an unprecedented scale, while achieving unparalleled latency, throughput and cost reduction. This systematic composition of system technologies for inference falls under the inference pillar. Learn more: DeepSpeed-Inference
To further increase the inference efficiency, DeepSpeed offers easy-to-use and flexible-to-compose compression techniques for researchers and practitioners to compress their models while delivering faster speed, smaller model size, and significantly reduced compression cost. Moreover, SoTA innovations on compression like ZeroQuant and XTC are included under the compression pillar. Learn more: DeepSpeed-Compression
In line with Microsoft's mission to solve humanity's most pressing challenges, the DeepSpeed team at Microsoft is responding to this opportunity by launching a new initiative called DeepSpeed4Science, aiming to build unique capabilities through AI system technology innovations to help domain experts to unlock today's biggest science mysteries. Learn more: DeepSpeed4Science website and tutorials
The DeepSpeed library (this repository) implements and packages the innovations and technologies in DeepSpeed Training, Inference and Compression Pillars into a single easy-to-use, open-sourced repository. It allows for easy composition of multitude of features within a single training, inference or compression pipeline. The DeepSpeed Library is heavily adopted by the DL community, and has been used to enable some of the most powerful models (see DeepSpeed Adoption).
Model Implementations for Inference (MII) is an open-sourced repository for making low-latency and high-throughput inference accessible to all data scientists by alleviating the need to apply complex system optimization techniques themselves. Out-of-box, MII offers support for thousands of widely used DL models, optimized using DeepSpeed-Inference, that can be deployed with a few lines of code, while achieving significant latency reduction compared to their vanilla open-sourced versions.
DeepSpeed users are diverse and have access to different environments. We recommend to try DeepSpeed on Azure as it is the simplest and easiest method. The recommended method to try DeepSpeed on Azure is through AzureML recipes. The job submission and data preparation scripts have been made available here. For more details on how to use DeepSpeed on Azure, please follow the Azure tutorial.
DeepSpeed has been used to train many different large-scale models, below is a list of several examples that we are aware of (if you'd like to include your model please submit a PR):
DeepSpeed has been integrated with several different popular open-source DL frameworks such as:
|Transformers with DeepSpeed|
|Accelerate with DeepSpeed|
|Lightning with DeepSpeed|
|MosaicML with DeepSpeed|
|Determined with DeepSpeed|
|MMEngine with DeepSpeed|
The quickest way to get started with DeepSpeed is via pip, this will install the latest release of DeepSpeed which is not tied to specific PyTorch or CUDA versions. DeepSpeed includes several C++/CUDA extensions that we commonly refer to as our 'ops'. By default, all of these extensions/ops will be built just-in-time (JIT) using torch's JIT C++ extension loader that relies on ninja to build and dynamically link them at runtime.
We regularly push releases to PyPI and encourage users to install from there in most cases.
pip install deepspeed
After installation, you can validate your install and see which extensions/ops your machine is compatible with via the DeepSpeed environment report.
If you would like to pre-install any of the DeepSpeed extensions/ops (instead of JIT compiling) or install pre-compiled ops via PyPI please see our advanced installation instructions.
Windows support is partially supported with DeepSpeed. On Windows you can build wheel with following steps, currently only inference mode is supported.
python setup.py bdist_wheelto build wheel in
All DeepSpeed documentation, tutorials, and blogs can be found on our website: deepspeed.ai
|Getting Started||First steps with DeepSpeed|
|DeepSpeed JSON Configuration||Configuring DeepSpeed|
|API Documentation||Generated DeepSpeed API documentation|
DeepSpeed welcomes your contributions! Please see our
contributing guide for more details on formatting, testing,
Thanks so much to all of our amazing contributors!
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.