Tengine Overview

Tengine, developed by OPEN AI LAB, is an AI application development platform for AIoT scenarios launched by OPEN AI LAB, which is dedicated to solving the fragmentation problem of aiot industrial chain and accelerating the landing of AI industrialization. Tengine is specially designed for AIoT scenarios, and it has several features, such as cross platform, heterogeneous scheduling, chip bottom acceleration, ultra light weight and independent, and complete development and deployment tool chain. Tengine is compatible with a variety of operating systems and deep learning algorithm framework, which simplifies and accelerates the rapid migration of scene oriented AI algorithm on embedded edge devices, as well as the actual application deployment;

Tengine is composed of five modules: core/operator/serializer/executor/driver.

core provides the basic components and functionalities of the system.
operator defines the schema of operators, such as convolution, relu, pooling, etc. al. Here is the current support operator list.
serializer is to load the saved model. The serializer framework is extensible to support different format, including the customized one. Caffe/ONNX/Tensorflow/MXNet and Tengine models can be loaded directly by Tengine.
executor implements the code to run graph and operators. Current version provides a highly optimized implementation for multi A72 cores.
driver is the adapter of real H/W and provides service to device executor by HAL API. It is possible for single driver to create multiple devices.

Build and Install

please refer to Wiki

Tengine examples and model zoo

please visit examples for demos on classification/detection and download models from Tengine model zoo (psw: hhgc)

tengine applications is a project for sharing android/linux applications powered by Tengine

Communication && Tech Support

Github issues
QQ group: 829565581 (Question:Tengine Answer:openailab)
email: [email protected]
Tengine Community: http://www.tengine.org.cn/

Benchmark

Test on RK3399-1*A72

Model	fp32	int8-hybrid	int8-e2e
Squeezenet v1.1	55.3ms	48.6ms	44.6ms
Mobilenet v1	108.7ms	74.6ms	64.2ms

More Benchmark data to be added.

Roadmap

2020.5 updated

Feature

[ ] More examples
[ ] Web-based convert tool
[ ] CV API
[ ] Support more ops of ONNX(PyTorch)

Optimization

[ ] arm v8.2 fp16 perf ops

Project Name	Stars	Repos Using This	Packages Using This	Most Recent Commit	Total Releases	Latest Release	Open Issues	License	Language
Transformers	101,817	64	911	9 hours ago	91	June 21, 2022	719	apache-2.0	Python
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
Pytorch	67,250		146	10 hours ago	23	August 10, 2022	12,004	other	Python
Tensors and Dynamic neural networks in Python with strong GPU acceleration
Yolov5	38,857			a day ago	35	May 21, 2022	346	agpl-3.0	Python
YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite
Made With Ml	33,193			a month ago	5	May 15, 2019	11	mit	Jupyter Notebook
Learn how to responsibly develop, deploy and maintain production machine learning applications.
Ray	25,767	80	199	10 hours ago	76	June 09, 2022	2,839	apache-2.0	Python
Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a toolkit of libraries (Ray AIR) for accelerating ML workloads.
Deepspeed	25,073		12	15 hours ago	53	May 25, 2022	864	apache-2.0	Python
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Fastai	23,942	184	117	a day ago	143	July 04, 2022	156	apache-2.0	Jupyter Notebook
The fastai deep learning library
Lightning	23,372	7	389	20 hours ago	221	June 01, 2022	678	apache-2.0	Python
Deep learning framework to train, deploy, and ship AI products Lightning fast.
Netron	22,896	4	63	13 hours ago	489	July 04, 2022	26	mit	JavaScript
Visualizer for neural network, deep learning, and machine learning models
Annotated_deep_learning_paper_implementations	22,464		1	16 days ago	76	June 27, 2022	17	mit	Jupyter Notebook
🧑‍🏫 59 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠