Bmw Tensorflow Inference Api Gpu

This is a repository for an object detection inference API using the Tensorflow framework.
Alternatives To Bmw Tensorflow Inference Api Gpu
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
25 days ago5March 05, 20191,947otherPython
Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow
13 hours ago2September 08, 2022495mitTypeScript
Annotate better with CVAT, the industry-leading data engine for machine learning. Used and trusted by teams at any scale, for data of any scale.
15 hours ago306mitPython
NVR with realtime local object detection for IP cameras
Tensorlayer7,1883464 months ago83February 15, 202230otherPython
Deep Learning and Reinforcement Learning Library for Scientists and Engineers
4 days agoapache-2.0
Techniques for deep learning with satellite & aerial imagery
2 months ago644gpl-3.0Python
Translate darknet to tensorflow. Load trained weights, retrain/fine-tune using tensorflow, export constant graph def to mobile devices
Ssd Tensorflow4,049
2 years ago298Jupyter Notebook
Single Shot MultiBox Detector in TensorFlow
8 months ago174otherPython
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Tensorflow Yolov33,596
7 months ago490mitPython
🔥 TensorFlow Code for technical report: "YOLOv3: An Incremental Improvement"
Tf Faster Rcnn3,595
2 years ago215mitPython
Tensorflow Faster RCNN for Object Detection
Alternatives To Bmw Tensorflow Inference Api Gpu
Select To Compare

Alternative Project Comparisons

Tensorflow GPU Inference API

This is a repository for an object detection inference API using the Tensorflow framework.

This repo is based on Tensorflow Object Detection API.

The Tensorflow version used is 1.13.1. The inference REST API works on GPU. It's only supported on Linux Operating systems.

Models trained using our training tensorflow repository can be deployed in this API. Several object detection models can be loaded and used at the same time. This repo also offers optical character recognition services to extract textboxes from images.

This repo can be deployed using either docker or docker swarm.

Please use docker swarm only if you need to:

  • Provide redundancy in terms of API containers: In case a container went down, the incoming requests will be redirected to another running instance.

  • Coordinate between the containers: Swarm will orchestrate between the APIs and choose one of them to listen to the incoming request.

  • Scale up the Inference service in order to get a faster prediction especially if there's traffic on the service.

If none of the aforementioned requirements are needed, simply use docker.

predict image


  • Ubuntu 18.04
  • NVIDIA Drivers (410.x or higher)
  • Docker CE latest stable release
  • NVIDIA Docker 2

Check for prerequisites

To check if you have docker-ce installed:

docker --version

To check if you have nvidia-docker installed:

nvidia-docker --version

To check your nvidia drivers version, open your terminal and type the command nvidia-smi


Install prerequisites

Use the following command to install docker on Ubuntu:

chmod +x && source

Install NVIDIA Drivers (410.x or higher) and NVIDIA Docker for GPU by following the official docs

Build The Docker Image

In order to build the project run the following command from the project's root directory:

sudo docker build -t tensorflow_inference_api_gpu -f docker/dockerfile .

Behind a proxy

sudo docker build --build-arg http_proxy='' --build-arg https_proxy='' -t tensorflow_inference_api_gpu -f ./docker/dockerfile .

Run the docker container

As mentioned before, this container can be deployed using either docker or docker swarm.

If you wish to deploy this API using docker, please issue the following run command.

If you wish to deploy this API using docker swarm, please refer to following link docker swarm documentation. After deploying the API with docker swarm, please consider returning to this documentation for further information about the API endpoints as well as the model structure sections.

To run the API, go the to the API's directory and run the following:

Using Linux based docker:

sudo NV_GPU=0 nvidia-docker run -itv $(pwd)/models:/models -v $(pwd)/models_hash:/models_hash -p <docker_host_port>:4343 tensorflow_inference_api_gpu

The <docker_host_port> can be any unique port of your choice.

The API file will be run automatically, and the service will listen to http requests on the chosen port.

NV_GPU defines on which GPU you want the API to run. If you want the API to run on multiple GPUs just enter multiple numbers seperated by a comma: (NV_GPU=0,1 for example)

API Endpoints

To see all available endpoints, open your favorite browser and navigate to:


The 'predict_batch' endpoint is not shown on swagger. The list of files input is not yet supported.

P.S: If you are using custom endpoints like /load, /detect, and /get_labels, you should always use the /load endpoint first and then use /detect or /get_labels

Endpoints summary

/load (GET)

Loads all available models and returns every model with it's hashed value. Loaded models are stored and aren't loaded again

load model

/detect (POST)

Performs inference on specified model, image, and returns bounding-boxes

detect image

/get_labels (POST)

Returns all of the specified model labels with their hashed values

get model labels

/models/{model_name}/predict_image (POST)

Performs inference on specified model, image, draws bounding boxes on the image, and returns the actual image as response

predict image

/models (GET)

Lists all available models

/models/{model_name}/load (GET)

Loads the specified model. Loaded models are stored and aren't loaded again

/models/{model_name}/predict (POST)

Performs inference on specified model, image, and returns bounding boxes.

/models/{model_name}/labels (GET)

Returns all of the specified model labels

/models/{model_name}/config (GET)

Returns the specified model's configuration

/models/{model_name}/predict_batch (POST)

Performs inference on specified model and a list of images, and returns bounding boxes

/models/{model_name}/one_shot_ocr (POST)

Takes an image and returns extracted text details. In first place a detection model will be used for cropping interesting areas in the uploaded image. Then, these areas will be passed to the OCR-Service for text extraction.

/models/{model_name}/ocr (POST)

Takes an image and returns extracted text details without using an object detection model

predict image

P.S: Custom endpoints like /load, /detect, /get_labels and /one_shot_ocr should be used in a chronological order. First you have to call /load, and then call /detect, /get_labels or /one_shot_ocr

Model structure

The folder "models" contains subfolders of all the models to be loaded. Inside each subfolder there should be a:

  • pb file (frozen_inference_graph.pb): contains the model weights

  • pbtxt file (object-detection.pbtxt): contains model classes

  • Config.json (This is a json file containing information about the model)

          "inference_engine_name": "tensorflow_detection",
          "confidence": 60,
          "predictions": 15,
          "number_of_classes": 2,
          "framework": "tensorflow",
          "type": "detection",
          "network": "inception"


    • You can change confidence and predictions values while running the API
    • The API will return bounding boxes with a confidence higher than the "confidence" value. A high "confidence" can show you only accurate predictions
      • The "predictions" value specifies the maximum number of bounding boxes in the API response


Windows Ubuntu
Network\Hardware Intel Xeon CPU 2.3 GHz Intel Xeon CPU 2.3 GHz Intel Xeon CPU 3.60 GHz GeForce GTX 1080
ssd_fpn 0.867 seconds/image 1.016 seconds/image 0.434 seconds/image 0.0658 seconds/image
frcnn_resnet_50 4.029 seconds/image 4.219 seconds/image 1.994 seconds/image 0.148 seconds/image
ssd_mobilenet 0.055 seconds/image 0.106 seconds/image 0.051 seconds/image 0.052 seconds/image
frcnn_resnet_101 4.469 seconds/image 4.985 seconds/image 2.254 seconds/image 0.364 seconds/image
ssd_resnet_50 1.34 seconds/image 1.462 seconds/image 0.668 seconds/image 0.091 seconds/image
ssd_inception 0.094 seconds/image 0.15 seconds/image 0.074 seconds/image 0.0513 seconds/image


Joe Sleiman, , Beirut, Lebanon

Antoine Charbel,, Beirut, Lebanon

Anis Ismail, Beirut, Lebanon

Popular Tensorflow Projects
Popular Object Detection Projects
Popular Machine Learning Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Deep Learning
Rest Api
Neural Network
Computer Vision
Object Detection
Deep Neural Networks
Docker Swarm
Tensorflow Models