A Kitti Road Segmentation model implemented in tensorflow.
Alternatives To Kittiseg
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Mit Deep Learning9,328
7 months ago15mitJupyter Notebook
Tutorials, assignments, and competitions for MIT Deep Learning related courses.
Background Matting4,622
6 months ago43Python
Background Matting: The World is Your Green Screen
Segmentation_models4,09512127 months ago8January 10, 2020237mitPython
Segmentation models with pretrained backbones. Keras and TensorFlow Keras.
8 months ago174otherPython
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
2 years ago143apache-2.0Python
Mask RCNN in TensorFlow
Imgclsmob2,3994a year ago67September 21, 20216mitPython
Sandbox for training deep learning networks
Semantic Segmentation Suite2,311
2 years ago83Python
Semantic Segmentation Suite in TensorFlow. Implement, train, and test new Semantic Segmentation models easily!
Lanenet Lane Detection2,023
17 days ago1apache-2.0Python
Unofficial implemention of lanenet model for real time lane detection
3 years ago85gpl-3.0Python
Generic U-Net Tensorflow implementation for image segmentation
3d Pointcloud1,523
14 days ago2Python
Papers and Datasets about Point Cloud.
Alternatives To Kittiseg
Select To Compare

Alternative Project Comparisons


KittiSeg performs segmentation of roads by utilizing an FCN based model. The model achieved first place on the Kitti Road Detection Benchmark at submission time. Check out our paper for a detailed model description.

The model is designed to perform well on small datasets. The training is done using just 250 densely labelled images. Despite this a state-of-the art MaxF1 score of over 96% is achieved. The model is usable for real-time application. Inference can be performed at the impressive speed of 95ms per image.

The repository contains code for training, evaluating and visualizing semantic segmentation in TensorFlow. It is build to be compatible with the TensorVision back end which allows to organize experiments in a very clean way. Also check out KittiBox a similar projects to perform state-of-the art detection. And finally the MultiNet repository contains code to jointly train segmentation, classification and detection. KittiSeg and KittiBox are utilized as submodules in MultiNet.


The code requires Tensorflow 1.0, python 2.7 as well as the following python libraries:

  • matplotlib
  • numpy
  • Pillow
  • scipy
  • commentjson

Those modules can be installed using: pip install numpy scipy pillow matplotlib commentjson or pip install -r requirements.txt.


  1. Clone this repository: git clone
  2. Initialize all submodules: git submodule update --init --recursive
  3. [Optional] Download Kitti Road Data:
    1. Retrieve kitti data url here:
    2. Call python --kitti_url URL_YOU_RETRIEVED

Running the model using does not require you to download kitti data (step 3). Step 3 is only required if you want to train your own model using or bench a model agains the official evaluation score Also note, that I recommend using instead of downloading the data yourself. The script will also extract and prepare the data. See Section Manage data storage if you like to control where the data is stored.

To update an existing installation do:
  1. Pull all patches: git pull
  2. Update all submodules: git submodule update --init --recursive

If you forget the second step you might end up with an inconstant repository state. You will already have the new code for KittiSeg but run it old submodule versions code. This can work, but I do not run any tests to verify this.


Getting started

Run: python --input_image data/demo/demo.png to obtain a prediction using demo.png as input.

Run: python to evaluate a trained model.

Run: python --hypes hypes/KittiSeg.json to train a model using Kitti Data.

If you like to understand the code, I would recommend looking at first. I have documented each step as thoroughly as possible in this file.

Manage Data Storage

KittiSeg allows to separate data storage from code. This is very useful in many server environments. By default, the data is stored in the folder KittiSeg/DATA and the output of runs in KittiSeg/RUNS. This behaviour can be changed by setting the bash environment variables: $TV_DIR_DATA and $TV_DIR_RUNS.

Include export TV_DIR_DATA="/MY/LARGE/HDD/DATA" in your .profile and the all data will be downloaded to /MY/LARGE/HDD/DATA/data_road. Include export TV_DIR_RUNS="/MY/LARGE/HDD/RUNS" in your .profile and all runs will be saved to /MY/LARGE/HDD/RUNS/KittiSeg

RUNDIR and Experiment Organization

KittiSeg helps you to organize large number of experiments. To do so the output of each run is stored in its own rundir. Each rundir contains:

  • output.log a copy of the training output which was printed to your screen
  • tensorflow events tensorboard can be run in rundir
  • tensorflow checkpoints the trained model can be loaded from rundir
  • [dir] images a folder containing example output images. image_iter controls how often the whole validation set is dumped
  • [dir] model_files A copy of all source code need to build the model. This can be very useful of you have many versions of the model.

To keep track of all the experiments, you can give each rundir a unique name with the --name flag. The --project flag will store the run in a separate subfolder allowing to run different series of experiments. As an example, python --project batch_size_bench --name size_5 will use the following dir as rundir: $TV_DIR_RUNS/KittiSeg/batch_size_bench/size_5_KittiSeg_2017_02_08_13.12.

The flag --nosave is very useful to not spam your rundir.

Modifying Model & Train on your own data

The model is controlled by the file hypes/KittiSeg.json. Modifying this file should be enough to train the model on your own data and adjust the architecture according to your needs. A description of the expected input format can be found here.

For advanced modifications, the code is controlled by 5 different modules, which are specified in hypes/KittiSeg.json.

"model": {
   "input_file": "../inputs/",
   "architecture_file" : "../encoder/",
   "objective_file" : "../decoder/",
   "optimizer_file" : "../optimizer/",
   "evaluator_file" : "../evals/"

Those modules operate independently. This allows easy experiments with different datasets (input_file), encoder networks (architecture_file), etc. Also see TensorVision for a specification of each of those files.

Utilize TensorVision backend

KittiSeg is build on top of the TensorVision TensorVision backend. TensorVision modularizes computer vision training and helps organizing experiments.

To utilize the entire TensorVision functionality install it using

$ cd KittiSeg/submodules/TensorVision
$ python install

Now you can use the TensorVision command line tools, which includes:

tv-train --hypes hypes/KittiSeg.json trains a json model.
tv-continue --logdir PATH/TO/RUNDIR trains the model in RUNDIR, starting from the last saved checkpoint. Can be used for fine tuning by increasing max_steps in model_files/hypes.json .
tv-analyze --logdir PATH/TO/RUNDIR evaluates the model in RUNDIR

Useful Flags & Variabels

Here are some Flags which will be useful when working with KittiSeg and TensorVision. All flags are available across all scripts.

--hypes : specify which hype-file to use
--logdir : specify which logdir to use
--gpus : specify on which GPUs to run the code
--name : assign a name to the run
--project : assign a project to the run
--nosave : debug run, logdir will be set to debug

In addition the following TensorVision environment Variables will be useful:

$TV_DIR_DATA: specify meta directory for data
$TV_DIR_RUNS: specify meta directory for output
$TV_USE_GPUS: specify default GPU behaviour.

On a cluster it is useful to set $TV_USE_GPUS=force. This will make the flag --gpus mandatory and ensure, that run will be executed on the right GPU.


Please have a look into the FAQ. Also feel free to open an issue to discuss any questions not covered so far.


If you benefit from this code, please cite our paper:

  title={MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving},
  author={Teichmann, Marvin and Weber, Michael and Zoellner, Marius and Cipolla, Roberto and Urtasun, Raquel},
  journal={arXiv preprint arXiv:1612.07695},
Popular Tensorflow Projects
Popular Segmentation Projects
Popular Machine Learning Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Computer Vision
Autonomous Driving