Semantic-Segmentation

A list of all papers and resoureces on Semantic Segmentation.

Dataset importance

SemanticSegmentation_DL

Some implementation of semantic segmantation for DL model

Dataset

• voc2012
• CitySpaces
• Mapillary
• ADE20K
• PASCAL Context
• COCO-Stuff 10K dataset v1.1
• 2D-3D-S dataset
• Mapillary Vistas
• Stanford Background Dataset
• Sift Flow Dataset
• Barcelona Dataset
• Microsoft COCO dataset
• MSRC Dataset
• LITS Liver Tumor Segmentation Dataset
• KITTI
• Pascal Context
• Data from Games dataset
• Human parsing dataset
• Mapillary Vistas Dataset
• Microsoft AirSim
• MIT Scene Parsing Benchmark
• COCO 2017 Stuff Segmentation Challenge
• ADE20K Dataset
• INRIA Annotations for Graz-02
• Daimler dataset
• ISBI Challenge: Segmentation of neuronal structures in EM stacks
• INRIA Annotations for Graz-02 (IG02)
• Pratheepan Dataset
• Clothing Co-Parsing (CCP) Dataset

Resources

Survey papers

• A 2017 Guide to Semantic Segmentation with Deep Learning by Qure AI [Blog about different sem. segm. methods]
• A Review on Deep Learning Techniques Applied to Semantic Segmentation [Survey paper with a special focus on datasets and the highest performing methods]
• Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art [Survey paper about all aspects of autonomous vehicles, including sem. segm.] [Webpage with a summary of all relevant publications]
• A Survey on Deep Learning in Medical Image Analysis [[Paper]](https://arxiv.org/pdf/1702.05747）

Online demos

• CRF as RNN
• SegNet

2D Semantic Segmentation

Papers:

• [2019-CVPR oral] CLAN: Category-level Adversaries for Semantics Consistent [paper] [code]

• [2019-CVPR] BRS: Interactive Image Segmentation via Backpropagating Refinement Scheme(***) [paper] [code]

• [2019-CVPR] DFANet：Deep Feature Aggregation for Real-Time Semantic Segmentation(used in camera) [paper] [code]

• [2019-CVPR] DeepCO3: Deep Instance Co-segmentation by Co-peak Search and Co-saliency [paper] [code]

• [2019-CVPR] Domain Adaptation(reducing the domain shif) [paper]

• [2019-CVPR] ELKPPNet: An Edge-aware Neural Network with Large Kernel Pyramid Pooling for Learning Discriminative Features in Semantic- Segmentation [paper] [code]

• [2019-CVPR oral] GLNet: Collaborative Global-Local Networks for Memory-Efficient Segmentation of Ultra-High Resolution Images[paper] [code]

• [2019-CVPR] Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth(***SOTA) [paper] [code]

• [2019-ECCV] ICNet: Real-Time Semantic Segmentation on High-Resolution Images [paper] [code]

• [2019-CVPR] LEDNet: A Lightweight Encoder-Decoder Network for Real-Time Semantic Segmentation(***SOTA) [paper] [code]

• [2019-arXiv] LightNet++: Boosted Light-weighted Networks for Real-time Semantic Segmentation [paper] [code]

• [2019-CVPR] PTSNet: A Cascaded Network for Video Object Segmentation [paper] [code]

• [2019-CVPR] PPGNet: Learning Point-Pair Graph for Line Segment Detection [paper] [code]

• [2019-CVPR] Show, Match and Segment: Joint Learning of Semantic Matching and Object Co-segmentation [paper] [code]

• [2019-CVPR] Video Instance Segmentation [paper] [code]

• Arxiv-2018 ExFuse: Enhancing Feature Fusion for Semantic Segmentation 87.9% mean Iou->voc2012 [Paper]
• CVPR-2018 spotlight Learning to Adapt Structured Output Space for Semantic Segmentation [Paper] [Code]
• Arfix-2018 Adversarial Learning for Semi-supervised Semantic Segmentation [Paper] [Code]
• Arxiv-2018 Context Encoding for Semantic Segmentation [Paper] [Code]
• CVPR-2018 Learning to Adapt Structured Output Space for Semantic Segmentation [Paper][Code]
• CVPR-2018 Dynamic-structured Semantic Propagation Network [Paper]
• Deeplab v4: Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation [Paper] [Code]
• Deep Value Networks Learn to Evaluate and Iteratively Refine Structured Outputs [Paper][Code]
• ICCV-2017 Semantic Line Detection and Its Applications [Paper]
• ICCV-2017 Attentive Semantic Video Generation Using Captions [Paper]
• ICCV-2017 BlitzNet: A Real-Time Deep Network for Scene Understanding [Paper] [Code]
• ICCV-2017 SCNet: Learning Semantic Correspondence [Code]
• CVPR-2017 End-to-End Instance Segmentation with Recurrent Attention [Code]
• CVPR-2017 Deep Watershed Transform for Instance Segmentation [Code]
• Piecewise Flat Embedding for Image Segmentation [Paper]
• ICCV-2017 Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes [Paper][Code]
• CVPR-2017 Not All Pixels Are Equal: Difficulty-Aware Semantic Segmentation via Deep Layer Cascade-2017 [Paper]
• CVPR-2017 Annotating Object Instances with a Polygon-RNN-2017 [Project] [Paper]
• CVPR-2017 Loss maxpooling for semantic image segmentation [Paper]
• ICCV-2017 Scale-adaptive convolutions for scene parsing [Paper]
• Towards End-to-End Lane Detection: an Instance Segmentation Approach [Paper]arxiv-1802
• AAAI-2018 Mix-and-Match Tuning for Self-Supervised Semantic Segmentation [Paper] arxiv-1712
• NIPS-2017-Learning Affinity via Spatial Propagation Networks [Paper]
• AAAI-2018-Spatial As Deep: Spatial CNN for Traffic Scene Understanding [Paper]
• Stacked Deconvolutional Network for Semantic Segmentation-2017 [Paper]
• Deeplab v3: Rethinking Atrous Convolution for Semantic Image Segmentation-2017(DeeplabV3) [Paper]
• CVPR-2017 Learning Object Interactions and Descriptions for Semantic Image Segmentation-2017 [Paper]
• Pixel Deconvolutional Networks-2017 [Code-Tensorflow] [Paper]
• Dilated Residual Networks-2017 [Paper]
• A Review on Deep Learning Techniques Applied to Semantic Segmentation-2017 [Paper]
• BiSeg: Simultaneous Instance Segmentation and Semantic Segmentation with Fully Convolutional Networks [Paper]
• ICNet for Real-Time Semantic Segmentation on High-Resolution Images-2017 [Project] [Code] [Paper] [Video]

• Feature Forwarding: Exploiting Encoder Representations for Efficient Semantic Segmentation-2017 [Project] [Code-Torch7]
• Reformulating Level Sets as Deep Recurrent Neural Network Approach to Semantic Segmentation-2017 [Paper]
• Adversarial Examples for Semantic Image Segmentation-2017 [Paper]
• Large Kernel Matters - Improve Semantic Segmentation by Global Convolutional Network-2017 [Paper]
• HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection [Paper]
• Hypercolumns for Object Segmentation and Fine-grained Localization [Paper]
• Matching-CNN meets KNN: Quasi-parametric human parsing[Paper]
• Deep Human Parsing with Active Template Regression [Paper]
• TPAMI-2012 Learning Hierarchical Features for Scene Labeling The first paper for applying dl on semantic segmentation !!! [Paper]
• Label Refinement Network for Coarse-to-Fine Semantic Segmentation-2017 [Paper]
• Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation [Paper]
• ParseNet: Looking Wider to See Better [Paper]
• CVPR-2016 Recombinator Networks: Learning Coarse-to-Fine Feature Aggregation [Paper]
• PixelNet: Representation of the pixels, by the pixels, and for the pixels-2017 [Project] [Code-Caffe] [Paper]
• LabelBank: Revisiting Global Perspectives for Semantic Segmentation-2017 [Paper]
• Progressively Diffused Networks for Semantic Image Segmentation-2017 [Paper]
• Understanding Convolution for Semantic Segmentation-2017 [Model-Mxnet] [Paper] [Code]
• ICCV-2017 Predicting Deeper into the Future of Semantic Segmentation-2017 [Paper]
• CVPR-2017 Pyramid Scene Parsing Network-2017 [Project] [Code-Caffe] [Paper] [Slides]
• FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation-2016 [Paper]
• FusionNet: A deep fully residual convolutional neural network for image segmentation in connectomics-2016 [Code-PyTorch] [Paper]
• RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation-2016 [Code-MatConvNet] [Paper]
• CVPRW-2017 The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation [Code-Theano] [Code-Keras1] [Code-Keras2] [Paper]
• CVPR-2017 Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes [Code-Theano] [Paper]
• PixelNet: Towards a General Pixel-level Architecture-2016 [Paper]
• Recalling Holistic Information for Semantic Segmentation-2016 [Paper]
• Semantic Segmentation using Adversarial Networks-2016 [Paper] [Code-Chainer]
• Region-based semantic segmentation with end-to-end training-2016 [Paper]
• Exploring Context with Deep Structured models for Semantic Segmentation-2016 [Paper]</
• Multi-scale context aggregation by dilated convolutions [Paper]
• Better Image Segmentation by Exploiting Dense Semantic Predictions-2016 [Paper]
• Boundary-aware Instance Segmentation-2016 [Paper]
• Improving Fully Convolution Network for Semantic Segmentation-2016 [Paper]
• Deep Structured Features for Semantic Segmentation-2016 [Paper]
• DeepLab v2:Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs-2016** [Project] [Code-Caffe] [Code-Tensorflow] [Code-PyTorch] [Paper]
• DeepLab v1: Semantic Image Segmentation With Deep Convolutional Nets and Fully Connected CRFs-2014** [Code-Caffe1] [Code-Caffe2] [Paper]
• Deep Learning Markov Random Field for Semantic Segmentation-2016 [Project] [Paper]
• ECCV2016 Salient Deconvolutional Networks [Code]
• Convolutional Random Walk Networks for Semantic Image Segmentation-2016 [Paper]
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation-2016 [Code-Caffe1][Code-Caffe2] [Paper] [Blog]
• High-performance Semantic Segmentation Using Very Deep Fully Convolutional Networks-2016 [Paper]
• CVPR-2016-oral ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation-2016 [Paper]
• Object Boundary Guided Semantic Segmentation-2016 [Code-Caffe] [Paper]
• Segmentation from Natural Language Expressions-2016 [Project] [Code-Tensorflow] [Code-Caffe] [Paper]
• Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation-2016 [Code-Caffe] [Paper]
• Global Deconvolutional Networks for Semantic Segmentation-2016 [Paper]
• Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network-2015 [Project] [Code-Caffe] [Paper]
• Learning Dense Convolutional Embeddings for Semantic Segmentation-2015 [Paper]
• ParseNet: Looking Wider to See Better-2015 [Code-Caffe] [Model-Caffe] [Paper]
• Decoupled Deep Neural Network for Semi-supervised Semantic Segmentation-2015 [Project] [Code-Caffe] [Paper]
• Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding [Paper]
• SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation-2015 [Project] [Code-Caffe] [Paper] [Tutorial1] [Tutorial2]
• Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform-2015 [Paper]
• Semantic Segmentation with Boundary Neural Fields-2015 [Code] [Paper]
• Semantic Image Segmentation via Deep Parsing Network-2015 [Project] [Paper1] [Paper2] [Slides]
• What’s the Point: Semantic Segmentation with Point Supervision-2015 [Project] [Code-Caffe] [Model-Caffe] [Paper]
• U-Net: Convolutional Networks for Biomedical Image Segmentation-2015 [Project] [Code+Data] [Code-Keras] [Code-Tensorflow] [Paper] [Notes]
• Learning Deconvolution Network for Semantic Segmentation(DeconvNet)-2015 [Project] [Code-Caffe] [Paper] [Slides]
• Multi-scale Context Aggregation by Dilated Convolutions-2015 [Project] [Code-Caffe] [Code-Keras] [Paper] [Notes]
• ReSeg: A Recurrent Neural Network-based Model for Semantic Segmentation-2015 [Code-Theano] [Paper]
• ICCV-2015 BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation-2015 [Paper]
• Feedforward semantic segmentation with zoom-out features-2015 [Code] [Paper] [Video]
• Conditional Random Fields as Recurrent Neural Networks-2015 [Project] [Code-Caffe1] [Code-Caffe2] [Demo] [Paper1] [Paper2]
• Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation-2015 [Paper]
• Fully Convolutional Networks for Semantic Segmentation-2015 [Code-Caffe] [Model-Caffe] [Code-Tensorflow1] [Code-Tensorflow2] [Code-Chainer] [Code-PyTorch] [Paper1] [Paper2] [Slides1] [Slides2]
• Deep Joint Task Learning for Generic Object Extraction-2014 [Project] [Code-Caffe] [Dataset] [Paper]
• Highly Efficient Forward and Backward Propagation of Convolutional Neural Networks for Pixelwise Classification-2014 [Code-Caffe] [Paper]
• Wider or deeper: Revisiting the resnet model for visual recognition [Paper]
• Describing the Scene as a Whole: Joint Object Detection, Scene Classification and Semantic Segmentation[Paper]
• Analyzing Semantic Segmentation Using Hybrid Human-Machine CRFs[Paper]
• Convolutional Patch Networks with Spatial Prior for Road Detection and Urban Scene Understanding[Paper]
• Deep Deconvolutional Networks for Scene Parsing[Paper]
• FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos[Paper][Poject]
• ICCV-2017 Deep Dual Learning for Semantic Image Segmentation [Paper]
• From image-level to pixel level labeling with convolutional networks [Paper]
• Scene Segmentation with DAG-Recurrent Neural Networks [Paper]
• Learning to Segment Every Thing [Paper]
• Panoptic Segmentation [Paper]
• The Devil is in the Decoder [Paper]
• Attention to Scale: Scale-aware Semantic Image Segmentation [Paper][Project]
• Convolutional Oriented Boundaries: From Image Segmentation to High-Level Tasks [Paper] [Project]
• Scale-Aware Alignment of Hierarchical Image Segmentation [Paper] [Project]
• ICCV-2017 Semi Supervised Semantic Segmentation Using Generative Adversarial Network[Paper]
• Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach [Paper]

• CVPR-2016 Convolutional Feature Masking for Joint Object and Stuff Segmentation [Paper]
• ECCV-2016 Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation [Paper]

• FastMask: Segment Object Multi-scale Candidates in One Shot-2016 [Code-Caffe] [Paper]
• Pixel Objectness-2017 [Project] [Code-Caffe] [Paper]

3D Semantic Segmentation

Papers

• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation [Paper]
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space (2017) [Paper]
• Learning 3D Mesh Segmentation and Labeling (2010) [Paper]
• Unsupervised Co-Segmentation of a Set of Shapes via Descriptor-Space Spectral Clustering (2011) [Paper]
• Single-View Reconstruction via Joint Analysis of Image and Shape Collections (2015) [Paper]
• 3D Shape Segmentation with Projective Convolutional Networks (2017) [Paper]
• Learning Hierarchical Shape Segmentation and Labeling from Online Repositories (2017) [Paper]
• 3D Graph Neural Networks for RGBD Semantic Segmentation (2017) [Paper]
• 3DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-scale 3D Point Clouds (2017)[Paper]
• Multi-view deep learning for consistent semantic mapping with rgb-d cameras [Paper]

• ICCV-2017 Large-scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55 [Paper][Project]

Instance Segmentation

• Mask Scoring R-CNN (MS R-CNN) [Code][Paper]
• Predicting Future Instance Segmentations by Forecasting Convolutional Features [Paper]
• CVPR-2018 Path Aggregation Network for Instance Segmentation [Paper] better than Mask-rcnn!！COCO-2017 1st!
• Pixelwise Instance Segmentation with a Dynamically Instantiated Network-2017 [Paper]
• Semantic Instance Segmentation via Deep Metric Learning-2017 [Paper]
• CVPR-2017 FastMask: Segment Multi-scale Object Candidates in One Shot [Code-Tensorflow] [Paper]
• Pose2Instance: Harnessing Keypoints for Person Instance Segmentation-2017 [Paper]
• Pixelwise Instance Segmentation with a Dynamically Instantiated Network-2017 [Paper]
• CVPR-2017-spotlight Fully Convolutional Instance-aware Semantic Segmentation-2016 [Code] [Paper]
• CVPR-2016-oral Instance-aware Semantic Segmentation via Multi-task Network Cascades-2015 [Code] [Paper]
• Recurrent Instance Segmentation-2015 [Project] [Code-Torch7] [Paper] [Poster] [Video]
• Annotating Object Instances with a Polygon-RNN [Paper]
• MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features [Paper]
• FCIS:Fully Convolutional Instance-aware Semantic Segmentation [Paper]Code
• MNC:Instance-aware Semantic Segmentation via Multi-task Network Cascades [Paper]Code
• DeepMask:Learning to Segment Object Candidates [Paper] Code
• SharpMask:Learning to Refine Object Segments [Paper]Code
• RIS:Recurrent Instance Segmentation [Paper]Code
• FastMask: Segment Multi-scale Object Candidates in One Shot [Paper]Code
• Proposal-free network for instance-level object segmentation [Paper]
• ECCV-2016 Instance-sensitive Fully Convolutional Networks [Paper]
• Pixel-level encoding and depth layering for instance-level semantic labeling [Paper]

Robotics

• Virtual-to-Real: Learning to Control in Visual Semantic Segmentation [Paper]
• End-to-End Tracking and Semantic Segmentation Using Recurrent Neural Networks [Paper]
• Semantic Segmentation using Adversarial Networks [Paper]

Adversarial Training

• CVPR-2017-Image-to-Image Translation with Conditional Adversarial Networks [Paper]
• ICCV-2017-Adversarial Examples for Semantic Segmentation and Object Detection [Paper]

Scene Understanding

Papers

1.Spatial As Deep: Spatial CNN for Traffic Scene Understanding [Paper]

Dataset & Resources

• SUNRGB-D 3D Object Detection Challenge [Link] 19 object categories for predicting a 3D bounding box in real world dimension Training set: 10,355 RGB-D scene images, Testing set: 2860 RGB-D images
• SceneNN (2016) [Link] 100+ indoor scene meshes with per-vertex and per-pixel annotation.
• ScanNet (2017) [Link] An RGB-D video dataset containing 2.5 million views in more than 1500 scans, annotated with 3D camera poses, surface reconstructions, and instance-level semantic segmentations.
• Matterport3D: Learning from RGB-D Data in Indoor Environments (2017) [Link]
  10,800 panoramic views (in both RGB and depth) from 194,400 RGB-D images of 90 building-scale scenes of private rooms. Instance-level semantic segmentations are provided for region (living room, kitchen) and object (sofa, TV) categories.
• SUNCG: A Large 3D Model Repository for Indoor Scenes (2017) [Link]
  The dataset contains over 45K different scenes with manually created realistic room and furniture layouts. All of the scenes are semantically annotated at the object level.
• MINOS: Multimodal Indoor Simulator (2017) [Link] MINOS is a simulator designed to support the development of multisensory models for goal-directed navigation in complex indoor environments. MINOS leverages large datasets of complex 3D environments and supports flexible configuration of multimodal sensor suites. MINOS supports SUNCG and Matterport3D scenes.
• Facebook House3D: A Rich and Realistic 3D Environment (2017) [Link]
  House3D is a virtual 3D environment which consists of 45K indoor scenes equipped with a diverse set of scene types, layouts and objects sourced from the SUNCG dataset. All 3D objects are fully annotated with category labels. Agents in the environment have access to observations of multiple modalities, including RGB images, depth, segmentation masks and top-down 2D map views.
• HoME: a Household Multimodal Environment (2017) [Link]
  HoME integrates over 45,000 diverse 3D house layouts based on the SUNCG dataset, a scale which may facilitate learning, generalization, and transfer. HoME is an open-source, OpenAI Gym-compatible platform extensible to tasks in reinforcement learning, language grounding, sound-based navigation, robotics, multi-agent learning.
• AI2-THOR: Photorealistic Interactive Environments for AI Agents [Link]
  AI2-THOR is a photo-realistic interactable framework for AI agents. There are a total 120 scenes in version 1.0 of the THOR environment covering four different room categories: kitchens, living rooms, bedrooms, and bathrooms. Each room has a number of actionable objects.

Weakly-Supervised-Segmentation && Interactive Segmentation && Transferable Semantic Segmentation

• arxiv-2018 WebSeg: Learning Semantic Segmentation from Web Searches [Paper]
• Weakly Supervised Object Localization Using Things and Stuff Transfer [Paper]
• Semi and Weakly Supervised Semantic Segmentation Using Generative Adversarial Network [Paper]
• Weakly- and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation [Paper]

• Weakly Supervised Structured Output Learning for Semantic Segmentation [Paper]
• ICCV-2011 Weakly supervised semantic segmentation with a multi-image model [Paper]
• ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016[Paper]
• Constrained convolutional neural networks for weakly supervised segmentation. Proceedings of the IEEE International Conference on Computer Vision. 2015.[Paper]
• Weakly-and semi-supervised learning of a DCNN for semantic image segmentation. arXiv preprint arXiv:1502.02734 (2015).[Paper]
• Learning to segment under various forms of weak supervision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.[Paper]
• STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation 2017 TPAMI [Paper] [Project]
• [Paper]
• CVPR-2017-Simple Does It: Weakly Supervised Instance and Semantic Segmentation [Paper] [tensorflow]
• CVPR-2017-Weakly Supervised Semantic Segmentation using Web-Crawled Videos [Paper]
• AAAI-2017-Weakly Supervised Semantic Segmentation Using Superpixel Pooling Network [Paper]
• ICCV-2015-Weakly supervised graph based semantic segmentation by learning communities of image-parts [Paper]
• Towards Weakly Supervised Semantic Segmentation by Means of Multiple Instance and Multitask Learning [Paper]
• Weakly-Supervised Semantic Segmentation using Motion Cues [Paper] [Project]
• Weakly Supervised Semantic Segmentation Based on Web Image Co-segmentation [Paper]
• Learning to Rene Object Segments [Paper]

• Weakly-Supervised Dual Clustering for Image Semantic Segmentation [Paper]

• Interactive Video Object Segmentation in the Wild [Paper]

Video Semantic Segmentation

• CVPR-2017 Video Object Segmentation Without Temporal Information One-Shot Video Object Segmentation [Project]
• Feature Space Optimization for Semantic Video Segmentation[Paper][Slides]
• The Basics of Video Object Segmentation [Blog]
• ICCV2017----SegFlow_Joint Learning for Video Object Segmentation and Optical Flow
• OSVOS:One-Shot Video Object Segmentation
• Surveillance Video Parsing with Single Frame Supervision
• The 2017 DAVIS Challenge on Video Object Segmentation
• Video Propagation Networks
• OnAVOS: Online Adaptation of Convolutional Neural Networks for Video Object Segmentation. P. Voigtlaender, B. Leibe, BMVC 2017. [Project Page] [Precomputed results]
• MSK: Learning Video Object Segmentation from Static Images. F. Perazzi*, A. Khoreva*, R. Benenson, B. Schiele, A. Sorkine-Hornung, CVPR 2017. [Project Page] [Precomputed results]
• SFL: SegFlow: Joint Learning for Video Object Segmentation and Optical Flow. J. Cheng, Y.-H. Tsai, S. Wang, M.-H. Yang, ICCV 2017. [Project Page] [Precomputed results]
• CTN: Online Video Object Segmentation via Convolutional Trident Network. W.-D. Jang, C.-S. Kim, CVPR 2017. [Project Page] [Precomputed results]
• VPN: Video Propagation Networks. V. Jampani, R. Gadde, P. V. Gehler, CVPR 2017. [Project Page] [Precomputed results]
• PLM: Pixel-level Matching for Video Object Segmentation using Convolutional Neural Networks. J. Shin Yoon, F. Rameau, J. Kim, S. Lee, S. Shin, I. So Kweon, ICCV 2017. [Project Page] [Precomputed results]
• OFL: Video Segmentation via Object Flow. Y.-H. Tsai, M.-H. Yang, M. Black, CVPR 2016. [Project Page] [Precomputed results]
• BVS: Bilateral Space Video Segmentation. N. Marki, F. Perazzi, O. Wang, A. Sorkine-Hornung, CVPR 2016. [Project Page] [Precomputed results]
• FCP: Fully Connected Object Proposals for Video Segmentation. F. Perazzi, O. Wang, M. Gross, A. Sorkine-Hornung, ICCV 2015. [Project Page] [Precomputed results]
• JMP: JumpCut: Non-Successive Mask Transfer and Interpolation for Video Cutout. Q. Fan, F. Zhong, D. Lischinski, D. Cohen-Or, B. Chen, SIGGRAPH 2015. [Project Page] [Precomputed results]
• HVS: Efficient hierarchical graph-based video segmentation. M. Grundmann, V. Kwatra, M. Han, I. A. Essa, CVPR 2010. [Project Page] [Precomputed results]
• SEA: SeamSeg: Video Object Segmentation Using Patch Seams. S. Avinash Ramakanth, R. Venkatesh Babu, CVPR 2014. [Project Page] [Precomputed results]
• ARP: Primary Object Segmentation in Videos Based on Region Augmentation and Reduction. Y.J. Koh, C.-S. Kim, CVPR 2017. [Project Page] [Precomputed results]
• LVO: Learning Video Object Segmentation with Visual Memory. P. Tokmakov, K. Alahari, C. Schmid, ICCV 2017. [Project Page] [Precomputed results]
• FSEG: FusionSeg: Learning to combine motion and appearance for fully automatic segmentation of generic objects in videos. S. Jain, B. Xiong, K. Grauman, CVPR 2017. [Project Page] [Precomputed results]
• LMP: Learning Motion Patterns in Videos. P. Tokmakov, K. Alahari, C. Schmid, CVPR 2017. [Project Page] [Precomputed results]
• SFL: SegFlow: Joint Learning for Video Object Segmentation and Optical Flow. J. Cheng, Y.-H. Tsai, S. Wang, M.-H. Yang, ICCV 2017. [Project Page] [Precomputed results] FST: Fast Object Segmentation in Unconstrained Video. A. Papazoglou, V. Ferrari, ICCV 2013. [Project Page] [Precomputed results]
• CUT: Motion Trajectory Segmentation via Minimum Cost Multicuts. M. Keuper, B. Andres, T. Brox, ICCV 2015. [Project Page] [Precomputed results]
• NLC: Video Segmentation by Non-Local Consensus voting. A. Faktor, M. Irani, BMVC 2014. [Project Page] [Precomputed results]
• MSG: Object segmentation in video: A hierarchical variational approach for turning point trajectories into dense regions. P. Ochs, T. Brox, ICCV 2011. [Project Page] [Precomputed results]
• KEY: Key-segments for video object segmentation. Y. Lee, J. Kim, K. Grauman, ICCV 2011. [Project Page] [Precomputed results]
• CVOS: Causal Video Object Segmentation from Persistence of Occlusions. B. Taylor, V. Karasev, S. Soatto, CVPR 2015. [Project Page] [Precomputed results]
• TRC: Video segmentation by tracing discontinuities in a trajectory embedding. K. Fragkiadaki, G. Zhang, J. Shi, CVPR 2012. [Project Page] [Precomputed results]
• Instance Embedding Transfer to Unsupervised Video Object Segmentation [Paper]

• Result of DAVIS-Challenge 2017
• Benchmark 2016----A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation 2016----Clockwork Convnets for Video Semantic Segmentation 2016----MaskTrack ----Learning Video Object Segmentation from Static Images 2017----DAVIS-Challenge-1st----Video Object Segmentation with Re-identification 2017----DAVIS-Challenge-2nd----Lucid Data Dreaming for Multiple Object Tracking 2017----DAVIS-Challenge-3rd----Instance Re-Identification Flow for Video Object Segmentation 2017----DAVIS-Challenge-4th----Multiple-Instance Video Segmentation with Sequence-Specific Object Proposals 2017----DAVIS-Challenge-5th Online Adaptation of Convolutional Neural Networks for the 2017 DAVIS Challenge on Video Object Segmentation 2017----DAVIS-Challenge-6th ----Learning to Segment Instances in Videos with Spatial Propagation Network 2017----DAVIS-Challenge-7th----Some Promising Ideas about Multi-instance Video Segmentation 2017----DAVIS-Challenge-8th----One-Shot Video Object Segmentation with Iterative Online Fine-Tuning 2017----DAVIS-Challenge-9th----Video Object Segmentation using Tracked Object Proposals

Multi-Task Learning

Papers:

• Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics [Paper]
• Multi-task Learning using Multi-modal Encoder-Decoder Networks with Shared Skip Connections [Paper]

Road Segmentation && Real Time Segmentation

Papers:

• Deep Semantic Segmentation for Automated Driving: Taxonomy, Roadmap and Challenges [Paper]
• 2018-arxiv Real-time Semantic Segmentation Comparative Study[Paper][Code]
• MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving [Paper]
• self-driving-car-road-segmentation [Link]
• Efficient Deep Models for Monocular Road Segmentation[Paper]
• Semantic Road Segmentation via Multi-scale Ensembles of Learned Features [Paper]
• Distantly Supervised Road Segmentation [Paper]
• Deep Fully Convolutional Networks with Random Data Augmentation for Enhanced Generalization in Road Detection [Paper]
• ICCV-2017 Real-time category-based and general obstacle detection for autonomous driving [Paper]
• ICCV-2017 FoveaNet: Perspective-aware Urban Scene Parsing [Paper]
• CVPR-2017 UberNet: Training a universal convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory [Paper]

• LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation [Paper]
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation-2016 [Code-Caffe1][Code-Caffe2] [Paper] [Blog]
• Efficient Deep Models for Monocular Road Segmentation[Paper]
• Real-Time Coarse-to-fine Topologically Preserving Segmentation[Paper]
• ICNet for Real-Time Semantic Segmentation on High-Resolution Images [Paper]
• Efficient and robust deep networks for semantic segmentation [Paper]
• NIPSW-2017 Speeding up semantic segmentation for autonomous driving [Paper]

• ECCV-2012 Road Scene Segmentation from a Single Image [Paper]

Codes

• MarvinTeichmann/MultiNet
• MarvinTeichmann/KittiSeg
• vxy10/p5_VehicleDetection_Unet [Keras]
• ndrplz/self-driving-car
• mvirgo/MLND-Capstone
• https://github.com/zhujun98/semantic_segmentation/tree/master/fcn8s_road

Medical Image Semantic Segmentation

Papers

• Arxiv-2018 Deep learning and its application to medical image segmentation [Paper]

• Deep neural networks segment neuronal membranes in electron microscopy images
• Semantic Image Segmentation with Deep Learning [Paper]
• Automatic Liver and Tumor Segmentation of CT and MRI Volumes Using Cascaded Fully Convolutional Neural Networks [Paper]
• DeepNAT: Deep Convolutional Neural Network for Segmenting Neuroanatomy [Paper]
• CNN-based Segmentation of Medical Imaging Data [Paper]
• Deep Retinal Image Understanding (http://www.vision.ee.ethz.ch/~cvlsegmentation/driu/data/paper/DRIU_MICCAI2016.pdf)
• Model-based segmentation of vertebral bodies from MR images with 3D CNNs
• Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation
• U-net: Convolutional networks for biomedical image segmentation
• 3D U-Net: Learning dense volumetric segmentation from sparse annotation.
• V-Net: Fully convolutional neural networks for volumetric medical image segmentation.arXiv:1606.04797
• The importance of skip connections in biomedical image segmentation Spatial clockwork recurrent neural network for muscle perimysium segmentation
• NPIS-2015 Parallel multi-dimensional LSTM, with application to fast biomedical volumetric image segmentation
• Multi-dimensional gated recurrent units for the segmentation of biomedical 3D-data
• Combining fully convolutional and recurrent neural networks for 3D biomedical image segmentation
• Recurrent fully convolutional neural networks for multi-slice MRI cardiac segmentation. arXiv:1608.03974
• Automatic detection and classification of colorectal polyps by transferring low-level CNN features from nonmedical domain
• Deep learning for multi-task medical image segmentation in multiple modalities
• Sub-cortical brain structure segmentation using F-CNNs
• Segmentation label propagation using deep convolutional neural networks and dense conditional random field
• Fast fully automatic segmentation of the human placenta from motion corrupted MRI
• Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks
• Non-uniform patch sampling with deep convolutional neural networks for white matter hyperintensity segmentation
• A unified framework for automatic wound segmentation and analysis with deep convolutional neural networks
• Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to Multiple Sclerosis lesion segmentation
• Brain tumor segmentation using convolutional neural networks in MRI images
• Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network
• Automatic Coronary Calcium Scoring in Cardiac CT Angiography Using Convolutional Neural Networks [Paper]
• Improving computer-aided detection using convolutional neural networks and random view aggregation [Paper]
• Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks [Paper]

Codes

Part Semantic Segmentation

• Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing-2017 [Project] [Code-Caffe] [Paper]
• Deep Learning for Human Part Discovery in Images-2016 [Code-Chainer] [Paper]
• A CNN Cascade for Landmark Guided Semantic Part Segmentation-2016 [Project] [Paper]
• Deep Learning for Semantic Part Segmentation With High-level Guidance-2015 [Paper]
• Neural Activation Constellations-Unsupervised Part Model Discovery with Convolutional Networks-2015 [Paper]
• Human Parsing with Contextualized Convolutional Neural Network-2015 [Paper]
• Part detector discovery in deep convolutional neural networks-2014 [Code] [Paper]
• Hypercolumns for object segmentation and fine-grained localization [Paper]

Clothes Parsing

• Looking at Outfit to Parse Clothing-2017 [Paper]
• Semantic Object Parsing with Local-Global Long Short-Term Memory-2015 [Paper] -A High Performance CRF Model for Clothes Parsing-2014 [Project] [Code] [Dataset] [Paper]
• Clothing co-parsing by joint image segmentation and labeling-2013 [Project] [Dataset] [Paper]
• Parsing clothing in fashion photographs-2012 [Project] [Paper]

Popular Methods and Implementations

• U-Net [https://arxiv.org/pdf/1505.04597.pdf]Pytorch
• SegNet [https://arxiv.org/pdf/1511.00561.pdf]Caffe
• DeepLab [https://arxiv.org/pdf/1606.00915.pdf]Caffe
• FCN [https://arxiv.org/pdf/1605.06211.pdf]tensorflow
• ENet [https://arxiv.org/pdf/1606.02147.pdf]Caffe
• LinkNet [https://arxiv.org/pdf/1707.03718.pdf]Torch
• DenseNet [https://arxiv.org/pdf/1608.06993.pdf]
• Tiramisu [https://arxiv.org/pdf/1611.09326.pdf]
• DilatedNet [https://arxiv.org/pdf/1511.07122.pdf]
• PixelNet [https://arxiv.org/pdf/1609.06694.pdf]Caffe
• ICNet [https://arxiv.org/pdf/1704.08545.pdf]Caffe
• ERFNet [http://www.robesafe.uah.es/personal/eduardo.romera/pdfs/Romera17iv.pdf]Torch
• RefineNet [https://arxiv.org/pdf/1611.06612.pdf]tensorflow
• PSPNet [https://arxiv.org/pdf/1612.01105.pdf,https://hszhao.github.io/projects/pspnet/]Caffe
• Dilated convolution [https://arxiv.org/pdf/1511.07122.pdf]Caffe
• DeconvNet [https://arxiv.org/pdf/1505.04366.pdf]Caffe
• FRRN [https://arxiv.org/pdf/1611.08323.pdf]Lasagne
• GCN [https://arxiv.org/pdf/1703.02719.pdf]PyTorch
• LRR [https://arxiv.org/pdf/1605.02264.pdf]Matconvnet
• DUC, HDC [https://arxiv.org/pdf/1702.08502.pdf]PyTorch
• MultiNet [https://arxiv.org/pdf/1612.07695.pdf] tensorflow1tensorflow2
• Segaware [https://arxiv.org/pdf/1708.04607.pdf]Caffe
• Semantic Segmentation using Adversarial Networks [https://arxiv.org/pdf/1611.08408.pdf] [Chainer](+ oyam/Semantic-Segmentation-using-Adversarial-Networks )
• In-Place Activated BatchNorm:obtain #1 positions [https://arxiv.org/abs/1712.02616] Pytorch

Annotation Tools:

• AKSHAYUBHAT/ImageSegmentation
• kyamagu/js-segment-annotator
• CSAILVision/LabelMeAnnotationTool
• seanbell/opensurfaces-segmentation-ui
• lzx1413/labelImgPlus
• wkentaro/labelme

Distinguished Researchers & Teams:

• Liang-Chieh (Jay) Chen Deeplab-Google
• Jianping Shi PSPNet
• Kaiming He Mask-RCNN
• Ming-Ming Cheng
• Joachim M. Buhmann
• Jifeng Dai FCIS-MSRA
• Alex Kendall SegNet

Results:

• MSRC-21
• Cityscapes
• VOC2012

Reference

nightrome/really-awesome-semantic-segmentation

mrgloom/awesome-semantic-segmentation