Objectron

Objectron is a dataset of short, object-centric video clips. In addition, the videos also contain AR session metadata including camera poses, sparse point-clouds and planes. In each video, the camera moves around and above the object and captures it from different views. Each object is annotated with a 3D bounding box. The 3D bounding box describes the object’s position, orientation, and dimensions. The dataset contains about 15K annotated video clips and 4M annotated images in the following categories: bikes, books, bottles, cameras, cereal boxes, chairs, cups, laptops, and shoes
Alternatives To Objectron
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Objectron1,958
a year ago21otherJupyter Notebook
Objectron is a dataset of short, object-centric video clips. In addition, the videos also contain AR session metadata including camera poses, sparse point-clouds and planes. In each video, the camera moves around and above the object and captures it from different views. Each object is annotated with a 3D bounding box. The 3D bounding box describes the object’s position, orientation, and dimensions. The dataset contains about 15K annotated video clips and 4M annotated images in the following categories: bikes, books, bottles, cameras, cereal boxes, chairs, cups, laptops, and shoes
Jeelizweboji1,008
a month ago2April 30, 2021apache-2.0JavaScript
JavaScript/WebGL real-time face tracking and expression detection library. Build your own emoticons animated in real time in the browser! SVG and THREE.js integration demos are provided.
Awesome Virtual Try On995
16 days ago4
A curated list of awesome research papers, projects, code, dataset, workshops etc. related to virtual try-on.
Opensse876
2 years agootherC++
Open Sketch Search Engine- 3D object retrieval based on sketch image as input
Video Classification 3d Cnn Pytorch749
5 years ago33mitPython
Video classification tools using 3D ResNet
Video To Pose3d601
4 days ago18mitPython
Convert video to 3D pose in one-key.
Efficient 3dcnns549
a year ago13mitPython
PyTorch Implementation of "Resource Efficient 3D Convolutional Neural Networks", codes and pretrained models.
Objectposeestimationsummary518
7 months agomitPython
Resources (papers, datasets, rendering methods) in the domain of object pose estimation.
Video Classification498
3 years ago26Jupyter Notebook
Tutorial for video classification/ action recognition using 3D CNN/ CNN+RNN on UCF101
Universalviewer43329 days ago5February 28, 2018247otherTypeScript
A community-developed open source project on a mission to help you share your 📚📜📰📽️📻🗿 with the 🌎
Alternatives To Objectron
Select To Compare


Alternative Project Comparisons
Readme

Objectron Dataset

Objectron is a dataset of short object centric video clips with pose annotations.


WebsiteDataset FormatTutorialsLicense

The Objectron dataset is a collection of short, object-centric video clips, which are accompanied by AR session metadata that includes camera poses, sparse point-clouds and characterization of the planar surfaces in the surrounding environment. In each video, the camera moves around the object, capturing it from different angles. The data also contain manually annotated 3D bounding boxes for each object, which describe the object’s position, orientation, and dimensions. The dataset consists of 15K annotated video clips supplemented with over 4M annotated images in the following categories: bikes, books, bottles, cameras, cereal boxes, chairs, cups, laptops, and shoes. In addition, to ensure geo-diversity, our dataset is collected from 10 countries across five continents. Along with the dataset, we are also sharing a 3D object detection solution for four categories of objects — shoes, chairs, mugs, and cameras. These models are trained using this dataset, and are released in MediaPipe, Google's open source framework for cross-platform customizable ML solutions for live and streaming media.

Key Features

  • 15000 annotated videos and 4M annotated images
  • All samples include high-res images, object pose, camera pose, point-cloud, and surface planes.
  • Ready to use examples in various tf.record formats, which can be used in Tensorflow/PyTorch.
  • Object-centric multi-views, observing the same object from different angles.
  • Accurate evaluation metrics, like 3D IoU for oriented 3D bounding boxes.

Dataset Format

The data is stored in the objectron bucket on Google Cloud storage. Check out the Download Data notebook for a quick review of how to download/access the dataset. The following assets are available:

  • The video sequences (located in /videos/class/batch-i/j/video.MOV files)
  • The annotation labels containing the 3D bounding boxes for objects. The annotation protobufs are located in /videos/class/batch-i/j/geometry.pbdata files. They are formatted using the object.proto. See [example] on how to parse the annotation files.
  • AR metadata (such as camera poses, point clouds, and planar surfaces). They are based on a_r_capture_metadata.proto. See example on how to parse these files.
  • Processed dataset: sharded and shuffled tf.records of the annotated frames, in tf.example format and videos in tf.SequenceExample format. These are used for creating the input data pipeline to your models. These files are located in /v1/records_shuffled/class/ and /v1/sequences/class/.
  • Supporting scripts to run evaluation based on the 3D IoU metric.
  • Supporting scripts to load the data into Tensorflow, Jax and Pytorch and visualize the dataset, including “Hello World” examples.
  • Supporting Apache Beam jobs to process the datasets on Google Cloud infrastructure.
  • The index of all available samples, as well as train/test splits for easy access and download.

Raw dataset size is 1.9TB (including videos and their annotations). Total dataset size is 4.4TB (including videos, records, sequences, etc.). This repository provides the required schemas and tools to parse the dataset.

class bike book bottle camera cereal_box chair cup laptop shoe
#videos 476 2024 1928 815 1609 1943 2204 1473 2116
#frames 150k 576k 476k 233k 396k 488k 546k 485k 557k

Tutorials

License

Objectron is released under Computational Use of Data Agreement 1.0 (C-UDA-1.0). A copy of the license is available in this repository.

BibTeX

If you found this dataset useful, please cite our paper.

@article{objectron2021,
  title={Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations},
  author={Adel Ahmadyan, Liangkai Zhang, Artsiom Ablavatski, Jianing Wei, Matthias Grundmann},
  journal={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year={2021}
}

This is not an officially supported Google product. If you have any question, you can email us at [email protected] or join our mailing list at [email protected]

Popular Video Projects
Popular 3d Graphics Projects
Popular Media Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Jupyter Notebook
Machine Learning
Deep Learning
Dataset
Pytorch
Tensorflow
Ai
3d
Camera
Neural Network
Computer Vision
Augmented Reality
3d Reconstruction