Awesome Open Source
Awesome Open Source

MNIST-baselines

This repository contains various baseline models for comparison on the polluted MNIST dataset

Requirements

Usage

Create a model

  1. Prepare Model_Name.json in config/
  2. Prepare Model_Name.py in models/
  3. Prepare Trainer_Name.py in trainers/ (optional)

Train a model

python3 main.py --method Model_Name

Test a model

python3 main.py --method Model_Name --test

Trained models

Available at https://drive.google.com/open?id=1jXe2kJJLkhGH8T30juGbyUgsaDQZXISu

Polluted MNIST

Feature Extraction

A visualization of the first 15 principle components:

Model Accuracy / % Model Accuracy / %
Plain DNN 90.72 DNN + Dropout 90.74
DNN + Batch Normalization 92.39 DNN + PCA (0.99 variance) 21.40
DNN + PCA (0.95 variance) 21.34 DNN + PCA (0.90 variance) 20.55
DNN + ICA 11.93 DNN + NMF 11.30
DNN + VAE 95.24

Conventional Classification Approaches

Model Accuracy / % Model Accuracy / %
Empty Model 11.04 Naive Bayes 19.14
Logistic Regression 24.37 Decision Tree (gini) 51.24
Decision Tree (entropy) 50.91 KNN (3 neighbours) 78.82
KNN (5 neighbours) 78.46 KNN (10 neighbours) 76.40
SGD 22.30 SVM (sigmoid kernel) 10.84
SVM (rbf kernel) 85.94 SVM (polynomial kernel) 87.11

The training curves of the above DNN models:

Deep Convolutional Neural Networks

Model Accuracy / % Model Accuracy / %
LeNet-5 98.35 MobileNetV2 99.63
VGG-19 99.64 ShuffleNetG3 99.65
ResNet-101 99.71 GoogLeNet 99.82
PreAct ResNet-152 99.68 PNASNet 99.75
DenseNet-161 99.75 ResNeXt-29-8x64d 99.71
DPN-92 99.76 SENet-18 99.69
MobileNet 99.44 CapsNet 98.84

Error Analysis

Team Members


Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Python (1,147,256
Pytorch (11,655
Resnet (561
Mobilenet (275
Mnist Classification (244
Densenet (199
Mobilenetv2 (198
Baseline (151
Vgg (146
Googlenet (64
Capsnet (57
Resnext (53
Lenet (48
Senet (43
Shufflenet (25
Dual Path Networks (7
Preact Resnet (3
Pnasnet (2
Related Projects