Label Embeddings In Image Classification

Convolutional Neural Networks (CNNs) are being widely used for various tasks in Computer Vision. We focus on the task of image classification particularly using CNNs with more focus on the relation or similarity between class labels. The similarity between labels is judged using label word embeddings and incorporated into the loss layer. We propose that shallower networks be learnt with more complex and structured losses, in order to gain from shorter training time and equivalent complexity. We train two variants of CNNs with multiple architectures , all limited to a maximum of ten convolution layers to obtain an accuracy of 93.27% on the Fashion-MNIST dataset and 86.40% on the CIFAR 10 dataset. We further probe the adversarial robustness of the model as well the classspecific behavior by visualizing the class confusion matrix.We also show some preliminary results towards extending a trained variant to zero-shot learning.