From Zero to Research Scientist full resources guide.

Guide description

This guide is designated to anybody with basic programming knowledge or a computer science background interested in becoming a Research Scientist with 🎯 on Deep Learning and NLP.

You can go Bottom-Up or Top-Down both works well and it is actually crucial to know which approach suites you the best. If you are okay with studying lots of mathematical concepts without application then use Bottom-Up. If you want to go hands-on first then use the Top-Down first.

Contents:

• Mathematical Foundation
  • Linear Algebra
  • Probability
  • Calculus
  • Optimization Theory
• Machine Learning
• Deep Learning
• Reinforcement Learning
• Natural Language Processing

Mathematical Foundations:

The Mathematical Foundation part is for all Artificial Intelligence branches such as Machine Learning, Reinforcement Learning, Computer Vision and so on. AI is heavily math-theory based so a solid foundation is essential.

Linear Algebra

:infinity:

This branch of Math is crucial for understanding the mechanism of Neural Networks which are the norm for NLP methodologies in nowadays State-of-The-Art.

Resource	Difficulty	Relevance
MIT Gilbert Strang 2005 Linear Algebra
Linear Algebra 4th Edition by Friedberg
Mathematics for Machine Learning Book: Chapter 2
James Hamblin Awesome Lecture Series
3Blue1Brown Essence of Linear Algebra
Mathematics For Machine Learning Specialization: Linear Algebra
Matrix Methods for Linear Algebra for Gilber Strang UPDATED!

Probability

:atom:

Most of Natural Language Processing and Machine Learning Algorithms are based on Probability theory. So this branch is extremely important for grasping how old methods work. Resource | Difficulty | Relevance ------------------------- | --------------- | ------------------------------- Joe Blitzstein Harvard Probability and Statistics Course |

| MIT Probability Course 2011 Lecture videos |

| MIT Probability Course 2018 short videos UPDATED! |

| Mathematics for Machine Learning Book: Chapter 6 |

| Probalistic Graphical Models CMU Advanced |

| Probalistic Graphical Models Stanford Daphne Advanced |

| A First Course In Probability Book by Ross |

| Joe Blitzstein Harvard Professor Probability Awesome Book |

|

Calculus

📐

Resource	Difficulty	Relevance
Essence of Calculus by 3Blue1Brown
Single Variable Calculus MIT 2007
Strang's Overview of Calculus
MultiVariable Calculus MIT 2007
Princeton University Multivariable Calculus 2013
Calculus Book by Stewart
Mathematics for Machine Learning Book: Chapter 5

Optimization Theory

-Resource	Difficulty	Relevance
CMU optimization course 2018
CMU Advanced optimization course
Stanford Famous optimization course
Boyd Convex Optimization Book

Machine Learning

Considered a fancy name for Statistical models where its main goal is to learn from data for several usages. It is considered highly recommended to master these statistical techniques before Research as most of research is inspired by most of the Algorithms.

Resource	Difficulty Level
Mathematics for Machine Learning Part 2
Pattern Recognition and Machine Leanring
Elements of Statistical Learning
Introduction to Statistical Learning
Machine Learning: A Probalisitic Perspective
Berkley CS188 Introduction to AI course
MIT Classic AI course taught by Prof. Patrick H. Winston
Stanford AI course 2018
California Instuite of Technology Learning from Data course
CMU Machine Learning 2015 10-601
CMU Statistical Machine Learning 10-702
Information Theory, Pattern Recognition ML course 2012
Large Scale Machine Learning Toronto University 2015
Algorithmic Aspects of Machine Learning MIT
MIT Course 9.520 - Statistical Learning Theory and Applications, Fall 2015
Undergraduate Machine Learning Course University of British Columbia 2013

Deep Learning

One of the major breakthroughs in the field of intersection between Artificial Intelligence and Computer Science. It lead to countless advances in technology and considered the standard way to do Artificial Intelligence.

Resource	Difficulty Level
Deep Learning Book by Ian Goodfellow
UCL Deepmind Deep Learning
Advanced Talks by Deep Learning Pioneers
Stanford Autumn 2018 Deep Learning Lectures
FAU Deep Learning 2020 Series
CMU Deep Learning course 2020
Stanford Convolutional Neural Network 2017
Oxford Deep Learning Awesome Lectures 2015
Stanford NLP with Deep Learning 2019
Deep Learning from Probability and Statistics POV
Advanced Deep Learning UCL 2017 course + Reinforcement Learning
Deep Learning UC Berkley 2020 Course
NYU Deep Learning with Pytorch hands on
Classic Jeoffrey Hinton Old course OUTDATED
Pieter Abdeel Deep Unsupervised Learning
Hugo Larochelle Deep Learning series
Deep Learning Book Explanation Series
Deep Learning Introduction by Durham University
Fast.ai Practical Deep Learning
Fast.ai Deep Learning From Foundations
Deep Learning with Python (Keras Author)

Reinforcement Learning

It is a sub-field of AI which focuses on learning by observation/rewards.

Resource	Difficulty Level
Introduction to Reinforcement Learning
David Silver Deep Mind Introductory Lectures
Stanford 2018 cs234 Reinforcement Learning
Stanford 2019 cs330 Meta Learning advanced course
Sergie Levine 2018 UC Berkley Lecture Videos
Waterloo cs885 Reinforcement Learing
Sergie Levine 2020 Deep Reinforcement Learning
Reinforcement Learning Specialization Coursea GOLDEN courses (Though it is not free but you can apply for financial aid)

Natural Language Processing

It is a sub-field of AI which focuses on the interpretation of Human Language.

Resource	Difficulty Level
Jurafsky Speech and Language Processing
Christopher Manning Foundations of Statistical NLP
Christopher Manning Introduction to Information Retrieval
cs224n Natural Language Processing with Deep Learning GOLDEN 2019
Oxford Natural Language Processing with Deep Learning 2017
Michigan Introduction to NLP
cs224u Natural Language Understanding 2019
cmu 2021 Neural Nets for NLP 2021
Jurafsky and Manning Introduction to Natural Language Processing

Must Read NLP Papers:

In this section, I am going to list the most influential papers that help people who want to dig deeper into the research world of NLP to catch up. Paper | Comment ------------------------- | ---------------

TODO