Awesome Open Source
Awesome Open Source

From Zero to Research Scientist full resources guide.

Full Guide Version 0.0.1

Guide description

This guide is designated to anybody with basic programming knowledge or a computer science background interested in becoming a Research Scientist with 🎯 on Deep Learning and NLP.

You can go Bottom-Up or Top-Down both works well and it is actually crucial to know which approach suites you the best. If you are okay with studying lots of mathematical concepts without application then use Bottom-Up. If you want to go hands-on first then use the Top-Down first.

Contents:

Mathematical Foundations:

The Mathematical Foundation part is for all Artificial Intelligence branches such as Machine Learning, Reinforcement Learning, Computer Vision and so on. AI is heavily math-theory based so a solid foundation is essential.

Linear Algebra

:infinity:

This branch of Math is crucial for understanding the mechanism of Neural Networks which are the norm for NLP methodologies in nowadays State-of-The-Art.

Resource Difficulty Relevance
MIT Gilbert Strang 2005 Linear Algebra
100% 50% 75%
Linear Algebra 4th Edition by Friedberg
100%
Mathematics for Machine Learning Book: Chapter 2
50% 75%
James Hamblin Awesome Lecture Series
100%
3Blue1Brown Essence of Linear Algebra
25% 100%
Mathematics For Machine Learning Specialization: Linear Algebra
50% 100%
Matrix Methods for Linear Algebra for Gilber Strang UPDATED!
100%

Probability

:atom:

Most of Natural Language Processing and Machine Learning Algorithms are based on Probability theory. So this branch is extremely important for grasping how old methods work. Resource | Difficulty | Relevance ------------------------- | --------------- | ------------------------------- Joe Blitzstein Harvard Probability and Statistics Course |

| 50% 25% 100% MIT Probability Course 2011 Lecture videos |
| 50% 75% MIT Probability Course 2018 short videos UPDATED! |
| 25% 25% 100% Mathematics for Machine Learning Book: Chapter 6 |
| 75% 25% 75% Probalistic Graphical Models CMU Advanced |
| 50% 25% 100% Probalistic Graphical Models Stanford Daphne Advanced |
| 50% 25% 25% A First Course In Probability Book by Ross |
| 50% Joe Blitzstein Harvard Professor Probability Awesome Book |
| 50%

Calculus

📐
Resource Difficulty Relevance
Essence of Calculus by 3Blue1Brown
75%
Single Variable Calculus MIT 2007
75%
Strang's Overview of Calculus
100%
MultiVariable Calculus MIT 2007
100%
Princeton University Multivariable Calculus 2013
100%
Calculus Book by Stewart
100% 25%
Mathematics for Machine Learning Book: Chapter 5
75% 50%

Optimization Theory

-Resource Difficulty Relevance
CMU optimization course 2018
100% 25%
CMU Advanced optimization course
100%
Stanford Famous optimization course
100%
Boyd Convex Optimization Book
100%

Machine Learning

Considered a fancy name for Statistical models where its main goal is to learn from data for several usages. It is considered highly recommended to master these statistical techniques before Research as most of research is inspired by most of the Algorithms.

Resource Difficulty Level
Mathematics for Machine Learning Part 2 Intermediate
Pattern Recognition and Machine Leanring Intermediate
Elements of Statistical Learning Advanced
Introduction to Statistical Learning Introductory
Machine Learning: A Probalisitic Perspective Advanced
Berkley CS188 Introduction to AI course Introductory
MIT Classic AI course taught by Prof. Patrick H. Winston Introductory
Stanford AI course 2018 Intermediate
California Instuite of Technology Learning from Data course Intermediate
CMU Machine Learning 2015 10-601 Intermediate
CMU Statistical Machine Learning 10-702 Intermediate
Information Theory, Pattern Recognition ML course 2012 Intermediate
Large Scale Machine Learning Toronto University 2015 Advanced
Algorithmic Aspects of Machine Learning MIT Advanced
MIT Course 9.520 - Statistical Learning Theory and Applications, Fall 2015 Advanced
Undergraduate Machine Learning Course University of British Columbia 2013 Introductory

Deep Learning

One of the major breakthroughs in the field of intersection between Artificial Intelligence and Computer Science. It lead to countless advances in technology and considered the standard way to do Artificial Intelligence.

Resource Difficulty Level
Deep Learning Book by Ian Goodfellow Advanced
UCL Deepmind Deep Learning Intermediate
Advanced Talks by Deep Learning Pioneers Advanced
Stanford Autumn 2018 Deep Learning Lectures Intermediate
FAU Deep Learning 2020 Series Introductory
CMU Deep Learning course 2020 Introductory
Stanford Convolutional Neural Network 2017 Intermediate
Oxford Deep Learning Awesome Lectures 2015 Intermediate
Stanford NLP with Deep Learning 2019 Intermediate
Deep Learning from Probability and Statistics POV Introductory
Advanced Deep Learning UCL 2017 course + Reinforcement Learning Intermediate
Deep Learning UC Berkley 2020 Course Introductory
NYU Deep Learning with Pytorch hands on Intermediate
Classic Jeoffrey Hinton Old course OUTDATED Intermediate
Pieter Abdeel Deep Unsupervised Learning Advanced
Hugo Larochelle Deep Learning series Introductory
Deep Learning Book Explanation Series Advanced
Deep Learning Introduction by Durham University Introductory
Fast.ai Practical Deep Learning Introductory
Fast.ai Deep Learning From Foundations Introductory
Deep Learning with Python (Keras Author) Intermediate

Reinforcement Learning

It is a sub-field of AI which focuses on learning by observation/rewards.

Resource Difficulty Level
Introduction to Reinforcement Learning Intermediate
David Silver Deep Mind Introductory Lectures Introductory
Stanford 2018 cs234 Reinforcement Learning Intermediate
Stanford 2019 cs330 Meta Learning advanced course Advanced
Sergie Levine 2018 UC Berkley Lecture Videos Advanced
Waterloo cs885 Reinforcement Learing Advanced
Sergie Levine 2020 Deep Reinforcement Learning Advanced
Reinforcement Learning Specialization Coursea GOLDEN courses (Though it is not free but you can apply for financial aid) Intermediate

Natural Language Processing

It is a sub-field of AI which focuses on the interpretation of Human Language.

Resource Difficulty Level
Jurafsky Speech and Language Processing Intermediate
Christopher Manning Foundations of Statistical NLP Advanced
Christopher Manning Introduction to Information Retrieval Advanced
cs224n Natural Language Processing with Deep Learning GOLDEN 2019 Intermediate
Oxford Natural Language Processing with Deep Learning 2017 Intermediate
Michigan Introduction to NLP Introductory
cs224u Natural Language Understanding 2019 Intermediate
cmu 2021 Neural Nets for NLP 2021 Intermediate
Jurafsky and Manning Introduction to Natural Language Processing Introductory

Must Read NLP Papers:

In this section, I am going to list the most influential papers that help people who want to dig deeper into the research world of NLP to catch up. Paper | Comment ------------------------- | ---------------

TODO


Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Machine Learning (30,746
Deep Learning (22,824
Book (2,372
Linear Algebra (821
Probability (612
Calculus (211
Related Projects