Awesome Open Source
Awesome Open Source


A running log of things I'm learning to build strong core software engineering skills while also expanding breadth of knowledge on adjacent technologies a little bit everyday.

Core Skills

Business Understanding
Concept Resource Done
Book: Delivering Happiness
Book: Good to Great: Why Some Companies Make the Leap...And Others Don't
Book: Hello, Startup: A Programmer's Guide to Building Products, Technologies, and Teams
Book: How Google Works
Book: Learn to Earn: A Beginner's Guide to the Basics of Investing and Business
Book: Rework
Book: The Airbnb Story
Book: The Personal MBA
Udacity: How to Build a Startup
Marketing Smartly: Marketing Fundamentals
Udacity: App Marketing
Facebook: Digital marketing: get started
Facebook: Digital marketing: go further
Google Analytics for Beginners
Moz: The Beginner's Guide to SEO
Treehouse: SEO Basics
Udacity: App Monetization
Python Programming
Concept Resource Done
Language Datacamp: Python for R Users
Datacamp: Python for Spreadsheet Users
Datacamp: Intro to Python for Finance
Book: A Byte of Python
Book: Learn Python The Hard way
Datacamp: Writing Efficient Python Code
Datacamp: Writing Functions in Python
Datacamp: Working with Dates and Times in Python
Datacamp: Object-Oriented Programming in Python
Datacamp: Importing Data in Python (Part 1)
Datacamp: Intermediate Python for Data Science
Datacamp: Python Data Science Toolbox (Part 1)
Datacamp: Python Data Science Toolbox (Part 2)
Standard Library Book: Python 201
Book: The Python 3 Standard Library By Example
Calmcode: logging
Calmcode: virtualenv
Calmcode: tqdm
Datacamp: Command Line Automation in Python
Regular Expression Regex For Noobs (like me!) - An Illustrated Guide
Youtube: Python 3 Programming Tutorial - Regular Expressions / Regex with re
Youtube: Python Tutorial: re Module - How to Write and Match Regular Expressions (Regex)
Concurrency Article: Python Concurrency: The Tricky Bits
Article: Speeding Up Python with Concurrency, Parallelism, and asyncio
Article: Speed Up Your Python Program With Concurrency
Youtube: Python Concurrency and Multithreading
Youtube: Aaron Richter- Parallel Processing in Python| PyData Global 2020
Packaging Datacamp: Developing Python Packages
Datacamp: Conda Essentials
Datacamp: Conda for Building & Distributing Packages
Article: Push and pull: when and why to update your dependencies
Article: Reproducible and upgradable Conda environments: dependency management with conda-lock
Article: Options for packaging your Python code: Wheels, Conda, Docker, and more
Project Organization Youtube: Tutorial: Sebastian Witowski - Modern Python Developer's Toolkit
Book: Writing Idiomatic Python 3
Article: Hypermodern Python
Article: Hypermodern Python Chapter 2: Testing
Article: Hypermodern Python Chapter 3: Linting
Article: Hypermodern Python Chapter 4: Typing
Article: pydantic
Article: Hypermodern Python Chapter 5: Documentation
Article: Hypermodern Python Chapter 6: CI/CD
Article: Stop using print, start using loguru in Python
Datacamp: Creating Robust Python Workflows
Datacamp: Software Engineering for Data Scientists in Python
Datacamp: Designing Machine Learning Workflows in Python
Youtube: Hydra configuration
Data Structures and Algorithms
Concept Resource Done
Book: Grokking Algorithms
Codecademy: Big O
Udacity: Intro to Data Structures and Algorithms
Linux & Command Line
Concept Resource Done
Codecademy: Learn the Command Line
Datacamp: Introduction to Shell for Data Science
Datacamp: Introduction to Bash Scripting
Datacamp: Data Processing in Shell
Lecture 1: Course Overview + The Shell (2020) 0:48:16
Lecture 2: Shell Tools and Scripting (2020) 0:48:55
Lecture 3: Editors (vim) (2020) 0:48:26
Lecture 4: Data Wrangling (2020) 0:50:03
Lecture 5: Command-line Environment (2020) 0:56:06
Lecture 7: Debugging and Profiling (2020) 0:54:13
Lecture 8: Metaprogramming (2020) 0:49:52
Lecture 9: Security and Cryptography (2020) 1:00:59
Udacity: Linux Command Line Basics
Udacity: Shell Workshop
Udacity: Configuring Linux Web Servers
Article: Streamline your projects using Makefile
Article: Understand Linux Load Averages and Monitor Performance of Linux
Article: Command-line Tools can be 235x Faster than your Hadoop Cluster
Calmcode: makefiles
Calmcode: entr
Version Control
Concept Resource Done
Git Udacity: Version Control with Git
Datacamp: Introduction to Git for Data Science
Thoughtbot: Mastering Git
MIT Lecture 6: Version Control (git) (2020) 1:24:59
Article: Mastering Git Stash Workflow
Article: How to Become a Master of Git Tags
Article: Keep your git directory clean with git clean and git trash
GitHub Udacity: GitHub & Collaboration
Udacity: How to Use Git and GitHub
LFS Youtube: 045 Introduction to Git LFS
Article: How to track large files in Github / Bitbucket? Git LFS to the rescue
Code Editor / IDE
Concept Resource Done
PyCharm Article: Work remotely with PyCharm, TensorFlow and SSH
Article: Docker as Remote Interpreter for PyCharm Professional
Article: Python remote debugging with PyCharm, CUDA, and Conda
VSCode Article: How To Use Visual Studio Code for Remote Development via the Remote-SSH Plugin
Youtube: Getting Started with Python in Visual Studio Code
Visual Studio Code Crash Course
Youtube: VSCode Keyboard Shortcuts For Productivity
Youtube: Getting Started with Jupyter Notebooks in VS Code
Youtube: Notebooks in VS Code Are Getting Revamped!
Youtube: Getting Started with PyTorch in VS Code
Youtube: What every GitHub user should know about VS Code - GitHub Satellite 2020
VS Code and GitHub
Test-Driven Development
Concept Resource Done
Test Cases Article: Test-Driven Machine Learning Development (Deployment Series: Guide 07)
Pluralsight: Test-driven Development: The Big Picture
Test Driven Development with Python
Datacamp: Unit Testing for Data Science in Python
Article: How to cheat at unit tests with pytest and Black
Youtube: Lab 8: Testing and Continuous Integration (Full Stack Deep Learning - Spring 2021) 0:13:26
Article: 4 Lesser-Known Yet Awesome Tips for Pytest
Article: How to Unit Test Deep Learning: Tests in TensorFlow, mocking and test coverage
Article: Unit Testing for Data Scientists
ML Article: Effective testing for machine learning systems
Youtube: Beyond Accuracy: Behavioral Testing of NLP Models with CheckList | AISC
Youtube: Lecture 10: ML Testing & Explainability (Full Stack Deep Learning - Spring 2021) 1:41:12
Web Technology
Concept Resource Done
Design Book: Refactoring UI
Code School: Fundamentals of Design
Thoughtbot: Design for Developers
Udacity: Product Design
Udacity: Rapid Prototyping
HTML Codecademy: Learn HTML
Codecademy: Make a website
Treehouse: HTML
CSS Pluralsight: CSS Positioning
Pluralsight: Introduction to CSS
Pluralsight: CSS: Specificity, the Box Model, and Best Practices
Pluralsight: CSS: Using Flexbox for Layout
Code School: Blasting Off with Bootstrap
Pluralsight: UX Fundamentals
Codecademy: Learn SASS
Javascript Treehouse: Javascript Booleans
Udacity: ES6 - JavaScript Improved
Udacity: Intro to Javascript
Udacity: Object Oriented JS 1
Udacity: Object Oriented JS 2
(ES6) - Beau teaches JavaScript
Udemy: Understanding Typescript
Codecademy: Learn ReactJS: Part I
Codecademy: Learn ReactJS: Part II
Codecademy: Learn JavaScript
Codecademy: Jquery Track
Pluralsight: Using The Chrome Developer Tools
Backend & Web Servers
Concept Resource Done
Theory Udacity: Authentication & Authorization: OAuth
Udacity: HTTP & Web Servers
Udacity: Client-Server Communication
Udacity: Designing RESTful APIs
Udacity: Networking for Web Developers
FastAPI Article: Microservice in Python using FastAPI
Youtube: PyConBY 2020: Sebastian Ramirez - Serve ML models easily with FastAPI
Youtube: FastAPI from the ground up
Youtube: Python pydantic Introduction – Give your data classes super powers
Gunicorn Article: Selecting gunicorn worker types for different python web applications.
Article: Better performance by optimizing Gunicorn config
Tensorflow Serving Article: Understanding TensorFlow Serving
Article: Serving models using Tensorflow Serving and Docker
Cortex Youtube: PyData Vancouver meetup: : Serving machine learning models in production
Celery Article: Celery Execution Pools: What is it all about?
Article: Distill: Why do we need Flask, Celery, and Redis? (with McDonalds in Between)
Article: Celery: an overview of the architecture and how it works
Article: Unit Testing Celery Tasks
Article: Testing Celery Chains
Article: Task Routing in Celery
Article: Dynamic Task Routing in Celery
Article: Dockerize a Celery app with Django and RabbitMQ
Article: How to call a Celery task from another app
Article: Distributed Monte Carlo with Celery chords
Article: An incredibly simple no-frills Celery setup
Article: 3 Strategies to Customise Celery logging handlers
Article: Celery task exceptions and automatic retries
Article: Concurrency and Parallelism
Article: Celery, docker and the missing startup banner
Article: Monitoring a Dockerized Celery Cluster with Flower
Article: Quick Guide: Custom Celery Task Logger
Article: Celery on Docker: From the Ground up
Article: Auto-reload Celery on code changes
Concept Resource Done
Udacity: Intro to relational database
Udacity: Database Systems Concepts & Design
Datacamp: Database Design
Datacamp: Introduction to Databases in Python
Codecademy: SQL Track
Datacamp: Intro to SQL for Data Science
Datacamp: Intermediate SQL
Datacamp: Querying with TransactSQL
Datacamp: Joining Data in PostgreSQL
Udacity: SQL for Data Analysis
Datacamp: Exploratory Data Analysis in SQL
Datacamp: Applying SQL to Real-World Problems
Datacamp: Analyzing Business Data in SQL
Datacamp: Reporting in SQL
Datacamp: Data-Driven Decision Making in SQL
Production Environment
Concept Resource Done
A/B Testing Datacamp: Customer Analytics & A/B Testing in Python
Udacity: A/B Testing
Udacity: A/B Testing for Business Analysts
Load Testing Youtube: Loading Testing with Python
Monitoring Article: Production Machine Learning Monitoring: Outliers, Drift, Explainers & Statistical Performance
Article: How to Monitor Models
Article: The Playbook to Monitor Your Model’s Performance in Production
Article: Monitoring your Machine Learning Model
Article: Preventing model drift with continuous monitoring and deployment using Github Actions and Algorithmia Insights
Article: Continuous monitoring for data projects
Article: Lessons Learned from 15 Years of Monitoring Machine Learning in Production
Article: Using Statistical Distances for Machine Learning Observability
Youtube: Instrumentation, Observability & Monitoring of Machine Learning Models
Article: Incident Management in Machine Learning Systems
Article: ML Infrastructure Tools — ML Observability
Youtube: MLOps #24 Monitoring the ML stack // Lina Weichbrodt 0:55:32
Youtube: Josh Wills: Visibility and Monitoring for Machine Learning Models
Youtube: Lecture 11B: Monitoring ML Models (Full Stack Deep Learning - Spring 2021) 0:36:55
Youtube: OpML '20 - How ML Breaks: A Decade of Outages for One Large ML Pipeline
Youtube: MLOps #28 ML Observability // Aparna Dhinakaran - Chief Product Officer at Arize AI 0:55:04
Youtube: MLOps #29 Continuous Evaluation & Model Experimentation // Danny Ma - Founder of Sydney Data Science 1:00:46
Youtube: SE4AI: Quality Assessment in Production 1:18:45
Youtube: SE4AI: Infrastructure Quality, Deployment and Operations 1:04:54
System and Infrastructure Design
Concept Resource Done
Datacamp: Data Engineering for Everyone
Article: Batch Inference vs Online Inference
Article: Machine Learning System Design: Real-time processing
Article: Machine Learning System Design: Models-as-a-service
Article: What Does it Mean to Deploy a Machine Learning Model? (Deployment Series: Guide 01)
Article: Software Interfaces for Machine Learning Deployment (Deployment Series: Guide 02)
Article: Batch Inference for Machine Learning Deployment (Deployment Series: Guide 03)
Article: The Challenges of Online Inference (Deployment Series: Guide 04)
Article: Online Inference for ML Deployment (Deployment Series: Guide 05)
Article: Model Registries for ML Deployment (Deployment Series: Guide 06)
Youtube: A friendly introduction to System Design
Youtube: System Design Basics: Horizontal vs. Vertical Scaling
Youtube: What is a microservice architecture and it's advantages?
Youtube: Service discovery and heartbeats in micro-services
Youtube: Avoid cascading failures in a distributed system
Youtube: How databases scale writes: The power of the log
Youtube: How to avoid a single point of failure in distributed systems
Youtube: How to start with distributed systems? Beginner's guide to scaling systems.
Youtube: What's an Event Driven System?
Youtube: Why do Databases fail? AntiPatterns to avoid!
Youtube: What is Consistent Hashing and Where is it used?
Youtube: What is a Message Queue and Where is it used?
Youtube: What is an API and how do you design it?
Youtube: Introduction to NoSQL databases
Article: Exponential Backoff And Jitter
Youtube: What is Database Sharding?
Youtube: What is the Publisher Subscriber Model?
Article: Shadow mode deployments
Youtube: Relational database index vs. NoSQL index
Youtube: Capacity Estimation: How much data does YouTube store daily?
Youtube: What is Load Balancing?
Youtube: Distributed Consensus and Data Replication strategies on the server
Youtube: What is Distributed Caching? Explained with Redis!
Youtube: Designing Instagram: System Design of News Feed
Youtube: System Design: Tinder as a microservice architecture
Youtube: System design : Design Autocomplete or Typeahead Suggestions for Google search
Youtube: Whatsapp System Design: Chat Messaging Systems for Interviews
Youtube: How Netflix onboards new content: Video Processing at scale
Article: Building a feature store
Article: Model artifacts: the war stories
Youtube: Feature Stores: An essential part of the ML stack to build great data / Kevin Stumpf - CTO at Tecton 1:05:46
Youtube: MLOps Meetup #6: Mid-Scale Production Feature Engineering with Dr. Venkata Pingali 1:01:35
Article: How to Deploy a Machine Learning Model
Article: How to properly ship and deploy your machine learning model
Article: The Ultimate Guide to Model Retraining
Youtube: Lecture 11A: Deploying ML Models (Full Stack Deep Learning - Spring 2021) 0:53:25
Article: Deploying Machine Learning Models: A Checklist
Article: How to put machine learning models into production
Youtube: MLOps meetup #5 High Stakes ML with Flavio CLesio 0:55:27
Youtube: MLOps meetup #7 Alex Spanos // TrueLayer 's MLOps Pipeline 0:56:17
Youtube: The Current MLOps Landscape // Nathan Benaich & Timothy Chen // MLOps Meetup #43 0:58:31
Article: How to build scalable Machine Learning systems — Part 1/2
Article: Machine learning is going real-time
Book: Machine Learning Systems Design
Article: ML Infrastructure Tools for Model Building
Article: ML Infrastructure Tools for Production (Part 1)
Article: ML Infrastructure Tools for Production
Article: Data Lineage — An Operational perspective
Article: Data Pipelines — Agile considerations
Article: Securing ML applications
Article: Getting machine learning to production
Article: Machine Learning to Production
Youtube: SE4AI: Invited Talk Molham Aref "Business Systems with Machine Learning" 0:47:53
Youtube: SE4AI: Software Architecture of AI-Enabled Systems 1:14:24
Youtube: MLOps #31 Path to Production and Monetizing Machine Learning // Vin Vashishta - Data Scientist 0:56:35
Youtube: MLOps #35: Streaming Machine Learning with Apache Kafka and Tiered Storage // Kai Waehner, Confluent 0:52:50
Youtube: MLOps #15 - Scaling Human in the Loop Machine Learning with Robert Munro 0:55:04
Youtube: MLOps #4: Shubhi Jain - Building an ML Platform @SurveyMonkey 0:55:42
Youtube: #11 Machine Learning at scale in Mercado Libre with Carlos de la Torre 0:59:28
Youtube: MLOps #18 // Nubank - Running a fintech on ML 0:53:19
Youtube: Shawn Scully: Production and Beyond: Deploying and Managing Machine Learning Models
Doc: Lecture 3: Data engineering
Youtube: MLOps #14: Kubeflow vs MLflow with Byron Allen 0:54:57
Youtube: Luigi in Production // MLOps Coffee Sessions #18 // Luigi Patruno ML in Production 0:47:23
Stanford MLSys Seminar Episode 1: Marco Tulio Ribeiro 1:00:38
Stanford MLSys Seminar Episode 2: Matei Zaharia 0:59:44
Stanford MLSys Seminar Episode 3: Virginia Smith 1:00:55
Stanford MLSys Seminar Episode 4: Alex Ratner 1:13:34
Stanford MLSys Seminar Episode 5: Chip Huyen 1:06:44
Youtube: Xavier Amatriain on Practical Deep Learning Systems (Full Stack Deep Learning - November 2019)
Concept Resource Done
Probability Article: Entropy, Cross Entropy, and KL Divergence
Article: Interview Guide to Probability Distributions
Article: Entropy of a probability distribution — in layman’s terms
Article: KL Divergence — in layman’s terms
Article: Probability Distributions
Article: Cross-Entropy and KL Divergence
Article: Why Randomness Is Information?
Article: Basic Probability Theory
Datacamp: Foundations of Probability in Python
Statistics Datacamp: Introduction to Statistics
Datacamp: Introduction to Statistics in Python
Datacamp: Hypothesis Testing in Python
Datacamp: Statistical Thinking in Python (Part 1)
Datacamp: Statistical Thinking in Python (Part 2)
Datacamp: Experimental Design in Python
Datacamp: Statistical Simulation in Python
edX: Essential Statistics for Data Analysis using Excel
StatQuest: Histograms, Clearly Explained 0:03:42
StatQuest: What is a statistical distribution? 0:05:14
StatQuest: The Normal Distribution, Clearly Explained!!! 0:05:12
Statistics Fundamentals: Population Parameters 0:14:31
Statistics Fundamentals: The Mean, Variance and Standard Deviation 0:14:22
StatQuest: What is a statistical model? 0:03:45
StatQuest: Sampling A Distribution 0:03:48
Hypothesis Testing and The Null Hypothesis 0:14:40
Alternative Hypotheses: Main Ideas!!! 0:09:49
p-values: What they are and how to interpret them 0:11:22
How to calculate p-values 0:25:15
p-hacking: What it is and how to avoid it! 0:13:44
Statistical Power, Clearly Explained!!! 0:08:19
Power Analysis, Clearly Explained!!! 0:16:44
Covariance and Correlation Part 1: Covariance 0:22:23
Covariance and Correlation Part 2: Pearson's Correlation 0:19:13
StatQuest: R-squared explained 0:11:01
The Central Limit Theorem 0:07:35
StatQuickie: Standard Deviation vs Standard Error 0:02:52
StatQuest: The standard error 0:11:43
StatQuest: Technical and Biological Replicates 0:05:27
StatQuest - Sample Size and Effective Sample Size, Clearly Explained 0:06:32
Bar Charts Are Better than Pie Charts 0:01:45
StatQuest: Boxplots, Clearly Explained 0:02:33
StatQuest: Logs (logarithms), clearly explained 0:15:37
StatQuest: Confidence Intervals 0:06:41
StatQuickie: Thresholds for Significance 0:06:40
StatQuickie: Which t test to use 0:05:10
StatQuest: One or Two Tailed P-Values 0:07:05
The Binomial Distribution and Test, Clearly Explained!!! 0:15:46
StatQuest: Quantiles and Percentiles, Clearly Explained!!! 0:06:30
StatQuest: Quantile-Quantile Plots (QQ plots), Clearly Explained 0:06:55
StatQuest: Quantile Normalization 0:04:51
StatQuest: Probability vs Likelihood 0:05:01
StatQuest: Maximum Likelihood, clearly explained!!! 0:06:12
Maximum Likelihood for the Exponential Distribution, Clearly Explained! V2.0 0:09:39
Why Dividing By N Underestimates the Variance 0:17:14
Maximum Likelihood for the Binomial Distribution, Clearly Explained!!! 0:11:24
Maximum Likelihood For the Normal Distribution, step-by-step! 0:19:50
StatQuest: Odds and Log(Odds), Clearly Explained!!! 0:11:30
StatQuest: Odds Ratios and Log(Odds Ratios), Clearly Explained!!! 0:16:20
Live 2020-04-20!!! Expected Values 0:33:00
Udacity: Statistics
Udacity: Intro to Inferential Statistics
Calculus The Essence of Calculus, Chapter 1 0:17:04
The paradox of the derivative | Essence of calculus, chapter 2 0:17:57
Derivative formulas through geometry | Essence of calculus, chapter 3 0:18:43
Visualizing the chain rule and product rule | Essence of calculus, chapter 4 0:16:52
What's so special about Euler's number e? | Essence of calculus, chapter 5 0:13:50
Implicit differentiation, what's going on here? | Essence of calculus, chapter 6 0:15:33
Limits, L'Hôpital's rule, and epsilon delta definitions | Essence of calculus, chapter 7 0:18:26
Integration and the fundamental theorem of calculus | Essence of calculus, chapter 8 0:20:46
What does area have to do with slope? | Essence of calculus, chapter 9 0:12:39
Higher order derivatives | Essence of calculus, chapter 10 0:05:38
Taylor series | Essence of calculus, chapter 11 0:22:19
What they won't teach you in calculus 0:16:22
But what is a Neural Network? | Deep learning, chapter 1 0:19:13
Gradient descent, how neural networks learn | Deep learning, chapter 2 0:21:01
What is backpropagation really doing? | Deep learning, chapter 3 0:13:54
Backpropagation calculus | Deep learning, chapter 4 0:10:17
Article: A Visual Tour of Backpropagation
Linear Algebra Vectors, what even are they? | Essence of linear algebra, chapter 1 0:09:52
Linear combinations, span, and basis vectors | Essence of linear algebra, chapter 2 0:09:59
Linear transformations and matrices | Essence of linear algebra, chapter 3 0:10:58
Matrix multiplication as composition | Essence of linear algebra, chapter 4 0:10:03
Three-dimensional linear transformations | Essence of linear algebra, chapter 5 0:04:46
The determinant | Essence of linear algebra, chapter 6 0:10:03
Inverse matrices, column space and null space | Essence of linear algebra, chapter 7 0:12:08
Nonsquare matrices as transformations between dimensions | Essence of linear algebra, chapter 8 0:04:27
Dot products and duality | Essence of linear algebra, chapter 9 0:14:11
Cross products | Essence of linear algebra, Chapter 10 0:08:53
Cross products in the light of linear transformations | Essence of linear algebra chapter 11 0:13:10
Cramer's rule, explained geometrically | Essence of linear algebra, chapter 12 0:12:12
Change of basis | Essence of linear algebra, chapter 13 0:12:50
Eigenvectors and eigenvalues | Essence of linear algebra, chapter 14 0:17:15
Abstract vector spaces | Essence of linear algebra, chapter 15 0:16:46
Article: Introduction to Linear Algebra for Applied Machine Learning with Python
Article: Relearning Matrices as Linear Functions
Article: You Could Have Come Up With Eigenvectors - Here's How
Article: PageRank - How Eigenvectors Power the Algorithm Behind Google Search
Article: Interactive Visualization of Why Eigenvectors Matter
Book: Basics of Linear Algebra for Machine Learning
Computational Linear Algebra for Coders
1. The Geometry of Linear Equations 0:39:49
2. Elimination with Matrices. 0:47:41
3. Multiplication and Inverse Matrices 0:46:48
4. Factorization into A = LU 0:48:05
5. Transposes, Permutations, Spaces R^n 0:47:41
6. Column Space and Nullspace 0:46:01
9. Independence, Basis, and Dimension 0:50:14
10. The Four Fundamental Subspaces 0:49:20
11. Matrix Spaces; Rank 1; Small World Graphs 0:45:55
14. Orthogonal Vectors and Subspaces 0:49:47
15. Projections onto Subspaces 0:48:51
16. Projection Matrices and Least Squares 0:48:05
17. Orthogonal Matrices and Gram-Schmidt 0:49:09
21. Eigenvalues and Eigenvectors 0:51:22
22. Diagonalization and Powers of A 0:51:50
24. Markov Matrices; Fourier Series 0:51:11
25. Symmetric Matrices and Positive Definiteness 0:43:52
27. Positive Definite Matrices and Minima 0:50:40
29. Singular Value Decomposition 0:40:28
30. Linear Transformations and Their Matrices 0:49:27
31. Change of Basis; Image Compression 0:50:13
33. Left and Right Inverses; Pseudoinverse 0:41:52
Udacity: Eigenvectors and Eigenvalues
Udacity: Linear Algebra Refresher
Interview Preparation
Concept Resource Done
Book: Machine Learning Interviews
Datacamp: Preparing for Statistics Interview Questions in Python
Datacamp: Practicing Machine Learning Interview Questions in Python
Datacamp: Kaggle Competition
Udacity: Optimize your GitHub
Udacity: Strengthen Your LinkedIn Network & Brand
Udacity: Data Science Interview Prep
Udacity: Full-Stack Interview Prep
Udacity: Refresh Your Resume
Udacity: Craft Your Cover Letter
Youtube: Guest Lecture - Chip Huyen - Machine Learning Interviews - Full Stack Deep Learning
Youtube: Tutorial: Technical Blogging for Python Programmers

Specialized Skills

Machine Learning Fundamentals
Concept Resource Done
Regression Article: Linear regression
Article: Polynomial regression
StatQuest: Fitting a line to data, aka least squares, aka linear regression. 0:09:21
StatQuest: Linear Models Pt.1 - Linear Regression 0:27:26
StatQuest: StatQuest: Linear Models Pt.2 - t-tests and ANOVA 0:11:37
StatQuest: Fiitting a curve to data, aka lowess, aka loess 0:10:10
Naive Bayes Article: Naive Bayes classification
Naive Bayes, Clearly Explained!!! 0:15:12
Gaussian Naive Bayes, Clearly Explained!!! 0:09:41
Logistic Regression Article: Logistic regression
Datacamp: Foundations of Predictive Analytics in Python (Part 1)
Datacamp: Foundations of Predictive Analytics in Python (Part 2)
StatQuest: Odds and Log(Odds), Clearly Explained!!! 0:11:30
StatQuest: Odds Ratios and Log(Odds Ratios), Clearly Explained!!! 0:16:20
StatQuest: Logistic Regression 0:08:47
Logistic Regression Details Pt1: Coefficients 0:19:02
Logistic Regression Details Pt 2: Maximum Likelihood 0:10:23
Logistic Regression Details Pt 3: R-squared and p-value 0:15:25
Saturated Models and Deviance 0:18:39
Deviance Residuals 0:06:18
Regularization Part 1: Ridge (L2) Regression 0:20:26
Regularization Part 2: Lasso (L1) Regression 0:08:19
Ridge vs Lasso Regression, Visualized!!! 0:09:05
Regularization Part 3: Elastic Net Regression 0:05:19
Article: One-vs-Rest strategy for Multi-Class Classification
Article: Multi-class Classification — One-vs-All & One-vs-One
Article: One-vs-Rest and One-vs-One for Multi-Class Classification
Decision Trees Article: Decision trees
StatQuest: Decision Trees 0:17:22
StatQuest: Decision Trees, Part 2 - Feature Selection and Missing Data 0:05:16
Decision Trees in Python from Start to Finish 1:06:23
Regression Trees, Clearly Explained!!! 0:22:33
How to Prune Regression Trees, Clearly Explained!!! 0:16:15
KNN Article: K-nearest neighbors
SVM Article: Support Vector Machines
Support Vector Machines, Clearly Explained!!! 0:20:32
Support Vector Machines Part 2: The Polynomial Kernel 0:07:15
Support Vector Machines Part 3: The Radial (RBF) Kernel 0:15:52
Bagging Article: Random forests
StatQuest: Random Forests Part 1 - Building, Using and Evaluating 0:09:54
StatQuest: Random Forests Part 2: Missing data and clustering 0:11:53
Boosting Article: Boosted trees
AdaBoost, Clearly Explained 0:20:54
Gradient Boost Part 1: Regression Main Ideas 0:15:52
Gradient Boost Part 2: Regression Details 0:26:45
Gradient Boost Part 3: Classification 0:17:02
Gradient Boost Part 4: Classification Details 0:36:59
Datacamp: Ensemble Methods in Python
XGBoost Part 1: Regression 0:25:46
XGBoost Part 2: Classification 0:25:17
XGBoost Part 3: Mathematical Details 0:27:24
XGBoost Part 4: Crazy Cool Optimizations 0:24:27
Datacamp: Extreme Gradient Boosting with XGBoost
Dimensionality Reduction StatQuest: Principal Component Analysis (PCA), Step-by-Step 0:21:57
StatQuest: PCA main ideas in only 5 minutes!!! 0:06:04
StatQuest: PCA - Practical Tips 0:08:19
StatQuest: PCA in Python 0:11:37
StatQuest: Linear Discriminant Analysis (LDA) clearly explained. 0:15:12
StatQuest: MDS and PCoA 0:08:18
StatQuest: t-SNE, Clearly Explained 0:11:47
Clustering StatQuest: Hierarchical Clustering 0:11:19
StatQuest: K-means clustering 0:08:57
StatQuest: K-nearest neighbors, Clearly Explained 0:05:30
Datacamp: Customer Segmentation in Python
Datacamp: Unsupervised Learning in Python
Udacity: Segmentation and Clustering
Youtube: Clustering Algorithms
Neural Networks Coursera: Neural Networks and Deep Learning Deep Learning for Coder (2020)
Gradient Descent, Step-by-Step 0:23:54
The Chain Rule 0:18:23
Stochastic Gradient Descent, Clearly Explained!!! 0:10:53
Article: Neural networks: activation functions
Article: Neural networks: training with backpropagation
Article: Neural Network from scratch-part 1
Article: Neural Network from scratch-part 2
Article: Perceptron to Deep-Neural-Network
Neural Networks from Scratch - P.1 Intro and Neuron Code 0:16:59
Neural Networks from Scratch - P.2 Coding a Layer 0:15:06
Neural Networks from Scratch - P.3 The Dot Product 0:25:17
Neural Networks from Scratch - P.4 Batches, Layers, and Objects 0:33:46
Neural Networks from Scratch - P.5 Hidden Layer Activation Functions 0:40:05
An overview of gradient descent optimization algorithms
Article: Optimization for Deep Learning Highlights in 2017
Article: Gradient descent
Article: Setting the learning rate of your neural network
Article: Dismantling Neural Networks to Understand the Inner Workings with Math and Pytorch
Youtube: Deep Double Descent
Article: Connections: Log Likelihood, Cross Entropy, KL Divergence, Logistic Regression, and Neural Networks
Article: MLE and MAP — in layman’s terms
Article: Cross-entropy for classification
Evaluation Metrics Article: Measuring Performance: AUPRC and Average Precision
Article: Measuring Performance: AUC (AUROC)
Article: Measuring Performance: The Confusion Matrix
Article: Measuring Performance: Accuracy
Article: ROC Curves: Intuition Through Visualization
Article: Precision, Recall, Accuracy, and F1 Score for Multi-Label Classification
Article: The Complete Guide to AUC and Average Precision: Simulations and Visualizations
Article: Proxy Metrics
Youtube: Applied ML 2020 - 09 - Model Evaluation and Metrics 1:18:23
Article: Validating your Machine Learning Model
Youtube: Machine Learning Fundamentals: Cross Validation 0:06:04
Youtube: Machine Learning Fundamentals: The Confusion Matrix 0:07:12
Youtube: Machine Learning Fundamentals: Sensitivity and Specificity 0:11:46
Youtube: Machine Learning Fundamentals: Bias and Variance 0:06:36
Youtube: ROC and AUC, Clearly Explained! 0:16:26
Article: The correct way to evaluate online machine learning models
Youtube: Accuracy as a Failure
Article: Best Use of Train/Val/Test Splits, with Tips for Medical Data
Machine Learning Libraries
Concept Resource Done
Numpy Article: A Visual Intro to NumPy and Data Representation
Article: Good practices with numpy random number generators
Article: NumPy Illustrated: The Visual Guide to NumPy
Article: NumPy Fundamentals for Data Science and Machine Learning
Datacamp: Intro to Python for Data Science
Pluralsight: Working with Multidimensional Data Using NumPy
Pandas Article: Visualizing Pandas' Pivoting and Reshaping Functions
Article: A Gentle Visual Intro to Data Analysis in Python Using Pandas
Article: Comprehensive Guide to Grouping and Aggregating with Pandas
Article: 8 Python Pandas Value_counts() tricks that make your work more efficient
Datacamp: pandas Foundations
Datacamp: Pandas Joins for Spreadsheet Users
Datacamp: Manipulating DataFrames with pandas
Datacamp: Merging DataFrames with pandas
Datacamp: Data Manipulation with pandas
Datacamp: Optimizing Python Code with pandas
Datacamp: Streamlined Data Ingestion with pandas
Datacamp: Analyzing Marketing Campaigns with pandas
edX: Implementing Predictive Analytics with Spark in Azure HDInsight
Modern Pandas (Part 1)
Modern Pandas (Part 2)
Modern Pandas (Part 3)
Modern Pandas (Part 4)
Modern Pandas (Part 5)
Modern Pandas (Part 6)
Modern Pandas (Part 7)
Modern Pandas (Part 8)
Jupyter Article: Securely storing configuration credentials in a Jupyter Notebook
Article: Automatically Reload Modules with %autoreload
Calmcode: ipywidgets
Documentation: Jupyter Lab
Pluralsight: Getting Started with Jupyter Notebook and Python
Youtube: William Horton - A Brief History of Jupyter Notebooks
Youtube: I Like Notebooks
Youtube: I don't like notebooks.- Joel Grus (Allen Institute for Artificial Intelligence)
Youtube: Ryan Herr - After, before you deploy| JupyterCon 2020
Youtube: nbdev live coding with Hamel Husain
Youtube: How to Use JupyterLab
DVC Versioning Data with DVC (Hands-On Tutorial!) 0:13:04
Sharing Data and Models with DVC (Hands-On Data Science Tutorial!) 0:08:53
Article: ML Ops: Data Science Version Control
Youtube: Data versioning in machine learning projects - Dmitry Petrov 0:34:44
Zoom: Data versioning with DVC Part 1
Zoom: Data versioning with DVC Part 2
scikit-learn Article: Stacking made easy with Sklearn
Article: Curve Fitting With Python
Article: A Guide to Calibration Plots in Python
Calmcode: human-learn
Datacamp: Supervised Learning with scikit-learn
Datacamp: Machine Learning with Tree-Based Models in Python
Datacamp: Introduction to Linear Modeling in Python
Datacamp: Linear Classifiers in Python
Datacamp: Generalized Linear Models in Python
Notebook: scikit-learn tips
Pluralsight: Building Machine Learning Models in Python with scikit-learn
Video: human learn
Youtube: dabl: Automatic Machine Learning with a Human in the Loop 00:25:43
Youtube: Multilabel and Multioutput Classification -Machine Learning with TensorFlow & scikit-learn on Python
Youtube: DABL: Automatic machine learning with a human in the loop- AI Latim American SumMIT Day 1
Tensorflow Coursera: Introduction to Tensorflow
Coursera: Convolutional Neural Networks in TensorFlow
Deeplizard: Keras - Python Deep Learning Neural Network API
Book: Deep Learning with Python (Page: 276)
Datacamp: Deep Learning in Python
Datacamp: Convolutional Neural Networks for Image Processing
Datacamp: Introduction to TensorFlow in Python
Datacamp: Introduction to Deep Learning with Keras
Datacamp: Advanced Deep Learning with Keras
Pluralsight: Deep Learning with Keras
Udacity: Intro to TensorFlow for Deep Learning
PyTorch Article: The One PyTorch Trick Which You Should Know
Article: How does automatic differentiation really work?
Article: 7 Tips To Maximize PyTorch Performance
Article: An introduction to PyTorch Lightning with comparisons to PyTorch
Article: Converting From Keras To PyTorch Lightning
Article: From PyTorch to PyTorch Lightning — A gentle introduction
Article: Introducing PyTorch Lightning Sharded: Train SOTA Models, With Half The Memory
Article: Sharded: A New Technique To Double The Size Of PyTorch Models
Article: Understanding Bidirectional RNN in PyTorch
Article: PyTorch Lightning Bolts — From Linear, Logistic Regression on TPUs to pre-trained GANs
Article: Scaling Logistic Regression Via Multi-GPU/TPU Training
Article: Training Neural Nets on Larger Batches: Practical Tips for 1-GPU, Multi-GPU & Distributed setups
Article: PyTorch Lightning 0.9 — synced BatchNorm, DataModules and final API!
Article: PyTorch Lightning: Metrics
Article: PyTorch Multi-GPU Metrics Library and More in PyTorch Lightning 0.8.1
Article: Faster Deep Learning Training with PyTorch – a 2021 Guide
Article: Fit More and Train Faster With ZeRO via DeepSpeed and FairScale
Article: PyTorch Lightning V1.2.0- DeepSpeed, Pruning, Quantization, SWA
Article: But what are PyTorch DataLoaders really?
Article: Using PyTorch + NumPy? You're making a mistake.
Article: How Wadhwani AI Uses PyTorch To Empower Cotton Farmers
Article: Taming LSTMs: Variable-sized mini-batches and why PyTorch is good for your health
Article: How to Build a Streaming DataLoader with PyTorch
Article: Transform your ML-model to Pytorch with Hummingbird
Article: PyTorch Loss Functions: The Ultimate Guide
Article: Pad pack sequences for Pytorch batch processing with DataLoader
Article: Model Parallelism
Notebook: Tensor Arithmetic
Notebook: Autograd
Notebook: Optimization
Notebook: Network modules
Notebook: Datasets and Dataloaders
Documentation: Pytorch Lightning
Datacamp: Introduction to Deep Learning with PyTorch
Deeplizard: Neural Network Programming - Deep Learning with PyTorch
Youtube: PyTorch Lightning 101
Youtube: SimCLR with PyTorch Lightning
Youtube: PyTorch Performance Tuning Guide 26:41:00
Youtube: Skin Cancer Detection with PyTorch
Youtube: Learn with Lightning
Youtube: PyTorch Tutorial - RNN & LSTM & GRU - Recurrent Neural Nets 00:15:51
Youtube: Pytorch Zero to All
PyTorch Developer Day 2020 | Full Livestream
Youtube: Lightning Chat: How a Grandmaster Won a Kaggle Competition Using Pytorch Lightning
Youtube: Production Inference Deployment with PyTorch
Youtube: What is Automatic Differentiation?
BeautifulSoup Docs: Beautiful Soup Documentation
Datacamp: Importing Data in Python (Part 2)
Datacamp: Web Scraping in Python
Docker and Containerization
Concept Resource Done
Pluralsight: Docker and Kubernetes: The Big Picture
Youtube: Docker
Youtube: Why Your Web Server Should Log to Stdout (Especially with Docker)
Article: How To Pass Environment Info During Docker Builds
Article: Pass Docker Environment Variables During The Image Build
Article: Setting Default Docker Environment Variables During Image Build
Article: Docker Explained Visually, For Non-Technical Folks
Article: Tensorflow in Docker
Article: Enough Docker to be Dangerous
Article: How Docker Can Help You Become A More Effective Data Scientist
Article: Deploying conda environments in (Docker) containers - how to do it right
Article: Configuring Gunicorn for Docker
Article: How to scale services using Docker Compose
Article: A Beginner-Friendly Introduction to Containers, VMs and Docker
Article: Smaller Docker images with Conda
Pluralsight: Docker and Containers: The Big Picture
Article: Docker for Machine Learning – Part I
Article: Docker for Machine Learning – Part II
Article: Docker for Machine Learning – Part III
Article: Using Docker to Generate Machine Learning Predictions in Real Time
Article: Connection refused? Docker networking and how it impacts your image
Article: Faster or slower: the basics of Docker build caching
Article: Where’s your code? Debugging ImportError and ModuleNotFoundErrors in your Docker image
Article: A tableau of crimes and misfortunes: the ever-useful docker history
Article: Broken by default: why you should avoid most Dockerfile examples
Article: A review of the official Dockerfile best practices: good, bad, and insecure
Article: The best Docker base image for your Python application (February 2021)
Article: A deep dive into the official Docker image for Python
Article: Using Alpine can make Python Docker builds 50× slower
Article: Building on solid ground: ensuring reproducible Docker builds for Python
Article: Installing system packages in Docker with minimal bloat
Article: Less capabilities, more security: minimizing privilege escalation in Docker
Article: Avoiding insecure images from Docker build caching
Article: Build secrets in Docker and Compose, the secure way
Article: Security scanners for Python and Docker: from code to dependencies
Article: The high cost of slow Docker builds
Article: Faster Docker builds with pipenv, poetry, or pip-tools
Article: Elegantly activating a virtualenv in a Dockerfile
Article: Poetry vs. Docker caching: Fight!
Article: Speed up pip downloads in Docker with BuildKit’s new caching
Article: Multi-stage builds #1: Smaller images for compiled code
Article: Multi-stage builds #2: Python specifics—virtualenv, –user, and other methods
Article: Multi-stage builds #3: Why your build is surprisingly slow, and how to speed it up
Article: Configuring Gunicorn for Docker
Article: Activating a Conda environment in your Dockerfile
Article: Shrink your Conda Docker images with conda-pack
Article: What’s running in production? Making your Docker images identifiable
Article: Your Docker build needs a smoke test
Article: Docker BuildKit: faster builds, new features, and now it’s stable
Article: Docker vs. Singularity for data processing: UIDs and filesystem access
Article: Where’s that log file? Debugging failed Docker builds
Article: An Introduction to Kubernetes for Data Scientists
Article: How to Use Kubernetes Pods for Machine Learning
Article: Kubernetes Jobs for Machine Learning
Article: Kubernetes CronJobs for Machine Learning
Article: Kubernetes Deployments for Machine Learning
Article: Kubernetes Services for Machine Learning
“Let’s use Kubernetes!” Now you have 8 problems
Article: Kubernetes for Python Developers: Part 1
Doc: Environment variables in Compose
Udacity: Scalable Microservices with Kubernetes
Cloud Computing
Concept Resource Done
Theory Datacamp: Cloud Computing for Everyone
Pluralsight: AWS Developer: The Big Picture
Pluralsight: AWS Networking Deep Dive: Virtual Private Cloud (VPC)
Pluralsight: AWS VPC Operations
Pluralsight: Building Applications Using Elastic Beanstalk
Udemy: AWS Concepts
Udemy: AWS Certified Developer - Associate 2018
Whitepaper: Architecting for the Cloud AWS Best Practices
Whitepaper: AWS Well-Architected Framework
Whitepaper: AWS Security Best Practices
Whitepaper: Blue/Green Deployments on AWS
Whitepaper: Microservices on AWS
Whitepaper: Optimizing Enterprise Economics with Serverless Architectures
Whitepaper: Practicing Continuous Integration and Continuous Delivery on AWS
Whitepaper: Running Containerized Microservices on AWS
Udemy: Serverless Concepts
Whitepaper: Serverless Architectures with AWS Lambda
Youtube: Deploying a machine learning model to the cloud using AWS Lambda
AWS: Amazon Transcribe Deep Dive: Using Feedback Loops to Improve Confidence Level of Transcription
AWS: Build a Text Classification Model with AWS Glue and Amazon SageMaker
AWS: Deep Dive on Amazon Rekognition: Building Computer Visions Based Smart Applications
AWS: Hands-on Rekognition: Automated Video Editing
AWS: Introduction to Amazon Comprehend
AWS: Introduction to Amazon Comprehend Medical
AWS: Introduction to Amazon Elastic Inference
AWS: Introduction to Amazon Forecast
AWS: Introduction to Amazon Lex
AWS: Introduction to Amazon Personalize
AWS: Introduction to Amazon Polly
AWS: Introduction to Amazon SageMaker Ground Truth
AWS: Introduction to Amazon SageMaker Neo
AWS: Introduction to Amazon Transcribe
AWS: Introduction to Amazon Translate
AWS: Introduction to AWS Marketplace - Machine Learning Category
AWS: Machine Learning Exam Basics
AWS: Neural Machine Translation with Sockeye
AWS: Process Model: CRISP-DM on the AWS Stack
AWS: Satellite Image Classification in SageMaker
Datacamp: Introduction to AWS Boto in Python
edX: Amazon SageMaker: Simplifying Machine Learning Application Development
Natural Language Processing
Concept Resource Done
Fundamentals Stanford CS224U: Natural Language Understanding | Spring 2019
Stanford CS224N: Stanford CS224N: NLP with Deep Learning | Winter 2019
Natural Language Processing with Transformers Book
Preprocessing Article: Fixing common Unicode mistakes with Python – after they’ve been made
Datacamp: Feature Engineering for NLP in Python
Datacamp: Natural Language Processing Fundamentals in Python
Datacamp: Regular Expressions in Python
Tokenization Article: 3 subword algorithms help to improve your NLP model performance
Keyword Extraction Article: Build A Keyword Extraction API with Spacy, Flask, and FuzzyWuzzy
Article: Unsupervised Auto-labeling of Websites
Article: Keyword Extraction with BERT
Article: Topic Modeling for Keyword Extraction
Simple Unsupervised Keyphrase Extraction using Sentence Embeddings | Research Paper Walkthrough 0:21:23
Embeddings Article: The Illustrated Word2vec
Article: Intuition & Use-Cases of Embeddings in NLP & beyond
Article: Learning Word Embedding
Article: On word embeddings - Part 1
Article: On word embeddings - Part 2: Approximating the Softmax
Article: On word embeddings - Part 3: The secret ingredients of word2vec
Article: Word embeddings in 2017: Trends and future direction
Rasa Algorithm Whiteboard - Embeddings 1: Just Letters 0:13:48
Rasa Algorithm Whiteboard - Embeddings 2: CBOW and Skip Gram 0:19:24
Rasa Algorithm Whiteboard - Embeddings 3: GloVe 0:19:12
Rasa Algorithm Whiteboard - Embeddings 4: Whatlies 0:14:03
Rasa Algorithm Whiteboard - StarSpace 0:11:46
Rasa Algorithm Whiteboard - Countvectors 0:13:32
Rasa Algorithm Whiteboard - Subword Embeddings 0:11:58
Rasa Algorithm Whiteboard - Implementation of Subword Embeddings 0:10:01
Rasa Algorithm Whiteboard - BytePair Embeddings 0:12:44
Vector 1 Word Meaning 0:09:09
Vector 2 Vector Semantics 0:06:37
Vector 3 Words and Vectors 0:05:16
Vector 4 Cosine Similarity 0:04:23
Vector 5 TF IDF 0:05:32
Vector 6 Word2vec 0:07:39
Vector 7 Learning in Word2vec 0:07:36
Vector 8 Properties of Embeddings 0:06:08
Youtube: Applied ML 2020 - 17 - Word vectors and document embeddings 1:03:04
RNN Article: Long Short-Term Memory: From Zero to Hero with PyTorch
Datacamp: RNN for Language Modeling
Article: Introduction to recurrent neural networks
Article: Explaining RNNs without neural networks
Article: Understanding LSTM Networks
Article: The Unreasonable Effectiveness of Recurrent Neural Networks
Article: Under the Hood of RNNs
Article: Exploring LSTMs
Article: Making sense of LSTMs by example
Article: Building RNNs is Fun with PyTorch and Google Colab
CMU Neural Nets for NLP 2021 (5): Recurrent Neural Networks 0:38:50
CMU Advanced NLP 2021 (5): Recurrent Neural Networks 1:13:43
Notebook: TextRNN - Predict Next Step
Notebook: TextLSTM - Autocomplete
Youtube: RNN and LSTM 26:13
Article: Understanding building blocks of ULMFIT
Article: Character level language model RNN
Text CNN Article: Understanding Convolutional Neural Networks for NLP
Transformers Article: Attention? An Other Perspective!: Part 1
UMass CS685 (Advanced NLP): Implementing a Transformer 1:12:36
Article: Attention and Memory in Deep Learning and NLP
Article: Attention? An Other Perspective!: Part 2
Article: Attention? An Other Perspective!: Part 3
Article: Attention? An Other Perspective!: Part 4
Article: Attention? An Other Perspective!: Part 5
Rasa Algorithm Whiteboard - Attention 1: Self Attention 0:14:32
Rasa Algorithm Whiteboard - Attention 2: Keys, Values, Queries 0:12:26
Rasa Algorithm Whiteboard - Attention 3: Multi Head Attention 0:10:55
Rasa Algorithm Whiteboard: Attention 4 - Transformers 0:14:34
Youtube: A brief history of the Transformer architecture in NLP
Youtube: The Transformer neural network architecture explained. “Attention is all you need” (NLP)
Youtube: How does a Transformer architecture combine Vision and Language? ViLBERT - NLP meets Computer Vision
Youtube: Strategies for pre-training the BERT-based Transformer architecture – language (and vision)
Article: The Illustrated Transformer
Article: The Annotated Transformer
Article: A Deep Dive into the Reformer
Article: A Survey of Long-Term Context in Transformers
Article: Why Rasa uses Sparse Layers in Transformers
Article: Transformer-based Encoder-Decoder Models
Article: Understanding BigBird's Block Sparse Attention"
Article: The Transformer Family
Article: Hugging Face Reads - 01/2021 - Sparsity and Pruning
Article: Hugging Face Reads, Feb. 2021 - Long-range Transformers
Article: The Transformer Explained
LSBert: A Simple Framework for Lexical Simplification | Research Paper Walkthrough 0:20:27
SpanBERT: Improving Pre-training by Representing and Predicting Spans | Research Paper Walkthrough 0:14:21
T5: Exploring Limits of Transfer Learning with Text-to-Text Transformer (Research Paper Walkthrough) 0:12:47
Hierarchical Transformers for Long Document Classification (Research Paper Walkthrough) 0:12:46
UMass CS685 (Advanced NLP): Attention mechanisms 0:48:53
UMass CS685 (Advanced NLP): Better BERTs 0:52:23
UMass CS685 (Advanced NLP): Retrieval-augmented language models 0:52:13
UMass CS685 (Advanced NLP): Model distillation and security threats 1:09:25
UMass CS685 (Advanced NLP): vision + language 1:06:28
UMass CS685 (Advanced NLP): Intermediate fine-tuning 1:10:35
UMass CS685 (Advanced NLP): probe tasks 0:54:30
UMass CS685 (Advanced NLP): semantic parsing 0:48:49
UMass CS685 (Advanced NLP): commonsense reasoning (guest lecture by Lorraine Li) 0:58:53
BERT Article: Deconstructing BERT
Youtube: BERT Research Series
Article: Maximizing BERT model performance
Article: The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)
Article: Smart Batching Tutorial - Speed Up BERT Training
Article: A review of BERT based models
Article: Understanding BERT’s Semantic Interpretations
Article: Examining BERT’s raw embeddings
Semantic Search Article: Semantic Search On Documents
7 1 Introduction to Information Retrieval 9 16 0:09:16
7 2 Term Document Incidence Matrices 8 59 0:08:59
7 3 The Inverted Index 10 42 0:10:43
7 4 Query Processing with the Inverted Index 6 43 0:06:44
7 5 The Boolean Retrieval Model 14 06 0:14:07
7 6 Phrase Queries and Positional Indexes 19 45 0:19:46
8 1 Introducing Ranked Retrieval 4 27 0:04:27
8 2 Scoring with the Jaccard Coefficient 5 06 0:05:07
8 3 Term Frequency Weighting 5 59 0:06:00
8 4 Inverse Document Frequency Weighting 10 16 0:10:17
8 5 TF IDF Weighting 3 42 0:03:42
8 6 The Vector Space Model 16 22 0:16:23
8 7 Calculating TF IDF Cosine Scores 12 47 0:12:48
8 8 Evaluating Search Engines 9 02 0:09:03
Article: Locality-sensitive Hashing and Singular to Plural Noun Conversion
Article: Haystack: The State of Search in 2021
Article: Search (Pt 1) — A Gentle Introduction
Article: Search (Pt 2) — A Semantic Horse Race
Article: Search (Pt 3) — Elastic Transformers
Article: Semantic search using BERT embeddings
Article: What Semantic Search Can do for You
Article: How To Create Natural Language Semantic Search For Arbitrary Objects With Deep Learning
Article: Building a sentence embedding index with fastText and BM25
Article: String Matching with BERT, TF-IDF, and more!
Article: Document search with fragment embeddings
Article: Text Similarities : Estimate the degree of similarity between two texts
Article: How we used Universal Sentence Encoder and FAISS to make our search 10x smarter
Article: Using embeddings to help find similar restaurants in Search
Article: Evolution of and experiments with feed ranking at Swiggy
Article: Personalizing Swiggy POP Recommendations
Article: Fan(s)tastic: Search for blazing-fast results
Article: Find My Food: Semantic Embeddings for Food Search Using Siamese Networks
Article: Learning To Rank Restaurants
Article: Comparison Of Ngram Fuzzy Matching Approaches
Article: String similarity — the basic know your algorithms guide!
Youtube: Billion-scale Approximate Nearest Neighbor Search
Youtube: Data Science - Fuzzy Record Matching
Youtube: Minimum Edit Distance Dynamic Programming
Youtube: Cheuk Ting Ho - Fuzzy Matching Smart Way of Finding Similar Names Using Fuzzywuzzy
Youtube: What's in a Name? Fast Fuzzy String Matching - Seth Verrinder & Kyle Putnam - 2015
Youtube: Jiaqi Liu Fuzzy Search Algorithms How and When to Use Them PyCon 2017
Youtube: 1 + 1 = 1 or Record Deduplication with Python | Flávio Juvenal @ PyBay2018
Youtube: Mike Mull: The Art and Science of Data Matching
Youtube: Record linkage: Join for real life by Rhydwyn Mcguire
Youtube: Approximate nearest neighbors and vector models, introduction to Annoy
Advanced Information Retrieval 2021 - 2021 Course Introduction 0:21:39
Advanced Information Retrieval 2021: Crash Course IR - Fundamentals 0:46:31
Advanced Information Retrieval 2021: Crash Course IR - Evaluation 0:37:15
Advanced Information Retrieval 2021: Crash Course IR - Test Collections 0:51:12
Advanced Information Retrieval 2021: Word Representation Learning 0:42:02
Advanced Information Retrieval 2021: Sequence Modelling with CNNs and RNNs 0:55:04
Advanced Information Retrieval 2021: Transformer and BERT Pre-training 0:47:15
Advanced Information Retrieval 2021: Introduction to Neural Re-Ranking 0:59:20
Advanced Information Retrieval 2021: Transformer Contextualized Re-Ranking 0:49:06
Advanced Information Retrieval 2021: Domain Specific Applications 0:38:32
Advanced Information Retrieval 2021: Dense Retrieval ❤ Knowledge Distillation 0:59:28
Introduction to Dense Text Representations - Part 1 0:12:56
Introduction to Dense Text Representations - Part 2 0:23:13
Introduction to Dense Text Representation - Part 3 0:38:07
Training State-of-the-Art Sentence Embedding Models 0:43:43
NER Article: Unsupervised NER using BERT
Introduction to Named Entity Tagging 0:05:06
Introduction to Part of Speech Tagging 0:09:03
Article: Zero shot NER using RoBERTA
Article: Existing Tools for Named Entity Recognition
Article: Solving NER with BERT for any entity type with very little training data (compared to past approaches)
Article: What is Hidden in the Hidden Markov Model?
Article: Part of Speech Tagging with Hidden Markov Chain Models
Article: Named-Entity evaluation metrics based on entity-level
Summarization Article: Automatically Summarize Trump’s State of the Union Address
Leveraging BERT for Extractive Text Summarization on Lectures | Research Paper Walkthrough 0:20:10
A Supervised Approach to Extractive Summarisation of Scientific Papers | Research Paper Walkthrough 0:19:01
Text Summarization of COVID-19 Medical Articles using BERT and GPT-2 | Research Paper Walkthrough 0:21:52
Extractive & Abstractive Summarization with Transformer Language Models | Research Paper Walkthrough 0:16:58
Unsupervised Multi-Document Summarization using Neural Document Model | Research Paper Walkthrough 0:15:11
SummPip: Multi-Document Summarization with Sentence Graph Compression | Research Paper Walkthrough 0:16:54
PEGASUS: Pre-training with Gap-Sentences for Abstractive Summarization | Research Paper Walkthrough 0:15:04
On Generating Extended Summaries of Long Documents (Research Paper Walkthrough) 0:14:24
Multilingual NLP Article: How to Apply BERT to Arabic and Other Languages
Article: A survey of cross-lingual word embedding models
Article: Unsupervised Cross-lingual Representation Learning
Article: DaCy: New Fast and Efficient State-of-the-Art in Danish NLP!
Article: Why You Should Do NLP Beyond English
CMU: Low-resource NLP Bootcamp 2020
CMU Multilingual NLP 2020
Domain Adaptation Article: Domain-Specific BERT Models
Text Generation Article: Text Generation
UMass CS685 (Advanced NLP): Text generation decoding and evaluation 1:02:32
UMass CS685 (Advanced NLP): Paraphrase generation 1:10:59
Article: The Illustrated GPT-2 (Visualizing Transformer Language Models)
Article: Controlling Text Generation with Plug and Play Language Models
Article: Poor man’s GPT-3: Few shot text generation with T5 Transformer
Article: Reducing Toxicity in Language Models
Article: How to Implement a Beam Search Decoder for Natural Language Processing
Article: Perplexity Intuition (and its derivation)
Article: The Annotated GPT-2
Datacamp: Natural Language Generation in Python
BLEURT: Learning Robust Metrics for Text Generation | Research Paper Walkthrough 0:13:38
Evaluation of Text Generation: A Survey | Human-Centric Evaluations | Research Paper Walkthrough 0:15:54
Nucleus Sampling: The Curious Case of Neural Text Degeneration (Research Paper Walkthrough) 0:12:48
Spelling Correction Article: Rebuilding the most popular spellchecker. Part 1
Article: Rebuilding the spellchecker, pt.2: Just look in the dictionary, they said!
Article: Rebuilding the spellchecker, pt.3: Lookup—compounds and solutions
Article: Rebuilding the spellchecker, pt.4: Introduction to suggest algorithm
Article: Rebuilding the spellchecker: Hunspell and the order of edits
Article: How to Use n-gram Models to Detect Format Errors in Datasets
Article: Spelling Correction: How to make an accurate and fast corrector
Article: Speller100: Zero-shot spelling correction at scale for 100-plus languages
Article: Breaking the spell of the spelling check
Article: How to Write a Spelling Corrector
Article: Spellchecking by computer
Article: A Spellchecker Used to Be a Major Feat of Software Engineering
Article: 1000x Faster Spelling Correction algorithm (2012)
Youtube: How to build a custom spell checker using python NLP
Topic Modeling Article: Automatic Topic Labeling in 2018: History and Trends
Youtube: Applied ML 2020 - 16 - Topic models for text data 1:18:34
Youtube: Extracting topics from reviews using NLP - Dr. Tal Perri
Article: Interactive Topic Modeling with BERTopic
Article: Understanding Climate Change Domains through Topic Modeling
Article: When Topic Modeling is Part of the Text Pre-processing
Article: pyLDAvis: Topic Modelling Exploration Tool That Every NLP Data Scientist Should Know
Article: Topic Modeling with BERT
Zero-Shot NLP Article: Zero-Shot Learning in Modern NLP
Article: Improved Few-Shot Text classification
Article: Text classification from few training examples
Article: Pattern-Exploiting Training
Paraphrasing Article: Paraphrasing
Conversational AI Article: What makes a good conversation?
Datacamp: Building Chatbots in Python
Rasa Algorithm Whiteboard - Diet Architecture 1: How it Works 0:23:27
Rasa Algorithm Whiteboard - Diet Architecture 2: Design Decisions 0:15:06
Rasa Algorithm Whiteboard - Diet Architecture 3: Benchmarking 0:22:34
Rasa Algorithm Whiteboard - TED Policy 0:16:10
Rasa Algorithm Whiteboard - TED in Practice 0:14:54
Rasa Algorithm Whiteboard - Response Selection 0:12:07
Rasa Algorithm Whiteboard - Response Selection: Implementation 0:09:25
Dialog 1 Overview 0:03:11
Dialogue 2 Human Conversation 0:10:31
Dialogue 3 ELIZA 0:09:27
Dialogue 4 Corpus Chatbots 0:09:35
Dialogue 5 Frame Based Dialogue 0:07:41
Dialogue 6 Dialogue State Architecture 0:08:58
Dialogue 7 Dialogue State Architecture Policy and Generation 0:08:23
Dialogue 8 Evaluation 0:04:38
Dialogue 9 Design and Ethical Issues 0:03:29
YouTube: Level 3 AI Assistant Conference 2020
Youtube: Conversational AI with Transformers and Rule-Based Systems 1:53:24
TOD-BERT: Pre-trained Transformers for Task-Oriented Dialogue Systems (Research Paper Walkthrough) 0:15:25
DialoGPT: Generative Training for Conversational Response Generation (Research Paper Walkthrough) 0:13:17
Youtube: Transformers 🤗 to Rule Them All? Under the Hood of the AI Recruiter Chatbot 🤖, with Keisuke Inoue
Youtube: Chatbots Revisted | by Abhishek Thakur | Kaggle Days Warsaw
Sentiment Analysis Article: NLP: Pre-trained Sentiment Analysis
Article: Key topics extraction and contextual sentiment of users reviews
Article: Aspect-Based Opinion Mining (NLP with Python)
Datacamp: Sentiment Analysis in Python
Youtube: Sentiment Analysis: Key Milestones, Challenges and New Directions
Talk: EmoTag1200: Understanding the Association between Emojis and Emotions
Youtube: Real life aspects of opinion sentiment analysis within customer reviews - Dr. Jonathan Yaniv
Youtube: Deep Learning Methods for Emotion Detection from Text - Dr. Liron Allerhand
Text Classification Article: Multi-Label Text Classification
Text Clustering Article: Document clustering
Datacamp: Clustering Methods with SciPy
Article: Gaussian Mixture Models for Clustering
Explainability Article: Explain NLP models with LIME & SHAP
Youtube: Explainability for Natural Language Processing
Usecases Article: How to solve 90% of NLP problems: a step-by-step guide
Article: Using an NLP Q&A System To Study Climate Hazards and Nature-Based Solutions
Article: How To Do Things With Words. And Counters
Talk: Practical NLP for the Real World
Youtube: Design Considerations for building ML-Powered Search Applications - Mark Moyou
Youtube: Analyze Customer Feedback in Minutes, Not Months
Youtube: NLP in Feedback Analysis - Yue Ning
Youtube: Productionizing an unsupervised machine learning model to understand customer feedback
Youtube: Bringing innovation to online retail: automating customer service with NLP
Youtube: Transform customer service with machine learning (Google Cloud Next '17)
Youtube: Artificial Intelligence and Natural Language Processing in E-Commerce by Katherine Munro | smec
Youtube: The giant leaps in language technology -- and who's left behind | Kalika Bali
Machine Translation Article: Introducing Translatotron: An End-to-End Speech-to-Speech Translation Model
Datacamp: Machine Translation in Python
Libraries Datacamp: Advanced NLP with spaCy
Spacy Tutorial
Youtube: spaCy v3.0: Bringing State-of-the-art NLP from Prototype to Production 00:22:40
Youtube: SpaCy for Digital Humanities with Python Tutorials
TextBlob Tutorial Series
YouTube: Intro to NLP with Spacy
Huggingface How-to Use HuggingFace's Datasets - Transformers From Scratch #1 0:14:21
Build a Custom Transformer Tokenizer - Transformers From Scratch #2 0:14:17
Building MLM Training Input Pipeline - Transformers From Scratch #3 0:23:11
Training and Testing an Italian BERT - Transformers From Scratch #4 0:30:38
Audio Datacamp: Spoken Language Processing in Python
Youtube: Librosa Audio and Music Signal Analysis in Python | SciPy 2015 | Brian McFee
Youtube: Deep Learning (for Audio) with Python
Gibberish Detection Youtube: Gibberish Detector
Constituency Parsing Youtube: NLP Lecture 7 Constituency Parsing
Youtube: LING 83 Teaching Video: Constituency Parsing
Question Answering UMass CS685 (Advanced NLP): Question answering 0:59:50
Data Annotation UMass CS685 (Advanced NLP): Crowdsourced text data collection 0:58:31
Ethics UMass CS685 (Advanced NLP): ethics in NLP 0:56:57
Related Awesome Lists
Top Programming Languages

Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Python (893,969
Learning (75,671
Machine Learning (41,050
Deep Learning (39,410
Artificial Intelligence (20,227
Book (20,203
Article (14,318
Udacity (5,006
Seo (3,403
Learning Resources (413
Datacamp (58
Codecademy (42