Awesome Open Source
Awesome Open Source

Complete-Life-Cycle-of-a-Data-Science-Project

CREDITS:All corresponding resources

MOTIVATION:Motivation to create this repository to help upcoming aspirants and help to others in the data science field

https://www.theinsaneapp.com/2021/03/how-to-build-machine-learning-project.html

**** If you like my work. please buy me a coffee it motivate me -> https://www.buymeacoffee.com/achuthasubhash?new=1 ****

Business understanding

1.Data collection

Data consists of 3 kinds

a.Structure data (tabular data,etc...)

b.Unstructured data (images,text,audio,etc...)

c.semi structured data (XML,JSON,etc...)

variable

a.qualitative (nominal,ordinal,binary)

b.quantitative(discrete,continuous)

https://www.chi2innovations.com/blog/discover-data-blog-series/data-types-101/

cleanlab https://l7.curtisnorthcutt.com/cleanlab-python-package https://awesomeopensource.com/project/cgnorthcutt/cleanlab https://awesomeopensource.com/project/cgnorthcutt/label-errors https://awesomeopensource.com/project/cgnorthcutt/rankpruning https://awesomeopensource.com/project/subeeshvasu/Awesome-Learning-with-Label-Noise

Measure Data Quality ydata-quality https://awesomeopensource.com/project/ydataai/ydata-synthetic https://towardsdatascience.com/how-can-i-measure-data-quality-9d31acfeb969

a.Web scraping best article to refer-https://towardsdatascience.com/choose-the-best-python-web-scraping-library-for-your-application-91a68bc81c4f

https://www.analyticsvidhya.com/blog/2019/10/web-scraping-hands-on-introduction-python/?utm_source=linkedin&utm_medium=KJ|link|weekend-blogs|blogs|44087|0.875

https://www.analyticsvidhya.com/blog/2019/10/web-scraping-hands-on-introduction-python/?utm_source=linkedin&utm_medium=AV|link|high-performance-blog|blogs|44204|0.375

https://www.kdnuggets.com/2021/02/6-web-scraping-tools.html

https://www.bigdatanews.datasciencecentral.com/profiles/blogs/top-30-free-web-scraping-software

https://towardsdatascience.com/6-web-scraping-tools-that-make-collecting-data-a-breeze-457c44e4411d

https://medium.com/analytics-vidhya/master-web-scraping-completly-from-zero-to-hero-38051423256b

1.Beautifulsoup  https://www.freecodecamp.org/news/how-to-scrape-websites-with-python-and-beautifulsoup-5946935d93fe/

  mechanicalsoup   https://analyticsindiamag.com/mechanicalsoup-web-scraping-custom-dataset-tutorial/

2.Scrapy,PyScrappy,Pandas Datareader,Instaloader

3.Selenium     https://www.freecodecamp.org/news/better-web-scraping-in-python-with-selenium-beautiful-soup-and-pandas-d6390592e251/

4.Request to access data 

5.AUTOSCRAPER - https://github.com/alirezamika/autoscraper https://www.youtube.com/watch?v=9BQ353Yu1D0 https://www.analyticsvidhya.com/blog/2021/04/automate-web-scraping-using-python-autoscraper-library/

Scrap Images From E-Commerce Website Using AutoScraper https://www.analyticsvidhya.com/blog/2021/05/scrap-images-from-e-commerce-website-using-autoscraper-library/

FiftyOne Now easier to download and evaluate  https://towardsdatascience.com/googles-open-images-now-easier-to-download-and-evaluate-with-fiftyone-615ce0482c02

webbot https://pypi.org/project/webbot/

gazpacho https://github.com/maxhumber/gazpacho

html_scraper_streamlit_app https://www.youtube.com/watch?v=6U5xJ3mXRKA&feature=youtu.be

6.Twitter scraping tool (๐š๐š ๐š’๐š—๐š or tweepy)-https://github.com/twintproject/twint

  twitterscraper https://www.youtube.com/watch?v=MpIi4HtCiVk
  
  snscrape  extract twitterr data  https://github.com/JustAnotherArchivist/snscrape
  
  Scweet A simple and unlimited twitter scraper  https://github.com/Altimis/Scweet
  
  GetOldTweets3,GoogleNews

  Scrape Twitter for Tweets https://github.com/taspinar/twitterscraper
  
  HAR File Web Scraper https://stevesie.com/har-file-web-scraper https://www.youtube.com/watch?v=LcqVDfueb8g

  https://analyticsindiamag.com/complete-tutorial-on-twint-twitter-scraping-without-twitters-api/
  
  https://developer.twitter.com/en/docs
  
  pytrends  https://medium.com/nerd-for-tech/scraping-data-from-online-platforms-to-enhance-time-series-forecasts-6eec3c68636d
  
  Scraping Instagram -instaloader  https://thecleverprogrammer.com/2020/07/30/scraping-instagram-with-python/
  
  Instascrape   
  
  Scraping Reddit using Python Reddit API Wrapper  (PRAW)
  
  Scrape Wikipedia  wikipedia https://www.thepythoncode.com/article/access-wikipedia-python
  
  patang - Scrape Product details from eCommerce Sites with Puppeteer and DOM String  https://www.youtube.com/watch?v=3sgxRmyOuXs
  
  Download Wikipedia https://www.wikidata.org/wiki/Wikidata:Main_Page https://www.youtube.com/watch?v=hC1rY4lRY0s https://towardsdatascience.com/an-efficient-way-to-read-data-from-the-web-directly-into-python-a526a0b4f4cb
  
  Web Scraping to Create a CSV File  https://thecleverprogrammer.com/2020/08/08/web-scraping-to-create-csv/
  
  Amazon Web Scraper, Amazon Auto Scraper

7.urllib

8.pattern

9.Octoparse Easy Web Scraping   https://www.octoparse.com/ 

 ParseHub https://www.parsehub.com/  https://analyticsindiamag.com/parsehub-no-code-gui-based-web-scraping-tool/
 
 Apify https://apify.com/
 
 obsei https://github.com/lalitpagaria/obsei
 
 Diffbot  https://analyticsindiamag.com/diffbot/
 
 Trustpilot,webhose,scrapingbot
 
 lxml  https://lxml.de/index.html#introduction
 
 ScrapingBee  https://analyticsindiamag.com/scrapingbee-api/
 
 Scrape HTML tables https://www.youtube.com/watch?v=6U5xJ3mXRKA&feature=youtu.be  or pd.read_html
 
 requests-html https://github.com/kennethreitz/requests-html
 
 newspaper https://github.com/codelucas/newspaper  https://www.youtube.com/watch?v=Hfry5XnISyc
 
 newspaper3k: https://newspaper.readthedocs.io  # easily extract text from articles
 
 newscatcher https://github.com/kotartemiy/newscatcher https://www.youtube.com/watch?v=pHzOuizZq4I
 
 patang (extract product details) https://github.com/tejazz/patang
 
 lisc https://github.com/lisc-tools/lisc
 
 pandas(read_html)
 
 wget,curl,parsehub,webhouse,octoparse,scraping bot,scraping bee,Common,Content Grabber,Docparser,Scraper API,Import.io,Altair Monarch,WebAutomation.io,WebScraper.io 

 Crawl Crawly  https://crawly.diffbot.com/   

 HTML basics for web scraping,Web Scraping with Octoparse,Web Scraping with Selenium

 10-best-web-scraping-tools  https://www.scraperapi.com/blog/the-10-best-web-scraping-tools/
 
 https://www.kdnuggets.com/2021/02/6-web-scraping-tools.html
  
 https://analyticsindiamag.com/complete-learning-path-to-web-scraping-with-all-major-tools/ https://towardsdatascience.com/6-web-scraping-tools-that-make-collecting-data-a-breeze-457c44e4411d
 
 https://towardsdatascience.com/6-web-scraping-tools-that-make-collecting-data-a-breeze-457c44e4411d https://www.kdnuggets.com/2018/02/web-scraping-tutorial-python.html
 
 https://www.octoparse.com/ https://github.com/tirthajyoti/pydbgen https://www.mozenda.com/ https://www.mockaroo.com/ https://lionbridge.ai/ https://www.mturk.com/ https://appen.com/
 
 11.GoogleImageCrawler,google_images_download,bing_image

b.Web Crawling

https://python.libhunt.com/scrapy-alternatives

Flat Data https://octo.github.com/projects/flat-data

b.3rd party API'S

22 APIs every data scientist should learn https://www.springboard.com/library/data-science/top-apis-for-data-scientists/

c.creating own data (manual collection eg:google docx,servey,etc...) primary data

d.etl awesome ETL https://awesomeopensource.com/project/pawl/awesome-etl#python https://awesomeopensource.com/project/achuthasubhash/awesome-etl

38x faster data pipelines with tf.data

d.Databases

Databases are 2 kind sequel and no sequel database

sql,sql lite,mysql,mongodb,montydb,hadoop,elastic search,cassendra,amazon s3,hive,googlebigtable,AWS DynamoDB,HBase,oracle db

sql https://mode.com/sql-tutorial/ https://www.w3schools.com/sql/

sql in python https://medium.com/jbennetcodes/how-to-rewrite-your-sql-queries-in-pandas-and-more-149d341fc53e

PyMongo https://analyticsindiamag.com/guide-to-pymongo-a-python-wrapper-for-mongodb/

Cloud AI Data labeling service https://cloud.google.com/ai-platform/data-labeling/docs?utm_source=youtube&utm_medium=Unpaidsocial&utm_campaign=guo-20200503-Data-Labeling

e.Online resources - ultimate resource https://datasetsearch.research.google.com/ https://medium.com/swlh/where-to-find-awesome-machine-learning-datasets-6bb909a3f350

10 BEST DATA COLLECTION TOOLS FOR EFFECTIVE RESULTS https://www.analyticsinsight.net/10-best-data-collection-tools-for-effective-results/

https://www.freecodecamp.org/news/https-medium-freecodecamp-org-best-free-open-data-sources-anyone-can-use-a65b514b0f2d/ https://research.google/tools/datasets/

Machine learning datasets https://www.datasetlist.com/ https://wiki.pathmind.com/open-datasets

https://guides.library.cmu.edu/az.php https://docs.microsoft.com/en-us/azure/azure-sql/public-data-sets https://registry.opendata.aws/ https://paperswithcode.com/datasets https://datasets.quantumstat.com/ https://www.quandl.com/ http://dataportals.org/ https://opendatamonitor.eu/frontend/web/index.php?r=dashboard%2Findex https://en.wikipedia.org/wiki/List_of_datasets_for_machine-learning_research https://www.quora.com/Where-can-I-find-large-datasets-open-to-the-public https://www.reddit.com/r/datasets/ https://ourworldindata.org/ https://data.worldbank.org/ https://data.world/ https://data.census.gov/cedsci/ https://data.seattle.gov/

Worldโ€™s Largest Data Platform https://worlddata.ai/

Awesome list of datasets in 100+ categories https://www.kdnuggets.com/2021/05/awesome-list-datasets.html

https://sebastianraschka.com/blog/2021/ml-dl-datasets.html  https://enoumen.com/2021/04/23/data-sciences-datasets-data-visualization-data-analytics-big-data-data-lakes/

https://serokell.io/blog/best-machine-learning-datasets https://medium.com/@ODSC/25-excellent-machine-learning-open-datasets-940ca2124dfc  

1)kaggle-https://www.kaggle.com/datasets , ๐š™๐š’๐š™ ๐š’๐š—๐šœ๐š๐šŠ๐š•๐š• ๐š”๐šŠ๐š๐š๐š•๐šŽ๐š๐šŠ๐š๐šŠ๐šœ๐šŽ๐š๐šœ

Downloading Kaggle datasets directly into Google Colab -https://towardsdatascience.com/downloading-kaggle-datasets-directly-into-google-colab-c8f0f407d73a

How to Download Kaggle Datasets using Jupyter Notebook https://www.analyticsvidhya.com/blog/2021/04/how-to-download-kaggle-datasets-using-jupyter-notebook/

2)https://sebastianraschka.com/blog/2021/ml-dl-datasets.html

movielens-https://grouplens.org/datasets/movielens/latest/

3)data.gov-https://data.gov.in/

4)uci-https://archive.ics.uci.edu/ml/datasets.php     https://github.com/tirthajyoti/UCI-ML-API

5)Group Lens dataset https://grouplens.org/

Wikipedia ML Datasets https://en.wikipedia.org/wiki/List_of_datasets_for_machine-learning_research

6)world3bank  https://data.world/ , worldbank

7)Google Cloud BigQuery public datasets

  Google Public Datasets-cloud.google.com/bigquery/public-data/
  
  Google Cloud Data Catalog  https://cloud.google.com/data-catalog
  
  Academic Torrents-https://academictorrents.com/check.htm?returnto=%2Fbrowse.php

8)online hacktons

 Datasets  https://www.paperswithcode.com/datasets

9)image data from google_images_download

https://www.visualdata.io/discovery

http://xviewdataset.org/#dataset

https://ai.googleblog.com/2016/09/introducing-open-images-dataset.html

10)image data from Bing_Search

image data from simple_image_download  https://github.com/RiddlerQ/simple_image_download

11)https://www.columnfivemedia.com/100-best-free-data-sources-infographic

12)Reddit:https://lnkd.in/dv5UCD4       https://www.reddit.com/r/datasets/

13)https://datasets.bifrost.ai/?ref=producthunt

14)data.world:https://lnkd.in/gEK897K

15)https://data.world/datasets/open-data

   https://tinyletter.com/data-is-plural

16)FiveThirtyEight :-  https://lnkd.in/gyh-HDj , https://data.fivethirtyeight.com/

17)BuzzFeed :- https://lnkd.in/gzPWyHj

   Buzzfeed News -github.com/BuzzFeedNews
   
   Socrata - https://opendata.socrata.com/

18)Google public datasets :- https://lnkd.in/g5dH8qE

Statistics Canada https://www.statcan.gc.ca/eng/start  https://towardsdatascience.com/how-to-collect-data-from-statistics-canada-using-python-db8a81ce6475

Deep Image Search AI-based image search engine https://github.com/TechyNilesh/DeepImageSearch

https://www.datasciencecentral.com/profiles/blogs/big-data-sets-available-for-free

19)Quandl :- https://www.quandl.com  stock data

   statista : https://www.statista.com/ stock data

20)socorateopendata :- https://lnkd.in/gea7JMz

21)AcedemicTorrents :- https://lnkd.in/g-Ur9Xy

22) Automates Image Annotation for Deep Learning Models https://medium.com/towards-artificial-intelligence/improving-data-labeling-efficiency-with-auto-labeling-uncertainty-estimates-and-active-learning-5848272365be

https://neptune.ai/blog/annotation-tool-comparison-deep-learning-data-annotation?utm_source=linkedin&utm_medium=post&utm_campaign=blog-annotation-tool-comparison-deep-learning-data-annotation

Diffgram,Label Studio ,CVAT,SuperAnnotate,Datasaur https://anthony-sarkis.medium.com/the-5-best-ai-data-annotation-platforms-for-machine-learning-2021-ec17c15142f3

https://foobar167.medium.com/open-source-free-software-for-image-segmentation-and-labeling-4b0332049878

***Label Assist: Model Assisted Pre-Annotation for Computer Vision https://blog.roboflow.com/announcing-label-assist/ https://www.youtube.com/watch?v=919CihTlkZw&feature=youtu.be***

https://github.com/jsbroks/awesome-dataset-tools

jupyter-innotater data annotator for Jupyter notebooks https://github.com/ideonate/jupyter-innotater

semi-auto-image-annotation-tool https://github.com/virajmavani/semi-auto-image-annotation-tool

labelimage:- https://github.com/wkentaro/labelme  ,  https://github.com/tzutalin/labelImg 

labelCloud  lightweight tool for labeling 3D bounding boxes in point clouds https://github.com/ch-sa/labelCloud

labeller https://www.labellerr.com/

prodigy Radically efficient machine teaching An annotation tool powered by active learning  https://prodi.gy/

Labelbox-https://labelbox.com/

Playment-https://playment.io/

SuperAnnotate -https://www.superannotate.com/

CVAT-https://github.com/openvinotoolkit/cvat

Lionbridge- https://lionbridge.ai/

LinkedAI: A No-code Data Annotations- https://analyticsindiamag.com/linkedai/

Dataturks

V7 Darwin The Rapid Image Annotator   https://docs.v7labs.com/docs/loading-a-dataset-in-python   https://github.com/v7labs/darwin-py#usage-as-a-python-library

https://waliamrinal.medium.com/top-and-easy-to-use-open-source-image-labelling-tools-for-machine-learning-projects-ffd9d5af4a20

https://github.com/heartexlabs/awesome-data-labeling  

Label a Dataset with a Few Lines of Code https://eric-landau.medium.com/label-a-dataset-with-a-few-lines-of-code-45c140ff119d

https://analyticsindiamag.com/complete-guide-to-data-labelling-tools/ https://neptune.ai/blog/data-labeling-software

Extraction of Objects In Images and Videos Using 5 Lines of Code https://towardsdatascience.com/extraction-of-objects-in-images-and-videos-using-5-lines-of-code-6a9e35677a31

https://neptune.ai/blog/data-labeling-software?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-data-labeling-software

23)tensorflow_datasets as tfds  https://www.tensorflow.org/datasets  (import tensorflow_datasets as tfds)

https://lionbridge.ai/datasets/tensorflow-datasets-machine-learning/

24)https://datasets.bifrost.ai/?ref=producthunt

25)https://ourworldindata.org/

26)https://data.worldbank.org/

27)google open images:https://storage.googleapis.com/openimages/web/download.html

30 Largest TensorFlow Datasets for Machine Learning  https://lionbridge.ai/datasets/tensorflow-datasets-machine-learning/

https://cloud.google.com/bigquery/public-data/   https://towardsdatascience.com/bigquery-public-datasets-936e1c50e6bc  

https://christopherzita.medium.com/how-to-download-google-images-using-python-2021-82e69c637d59

28)https://data.gov.in/

29)imagenet dataset-http://www.image-net.org/

30)https://parulpandey.com/2020/08/09/getting-datasets-for-data-analysis-tasks%e2%80%8a-%e2%80%8aadvanced-google-search/

31)https://storage.googleapis.com/openimages/web/index.html  , 

   https://storage.googleapis.com/openimages/web/visualizer/index.html?set=train&type=segmentation&r=false&c=%2Fm%2F09qck
   
   https://console.cloud.google.com/marketplace/browse?filter=solution-type:dataset&_ga=2.35328417.1459465882.1589693499-869920574.1589693499
   
   https://catalog.data.gov/dataset?groups=education2168#topic=education_navigation
   
   https://vincentarelbundock.github.io/Rdatasets/datasets.html
 
32)coco dataset https://cocodataset.org/#explore
 
33)huggingface datasets-https://github.com/huggingface/datasets  https://huggingface.co/datasets  https://huggingface.co/languages

pip install datasets

34)Big Bad NLP Database-https://datasets.quantumstat.com/

fast.ai Datasets https://course.fast.ai/datasets

https://github.com/niderhoff/nlp-datasets

600 NLP Datasets and Glory  https://pub.towardsai.net/600-nlp-datasets-and-glory-4b0080bf5ab

nlp-datasets https://github.com/karthikncode/nlp-datasets

https://analyticsindiamag.com/15-most-important-nlp-datasets/      https://medium.com/ai-in-plain-english/25-free-datasets-for-natural-language-processing-57e407402c60

35)https://www.edureka.co/blog/25-best-free-datasets-machine-learning/

36)bigquery public dataset ,Google Public Data Explorer

https://cloud.google.com/public-datasets        https://guides.library.cmu.edu/machine-learning/datasets

37)inbuilt library data eg:iris dataset,mnist dataset,etc...

pandas-datareader  https://github.com/pydata/pandas-datareader

tf.data.Datasets for TensorFlow Datasets 

38)https://data.gov.sg/    https://data.gov.au/   https://data.europa.eu/euodp/en/data   https://data.europa.eu/euodp/en/data    https://data.govt.nz/

data.gov.be ,data.egov.bg/ ,data.gov.cz/english ,portal.opendata.dk,govdata.de,opendata.riik.ee,data.gov.ie,data.gov.gr,datos.gob.es,data.gouv.fr,data.gov.hr

dati.gov.it,data.gov.cy,opendata.gov.lt,data.gov.lv,data.public.lu,data.gov.mt,data.overheid.nl,data.gv.at,danepubliczne.gov.pl,dados.gov.pt,data.gov.ro,podatki.gov.si

data.gov.sk,avoindata.fi,oppnadata.se,https://data.adb.org/ ,https://data.iadb.org/ ,https://www.weforum.org/agenda/2018/03/latin-america-smart-cities-big-data/

https://data.fivethirtyeight.com/ , https://wiki.dbpedia.org/ ,https://www.europeandataportal.eu/en ,https://data.europa.eu/ ,https://www.census.gov/,

https://www.who.int/data/gho ,https://data.unicef.org/open-data/ ,http://data.un.org/ ,https://data.oecd.org/ ,https://data.worldbank.org/  

39.Awesome Public Dataset- https://github.com/awesomedata/awesome-public-datasets

Get OpenMLโ€™s Dataset in One Line of Code https://mathdatasimplified.com/2021/04/23/fetch_openml-get-openmls-dataset-in-one-line-of-code/

https://github.com/the-pudding/data

datasets  https://github.com/benedekrozemberczki/datasets

kdnuggets  https://www.kdnuggets.com/datasets/index.html

Hub https://github.com/activeloopai/Hub

40.Datasets for Machine Learning on Graphs-https://ogb.stanford.edu/

41.https://www.johnsnowlabs.com/data/

42.30 largest tensorflow datasets-https://lionbridge.ai/datasets/tensorflow-datasets-machine-learning/

43. coco dataset-https://cocodataset.org/#home

flickr-downloader https://github.com/renatoviolin/flickr-downloader/

Synthetic Image Datasets https://analyticsindiamag.com/unity-launches-synthetic-image-datasets-to-train-ai-models-faster/

Synthetic structured data generators https://github.com/ydataai/ydata-synthetic

DatasetGAN: an automatic procedure to generate massive datasets of high-quality images 

Generating synthetic tabular data with GANs,Synthetic Time-Series Data by A GAN approach

Unity Launches Synthetic Image Datasets https://www.marktechpost.com/2021/04/23/unity-launches-synthetic-image-datasets-to-train-ai-and-computer-vision-models-faster/

Generate Your Own Dataset using GAN https://www.analyticsvidhya.com/blog/2021/04/generate-your-own-dataset-using-gan/

Google Open images-https://opensource.google/projects/open-images-dataset  https://storage.googleapis.com/openimages/web/index.html

50+ Object Detection Datasets-https://medium.com/towards-artificial-intelligence/50-object-detection-datasets-from-different-industry-domains-1a53342ae13d

70+ Image Classification Datasets from different Industry domains-https://medium.com/towards-artificial-intelligence/70-image-classification-datasets-from-different-industry-domains-part-2-cd1af6e48eda

VisualData Discovery https://www.visualdata.io/discovery   https://guides.library.cmu.edu/machine-learning/datasets
 
 data https://storage.googleapis.com/openimages/web/visualizer/index.html?c=%2Fm%2F04yqq2&r=false&set=train&type=segmentation&utm_campaign=Weekly%20Machine%20Learning%20news&utm_medium=email&utm_source=Revue%20newsletter
 
VisualData https://www.visualdata.io/discovery
   
bifrost-   https://datasets.bifrost.ai/

satellite images https://towardsdatascience.com/finding-satellite-images-for-your-data-science-project-888695361925

https://public.roboflow.com/

https://www.visualdata.io/discovery        http://www.image-net.org/      https://www.cs.toronto.edu/~kriz/cifar.html  
   
tensorflow_datasets.object_detection - https://storage.googleapis.com/openimages/web/index.html

https://github.com/google-research-datasets/Objectron/  https://ai.googleblog.com/2020/11/announcing-objectron-dataset.html?m=1 

http://idd.insaan.iiit.ac.in/   http://database.mmsp-kn.de/koniq-10k-database.html

https://ai.googleblog.com/2020/11/announcing-objectron-dataset.html

https://www.visualdata.io/discovery  https://blogs.bing.com/maps/2019-03/microsoft-releases-12-million-canadian-building-footprints-as-open-data

https://blogs.bing.com/maps/2019-09/microsoft-releases-18M-building-footprints-in-uganda-and-tanzania-to-enable-ai-assisted-mapping

https://datasets.bifrost.ai/     https://storage.googleapis.com/openimages/web/download.html  https://computervisiononline.com/datasets  http://yacvid.hayko.at/

https://www.cogitotech.com/use-cases/biodiversity/

ImageNet data -http://image-net.org/

ApolloScape Dataset-http://apolloscape.auto/

https://github.com/chrieke/awesome-satellite-imagery-datasets

44.https://github.com/fivethirtyeight/data

45.Recommender Systems Datasets-https://cseweb.ucsd.edu/~jmcauley/datasets.html

46.indiadataportal-https://indiadataportal.com/

47.US Government Open Dataset: https://www.data.gov/

https://censusreporter.org/   https://data.census.gov/cedsci/

48.AWS Public Data Sets:https://registry.opendata.aws/    https://aws.amazon.com/opendata/?wwps-cards.sort-by=item.additionalFields.sortDate&wwps-cards.sort-order=desc

49.https://the-eye.eu/public/AI/pile_preliminary_components/

  Reddit -https://www.reddit.com/r/datasets/  
  
  wikipedia-https://en.wikipedia.org/wiki/List_of_datasets_for_machine-learning_research
  
  http://opendata.cern.ch/  ,  https://www.imf.org/en/Data
  
  Global Health Observatory data repository-https://apps.who.int/gho/data/node.main
  
  CERN Open Data Portal-http://opendata.cern.ch/
  
  TensorFlow Datasets https://www.tensorflow.org/datasets
  
50.openblender- https://www.openblender.io/#/welcome

51.Top 10 Datasets For Cybersecurity Projects- https://analyticsindiamag.com/top-10-datasets-for-cybersecurity-projects/

52.Datasets from Web Crawl Data (nlp)-http://data.statmt.org/cc-100/

53.https://www.springboard.com/blog/free-public-data-sets-data-science-project/

54.NASA - https://nasa.github.io/data-nasa-gov-frontpage/ace 

55.Academic Torrents,GitHub Datasets,CERN Open Data Portal,Global Health Observatory Data Repository

56.32 Data Sets to Uplift your Skills in Data Science-https://blog.datasciencedojo.com/data-sets-data-science-skills/?utm_content=144243072&utm_medium=social&utm_source=linkedin&hss_channel=lcp-3740012

https://lionbridge.ai/datasets/the-50-best-free-datasets-for-machine-learning/

57.OpenDaL-https://opendatalibrary.com/

Data Is Plural-https://docs.google.com/spreadsheets/d/1wZhPLMCHKJvwOkP4juclhjFgqIY8fQFMemwKL2c64vk/edit#gid=0

VisualData-https://www.visualdata.io/discovery

https://medium.com/towards-artificial-intelligence/best-datasets-for-machine-learning-data-science-computer-vision-nlp-ai-c9541058cf4f
 
58.Pandas Data Reader-https://pandas-datareader.readthedocs.io/en/latest/remote_data.html

59.ieee-dataport-https://ieee-dataport.org/datasets

https://medium.com/towards-artificial-intelligence/best-datasets-for-machine-learning-data-science-computer-vision-nlp-ai-c9541058cf4f

https://github.com/neomatrix369/awesome-ai-ml-dl/blob/master/data/datasets.md#datasets-and-sources-of-raw-data

60.Generating Realistic Fake Data https://towardsdatascience.com/free-resources-for-generating-realistic-fake-data-da63836be1a8

Faker is a Python package that generates fake data-https://github.com/joke2k/faker

ydata-synthetic


Mockaroo  https://www.mockaroo.com/ 

GenerateData  https://site.generatedata4.com/ 

JSON Schema Faker  https://json-schema-faker.js.org/ 

FakeStoreAPI  https://fakestoreapi.com/

graviti dataset https://gas.graviti.com/open-datasets

Synthetic data for computer vision https://github.com/ZumoLabs/zpy

Synthetic data library https://github.com/finos/datahub https://github.com/agmmnn/awesome-blender https://opendata.blender.org/ https://www.youtube.com/watch?v=eZwOeBkLL8E

https://www.kdnuggets.com/2019/09/scikit-learn-synthetic-dataset.html

Fully Synthetic Data,Partially Synthetic Data ,Hybrid Synthetic Data https://towardsdatascience.com/synthetic-data-key-benefits-types-generation-methods-and-challenges-11b0ad304b55

Synthetic Image Datasets https://analyticsindiamag.com/unity-launches-synthetic-image-datasets-to-train-ai-models-faster/  https://dockship.io/articles/607847e461373d1b994cc2dc/create-synthetic-images-using-opencv-(python)

Synthetic Data Generation Using Gaussian Mixture Model https://deepnote.com/@chanakya-vivek-kapoor/Synthetic-Data-Generation-QaaTRs73T2iCb0amHFbwpQ

Synthetic Data Vault  https://analyticsindiamag.com/guide-to-synthetic-data-vault-an-ecosystem-of-synthetic-data-generation-libraries/ https://github.com/sdv-dev/SDV

Create Your own Image Dataset using Opencv https://www.analyticsvidhya.com/blog/2021/05/create-your-own-image-dataset-using-opencv-in-machine-learning/

ydata-synthetic https://github.com/ydataai/ydata-synthetic

61.Text Data Annotator Tool - Datasaur  https://datasaur.ai/

Tagalog is our state-of-the-art solution for data management and labeling in Natural Language Processing  https://www.tagalog.ai/tagalog/

62.Google Analytics cost data import https://segmentstream.com/google-analytics?utm_source=twitter&utm_medium=cpc&utm_campaign=ga_costs_import_en&utm_content=guide

63.https://lionbridge.ai/services/crowdsourcing/    https://lionbridge.ai/     https://www.clickworker.com/  https://appen.com/  https://www.globalme.net/

64.Azure Open Datasets https://azure.microsoft.com/en-us/services/open-datasets/       https://azure.microsoft.com/en-in/services/open-datasets/catalog/
  
Yelp Open Dataset  https://www.yelp.com/dataset

https://data.world/

ODK Open Data Kit- https://getodk.org/

World Bank Open Data https://data.worldbank.org/

https://analyticsindiamag.com/10-biggest-data-breaches-that-made-headlines-in-2020/

https://data.mendeley.com/

https://github.com/iamtekson/geospatial-data-download-sites

https://eugeneyan.com/writing/data-discovery-platforms/

65.https://medium.com/towards-artificial-intelligence/best-datasets-for-machine-learning-data-science-computer-vision-nlp-ai-c9541058cf4f

https://towardsdatascience.com/data-repositories-for-almost-every-type-of-data-science-project-7aa2f98128b

https://github.com/MTG/freesound-datasets

https://dataform.co/

https://github.com/rfordatascience/tidytuesday https://www.youtube.com/watch?v=vCBeGLpvoYM

https://www.analyticsvidhya.com/blog/2020/12/top-15-datasets-of-2020-that-every-data-scientist-should-add-to-their-portfolio/?utm_source=linkedin&utm_medium=AV|link|high-performance-blog|blogs|44181|0.375

https://cseweb.ucsd.edu/~jmcauley/datasets.html

66.https://en.wikipedia.org/wiki/List_of_datasets_for_machine-learning_research  

https://archive.org/details/datasets

https://commoncrawl.org/

https://www.youtube.com/watch?v=1aUt8zAG09E

67. 6 Sources of Financial Data https://medium.datadriveninvestor.com/financial-data-431b75975bb

yfinance for finance data using     https://github.com/ranaroussi/yfinance  https://medium.com/towards-artificial-intelligence/algorithmic-trading-with-python-and-machine-learning-part-1-47c56706c182

import fix_yahoo_finance as yf , yahoofinancials ,Pandas DataReaders,Twelve Data

financeapi https://towardsdatascience.com/pull-and-analyze-financial-data-using-a-simple-python-package-83e47759c4a7

Investing.com pip install investpy ,Kite by Zerodha pip install kiteconnect,quandl  pip install quandl

https://www.analyticsvidhya.com/blog/2021/01/bear-run-or-bull-run-can-reinforcement-learning-help-in-automated-trading/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+AnalyticsVidhya+%28Analytics+Vidhya%29

Downloading Historical Stock prices with Alpha Vantage  https://medium.com/towards-artificial-intelligence/downloading-historical-stock-prices-with-alpha-vantage-688edad46a6d

Pandas Datareader https://pandas-datareader.readthedocs.io/en/latest/  https://www.youtube.com/watch?v=f2BCmQBCwDs

Get Financial Data Directly into Python https://www.quandl.com/tools/python   https://medium.com/nerd-for-tech/how-to-get-financial-data-using-python-7a508f25fc39

openml https://www.openml.org/search?type=data

https://registry.opendata.aws/

voice_datasets https://github.com/jim-schwoebel/voice_datasets
 
Dynamically-Generated-Hate-Speech-Dataset https://github.com/bvidgen/Dynamically-Generated-Hate-Speech-Dataset

68.DOCANO, an open source text annotation tool  https://github.com/doccano/doccano

69.https://www.dataquest.io/blog/free-datasets-for-projects/

70.audio set https://research.google.com/audioset/

71.FlatData Flat explores how to make it easy to work with data in git and GitHub https://octo.github.com/projects/flat-data?utm_campaign=Data_Elixir&utm_source=Data_Elixir_337

2.Feature engineering

https://towardsdatascience.com/practical-code-implementations-of-feature-engineering-for-machine-learning-with-python-f13b953d4bcd

Feature-engine https://trainindata.medium.com/feature-engine-a-new-open-source-python-package-for-feature-engineering-29a0ab88ea7c https://feature-engine.readthedocs.io/en/latest/ https://awesomeopensource.com/project/solegalli/feature_engine https://www.datasciencecentral.com/profiles/blogs/feature-engine-python-package-for-feature-engineering

Automated feature engineering https://medium.com/ibm-data-ai/automated-feature-engineering-for-relational-data-with-autoai-3612fafe9f89

Automated Data Wrangling https://catalyst.coop/2021/05/23/automated-data-wrangling/

Automatic Feature Engineering Using Featurewiz https://towardsdatascience.com/automate-your-feature-selection-workflow-in-one-line-of-python-code-3d4f23b7e2c4 https://awesomeopensource.com/project/AutoViML/featurewiz

Automatic Feature Engineering Using AutoFeat https://analyticsindiamag.com/guide-to-automatic-feature-engineering-using-autofeat/

Categorical Encoding https://awesomeopensource.com/project/scikit-learn-contrib/category_encoders

Streamlining Feature Engineering Pipelines with Feature-engine https://towardsdatascience.com/streamlining-feature-engineering-pipelines-with-feature-engine-e781d551f470 https://feature-engine.readthedocs.io/en/latest/#

Validate your Data (Schema) https://towardsdatascience.com/introduction-to-schema-a-python-libary-to-validate-your-data-c6d99e06d56a

Data cleaning-Pyjanitor-https://analyticsindiamag.com/beginners-guide-to-pyjanitor-a-python-tool-for-data-cleaning/

data cleaning library https://www.analyticsvidhya.com/blog/2021/05/data-cleaning-libraries-in-python-a-gentle-introduction/

Cleaner Data Analysis with Pandas Using Pipes https://towardsdatascience.com/cleaner-data-analysis-with-pandas-using-pipes-4d73770fbf3c

DataPrep https://dataprep.ai/ https://awesomeopensource.com/project/sfu-db/dataprep https://towardsdatascience.com/dataprep-v0-3-0-has-been-released-be49b1be0e72

Dora (pip library) - data cleaning

cleantext https://www.youtube.com/watch?v=i2TjAgga1YU&feature=youtu.be

CleanText: A Python Package to Clean Raw Text Data https://analyticsindiamag.com/guide-to-cleantext-a-python-package-to-clean-raw-text-data/

ATOM https://awesomeopensource.com/project/tvdboom/ATOM https://towardsdatascience.com/how-to-test-multiple-machine-learning-pipelines-with-just-a-few-lines-of-python-1a16cb4686d

openrefine A free, open source, powerful tool for working with messy data https://openrefine.org/#

data leaning library https://www.analyticsvidhya.com/blog/2021/05/data-cleaning-libraries-in-python-a-gentle-introduction/

https://machinelearningmastery.com/basic-data-cleaning-for-machine-learning/

Speed Up Data Cleaning and Exploratory Data Analysis in Python with klib https://awesomeopensource.com/project/akanz1/klib https://towardsdatascience.com/speed-up-your-data-cleaning-and-preprocessing-with-klib-97191d320f80

missingno https://awesomeopensource.com/project/ResidentMario/missingno

Take the Pain Out of Data Cleaning for Machine Learning https://towardsdatascience.com/take-the-pain-out-of-data-cleaning-for-machine-learning-20a646a277fd

dabl https://ms-bharti.medium.com/jump-start-your-supervised-learning-task-with-dabl-e479323e81fe

Easy to use Python library of customized functions for cleaning and analyzing data https://awesomeopensource.com/project/akanz1/klib

PyOD https://pyod.readthedocs.io/en/latest/ https://github.com/yzhao062/pyod/blob/development/docs/index.rst https://towardsdatascience.com/how-to-detect-outliers-with-python-pyod-aa7147359e4b

Amazonโ€™s New Visual Data Cleaning Tool Can Speed Up Your AI Projects https://medium.com/dataseries/how-amazons-new-visual-data-tool-can-speed-up-your-ai-projects-68e3289382c

Featuretools https://www.featuretools.com/ https://towardsdatascience.com/why-automated-feature-engineering-will-change-the-way-you-do-machine-learning-5c15bf188b96

https://awesomeopensource.com/project/alteryx/featuretools https://analyticsindiamag.com/introduction-to-featuretools-a-python-framework-for-automated-feature-engineering/

Feature Selection using Genetic Algorithm https://awesomeopensource.com/project/kaushalshetty/FeatureSelectionGA

AutoFeat https://analyticsindiamag.com/guide-to-automatic-feature-engineering-using-autofeat/ https://awesomeopensource.com/project/cod3licious/autofeat

feast Feature Store for Machine Learning https://awesomeopensource.com/project/feast-dev/feast https://www.youtube.com/watch?v=ZeJdr0nZ9PA

FeatureTools,AutoFeat,TsFresh,Cognito,OneBM,ExploreKit,PyFeat

Automated Feature Selection: Featurewiz https://awesomeopensource.com/project/AutoViML/featurewiz https://towardsdatascience.com/featurewiz-fast-way-to-select-the-best-features-in-a-data-9c861178602e

zoofs a Python library for performing feature selection https://awesomeopensource.com/project/jaswinder9051998/zoofs

Feature Engineering of DateTime Variables for Data Science, Machine Learning https://www.kdnuggets.com/2021/04/feature-engineering-datetime-variables-data-science-machine-learning.html

NeatText a simple NLP package for cleaning textual data and text preprocessing https://awesomeopensource.com/project/Jcharis/neattext

Remove duplicate data in dataset,Data validity check,Contaminated Data,Inconsistent Data,Invalid Data,

a.Handle missing value

 Types of missing value  https://datamuni.com/@atsunorifujita/missing-value-imputation-using-datawig
 
 Handling Missing Values in Pandas https://pub.towardsai.net/handling-missing-values-in-pandas-f87cec928937
 
 i.missing completely at random(no correlation b/w missing and observed data) we can delete no disturbance of data distribution
 
 ii.missing at random (randomness in missing data, missing value have correlation by data) we can't delete because disturbance of data distribution
 
 iii.missing not at random  (there is reason for missing value and directly related to value)
 
 Identify Missingness Types With Missingno  https://towardsdev.com/how-to-identify-missingness-types-with-missingno-61cfe0449ad9
 
 Univariate,Multivariate  https://medium.com/fintechexplained/what-are-imputers-in-data-science-b72f8308322b

 1.if missing data too small then delete it a.row deletion b.column deletion c.pairwise deletion and listwise
 
 2.replace by statistical method mean(influenced by outiler),median(not influenced by outiler),mode
 
 3.apply classifier algorithm to predict missing value
 
 Using Algorithms that support missing values
 
 Imputation using Deep Learning Library โ€” Datawig  https://github.com/awslabs/datawig
 
 4.Iterative imputer,knn imputer, multivariate imputation
 
 5.apply unsupervised 
 
 6.Random Imputation
 
 7.Adding a variable to capture NAN(missing term)
 
 8.Arbitrary Value Imputation
 
 9.hot deck Imputation,Cold deck imputation
 
 10.regression Imputation,Stochastic Regression Imputation
 
 11.End of Distribution Imputation
 
 12.Arbitrary Value Imputation
 
 13.Frequent Category Imputation
 
 14.MICE Imputation
 
 15.interpolation  https://www.analyticsvidhya.com/blog/2021/06/power-of-interpolation-in-python-to-fill-missing-values/
 
 Extrapolation and Interpolation
 
 Imputation using K-NN
 
 Imputation Using Deep Learning (Datawig)
 
 15.autoimpute-https://github.com/kearnz/autoimpute
 
 16.bfill / ffill
 
 17.Adding a variable to capture NAN
 
 18.replace NAN with a new category
 
 https://www.kdnuggets.com/2021/05/deal-with-categorical-data-machine-learning.html
 
 https://towardsdatascience.com/6-different-ways-to-compensate-for-missing-values-data-imputation-with-examples-6022d9ca0779
 
 https://stefvanbuuren.name/fimd/want-the-hardcopy.html  https://www.datasciencecentral.com/profiles/blogs/how-to-treat-missing-values-in-your-data-1

b.Handle imbalance Collect More Data if possible,Try Resampling Your Dataset

 1.Under Sampling - mostly not prefer because lost of data
 
 2.Over Sampling  (RandomOverSampler (here new points create by same dot)) ,  SMOTETomek(new points create by nearest point so take long time),BorderLine Smote, SMOTE-ENN ,KMeans Smote,SVM Smote,SMOTNC,ADASYN,Smote-NC,Random Over Sampling,RandomUnderSampler,SMOTEN, Cluster-Based Over Sampling, Informed Over Sampling,MSMOTE,Oversampling Using Gaussian Mixture Models,SMOTE + Tomek Links, SMOTE + ENN,Crucio SMOTEENN
 
 smote_variants https://github.com/analyticalmindsltd/smote_variants

 https://towardsdatascience.com/5-smote-techniques-for-oversampling-your-imbalance-data-b8155bdbe2b5
 
 https://www.analyticsvidhya.com/blog/2017/03/imbalanced-data-classification/
 
 ensmble based -Bagging Based techniques, Boosting-Based techniques,Adaptive Boosting- Ada Boost techniques,Gradient Tree Boosting,XG Boost 
 
 tools Imb-learn,SMOTE-Variants,Regression-ReSampling  https://towardsdatascience.com/tools-to-handle-class-imbalance-bff20c3bf099

 Balancing data sets with Crucio ADASYN  https://medium.com/softplus-publication/balancing-data-sets-with-crucio-adasyn-79f04ff0779d

 LoRAS: A Better Oversampling Algorithm https://analyticsindiamag.com/hands-on-guide-to-loras-a-better-oversampling-algorithm/  https://github.com/narek-davtyan/LoRAS   
 
 https://towardsdatascience.com/7-over-sampling-techniques-to-handle-imbalanced-data-ec51c8db349f
 
 3.class_weight give more importance(weight) to that small class ( Cost-Sensitive Algorithms)
 
 4.use Stratified kfold to keep the ratio of classess constantly
 
 5.Weighted Neural Network
 
 6.MESA https://analyticsindiamag.com/guide-to-mesa-boost-ensemble-imbalanced-learning-with-meta-sampler/
 
 7.choose  Proper Evaluation Metric metric roc,f1,etc...
   
 https://machinelearningmastery.com/framework-for-imbalanced-classification-projects/  https://www.kdnuggets.com/2020/01/5-most-useful-techniques-handle-imbalanced-datasets.html
 
 8.Deep Imbalanced Regression https://github.com/YyzHarry/imbalanced-regression https://analyticsindiamag.com/deep-imbalanced-regression-complete-guide/
 
 9.BalancedBaggingClassifier,Threshold moving
 
 10.Try Different Algorithms (Bagging Based techniques,Boosting-Based techniques)
 
 Algorithmic Ensemble Techniques,Tree-Based Algorithms
 
 11.Try a Different Perspective ( consider as anomaly detection or change detection)
 
 12.databalancer  https://github.com/pradeepdev-1995/databalancer

c.Remove noise data

d.Format data

d.Discretize a.Equal width binning b.Equal frequency binning c.K-means Binning d.Discretization by Decision Trees e.ChiMerge

e.Handle categorical data Ordinal,Nominal,cyclic,binary categorical variables

 1.One Hot Encoding
 
 2.Count Or Frequency Encoding
 
 3.Target Guided Ordinal Encoding,Target Guided Mean Encoding
 
 4.Mean Encoding
 
 5.Probability Ratio Encoding
 
 6.label encoding  or .cat.codes
 
 7.probability ratio encoding
 
 8.woe(Weight_of_evidence)
 
 9.one hot encoding with multi category (keep most frequently repeated only)
 
 10.feature hashing 
 
 11.sparse csr matrix
 
 12.entity embeddings
 
 13.binary encoding
 
 14.Rare label encoding
 
 15.Leave-one-out(Loo) encoding

 16.hash encoding
 
 17.dummy encoding
 
 18.Helmert Encoding,Backward Difference Encoding,James-Stein Encoding,M-estimator Encoding,Thermometer Encoder,Bayesian Encoders,Effect Encoding

 Helmert Encoding,Base N Encoding,Hash Encoding,Effect or Sum or Deviation Encoding,Backward Difference Encoding,M-Estimator Encoding,James- Stein Encoding,Thermometer Encoding,CatBoost Encoding,Binary Encoding,NaN encoding Polynomial encoding,Expansion encoding 
 
 Transform your categorical columns with imperio SmoothingTransformer
 
 Improve ML Model Performance by Combining Categorical Features https://towardsdatascience.com/improve-ml-model-performance-by-combining-categorical-features-a23efbb6a215
 
 https://towardsdatascience.com/beyond-one-hot-17-ways-of-transforming-categorical-features-into-numeric-features-57f54f199ea4
 
 https://towardsdatascience.com/how-to-encode-categorical-data-d44dde313131  https://towardsdatascience.com/python-for-finance-7-useful-libraries-that-you-should-know-e422b9e9aaba

f.Scaling of data

   1.Normalisation  

   2.Standardization
 
   3.Robust Scaler not influenced by outliers because using of median,IQR
   
   4.Min Max Scaling
   
   5.Mean normalization
   
   6.maximum absolute scaling
   
   7.Power Transformer Scaler
   
   8.Scaling To Median And Quantiles
   
   9.unit vector scaler
   
   https://www.analyticsvidhya.com/blog/2020/07/types-of-feature-transformation-and-scaling/?utm_source=linkedin&utm_medium=KJ|link|high-performance-blog|blogs|44204|0.375

Q-Q plot or Shapiro-Wilk Normality Test or lilliefors test is used to check whether feature is guassian or normal distributed required for linear regression,logistic regression to Improve performance if not distributed then use below methods to bring it guassian distribution

normal test,Histogram,Q-Q plot,KDE plot,Skewness and Kurtosis for check normal distribution

Fitter Library Finding the Best Distribution that Fits Your Data https://towardsdatascience.com/finding-the-best-distribution-that-fits-your-data-using-pythons-fitter-library-319a5a0972e9

anderson teset use for check any distribution

       b.Logarithmic Transformation
    
       c.Reciprocal Trnasformation
    
       d.Square Root Transformation
    
       e.Exponential Transdormation
    
       f.BoxCOx and Yeo-Johnson Transformation
    
       g.log(1+x) Transformation
       
       h.johnson
       
       i.power transformations  https://towardsdatascience.com/when-and-how-to-use-power-transform-in-machine-learning-2c6ad75fb72e
       
       g.Quantile Transformation
       
       Root transformation,Cube root transformation
       
       Left skewness (use powers) Squares transformation,Cubes transformation,High powers

g.Remove low variance feature by using VarianceThreshold

h.Same variable(only 1 variable) in feature then remove feature

i.Outilers removing outilers depond on problem we are solving https://awesomeopensource.com/project/jainyk/package-outlier

  2 type of outilers available: Global outiler, Local outiler

  eg: incase of fraud detection outilers are very important
  
  methods to find outiler: Standard Deviation,zscore,boxplot,scatter plot,histogram,IQR,TensorFlow_Data_Validation,Scatterplot
  
  Automatic Outlier Detection:Isolation Forest,Local Outlier Factor,Minimum Covariance Determinant,Robust Random Cut Forest,DBScan Clustering,One-Class Classification,One-Class SVM,Autoencoder,Outlier Detection using In-degree Number,Histogram-based Outlier Detection
  
  outiler treatment: mean/median/random imputation,drop,discretization (binning),treat as seperate group,replace with resperctive percentiles,transforation(log,scaling,sqrt,power)
  
  ML model which are not sensitive to outliers Like:-KNN,Decision Tree,SVM,NaรฏveBayes,Ensemble 
  
  PyOD: A Python Toolkit For Outlier Detection https://analyticsindiamag.com/guide-to-pyod-a-python-toolkit-for-outlier-detection/ 
  
  TODS: An Automated Time-series Outlier Detection System https://github.com/datamllab/tods https://towardsdatascience.com/tods-detecting-outliers-from-time-series-data-2d4bd2e91381
  
  if outiler present then use robust scaling
  
  alibi-detect https://github.com/SeldonIO/alibi-detect#adversarial-detection   https://docs.seldon.io/projects/alibi-detect/en/latest/
  
  https://medium.com/towards-artificial-intelligence/outlier-detection-and-treatment-a-beginners-guide-c44af0699754
  
  https://towardsdatascience.com/two-outlier-detection-techniques-you-should-know-in-2021-1454bef89331

j.Anomaly anomaly-detection-resources https://awesomeopensource.com/project/yzhao062/anomaly-detection-resources

 clustering techniques to find it
 
 Timetk https://towardsdatascience.com/timetk-the-r-library-for-time-series-analysis-9822f7720318
 
 Isolation Forest(for Big Data),dbscan,Local Outlier Factor,One-Class Support Vector Machine,Autoencoders,knn,Time Series Analysis,Elliptic EnvelopeInterquartile Range,Median Absolute Deviation,K-Nearest Neighbours
 
 Anomaly detection using PyOD  https://pyod.readthedocs.io/en/latest/   https://www.youtube.com/watch?v=QPjG_313GOw  https://github.com/yzhao062/pyod
 
 divided into three types:Point/Global Anomalies,Collective Anomalies,Contextual Anomalies https://towardsdatascience.com/a-comprehensive-beginners-guide-to-the-diverse-field-of-anomaly-detection-8c818d153995
 
 https://github.com/zhuyiche/awesome-anomaly-detection
 
 https://medium.com/@ODSC/data-sciences-role-in-anomaly-detection-bd976f93d7e3

k.Sampling techniques

 a.biased sampling
 
 b.unbiased sampling

3.Exploratory Data Analysis(eda)

Explore the dataset by using  python or microsoft Excel,Atoti,Power BI,Datapaneโ€™s,Tableau,TabPy,SAS Business Intelligence and Analytics Tool,QlikView,PyToQlik ,KNIME,Splunk,RapidMiner,Zoho Analytics,Sisense etc...

TabPy: Combining Python and Tableau https://www.kdnuggets.com/2020/11/tabpy-combining-python-tableau.html

atoti https://www.atoti.io/  https://www.youtube.com/watch?v=Hb6mSXa14oo   Datapaneโ€™s Create a Beautiful Dashboard in Python in a Few Lines of Code https://towardsdatascience.com/datapanes-new-features-create-a-beautiful-dashboard-in-python-in-a-few-lines-of-code-a3c44523292b

Switching from Spreadsheets to Neptune.ai https://neptune.ai/blog/switching-from-spreadsheets-to-neptune-ai

Data Analysis using excel https://www.excel-easy.com/data-analysis.html https://www.educba.com/data-analysis-tool-in-excel/ https://www.youtube.com/watch?v=OOWAk2aLEfk

Power BI In Jupyter Notebooks https://github.com/microsoft/powerbi-jupyter  https://analyticsindiamag.com/microsoft-releases-power-bi-in-jupyter-notebooks/

Mito Generating Python By Editing Spreadsheet  https://www.youtube.com/watch?v=yy3-C39ra6s https://trymito.io/?source=twitter1

Automate Pivot Table with Python https://towardsdatascience.com/automate-excel-with-python-pivot-table-899eab993966

OpenPyXL: A Python Module For Excel https://analyticsindiamag.com/guide-to-openpyxl-a-python-module-for-excel/

causal interactive dashboards and beautiful visuals https://www.causal.app/,

Visual Programming (Orange) https://orange.biolab.si/

Integrating Tableau With Python https://analyticsindiamag.com/tabpy/  Qlib https://analyticsindiamag.com/qlib/

Data visualization (Matplotlib,Seaborn,Plotly,pyqtgraph,Bokeh,Pygal,Dash,Pydot,Geoplotlib,ggplot,visualizer,Altair,folium,geoplot,etc...)

Scatterplot,multi line plot,bubble chart,bar chart,histogram,boxplot,distplot,bubble charts,area plot,heat map,index plot,violin plot,time series plot,density plot,dot plot,strip plot,plotly,Choropleth Map,Kepler,PDF,Kernel density function,networkx,Scatter_matrix,Bootstrap_plot,functionvis,Higher-Dimensional Plots,3-D Plots,3D Plots With Matplotlib,3D Plots With Plotly,Animated Plot With Plotly,Word Clouds,HoloViz,Horizontal Bar Graphs,Stacked Bar Graphs,Group Bar Graphs,Raincloud Plots

50-charts https://towardsdatascience.com/how-did-i-classify-50-chart-types-by-purpose-a6b0aa5b812d

all in one https://app.learney.me/

Python Tool For Visualizing and Plotting 2D/3D Scientific Data https://analyticsindiamag.com/guide-to-mayavi-a-python-tool-for-visualizing-and-plotting-2d-3d-scientific-data/

7 Techniques to Visualize Geospatial Data https://www.kdnuggets.com/2017/10/7-techniques-visualize-geospatial-data.html

data to viz  https://www.data-to-viz.com/ 

Interactive plots directly with pandas https://towardsdatascience.com/get-interactive-plots-directly-with-pandas-13a311ebf426

Top 10 Data Visualization Tools https://www.analyticsvidhya.com/blog/2021/04/top-10-data-visualization-tools/  https://www.xenonstack.com/blog/data-visualization-tools/

https://www.analyticsvidhya.com/blog/2021/03/when-to-use-what-plot-a-beginners-guide-to-select-plots-for-visualization/

https://towardsdatascience.com/8-free-tools-to-make-interactive-data-visualizations-in-2021-no-coding-required-2b2c6c564b5b

https://datavizproject.com/   https://datavizcatalogue.com/ 

https://attachments.convertkitcdnm.com/232198/ee18f415-1406-4e5c-94f1-49a2c6e3ec4e/Statistics-The-Big-Picture-Poster.pdf

https://towardsdatascience.com/8-free-tools-to-make-interactive-data-visualizations-in-2021-no-coding-required-2b2c6c564b5b

HiPlot (high dimensional data)-https://github.com/facebookresearch/hiplot https://levelup.gitconnected.com/learn-hiplot-in-6-mins-facebooks-python-library-for-machine-learning-visualizations-330129d558ac

https://towardsdatascience.com/top-6-python-libraries-for-visualization-which-one-to-use-fe43381cd658

https://www.kaggle.com/abhishekvaid19968/data-visualization-using-matplotlib-seaborn-plotly

๐—ž๐—ฒ๐—ฟ๐—ฎ๐˜€ ๐— ๐—ผ๐—ฑ๐—ฒ๐—น ๐˜ƒ๐—ถ๐˜€๐˜‚๐—ฎ๐—น๐—ถ๐˜‡๐—ฎ๐˜๐—ถ๐—ผ๐—ป ๐—ด๐—ฒ๐—ป๐—ฒ๐—ฟ๐—ฎ๐˜๐—ผ๐—ฟ(ann-visualizer)- ๐—ฝ๐—ถ๐—ฝ๐Ÿฏ ๐—ถ๐—ป๐˜€๐˜๐—ฎ๐—น๐—น ๐—ด๐—ฟ๐—ฎ๐—ฝ๐—ต๐˜ƒ๐—ถ๐˜‡

univariate and bivariate and multivariate analysis

model visualization Tensorboard,netron,playground tensorflow,plotly,TensorDash,Dash,Microscope,Lucid

distributions(discerte,continous)

data distributions-normal distribution,Standard Normal Distribution,Student's t-Distribution,Bernoulli Distribution,Binomial Distribution,Poisson Distribution,๏‚ทUniform Distribution,F Distribution,Covariance and Correlation

Pingouin  statistical package https://pingouin-stats.org/index.html https://www.youtube.com/watch?v=zqi51Wu5qC0 

Types of Statistics  

1.Descriptive

2.Inferential

Types of data

1) Categorical (nomial,ordinal)
 
2) Numerical   (discerte,continous)

random variable(discerte random variable ,continous random variable)

Central Limit Theorem,Bayes Theorem,Confidence Interval,Hypothesis Testing,z test, t test,f test,Confidence Interval,1 tail test, 2 tail test,chisquare test,anova test,A/B testing

4.Feature selection https://awesomeopensource.com/project/solegalli/feature-selection-for-machine-learning

FeatureSelector https://github.com/WillKoehrsen/feature-selector       feature_engine https://github.com/solegalli/feature_engine

1.Filter methods (correleation,feature importance,chisquare test,Ttest,vif,anova test,mutal information,hypothesis test,information gain,Univariate Selection Methods,SelectKBest,SelectPercentile,Variance threshold etc...)

2.Wrapper methods (recursive feature eliminiation,SelectKbest,boruta,forward feature selection,backward feature elimination,Bi-directional selection,exhaustic feature selection,stepwise selection etc...)

3.Embedded method (lasso regression,ridge regression,elastic net regression,tree based etc...)

4.Hybrid Method(Recursive Feature Selection,Recursive Feature addition)

5.Feature creation

feature selection  https://medium.com/analytics-vidhya/feature-selection-extended-overview-b58f1d524c1c

DropConstantFeatures  DropDuplicateFeatures    DropCorrelatedFeatures  

step forward feature selection https://www.kdnuggets.com/2018/06/step-forward-feature-selection-python.html

4.Feature Importance

   a.ExtraTreesClassifier,ExtraTreesregressor

   b.SelectKBest

   c.Logistic Regression

   d.Random_forest_importance
   
   e.decision tree
   
   f.Linear Regression
   
   g.xgboost
   
   h.Pearson correlation
   
   Forward selection,Chi-square,Logit (Logistic Regression model)

5.curse of dimensionality (as dimension increases performance decreases)

6.highly correleated features then can take any 1 feature (multicollinearity)

7.dimension reduction

8.lasso regression to penalise unimportant features

9.VarianceThreshold ,selectkbest

10.model based selection

11.Mutual Information Feature Selection

12.remove features with very low variance (quasi constant feature dropping)

13.Univariate  feature selection

14.importance of feature (random forest importance)

15.feature importance with decision trees

16.PyImpetus

17.drop constant features (variance=0)

18.variance inflation factor(vif)

19.Recursive Feature Elimination     RecursiveFeatureAddition

20.exchaustive feature selection

21.Statistical Methods , Hypothesis Testing ,Recursive Feature Elimination

22.Boruta https://github.com/scikit-learn-contrib/boruta_py https://analyticsindiamag.com/hands-on-guide-to-automated-feature-selection-using-boruta/

23.Sequence Feature Selection

PyImpetus https://github.com/atif-hassan/PyImpetus

https://www.analyticsvidhya.com/blog/2016/12/introduction-to-feature-selection-methods-with-an-example-or-how-to-select-the-right-variables/

Automate your Feature Selection Workflow in one line of Python code https://github.com/AutoViML/featurewiz  https://towardsdatascience.com/automate-your-feature-selection-workflow-in-one-line-of-python-code-3d4f23b7e2c4

https://machinelearningmastery.com/feature-selection-with-real-and-categorical-data/ https://machinelearningmastery.com/statistical-hypothesis-tests-in-python-cheat-sheet/

https://www.analyticsvidhya.com/blog/2020/10/a-comprehensive-guide-to-feature-selection-using-wrapper-methods-in-python/

https://towardsdatascience.com/5-feature-selection-method-from-scikit-learn-you-should-know-ed4d116e4172

Feature Engineering Tools https://neptune.ai/blog/feature-engineering-tools?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-feature-engineering-tools

https://towardsdatascience.com/practical-code-implementations-of-feature-engineering-for-machine-learning-with-python-f13b953d4bcd

Featuretools: https://github.com/alteryx/featuretools  https://analyticsindiamag.com/introduction-to-featuretools-a-python-framework-for-automated-feature-engineering/
AutoFeat: https://github.com/cod3licious/autofeat
TSFresh: https://github.com/blue-yonder/tsfresh
FeatureSelector: https://github.com/WillKoehrsen/feature-selector

5.Data splitting

 Splitting ratio of data deponds on size of dataset available

 Training data,Validation data,Testing data

6.Model selection

Machine learning https://scikit-learn.org/stable/index.html

Choose the Right Machine Learning Algorithm for Your Application https://towardsdatascience.com/how-to-choose-the-right-machine-learning-algorithm-for-your-application-1e36c32400b9

Time Complexity Of Machine Learning Models -https://www.thekerneltrip.com/machine/learning/computational-complexity-learning-algorithms/

interactive tools https://awesomeopensource.com/project/Machine-Learning-Tokyo/Interactive_Tools

HTML tables into Google Sheets -https://towardsdatascience.com/import-html-tables-into-google-sheets-effortlessly-f471eae58ac9

Machine Learning Playground https://ml-playground.com/

visual introduction to machine learning http://www.r2d3.us/visual-intro-to-machine-learning-part-1/

draw a dataset from inside jupyter https://pypi.org/project/drawdata/ https://www.youtube.com/watch?v=b0rsDPQ3bjg

Visual programming language for machine learning - Kobra https://kobra.dev/

compose generate labels for supervised learning https://awesomeopensource.com/project/alteryx/compose https://analyticsindiamag.com/guide-to-prediction-engineering-with-compose/

human-learn https://towardsdatascience.com/human-learn-create-rules-by-drawing-on-the-dataset-bcbca229f00

Neural Network https://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=circle&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=0&networkShape=4,2&seed=0.46672&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false

Microscope https://microscope.openai.com/models https://www.youtube.com/watch?v=y0-ISRhL4Ks

Ptpython Autocompletion, Autosuggestion, Docstring https://awesomeopensource.com/project/prompt-toolkit/ptpython https://towardsdatascience.com/ptpython-a-better-python-repl-6e21df1eb648

3 Tools to Track and Visualize the Execution of your Python Code https://towardsdatascience.com/3-tools-to-track-and-visualize-the-execution-of-your-python-code-666a153e435e

ML Code memory Consuming https://towardsdatascience.com/how-much-memory-is-your-ml-code-consuming-98df64074c8f

PyGrid Privacy-preserving, Decentralized Data Science https://github.com/OpenMined/PyGrid/

Best and Worst Cases of Machine-Learning Models https://medium.com/towards-artificial-intelligence/best-and-worst-cases-of-machine-learning-models-part-1-36cdb9296611

https://www.youtube.com/watch?v=mlumJPFvooQ&list=PLZoTAELRMXVM0zN0cgJrfT6TK2ypCpQdY

skater Machine Learning Model Interpretation https://towardsdatascience.com/machine-learning-model-interpretation-47b4bc29d17f

Speedml Speeding up Machine Learning https://towardsdatascience.com/speedml-speeding-up-machine-learning-5dccbf21effd

pdpipe https://awesomeopensource.com/project/pdpipe/pdpipe pipeline https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html

PHOTONAI A high level Python API for designing and optimizing machine learning pipelines https://www.photon-ai.com/

Machine Learning in Tableau with PyCaret https://towardsdatascience.com/machine-learning-in-tableau-with-pycaret-166ffac9b22e

TabNet balances explainability and model performance on tabular data https://towardsdatascience.com/tabnet-e1b979907694

FreaAI That Automatically Finds Weaknesses In ML models https://analyticsindiamag.com/ibm-launches-freaai-that-automatically-finds-weaknesses-in-ml-models/

A.Supervised learning (have label data)

 1.Regression (output feature in continous data form)
 
   linear regression,polynomial regression,Robust Regression,support vector regression,Decision Tree Regression,Random Forest Regression,TensorFlow Decision Forests,
   
   least square method,linear-tree,Random Forest Regression,xgboost,ridge(L2 Regularization),lasso(L1 Regularization (more sparse)),catboost,gradientboosting,adaboost,Explainable Boosting Machine,Histogram-Based Gradient Boost,LightBoost,CatBoost,XBNet,Chefboost
   
   elsatic net,light gbm,ordinary least squares,cart,Stepwise Regression,Multivariate Adaptive Regression Splines ,Generalised Additive Model(learn non-linear feature),tabnet,Linear Tree regression
   
   Locally Weighted Linear Regression https://towardsdatascience.com/locally-weighted-linear-regression-in-python-3d324108efbf
   
   TuringBot https://www.youtube.com/watch?v=LyKzKvjyIPo
   
   chefboost is an alternative library for training tree-based models https://github.com/serengil/chefboost
   
   growtrees About Cost-Aware Robust Tree Ensembles for Security Applications https://github.com/surrealyz/growtrees

 2.Classification (output feature in categorical data form)
 
    Binary,Multi-class,Multi-labe
 
    Logistic Regression,K-Nearest Neighbors,Support Vector Machine,Kernel SVM,Naive Bayes,Decision Tree Classification,linear-tree,TensorFlow Decision Forests,
    
    Random Forest Classification,TensorFlow Decision Forests,xgboost,adaboost,Gradient Boost,XBNet,catboost,gaussian NB,LGBMClassifier,LinearDiscriminantAnalysis, Extreme Gradient Boosting Machine, Explainable Boosting Machine,Chefboost,passive aggressive classifier algorithm,cart,c4.5,c5.0,tabnet
    
    https://mlwhiz.com/blog/2019/11/12/dtsplits/?utm_campaign=the-simple-math-behind-3-decision-tree-splitting-criterions&utm_medium=social_link&utm_source=missinglettr-linkedin
    
    4 Useful techniques avoid overfitting in decision trees https://towardsdatascience.com/4-useful-techniques-that-can-mitigate-overfitting-in-decision-trees-87380098bd3c
    
    Machine Learning โ€“ itโ€™s all about assumptions  https://www.kdnuggets.com/2021/02/machine-learning-assumptions.html
    
    GPBoost: A Library To Combine Tree-Boosting With Gaussian Process And Mixed-Effects Models https://analyticsindiamag.com/guide-to-gpboost-a-machine-learning-library-to-combine-tree-boosting/
    
    
    Data and Concept Drift  https://evidentlyai.com/blog/machine-learning-monitoring-data-and-concept-drift

B.Unsupervised learning(no label(target) data)

 1.Dimensionality reduction - PCA,SVD,LDA,som,tsne,openTSNE,plsr,pcr,autoencoders,kernelpca,Latent Semantic Analysis,Factor Analysis,Locality Preserving Projections,Isometric Mapping 
 
 t-SNE Effectively https://distill.pub/2016/misread-tsne/
 
 17 clustering  https://towardsdatascience.com/17-clustering-algorithms-used-in-data-science-mining-49dbfa5bf69a
 
 https://neptune.ai/blog/clustering-algorithms?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-clustering-algorithms

 2.Clustering : Centroid-based Model ,Density-based Model ,Distribution-based Model,Connectivity-based model
 
 https://www.mygreatlearning.com/blog/unsupervised-machine-learning/?highlight=unsupervised%20machine%20learning&utm_source=GLA&utm_medium=Blog&utm_campaign=1-16th%20May
 
 https://scikit-learn.org/stable/modules/clustering.html  https://machinelearningmastery.com/clustering-algorithms-with-python/

 https://towardsdatascience.com/17-clustering-algorithms-used-in-data-science-mining-49dbfa5bf69a
 
 RFM Segmentation in E-Commerce https://towardsdatascience.com/rfm-segmentation-in-e-commerce-e0209ce8fcf6
 
 kmodes https://www.youtube.com/watch?v=8eATPLDJ0NQ
 
 Agglomerative Hierarchical Clustering Using AGNES Algorithm https://analyticsindiamag.com/perform-agglomerative-hierarchical-clustering-using-agnes-algorithm/
 
 CLARANS Clustering Algorithm https://analyticsindiamag.com/comprehensive-guide-to-clarans-clustering-algorithm/
 
 https://pub.towardsai.net/fully-explained-birch-clustering-for-outliers-with-python-2ad6243f126b
 
 https://www.kdnuggets.com/2020/12/algorithms-explained-k-means-k-medoids-clustering.html 
 
 https://www.kdnuggets.com/2017/03/naive-sharding-centroid-initialization-method.html
 
 K-Means 8x faster, 27x lower error than Scikit-learn in 25 lines   https://www.kdnuggets.com/2021/01/k-means-faster-lower-error-scikit-learn.html#.YAHAAIpnx4A.linkedin

 k-Means Clustering by up to 10x Over Scikit-Learn  https://towardsdatascience.com/how-to-speed-up-your-k-means-clustering-by-up-to-10x-over-scikit-learn-5aec980ebb72
 
 3.Association Rule Learning - support,lift,confidence,leverage,aprior,elcat,Fp-growth,Fp-tree construction, association_rules,Frequent Itemset Mining

 4.Recommendation system -
 
     a.collaborative Recommendation system (model based, memory based(item based,user based))  user-item interaction matrix
    
     b.content based Recommendation system 
     
     similarity based(user-user similarity,item-item similarity)
     
     matrix factorization
     
     c.utility based Recommendation system 
     
     d.knowledge based Recommendation system 
     
     e.demographic based Recommendation system 
     
     f.hybrid based Recommendation system 
     
     g.Average Weighted Recommendation
     
     h.using K Nearest Neighbor
     
     i.cosine distance recommender system
     
     j.TensorFlow Recommenders https://www.tensorflow.org/recommenders
     
     recommenders  https://github.com/microsoft/recommenders
     
     k.suprise baseline model
     
     l.Tf-Rec https://github.com/Praful932/Tf-Rec
     
     m.Deep Learning Recommendation Models https://www.kdnuggets.com/2021/04/deep-learning-recommendation-models-dlrm-deep-dive.html
     
     Restricted Boltzmann Machines 
     
     TOROS Buffalo https://github.com/kakao/buffalo
     
     recommenders-https://github.com/microsoft/recommenders
     
     PyTorch implementations of deep reinforcement learning algorithms and environments https://github.com/p-christ/Deep-Reinforcement-Learning-Algorithms-with-PyTorch
     
     recmetrics library of metrics for evaluating recommender systems   https://github.com/statisticianinstilettos/recmetrics
     
     Downsize Recommendation Models By 112 Times https://analyticsindiamag.com/explained-facebooks-novel-method-to-downsize-recommendation-models-by-112-times/
     
     https://analyticsindiamag.com/top-open-source-recommender-systems-in-python-for-your-ml-project/
     
     https://towardsdatascience.com/modern-recommender-systems-a0c727609aa8
     
     https://machinelearningmastery.com/recommender-systems-resources/

C.Ensemble methods

 1.Stacking models https://www.analyticsvidhya.com/blog/2021/03/advanced-ensemble-learning-technique-stacking-and-its-variants/?

   vecstack https://github.com/vecxoz/vecstack

 2.Bagging models

 3.Boosting models
 
 4.Blending
 
 5.Voting (Hard Voting,Soft Voting)
 
 https://analyticsindiamag.com/do-ensemble-methods-always-work/
 
 Shapley value of players (models) in weighted voting games  https://github.com/benedekrozemberczki/shapley

D.Reinforcement learning https://neptune.ai/blog/best-reinforcement-learning-tutorials-examples-projects-and-courses

  2 types a)model free   b)model based
  
  gym-https://github.com/openai/gym           reinforcement learning by using  PyTorch-https://github.com/SforAiDl/genrl

  agent,environment,policy(On-Policy vs Off-Policy),reward function,value function,state,action,episode,actor-critic

  agent apply action to environment get corresponding reward so that it learn environment
  
  How to get started with Reinforcement Learning https://gordicaleksa.medium.com/how-to-get-started-with-reinforcement-learning-rl-4922fafeaf8c
  
  1.Q-Learning
  
  2.Deep Q-Learning
  
  3.Deep Convolutional Q-Learning
  
  Deep Deterministic Policy Gradient
  
  4.Twin Delayed DDPG,DQN
  
  5.A3C  (Actor Critic) ,A2C, Soft Actor Critic (SAC)
  
  6.Advantage weighted actor critic (AWAC). 
  
  7.XCS
  
  8.genetic algorithm,sarsa,natural policy gradient
  
  https://simoninithomas.github.io/deep-rl-course/
  
  SARSA,REINFORCE,PPO,DDPG,TD3
  
  AUTORL: AUTOML FOR RL https://www.automl.org/blog-autorl/
  
   Environments-OpenAI Gym, DeepMind Lab, Unity ML-Agents
   
   https://data-flair.training/news/python-libraries-for-reinforcement-learning/
   
   https://analyticsindiamag.com/8-best-free-resources-to-learn-deep-reinforcement-learning-using-tensorflow/   
   
   https://analyticsindiamag.com/top-8-autonomous-driving-open-source-projects-one-must-try-hands-on/
   
   https://analyticsindiamag.com/8-toolkits-for-reinforcement-learning-models-that-make-reasoning-explainability-core-to-ai/
   
   https://neptune.ai/blog/best-reinforcement-learning-tutorials-examples-projects-and-courses
   
   https://towardsdatascience.com/value-based-methods-in-deep-reinforcement-learning-d40ca1086e1
   
   https://neptune.ai/blog/best-reinforcement-learning-tutorials-examples-projects-and-courses?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-best-reinforcement-learning-tutorials-examples-projects-and-courses
   
   TensorForce: A TensorFlow-based Reinforcement Learning Framework https://analyticsindiamag.com/guide-to-tensorforce-a-tensorflow-based-reinforcement-learning-framework/
  
   Decision Transformer: Reinforcement Learning via Sequence Modeling https://github.com/kzl/decision-transformer
   
   Open AI Gym - https://gym.openai.com/
   
   DeepMindโ€™s MuZero  https://deepmind.com/blog/article/muzero-mastering-go-chess-shogi-and-atari-without-rules?utm_campaign=Learning%20Posts&utm_content=150411901&utm_medium=social&utm_source=twitter&hss_channel=tw-3018841323
   
   KerasRL https://github.com/keras-rl/keras-rl
   
   pyqlearning
   
   tensorforce https://tensorforce.readthedocs.io/en/latest/index.html  
   
   Practical_RL https://github.com/yandexdataschool/Practical_RL
   
   rl_coach https://github.com/IntelLabs/coach#installation        MushroomRL https://mushroomrl.readthedocs.io/en/latest/
   
   TFAgents  https://github.com/tensorflow/agents (https://www.tensorflow.org/agents)   https://deepmind.com/blog/article/trfl    
   
   Automate The Stock Market Using FinRL (Deep Reinforcement Learning Library)  https://analyticsindiamag.com/stock-market-prediction-using-finrl/
   
   Stable Baselines  https://github.com/openai/baselines
   
   https://www.youtube.com/playlist?list=PL_iWQOsE6TfURIIhCrlt-wj9ByIVpbfGc
   
   https://neptune.ai/blog/the-best-tools-for-reinforcement-learning-in-python?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-the-best-tools-for-reinforcement-learning-in-python

Semi-Supervised Learning-small amount of labeled data with a large amount of unlabeled data during training

Machine Learning with Graphs http://web.stanford.edu/class/cs224w/

E.Deep-learning (use when have huge data and data is highly complex and state of art for unstructured data) https://www.kdnuggets.com/2019/11/designing-neural-networks.html

Model Zoo Discover open source deep learning code and pretrained models https://modelzoo.co/

Visualizing your Neural Network with Netron

Sharing the best resources on various machine learning topics https://www.backprop.org/

deeplearning-models-https://awesomeopensource.com/project/rasbt/deeplearning-models

Deep-Learning-with-PyTorch- https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf

Frameworks:Pytorch,Tensorflow,Keras,caffe,theano,MXNet,Matlab,Microsoft Cognitive Toolkit,opacus(Train PyTorch models with Differential Privacy)

https://towardsdatascience.com/the-mostly-complete-chart-of-neural-networks-explained-3fb6f2367464 https://docs.deepstack.cc/getting-started/index.html

fastest way to build, debug, and interpret neural networks https://www.perceptilabs.com/

Nengo: A New Neural Network Building and Deployment Tool https://pub.towardsai.net/nengo-a-new-neural-network-building-and-deployment-tool-66677c65fa19

Binarized Neural Network memory size is reduced, and bitwise operations improve the power efficiency https://neptune.ai/blog/binarized-neural-network-bnn-and-its-implementation-in-ml

paddlehub https://awesomeopensource.com/project/PaddlePaddle/PaddleHub Performing Computer Vision & NLP Tasks in a Single Of Code https://towardsdatascience.com/performing-computer-vision-nlp-tasks-in-a-single-of-code-f7205f212d34

scikit-neuralnetwork https://towardsdatascience.com/the-simplest-way-to-train-a-neural-network-in-python-17613fa97958 https://awesomeopensource.com/project/aigamedev/scikit-neuralnetwork

NVIDIAโ€™s Kaolin: A 3D Deep Learning Library https://analyticsindiamag.com/nvidias-kaolin-3d-deep-learning-library/ https://awesomeopensource.com/project/NVIDIAGameWorks/kaolin

PySyft is a Python library for secure and private Deep Learning https://awesomeopensource.com/project/OpenMined/PySyft

keras-vis Visualizing Learning of a Deep Neural Network https://towardsdatascience.com/deep-learning-model-visualization-6cf6290dc981

Deep Replay Visualizing Learning of a Deep Neural Network https://towardsdatascience.com/visualizing-learning-of-a-deep-neural-network-b05f1711651c

keras-visualizer Visualizing Keras Models https://towardsdatascience.com/visualizing-keras-models-4d0063c8805e

Lucid Library is an open source framework to improve the interpretation of deep neural networks

Gradient-Centralization-TensorFlow improve your training performance of TensorFlow models with just 2 lines of code! https://awesomeopensource.com/project/Rishit-dagli/Gradient-Centralization-TensorFlow

XBNet: An Extremely Boosted Neural Network

MIL-WebDNN Fastest DNN Execution Framework on Web Browser https://mil-tokyo.github.io/webdnn/

1.Multilayer perceptron(MLP)

 1.Regression task

 2.Classification task
 
 Tabnet and deep tables for tabular dataset using deep learning

2.Convolutional neural network ( use for image data)

 Best MLOps Tools for Your Computer Vision Project Pipeline https://neptune.ai/blog/best-mlops-tools-for-computer-vision-project?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-best-mlops-tools-for-computer-vision-project

  mediapipe https://google.github.io/mediapipe/        cv modelhub https://modelplace.ai/
  
  all openmmlab https://github.com/open-mmlab  mmdetection,mmsegmentation,mmediting,mmdetection3d,mmaction2,mmocr,mmpose,etc...
  
  glasses High-quality Neural Networks for Computer Vision  https://github.com/FrancescoSaverioZuppichini/glasses
  
  IceVision https://airctic.com/0.8.0/
  
  Top Computer Vision Google Colab Notebooks- https://www.qblocks.cloud/creators/computer-vision-google-colab-notebooks
  
  for low code object detection (detecto)- https://github.com/alankbi/detecto
  
  CV-pretrained-model-   https://github.com/balavenkatesh3322/CV-pretrained-modelCV-pretrained-model-
  
  Fast Computer Vision Model Building PyTorch Lightning Flash and FiftyOne https://towardsdatascience.com/open-source-tools-for-fast-computer-vision-model-building-b39755aab490
  
  5 Open-Source Facial Recognition   https://medium.com/analytics-vidhya/ways-to-boost-your-computer-vision-projects-by-using-5-open-source-facial-recognition-projects-56668f170cb9


 1.Classification of image 
 
   albumentations https://github.com/albumentations-team/albumentations  AugLy https://github.com/facebookresearch/AugLy
 
   create own model,Lenet,Alexnet,DenseNet,MobileNet,ShuffleNet,SqueezeNet,Resenet,GoogleNet,Inception,Vgg16,vgg19,Efficient,EfficientNetV2,EfficientDet,residualnet,Nasnet,STN,nasneta,senet,amoebanetc,DeiT (tiny,small,base),Meta Pseudo Labels,res-mlp-pytorch,MLP-Mixer,vit,DynamicViT, FNet,gMLP models,nfnet
   
   https://theaisummer.com/cnn-architectures/  https://paperswithcode.com/sota/image-classification-on-imagenet 
   
   timm https://pypi.org/project/timm/  https://github.com/rwightman/pytorch-image-models
 
 2.Localization of object in image
 
 3.Object detection and object segmentation 
 
   rcnn,fastrcnn,fastercnn,TensorFlow Object Detection,yolo v1,yolo v2,yolo v3,SlimYOLOv3,yolo v4,PP-YOLO,scaled yolov4,YOLOR,YoloV5,YOLOS,efficinetdet,fast yolo,yolo tiny,yolo lite,yolo tiny++,yolo act++,
   
   maskrcnn,DeepLab-v3-plus,ssd,detectron,detectron2,D2Go,mobilenet,retinanet,R-fcn,Libra_R-CNN,detr facebook,mdetr,pspnet,segnet,U-net,UNet++,Efficient U-Nets, ๐——๐—ฒ๐—ป๐˜€๐—ฒ-๐—š๐—ฎ๐˜๐—ฒ๐—ฑ ๐—จ-๐—ก๐—ฒ๐˜, nnU-Net,v-net,TransUNet, H-DenseUNet, MultiResUNet ,deeplab,globalconvolutionnetwork,fcn,EfficientDet,Vision Transformer,deit,VarifocalNet (VF-Net),DINO,BodyPix,vit,AugFPN,mlsd
   
   PixelLib Simplifying Object Segmentation with PixelLib Library  https://github.com/ayoolaolafenwa/PixelLib

   mmdetection https://github.com/open-mmlab/mmdetection    https://towardsdatascience.com/mmdetection-tutorial-an-end2end-state-of-the-art-object-detection-library-59064deeada3
   
   imageai.Detection ObjectDetection       Segmentation models https://github.com/qubvel/segmentation_models
   
   Image-Segmentation-Using-Pixellib
   
   IceVision https://airctic.com/0.8.0/
   
   Image Generation Using TensorFlow Keras https://analyticsindiamag.com/getting-started-image-generation-tensorflow-keras/

   Video Understanding https://towardsdatascience.com/video-understanding-made-simple-with-pytorch-video-and-lightning-flash-c7d65583c37e

   Getting Started With Object Detection Using TensorFlow https://analyticsindiamag.com/object-detection-using-tensorflow/
  
   Instance Segmentation using Mask-RCNN with PixelLib and Python https://www.youtube.com/watch?v=i_-ud01wFhc
   
   MLP   MLP solution for Vision, from Google AI  https://github.com/lucidrains/mlp-mixer-pytorch
   
   MMDetection https://analyticsindiamag.com/guide-to-mmdetection-an-object-detection-python-toolbox/  mediapipe  https://github.com/google/mediapipe
   
   SSL Framework For Object Detection https://analyticsindiamag.com/googles-stac-ssl-framework-for-object-detection/ 
   
   GSDT https://analyticsindiamag.com/gsdt-gnns-for-simultaneous-detection-and-tracking/
   
   D2Go Brings Detectron2 To Mobile  https://analyticsindiamag.com/facebooks-d2go-brings-detectron2-to-mobile/
   
   AdelaiDet  open source toolbox for multiple instance-level detection and recognition tasks  https://github.com/aim-uofa/AdelaiDet
   
   3d object detection https://omdena.com/blog/3d-object-detection/?utm_source=linkedin&utm_medium=organic&utm_campaign=blog&utm_term=google-analytics
   
   PyMAF https://analyticsindiamag.com/guide-to-pymaf-pyramidal-mesh-alignment-feedback/
   
   3 kind of object segmentation are available semantic segmentation,instance segmentation,panoptic segmentation
   
   segmentation_models  https://github.com/qubvel/segmentation_models
   
   https://analyticsindiamag.com/guide-to-panoptic-segmentation-a-semantic-instance-segmentation-approach/  https://analyticsindiamag.com/semantic-vs-instance-vs-panoptic-which-image-segmentation-technique-to-choose/
   
   ResNeSt: A Better ResNet with the Same Costs https://analyticsindiamag.com/guide-to-resnest-a-better-resnet-with-the-same-costs/
   
   PAN: Pyramid Attention Network for Semantic Segmentation  https://medium.com/mlearning-ai/review-pan-pyramid-attention-network-for-semantic-segmentation-semantic-segmentation-8d94101ba24a
   
   PyTorch based low code object detection-https://github.com/alankbi/detecto  
   
   https://www.kdnuggets.com/2021/03/extraction-objects-images-videos-5-lines-code.html    
   
   autogluon 
   
   GluonCV  https://medium.com/apache-mxnet/start-fitting-cv-models-like-scikit-learn-with-gluoncv-0-10-931ff910a38 
   
   https://awesomeopensource.com/project/hoya012/deep_learning_object_detection
 
 4.objecttracking  (mean shit and optical flow and kalman filter)
 
   Tracktor++,Trackrcnn,Jde,DeepSORT,FairMOT
   
   mmtracking https://github.com/open-mmlab/mmtracking
 
 5.Deepdream,Neural style transfer, Pose estimation 

 Machine Learning for Art https://ml4a.net/#

 Pose estimation by mediapipe library  https://google.github.io/mediapipe/  https://www.youtube.com/watch?v=brwgBf6VB0I
 
 posemodule https://www.youtube.com/watch?v=5kaX3ta398w   Pose Tracking  https://www.youtube.com/watch?v=0JU3kpYytuQ&t=1650s
 
 6.DEEP LEARNING METHODS FOR 2D :OpenPose,DeepPose,tfpose,MultiPoseNet,AlphaPose,Movenet lighting,VIBE,DeeperCut,Mask RCNN,DeepCut,Convolutional Pose Machines,PoseNet,MoveNet,Adobeโ€™s BodyNet,MoveNet and TensorFlow.js 
 
 openpose wrnchai  densepose
 
 Pose Estimation using OpenCV https://www.analyticsvidhya.com/blog/2021/05/pose-estimation-using-opencv/
 
 https://medium.com/beyondminds/an-overview-of-human-pose-estimation-with-deep-learning-d49eb656739b
 
 3D POSE ESTIMATION
 
 3D Image Classification https://keras.io/examples/vision/3D_image_classification/
 
 TensorFlow 2 Object Detection API tutorial https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/
 
 https://blog.paperspace.com/how-to-train-scaled-yolov4-object-detection/
 
 albumentations https://github.com/albumentations-team/albumentations https://towardsdatascience.com/getting-started-with-albumentation-winning-deep-learning-image-augmentation-technique-in-pytorch-47aaba0ee3f8
 
 AugLy  A Modern Data Augmentation Library  https://analyticsindiamag.com/complete-guide-to-augly-a-modern-data-augmentation-library/ https://github.com/facebookresearch/AugLy
 
 Data augmentation with tf.data
 
 ImageGenerator  image augmentation  ImageDataGenerator 
 
 Augmentor Image augmentation library in Python for machine learning  https://github.com/mdbloice/Augmentor
 
 albumentations  https://github.com/albumentations-team/albumentations 
 
 HiSD: Image-to-Image translation via Hierarchical Style Disentanglement https://analyticsindiamag.com/hisd-python-implementation-of-image-to-image-translation/
 
 Zooming Slow-Mo https://analyticsindiamag.com/guide-to-zooming-slow-mo-one-stage-space-time-video-super-resolution/
 
 Image Augmentation Pipelines with Tensorflow  https://towardsai.net/p/machine-learning/building-complex-image-augmentation-pipelines-with-tensorflow-bed1914278d2
 
 TensorFlow2.0-Examples  https://github.com/YunYang1994/TensorFlow2.0-Examples
 
 unadversarial  https://github.com/microsoft/unadversarial/ https://analyticsindiamag.com/microsoft-research-unadversarial/
 
 CNNs 'see' - FilterVisualizations, Heatmaps,Saliency Maps,saliency_map_guided,Heat Map Visualizations,GradCAM,Class Activation Maps,ZFNet,Lucid,Activation Atlas,Blur Integrated Gradients,concept whitening,Integrated Gradients,SmoothGrad,PytorchRevelio,Feature Visualizer, Guided Gradients, grad_cam,sensitivity_analysis,Captum,Preliminary Methods,Plot Model Architecture,Visualize Filters,Activation based Methods,Maximal Activation,Image Occlusion,Gradient based Methods,Gradient based Class Activation Map

 
 Tools to Design or Visualize Architecture of Neural Network https://github.com/ashishpatel26/Tools-to-Design-or-Visualize-Architecture-of-Neural-Network
 
 quiver Interactive convnet features visualization for Keras https://github.com/keplr-io/quiver
 
 https://jair-neto.medium.com/a-powerful-method-for-explainability-of-object-detection-algorithms-ace0fe4623e7
 
 https://github.com/utkuozbulak/pytorch-cnn-visualizations  https://microscope.openai.com/models  https://github.com/balavenkatesh3322/CV-pretrained-model
 
 Mediapipe for Python https://google.github.io/mediapipe/
 
 imageai.Detection for Object detection
 
 cnn-raccoon  interactive dashboards for your Convolutional Neural Networks with a single line of code https://github.com/lucko515/cnn-raccoon
 
 deit https://github.com/facebookresearch/deit   https://wandb.ai/thibault-neveu/detr-tensorflow-log/reports/Finetuning-DETR-Object-Detection-with-Transformers-on-Tensorflow-A-step-by-step-tutorial--VmlldzozOTYyNzQ  https://github.com/Visual-Behavior/detr-tensorflow
   
 awesome-computer-vision-models https://github.com/nerox8664/awesome-computer-vision-models
 
 EfficientDet https://github.com/ravi02512/efficientdet-keras
 
 Vision Transformer - Pytorch  https://github.com/lucidrains/vit-pytorch   https://github.com/alohays/awesome-visual-representation-learning-with-transformers 
 
 T2T-ViT https://analyticsindiamag.com/complete-guide-to-t2t-vit-training-vision-transformers-efficiently-with-minimal-data/ https://github.com/yitu-opensource/T2T-ViT
 
 Explainability for Vision Transformers https://github.com/jacobgil/vit-explain
 
 https://keras.io/examples/vision/image_classification_with_vision_transformer/
 
 https://github.com/ashishpatel26/Vision-Transformer-Keras-Tensorflow-Pytorch-Examples https://github.com/google-research/vision_transformer 
 
 DeepLab-v3-plus Semantic Segmentation in TensorFlow https://github.com/rishizek/tensorflow-deeplab-v3-plus
 
 DEEP LEARNING METHODS FOR 3D:3D human pose estimation= 2D pose estimation + matching,Integral Human Pose Regression,Towards 3D Human Pose Estimation in the

Wild: a Weakly-supervised Approach,A Simple Yet Effective Baseline for 3d Human Pose Estimation,

 Data Augmentation apply to increase size of dataset and performance of model
 
 low code object detection -  detecto  https://github.com/alankbi/detecto 
 
 AutoML  https://github.com/dataloop-ai/AutoML
 
 Object Detection with 10 lines of code-https://www.datasciencecentral.com/profiles/blogs/object-detection-with-10-lines-of-code  https://towardsdatascience.com/object-detection-with-10-lines-of-code-d6cb4d86f606
 
 Detecto https://github.com/alankbi/detecto https://medium.com/analytics-vidhya/computer-vision-in-healthcare-detection-of-fractures-3313fe6452fc
 
 OneNet-https://analyticsindiamag.com/onenet/
 
 Norfair https://github.com/tryolabs/norfair
 
 Remo Improves Image Management  https://www.freecodecamp.org/news/manage-computer-vision-datasets-in-python-with-remo/
 
 yolo https://github.com/zzh8829/yolov3-tf2 https://github.com/ultralytics/yolov5 https://github.com/ashishpatel26/Yolov5-King-of-object-Detection  https://github.com/sicara/tf2-yolov4
 
 clip https://github.com/openai/CLIP

3.Recurrent neural network (use when series of data)

 1.RNN
 
 2.GRU
 
 3.LSTM (have memory cell,forget gate  etc..)

 Depth Gated RNNs,Clockwork RNNs
 
 ๐—ง๐—ฒ๐—บ๐—ฝ๐—ผ๐—ฟ๐—ฎ๐—น ๐—–๐—ผ๐—ป๐˜ƒ๐—ผ๐—น๐˜‚๐˜๐—ถ๐—ผ๐—ป๐—ฎ๐—น ๐—ก๐—ฒ๐˜๐˜„๐—ผ๐—ฟ๐—ธ  better than LSTM/GRU https://github.com/ashishpatel26/tcn-keras-Examples
 
 4.Information Discrimination Units (IDU) https://github.com/hjeun/idu

 Train an LSTM Model ~30x Faster Using PyTorch with GPU https://towardsdatascience.com/how-to-train-an-lstm-model-30x-faster-using-pytorch-with-gpu-e6bcd3134c86
 
 all above 3 models have bidirectional also based on problem statement use bidirectional models
 
 Quasi-Recurrent Neural Network  https://github.com/salesforce/pytorch-qrnn
 
 textgenrnn https://github.com/minimaxir/textgenrnn

4.Generative adversarial network https://poloclub.github.io/ganlab/ https://developers.google.com/machine-learning/gan/training

 gan lab https://poloclub.github.io/ganlab/

 https://neptune.ai/blog/generative-adversarial-networks-gan-applications?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-generative-adversarial-networks-gan-applications
 
 Diffusion Models Beat GANs on Image Synthesis https://paperswithcode.com/paper/diffusion-models-beat-gans-on-image-synthesis?from=n9
 
 MUNIT: Multimodal Unsupervised Image-to-Image Translation (GAN)
 
 https://jonathan-hui.medium.com/gan-gan-series-2d279f906e7b

 generative adversarial transformers https://github.com/dorarad/gansformer
 
 LipGAN https://github.com/Rudrabha/LipGAN  Wav2Lip https://github.com/Rudrabha/Wav2Lip
 
 BigGAN https://analyticsindiamag.com/hands-on-guide-to-biggan-with-python-code/

 Cycle gan,Dcgan,cGAN,SRGAN,InfoGAN,stargan,attan gan,stylegan,,PixelRNN,StackGAN,DiscoGAN,lsGAN,Conditional GAN(Pix2Pix),Progressive GANs( produces higher resolution images,Image-to-Image Translation),Face Inpainting,Super-resolution,Progressive Growing GAN,Instance-Conditioned GAN,Wasserstein GAN(improve image generation),ChromaGan,GANsformers,Conditional GAN and Unconditional GAN,Least Square GAN,Auxilary Classifier GAN,Dual Video Discriminator GAN,SRGAN
 
 diffusion https://github.com/openai/guided-diffusion
 
 https://www.analyticsvidhya.com/blog/2021/05/progressive-growing-gan-progan/
 
 5 Alternatives To Deep Nostalgia https://analyticsindiamag.com/top-5-alternatives-to-deep-nostalgia/
 
 MixNMatch https://github.com/Yuheng-Li/MixNMatch
 
 Quantum GAN  https://analyticsindiamag.com/now-gans-are-being-used-for-drug-discovery-complete-guide-to-quantum-gan-with-python-code/
 
 https://analyticsindiamag.com/guide-to-differentiable-augmentation-for-data-efficient-gan-training/  https://analyticsindiamag.com/hands-on-python-guide-to-style-based-age-manipulation-sam-technique/
 
 Imaginaire https://analyticsindiamag.com/guide-to-nvidia-imaginaire-gan-library-in-python/
 
 Disentanglement https://analyticsindiamag.com/what-is-face-identity-disentanglement-and-how-it-outperformed-gans/
 
 StyleFlow https://github.com/RameenAbdal/StyleFlow 
 
 https://github.com/hindupuravinash/the-gan-zoo  https://analyticsindiamag.com/top-10-tools-for-generative-adversarial-networks/

5.Autoencoder

  1.sparse Autoencoder
  
  2.denoising Autoencoder
  
  3.Contractive Autoencoder
  
  4.stacked Autoencoder
  
  5.deep Autoencoder
  
  6.variational autoencoder
 
  7.convolutional   autoencoder
  
  https://github.com/zc8340311/RobustAutoencoder
  
  Applications of AutoEncoders,Dimensionality reduction,Anomaly detection,Image denoising,Image compression,Image generation

6.BoltzmannMachines,Restricted Boltzmann Machine,deep belief network,deep BoltzmannMachines

7.Self Organizing Maps (SOM)

8.Natural language processing

 Clean data(removing stopwords depond on problem ,lowering data,tokenization,postagging,stemmimg or lemmatization depond on problem,skipgram,n-gram,chunking)
 
 clean text https://github.com/jfilter/clean-text
 
 Cleaning and Pre-processing textual data with NeatText library   Automated NLP Pre-Processing using Data-Purifier Library https://github.com/Elysian01/Data-Purifier  
 
 Nltk,spacy,genism,textblob,inltk,Indic NLP,StanfordNLP,Pattern,stanza,OpenNLP,polygot,corenlp,polyglot,PyDictionary,Huggiing face,spark nlp,allen nlp,rasa nlu,Megatron,texthero,Flair,textacy,finetune,gluon-nlp,VnCoreNLP,fasttext,Langid,PyCLD3,Guesslang,Parrot  libraries
 
 jiant is an NLP toolkit https://github.com/nyu-mll/jiant
 
 clean-text  https://github.com/jfilter/clean-text https://www.youtube.com/watch?v=i2TjAgga1YU
 
 indicnlp https://indicnlp.ai4bharat.org/samanantar/
 
 Augmenting Data For NLP Tasks  https://towardsdatascience.com/tips-tricks-augmenting-data-for-nlp-tasks-983e33ad55a7 https://amitness.com/2020/05/data-augmentation-for-nlp/ https://github.com/makcedward/nlpaug https://towardsdatascience.com/data-augmentation-in-nlp-2801a34dfc28
 
 NLP Data Augmenting  https://lnkd.in/eHa2cH6
 
 Tagalog is our state-of-the-art solution for data management and labeling in Natural Language Processing https://www.tagalog.ai/tagalog/
 
 https://github.com/jasonwei20/eda_nlp  https://github.com/dsfsi/textaugment https://github.com/QData/TextAttack https://github.com/makcedward/nlpaug
 
 nlp_profiler https://analyticsindiamag.com/complete-guide-on-nlp-profiler-python-tool-for-profiling-of-textual-dataset/
 
 doccano text annotation tool https://github.com/doccano/doccano https://www.youtube.com/watch?v=vT-GE_jssPk https://github.com/doccano/auto-labeling-pipeline   https://github.com/doccano/doccano-client https://doccano.herokuapp.com/
 
 Data augmentation for NLP-https://github.com/makcedward/nlpaug
 
 Data Augmentation library for text nlpaug https://towardsdatascience.com/data-augmentation-library-for-text-9661736b13ff
 
 detext-https://github.com/linkedin/detext
 
 nlpaug-https://github.com/makcedward/nlpaug
 
 NLP-progress -https://github.com/sebastianruder/NLP-progress
 
 Super Duper NLP Repo- https://notebooks.quantumstat.com/
 
 Multilingual Representations for Indian Languages https://tfhub.dev/google/MuRIL/1
 
 Natural Language Processing 365- https://ryanong.co.uk/natural-language-processing-365/
 
 1 line for hundreds of NLP models and algorithms- https://github.com/JohnSnowLabs/nlu
 
 simpletransformers
 
 beautiful Wordclouds in Python  https://towardsdatascience.com/how-to-easily-make-beautiful-wordclouds-in-python-55789102f6f5
 
 Automate your Text Processing workflow in a single line of Python Code https://towardsdatascience.com/automate-your-text-processing-workflow-in-a-single-line-of-python-code-e276755e45de
 
 quantumstat  https://index.quantumstat.com/
 
 Dynaboard: Moving beyond accuracy to holistic model evaluation in NLP https://ai.facebook.com/blog/dynaboard-moving-beyond-accuracy-to-holistic-model-evaluation-in-nlp/
 
 gobbli for interactive NLP  https://medium.com/rti-cds/using-gobbli-for-interactive-nlp-f60feb41e5cb
 
 AutoReg  Regex of string in Python https://github.com/SusmitPanda/AutoReg
 
 NLU,NLG,NER,text summarization,Sentiment Analysis,Text Classifications,machine translation,chat bot,Text Generation,Speech Recognition
  
 1.bag of words
 
 2.Tfidf , Hashing
 
 3.wordembedding
    
    a.using pretrained model 
      
      i)word2vec( cbow,skipgram)
      
      ii)glove        https://medium.com/spark-nlp/1-line-to-glove-word-embeddings-with-nlu-in-python-baed152fff4d
      
      iiI)fasttext
    
    b.creating own embedding  (use when have huge data)
    
      i)word2vec library
      
      ii)keras embedding 
      
  elmo (store semantic of word)
    
 4.Document embedding-Doc2vec
  
 5.sentence embedding

   sense2vec,SENT2VEC,Universal sentence encoder
   
 Top2Vec 
 
 Topic Modelling https://towardsdatascience.com/april-edition-adventures-in-topic-modelling-7ee9081a48a0
 
 6.using rnn,lstm,gru
 
   for above 3 models have bidirectional also
 
 7.Encoder and Decoder(sequence to sequence), ProphetNet(new pretrained seq2seq model)
  
 8.attention 
 
   self attention,Global Attention,Multi-Head Attention,Local Attention (monotonic,predictive)    https://github.com/uzaymacar/attention-mechanisms
 
 9.Transformer (big breakthrough in NLP) - http://jalammar.github.io/illustrated-transformer/  

    Build a Transformer in JAX from scratch https://theaisummer.com/jax-transformer/
    
    Trankit is a Light-Weight Transformer-based Python Toolkit for Multilingual Natural Language Processing  https://github.com/nlp-uoregon/trankit
 
    FastFormers  https://medium.com/ai-in-plain-english/fastformers-233x-faster-transformers-inference-on-cpu-4c0b7a720e1
 
    Shrinking Transformers (reduce size)  1.quantization,distillation,pruning,
    
    Reformer,Performers,vision transformer
    
    Reformer: The Efficient Transformer
    
    Longformer: The Long-Document Transformer
    
    ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
    
    DeLighT: Deep and Light-weight Transformer  https://analyticsindiamag.com/complete-guide-to-delight-deep-and-light-weight-transformer/
    
    https://github.com/balavenkatesh3322/NLP-pretrained-model
    
    Tree-Transformer https://github.com/yaushian/Tree-Transformer
    
    Scalable Transformer-based Model  https://analyticsindiamag.com/guide-to-perceiver-a-scalable-transformer-based-model/
    
    Transformers Interpret https://towardsdatascience.com/introducing-transformers-interpret-explainable-ai-for-transformers-890a403a9470 https://github.com/cdpierse/transformers-interpret  https://analyticsindiamag.com/hands-on-guide-to-the-evolved-transformer-on-neural-machine-translation/
    
    Novel Interpretable Transformer https://github.com/hila-chefer/Transformer-Explainability  https://analyticsindiamag.com/compute-relevancy-of-transformer-networks-via-novel-interpretable-transformer/
    
    https://www.kdnuggets.com/2021/02/hugging-face-transformer-basics.html#.YE7gRy9s-LA.linkedin
    
    mBART-50 https://www.youtube.com/watch?v=fxZtz0LPJLE&feature=youtu.be
  
 10.BERT,Packed BERT,BART,DynaBERT,SBERT,ConvBert,Quantized MobileBERT,ALBERT,ELECTRA,ARBERT,MARBERTElectra,Transformer-XL,Longformer,Reformer,DistilBERT,ELMo,ROBERTA,XLNet,XLM-RoBERTa,DeBERTa,T5,fastT5, CodeT5,mT5,ByT5,simpleT5,byt5,OnnxT5,FastT5,Linformer,DISTILBERT,GPT,GPT2,GPT3,gpt-neo,gpt-neox,GPT-J,aitextgen,PRADO,PET,BORT,MuRIL,Multitask Unified Model,aitextgen,AI21's 'Jurassic' language model,Turing NLG,Wu Dao 2.0,PanGu-Alpha

    https://neptune.ai/blog/bert-and-the-transformer-architecture-reshaping-the-ai-landscape
    
    gpt3 https://www.producthunt.com/posts/100-resources-on-gpt-3
    
    Graph4NLP  https://dlg4nlp.github.io/index.html

    Feedback Transformers from Facebook AI https://towardsdatascience.com/feedback-transformers-from-facebook-ai-221c5dd09e3f

    DETR  https://analyticsindiamag.com/how-to-detect-objects-with-detection-transformers/  https://github.com/dddzg/up-detr
    
    DeiT https://analyticsindiamag.com/introducing-deit-data-efficient-image-transformers/ https://github.com/facebookresearch/deit
    
    80+ NLP tasks  https://medium.com/innerdoc/80-natural-language-processing-tasks-described-c777bc4974b3
    
    Text-to-Image  https://www.datasciencecentral.com/profiles/blogs/summarizing-popular-text-to-image-synthesis-methods-with-python
    
    NLP: Pre-trained Sentiment Analysis https://medium.com/@b.terryjack/nlp-pre-trained-sentiment-analysis-1eb52a9d742c
    
    Awesome-NLP-Resources -https://github.com/Robofied/Awesome-NLP-Resources  https://shivanandroy.com/awesome-nlp-resources/   https://github.com/keon/awesome-nlp
    
    10 Popular Keyword Extraction Algorithms in Natural Language Processing https://prakhar-mishra.medium.com/10-popular-keyword-extraction-algorithms-in-natural-language-processing-8975ada5750c
    
    https://medium.com/@jatinmandav3/opinion-mining-sometimes-known-as-sentiment-analysis-or-emotion-ai-refers-to-the-use-of-natural-874f369194c0#:~:text=fastText%20is%20a%20library%20for,pretrained%20models%20for%20294%20languages
 
    https://analyticsindiamag.com/top-ten-bert-alternatives-for-nlu-projects/  https://towardsdatascience.com/from-pre-trained-word-embeddings-to-pre-trained-language-models-focus-on-bert-343815627598
    
    
    GPT2 generated Indian Food Recipes https://www.kaggle.com/nulldata/gpt2-generated-indian-food-recipes
 
    http://jalammar.github.io/    http://jalammar.github.io/illustrated-bert/   http://jalammar.github.io/a-visual-guide-to-using-bert-for-the-first-time/
    
    https://jalammar.github.io/explaining-transformers/    https://jalammar.github.io/hidden-states/
    
    https://www.kdnuggets.com/2019/09/bert-roberta-distilbert-xlnet-one-use.html
    
 11.Speech  (Braina,Dragon Speech Recognition Solutions ,Winscribe,Gboard,Windows 10 Speech Recognition,Otter,Speechnotes,tts,OpenSpeech,FRILL,Vakyansh)
   
    speech to text   
    
    text to speech  https://towardsdatascience.com/text-to-speech-one-small-step-by-mankind-to-create-lifelike-robots-54e19f843b21
    
    Acoustic model,Speaker diarisation,apis,apiai,assemblyai,google-cloud-speech,pocketsphinx,SpeechRecognition,watson-developer-cloud,wit
    
    Microsoft IceCAPS is an Open Source Framework for Conversational Modeling https://pub.towardsai.net/microsoft-icecaps-is-an-open-source-framework-for-conversational-modeling-4f78492ca685
    
    State-of-the-art Approaches to Building Open-Domain Conversational Agents https://www.topbots.com/conversational-ai-open-domain-chatbots/?utm_source=twitter&utm_medium=company_post&utm_campaign=conversational_open_domain_chatbots
    
    LaMDA: our breakthrough conversation technology https://www.blog.google/technology/ai/lamda
    
    assemblyai https://www.assemblyai.com/
    
    SpeechBrain A PyTorch Powered Speech Toolkit https://speechbrain.github.io/  https://github.com/speechbrain/speechbrain
    
    Wav2vec-U learns to recognize #speech from unlabeled data https://venturebeat.com/2021/05/21/facebook-wav2vec-u-learns-to-recognize-speech-from-unlabeled-data/?utm_source=dlvr.it&utm_medium=linkedin
    
    Wav2Vec2 https://huggingface.co/transformers/model_doc/wav2vec2.html https://www.youtube.com/watch?v=dJAoK5zK36M&feature=youtu.be
    
    SincNet is a neural architecture for efficiently processing raw audio samples https://github.com/mravanelli/SincNet
    
    HuggingFace Transformers ASR  https://github.com/dennisbakhuis/Ecare_Brunch_ASR
    
    https://github.com/balavenkatesh3322/audio-pretrained-model
    
 SpeechRecognition
 
 audiomentations  Python library for audio data augmentation  https://github.com/iver56/audiomentations
 
 googletrans (google Translator)   https://pypi.org/project/googletrans/
 
 lang-identification   Google Compact Language Detector,FastText
 
 ๐—ด๐—ง๐—ง๐—ฆ for text to speech conversion , ๐˜€๐—ฝ๐—ฒ๐—ฒ๐—ฐ๐—ต_๐—ฟ๐—ฒ๐—ฐ๐—ผ๐—ด๐—ป๐—ถ๐˜๐—ถ๐—ผ๐—ป,TTS
 
 Python/Pytorch app for easily synthesising human voices https://github.com/BenAAndrew/Voice-Cloning-App
 
 Speech-Transformer-tf2.0 https://github.com/xingchensong/Speech-Transformer-tf2.0
 
 The Super Duper NLP Repo  https://notebooks.quantumstat.com/
 
 ecco https://github.com/jalammar/ecco  https://www.eccox.io/   https://www.youtube.com/watch?v=rHrItfNeuh0&feature=youtu.be 
 
 Language Interpretability Tool (LIT) is an open-source platform for visualization and understanding of NLP models https://pair-code.github.io/lit/
 
 Language Interpretability Tool https://github.com/pair-code/lit https://ai.googleblog.com/2020/11/the-language-interpretability-tool-lit.html
 
 autonlp  https://analyticsindiamag.com/hands-on-guide-to-using-autonlp-for-automating-sentiment-analysis/
    
 https://medium.com/towards-artificial-intelligence/natural-language-processing-nlp-with-python-tutorial-for-beginners-1f54e610a1a0
 
 https://pakodas.substack.com/p/neural-search-on-indian-languages     
 
 https://www.linkedin.com/pulse/natural-language-processing-2020-year-review-ivan-bilan/?trackingId=CYfd1ZyLStu6x09tjVIoGw%3D%3D
 
 ConvBert https://github.com/yitu-opensource/ConvBert
 
 Python interface for building, loading, and using GloVe vectors https://github.com/Lguyogiro/pyglove
 
 SentenceTransformers  https://www.sbert.net/
 
 Reformer โ€“ The Efficient Transformer  https://analyticsindiamag.com/hands-on-guide-to-reformer-the-efficient-transformer/
 
 Funnel-Transformer https://github.com/laiguokun/Funnel-Transformer
 
 CLIP โ€“ Connecting Text To Images  https://analyticsindiamag.com/hands-on-guide-to-openais-clip-connecting-text-to-images/ 
 
 Topic Modeling in One Line with Top2Vec https://towardsdatascience.com/topic-modeling-in-one-line-with-top2vec-a413991aa0ef
 
 MT5-https://venturebeat.com/2020/10/26/google-open-sources-mt5-a-multilingual-model-trained-on-over-101-languages/?utm_content=144321587&utm_medium=social&utm_source=linkedin&hss_channel=lcp-3740012
 
 VADER does not require any training data https://pypi.org/project/vaderSentiment/  https://analyticsindiamag.com/sentiment-analysis-made-easy-using-vader/
 
 APPLICATIONS OF MACHINE TRANSLATIO-Text-to-text,Text-to-speech,Speech-to-text,Speech-to-speech,Image (of words)-to-text
 
 Google-GNMT (Tensorflow),Facebook-fairseq (Torch),Amazon-Sockeye (MXNet),NEMATUS (Theano),THUMT (Theano),OpenNMT (PyTorch),StanfordNMT (Matlab),DyNet-lamtram(CMU),EUREKA(MangoNMT
 
 awesome-gpt3 https://github.com/elyase/awesome-gpt3
 
 Robustness Gym: Evaluation Toolkit for NLP https://github.com/robustness-gym/robustness-gym
 
 https://analyticsindiamag.com/best-nlp-based-seo-tools-for-2021/  https://towardsdatascience.com/5-nlp-models-that-you-need-to-know-about-754594a3225b
 
 https://www.kdnuggets.com/2020/05/best-nlp-deep-learning-course-free.html   https://analyticsindiamag.com/flair-hands-on-guide-to-robust-nlp-framework-built-upon-pytorch/
 
 https://medium.com/modern-nlp/nlp-metablog-a-blog-of-blogs-693e3a8f1e0c
 
 summarization https://github.com/hyunwoongko/summarizers ctrl-sum  https://github.com/salesforce/ctrl-sum

classification,clustering,recommender systems,topic modelling,sentiment analysis,semantic analysis,summarization,machine translation,conversational interface,named entity recognition

F.Time Series Hands-On Guide To Atspy For Automating The Time-Series Forecasting https://awesomeopensource.com/project/Apress/hands-on-time-series-analylsis-python

  here data split is different (train,test,validate)
  
  here handling missing data different 
  
  Time Series Decomposition In Python  trend, seasonality, and noise https://towardsdatascience.com/time-series-decomposition-in-python-8acac385a5b2
  
  generally used  to impute data in Time Series
  
  1.ffill
  
  2.bfill
  
  3.do mean of previous or future x samples and impute
  
  4.take previous season value and impute (data with trend)
  
  5.mean,mode,median,random sample imputation (data without trend and without seasonality)
  
  6.linear interpolation(data with trend and without seasonality)
  
  7.seasonal +interpolation(data with trend and with seasonality)
  
  here model selection deponds on different property of data like stationary,trend,seasonality,cyclic 
  
  Anomaly Detection using Isolation Forest,AutoEncoders
  
  Granger Causality Statistical Test use for variable usable for forecast 
  
  adfuller test  for  Stationarity        Non Stationary Statistical Test - KPSS and ADF
  
  Handling Data with Regular Gaps using Facebook Prophet
  
  models 
  
  1.arma,Arima , auto arima(pmd arima) ,seasonal arima
  
  2.Autoregressive
  
  3.Moving average,Exponential Moving average,Exponential Smoothing,Simple average, Holtโ€™s linear trend method, Holtโ€™s Winter seasonal method
  
  11 Classical Time Series Forecasting Methods in Python https://machinelearningmastery.com/time-series-forecasting-methods-in-python-cheat-sheet/
  
  4.Lstm(neural network)
  
  5.GARCH
  
  atspy	Automated time-series models
  
  6.Navie forecasts
  
  7.Smoothing (moving average,exponential smoothing)
  
  8.Facebook prophet (note:expceted date column as ds and target column as y) https://thecleverprogrammer.com/2020/12/14/facebook-prophet-model-with-python/
  
  NeuralProphet Model- https://ourownstory.github.io/neural_prophet/model-overview/ https://thecleverprogrammer.com/2021/01/28/neuralprophet-model-with-python/
  
  bulbea Deep Learning based Python Library for Stock Market Prediction and Modelling https://github.com/achillesrasquinha/bulbea
  
  PyTorch Forecasting enables deep learning models for time-series forecasting  pytorch-ts https://github.com/zalandoresearch/pytorch-ts
  
  Transformer Networks to build a Forecasting model https://towardsdatascience.com/how-to-use-transformer-networks-to-build-a-forecasting-model-297f9270e630
  
  hmmlearn https://github.com/ushareng/StockPricePredictionUsingHMM_Byte/blob/master/StockPricePredictionUsingHMM.ipynb
  
  pyramid-arima https://github.com/tgsmith61591/pyramid
  
  pyflux: time series library: https://github.com/RJT1990/pyflux
  
  orbit https://eng.uber.com/orbit/ 
  
  greykite A flexible, intuitive and fast forecasting library https://github.com/linkedin/greykite https://www.analyticsvidhya.com/blog/2021/05/greykite-time-series-forecasting-in-python/
  
  Silverkite
  
  LinkedIn open-sources Greykite, a library for time series forecasting  https://github.com/linkedin/greykite/stargazers
  
  stumpy https://github.com/TDAmeritrade/stumpy
  
  Giotto-Time Time-Series Forecasting Python Library https://github.com/giotto-ai/giotto-time https://analyticsindiamag.com/guide-to-giotto-time-a-time-series-forecasting-python-library/
  
  Informer (for Long Sequence Time-Series Forecasting) https://analyticsindiamag.com/informer/ 
  
  tfcausalimpact https://github.com/WillianFuks/tfcausalimpact
  
  deepar is global model https://www.youtube.com/watch?v=xcbj0RE3kfI&list=PL3N9eeOlCrP5cK0QRQxeJd6GrQvhAtpBK&index=14
  
  pmdarima for Auto ARIMA
  
  GluonTS https://github.com/awslabs/gluon-ts
  
  sktime โ€” a unified time-series framework for Scikit-Learn
  
  tsfresh โ€” a magical library for feature extraction in time-series datasets
  
  9.Holts winter,Holts linear trend
  
  10.Auto_Timeseries by auto-ts    https://www.youtube.com/watch?v=URUiVD37fns&list=PL3N9eeOlCrP5cK0QRQxeJd6GrQvhAtpBK&index=24 tell best model for data
  
  AutoTS-https://analyticsindiamag.com/hands-on-guide-to-autots-effective-model-selection-for-multiple-time-series/  https://github.com/AutoViML/Auto_TS
  
  AutoML for time series: advanced approaches with FEDOT framework  https://towardsdatascience.com/automl-for-time-series-advanced-approaches-with-fedot-framework-4f9d8ea3382c

  AutoML for time series: definitely a good idea https://towardsdatascience.com/automl-for-time-series-definitely-a-good-idea-c51d39b2b3f
  
  AutoReg  Regex of string in Python https://github.com/SusmitPanda/AutoReg
  
  pytsal  low-code open-source python framework for Time Series analysis,visualization,forecasting along with AutoTS  https://github.com/KrishnanSG/pytsal
  
  Automated Time Series Forecasting https://github.com/winedarksea/AutoTS
  
  Forecasting with H2O AutoML https://github.com/business-science/modeltime.h2o/
  
  Forecasting Stock Prices Using Stocker  https://medium.com/mlearning-ai/forecasting-stock-prices-using-stocker-7d2ac15966f5
  
  MiniRocket: Fast(er) and Accurate Time Series Classification https://towardsdatascience.com/minirocket-fast-er-and-accurate-time-series-classification-cdacca2dcbfa
  
  modeltime https://github.com/business-science/modeltime
  
  GluonTS , PytorchTS   https://analyticsindiamag.com/gluonts-pytorchts-for-time-series-forecasting/
  
  stocker https://medium.datadriveninvestor.com/forecasting-stock-prices-using-stocker-66503c26307a
  
  11.Temporal Convolutional Neural
  
  12.Atspy For Automating The Time-Series Forecasting-https://analyticsindiamag.com/hands-on-guide-to-atspy-for-automating-the-time-series-forecasting/
  
  13.Darts-https://analyticsindiamag.com/hands-on-guide-to-darts-a-python-tool-for-time-series-forecasting/
  
  14.Bayesian Neural Network , TsEuler
  
  15.PyFlux (easy way to compare different models)-https://analyticsindiamag.com/pyflux-guide-python-library-for-time-series-analysis-and-prediction/
  
  16.Orbit , DeepAR ,NeuralProphet(https://github.com/ourownstory/neural_prophet    https://ourownstory.github.io/neural_prophet/model-overview/)

  IBMโ€™s AutoAI automates time series forecasting https://www.ibm.com/blogs/research/2021/03/autoai-time-series/?utm_campaign=Learning%20Posts&utm_content=159454790&utm_medium=social&utm_source=twitter&hss_channel=tw-3018841323
  
  Kats all in 1  time seres data https://github.com/facebookresearch/kats  https://facebookresearch.github.io/Kats/

  orbit https://analyticsindiamag.com/hands-on-guide-to-orbit-ubers-python-framework-for-bayesian-forecasting-inference/ https://github.com/uber/orbit
  
  best article-https://www.analyticsvidhya.com/blog/2018/02/time-series-forecasting-methods/
  
  TimeSynth https://github.com/TimeSynth/TimeSynth  https://analyticsindiamag.com/guide-to-timesynth-a-python-library-for-synthetic-time-series-generation/
  
  time series visualization tool https://plotjuggler.io/
  
  Time Series Anomaly Detection using Generative Adversarial Networks(TadGAN) https://analyticsindiamag.com/hands-on-guide-to-tadgan-with-python-codes/
  
  fastquant โ€” Backtest and optimize your trading strategies with only 3 lines of code  https://github.com/enzoampil/fastquant 

  pytorch-forecasting  https://github.com/jdb78/pytorch-forecasting  https://analyticsindiamag.com/guide-to-pytorch-time-series-forecasting/ 
  
  https://pytorch-forecasting.readthedocs.io/en/latest/  https://pytorch-forecasting.readthedocs.io/en/latest/tutorials/ar.html
  
  sktime-https://github.com/alan-turing-institute/sktime  https://analyticsindiamag.com/sktime-library/
  
  tsfresh โ€“ a magical library for feature extraction in time-series datasets.
  
  atspy  https://github.com/firmai/atspy
  
  tcn https://towardsdatascience.com/farewell-rnns-welcome-tcns-dd76674707c8
  
  Pastas https://analyticsindiamag.com/guide-to-pastas-a-python-framework-for-hydrogeological-time-series-analysis/ https://github.com/pastas/pastas
  
  stockDL https://github.com/ashishpapanai/stockDL
  
  decompsition https://towardsdatascience.com/time-series-decomposition-in-python-8acac385a5b2
  
  Bayesian Diffusion Modeling  https://www.topbots.com/bayesian-diffusion-modeling/
  
  Top 10 Python Tools For Time Series Analysis https://analyticsindiamag.com/top-10-python-tools-for-time-series-analysis/
  
  fine Tune Your Machine Learning Models To Improve Forecasting Accuracy https://www.kdnuggets.com/2019/01/fine-tune-machine-learning-models-forecasting.html
  
  add extra features https://towardsdatascience.com/the-demand-sales-forecast-technique-every-data-scientist-should-be-using-to-reduce-error-1c6f25add9cb
  
  https://machinelearningmastery.com/time-series-forecasting-methods-in-python-cheat-sheet/
  
  https://www.machinelearningplus.com/time-series/time-series-analysis-python/  https://www.datasciencecentral.com/profiles/blogs/list-of-time-series-methods-in-one-picture
  
  https://github.com/Apress/hands-on-time-series-analylsis-python
  
  https://otexts.com/fpp2/simple-methods.html
      
  https://analyticsindiamag.com/top-time-series-deep-learning-methods/
  
  book https://otexts.com/fpp2/

deep_autoviml Build tensorflow keras model pipelines in a single line of code https://awesomeopensource.com/project/AutoViML/deep_autoviml

G.Semi supervised learning,Self-Supervised Learning,Multi-Instance Learning

self-training meta-estimator for semi-supervised learning

skweak: A Python Toolkit For Applying Weak Supervision To NLP Tasks https://analyticsindiamag.com/meet-skweak-a-python-toolkit-for-applying-weak-supervision-to-nlp-tasks/

10 Self-Supervised Learning Frameworks & Libraries To Use In 2021 analyticsindiamag.com/10-self-supervised-learning-frameworks-libraries-to-use-in-2021/

Self-Supervised Learning https://awesomeopensource.com/project/jason718/awesome-self-supervised-learning

awesome-self-supervised-learning https://awesomeopensource.com/project/jason718/awesome-self-supervised-learning

Self-supervised Video Object Segmentation https://charigyang.github.io/motiongroup/

lightly A python library for self-supervised learning on images https://awesomeopensource.com/project/lightly-ai/lightly

Weak Supervision: The Art Of Training ML Models From Noisy Data https://analyticsindiamag.com/weak-supervision-the-art-of-training-ml-models-from-noisy-data/

snorkel and skweak, are there other libraries to explore for weak supervision in NLP

8 Resources To Learn Self-Supervised Learning In 2021 https://analyticsindiamag.com/top-8-resources-to-learn-self-supervised-learning-in-2021/

Barlow Twins: Self-Supervised Learning via Redundancy Reduction https://analyticsindiamag.com/a-guide-to-barlow-twins-self-supervised-learning-via-redundancy-reduction/ https://awesomeopensource.com/project/facebookresearch/barlowtwins

skweak: A Python Toolkit For Applying Weak Supervision To NLP Tasks https://analyticsindiamag.com/meet-skweak-a-python-toolkit-for-applying-weak-supervision-to-nlp-tasks/

H.Active learning,Multi-Task Learning,Online Learning

Active Learning Frameworks https://towardsdatascience.com/a-summary-of-active-learning-frameworks-3165159baae9

Meta Learning https://awesomeopensource.com/project/sudharsan13296/Awesome-Meta-Learning

Avalanche: A Python Library for Continual Learning https://analyticsindiamag.com/avalanche-a-python-library-for-continual-learning/

Reptile (OpenAIโ€™s Latest Meta-Learning Algorithm) https://awesomeopensource.com/project/openai/supervised-reptile https://analyticsindiamag.com/reptile-openais-latest-meta-learning-algorithm/

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks" https://awesomeopensource.com/project/cbfinn/maml

I.Transfer learning(Inductive Transfer learning(similar domain,different task),Unsupervised Transfer Learning(different task,different domain but similar enough) ,Transductive Transfer Learning(similar task,different domain))

Transfer Learning Using TensorFlow Keras https://analyticsindiamag.com/transfer-learning-using-tensorflow-keras/

https://awesomeopensource.com/project/artix41/awesome-transfer-learning

https://towardsdatascience.com/a-comprehensive-hands-on-guide-to-transfer-learning-with-real-world-applications-in-deep-learning-212bf3b2f27a

J.Deep dream,Style transfer

K.One-shot learning,Zero-shot learning

l.Incremental Training https://blog.rasa.com/rasa-new-incremental-training/

https://awesomeopensource.com/project/ChristosChristofidis/awesome-deep-learning

101 Machine Learning Algorithms for Data Science with Cheat Sheets https://blog-datasciencedojo-com.cdn.ampproject.org/c/s/blog.datasciencedojo.com/machine-learning-algorithms/amp/

TYPES OF ACTIVATION FUNCTIONS: LINEAR ACTIVATION,RELU,LEAKY RELU,GELU,SIGMOID ACTIVATION,TANH ACTIVATION,elu,PReLU,Softmax,Swish,Softplus,Mish

Optimizer- Gradient Descent(Batch Gradient Descent,Stochastic Gradient Descent,Mini batch Gradient Descent),sgd with momentum,Adagrad,RMSProp,AMSGrad,Adam,AdaBelief,MADGRAD,Nero

https://analyticsindiamag.com/ultimate-guide-to-pytorch-optimizers/ https://analyticsindiamag.com/guide-to-tensorflow-keras-optimizers/

Regularization- L1, L2, dropout, early stopping, and data augmentation,batch normalisation,Layer Normalization,Group Normalization,tree purning,DropBlock,DropConnect,Learning rate schedulingWeight Decay,Gradient clipping,Adaptive optimizer

DropBlock: A New Regularization Technique https://pub.towardsai.net/dropblock-a-new-regularization-technique-e926bbc74adb

Learning rate scheduling (Learning rate finder),Weight Decay,Gradient clipping,Cyclic Learning Rate

weight initialization Normal Distribution,initialized to the same value,Xavier Initialization,He Norm Initialization,

Different Normalization Layers - https://towardsdatascience.com/different-normalization-layers-in-deep-learning-1a7214ff71d6

Hyperparameters Number of hidden layers,Dropout,activation function,Weights initialization , learning rate,epoch, iterations and batch size

DropBlock-Keras-Implementation https://awesomeopensource.com/project/iantimmis/DropBlock-Keras-Implementation https://awesomeopensource.com/project/miguelvr/dropblock https://awesomeopensource.com/project/DHZS/tf-dropblock

Hyperparameter tuning

https://analyticsindiamag.com/top-8-approaches-for-tuning-hyperparameters-of-machine-learning-models/  https://analyticsindiamag.com/top-10-open-source-hyperparameter-optimisation-libraries-for-ml-models/

https://github.com/balavenkatesh3322/hyperparameter_tuning

manual search
 
a.GridSearchCV (check every given parameter so take long time)

HalvingGridSearch https://towardsdatascience.com/11-times-faster-hyperparameter-tuning-with-halvinggridsearch-232ed0160155 https://towardsdatascience.com/faster-hyperparameter-tuning-with-scikit-learn-71aa76d06f12

b.RandomizedSearchCV (search randomly narrow down our time) with Scikit-learn, Scikit-Optimize,Hyperopt

HalvingRandomSearchCV

c.Bayesian Optimization , bayes search,Hyperband and BOHB

Bayesian search with Gaussian processes,bayesian search with Random Forests,Bayesian search with GBMs

Bayesian Optimization Using BoTorch https://analyticsindiamag.com/guide-to-bayesian-optimization-using-botorch/

hyperparameter optimization https://github.com/LiYangHart/Hyperparameter-Optimization-of-Machine-Learning-Algorithms

Hyperopt hyperas https://www.kdnuggets.com/2018/12/keras-hyperparameter-tuning-google-colab-hyperas.html

hypertune-using-scikit-optimize  BayesSearchCV

HpBandSter https://github.com/automl/HpBandSter  hpsklearn https://medium.com/mlearning-ai/automatic-hyperparameter-optimization-6a1692c2ebee

hypopt  https://github.com/cgnorthcutt/hypopt https://medium.com/mlearning-ai/automatic-hyperparameter-optimization-6a1692c2ebee

HiPlot https://analyticsindiamag.com/this-new-tool-helps-developers-in-effective-hyperparameter-tuning/

OCTIS https://github.com/mind-lab/octis

hyperband  https://neptune.ai/blog/hyperband-and-bohb-understanding-state-of-the-art-hyperparameter-optimization-algorithms

Spearmint  https://github.com/JasperSnoek/spearmint/

NeuPy http://neupy.com/2016/12/17/hyperparameter_optimization_for_neural_networks.html#id24

Vizier

ConfigSpace https://automl.github.io/ConfigSpace/master/ https://towardsdatascience.com/tuning-xgboost-with-xgboost-writing-your-own-hyper-parameters-optimization-engine-a593498b5fba

NatureInspiredSearchCV https://github.com/timzatko/Sklearn-Nature-Inspired-Algorithms

d.Sequential Model Based Optimization(Tuning a scikit-learn estimator with skopt)

e.Optuna  https://analyticsindiamag.com/hands-on-python-guide-to-optuna-a-new-hyperparameter-optimization-tool/

f.Genetic Algorithms

darwin-mendel Genetic Algorithm for Hyper-Parameter Tuning https://manishagrawal-datascience.medium.com/genetic-algorithm-for-hyper-parameter-tuning-1ca29b201c08

g.Keras tuner (Random Search Keras Tuner,HyperBand Keras Tuner,Bayesian Optimization Keras Tuner,Hyperas  ) https://sukanyabag.medium.com/automated-hyperparameter-tuning-with-keras-tuner-and-tensorflow-2-0-31ec83f08a62

Keras Hyperparameter Tuning with aisaratuners Library https://aisaradeepwadi.medium.com/advance-keras-hyperparameter-tuning-with-aisaratuners-library-78c488ab4d6a

storm-tuner https://github.com/ben-arnao/StoRM https://medium.com/geekculture/finding-best-hyper-parameters-for-deep-learning-model-4df7a17546c2

Hyperas  https://towardsdatascience.com/automating-hyperparameter-tuning-of-keras-model-4fe69b8dedee

Deep AutoViML https://github.com/AutoViML/deep_autoviml

h.Scikit-Optimize

i.ray[tune] and aisaratuners https://towardsdatascience.com/choosing-a-hyperparameter-tuning-library-ray-tune-or-aisaratuners-b707b175c1d7

raytune https://docs.ray.io/en/master/tune/index.html

k.model_search https://github.com/google/model_search https://analyticsindiamag.com/hands-on-guide-to-model-search-a-tensorflow-based-framework-for-automl/

Optimize machine learning models https://www.tensorflow.org/model_optimization

Milano   https://github.com/NVIDIA/Milano

Tree-structured Parzen Estimators - TPE

Hyperparameter Tuning with the HParams Dashboard

baytune https://www.kdnuggets.com/2021/03/automating-machine-learning-model-optimization.html

Dragonfly https://analyticsindiamag.com/guide-to-scalable-and-robust-bayesian-optimization-with-dragonfly/

Pywedge https://www.analyticsvidhya.com/blog/2021/02/interactive-widget-based-hyperparameter-tuning-and-tracking-in-pywedge/ 

CapsNet Hyperparameter Tuning with Keras https://towardsdatascience.com/scikeras-tutorial-a-multi-input-multi-output-wrapper-for-capsnet-hyperparameter-tuning-with-keras-3127690f7f28

GPyTorch: A Python Library For Gaussian Process Models https://analyticsindiamag.com/guide-to-gpytorch-a-python-library-for-gaussian-process-models/

Auto-PyTorch https://github.com/automl/Auto-PyTorch

l.SMAC https://www.automl.org/automated-algorithm-design/algorithm-configuration/smac/  https://towardsdatascience.com/automl-for-fast-hyperparameters-tuning-with-smac-4d70b1399ce6

m.faster Hyper Parameter Tuning(sklearn-nature-inspired-algorithms) https://pypi.org/project/sklearn-nature-inspired-algorithms/

n.talos Neural network and hyperparameter optimization using Talos https://www.analyticsvidhya.com/blog/2021/05/neural-network-and-hyperparameter-optimization-using-talos/

https://towardsdatascience.com/10-hyperparameter-optimization-frameworks-8bc87bc8b7e3 

https://mlwhiz.com/blog/2020/02/22/hyperspark/?utm_campaign=100x-faster-hyperparameter-search-framework-with-pyspark&utm_medium=social_link&utm_source=missinglettr

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective https://github.com/microsoft/DeepSpeed

o.shap-hypetune https://github.com/cerlymarco/shap-hypetune https://towardsdatascience.com/shap-for-feature-selection-and-hyperparameter-tuning-a330ec0ea104

mlmachine,Polyaxon,BayesianOptimization,Talos,SHERPA,Scikit-Optimize,GPyOpt

p.Hyperactive https://github.com/SimonBlanke/Hyperactive
   
Hyperopt, Optuna, and Ray,SCIKIT-OPTIMIZE,SMAC

HyperOpt  http://hyperopt.github.io/hyperopt/   Optuna  https://optuna.org/   Scikit-optimize https://scikit-optimize.github.io/stable/  SigOpt  https://sigopt.com/

Cross validation techniques- https://towardsdatascience.com/understanding-8-types-of-cross-validation-80c935a4976d

 1.Loocv
 
 2.Kfoldcv
 
 3.Stratfied cross validation

 Stratified K-folds  
 
 4.Time Series cross-validation
 
 5.Holdout cross-validation
 
 6.Repeated cross-validation

 Repeated K-folds
 
 7.Leave P out ,Leave-one-out cross-validation

 8.Time Series cross-validation

 9.ShuffleSplit Cross Validation

 10.Group KFold Cross Validation

Tensorboard,Neptune,TensorFlow Profiler to visualization of model performance

Distributed Training with TensorFlow

6.Testing model

Text Robustness Evaluation Platform https://awesomeopensource.com/project/textflint/textflint

Generally used metrics

 Always check bias variance tradeoff to know how model is performing
 
 Locust Performance Testing ML Serving APIs With Locust https://www.analyticsvidhya.com/blog/2021/06/performance-testing-ml-serving-apis-with-locust/
 
 Model can be overfitting(low bias,high variance),underfitting(high bias,high variance),good fit(low bias,low variance)
 
 https://scikit-learn.org/stable/modules/model_evaluation.html   https://scikit-learn.org/stable/modules/classes.html#module-sklearn.linear_model
 
 https://stanford.edu/~shervine/teaching/cs-229/cheatsheet-machine-learning-tips-and-tricks
 
1.Regression task - mean-squared-error, Root-Mean-Squared-Error,mean-absolute error, Rยฒ, Adjusted Rยฒ,Cross-entropy loss,Mean percentage error 

2.Classification task-Accuracy,confusion matrix,Precision,Recall,F1 Score,Binary Crossentropy,Categorical Crossentropy,AUC-ROC curve,log loss,Average precision,Mean average precision

3.Reinforcement learning - generally  use rewards

4.Incase of machine translation use bleu score

5.Clustering then use External: Adjusted Rand index, Jaccard Score, Purity Score    Internal:silhouette_score, Davies-Bouldin Index, Dunn Index

https://towardsdatascience.com/performance-metrics-in-machine-learning-part-3-clustering-d69550662dc6

6.Object Detection loss-localization loss,classification loss,Focal Loss,IOU,L2 loss

7.Distance Metrics - Euclidean Distance,Manhattan Distance,Minkowski Distance,Hamming Distance  https://towardsdatascience.com/9-distance-measures-in-data-science-918109d069fa

8.Recommender Systems  https://parthchokhra.medium.com/evaluating-recommender-systems-590a7b87afa5

metric-Built-in metrics, Custom metric without external parameters,Custom metric with external parameters,Subclassing custom metric layer

Robustness Gym: Evaluation Toolkit for NLP https://github.com/robustness-gym/robustness-gym

https://medium.com/swlh/custom-loss-and-custom-metrics-using-keras-sequential-model-api-d5bcd3a4ff28

loss-Built-in loss, Custom loss without external parameters,Custom loss with external parameters,Subclassing loss layer

https://analyticsindiamag.com/all-pytorch-loss-function/   https://analyticsindiamag.com/ultimate-guide-to-loss-functions-in-tensorflow-keras-api-with-python-implementation/

tensorwatch  Debugging, monitoring and visualization for Python Machine Learning and Data Science https://github.com/microsoft/tensorwatch

Docker and Kubernetes

https://towardsdatascience.com/deploy-machine-learning-app-built-using-streamlit-and-pycaret-on-google-kubernetes-engine-fd7e393d99cb

simplest way to serve your ML models on Kubernetes https://towardsdatascience.com/the-simplest-way-to-serve-your-ml-models-on-kubernetes-5323a380bf9f

7.deployment https://awesomeopensource.com/project/piyushpathak03/Model-Deployment

Continuously Monitor the Behaviour of Deployed Models  https://se-ml.github.io/best_practices/04-monitor_models_prod/ 

Model Monitoring https://www.kdnuggets.com/2021/03/machine-learning-model-monitoring-checklist.html

Automate Model Deployment https://se-ml.github.io/best_practices/04-auto_model_packaging/

Platform as a Service (PaaS),Infrastructure as a Service (IaaS),SaaS (Software as a Service)

3 main approaches of Saving and Reloading an ML Model-Pickle Approach,Joblib Approach,JSON approach  

https://www.datacamp.com/community/tutorials/pickle-python-tutorial  https://github.com/balavenkatesh3322/model_deployment

1.Azure

2.Heroku

3.Amazon Web Services   Everything AWS  https://app.polymersearch.com/discover/aws

4.Google cloud platform

5.ngrok https://www.youtube.com/watch?v=AkEnjJ5yWV0

Deploy a Machine Learning Model for Free https://www.freecodecamp.org/news/deploy-your-machine-learning-models-for-free/

mlpack is a fast, flexible machine learning library suitable for both data science prototyping and deployment https://numfocus.org/project/mlpack  https://github.com/mlpack/mlpack

MODEL DEPLOYMENT USING TF SERVING

Dockerize  https://www.kdnuggets.com/2021/04/dockerize-any-machine-learning-application.html  https://pub.towardsai.net/how-to-dockerize-your-data-science-project-a-quick-guide-b6fa2d6a8ba1

bodywork-core MLOps tool for deploying machine learning projects to Kubernetes https://github.com/bodywork-ml/bodywork-core

Create ML model inside the docker container https://dev.to/niteshthapliyal/create-ml-model-inside-the-docker-container-3b23

LyftLearn: ML Model Training Infrastructure built on Kubernetes https://eng.lyft.com/lyftlearn-ml-model-training-infrastructure-built-on-kubernetes-aef8218842bb

Model Serving https://neptune.ai/blog/ml-model-serving-best-tools?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-ml-model-serving-best-tools
 
TensorFlow Extended (TFX) is an end-to-end platform for deploying production ML pipelines https://www.tensorflow.org/tfx  https://theaisummer.com/tfx/?utm_content=163294295&utm_medium=social&utm_source=linkedin&hss_channel=lcp-42461735

torchblaze https://github.com/MLH-Fellowship/torchblaze https://mlh-fellowship.github.io/torchblaze/

ML Aide Manage Machine Learning Lifecycle  https://mlaide.com/home https://medium.com/ml-aide/manage-machine-learning-lifecycle-with-ml-aide-dfe7710cbe53

Models visualization using Tensorboard,netron, TensorBoard.dev

Python web Frameworks for App Development- Flask,Streamlit,fastapi,Django,Web2py,Pyramid,CherryPy,Voila,Kivy and Kivymd  

streamlit,plotly jupyterdash,h2o wave,dash,gradio,PyWebIO,r shiny,sanic,panel,flask,django

Django models https://www.deploymachinelearning.com/#create-django-models  https://www.deploymachinelearning.com/

panel high-level app and dashboarding solution for Python https://github.com/holoviz/panel

sanic https://github.com/sanic-org/sanic

Gradio - take input from user  https://gradio.app/getting_started

Netlify  https://www.analyticsvidhya.com/blog/2021/04/easily-deploy-your-machine-learning-model-into-a-web-app-netlify/

streamlit https://www.kdnuggets.com/2019/10/write-web-apps-using-simple-python-data-scientists.html  https://www.youtube.com/watch?v=iUgNIFrVejc   https://blog.streamlit.io/introducing-theming/

Streamlit Flask App from Colab using remoteit and ngrok https://www.youtube.com/watch?v=O2enoygZwl4

Streamlit to databases https://docs.streamlit.io/en/0.83.0/tutorial/databases.html

https://github.com/jrieke/best-of-streamlit

https://neptune.ai/blog/streamlit-guide-machine-learning?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-streamlit-guide-machine-learning

streamlit-ace https://github.com/okld/streamlit-ace  https://www.youtube.com/watch?v=Iv2vt-7AYNQ 

customize the themes of your Streamlit web apps https://www.youtube.com/watch?v=3xJYP_C4KNE https://github.com/khuyentran1401/Data-science/tree/master/applications/pywebio_examples

colab_everything Python library to run streamlit, flask, fastapi, etc on google colab  https://github.com/Ankur-singh/colab_everything/

dash https://github.com/plotly/dash

panel-highcharts https://awesome-panel.org/  https://github.com/marcskovmadsen/panel-highcharts https://github.com/holoviz/panel https://github.com/holoviz/panel

opyrator Turns your machine learning code into microservices with web API, interactive GUI, and more  https://github.com/ml-tooling/opyrator

plotly https://plotly.com/  https://analyticsindiamag.com/how-to-use-plotly-in-colab/

Creating a Machine Learning App with Power BI and PyCaret

Streamlit vs. Dash vs. Shiny vs. Voila vs. Flask vs. Jupyter vs django vs PySimpleGUIvs pywebio vs Gradio   https://www.datarevenue.com/en-blog/data-dashboarding-streamlit-vs-dash-vs-shiny-vs-voila

Quickly deploy ML WebApps https://ngrok.com/

Deepnote https://deepnote.com/  https://www.youtube.com/watch?v=0ppptVxgEI8

booklet https://booklet.ai/  https://towardsdatascience.com/building-a-covid-19-project-recommendation-system-4607806923b9

https://analyticsindiamag.com/top-8-python-tools-for-app-development/

Voila This library can turn your Jupyter notebooks into standalone web apps that can be deployed to any cloud platform.  https://voila.readthedocs.io/en/stable/

H2O.ai https://www.h2o.ai/blog/data-to-production-ready-models-to-business-apps-in-just-a-few-steps/ 

PyQt and Tkinter , PySimpleGUI are GUI programming in Python  https://github.com/tirthajyoti/DS-with-PySimpleGUI

DearPyGui https://github.com/hoffstadt/DearPyGui

PySimpleGUI https://github.com/PySimpleGUI/PySimpleGUI

Gooey Turn (almost) any Python command line program into a full GUI application with one line https://github.com/chriskiehl/Gooey

snapyml Deploy AI Models For Free -http://snapyml.snapy.ai/

BentoML https://github.com/bentoml/BentoML

h20 wave-apps https://github.com/h2oai/wave-apps  https://h2oai.github.io/wave/docs/installation/  https://h2oai.github.io/wave/

h20 Wave ML (AutoML for Wave Apps) https://h2oai.github.io/wave/blog/ml-release-0.3.0/

fastapi https://towardsdatascience.com/deploying-ml-models-in-production-with-fastapi-and-celery-7063e539a5db

FastAPI + Uvicorn https://www.kdnuggets.com/2021/04/deploy-machine-learning-models-to-web.html  

FastAPI based template https://github.com/99sbr/fastapi-template  fastapi-log 0.0.3 https://pypi.org/project/fastapi-log/

chitra ๐—–๐—ฟ๐—ฒ๐—ฎ๐˜๐—ฒ ๐—”๐—ฃ๐—œ ๐—ณ๐—ผ๐—ฟ ๐—”๐—ป๐˜† ๐—Ÿ๐—ฒ๐—ฎ๐—ฟ๐—ป๐—ถ๐—ป๐—ด ๐— ๐—ผ๐—ฑ๐—ฒ๐—น  https://github.com/aniketmaurya/chitra

PyWebIO Write Interactive Web App in Script Way Using Python https://www.youtube.com/watch?v=vp1ZNapAy6Y  https://towardsdatascience.com/pywebio-write-interactive-web-app-in-script-way-using-python-14f50155af4e  https://github.com/tirthajyoti/PyWebIO

Katana https://github.com/shaz13/katana https://katana-demo.herokuapp.com/redoc  https://katana-demo.herokuapp.com/docs

DS-with-PySimpleGUI  https://github.com/tirthajyoti/DS-with-PySimpleGUI

pywinauto Windows GUI Automation with Python

tkinter to deploy machine learning model-https://analyticsindiamag.com/complete-tutorial-on-tkinter-to-deploy-machine-learning-model/

Tkinter-Designer Create Beautiful Tkinter GUIs by Drag and Drop https://github.com/ParthJadhav/Tkinter-Designer

Web-Based GUI (Gradio)- https://analyticsindiamag.com/guide-to-gradio-create-web-based-gui-applications-for-machine-learning/  https://www.gradio.app/

Bamboolib https://medium.com/ai-in-plain-english/bamboolib-a-data-warriors-weapon-9f734f4c2553

web application(dash)- https://dash.plotly.com/

Pyramid web framework https://trypyramid.com/documentation.html

Kivy /Kivymd creating an android app 

https://towardsdatascience.com/pycaret-2-1-is-here-whats-new-4aae6a7f636a

Create a Website with AI https://www.bookmark.com/ 

localhost to globalurl https://ngrok.com/  https://remote.it/
 
Jupyter Notebook into an interactive dashboard (voila)-https://voila.readthedocs.io/en/stable/

high-level app and dashboarding solution(Panel)-https://panel.holoviz.org/

MaaS Build ML Models As A Service  https://www.analyticsvidhya.com/blog/2021/05/maas-build-ml-models-as-a-service/

https://github.com/gradio-app/gradio

Tensorflow lite:Use of tensorflow lite to reduce size of model https://www.tensorflow.org/lite https://codelabs.developers.google.com/codelabs/recognize-flowers-with-tensorflow-on-android-beta/#0 https://tfhub.dev/s?deployment-format=lite https://www.tensorflow.org/lite/examples https://www.tensorflow.org/lite/microcontrollers https://www.tensorflow.org/lite/models

Adventures-in-TensorFlow-Lite https://awesomeopensource.com/project/sayakpaul/Adventures-in-TensorFlow-Lite

coral https://coral.ai/docs/edgetpu/models-intro/

TF Micro and SensiML https://blog.tensorflow.org/2021/05/building-tinyml-application-with-tf-micro-and-sensiml.html

six different types of methods:

  1. Pruning, Weight sharing
  2. Quantization ,TensorFlow Quantum Post-Training Quantization โ€” Reduce Float16 โ€” Hybrid Quantization โ€” Integer Quantization -dynamic range quantization 2. During-Training Quantization 3. Post-Training Pruning 4. Post-Training Clustering
  3. Knowledge distillation
  4. Parameter sharing
  5. Tensor decomposition
  6. Linear Transformer,Winograd Transformation
  7. Selective attention
  8. Low-rank factorisation
  9. 3LC https://research.google/pubs/pub47962/
  10. brevitas https://github.com/Xilinx/brevitas/
  11. aimet https://awesomeopensource.com/project/quic/aimet

QKeras: a quantization deep learning library for Tensorflow Keras

Knowledge Distillation knowledge are categorized into three different types: Response-based knowledge, Feature-based knowledge, and Relation-based knowledge three principal types of methods for training student and teacher models, namely offline, online and self distillation.

Distillation library KD_Lib https://awesomeopensource.com/project/SforAiDl/KD_Lib

ibm new tool https://www.zdnet.com/article/ibms-new-tool-lets-developers-add-quantum-computing-power-to-machine-learning/

qiskit-machine-learning https://awesomeopensource.com/project/Qiskit/qiskit-machine-learning https://qiskit.org/documentation/machine-learning/stubs/qiskit_machine_learning.neural_networks.SamplingNeuralNetwork.html

compressors https://awesomeopensource.com/project/elephantmipt/compressors

https://rachitsingh.com/deep-learning-model-compression/#quantization

model optimization (architecture)

TF Lite with iOS, Swift and TF Lite Swift

TinyML https://blog.tensorflow.org/2020/08/the-future-of-ml-tiny-and-bright.html

tinyml-papers-and-projects This is a list of interesting papers and projects about TinyML https://awesomeopensource.com/project/gigwegbe/tinyml-papers-and-projects

pennylane Python library for differentiable programming of quantum computers https://awesomeopensource.com/project/PennyLaneAI/pennylane

AI Engine for Edge Devices https://awesomeopensource.com/project/johnolafenwa/deepstack TensorFlow Lite Samples on Unity https://awesomeopensource.com/project/asus4/tf-lite-unity-sample

tflite-support TFLite Support is a toolkit that helps users to develop ML and deploy TFLite models onto mobile / ioT devices https://awesomeopensource.com/project/tensorflow/tflite-support

Post-training Quantization in TensorFlow Lite https://www.tensorflow.org/lite/performance/post_training_quantization

pruning

Custom Text Classification on Android using TensorFlow Lite https://www.analyticsvidhya.com/blog/2021/05/custom-text-classification-on-android-using-tensorflow-lite/

aimet advanced quantization and compression techniques for trained neural network models https://awesomeopensource.com/project/quic/aimet https://awesomeopensource.com/project/quic/aimet-model-zoo

Automatic Model Compression (AutoMC) framework for developing smaller and faster AI applications https://awesomeopensource.com/project/Tencent/PocketFlow

leverage of model architecture

Federated Learning https://www.analyticsvidhya.com/blog/2021/04/federated-learning-for-beginners/ https://www.tensorflow.org/federated

FEDERATED LEARNING https://blog.openmined.org/federated-learning-types/

Federated Learning with FEDn https://awesomeopensource.com/project/scaleoutsystems/fedn

plato scalable federated learning research framework https://awesomeopensource.com/project/TL-System/plato

FedNLP: A Research Platform for Federated Learning in Natural Language Processing https://awesomeopensource.com/project/FedML-AI/FedNLP

privacy https://awesomeopensource.com/project/tensorflow/privacy

Quantization:Use Quantization to reduce size of model https://medium.com/qiskit/introducing-qiskit-machine-learning-5f06b6597526

TensorFlow Quantum https://www.tensorflow.org/quantum

Qiskit Machine Learning https://awesomeopensource.com/project/Qiskit/qiskit-machine-learning

Quantum Machine Learning

Quantum Kernels https://github.com/Qiskit/qiskit-machine-learning/blob/master/docs/tutorials/03_quantum_kernel.ipynb

IBMs Qiskit,Googleโ€™s Cirq,Amazonโ€™s AWS Braket,Microsoftโ€™s Q# and Azure Quantum,Rigettiโ€™s Forest,Xanaduโ€™s Pennylane

On-Device Machine Learning https://developers.google.com/learn/topics/on-device-ml https://www.tensorflow.org/lite/guide/model_maker

Core ML for iOS, Tensorflow lite for Android, ML.NET for Windows and ML Kit

8.Mointoring model

CI CD pipeline used- circleci , jenkins

In real world project use pipeline -https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html

1.easy debugging

2.better readability

BIG DATA: hadoop,apache spark

project structure

data science project structure https://towardsdatascience.com/automate-your-data-science-project-structure-in-three-easy-steps-277c92328d24

research paper-https://arxiv.org/ ,https://arxiv.org/list/cs.LG/recent, https://www.kaggle.com/Cornell-University/arxiv

arXiv.org https://arxiv.org/list/cs.AI/recent https://arxiv.org/list/stat.ML/recent https://arxiv.org/list/cs.CL/recent https://arxiv.org/list/cs.CV/recent

https://awesomeopensource.com/project/amitness/papers-with-video

Datasets on arXiv https://medium.com/paperswithcode/datasets-on-arxiv-1a5a8f7bd104

code for research paper https://www.analyticsvidhya.com/blog/2021/06/steal-the-code-ethically-get-better-at-ml-ai-research/

papertalk https://papertalk.org/index

connected papers https://www.connectedpapers.com/

Stanford AI Lab Papers and Talks at ICLR 2021 https://ramseyelbasheer.io/2021/05/03/stanford-ai-lab-papers-and-talks-at-iclr-2021/

Semantic Scholar searches: https://www.semanticscholar.org/search?q=%22neural%20networks%22&sort=relevance&ae=false

https://www.semanticscholar.org/search?q=%22machine%20learning%22&sort=relevance&ae=false

https://www.semanticscholar.org/search?q=%22natural%20language%22&sort=relevance&ae=false

https://www.semanticscholar.org/search?q=%22computer%20vision%22&sort=relevance&ae=false

https://www.semanticscholar.org/search?q=%22deep%20learning%22&sort=relevance&ae=false

code for Research Papers-https://chrome.google.com/webstore/detail/find-code-for-research-pa/aikkeehnlfpamidigaffhfmgbkdeheil

Summarise Research Papers - https://www.semanticscholar.org/

Structure Your Data Science Projects https://towardsdatascience.com/structure-your-data-science-projects-6c6c8653c16a

programming language for data science is Python,R,Julia,Java,Scala,JAVA SCRIPT(Tensorflow.js),etc...

IDE:jupyter notebook,spyder,pycharm,visual studio

4 Tools for Reproducible Jupyter Notebooks https://towardsdatascience.com/4-tools-for-reproducible-jupyter-notebooks-d7423721bd04

12 Jupyter Notebook Extensions That Will Make Your Life Easier https://towardsdatascience.com/12-jupyter-notebook-extensions-that-will-make-your-life-easier-e0aae0bd181

BEST ONLINE COURSES

1.COURSERA

2.UDEMY

3.EDX

4.DATACAMP

5.Udacity

6.https://www.skillbasics.com/

BEST YOUTUBE CHANNEL TO FOLLOW

1.Krish Naik-https://www.youtube.com/user/krishnaik06

2.Codebasics-https://www.youtube.com/channel/UCh9nVJoWXmFb7sLApWGcLPQ  

3.Abhishek thakur-https://www.youtube.com/user/abhisheksvnit

4.AIEngineering-https://www.youtube.com/channel/UCwBs8TLOogwyGd0GxHCp-Dw

5.Ineuron-https://www.youtube.com/channel/UCb1GdqUqArXMQ3RS86lqqOw

6.Ken jee-https://www.youtube.com/c/KenJee1/featured       

7.3Blue1Brown-https://www.youtube.com/c/3blue1brown/featured

8.The AI Guy -https://www.youtube.com/channel/UCrydcKaojc44XnuXrfhlV8Q 

9.Unfold Data Science-https://www.youtube.com/channel/UCh8IuVJvRdporrHi-I9H7Vw  etc...

BEST BLOGS TO FOLLOW

https://www.cybrhome.com/topic/data-science-blogs    

AI Summary https://ai-summary.com/

https://www.datasciencecentral.com/profiles/blog/list  https://developer.nvidia.com/blog/?ncid=em-prom-48627

1.Towards data science-https://towardsdatascience.com/

2.Analyticsvidhya-https://www.analyticsvidhya.com/blog/?utm_source=feed&utm_medium=navbar       https://analyticsindiamag.com/  https://www.analyticsinsight.net/

3.Medium-https://medium.com/

4.Machinelearningmastery-https://machinelearningmastery.com/blog/

5.ML+  -https://www.machinelearningplus.com/

6.analyticsinsight https://www.analyticsinsight.net/category/latest-news/    https://www.analyticsinsight.net/

7.KDnuggets https://www.kdnuggets.com/  https://www.kdnuggets.com/news/index.html   

8.Artificial Intelligence Database https://www.wired.com/category/artificial-intelligence/?verso=true

https://machinelearningknowledge.ai/   

https://github.com/rushter/data-science-blogs

https://www.datamuni.com/

https://blog.ml.cmu.edu/?utm_source=towardsai.net&utm_medium=referral&utm_campaign=marketing&utm_term=machine-learning-blog&utm_content=best-machine-learning-blogs-to-follow

https://www.amazon.science/blog?utm_source=towardsai.net&utm_medium=referral&utm_campaign=marketing&utm_term=machine+learning+blog&utm_content=machine+learning+blog&f0=0000016e-2ff1-d205-a5ef-aff9651e0000&s=0

https://distill.pub/?utm_source=towardsai.net&utm_medium=referral&utm_campaign=marketing&utm_term=machine-learning-blog&utm_content=best-machine-learning-blogs-to-follow

https://ai.googleblog.com/search/label/Machine%20Learning?utm_source=towardsai.net&utm_medium=referral&utm_campaign=marketing&utm_term=machine-learning-blog&utm_content=best-machine-learning-blogs-to-follow

https://neptune.ai/blog?utm_source=towardsai.net&utm_medium=referral&utm_campaign=marketing&utm_term=machine+learning+blog&utm_content=machine+learning+blog

https://bair.berkeley.edu/blog/?utm_source=towardsai.net&utm_medium=referral&utm_campaign=marketing&utm_term=machine-learning-blog&utm_content=best-machine-learning-blogs-to-follow

https://deepmind.com/research?utm_source=towardsai.net&utm_medium=referral&utm_campaign=marketing&utm_term=machine-learning-blog&utm_content=machine-learning-blogs-to-follow&filters=%7B%22category%22:%5B%22Research%22%5D%7D

https://ai.facebook.com/blog/?utm_source=towardsai.net&utm_medium=referral&utm_campaign=marketing&utm_term=machine-learning-blog&utm_content=machine-learning-blogs-to-follow

https://becominghuman.ai/top-25-ai-and-machine-learning-blogs-for-data-scientists-9f121bcfd9a2

https://medium.com/towards-artificial-intelligence/best-machine-learning-blogs-to-follow-ml-research-ai-3994e01967f9

BEST RESOURCES

https://amitness.com/toolbox/ https://khuyentran1401.github.io/Data-science/ https://awesomeopensource.com/project/ml-tooling/best-of-ml-python

https://awesomeopensource.com/project/ml-tooling/best-of-ml-python#machine-learning-frameworks http://dfkoz.com/ai-data-landscape/ https://landscape.lfai.foundation/

https://towardsdatascience.com/data-science-tools-f16ecd91c95d https://mathdatasimplified.com/ https://awesomeopensource.com/project/neomatrix369/awesome-ai-ml-dl

https://amitness.com/ https://postsyoumighthavemissed.com/search/

1.paperswithcode-https://paperswithcode.com/methods https://www.paperswithcode.com/datasets

paperswithcode-client https://awesomeopensource.com/project/paperswithcode/paperswithcode-client https://paperswithcode.com/lib/torchvision

https://www.connectedpapers.com/main/4f2eda8077dc7a69bb2b4e0a1a086cf054adb3f9/EfficientNet-Rethinking-Model-Scaling-for-Convolutional-Neural-Networks/graph

2.madewithml-https://madewithml.com/topics/ https://madewithml.com/courses/applied-ml-in-production/ https://awesomeopensource.com/project/GokuMohandas/applied-ml

modelzoo https://modelzoo.co/

Weights & Biases- https://wandb.ai/gallery sotabench-https://sotabench.com/

3.Deep learning-https://course.fullstackdeeplearning.com/#course-content

4.pytorch deep learning-https://atcold.github.io/pytorch-Deep-Learning/

PYTORCH HUB https://pytorch.org/hub/ https://pytorch.org/hub/research-models

5.https://papers.labml.ai/papers/daily https://42papers.com/

https://www.kdnuggets.com/2019/08/pytorch-cheat-sheet-beginners.html https://www.kdnuggets.com/2019/04/nlp-pytorch.html https://www.kdnuggets.com/2019/08/9-tips-training-lightning-fast-neural-networks-pytorch.html

fairscale PyTorch extensions for high performance and large scale training https://awesomeopensource.com/project/facebookresearch/fairscale

PyTorch Lightning-https://awesomeopensource.com/project/PyTorchLightning/pytorch-lightning https://www.kdnuggets.com/2020/11/deploy-pytorch-lightning-models-production.html

https://pytorch-lightning.medium.com/lightning-flash-0-3-new-tasks-visualization-tools-data-pipeline-and-flash-registry-api-1e236ba9530

PYTORCH - https://pytorch.org/ https://pytorch.org/ecosystem/ https://pytorch.org/tutorials/ https://pytorch.org/docs/stable/index.html https://awesomeopensource.com/project/pytorch/pytorch

PYTORCH Lightning https://pytorchlightning.ai/community#projects https://seannaren.medium.com/introducing-pytorch-lightning-sharded-train-sota-models-with-half-the-memory-7bcc8b4484f2

ort Accelerate PyTorch models with ONNX Runtime https://awesomeopensource.com/project/pytorch/ort

lightning-flash https://awesomeopensource.com/project/PyTorchLightning/lightning-flash https://pytorch-lightning.medium.com/introducing-lightning-flash-the-fastest-way-to-get-started-with-deep-learning-202f196b3b98

torchflare easy-to-use PyTorch Framework https://awesomeopensource.com/project/Atharva-Phatak/torchflare

Lightning Bolts collection of well established, SOTA models and components https://awesomeopensource.com/project/PyTorchLightning/lightning-bolts

Sharded: A New Technique To Double The Size Of PyTorch Models https://towardsdatascience.com/sharded-a-new-technique-to-double-the-size-of-pytorch-models-3af057466dba

๐—ข๐—ฝ๐—ฎ๐—ฐ๐˜‚๐˜€ (๐˜๐—ฟ๐—ฎ๐—ถ๐—ป๐—ถ๐—ป๐—ด ๐—ฃ๐˜†๐—ง๐—ผ๐—ฟ๐—ฐ๐—ต ๐—บ๐—ผ๐—ฑ๐—ฒ๐—น๐˜€ ๐˜„๐—ถ๐˜๐—ต ๐—ฑ๐—ถ๐—ณ๐—ณ๐—ฒ๐—ฟ๐—ฒ๐—ป๐˜๐—ถ๐—ฎ๐—น ๐—ฝ๐—ฟ๐—ถ๐˜ƒ๐—ฎ๐—ฐ๐˜†)-https://opacus.ai/

light-face-detection https://awesomeopensource.com/project/borhanMorphy/light-face-detection

DALLE-pytorch https://awesomeopensource.com/project/lucidrains/DALLE-pytorch

PyTorch JIT -https://lernapparat.de/jit-optimization-intro/

jax- https://awesomeopensource.com/project/google/jax

incubator-mxnet - https://awesomeopensource.com/project/apache/incubator-mxnet

ignite-https://awesomeopensource.com/project/pytorch/ignite

fastText - https://awesomeopensource.com/project/facebookresearch/fastText

rapidminer-https://rapidminer.com/

5.deep-learning-drizzle-https://deep-learning-drizzle.github.io/ https://deep-learning-drizzle.github.io/index.html

6.Fastaibook-https://awesomeopensource.com/project/fastai/fastbook , https://course.fast.ai/ https://www.fast.ai/2019/07/08/fastai-nlp/ https://www.fast.ai/2020/08/21/fastai2-launch/

neptune.ai-https://docs.neptune.ai/index.html

Dive into Deep Learning http://d2l.ai/

7.TopDeepLearning-https://awesomeopensource.com/project/aymericdamien/TopDeepLearning

8.NLP-progress-https://awesomeopensource.com/project/sebastianruder/NLP-progress

9.EasyOCR-https://awesomeopensource.com/project/JaidedAI/EasyOCR textract,pytesseract, pyzbar, and pyocr,OCR With Detectron2,PymuPDF,Camelot,keras ocr,PDFTableExtract(by PyPDF2),tesseract-ocr,PyMuPDF,pyocr,Apache Tika,pdfPlumber,pdfMiner3,TextOCR,keras-CTPN,pytorch-CTPN,ocr.pytorch,layout-parser,tabula-py,Spark OCR,mmocr,Amazon Textract,Azure OCR, Google OCR,

10.Awesome-pytorch-list-https://awesomeopensource.com/project/bharathgs/Awesome-pytorch-list https://shivanandroy.com/awesome-nlp-resources/

11.free-data-science-books-https://awesomeopensource.com/project/chaconnewu/free-data-science-books

12.arcgis-https://awesomeopensource.com/project/Esri/arcgis-python-api https://geemap.org/

13.data-science-ipython-notebooks-https://awesomeopensource.com/project/donnemartin/data-science-ipython-notebooks

14.julia-https://awesomeopensource.com/project/JuliaLang/julia , https://docs.julialang.org/en/v1/

15.google-research-https://awesomeopensource.com/project/google-research/google-research

16.reinforcement-learning-https://awesomeopensource.com/project/dennybritz/reinforcement-learning

17.keras-applications-https://awesomeopensource.com/project/keras-team/keras-applications , https://awesomeopensource.com/project/keras-team/keras https://keras.io/examples/

18.opencv-https://awesomeopensource.com/project/opencv/opencv

19.transformers-https://awesomeopensource.com/project/huggingface/transformers

20.code implementations for research papers-https://chrome.google.com/webstore/detail/find-code-for-research-pa/aikkeehnlfpamidigaffhfmgbkdeheil

21.regarding satellite images - Geo AI,Arcgis,geemap

ersi arcgis-https://www.esri.com/en-us/arcgis/about-arcgis/overview

earthcube-https://www.earthcube.eu/

geemap-https://geemap.org/

22.Monk_Object_Detection-https://awesomeopensource.com/project/Tessellate-Imaging/Monk_Object_Detection

https://awesomeopensource.com/project/Tessellate-Imaging/monk_v1

https://analyticsindiamag.com/build-computer-vision-applications-with-few-lines-of-code-using-monk-ai/

pyradox https://awesomeopensource.com/project/Ritvik19/pyradox

23.NLP-progress - https://awesomeopensource.com/project/sebastianruder/NLP-progress

24.interview-question-data-science-https://awesomeopensource.com/project/iNeuronai/interview-question-data-science-

27.Tool for visualizing attention in the Transformer model-https://awesomeopensource.com/project/jessevig/bertviz

28.TransCoder-https://awesomeopensource.com/project/facebookresearch/TransCoder

29.Tessellate-Imaging-https://awesomeopensource.com/project/Tessellate-Imaging/monk_v1

Monk_Object_Detection-https://github.com/Tessellate-Imaging/Monk_Object_Detection/tree/master/application_model_zoo

Artificial-Intelligence-Deep-Learning-Machine-Learning-Tutorials- https://awesomeopensource.com/project/TarrySingh/Artificial-Intelligence-Deep-Learning-Machine-Learning-Tutorials

30.Machine-Learning-with-Python-https://awesomeopensource.com/project/tirthajyoti/Machine-Learning-with-Python

31.huggingface contain almost all nlp pretrained model and all tasks related to nlp field https://huggingface.co/course/chapter0?fw=pt

https://huggingface.co/models https://www.kdnuggets.com/2021/02/hugging-face-transformer-basics.html https://huggingface.co/modelsz

https://github.com/huggingface https://awesomeopensource.com/project/huggingface/transformers https://huggingface.co/transformers/ https://huggingface.co/transformers/master/ https://awesomeopensource.com/project/huggingface/tokenizers

Hugging Face pipelines https://towardsdatascience.com/effortless-nlp-using-pre-trained-hugging-face-pipelines-with-just-3-lines-of-code-a4788d95754f

Fine-tuning pretrained NLP models with Huggingfaceโ€™s Trainer https://towardsdatascience.com/fine-tuning-pretrained-nlp-models-with-huggingfaces-trainer-6326a4456e7b

Mixing Hugging Face Models with Gradio 2.0 https://gradio.app/blog/using-huggingface-models https://huggingface.co/blog/gradio

ktrain https://awesomeopensource.com/project/amaiya/ktrain

Top 6 Alternatives To Hugging Face https://analyticsindiamag.com/top-6-alternatives-to-hugging-face/

32.multi-task-NLP-https://awesomeopensource.com/project/hellohaptik/multi-task-NLP

33.gpt-2 - https://awesomeopensource.com/project/openai/gpt-2

34.Powerful and efficient Computer Vision Annotation Tool (CVAT)-https://awesomeopensource.com/project/openvinotoolkit/cvat, https://awesomeopensource.com/project/abreheret/PixelAnnotationTool

https://awesomeopensource.com/project/UniversalDataTool/universal-data-tool http://www.robots.ox.ac.uk/~vgg/software/via/

36.awesome Data Science-https://awesomeopensource.com/project/academic/awesome-datascience

39.Super Duper NLP Repo-https://notebooks.quantumstat.com/ https://models.quantumstat.com/ https://miro.com/app/board/o9J_kqndLls=/ https://datasets.quantumstat.com/

https://index.quantumstat.com/

https://notebooks.quantumstat.com/?utm_campaign=NLP%20News&utm_medium=email&utm_source=Revue%20newsletter

40.papers summarizing the advances in the field-https://awesomeopensource.com/project/eugeneyan/ml-surveys

41.deep-translator-https://awesomeopensource.com/project/nidhaloff/deep-translator

44.ipython-sql-https://awesomeopensource.com/project/catherinedevlin/ipython-sql

45.libra-https://awesomeopensource.com/project/Palashio/libra

46.opencv-https://awesomeopensource.com/project/opencv/opencv

47.learnopencv-https://awesomeopensource.com/project/spmallick/learnopencv , https://www.learnopencv.com/

48.math is fun-https://www.mathsisfun.com/ , https://pabloinsente.github.io/intro-linear-algebra, https://hadrienj.github.io/posts/Deep-Learning-Book-Series-Introduction/

49.DEEP LEARNING WITH PYTORCH: A 60 MINUTE BLITZ - https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html

50.https://data-flair.training/blogs/

https://data-flair.training/blogs/python-tutorials-home/ https://data-flair.training/blogs/hadoop-tutorials-home/ https://data-flair.training/blogs/spark-tutorials-home/

https://data-flair.training/blogs/tableau-tutorials-home/ https://data-flair.training/blogs/data-science-tutorials-home/

Spark Release 3.0.1-https://spark.apache.org/releases/spark-release-3-0-1.html https://neptune.ai/blog/apache-spark-tutorial

Koalas on Apache Spark - Pandas API https://www.youtube.com/watch?v=kOtAMiMe1JY&t=482s https://koalas.readthedocs.io/en/latest/

mllib https://spark.apache.org/docs/2.0.0/api/python/pyspark.mllib.html https://spark.apache.org/docs/2.0.0/api/python/index.html

https://data-flair.training/blogs/spark-tutorial/ Spark Core,Spark SQL,Spark Streaming,Spark MLlib,Spark GraphX,etc...

Machine Learning with Optimus on Apache Spark https://www.kdnuggets.com/2017/11/machine-learning-with-optimus.html

BigDL: Distributed Deep Learning Framework for Apache Spark https://awesomeopensource.com/project/intel-analytics/BigDL

51.for more cheatsheets-https://awesomeopensource.com/project/FavioVazquez/ds-cheatsheets , https://medium.com/swlh/the-ultimate-cheat-sheet-for-data-scientists-d1e247b6a60c

https://www.theinsaneapp.com/2020/12/machine-learning-and-data-science-cheat-sheets-pdf.html

https://stanford.edu/~shervine/teaching/cs-229/cheatsheet-supervised-learning

52.text2emotion-https://pypi.org/project/text2emotion/

53.ExploriPy-https://analyticsindiamag.com/hands-on-tutorial-on-exploripy-effortless-target-based-eda-tool/

54.TCN-https://awesomeopensource.com/project/philipperemy/keras-tcn

56.earthengine-py-notebooks-https://awesomeopensource.com/project/giswqs/earthengine-py-notebooks

58.numerical-linear-algebra -https://awesomeopensource.com/project/fastai/numerical-linear-algebra

61.chatbot- from scratch,google dialogflow,rasa nlu,azure luis, chatterbot,Amazon lex,Wit.ai,Luis.ai,IBM Watson,Parrot etc...

https://awesomeopensource.com/project/fendouai/Awesome-Chatbot https://medium.com/nerd-for-tech/make-money-building-a-fast-powerful-chatbot-in-10-minutes-using-nltk-91038e15ab17

https://www.analyticsinsight.net/category/chatbots/ https://www.promaticsindia.com/blog/here-are-the-most-popular-chatbot-development-frameworks/

https://neptune.ai/blog/building-machine-learning-chatbots-platforms-and-applications?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-building-machine-learning-chatbots-platforms-and-applications

https://blog.ubisend.com/optimise-chatbots/chatbot-training-data

OpenChat: Open Source Chatting Framework for Generative Models https://analyticsindiamag.com/a-brief-overview-of-openchat-open-source-chatting-framework-for-generative-models/

  1. No Code Machine Learning / Deep Learning https://analyticsindiamag.com/top-12-no-code-machine-learning-platforms-in-2021/ https://www.pye.ai/2021/06/01/2021-list-of-top-data-science-platforms-end-to-end-machine-learning/

https://serokell.io/blog/top-no-code-platforms https://www.nanalyze.com/2021/04/no-code-platforms-machine-learning/

Akkio, Obviously.ai, DataRobot, Levity, Clarifai, Teachable Machines, Lobe,pimer,DynaBench,APAflow

Teachable Machine-https://teachablemachine.withgoogle.com/ Vertex AI https://cloud.google.com/vertex-ai/docs/start/automl-users

Microsoft Lobe -https://lobe.ai/

Ludwig https://awesomeopensource.com/project/ludwig-ai/ludwig

WEKA - https://www.cs.waikato.ac.nz/ml/weka/ autoweka

Create ML https://developer.apple.com/documentation/createml

APAflow https://apaflow.com/?utm_medium=social&utm_source=linkedin&utm_campaign=postfity&utm_content=postfity0b527 https://apaflow.com/

Monk_Gui-https://awesomeopensource.com/project/Tessellate-Imaging/Monk_Gui

FlashML https://www.flash-ml.com/

JADBioโ€™s https://www.jadbio.com/

JOHN SNOW LABS https://www.johnsnowlabs.com/models-training-and-active-learning-in-john-snow-labs-annotation-lab/

igel https://awesomeopensource.com/project/nidhaloff/igel

BRYTER https://bryter.com

Ushur https://ushur.com

Accern https://accern.com

Signzy https://signzy.com

Runway https://runwayml.com

Fritz AI https://www.fritz.ai

BigML, Inc https://bigml.com

MyDataModels https://lnkd.in/eejjDbM

MonkeyLearn https://monkeylearn.com

Levity https://levity.ai

Nanonets https://nanonets.com

obviously https://www.obviously.ai/

machine learning straight from Microsoft Excel https://venturebeat.com/2020/12/30/you-dont-code-do-machine-learning-straight-from-microsoft-excel/

ENNUI-https://math.mit.edu/ennui/ https://awesomeopensource.com/project/martinjm97/ENNUI https://www.youtube.com/watch?v=4VRC5k0Qs2w

Knime https://www.knime.com/

Accord.net http://accord-framework.net/

DeepDev https://realmichaelye.github.io/DeepDev/deepdev.tech%20-%20Landing%20Page/ https://awesomeopensource.com/project/realmichaelye/DeepDev

H2O Driverless AI https://www.h2o.ai/products/h2o-driverless-ai/

Oracle AutoML https://medium.com/nerd-for-tech/oracles-automl-what-it-is-and-how-it-works-12e09a832c2 https://docs.oracle.com/en-us/iaas/tools/ads-sdk/latest/user_guide/overview/overview.html

Rapid Miner https://rapidminer.com/

opennn https://www.opennn.net/

datarobot https://www.datarobot.com/

dataiku https://www.dataiku.com/product/get-started/

orange https://orange.biolab.si/

Databricks AutoML Automate Machine Learning using Databricks AutoML https://pub.towardsai.net/automate-machine-learning-using-databricks-automl-a-glass-box-approach-and-mlflow-2543a8143687

OpenBlender https://openblender.io/#/welcome https://analyticsindiamag.com/how-to-use-openblender-the-leading-data-blending-tool/

create neural networks with one line of code https://github.com/PraneetNeuro/nnio.l

AWS SageMaker AutoPilot https://aws.amazon.com/sagemaker/autopilot/

Machine Learning in JUST ONE LINE OF CODE libra https://github.com/Palashio/libra/ https://www.youtube.com/watch?v=N_T_ljj5vc4

perceptilabs https://towardsdatascience.com/easy-model-building-with-perceptilabs-interactive-tensorflowvisualization-gui-834d5bb3c973

64.tensorflow development-https://blog.tensorflow.org/

TensorFlow Hub (trained ready-to-deploy machine learning models in one place) - https://tfhub.dev/

CrypTFlow: An End-to-end System for Secure TensorFlow Inference https://awesomeopensource.com/project/mpc-msri/EzPC https://pratik-bhatu.medium.com/privacy-preserving-machine-learning-for-healthcare-using-cryptflow-cc6c379fbab7

TensorBoard.dev - https://tensorboard.dev/

tutorials-https://www.tensorflow.org/tutorials https://www.tensorflow.org/guide

TensorFlow Graphics - https://www.tensorflow.org/graphics Lattice-https://www.tensorflow.org/lattice

TensorFlow Probability-https://www.tensorflow.org/probability TensorFlow Privacy- tensorflow-privacy

https://developers.google.com/learn/topics/on-device-ml https://www.tensorflow.org/lite/guide/model_maker https://tfhub.dev/ https://www.tensorflow.org/cloud

63.Data Science in the Cloud-Amazon SageMaker,Amazon Lex,Amazon Rekognition,Azure Machine Learning (Azure ML) Services,Azure Service Bot framework,Google Cloud AutoML

64.platforms to build and deploy ML models -Uber has Michelangelo,Google has TFX,Databricks has MLFlow,Amazon Web Services (AWS) has Sagemaker

66.ML from scratch-https://dafriedman97.github.io/mlbook/content/introduction.html

https://aihubprojects.com/machine-learning-from-scratch-python/

https://awesomeopensource.com/project/python-engineer/MLfromscratch https://www.youtube.com/watch?v=rLOyrWV8gmA

https://www.datasciencecentral.com/profiles/blogs/a-complete-tutorial-to-learn-data-science-with-python-from

https://medium.com/@mattybv3/learn-data-science-from-scratch-curriculum-with-20-free-online-courses-8cff96d6cbe5

67.turn-on visual training for most popular ML algorithms https://awesomeopensource.com/project/lucko515/ml_tutor https://pypi.org/project/ml-tutor/

68.mlcourse.ai is a free online- https://mlcourse.ai/

72.R for Data Science-https://r4ds.had.co.nz/ ,Fundamentals of Data Visualization-https://clauswilke.com/dataviz/

74.machine learning in JavaScript-https://www.tensorflow.org/js https://www.tensorflow.org/js/models https://tensorflow-js-object-detection.glitch.me/

TensorFlow.jl Julia with TensorFlow https://malmaud.github.io/tfdocs/ https://malmaud.github.io/TensorFlow.jl/latest/tutorial.html

Sonnet is a library built on top of TensorFlow 2 https://awesomeopensource.com/project/deepmind/sonnet

TensorFlow Federated (TFF) ( facilitate open research and experimentation with Federated Learning)-https://www.tensorflow.org/federated

TFX is an end-to-end platform for deploying production ML pipelines https://www.tensorflow.org/tfx https://awesomeopensource.com/project/tensorflow/tfx https://analyticsindiamag.com/guide-to-tensorflow-extendedtfx-end-to-end-platform-for-deploying-production-ml-pipelines/

Federated Learning -https://www.tensorflow.org/federated/tutorials/federated_learning_for_image_classification

Neural Structured Learning-https://www.tensorflow.org/neural_structured_learning/tutorials/graph_keras_mlp_cora

Responsible AI-https://www.tensorflow.org/resources/responsible-ai

https://www.tensorflow.org/graphics

75.free list of AI/ Machine Learning Resources/Courses-https://www.marktechpost.com/free-resources/

https://awesomeopensource.com/project/kabartay/OpenUnivCourses

Open ML University https://curriculum.openmlu.com/

https://www.kdnuggets.com/2018/11/10-free-must-see-courses-machine-learning-data-science.html

https://www.kdnuggets.com/2018/12/10-more-free-must-see-courses-machine-learning-data-science.html

https://www.theinsaneapp.com/2020/12/machine-learning-and-data-science-cheat-sheets-pdf.html

https://www.theinsaneapp.com/2020/11/free-machine-learning-data-science-and-python-books.html

65 Machine Learning and Data books for free- https://towardsdatascience.com/springer-has-released-65-machine-learning-and-data-books-for-free-961f8181f189

https://www.deeplearningbook.org/ http://d2l.ai/ https://www.theinsaneapp.com/2020/12/download-free-machine-learning-books.html

https://www.datasciencecentral.com/profiles/blogs/free-500-page-book-on-applications-of-deep-neural-networks-1 https://awesomeopensource.com/project/jeffheaton/t81_558_deep_learning

https://www.theinsaneapp.com/2020/12/free-data-science-books-pdf.html

https://www.datasciencecentral.com/profiles/blogs/free-500-page-book-on-applications-of-deep-neural-networks-1

https://awesomeopensource.com/project/chaconnewu/free-data-science-books

https://www.kdnuggets.com/2020/03/24-best-free-books-understand-machine-learning.html

https://www.kdnuggets.com/2020/12/15-free-data-science-machine-learning-statistics-ebooks-2021.html

http://introtodeeplearning.com/

https://www.theinsaneapp.com/2020/12/free-data-science-books-pdf.html

http://d2l.ai/index.html https://www.kdnuggets.com/2020/09/best-free-data-science-ebooks-2020-update.html

https://www.youtube.com/playlist?app=desktop&list=PLypiXJdtIca5ElZMWHl4HMeyle2AzUgVB https://mit6874.github.io/

79.For practice -https://www.confetti.ai/exams

80.Yellowbrick-https://towardsdatascience.com/introduction-to-yellowbrick-a-python-library-to-explain-the-prediction-of-your-machine-learning-d63ecee10ecc

81.Mathematics of Machine Learning,deep learning-https://towardsdatascience.com/the-mathematics-of-machine-learning-894f046c568

https://awesomeopensource.com/project/hrnbot/Basic-Mathematics-for-Machine-Learning

https://towardsdatascience.com/the-roadmap-of-mathematics-for-deep-learning-357b3db8569b

https://medium.com/towards-artificial-intelligence/basic-linear-algebra-for-deep-learning-and-machine-learning-ml-python-tutorial-444e23db3e9e

https://www.kdnuggets.com/2020/02/free-mathematics-courses-data-science-machine-learning.html

https://towardsai.net/p/data-science/how-much-math-do-i-need-in-data-science-d05d83f8cb19

https://www.mltut.com/how-to-learn-math-for-machine-learning-step-by-step-guide/

https://stanford.edu/~shervine/teaching/cs-229/cheatsheet-machine-learning-tips-and-tricks#

https://www.datasciencecentral.com/profiles/blogs/free-online-book-machine-learning-from-scratch

https://hadrienj.github.io/posts/Essential-Math-for-Data-Science-Introduction_to_matrices_and_matrix_product/?utm_source=linkedin&utm_medium=social&utm_campaign=linkedin_matrices

https://www.youtube.com/playlist?list=PLRDl2inPrWQW1QSWhBU0ki-jq_uElkh2a https://awesomeopensource.com/project/jonkrohn/ML-foundations

https://ocw.mit.edu/resources/res-18-001-calculus-online-textbook-spring-2005/textbook/

82.Googleai-https://ai.google/education

83.ELI5 is a Python package which helps to debug machine learning classifiers and explain their predictions

PyBrain is a modular Machine Learning Library for Python

84.Best Online Courses for Machine Learning and Data Science-https://www.mltut.com/best-online-courses-for-machine-learning-and-data-science/

Comprehensive Project Based Data Science Curriculum https://julienbeaulieu.github.io/2019/09/25/comprehensive-project-based-data-science-curriculum/

AI Expert Roadmap-https://i.am.ai/roadmap/#data-science-roadmap

86.Yann LeCunโ€™s Deep Learning Course at CDS-https://cds.nyu.edu/deep-learning/ https://atcold.github.io/pytorch-Deep-Learning/

https://atcold.github.io/pytorch-Deep-Learning/

https://www.cs.cmu.edu/~ninamf/courses/601sp15/lectures.shtml

88.Python Data Science Handbook https://jakevdp.github.io/PythonDataScienceHandbook/

91.AudioFeaturizer when deal with audio data- https://pypi.org/project/AudioFeaturizer/

liborsa library https://librosa.org/doc/latest/index.html

MAGENTA-https://magenta.tensorflow.org/

pydub https://awesomeopensource.com/project/jiaaro/pydub

DDSP: Differentiable Digital Signal Processing https://awesomeopensource.com/project/magenta/ddsp https://analyticsindiamag.com/guide-to-differentiable-digital-signal-processing-ddsp-library-with-python-code/

92.Palladium-https://palladium.readthedocs.io/en/latest/

94.Facebook Open Sourced New Frameworks to Advance Deep Learning Research https://www.kdnuggets.com/2020/11/facebook-open-source-frameworks-advance-deep-learning-research.html

95.Software Engineering for Machine Learning https://awesomeopensource.com/project/SE-ML/awesome-seml

96.Atlas web-based dashboard -https://www.atlas.dessa.com/

97.Pytest (test code) https://docs.pytest.org/en/latest/index.html (test code)

98.keras- https://keras.io/ https://keras.io/api/ https://keras.io/examples/

99.High-Performance Jupyter Notebook - BlazingSQL Notebooks https://blazingsql.com/notebooks

jupyter-tabnine https://awesomeopensource.com/project/wenmin-wu/jupyter-tabnine

101.Kubeflow Machine Learning Toolkit for Kubernetes https://www.kubeflow.org/

102.Daily AI updates to your inbox- https://sago-ai.news/#/

103.Three API styles - Sequential Model,functional API,Model subclassing

104.Deep Learning Toolkit for Medical Image Analysis -https://awesomeopensource.com/project/DLTK/DLTK

106.Interpret The ML Model https://towardsdatascience.com/explainable-artificial-intelligence-part-3-hands-on-machine-learning-model-interpretation-e8ebe5afc608

https://christophm.github.io/interpretable-ml-book/ https://www.kaggle.com/getting-started/209632 https://ex.pegg.io/

https://neptune.ai/blog/explainability-auditability-ml-definitions-techniques-tools?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-explainability-auditability-ml-definitions-techniques-tools

Ethical-AI Toolkits https://murat-durmus.medium.com/an-brief-overview-of-some-ethical-ai-toolkits-712afe9f3b3a

Awesome-explainable-AI https://ex.pegg.io/

tf-explain https://awesomeopensource.com/project/sicara/tf-explain

lime(explain black box models)- https://lime-ml.readthedocs.io/en/latest/ https://towardsdatascience.com/interpreting-image-classification-model-with-lime-1e7064a2f2e5

SHAP https://medium.com/towards-artificial-intelligence/explain-your-machine-learning-predictions-with-kernel-shap-kernel-explainer-fed56b9250b8

https://awesomeopensource.com/project/slundberg/shap https://www.kdnuggets.com/2020/01/explaining-black-box-models-ensemble-deep-learning-lime-shap.html https://analyticsindiamag.com/hands-on-guide-to-interpret-machine-learning-with-shap/

fastshap https://awesomeopensource.com/project/bgreenwell/fastshap

Shapash makes Machine Learning models transparent and understandable by everyone https://awesomeopensource.com/project/MAIF/shapash https://www.kdnuggets.com/2021/04/shapash-machine-learning-models-understandable.html

Captum is a model interpretability and understanding library for PyTorch https://awesomeopensource.com/project/pytorch/captum

Explainable AI https://ex.pegg.io/

Explainable AI dashboards https://awesomeopensource.com/project/oegedijk/explainerdashboard https://www.youtube.com/watch?v=ZgypAMRcmw8

interpret https://awesomeopensource.com/project/interpretml/interpret mlxtend's http://rasbt.github.io/mlxtend/

DiCE Generate Diverse Counterfactual Explanations for any machine learning model. https://awesomeopensource.com/project/interpretml/DiCE

tcav https://awesomeopensource.com/project/tensorflow/tcav yellowbrick https://www.scikit-yb.org/en/latest/quickstart.html

Language Interpretability Tool https://awesomeopensource.com/project/pair-code/lit https://ai.googleblog.com/2020/11/the-language-interpretability-tool-lit.html

Transformers Interpret https://towardsdatascience.com/introducing-transformers-interpret-explainable-ai-for-transformers-890a403a9470 https://awesomeopensource.com/project/cdpierse/transformers-interpret

treeinterpreter https://awesomeopensource.com/project/andosa/treeinterpreter

Adversarial Explainable AI https://awesomeopensource.com/project/hbaniecki/adversarial-explainable-ai https://medium.com/responsibleml/adversarial-attacks-on-explainable-ai-f65d41e83c5f

Captum Model Interpretability for PyTorch https://captum.ai/ https://awesomeopensource.com/project/pytorch/captum

ecco https://awesomeopensource.com/project/jalammar/ecco https://jalammar.github.io/explaining-transformers/ https://www.eccox.io/

dalex https://pypi.org/project/dalex/ https://blog.learningdollars.com/2021/01/02/ai-in-medical-diagnosis/ https://www.kdnuggets.com/2020/11/dalex-explain-tensorflow-model.html

google AI Explanations for AI Platform https://cloud.google.com/ai-platform/prediction/docs/ai-explanations/overview?utm_source=youtube&utm_medium=Unpaidsocial&utm_campaign=guo-20200423-Intro-Aiexp

eli5 https://eli5.readthedocs.io/en/latest/

TabNet: Attentive Interpretable Tabular Learning https://awesomeopensource.com/project/dreamquark-ai/tabnet

skater https://oracle.github.io/Skater/

lucid https://github.com/tensorflow/lucid/ https://www.kdnuggets.com/2020/04/openai-open-sources-microscope-lucid-library-neural-networks.html

what if tool https://pair-code.github.io/what-if-tool/ https://pair-code.github.io/what-if-tool/demos/uci.html

themis https://themis-ml.readthedocs.io/en/latest/

DeepLIFT https://awesomeopensource.com/project/kundajelab/deeplift

Arena https://medium.com/responsibleml/python-has-now-the-new-way-of-exploring-xai-explanations-4248846426cf

tabnet https://cloud.google.com/blog/products/ai-machine-learning/ml-model-tabnet-is-easy-to-use-on-cloud-ai-platform

explainerdashboard https://towardsdatascience.com/the-quickest-way-to-build-dashboards-for-machine-learning-models-ec769825070d

Responsible AI-https://www.tensorflow.org/resources/responsible-ai

fairlearn https://awesomeopensource.com/project/fairlearn/fairlearn fairml https://awesomeopensource.com/project/adebayoj/fairml https://www.datasciencecentral.com/profiles/blogs/fairml-auditing-black-box-predictive-models

fair https://medium.com/responsibleml/how-to-easily-check-if-your-ml-model-is-fair-2c173419ae4c

cleverhans https://awesomeopensource.com/project/cleverhans-lab/cleverhans

Google Facets https://pair-code.github.io/facets/

Googleโ€™s Model Card Toolkit

Opening the AI Black Box -https://zetane.com/gallery

Rulex Explainable AI https://www.rulex.ai/rulex-explainable-ai-xai/

AI Explainability 360 Toolkit from IBM Research https://aix360.mybluemix.net/ https://analyticsindiamag.com/guide-to-ai-explainability-360-an-open-source-toolkit-by-ibm/

onnx https://awesomeopensource.com/project/onnx/onnx

torch-dreams https://awesomeopensource.com/project/Mayukhdeb/torch-dreams

https://awesomeopensource.com/project/jphall663/awesome-machine-learning-interpretability

https://analyticsindiamag.com/8-explainable-ai-frameworks-driving-a-new-paradigm-for-transparency-in-ai/

https://christophm.github.io/interpretable-ml-book/ https://awesomeopensource.com/project/christophM/interpretable-ml-book

https://www.kdnuggets.com/2018/12/machine-learning-explainability-interpretability-ai.html https://www.kdnuggets.com/2019/09/python-libraries-interpretable-machine-learning.html https://www.kdnuggets.com/2019/08/open-black-boxes-explainable-machine-learning.html

Fairness https://analyticsindiamag.com/building-a-responsible-ai-eco-system/

How to easily check if your Machine Learning model is fair (dalex) https://www.kdnuggets.com/2020/12/machine-learning-model-fair.html

LinkedIn Fairness Toolkit,Fairlearn,AI Fairness 360,scikit-fairness,Algofairness,Aequitas,CERTIFAI,ML-fairness-gym,Algofairness,FairSight,GD-IQ,scikit-fairness,Mitigating Gender Bias In Captioning System

https://analyticsindiamag.com/guide-to-ai-fairness-360-an-open-source-toolkit-for-detection-and-mitigation-of-bias-in-ml-models/

107.deep-learning-drizzle -https://deep-learning-drizzle.github.io/

108.Machine Learning University - https://aws.amazon.com/machine-learning/mlu/

109.mlflow https://mlflow.org/ An open source platform for the machine learning lifecycle

https://www.kdnuggets.com/2021/01/5-tools-effortless-data-science.html

https://neptune.ai/

https://azure.microsoft.com/en-us/services/machine-learning/

https://awesomeopensource.com/project/VertaAI/modeldb

110.Data Preparation / ETL https://airflow.apache.org/ https://intake.readthedocs.io/en/latest/

111.fairlearn https://github.com/fairlearn/fairlearn/blob/master/README.md Evaluating fairness of AI/ML models and training data and for mitigating bias in models determined to be unfair.

AI Fairness 360 evaluating fairness of AI/ML models and training data and mitigating bias in current models https://aif360.mybluemix.net/

An ethics checklist for data scientists https://deon.drivendata.org/

112.https://analyticsindiamag.com/top-6-ai-powered-drug-discovery-tools-in-2021/

MONAI Framework For Medical Imaging Research https://analyticsindiamag.com/monai-datatsets-managers/

torchio https://awesomeopensource.com/project/fepegar/torchio https://analyticsindiamag.com/torchio-3d-medical-imaging/

MolBert: Molecular Representation learning with AI

medicalAI https://awesomeopensource.com/project/aibharata/medicalAI

Biopython is a set of freely available tools https://awesomeopensource.com/project/biopython/biopython

DeepIPW https://awesomeopensource.com/project/ruoqi-liu/DeepIPW

113.OpenVINO https://opencv.org/openvino-model-optimization/ https://opencv.org/how-to-speed-up-deep-learning-inference-using-openvino-toolkit-2/

114.https://neptune.ai/blog/machine-learning-model-management https://analyticsindiamag.com/top-mlops-tools-github-repos/ https://neu.ro/2021-mlops-platforms-vendor-analysis-report/

Best Workflow and Pipeline Orchestration Tools https://neptune.ai/blog/best-workflow-and-pipeline-orchestration-tools?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-best-workflow-and-pipeline-orchestration-tools

MLflow vs Kubeflow vs Neptune https://neptune.ai/blog/mlflow-vs-kubeflow-vs-neptune-differences?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-mlflow-vs-kubeflow-vs-neptune-differences

15 MLOps.toys https://mlops.toys/ AIOps,Data version control DVC,MLFlow,Docker foundation,Kubernetes Foundation,Tensorflow Extend (TFX),Kubeflow,AWS AIOps,Azure AIOps

Metaflow,MLReef,MLRun,ZenML,MLflow,Seldon,Bodywork,Pachyderm,DVC, or Data Version Control

MLOps https://analyticsindiamag.com/8-projects-to-kickstart-your-mlops-journey-in-2021/

Open MLOps https://awesomeopensource.com/project/datarevenue-berlin/OpenMLOps

Best Tools for Tracking Machine Learning Experiments https://neptune.ai/blog/best-ml-experiment-tracking-tools

mlops-https://awesomeopensource.com/project/visenger/awesome-mlops

mlflow https://towardsdatascience.com/get-started-with-mlops-fd7062cab018

MLOPS https://www.analyticsinsight.net/top-mlops-based-tools-for-enabling-effective-machine-learning-lifecycle/ https://neptune.ai/blog/best-mlops-tools

ML-Model-CI https://awesomeopensource.com/project/cap-ntu/ML-Model-CI

Easy MLOps with PyCaret + MLflow https://www.kdnuggets.com/2021/05/easy-mlops-pycaret-mlflow.html

https://www.kdnuggets.com/2021/03/overview-mlops.html https://medium.com/prosus-ai-tech-blog/towards-mlops-technical-capabilities-of-a-machine-learning-platform-61f504e3e281

omegaml https://awesomeopensource.com/project/omegaml/omegaml

https://neptune.ai/blog/8-best-data-science-and-machine-learning-platforms-for-mlops?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-8-best-data-science-and-machine-learning-platforms-for-mlops

https://neptune.ai/blog/ml-model-monitoring-best-tools?utm_source=email&utm_medium=newsletter&utm_campaign=blog-march&utm_content=ml-model-monitoring-best-tools

https://neptune.ai/blog/end-to-end-mlops-platforms?utm_source=email&utm_medium=newsletter&utm_campaign=blog-march&utm_content=end-to-end-mlops-platforms

https://neptune.ai/blog/mlops-at-greensteam-shipping-machine-learning-case-study?utm_source=email&utm_medium=newsletter&utm_campaign=blog-march&utm_content=mlops-at-greensteam-shipping-machine-learning-case-study

https://neptune.ai/blog/mlops-10-best-practices?utm_source=email&utm_medium=newsletter&utm_campaign=blog-march&utm_content=mlops-10-best-practices

https://neptune.ai/blog/machine-learning-model-management?utm_source=email&utm_medium=newsletter&utm_campaign=blog-march&utm_content=machine-learning-model-management

https://mlops.githubapp.com/ https://about.mlreef.com/blog/global-mlops-and-ml-tools-landscape https://awesomeopensource.com/project/paiml/practical-mlops-book

https://olympus.greatlearning.in/courses/12956?_gl=1*ljadx1*_ga*NjMxNjUxNjM2LjE2MDYyMDYzNDM.*_ga_TH52C020P8*MTYxMTIyOTQ0MS40Ny4wLjE2MTEyMjk0NDEuNjA.

https://docs.microsoft.com/en-us/azure/architecture/example-scenario/mlops/mlops-technical-paper https://neptune.ai/blog/end-to-end-mlops-platforms

https://awesomeopensource.com/project/kelvins/awesome-mlops#hyperparameter-tuning

ClearML https://analyticsindiamag.com/guide-to-clearml-zero-integration-mlops-solution/

https://neptune.ai/blog/mlops-what-it-is-why-it-matters-and-how-to-implement-it-from-a-data-scientist-perspective?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-mlops-what-it-is-why-it-matters-and-how-to-implement-it-from-a-data-scientist-perspective

https://ml-ops.org/content/mlops-principles

115.Code faster https://www.tabnine.com/

117.https://www.pye.ai/2021/03/19/machine-learning-model-management-what-why-and-how/ https://www.ambiata.com/blog/2020-12-07-mlops-tools/

mlflow https://mlflow.org/docs/latest/index.html https://awesomeopensource.com/project/amesar/mlflow-examples

MLFlow by pycaret https://pycaret.org/mlflow/?utm_medium=social&utm_source=linkedin&utm_campaign=postfity&utm_content=postfity2c1c2

labml https://ramith.fyi/tracking-your-ml-experiments-without-sending-data-to-the-cloud/

MLOps https://awesomeopensource.com/project/microsoft/MLOps https://mlops.githubapp.com/ https://huyenchip.com/2020/12/30/mlops-v2.html https://awesomeopensource.com/project/paiml/practical-mlops-book

7 Best Resources To Learn MLOps In 2021 https://analyticsindiamag.com/7-best-resources-to-learn-mlops-in-2021/

DevOps https://awesomeopensource.com/project/collections/devops-tools

airflow https://awesomeopensource.com/project/apache/airflow

kubeflow https://awesomeopensource.com/project/kubeflow/kubeflow

kubernetes https://awesomeopensource.com/project/kubernetes/kubernetes

Metaflow https://metaflow.org/ https://awesomeopensource.com/project/Netflix/metaflow

pipeline https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html

Tensorflow Extended https://www.tensorflow.org/tfx Tensorflow Transform https://www.tensorflow.org/tfx/transform/get_started

https://aniruddha-choudhury49.medium.com/mlops-kubeflow-with-tensorflow-tfx-pipelines-seamlessly-and-at-scale-92b432bd39b0

Serving Models https://www.tensorflow.org/tfx/guide/serving

Tensorflow Data Validation https://www.tensorflow.org/tfx/data_validation/get_started TensorFlow Model Analysis https://www.tensorflow.org/tfx/model_analysis/get_started

MLflow Open-source platform for tracking machine learning experiments https://mlflow.org/ https://analyticsindiamag.com/guide-to-mlflow-a-platform-to-manage-machine-learning-lifecycle/ https://www.kdnuggets.com/2021/01/model-experiments-tracking-registration-mlflow-databricks.html

ray https://docs.ray.io/en/master/serve/ https://awesomeopensource.com/project/ray-project/ray

https://medium.com/distributed-computing-with-ray/ray-mlflow-taking-distributed-machine-learning-applications-to-production-103f5505cb88

  1. Feature Stores https://neptune.ai/blog/feature-stores-components-of-a-data-science-factory-guide?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-feature-stores-components-of-a-data-science-factory-guide

Top 10 Leading Machine Learning Feature Stores https://www.pye.ai/2021/05/14/top-10-machine-learning-feature-store-systems/

118.algorithm to use by problem https://www.datasciencecentral.com/profiles/blogs/which-machine-learning-deep-learning-algorithm-to-use-by-problem

119.Connect the world to your data and fuel your ML.

OpenBlender Enrich ML Models with adding new Variables from Any Source to Boost Performance https://www.youtube.com/channel/UCCFN8DDrA6k7eHYLvZGdNVA https://openblender.io/

  1. Google's MuRIL (Multilingual Representations for Indian Languages) https://tfhub.dev/google/MuRIL/1

121.mxnet https://mxnet.apache.org/versions/master/api/python/docs/tutorials/getting-started/crash-course/index.html

122.tools-https://towardsdatascience.com/data-science-tools-f16ecd91c95d

123.Elements of AI free online course https://www.elementsofai.com/

124.Best_AI_paper_2020 https://awesomeopensource.com/project/louisfb01/Best_AI_paper_2020

125.roadmap https://awesomeopensource.com/project/graykode/nlp-roadmap https://www.theinsaneapp.com/2021/03/roadmap-series.html

https://www.freecodecamp.org/news/data-science-learning-roadmap/ https://www.kdnuggets.com/2020/12/roadmaps-ai-developer-data-scientist-machine-learning-engineer.html

https://mohammedazeem665.medium.com/plan-to-learn-machine-learning-data-science-in-2021-note-these-assets-from-2020-e84389d94097

https://awesomeopensource.com/project/AMAI-GmbH/AI-Expert-Roadmap

https://becominghuman.ai/cheat-sheets-for-ai-neural-networks-machine-learning-deep-learning-big-data-678c51b4b463

data-engineer-roadmap https://awesomeopensource.com/project/datastacktv/data-engineer-roadmap

126.https://neptune.ai/blog/best-data-science-tools-to-increase-machine-learning-model-understanding?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-best-data-science-tools-to-increase-machine-learning-model-understanding

Visualizing the Execution of Python Program http://pythontutor.com/ https://www.youtube.com/watch?v=pCSlWQjfCzA

MLPerf Model performance debugging tools https://mlperf.org/

Model debugging tools Manifold https://eng.uber.com/manifold/

Pytest for Data Scientists https://towardsdatascience.com/4-lessor-known-yet-awesome-tips-for-pytest-2117d8a62d9c

Icecream https://towardsdatascience.com/stop-using-print-to-debug-in-python-use-icecream-instead-79e17b963fcc

Experiment tracking tools WandB https://wandb.ai/site

Comet manage and organize machine learning experiments https://www.comet.ml/site/ https://analyticsindiamag.com/how-to-supercharge-your-machine-learning-experiments-with-comet-ml/

neptune https://neptune.ai/ https://analyticsindiamag.com/how-to-manage-ml-experiments-with-neptune-ai/

weights & biases https://wandb.ai/site https://analyticsindiamag.com/hands-on-guide-to-weights-and-biases-wandb-with-python-implementation/ https://docs.wandb.ai/

https://www.kdnuggets.com/2020/07/tour-end-to-end-machine-learning-platforms.html

127.19 Best JupyterLab Extensions for Machine Learning https://neptune.ai/blog/jupyterlab-extensions-for-machine-learning

128.coreml https://developer.apple.com/machine-learning/core-ml/

129.Protect Your Neural Networks Against Hacking Adversarial Robustness Toolbox (ART) https://analyticsindiamag.com/adversarial-robustness-toolbox-art/

130.https://www.kdnuggets.com/2021/01/10-underappreciated-python-packages-machine-learning-practitioners.html

131.datascience-fails https://awesomeopensource.com/project/xLaszlo/datascience-fails

132.Jupyter notebook integration for Microsoft Excel https://awesomeopensource.com/project/pyxll/pyxll-jupyter https://towardsdatascience.com/python-jupyter-notebooks-in-excel-5ab34fc6439

Voilร  turns Jupyter notebooks into standalone web applications https://awesomeopensource.com/project/voila-dashboards/voila https://awesomeopensource.com/project/voila-dashboards/voila-gridstack

How to Optimize Your Jupyter Notebook https://www.kdnuggets.com/2020/01/optimize-jupyter-notebook.html

TabNet: Attentive Interpretable Tabular Learning https://awesomeopensource.com/project/dreamquark-ai/tabnet

133.rapidly develop data applications with Python https://awesomeopensource.com/project/dstackai/dstack

134.Google Research: Looking Back at 2020, and Forward to 2021 https://ai.googleblog.com/2021/01/google-research-looking-back-at-2020.html

135.cortex Run inference at scale https://www.cortex.dev/ https://awesomeopensource.com/project/cortexlabs/cortex

136.https://www.theinsaneapp.com/2020/12/machine-learning-and-data-science-cheat-sheets-pdf.html

137.Federated Learning Systems

Flower โ€“ A Framework To Build Federated Learning Systems https://awesomeopensource.com/project/adap/flower https://flower.dev/

138.https://analyticsindiamag.com/top-ai-powered-writing-assistants-to-create-better-content/

139.Tensorflow Data Validation - Data Analysis At Scale https://www.youtube.com/watch?v=eGIG_qHgQ08

140.SciKeras https://scikeras.readthedocs.io/en/latest/#

141.debugging Data viewer https://devblogs.microsoft.com/python/python-in-visual-studio-code-january-2021-release/

142.Machine Learning Lifecycle in 2021 https://towardsdatascience.com/the-machine-learning-lifecycle-in-2021-473717c633bc

143.Introduction To ML.NET โ€“ An ML Framework For DOTNET Developers https://analyticsindiamag.com/introduction-to-ml-net-a-machine-learning-framework-for-dotnet-developers/

https://analyticsindiamag.com/step-by-step-guide-for-image-classification-using-ml-net/

144.https://www.perceptilabs.com/home http://deeplearninggallery.com/ https://www.kdnuggets.com/2019/01/practical-apache-spark-10-minutes.html

145.https://www.kdnuggets.com/2018/09/machine-learning-cheat-sheets.html https://www.kdnuggets.com/2018/09/meverick-lin-data-science-cheat-sheet.html

https://www.kdnuggets.com/2018/08/data-visualization-cheatsheet.html https://www.kdnuggets.com/2018/07/sql-cheat-sheet.html https://www.kdnuggets.com/2018/04/python-regular-expressions-cheat-sheet.html https://www.kdnuggets.com/2017/09/essential-data-science-machine-learning-deep-learning-cheat-sheets.html

https://www.analyticsvidhya.com/blog/2021/01/5-python-packages-every-data-scientist-must-know/

https://www.kdnuggets.com/2021/01/ultimate-scikit-learn-machine-learning-cheatsheet.html https://www.kdnuggets.com/2020/09/10-things-know-scikit-learn.html

146.Data Pipelines https://www.kdnuggets.com/2018/05/beginners-guide-data-science-pipeline.html https://www.kdnuggets.com/2019/03/data-pipelines-luigi-airflow-everything-need-know.html

  1. AI Habitat: A Platform For Embodied AI Research https://analyticsindiamag.com/hands-on-guide-to-ai-habitat-a-platform-for-embodied-ai-research/

149.onnx https://medium.com/towards-artificial-intelligence/onnx-for-model-interoperability-faster-inference-8709375db9bf

152.Best ML Frameworks & Extensions for Scikit-learn https://neptune.ai/blog/the-best-ml-framework-extensions-for-scikit-learn?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-the-best-ml-framework-extensions-for-scikit-learn

153.Multimodal Neurons, The Most Advanced Neural Networks Discovered By OpenAI https://analyticsindiamag.com/inside-multimodal-neurons-the-most-advanced-neural-networks-discovered-by-openai/

154.TensorGram https://awesomeopensource.com/project/ksdkamesh99/TensorGram https://www.youtube.com/watch?v=ItDBQB4YFuI

   knockknock https://towardsdatascience.com/how-to-get-notified-when-your-model-is-done-training-with-knockknock-483a0475f82c
   
   labmi Organize machine learning experiments and monitor training progress from mobile  https://labml.ai/
   
   WeightWatcher https://github.com/CalculatedContent/WeightWatcher
   
   labml Monitor deep learning model training and hardware usage from your mobile phone https://labml.ai/      https://github.com/labmlai/labml

155.r packages https://upurl.me/vkf3r http://r-bloggers.com/2021/04/15-essential-packages-in-r-for-data-science/ https://www.ubuntupit.com/best-r-machine-learning-packages/

Top 10 Free Resources To Learn R https://analyticsindiamag.com/top-10-free-resources-to-learn-r/

https://bluemind1988.medium.com/explore-r-libraries-for-end-to-end-data-science-projects-b4d0af3a9f5c

analyticsvidhya.com/blog/2021/04/top-10-r-packages-for-data-science-you-must-know-in-2021/

156.Top Julia Libraries for Machine Learning https://www.analyticsvidhya.com/blog/2021/05/top-julia-machine-learning-libraries/

156.openblender Fuel your ML Engines with Relevant Data to Boost Performance https://openblender.io/#/welcome

157.all Domain-based A.I. Platform for Data Scientists https://www.cluzters.ai/

158.2D images to 3D https://analyticsindiamag.com/python-guide-to-neural-body-converting-2d-images-to-3d/

Open3D: An Open Source Modern Library For 3D Data Processing https://awesomeopensource.com/project/intel-isl/Open3D

160.https://gallery.allennlp.org/ https://prior.allenai.org/projects/gpv

161.NVIDIA Unveils 50+ New, Updated AI Tools and Trainings for Developers https://www.hpcwire.com/off-the-wire/nvidia-unveils-50-new-updated-ai-tools-and-trainings-for-developers/

162.Best Workflow and Pipeline Orchestration Tools https://neptune.ai/blog/best-workflow-and-pipeline-orchestration-tools?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-best-workflow-and-pipeline-orchestration-tools

164.notes Data Science & Machine Learning https://chrisalbon.com/

165.black uncompromising Python code formatter https://awesomeopensource.com/project/psf/black

166.Feature stores https://www.kdnuggets.com/2021/05/feature-stores-how-avoid-feeling-every-day-is-groundhog-day.html https://neptune.ai/blog/feature-stores-components-of-a-data-science-factory-guide?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-feature-stores-components-of-a-data-science-factory-guide

167.Code and Notebook Versioning for ML Teams https://neptune.ai/blog/code-and-notebook-versioning-for-ml-teams-guide?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-code-and-notebook-versioning-for-ml-teams-guide

10 tools that can serve as a great alternative to the different parts of ClearML https://neptune.ai/blog/clear-ml-alternatives?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-clear-ml-alternatives

168.3 Tools to Track and Visualize the Execution of your Python Code https://towardsdatascience.com/3-tools-to-track-and-visualize-the-execution-of-your-python-code-666a153e435e

Follow leaders in the field to update yourself in the field

1.Linkedin

2.Twitter

CPU/GPU/TPU

1.Google cloab (FREE)  Jupyter Lab for Python, R, Swift from Google Colab with ColabCode  https://www.youtube.com/watch?v=Q35WIqZoUF4

https://www.analyticsvidhya.com/blog/2021/01/avid-user-of-google-colab-here-are-some-alternatives-of-google-colab-that-you-should-know-about/?utm_source=linkedin&utm_medium=social&utm_campaign=old-blog&utm_content=B&custom=FBI156

https://towardsdatascience.com/use-colab-more-efficiently-with-these-hacks-fc89ef1162d8 https://www.analyticsvidhya.com/blog/2021/05/10-colab-tips-and-hacks-for-efficient-use-of-it/

ColabCode This is an amazing extension to the already available resource, Google Colab  https://github.com/abhi1thakur/colabcode 

GitHub notebooks with Google Colab https://www.youtube.com/watch?v=LmIylxNmA-A&feature=youtu.be  

colab_everything Python library to run streamlit, flask, fastapi, etc on google colab  https://github.com/Ankur-singh/colab_everything/

2.Kaggle kernel(read terms and conditions before use) (FREE)

3.Paperspace Gradient(read terms and conditions before use)

4.knime - https://www.knime.com/(read terms and conditions before use)

5.RapidMiner (read terms and conditions before use)

https://github.com/zszazi/Deep-learning-in-cloud

So what next ?

participate online competition and do project and apply to intership ,job,solving real world problems, etc...

applications of data science in many industry

1.E-commerce- Identifying consumers,Recommending Products,Analyzing Reviews

2.Manufacturing- Predicting potential problems,Monitoring systems,Automating manufacturing units, Maintenance Scheduling,Anomaly Detection

3.Banking- Fraud detection,Credit risk modeling,Customer lifetime value

4.Healthcare- Medical image analysis, Drug discovery,Bioinformatics,Virtual Assistants,image segmentation

5.Transport- Self-driving cars,Enhanced driving experience,Car monitoring system,Enhancing the safety of passengers

6.Finance- Customer segmentation,Strategic decision making,Algorithmic trading,Risk analytics

7.Marketing (Added from comments Credits: Jawad Ali)- LTV predictions,Predictive analytics for customer behavior,Ad targeting

and many more fields - https://www.topbots.com/enterprise-ai-companies-2020/ , https://venturebeat.com/2020/10/21/the-2020-data-and-ai-landscape/

Research blogs https://www.theinsaneapp.com/2021/04/top-machine-learning-blogs-to-follow-in-2021.html

https://reconshell.com/top-ai-and-machine-learning-blogs-curated-for-ai-enthusiasts/

1.https://ai.facebook.com/ https://ai.facebook.com/blog/

2.https://ai.googleblog.com/

3.https://deepmind.com/blog https://deepai.org/definitions

4.https://openai.com/blog/

5.https://www.malongtech.com/en/research.html

6.https://blogs.nvidia.com/blog/tag/artificial-intelligence/ https://blogs.nvidia.com/

https://ai.googleblog.com/2021/01/google-research-looking-back-at-2020.html?m=1

7.https://blog.tensorflow.org/

8.https://pytorch.org/blog/

9.https://distill.pub/

kdnuggets.com

https://www.kdnuggets.com/2020/01/top-10-ai-ml-articles-to-know.html

RESEARCH LABS IN THE WORLD

https://ai.facebook.com/ https://ai.googleblog.com/ https://research.google/ https://ai.google/research/

1.The Alan Turing Institute:https://www.turing.ac.uk/

2.J.P. Morgan AI Research Lab:https://www.jpmorgan.com/insights/tec...

3.Oxford ML Research Group:http://www.robots.ox.ac.uk/~parg/proj...

4.Microsoft Research Lab- AI:https://www.microsoft.com/en-us/resea...

5.Berkeley AI Research:https://bair.berkeley.edu/

6.LIVIA:https://en.etsmtl.ca/Unites-de-recher...

7.MIT Computer Science and Artificial :https://www.csail.mit.edu/

online competitions:

Top 25 Machine Learning Hackathons https://medium.com/analytics-vidhya/top-25-machine-learning-hackathons-its-here-now-for-anyone-to-move-to-data-science-a93deb2a198a

1.Kaggle-https://www.kaggle.com/

kaggle-solutions https://awesomeopensource.com/project/faridrashidi/kaggle-solutions

2.hackerearth-https://www.hackerearth.com/challenges/

3.machinehack-https://www.machinehack.com/

4.analyticsvidhya-https://datahack.analyticsvidhya.com/contest/all/

5.zindi-https://zindi.africa/competitions

6.crowdai-https://www.crowdai.org/

7.driven data-https://www.drivendata.org/

8.dockship-https://dockship.io/Runway AI

9.SIGNATE Competition- https://signate.jp/about?rf=competition_about

9.International Data Analysis Olympiad (IDAHO)

10.Codalab

11.Iron Viz

12.Data Science Challenges

13.Tianchi Big Data Competition

14.https://www.techgig.com/hackathon/ml_hackathon

15.https://www.openml.org/

https://towardsdatascience.com/12-data-science-ai-competitions-to-advance-your-skills-in-2021-32e3fcb95d8c https://www.kdnuggets.com/2020/09/international-alternatives-kaggle-data-science-competitions.html

Some useful content :

  1. H20.ai automl, google automl,google ml kit(https://developers.google.com/ml-kit) ,Azure Cognitive Services,Azure Machine Learning Service,amazon ml,Azure Machine Learning Studio,Google Cloud Platform,gcp automl ision,Weka,AutoWeka,Microsoft Cognitive Toolkit,Google Cloud AutoML,DataRobot AutoML,Databricks AutoML,Azure ML,azure machine learning studio,IBM Watson ml studio,AWS Sagemaker Studio,aws rekognition,Google AI Platform,Databricks,Domino Data Lab,roboflow

H2O Driverless AI https://www.h2o.ai/products/h2o-driverless-ai/

H2O Flow - Web Based Machine Learning Development https://docs.h2o.ai/h2o/latest-stable/h2o-docs/flow.html https://www.analyticsvidhya.com/blog/2021/05/a-step-by-step-guide-to-automl-with-h2o-flow/

https://docs.microsoft.com/en-us/azure/machine-learning/algorithm-cheat-sheet

https://neptune.ai/blog/best-machine-learning-as-a-service-platforms-mlaas?utm_source=twitter&utm_medium=tweet&utm_campaign=blog-best-machine-learning-as-a-service-platforms-mlaas

https://codegnan.com/blog/35-best-data-sciecne-tools-for-beginners-to-master/ https://analyticsindiamag.com/free-online-resources-to-learn-automl/

https://analyticsindiamag.com/10-popular-automl-tools-developers-can-use/ https://analyticsindiamag.com/8-best-open-source-tools-for-data-mining/

mlkit-https://firebase.google.com/products/ml runway https://runwayml.com/ fritz https://www.fritz.ai/

obviously https://www.obviously.ai/ createml https://developer.apple.com/machine-learning/create-ml/ makeml https://makeml.app/

superannotate https://superannotate.com/ https://rapidminer.com/ https://monkeylearn.com/monkeylearn-studio/ https://nanonets.com/

GCP Professional ML Engineer certification in 8 days https://ml-rafiqhasan.medium.com/how-i-cracked-the-gcp-professional-ml-engineer-certification-in-8-days-f341cf0bc5a0

Vertex AI, one platform, every ML tool you need https://cloud.google.com/vertex-ai

2.Automated model architecture search tools (e.g. darts, enas) https://awesomeopensource.com/projects/automl

https://github.com/search?q=automl https://www.kdnuggets.com/2016/03/automated-data-science.html https://www.kdnuggets.com/software/automated-data-science.html

Tpot https://awesomeopensource.com/project/EpistasisLab/tpot

ATOM https://awesomeopensource.com/project/tvdboom/ATOM https://towardsdatascience.com/how-to-test-multiple-machine-learning-pipelines-with-just-a-few-lines-of-python-1a16cb4686d

mljar-supervised https://awesomeopensource.com/project/mljar/mljar-supervised

libra end-to-end machine learning process in just one line of code https://awesomeopensource.com/project/Palashio/libra

featurewiz, boruta_py ,AutoWeka,Auto-Sklearn,AutoGluon

GML https://awesomeopensource.com/project/Muhammad4hmed/GML

auto_ml https://awesomeopensource.com/project/ClimbsRocks/auto_ml

pywedge https://awesomeopensource.com/project/taknev83/pywedge https://towardsdatascience.com/automated-interactive-package-for-eda-modeling-and-hyperparameter-tuning-in-a-few-lines-of-228c561fa63c

LightAutoML https://awesomeopensource.com/project/sberbank-ai-lab/LightAutoML https://lightautoml.readthedocs.io/en/latest/ https://towardsdatascience.com/lightautoml-preset-usage-tutorial-2cce7da6f936

FLAML fast and lightweight AutoML library https://awesomeopensource.com/project/microsoft/FLAML

LightAutoML LAMA - automatic model creation framework https://awesomeopensource.com/project/sberbank-ai-lab/LightAutoML

EvalML is an AutoML library https://awesomeopensource.com/project/alteryx/evalml https://evalml.alteryx.com/en/stable/ https://www.kdnuggets.com/2021/04/easy-automl-python.html https://www.youtube.com/watch?v=uuYEQqrExBQ https://www.analyticsvidhya.com/blog/2021/05/machine-learning-automation-using-evalml-library/

dataprep Beginners Guide to Automation in Data Science https://www.analyticsvidhya.com/blog/2021/04/beginners-guide-to-automation-in-data-science/

A machine learning tool for automated prediction engineering https://awesomeopensource.com/project/alteryx/compose

adanet https://awesomeopensource.com/project/tensorflow/adanet

mljar-supervised https://github.com/mljar/mljar-supervised/ https://www.kdnuggets.com/2021/05/binary-classification-automated-machine-learning.html

ludwig https://awesomeopensource.com/project/ludwig-ai/ludwig

carefree-learn is a minimal Automatic Machine Learning (AutoML) solution for tabular datasets based on PyTorch https://carefree0910.me/carefree-learn-doc/

autoweka https://awesomeopensource.com/project/automl/autoweka

autokeras https://autokeras.com/ autoSklearn https://automl.github.io/auto-sklearn/master/

adanet https://awesomeopensource.com/project/tensorflow/adanet

AlphaPy Automated Machine Learning https://awesomeopensource.com/project/ScottfreeLLC/AlphaPy

TransmogrifAI https://awesomeopensource.com/project/salesforce/TransmogrifAI

Hugging Faceโ€™s AutoNLP https://www.analyticsvidhya.com/blog/2021/03/a-hands-on-introduction-to-hugging-faces-autonlp-101/

Automated Text Classification with EvalML https://www.kdnuggets.com/2021/04/automated-text-classification-evalml.html

Pywedge A complete package for EDA, Data Preprocessing and Modelling https://towardsdatascience.com/pywedge-a-complete-package-for-eda-data-preprocessing-and-modelling-32171702a1e0

3.awesome-AutoML https://awesomeopensource.com/project/windmaple/awesome-AutoML , automl-gs github.com/minimaxir/automl-gs

autopandas,Auto-Sklearn,scikit-optimize,Hyperopt-Sklearn,smac3,alphapy,nni,adanet,ludwig, TPOT,flaml, H2OAutoML ,automl ,LightAutoML,auto keras,MLJAR

Auto-PyTorch,Keras Tuner,DataRobot, DriverlessAI , MLBox, AutoGluon, autoweka, Amazon Lex,Darwin,AdaNet, Microsoft NNI,GradsFlow,Ludwig

Automated Tensorflow https://awesomeopensource.com/project/rafiqhasan/auto-tensorflow

MLBox https://awesomeopensource.com/project/AxeldeRomblay/MLBox

quick_ml https://pypi.org/project/quick-ml/ https://www.quickml.info/

MLJAR https://github.com/mljar/mljar-supervised/ https://towardsdatascience.com/binary-classification-with-automated-machine-learning-1a36e78ba50f

TransmogrifAI https://awesomeopensource.com/project/salesforce/TransmogrifAI darwin http://drwn.anu.edu.au/

GenoML (AutoML) for Genomics https://genoml.com/ https://github.com/GenoML

baytune https://www.kdnuggets.com/2021/03/automating-machine-learning-model-optimization.html https://awesomeopensource.com/project/MLBazaar/BTB

adanet https://awesomeopensource.com/project/tensorflow/adanet

FEDOT Automated modeling and machine learning framework FEDOT https://awesomeopensource.com/project/nccr-itmo/FEDOT

4.AutoGluon AutoML for Text, Image, and Tabular Data https://analyticsindiamag.com/how-to-automate-machine-learning-tasks-using-autogluon/

AutoGL: The First Ever AutoML Framework for Graph Datasets https://analyticsindiamag.com/meet-autogl-the-first-ever-automl-framework-for-graph-datasets/

  1. autosklearn,autokeras,LightAutoML,xcessiv,kerastuner ,LAMA, NNI, FEDOT (https://awesomeopensource.com/project/sberbank-ai-lab/LightAutoML)

Keras Tuner or storm-tuner - Decide Number of Hidden Layers And Neuron In Neural Network

AutoNeuro https://autoneuro.challenge-ineuron.in/

ATOM https://towardsdatascience.com/atom-a-python-package-for-fast-exploration-of-machine-learning-pipelines-653956a16e7b https://awesomeopensource.com/project/tvdboom/ATOM

  1. autoviml https://awesomeopensource.com/project/AutoViML/Auto_ViML https://towardsdatascience.com/autoviml-automating-machine-learning-4792fee6ae1e

    deep_autoviml https://awesomeopensource.com/project/AutoViML/deep_autoviml

    ๐—ฎ๐˜‚๐˜๐—ผ๐—บ๐—ฎ๐˜๐—ฒ ๐—บ๐—ผ๐˜€๐˜ ๐—ผ๐—ณ ๐˜๐—ต๐—ฒ ๐—ฑ๐—ฎ๐˜๐—ฎ ๐˜€๐—ฐ๐—ถ๐—ฒ๐—ป๐—ฐ๐—ฒ https://awesomeopensource.com/project/Muhammad4hmed/GML

    CodeLess https://pypi.org/project/codeless/ https://awesomeopensource.com/project/porky5191/codeless_demo_project

    AUTORL: AUTOML FOR RL https://www.automl.org/blog-autorl/

  2. sweetviz (EDA purpose) - https://pypi.org/project/sweetviz/ https://www.kdnuggets.com/2021/03/know-your-data-much-faster-sweetviz-python-library.html

  3. pandasprofiling(display whole EDA) - https://pypi.org/project/pandas-profiling/ https://pandas-profiling.github.io/pandas-profiling/docs/master/rtd/index.html

  4. autokeras,AutoSklearn,Neural Network Intelligence

    FeatureTools automated feature engineering.

    MLBox,Lightwood,mindsdb(machine learning models using SQL queries),mljar-supervised,Ludwig(deep learning models without the need to write code)

    AdaNet is a lightweight TensorFlow-based framework

  5. pycaret- https://pycaret.org/ https://www.kdnuggets.com/2020/08/build-automl-pycaret.html https://www.kdnuggets.com/2020/08/github-best-automl-ever-need.html https://www.kdnuggets.com/2020/07/5-things-pycaret.html

Machine Learning in Power BI using PyCaret https://www.kdnuggets.com/2020/05/machine-learning-power-bi-pycaret.html

https://towardsdatascience.com/build-your-first-anomaly-detector-in-power-bi-using-pycaret-2b41b363244e

https://www.kdnuggets.com/2020/06/deploy-machine-learning-pipeline-cloud-docker.html https://www.kdnuggets.com/2020/08/github-best-automl-ever-need.html

mindsdb Machine Learning in 5 Lines of Code https://mindsdb.com/

automated feature engineering https://awesomeopensource.com/project/alteryx/featuretools https://towardsdatascience.com/why-automated-feature-engineering-will-change-the-way-you-do-machine-learning-5c15bf188b96

Featuretools https://www.featuretools.com/

Automate your ML Pipelines with EvalML https://analyticsindiamag.com/automate-your-ml-pipelines-with-evalml/

Aethos โ€” A Data Science Library to Automate your Workflow https://towardsdatascience.com/aethos-a-data-science-library-to-automate-workflow-17cd76b073a4

AutoAI โ€” Automating the AI Workflow to Build & Deploy Machine Learning model https://medium.com/geekculture/autoai-automating-the-ai-workflow-to-build-deploy-machine-learning-model-bb2b727cda28

AutoML toolkit https://awesomeopensource.com/project/microsoft/nni

LightAutoML LAMA - automatic model creation framework https://awesomeopensource.com/project/sberbank-ai-lab/LightAutoML https://analyticsindiamag.com/hands-on-python-guide-to-lama-an-automatic-ml-model-creation-framework/

mljar-supervised Automates Machine Learning Pipeline with Feature Engineering and Hyper-Parameters Tuning https://awesomeopensource.com/project/mljar/mljar-supervised

MLBox is a powerful Automated Machine Learning python library https://awesomeopensource.com/project/AxeldeRomblay/MLBox

12.Auto_Timeseries by auto_ts

13.AutoNLP_Sentiment_Analysis by autoviml

14.automl lazypredict https://awesomeopensource.com/project/shankarpandala/lazypredict

AutoML Toolkit for Graph Datasets & Tasks AutoGL(Auto Graph Learning)https://medium.com/syncedreview/tsinghua-university-releases-first-automl-toolkit-for-graph-datasets-tasks-c61ea0261d78

AutoFeat-https://analyticsindiamag.com/guide-to-automatic-feature-engineering-using-autofeat/

15.https://awesomeopensource.com/project/mstaniak/autoEDA-resources

bamboolib or pandas-ui or pandas-summary or pandas_visual_analysis or Dtale(get code also) (python package for easy data exploration & transformation)

Automating EDA using Pandas Profiling, streamlit_pandas_profiling,Sweetviz and Autoviz,DataPrep,vaex,Datapane,Sweetviz,PandasGUI,Datatable,Dora,Pywedge,D-Tale,lux,Dabl,Pretty pandas,AWS Glue DataBrew,speedML,edaviz,Altair,voyager,Mito,Facets,KNIME,lux,datatable,Pandas-visual-analysis,ExploriPy,Holoviews,lux,Dataprep,atoti,QuickDA ,panel-highcharts,Know Your Data,Atoti ,ExploriPy,autoplotter,tensorflow data validation,skimpy,OpenRefine

QuickDA https://towardsdatascience.com/save-hours-of-work-doing-a-complete-eda-with-a-few-lines-of-code-45de2e60f257

Dataprep https://towardsdatascience.com/dataprep-eda-accelerate-your-eda-eb845a4088bc https://www.analyticsvidhya.com/blog/2021/05/dataprep-library-perform-eda-faster/

explainerdashboard https://towardsdatascience.com/the-quickest-way-to-build-dashboards-for-machine-learning-models-ec769825070d

Facets https://awesomeopensource.com/project/PAIR-code/facets https://towardsdatascience.com/visualize-your-data-with-facets-d11b085409bc

pywedge https://awesomeopensource.com/project/taknev83/pywedge https://towardsdatascience.com/automated-interactive-package-for-eda-modeling-and-hyperparameter-tuning-in-a-few-lines-of-228c561fa63c

Datapane makes it simple to build shareable reports from Python https://awesomeopensource.com/project/datapane/datapane https://towardsdatascience.com/datapanes-new-features-create-a-beautiful-dashboard-in-python-in-a-few-lines-of-code-a3c44523292b https://towardsdatascience.com/introduction-to-datapane-a-python-library-to-build-interactive-reports-4593fd3cb9c8

lux https://medium.com/swlh/automating-exploratory-data-analysis-part-3-d04352b83072 https://pub.towardsai.net/speed-up-eda-with-the-intelligent-lux-37f96542527b

lux Python API for Intelligent Visual Data Discovery https://awesomeopensource.com/project/lux-org/lux https://analyticsindiamag.com/python-guide-to-lux-an-interactive-visual-discovery/

Automatic EDA https://thecleverprogrammer.com/2021/02/06/automatic-eda-using-python/

Automated Interactive Package for EDA, Modeling, and Hyperparameter Tuning in a few lines of Python Code https://towardsdatascience.com/automated-interactive-package-for-eda-modeling-and-hyperparameter-tuning-in-a-few-lines-of-228c561fa63c

Arena https://awesomeopensource.com/project/ModelOriented/Arena

https://awesomeopensource.com/project/mstaniak/autoEDA-resources https://thecleverprogrammer.com/2021/02/06/automatic-eda-using-python/

ExploriPy import EDA-https://analyticsindiamag.com/hands-on-tutorial-on-exploripy-effortless-target-based-eda-tool/

Lens- Statistical Analysis of Data https://analyticsindiamag.com/hands-on-tutorial-on-lens-python-tool-for-swift-statistical-analysis/

Dashboard in Less Than 10 Lines of Code https://towardsdatascience.com/build-dashboards-in-less-than-10-lines-of-code-835e9abeae4b

Plotly Express Interprete data through interactive visualization https://pub.towardsai.net/matplotlib-is-dead-long-life-to-plotly-express-e1671dce0d18

Rich terminal dashboards https://www.willmcgugan.com/blog/tech/post/building-rich-terminal-dashboards/

Explainable AI dashboards https://awesomeopensource.com/project/oegedijk/explainerdashboard https://www.youtube.com/watch?v=ZgypAMRcmw8

Machine Learning Model Dashboard https://towardsdatascience.com/machine-learning-model-dashboard-4544daa50848

Creating Automated Python Dashboards using Plotly, Datapane, and GitHub Actions https://towardsdatascience.com/creating-automated-python-dashboards-using-plotly-datapane-and-github-actions-ff8aa8b4e3

atoti Python library to quickly build BI analytics dashboards https://docs.atoti.io/latest/tutorial/tutorial.html

interactive dashboards https://medium.com/analytics-vidhya/explainer-dashboard-build-interactive-dashboards-for-machine-learning-models-fda63e0eab9

MitoSheets https://analyticsindiamag.com/guide-to-mitosheets-harnessing-power-of-spreadsheets-in-python/

Datacleaner-https://analyticsindiamag.com/tutorial-on-datacleaner-python-tool-to-speed-up-data-cleaning-process/

Datacleaner :dora ,Voilร  -Jupyter Notebooks quickly into standalone web applications , Plotly Dash - for more advanced and production level dashboards

featurewiz(Select the best features from your data set fast with a single line of code) - https://awesomeopensource.com/project/AutoViML/featurewiz

explainerdashboard https://medium.com/analytics-vidhya/explainer-dashboard-build-interactive-dashboards-for-machine-learning-models-fda63e0eab9

interpret Dashboards for Interpreting & Comparing Machine Learning Models https://hmix13.medium.com/dashboards-for-interpreting-comparing-machine-learning-models-ffcfb4c05152

https://www.kdnuggets.com/2019/07/10-simple-hacks-speed-data-analysis-python.html

Panel - web apps

Automating report generation with Jupyter Notebooks https://medium.com/applied-data-science/full-stack-data-scientist-5-automating-report-generation-with-jupyter-notebooks-919e32e88d18

10 Useful Jupyter Notebook Extensions for a Data Scientist https://towardsdatascience.com/10-useful-jupyter-notebook-extensions-for-a-data-scientist-bd4cb472c25e

Datapane ( Build Interactive Reports) https://towardsdatascience.com/introduction-to-datapane-a-python-library-to-build-interactive-reports-4593fd3cb9c8 https://www.kdnuggets.com/news/index.html

pomegranate probabilistic modelling in Python https://awesomeopensource.com/project/jmschrei/pomegranate https://www.kdnuggets.com/2020/12/fast-intuitive-statistical-modeling-pomegranate.html

16.CUPY (array process parallel in gpu) https://pypi.org/project/cupy/

17.Dabl-automate the known 80% of Data Science which is data preprocessing, data cleaning, and feature engineering https://pypi.org/project/dabl/

18.dask (parallel comptataion) https://docs.dask.org/en/latest/ https://medium.com/rapids-ai/reading-larger-than-memory-csvs-with-rapids-and-dask-e6e27dfa6c0f#cid=av01_so-nvsh_en-us

pandarallel https://towardsdatascience.com/make-pandas-run-blazingly-fast-3dbcd621f75b

Dask Dataframe and SQL https://docs.dask.org/en/latest/dataframe-sql.html

Swiftapplyโ€Š โ€“ Automatically efficient pandas apply operations https://www.kdnuggets.com/2018/04/swiftapply-automatically-efficient-pandas-apply-operations.html

Dask CUDA

Numba https://awesomeopensource.com/project/numba/numba https://www.youtube.com/watch?v=3O-Pvnrbsu0 https://www.analyticsvidhya.com/blog/2021/04/numba-for-data-science-make-your-py-code-run-1000x-faster/

Arrow https://towardsdatascience.com/how-fast-is-reading-parquet-file-with-arrow-vs-csv-with-pandas-2f8095722e94

Cython,Numba,PyPy,ray,loky,Dask,p_tqdm (aka Pathos + tqdm),modin,connectorx,cudf, cuML

Reducing Pandas memory https://pythonspeed.com/articles/pandas-load-less-data/ https://www.youtube.com/watch?v=HNE0qHJ9A9o

Speed up Scikit-Learn Model Training https://www.kdnuggets.com/2021/02/speed-up-scikit-learn-model-training.html

thundergbm Fast GBDTs and Random Forests on GPUs https://awesomeopensource.com/project/Xtra-Computing/thundergbm

thundersvm https://awesomeopensource.com/project/Xtra-Computing/thundersvm

NumPy API on TensorFlow https://www.tensorflow.org/guide/tf_numpy https://www.youtube.com/watch?v=mgY46AEXnG0

pandas chunksize,Pandas vectorization,Modin ,Vaex,ray,Dask,PyPolars,Polars,spark,cuDF,cupy,mars,ray,Caching,rapids,joblib,snorkel,arrow,Pyarrow,Fastparquet,dampr, pandarallel ,numba,bolt, numexpr,ipython parallel,Nim,speedML,ConnectorX , apache arrow,jax https://www.youtube.com/watch?v=eJyjB3cNIB0&feature=youtu.be

19.dataprep (Understand your data with a few lines of code in seconds)

data-preparation-tools - https://improvado.io/blog/data-preparation-tools

20.Dora library is another data analysis library designed to simplify exploratory data analysis. https://pypi.org/project/Dora/

23.FlashText (A library faster than Regular Expressions for NLP tasks) https://pypi.org/project/flashtext/

24.Guietta (tool that makes simple GUIs simple) https://pypi.org/project/guietta/

pandas-visual-analysis -https://analyticsindiamag.com/hands-on-guide-to-pandas-visual-analysis-way-to-speed-up-data-visualization/

25.hummingbird (make code fastly exexcute) https://pypi.org/project/Hummingbird/ https://analyticsindiamag.com/guide-to-hummingbird-a-microsofts-library-for-expediting-traditional-machine-learning-models/

CUML- increase the speed of training your machine learning model https://towardsdatascience.com/train-your-machine-learning-model-150x-faster-with-cuml-69d0768a047a

https://docs.rapids.ai/api/cuml/stable/

modin https://www.kdnuggets.com/2021/03/speed-up-pandas-modin.html

Datatable speed up pandas https://www.youtube.com/watch?v=mQi6QIGGJ5U

Process large datasets without running out of memory https://pythonspeed.com/memory/?utm_medium=email&utm_source=topic+optin&utm_campaign=awareness&utm_content=20210426+data+ai+nl&mkt_tok=MTA3LUZNUy0wNzAAAAF8rA-uJucI5nYkInNB60OO8SozgyRZZ2ptfW-Dt-5HR3I0ysFHju2OYpeK_JZRtxcnmHGSefwL-1zg9Be3zse6zZVklh3zcWYSCxLRvJqd5LfAJMaF

Snap ML โ€” Speed Up Model Training https://medium.com/ibm-data-ai/snap-ml-speed-up-model-training-2ef36fbbf101

26.memory-profiler (tell memory consumption line by line) https://pypi.org/project/memory-profiler/

Cython A Speed-Up Tool for your Python Function https://towardsdatascience.com/cython-a-speed-up-tool-for-your-python-function-9bab64364bfd

PyPy Run Your Python Code as Fast as C https://towardsdatascience.com/run-your-python-code-as-fast-as-c-4ae49935a826

Python Tricks for Keeping Track of Your Data https://towardsdatascience.com/python-tricks-for-keeping-track-of-your-data-aef3dc817a4e

27.numexpr (incerease speed of execution of numpy) https://awesomeopensource.com/project/pydata/numexpr

pypolars instead of pandas (beating-pandas-performance) https://www.youtube.com/watch?v=1-O_KnLZEso https://towardsdatascience.com/3x-times-faster-pandas-with-pypolars-7550e605805e

50X speed up your Pandas apply function https://awesomeopensource.com/project/jmcarpenter2/swifter

sklearn 100x Faster https://www.kdnuggets.com/2019/09/train-sklearn-100x-faster.html

JAX Autograd and XLA, facilitating high-performance machine learning research https://awesomeopensource.com/project/google/jax

Numba (optimise performance of numpy and high performance python compiler) http://numba.pydata.org/

Pyston project open sources its faster Python https://www.infoworld.com/article/3618169/pyston-project-open-sources-its-faster-python.html

28.pandarallel (simple and efficient tool to parallelize your pandas computation on all your CPUs) https://pypi.org/project/pandarallel/

Pandarallel, Pandarallelโ€™s parallel_apply()

29.PDFTableExtract(by PyPDF2) https://awesomeopensource.com/project/ashima/pdf-table-extract

Camelot-https://towardsdatascience.com/extracting-tabular-data-from-pdfs-made-easy-with-camelot-80c13967cc88

30.PyImpuyte(Python package that simplifies the task of imputing missing values in big datasets) https://pypi.org/project/PyImpuyte/

31.libra(Automates the end-to-end machine learning process in just one line of code) https://pypi.org/project/libra/

32.debug code by puyton -m pdp -c continue

33.cURL (This is a useful tool for obtaining data from any server via a variety of protocols including HTTP.) https://stackabuse.com/using-curl-in-python-with-pycurl/

34.csvkit https://pypi.org/project/csvkit/

35.IPython IPython gives access to enhanced interactive python from the shell.

36.pip install faker (Create our own Dataset) https://pypi.org/project/Faker/

37.Python debugger %pdb

38.๐šŸ๐š˜๐š’๐š•๐šŠ-From notebooks to standalone web applications and dashboards https://voila.readthedocs.io/en/stable/ https://awesomeopensource.com/project/voila-dashboards/voila

39.๐š๐šœ๐š•๐šŽ๐šŠ๐š›๐š— for timeseries data https://awesomeopensource.com/project/tslearn-team/tslearn

40.texthero text-based dataset in Pandas Dataframe quickly and effortlessly https://awesomeopensource.com/project/jbesomi/texthero

41.๐š”๐šŠ๐š•๐šŽ๐š’๐š๐š˜(web-based visualization libraries like your Jupyter Notebook with zero dependencies) https://pypi.org/project/kaleido/

42.Vaex- Reading And Processing Huge Datasets in seconds https://awesomeopensource.com/project/vaexio/vaex

43.Uberโ€™s Ludwig is an Open Source Framework for Low-Code Machine Learning https://eng.uber.com/introducing-ludwig/

44.Google's TAPAS, a BERT-Based Model for Querying Tables Using Natural Language https://awesomeopensource.com/project/google-research/tapas

45.RAPIDS open GPU Data Science https://rapids.ai/

RAPIDS cuML,cudf

tick is a lightweight machine learning library https://x-datainitiative.github.io/tick/

modular machine learning framework http://www.pybrain.org/docs/

machine learning framework It supports several programming languages notably: Python, R, Java, Scala, Ruby and Lua Shogun https://github.com/shogun-toolbox/shogun/

46.pyforest Lazy-import of all popular Python Data Science libraries. Stop writing the same imports over and over again. https://pypi.org/project/pyforest/0.1.1/

47.Modin Get faster Pandas with Modin https://awesomeopensource.com/project/modin-project/modin

48.Text2Code for Jupyter notebook - https://awesomeopensource.com/project/deepklarity/jupyter-text2code , https://towardsdatascience.com/data-analysis-made-easy-text2code-for-jupyter-notebook-5380e89bb493

49.Openrefine Tool-For Data Preprocessing Without Code https://analyticsindiamag.com/openrefine-tutorial-a-tool-for-data-preprocessing-without-code/

50.Microsoft Releases Latest Version Of DeepSpeed deep learning optimisation library known as DeepSpeed- https://awesomeopensource.com/project/microsoft/DeepSpeed

https://analyticsindiamag.com/microsoft-releases-latest-version-of-deepspeed-its-python-library-for-deep-learning-optimisation/

51.4-pandas-tricks-https://towardsdatascience.com/4-pandas-tricks-that-most-people-dont-know-86a70a007993

53.autoplotter is a python package for GUI based exploratory data analysis-https://awesomeopensource.com/project/ersaurabhverma/autoplotter

54.3 NLP Interpretability Tools For Debugging Language Models-https://www.topbots.com/nlp-interpretability-tools/

55.New Algorithm For Training Sparse Neural Networks (RigL)-https://analyticsindiamag.com/rigl-google-algorithm-neural-networks/

56.Read Data from pdf and Word-PyPDF2,PDFMiner,PDFQuery,tabula-py,pdflib for Python,PDFTables,PyFPDF2

OpenCV to Extract Information From Table Images-https://analyticsindiamag.com/how-to-use-opencv-to-extract-information-from-table-images/

57.Text Annotation-https://towardsdatascience.com/tortus-e4002d95134b

58.GDMix, A Framework That Trains Efficient Personalisation Models - https://analyticsindiamag.com/linkedin-open-sources-gdmix-a-framework-that-trains-efficient-personalisation-models/

59.Learn Machine Learning Concepts Interactively-https://towardsdatascience.com/learn-machine-learning-concepts-interactively-6c3f64518da2

60.Folium, Python Library For Geographical Data Visualization-https://analyticsindiamag.com/hands-on-tutorial-on-folium-python-library-for-geographical-data-visualization/

61.GPU Technology Conference (GTC) Keynote Oct 2020-https://www.youtube.com/watch?v=Dw4oet5f0dI&list=PLZHnYvH1qtOYOfzAj7JZFwqtabM5XPku1

62.jiant nlp task-https://awesomeopensource.com/project/nyu-mll/jiant

63.painted your machine learning model-https://koaning.github.io/human-learn/

64.Vector AI-https://awesomeopensource.com/project/vector-ai/vectorai

65.NVIDIA NeMo(for Conversational AI)-https://awesomeopensource.com/project/NVIDIA/NeMo

66.Deep Learning Models Without Coding(DeepCognition)-https://analyticsindiamag.com/how-to-use-deepcognition-to-build-drag-and-drop-deep-learning-models-without-coding/

67.100 Machine Learning Projects-https://medium.com/@amankharwal/100-machine-learning-projects-aff22b22dd6e

68.Question generation using Natural Language Processing-https://github.com/ramsrigouthamg/Questgen.ai

69.PixelLib(image segmentation,Blur Background,Gray Background,Background Colour Change,Background Change)-https://awesomeopensource.com/project/ayoolaolafenwa/PixelLib

70.High-Resolution 3D Human Digitization-https://shunsukesaito.github.io/PIFuHD/

71.AI model that translates 100 languages without relying on English data - https://ai.facebook.com/blog/introducing-many-to-many-multilingual-machine-translation/

72.800 free textbooks - https://open.umn.edu/opentextbooks

73.TensorDash is an application that lets you remotely monitor your deep learning model's metrics and notifies you when your model training is completed or crashed.

https://awesomeopensource.com/project/CleanPegasus/TensorDash

HyperDash https://towardsdatascience.com/how-to-monitor-and-log-your-machine-learning-experiment-remotely-with-hyperdash-aa7106b15509

74.YellowBrick -select features, tune hyperparameters, select the best models, and understand the performance metrics.

75.Freely Available Python Books-https://rajukumarmishrablog.com/freely-available-python-books/

Collection of Python Cheat Sheets- https://rajukumarmishrablog.com/collection-of-python-cheat-sheets/

76.Add External Data to Your Pandas Dataframe - https://towardsdatascience.com/add-external-data-to-your-pandas-dataframe-with-a-one-liner-f060f80daaa4

https://www.openblender.io/#/welcome

77.visualize the model architecture-https://awesomeopensource.com/project/PerceptiLabs/PerceptiLabs

78.Train Conversational AI in 3 lines of code with NeMo and Lightning-https://towardsdatascience.com/train-conversational-ai-in-3-lines-of-code-with-nemo-and-lightning-a6088988ae37

79.Machine Learning for Healthcare by mit-https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-s897-machine-learning-for-healthcare-spring-2019/

80.pydot is an interface to Graphviz ,AutoGraph-Easy control flow for graphs,Neo4j-Graph Data Science Library,pyRDF2Vec-Representations of Entities in a Knowledge Graph,igraph,NetworkX,euler,pyvis,NEuler: No-code graph algorithms,dgl ease deep learning on graph,Graph4nlp,

7 Open Source Libraries for Deep Learning Graphs https://www.kdnuggets.com/2021/07/7-open-source-libraries-deep-learning-graphs.html

GeometricFlux.jl,PyTorch GNN, Jraph,Spektral,Graph Nets,Deep Graph Library , PyTorch Geometric

https://www.tensorflow.org/neural_structured_learning https://awesomeopensource.com/project/deepmind/graph_nets https://deepmind.com/research/open-source/graph-nets-library

https://www.kdnuggets.com/2019/09/5-graph-algorithms-data-scientists-know.html https://towardsdatascience.com/visualizing-networks-in-python-d70f4cbeb259

Pyviz https://towardsdatascience.com/interactive-network-visualization-757af376621

AutoGL: The First Ever AutoML Framework for Graph Datasets https://analyticsindiamag.com/meet-autogl-the-first-ever-automl-framework-for-graph-datasets/

https://analyticsindiamag.com/complete-guide-to-autogl-the-latest-automl-framework-for-graph-datasets/ http://mn.cs.tsinghua.edu.cn/AutoGL/

Graph Neural Networks, PySpark, Neural Cellular Automata, FB Prophet, Google Cloud and NLP codes https://awesomeopensource.com/project/RubensZimbres/Repo-2021

AmpliGraph: A Machine Learning Library For Knowledge Graphs https://analyticsindiamag.com/guide-to-ampligraph-a-machine-learning-library-for-knowledge-graphs/

open-source project for analysis of graphs or networks GrasPy / graspologic https://graspy.neurodata.io/

Pykg2vec: A Python Library for Knowledge Graph Embedding https://analyticsindiamag.com/pykg2vec/

https://www.kdnuggets.com/2019/05/60-useful-graph-visualization-libraries.html https://www.kdnuggets.com/2015/06/top-30-social-network-analysis-visualization-tools.html

84.Google Introduces Document AI (DocAI) https://www.marktechpost.com/2020/11/05/google-introduces-document-ai-docai-platform-for-automated-document-processing/

85.100 Machine Learning Projects-https://amankharwal.medium.com/100-machine-learning-projects-aff22b22dd6e

86.https://towardsdatascience.com/25-hot-new-data-tools-and-what-they-dont-do-31bf23bd8e56

87.Opacus: A high-speed library for training PyTorch models-https://ai.facebook.com/blog/introducing-opacus-a-high-speed-library-for-training-pytorch-models-with-differential-privacy

88.lazynlp https://awesomeopensource.com/project/chiphuyen/lazynlp

90.Pseudo-Labeling (deal with small datasets)https://towardsdatascience.com/pseudo-labeling-to-deal-with-small-datasets-what-why-how-fd6f903213af

91.Project List A - Comparatively Easy Wine Quality Analysis,Boston Housing Prediction,Spam Email Classification,Survival Prediction - Titanic Disaster,Stock Market Prediction Class of Flower Prediction,Bigmart Sales Prediction,Air Pollution Prediction,IMDB Prediction,Optimizing Product Price,Web Traffic Time Series Forecasting,Insurance Purchase Prediction,Tweet Classification

Project List B - Comparatively Difficult,Domain-Specific Chatbot,Fake News Detection,Human Action Recognition,Video Classification,Driver Drowsiness Detection,Medical Report Gen Using CT Scans,Sign Language Detection,Image Caption Generator,Celebrity Voice Prediction,Speech Emotion Recognition,Job Recommendation System,Interest Level in Rental Properties,Google Ads Keywords Generator

https://www.analyticsvidhya.com/blog/2018/05/24-ultimate-data-science-projects-to-boost-your-knowledge-and-skills/

https://ml-showcase.paperspace.com/ https://awesomeopensource.com/project/ashishpatel26/500-AI-Machine-learning-Deep-learning-Computer-vision-NLP-Projects-with-code

https://dev.to/hb/30-machine-learning-ai-data-science-project-ideas-gf5 https://www.theinsaneapp.com/2021/01/top-30-ai-and-ml-projects-for-2021.html

https://medium.com/coders-camp/180-data-science-and-machine-learning-projects-with-python-6191bc7b9db9

https://www.analyticsvidhya.com/blog/2020/12/10-data-science-projects-for-beginners/?utm_source=linkedin&utm_medium=AV|link|high-performance-blog|blogs|44195|0.375

https://medium.com/the-innovation/130-machine-learning-projects-solved-and-explained-605d188fb392 https://medium.com/coders-camp/96-python-projects-with-source-code-4069eb58beef

https://thecleverprogrammer.com/machine-learning/ https://www.kdnuggets.com/2020/03/20-machine-learning-datasets-project-ideas.html

https://www.analyticsvidhya.com/blog/2018/05/24-ultimate-data-science-projects-to-boost-your-knowledge-and-skills/?utm_source=linkedin&utm_medium=KJ|link|blackbelt|blogs|44081|0.625

https://www.kdnuggets.com/2021/03/10-amazing-machine-learning-projects-2020.html?utm_content=bufferc38bd&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer

https://data-flair.training/blogs/machine-learning-datasets/# https://data-flair.training/blogs/machine-learning-project-ideas/

https://data-flair.training/blogs/artificial-intelligence-ai-tutorial/ https://www.theinsaneapp.com/2020/11/data-science-projects-with-source-code.html

https://data-flair.training/blogs/cartoonify-image-opencv-python/ https://data-flair.training/blogs/python-project-calorie-calculator-django/

https://www.theinsaneapp.com/2020/11/machine-learning-projects-with-source-codes.html https://www.theinsaneapp.com/2020/11/data-science-projects-with-source-code.html

https://amankharwal.medium.com/20-machine-learning-projects-on-future-prediction-with-python-93932d9a7f7f

https://medium.com/coders-camp/20-deep-learning-projects-with-python-3c56f7e6a721 https://amankharwal.medium.com/12-machine-learning-projects-on-object-detection-46b32adc3c37

https://amankharwal.medium.com/7-python-gui-projects-for-beginners-87ae2c695d78 https://awesomeopensource.com/project/Kushal997-das/Project-Guidance

https://amankharwal.medium.com/20-machine-learning-projects-for-portfolio-81e3dbd167b1 https://amankharwal.medium.com/4-chatbot-projects-with-python-5b32fd84af37

https://amankharwal.medium.com/30-python-projects-solved-and-explained-563fd7473003

https://www.aiquotient.app/projects https://www.aiquotient.app/ https://www.mltut.com/best-machine-learning-projects-for-beginners/

https://medium.com/coders-camp/20-machine-learning-projects-on-nlp-582effe73b9c

93.The Linux Command Handbook-https://www.freecodecamp.org/news/the-linux-commands-handbook/

94.130 Machine Learning Projects Solved and Explained-https://medium.com/the-innovation/130-machine-learning-projects-solved-and-explained-605d188fb392

95.DataBrew-do drag-and-drop data cleansing

96.stratascratch- https://www.stratascratch.com/

97.5 ways to celebrate TensorFlow's 5th birthday-https://blog.google/technology/ai/5-ways-celebrate-tensorflows-5th-birthday/

98.TensorFlow.js: Machine Learning in Javascript https://blog.tensorflow.org/2018/03/introducing-tensorflowjs-machine-learning-javascript.html

99.Language Interpretability Tool open-source platform for visualization and understanding of NLP models - https://pair-code.github.io/lit/

100.Deep Learning Hardware Guide https://towardsdatascience.com/another-deep-learning-hardware-guide-73a4c35d3e86

101.johnsnowlabs- https://nlp.johnsnowlabs.com/ https://nlp.johnsnowlabs.com/docs/en/quickstart https://nlp.johnsnowlabs.com/docs/en/licensed_release_notes

104.Clarifai-https://www.clarifai.com/ https://analyticsindiamag.com/clarifai/

105.rapidly build and deploy machine learning models https://analyticsindiamag.com/top-10-datarobot-alternatives-one-must-know/

106.Hive Data full-stack AI https://thehive.ai/hive-data

107.real-time remote service to get the Keras callbacks to the telegram including the details of metrics https://awesomeopensource.com/project/ksdkamesh99/TensorGram

108.Language Interpretability Tool - https://pair-code.github.io/lit/demos/

109.Docly will handle the comments http://thedocly.io/

110.machine-learning-roadmap-2020 https://whimsical.com/machine-learning-roadmap-2020-CA7f3ykvXpnJ9Az32vYXva

112.freecodecamp - https://www.freecodecamp.org/learn

113.image_to_string (pytesseract)

Extract Tables in PDFs to pandas DataFrames - tabula-py

114.NLP Pipelines in a single line of code https://medium.com/analytics-vidhya/nlp-pipelines-in-a-single-line-of-code-500b3266ac7b

116.aitextgen #for ai text generation

117.http://introtodeeplearning.com/ http://cs231n.stanford.edu/ http://web.stanford.edu/class/cs224n/index.html#schedule https://www.youtube.com/playlist?list=PLkFD6_40KJIwhWJpGazJ9VSj9CFMkb79A https://www.youtube.com/playlist?list=PLkFD6_40KJIwhWJpGazJ9VSj9CFMkb79A https://www.youtube.com/playlist?list=PLwRJQ4m4UJjPiJP3691u-qWwPGVKzSlNP https://www.youtube.com/playlist?list=PLoROMvodv4rMC6zfYmnD7UG3LVvwaITY5

117.https://data-flair.training/blogs/data-science-tutorials-home

119.Pystiche - Create Your Artistic Image Using Pystiche https://analyticsindiamag.com/pystiche/ https://pystiche.readthedocs.io/en/latest/index.html

120.Low Light Image Enhancement using Python & Deep Learning https://github.com/soumik12345/MIRNet/ https://www.youtube.com/watch?v=b5Uz_c0JLMs

121.Koalas on Apache Spark - Pandas API https://www.youtube.com/watch?v=kOtAMiMe1JY&t=482s https://koalas.readthedocs.io/en/latest/

122.DALLยทE https://openai.com/blog/dall-e/ https://analyticsindiamag.com/comprehensive-guide-to-dall-e-by-openai-creating-images-from-text/

https://awesomeopensource.com/project/lucidrains/big-sleep https://awesomeopensource.com/project/lucidrains/deep-daze https://www.youtube.com/watch?v=lVR5kN7SjQ8&feature=youtu.be

DALLยทE Mini

Beginnerโ€™s Guide to the CLIP Model https://www.kdnuggets.com/2021/03/beginners-guide-clip-model.html https://www.kdnuggets.com/2021/03/multilingual-clip--huggingface-pytorch-lightning.html

StyleCLIP: Text Driven Image Manipulation https://analyticsindiamag.com/guide-to-styleclip-text-driven-image-manipulation/

https://sachinruk.github.io/blog/pytorch/pytorch%20lightning/loss%20function/gpu/2021/03/07/CLIP.html

123.SpeechBrain https://speechbrain.github.io/

124.Real-Time High-Resolution Background Replacement https://analyticsindiamag.com/introducing-real-time-high-resolution-background-replacement/ https://awesomeopensource.com/project/PeterL1n/BackgroundMattingV2

**** If you like my work. please buy me a coffee it motivate me -> https://www.buymeacoffee.com/achuthasubhash?new=1 ****

                                                    HAPPY LEARNING

Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
Machine Learning (30,759)ย 
Deep Learning (22,836)ย 
Pytorch (11,058)ย 
Data Science (8,790)ย 
Reinforcement Learning (3,853)ย 
Data Analysis (3,649)ย 
Dataset (3,626)ย 
Analysis (2,049)ย 
Python Library (1,273)ย 
Transfer Learning (1,011)ย 
Unsupervised Learning (930)ย 
Supervised Learning (617)ย 
Eda (606)ย 
Exploratory Data Analysis (571)ย 
Feature Engineering (440)ย 
Series (332)ย 
Federated Learning (218)ย 
Related Projects
Advertising ๐Ÿ“ฆย 9
All Projects
Application Programming Interfaces ๐Ÿ“ฆย 120
Applications ๐Ÿ“ฆย 181
Artificial Intelligence ๐Ÿ“ฆย 72
Blockchain ๐Ÿ“ฆย 70
Build Tools ๐Ÿ“ฆย 111
Cloud Computing ๐Ÿ“ฆย 79
Code Quality ๐Ÿ“ฆย 28
Collaboration ๐Ÿ“ฆย 30
Command Line Interface ๐Ÿ“ฆย 48
Community ๐Ÿ“ฆย 81
Companies ๐Ÿ“ฆย 60
Compilers ๐Ÿ“ฆย 60
Computer Science ๐Ÿ“ฆย 74
Configuration Management ๐Ÿ“ฆย 39
Content Management ๐Ÿ“ฆย 167
Control Flow ๐Ÿ“ฆย 197
Data Formats ๐Ÿ“ฆย 77
Data Processing ๐Ÿ“ฆย 266
Data Storage ๐Ÿ“ฆย 132
Economics ๐Ÿ“ฆย 60
Frameworks ๐Ÿ“ฆย 198
Games ๐Ÿ“ฆย 122
Graphics ๐Ÿ“ฆย 103
Hardware ๐Ÿ“ฆย 148
Integrated Development Environments ๐Ÿ“ฆย 47
Learning Resources ๐Ÿ“ฆย 147
Legal ๐Ÿ“ฆย 28
Libraries ๐Ÿ“ฆย 119
Lists Of Projects ๐Ÿ“ฆย 21
Machine Learning ๐Ÿ“ฆย 336
Mapping ๐Ÿ“ฆย 61
Marketing ๐Ÿ“ฆย 15
Mathematics ๐Ÿ“ฆย 55
Media ๐Ÿ“ฆย 228
Messaging ๐Ÿ“ฆย 97
Networking ๐Ÿ“ฆย 304
Operating Systems ๐Ÿ“ฆย 84
Operations ๐Ÿ“ฆย 120
Package Managers ๐Ÿ“ฆย 52
Programming Languages ๐Ÿ“ฆย 229
Runtime Environments ๐Ÿ“ฆย 96
Science ๐Ÿ“ฆย 42
Security ๐Ÿ“ฆย 375
Social Media ๐Ÿ“ฆย 26
Software Architecture ๐Ÿ“ฆย 70
Software Development ๐Ÿ“ฆย 68
Software Performance ๐Ÿ“ฆย 57
Software Quality ๐Ÿ“ฆย 127
Text Editors ๐Ÿ“ฆย 45
Text Processing ๐Ÿ“ฆย 131
User Interface ๐Ÿ“ฆย 310
User Interface Components ๐Ÿ“ฆย 465
Version Control ๐Ÿ“ฆย 29
Virtualization ๐Ÿ“ฆย 68
Web Browsers ๐Ÿ“ฆย 38
Web Servers ๐Ÿ“ฆย 25
Web User Interface ๐Ÿ“ฆย 194