Learning Resources

Other Resources

Resources from Fuse.ai OpenSessions

This resource is intended for those completely new to Machine Learning as a discipline. It includes the basics of programming with the Python programming language and contains a shallow overview of few of the several subfields of Artificial Intelligence. There are some handy exercises and mini-projects that the student may wish to use to practically apply whatever they have learnt. The content of this course is approximately 12 hours long, but the exercises and projects to be completed would extend it to be much longer and would be better if it is spread out over a couple of weeks. https://github.com/aayush-py/OpenSessions

The Open Sessions Format:

• Python Syntax, Object-oriented (anatomy of a class), and vector based calculations using NumPy.
• Basic Data Manipulation and Data Exploration using Pandas. Visualizations using matplotlib and seaborn libraries.
• Familiarity with common jargon, scikit-learn syntax, and approaches in ML using several different algorithms. Elaboration of the concept of regression (univariate and multivariate).
• Familiarity with the concepts of classification and model ensembles. Understanding the bias-variance tradeoff.
• Conceptual understanding of dimensionality reduction using PCA, t-SNE. Concept of unsupervised learning and examples with few clustering algorithms.
• Introduction to Deep Learning and related libraries.

Short Course

This course is suitable for those who wish to build a better foundation for their understanding than the first course. While this one is more intensive and would take considerably more time to complete, it is much more rewarding in the academic sense due to the depth and breadth of the subjects within. It also suits those who have already familiarized themselves with the basics of Machine Learning and wish to expand their understanding from the point they believe would be the best. The resources provided here may be used to structure your learning endeavor or as a starting point for your exploration. Additional resources in the sections at the end may be useful for practice projects based on the knowledge obtained from this course.

• A broad understanding of ML and applications.

• Python and Prerequisites

  • Python Syntax
    • https://automatetheboringstuff.com
    • https://www.oreilly.com/programming/free/files/a-whirlwind-tour-of-python.pdf
    • http://math.ecnu.edu.cn/~lfzhou/seminar/[Joel_Grus]_Data_Science_from_Scratch_First_Princ.pdf
  • Jupyter Notebook interface
    • https://www.youtube.com/watch?v=HW29067qVWk
    • https://ipython.org/notebook.html
  • Object-Oriented Programming
    • https://docs.python.org/3/tutorial/classes.html
    • https://www.youtube.com/watch?v=ZDa-Z5JzLYM
    • https://medium.com/the-renaissance-developer/python-101-object-oriented-programming-part-1-7d5d06833f26
  • Numerical calculations with NumPy
    • https://www.youtube.com/watch?v=xECXZ3tyONo
    • http://cs231n.github.io/python-numpy-tutorial/#numpy
    • https://docs.scipy.org/doc/numpy/reference/
  • Data Manipulation, Exploration, and Analysis with Pandas
    • https://www.youtube.com/watch?v=PfVxFV1ZPnk
    • https://www.youtube.com/watch?v=yzIMircGU5I&list=PL5-da3qGB5ICCsgW1MxlZ0Hq8LL5U3u9y
  • Visualization
    • Importance of visualization
    • Matplotlib documentation
    • https://www.youtube.com/watch?v=q7Bo_J8x_dw&list=PLQVvvaa0QuDfefDfXb9Yf0la1fPDKluPF
    • https://www.youtube.com/watch?v=qErBw-R2Ybk
    • Seaborn documentation
    • https://www.youtube.com/watch?v=z7ZINBk8EUk&list=PL998lXKj66MpNd0_XkEXwzTGPxY2jYM2d
    • https://www.tutorialspoint.com/seaborn/index.htm

• Machine Learning

  • Understanding ML
    • Some preliminary reading
    • Important blogs
    • 101
  • Scikit-learn
    • https://scikit-learn.org/stable/tutorial/basic/tutorial.html
    • https://www.dataquest.io/blog/sci-kit-learn-tutorial/
    • http://gael-varoquaux.info/scikit-learn-tutorial/
  • Bias-Variance tradeoff
    • http://scott.fortmann-roe.com/docs/BiasVariance.html
    • https://towardsdatascience.com/understanding-the-bias-variance-tradeoff-165e6942b229
    • CMU Slides, goes into more detail
  • Hyperparameter tuning using Grid Search, Random Search, Evolutionary Search
    • Grid Search
      • https://medium.com/datadriveninvestor/an-introduction-to-grid-search-ff57adcc0998
      • https://chrisalbon.com/machine_learning/model_evaluation/cross_validation_parameter_tuning_grid_search/
      • https://scikit-learn.org/stable/auto_examples/model_selection/plot_grid_search_digits.html
    • Random Search
    • Comparing GridSearch to RandomSearch
    • Evolutionary Search
  • Model saving and pickling
    • What Pickling (and serialization) is and why it may be used
    • How to pickle
      • https://machinelearningmastery.com/save-load-machine-learning-models-python-scikit-learn/
      • https://scikit-learn.org/stable/modules/model_persistence.html
      • Being cautious while using Pickles
  • Pipelining
    • Why and how to Pipeline
    • Documentation
  • Metrics for models
    • https://medium.com/thalus-ai/performance-metrics-for-classification-problems-in-machine-learning-part-i-b085d432082b

• Supervised Learning

  • Linear Regression
    • Simple Linear Regression
    • https://towardsdatascience.com/introduction-to-machine-learning-algorithms-linear-regression-14c4e325882a
    • https://machinelearningmastery.com/linear-regression-for-machine-learning/
  • Regularization with LASSO, Ridge, ElasticNet
    • https://towardsdatascience.com/regularization-in-machine-learning-76441ddcf99a
    • https://www.analyticsvidhya.com/blog/2017/06/a-comprehensive-guide-for-linear-ridge-and-lasso-regression/
  • Logistic Regression
    • https://www.youtube.com/watch?v=hjrYrynGWGA
  • Naive Bayes and text classification using vectorizers
  • Support Vector Machines
    • https://medium.com/machine-learning-101/chapter-2-svm-support-vector-machine-theory-f0812effc72
    • https://towardsdatascience.com/understanding-support-vector-machine-part-2-kernel-trick-mercers-theorem-e1e6848c6c4d

• Ensemble models and meta-algorithms

  • Bootstrap
    • https://machinelearningmastery.com/a-gentle-introduction-to-the-bootstrap-method/
  • Bagging
    • Random Forest
      • https://medium.com/machine-learning-101/chapter-5-random-forest-classifier-56dc7425c3e1
      • https://www.datacamp.com/community/tutorials/random-forests-classifier-python
    • Random Subspace
  • Boosting
    • https://medium.com/greyatom/a-quick-guide-to-boosting-in-ml-acf7c1585cb5
    • Adaptive Boosting
      • https://towardsdatascience.com/boosting-algorithm-adaboost-b6737a9ee60c
      • https://towardsdatascience.com/adaboost-for-dummies-breaking-down-the-math-and-its-equations-into-simple-terms-87f439757dcf
    • Gradient Boosting
      • http://blog.kaggle.com/2017/01/23/a-kaggle-master-explains-gradient-boosting/
      • https://towardsdatascience.com/understanding-gradient-boosting-machines-9be756fe76ab
    • XGBoost
      • https://xgboost.readthedocs.io/en/latest/
      • https://www.datacamp.com/community/tutorials/xgboost-in-python

• Feature Engineering

  • https://www.kaggle.com/kashnitsky/topic-6-feature-engineering-and-feature-selection
  • https://towardsdatascience.com/feature-engineering-what-powers-machine-learning-93ab191bcc2d
  • https://www.kaggle.com/willkoehrsen/automated-feature-engineering-tutorial

• Unsupervised Learning

  • Dimensionality Reduction
    • https://www.datacamp.com/community/tutorials/introduction-t-sne
    • https://skymind.ai/wiki/eigenvector
    • https://www.analyticsvidhya.com/blog/2016/03/practical-guide-principal-component-analysis-python/
  • Clustering
    • https://www.datacamp.com/community/tutorials/k-means-clustering-python
    • https://towardsdatascience.com/an-introduction-to-clustering-algorithms-in-python-123438574097
    • https://stackabuse.com/hierarchical-clustering-with-python-and-scikit-learn/

• Deep Learning Foundations

  • Neurons and Neural Networks
  • Gradient Descent
  • Loss Functions
  • Back Propagation
  • Advanced Optimization (Momentum etc.)
    • http://neuralnetworksanddeeplearning.com/chap1.html
    • http://neuralnetworksanddeeplearning.com/chap2.html
    • http://neuralnetworksanddeeplearning.com/chap3.html
    • https://distill.pub/2017/momentum/
    • https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi

• Computer Vision and Convolutional Neural Networks

  • 1D and 2D Convolution
  • Motivation for CNNs
  • Back propagation in CNNs
  • What filters learn
  • Attention and Residual CNNs
  • Object detection with CNNs
    • http://cs231n.github.io/convolutional-networks/
    • https://distill.pub/2017/feature-visualization/
    • https://towardsdatascience.com/an-overview-of-resnet-and-its-variants-5281e2f56035
    • https://towardsdatascience.com/r-cnn-fast-r-cnn-faster-r-cnn-yolo-object-detection-algorithms-36d53571365e

• Recurrent Neural Networks and Sequence Modelling

  • Motivation for RNNs
  • Simple RNN cell and Backpropagation
  • LSTM cells (and GRU)
  • Bidirectional networks
  • Sequence to Sequence (seq2seq)
    • http://colah.github.io/posts/2015-08-Understanding-LSTMs/
    • https://distill.pub/2016/augmented-rnns/

• Unsupervised Deep Learning

  • Autoencoder
    • http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/
    • https://www.jeremyjordan.me/autoencoders/
    • https://medium.com/datadriveninvestor/deep-learning-different-types-of-autoencoders-41d4fa5f7570
  • Generative Adversarial Networks
    • https://www.youtube.com/watch?v=9JpdAg6uMXs
    • https://skymind.ai/wiki/generative-adversarial-network-gan

• Natural Language Processing

  • Word Vectors (word2vec, GloVe etc.)
    • https://skymind.ai/wiki/word2vec
    • https://adventuresinmachinelearning.com/word2vec-keras-tutorial/
  • Text Classification (Intent Classification, doc2vec for Document Classification etc.)
    • https://towardsdatascience.com/machine-learning-text-processing-1d5a2d638958
    • https://towardsdatascience.com/machine-learning-word-embedding-sentiment-classification-using-keras-b83c28087456
    • https://medium.com/scaleabout/a-gentle-introduction-to-doc2vec-db3e8c0cce5e
  • Recent Approaches to NLP with Deep Learning (ELMo, BERT etc.)
    • http://jalammar.github.io/illustrated-bert/
    • https://florianwilhelm.info/2018/08/multiplicative_LSTM_for_sequence_based_recos/

Useful Blogs

• Towards Data Science
• Machine Learning Mastery
• Analytics Vidhya
• https://colah.github.io/
• http://www.wildml.com/