Resources


I keep finding cool resources all over the internet, friends send me stuff, and so on and so forth. I always loose track of them so I’m making a little bookmark page to help me keep track of stuff. If you’re reading this and you have something to add or find a dead link please put it in the comments.

Machine Learning 101:

I. Introduction to Machine Learning
http://homepages.inf.ed.ac.uk/rbf/IAPR/researchers/MLPAGES/mltut.htm
http://jeremykun.com/2012/08/04/machine-learning-introduction/
http://www.omidrouhani.com/research/machinelearning/html/machinelearning.htm
http://www.youtube.com/playlist?list=PLD63A284B7615313A (cal tech class)

II.  Linear Regression
http://en.wikipedia.org/wiki/Linear_regression
http://www.youtube.com/watch?v=ExVhaN36jBs
http://en.wikipedia.org/wiki/Simple_linear_regression
http://www.youtube.com/watch?v=ocGEhiLwDVc

III) Linear Algebra
http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebra-fall-2011/Syllabus/
https://www.khanacademy.org/math/linear-algebra
online text
http://joshua.smcvt.edu/linearalgebra/book.pdf
- see http://joshua.smcvt.edu/linearalgebra/ for usage rights

V) Linear Regression with Multiple Variables
- Gradient Descent
http://en.wikipedia.org/wiki/Gradient_descent
http://www.youtube.com/watch?v=umAeJ7LMCfU (discusses above wiki article)
http://www.youtube.com/watch?v=Dgn1ssi2p40
- Optimization
http://www.stanford.edu/class/ee364a/videos/video01.html

IV) Octave Tutorial
http://en.wikibooks.org/wiki/Octave_Programming_Tutorial

VI) Logistic Regression (LR)
http://en.wikipedia.org/wiki/Logistic_regression
http://alias-i.com/lingpipe/demos/tutorial/logistic-regression/read-me.html
http://www.ats.ucla.edu/stat/sas/library/logistic.pdf
http://www.youtube.com/watch?v=-Z2a_mzl9LM&feature=c4-overview&playnext=1&list=TLIxwITi7ngG0 (refers to LR as a classifier)

VII) Regularization
http://en.wikipedia.org/wiki/Regularization_(mathematics)
http://solon.cma.univie.ac.at/regul.html
http://www.di.ens.fr/~fbach/ecml2010tutorial/ecml_tutorial_part1.pdf
overview using advanced math
http://solon.cma.univie.ac.at/ms/regtutorial.pdf

VIII and IX) Neural Networks
http://www.youtube.com/watch?v=KuPai0ogiHk
http://www.youtube.com/watch?v=Ih5Mr93E-2c&list=PLD63A284B7615313A&index=10
- backpropagation
http://www.youtube.com/watch?v=aVId8KMsdUU
http://www.speech.sri.com/people/anand/771/html/node37.html
http://blog.zabarauskas.com/backpropagation-tutorial/

XI) Machine Learning System Design
http://people.cs.pitt.edu/~milos/courses/cs2750-Spring03/lectures/class2.pdf

Precision, recall, accuracy, …
http://en.wikipedia.org/wiki/Precision_and_recall
https://en.wikipedia.org/wiki/Accuracy_and_precision
http://stats.stackexchange.com/questions/34193/how-to-choose-an-error-metric-when-evaluating-a-class…
http://www.cs.cornell.edu/courses/cs578/2003fa/performance_measures.pdf

XII) Support Vector Machines
http://www.cs.ucf.edu/courses/cap6412/fall2009/papers/Berwick2003.pdf
http://www.cs.columbia.edu/~kathy/cs4701/documents/jason_svm_tutorial.pdf
http://www.youtube.com/watch?v=eHsErlPJWUU
http://web.mit.edu/zoya/www/SVM.pdf

XIII) Clustering
http://en.wikipedia.org/wiki/Cluster_analysis
http://en.wikipedia.org/wiki/K-means_clustering
http://www.youtube.com/watch?v=0MQEt10e4NM&feature=c4-overview&playnext=1&list=TLT3EED0Azl4Y

XIV) Dimensionality Reduction
http://en.wikipedia.org/wiki/Dimensionality_reduction
http://research.cs.tamu.edu/prism/lectures/iss/iss_l10.pdf
http://www.math.uwaterloo.ca/~aghodsib/courses/f06stat890/readings/tutorial_stat890.pdf
http://www.youtube.com/watch?v=EHIZ7Pk1XVY
http://www.youtube.com/watch?v=mz618Tesra4

XV) Anomaly Detection
www.siam.org/meetings/sdm08/TS2.ppt
http://en.wikipedia.org/wiki/Anomaly_detection

- Google Analytics http://www.google.com/analytics/
- anomaly detection with Google Analytics (example)
http://www.youtube.com/watch?v=PulNjqfToAo

Must purchase this article (I did not purchase but appears to be good) http://www.sciencedirect.com/science/article/pii/S138912860700062X

- Gaussian distribution
http://www.youtube.com/watch?v=4uiJoYVPmMw (no math)
https://en.wikipedia.org/wiki/Normal_distribution
http://www.r-tutor.com/elementary-statistics/probability-distributions/normal-distribution
https://en.wikipedia.org/wiki/Multivariate_normal_distribution

XVI) Recommender Systems
http://pages.cs.wisc.edu/~beechung/icml11-tutorial/
http://ijcai-11.iiia.csic.es/files/proceedings/Tutorial%20IJCAI%202011%20Gesamt.pdf
http://muricoca.github.io/crab/tutorial.html (using Python)

- Collaborative Filtering
www.cs.cmu.edu/~wcohen/collab-filtering-tutorial.ppt

XVII) Large Scale Machine Learning
http://i.stanford.edu/~ullman/pub/ch12.pdf
http://www.sanjivk.com/EECS6898/ (introduction to class)
(lectures) http://www.sanjivk.com/EECS6898/lectures.html
http://techtalks.tv/talks/introduction-5/57923/

- stochastic gradient descent
http://en.wikipedia.org/wiki/Stochastic_gradient_descent
http://www.youtube.com/watch?v=HvLJUsEc6dw (visualization)
http://work.caltech.edu/library/101.html

http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-832-underactuated-robotics-…
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-832-underactuated-robotics-…

- parallelized stochastic gradient descent
http://www.research.rutgers.edu/~lihong/pub/Zinkevich11Parallelized.pdf

- recursive partitioning:

http://cran.r-project.org/web/packages/rpart/vignettes/longintro.pdf

Machine Learning 201:

Advanced Machine Learning Course (CMU)

Lecture 1: Machine Learning With Scikit-Learn

Lecture 2: Machine Learning With Scikit-Learn

Lecture 3: Machine Learning from the Boston Python User Group

Andrew Ng’s Standford ML Class

An Introduction to Machine Learning

Andrew Ng’s Coursera Class Wiki

The Machine Learning Library

JMLR

CMU Google Slides 

NN Course

Some good articles on working with the command line:

command line nuggets for data science (article focuses on unix but all will work in linux bash)

intro to the command line

7 Command Line Tools for Data Scientists

Jacobian Iteration for Singular Value Decomposition:

Basic Explanation 

Stream Algorithm for SVD

Fortran:

Fortran for Beginners 

Fortran 77 Stanford Tutorial

Professional Programmer’s Guide to Fortran 77

BLAS

Fortran 77 Intrinsic Functions

Mathematics, Statistical Theory and Probability Theory:

Introduction to Probability

Rice

Chang Stochastic Processes

Durrett Probability

Very nice list of distributions

Methods of Optimization:

Gradient Descent

Basic Steepest Decent 

Newton’s Method in Optimization

CRAN Optimization and Mathematical Programming Task View

MIT OCW Optimization Methods

Boyd Optimization

Boyd Solutions Manual

Convex Optimization in R

Theoretical Computer Science:

Foundations of Computer Science

Complexity Theory a Modern Approach 

Some Really Random Stuff:

A Little Stats Cheat Sheet. Pretty basic stuff but it is a nice quick reference.

Regex Cheat Sheet

Proof wiki list of symbols with LaTex code!!

LaTex greeks, very useful.

LaTeX fonts

R:

R One pagers

R Time Series

R Statistical And Machine Learning Task View