22. Gradient Descent: Downhill to a Minimum

Lecture 24 of 36 in Matrix Methods in Data Analysis (MIT 18.065) by Free Academy. 53 min. Watch on YouTube

Course: Matrix Methods in Data Analysis (MIT 18.065) (Mathematics). Covers: svd pca least squares low rank deep learning linear algebra.

All lectures in Matrix Methods in Data Analysis (MIT 18.065)

  1. Course Introduction of 18.065 by Professor Strang · 7 min
  2. An Interview with Gilbert Strang on Teaching Matrix Methods in Data Analysis, Signal Processing,... · 8 min
  3. Lecture 1: The Column Space of A Contains All Vectors Ax · 52 min
  4. Lecture 2: Multiplying and Factoring Matrices · 48 min
  5. 3. Orthonormal Columns in Q Give Q'Q = I · 49 min
  6. 4. Eigenvalues and Eigenvectors · 49 min
  7. 5. Positive Definite and Semidefinite Matrices · 45 min
  8. 6. Singular Value Decomposition (SVD) · 54 min
  9. 7. Eckart-Young: The Closest Rank k Matrix to A · 47 min
  10. Lecture 8: Norms of Vectors and Matrices · 49 min
  11. 9. Four Ways to Solve Least Squares Problems · 50 min
  12. Lecture 10: Survey of Difficulties with Ax = b · 50 min
  13. Lecture 11: Minimizing ‖x‖ Subject to Ax = b · 50 min
  14. 12. Computing Eigenvalues and Singular Values · 49 min
  15. Lecture 13: Randomized Matrix Multiplication · 52 min
  16. 14. Low Rank Changes in A and Its Inverse · 51 min
  17. 15. Matrices A(t) Depending on t, Derivative = dA/dt · 51 min
  18. 16. Derivatives of Inverse and Singular Values · 43 min
  19. Lecture 17: Rapidly Decreasing Singular Values · 51 min
  20. Lecture 18: Counting Parameters in SVD, LU, QR, Saddle Points · 49 min
  21. 19. Saddle Points Continued, Maxmin Principle · 52 min
  22. 20. Definitions and Inequalities · 55 min
  23. Lecture 21: Minimizing a Function Step by Step · 54 min
  24. 22. Gradient Descent: Downhill to a Minimum · 53 min
  25. 23. Accelerating Gradient Descent (Use Momentum) · 49 min
  26. 24. Linear Programming and Two-Person Games · 54 min
  27. 25. Stochastic Gradient Descent · 53 min
  28. 26. Structure of Neural Nets for Deep Learning · 53 min
  29. 27. Backpropagation: Find Partial Derivatives · 53 min
  30. Lecture 30: Completing a Rank-One Matrix, Circulants! · 50 min
  31. 31. Eigenvectors of Circulant Matrices: Fourier Matrix · 53 min
  32. Lecture 32: ImageNet is a Convolutional Neural Network (CNN), The Convolution Rule · 47 min
  33. 33. Neural Nets and the Learning Function · 56 min
  34. 34. Distance Matrices, Procrustes Problem · 29 min
  35. 35. Finding Clusters in Graphs · 35 min
  36. Lecture 36: Alan Edelman and Julia Language · 38 min