## SI231B: Matrix ComputationsProf. Ziping Zhao, ShanghaiTech University, Fall Term 2020-21.
## Course Descriptions
This course consists of several parts. The first part focuses on various matrix factorizations, such as eigendecomposition, singular value decomposition, Schur decomposition, QZ decomposition and nonnegative factorization. The second part considers important matrix operations and solutions such as matrix inversion lemmas, linear system of equations, least squares, subspace projections, Kronecker product, Hadamard product and the vectorization operator. Sensitivity and computational aspects are also studied. The third part explores presently frontier or further advanced topics, such as matrix calculus and its various applications, deep learning, tensor decomposition, and compressive sensing (or managing undetermined systems of equations via sparsity). Especially, matrix concepts are key for understanding and creating machine learning algorithms, and hence, a special focus will be given on how matrix computations are applied to neural networks.
In each part, the relevance to engineering fields is emphasized and applications are showcased.
## AnnouncementsTime: Tue/Thu 10:15am-11:55am, Venue: Rm. 101, Teaching Center. Piazza: https://piazza.com/shanghaitech/fall2020/si231b Gradescope: see the HW's
## PrerequisitesCompulsory: Linear Algebra, Mathematical Analysis or Advanced Calculus, Probability and Statistics, Linear Programming, SI151 Optimization and Machine Learning. Recommended/Postrequisites: Mathematical and Numerical Optimization, Machine Learning.
## Textbooks and Optional References## TextbookGene H. Golub and Charles F. Van Loan, *Matrix Computations (Fourth edition)*, The John Hopkins University Press, 2013.
## ReferencesRoger A. Horn and Charles R. Johnson, *Matrix Analysis (Second Edition)*, Cambridge University Press, 2012.Jan R. Magnus and Heinz Neudecker, *Matrix Differential Calculus with Applications in Statistics and Econometrics (Third Edition)*, John Wiley and Sons, New York, 2007.Gilbert Strang, *Linear Algebra and Learning from Data*, Wellesley-Cambridge Press, 2019.Carl D. Meyer, *Matrix Analysis and Applied Linear Algebra*, SIAM (Society for Industrial and Applied Mathematics), 2000.Alan J. Laub, *Matrix Analysis for Scientists & Engineers*, SIAM (Society for Industrial and Applied Mathematics), 2004.Xianda Zhang, *Matrix Analysis and Applications*, Cambridge University Press, 2017.Xianda Zhang, *Matrix Analysis and Applications (Second Edition)*, Tsinghua University Press, 2013. (a ref. in Chinese)
## SchedulePart I Lecture 0: Overview (0.5 week) Lecture 1: Basic Concepts (1 week)
Part II Lecture 2: Linear Systems (1.5 week) Lecture 3: Least Squares (1 weeks) Lecture 4: Orthogonalization and QR Decomposition (1 week) Lecture 5: Eigenvalues, Eigenvectors, and Eigendecomposition (2.5 weeks) Lecture 6: Positive Semidefinite Matrices (0.5 week) Lecture 7: Singular Values, Singular Vectors, and Singular Value Decomposition (1 week) Lecture 8: Least Squares Revisited (1 week) Lecture 9: Kronecker Product and Hadamard Product (0.5 week) Lecture 10: Review (0.5 week)
Part III (optional) Extra Lecture 1: Multilinear/Tensor Algebra and Tensor Decomposition (guest lecture: [Part 1 slides] [Part 2 slides]) (Last updated: Nov. 28th) Extra Lecture 2: Neural Networks Extra Lecture 3: Matrix Calculus Extra Lecture 4: Reduced-Rank Regression Extra Lecture 5: Randomized Linear Algebra Extra Lecture 6: Gaussian Mixture Models Extra Lecture 7: Regression, Lasso, and Graphical Lasso
## Assessment30% assignments, 40% mid-term exam, 30% final project. ## Academic Integrity PolicyGroup study and collaboration on problem sets are encouraged, as working together is a great way to understand new materials. Students are free to discuss the homework problems with anyone under the following conditions: Students must write down their own solutions. Plagiarism is never allowed. Similar answers, MATLAB/Python/R codes, etc., found in HWs will invite you into suspected plagiarism investigation. Students must list the names of their collaborators (i.e., anyone with whom the assignment was discussed). Students can not use old solution sets from other classes under any circumstances, unless the instructor grants special permission.
Students are encouraged to read the ShanghaiTech Policy on Academic Integrity. |