Back to Teaching page

Elements of Linear Algebra

Constructor University, Fall 2024

Official Class Description from Campusnet

This module is the first in a sequence introducing mathematical methods at the university level in a form relevant for study and research in the quantitative natural sciences, engineering, Computer Science. The emphasis in these modules is on training operational skills and recognizing mathematical structures in a problem context. Mathematical rigor is used where appropriate. However, a full axiomatic treatment of the subject is provided in the first-year modules "Analysis" and "Linear Algebra". The lecture comprises the following topics

• Review of elementary analytic geometry
• Vector spaces, linear independence, bases, coordinates
• Matrices and matrix algebra
• Solving linear systems by Gauss elimination, structure of general solution
• LU decomposition and matrix inverse
• Linear maps and connection to matrices
• Determinant
• Eigenvalues and eigenvector
• Hermitian and skew-Hermitian matrices
• Orthonormal bases, Gram-Schmidt orthonormalization and QR decomposition
• Fourier transform
• Singular value decomposition
• Principal Component Analysis and best low rank approximations

News

• First class is on Mon., Sep. 2, 2024, in RLH-172 (Conrad Naber Lecture Hall). The first tutorial is on Wed., Sep. 4, 2024 in East Wing (IRC).

Contact Information

Time and Place

How is this class organized?

In each week, you are supposed to:

• Watch the videos of the two corresponding sessions via MS Teams (links below).
• Come to class (example/question session) on Tuesdays at 14:15 (held by the instructor).
• Come to the tutorial on Mondays at 14:15 (held by the TAs).
• Work on the moodle exercises/quizzes (online multiple choice questions) and submit them before the following tutorial.
• Every other week, work on the homework sheets (linked below as pdfs), and submit your handwritten solution to moodle on the due date before the tutorial.

Textbooks

• Gilbert Strang: Introduction to Linear Algebra (Fifth Edition, 2016)
• S.A. Leduc: Linear Algebra (Hoboken: Wiley, 2003)
• Riley, Hobson, Bence (RHB): Mathematical Methods for Physics and Engineering (3rd edition, Cambridge, 2006)

Table of Contents

Chapter 1: Basic Calculus Review
1.1: Numbers and Polynomials
1.2: Functions

Chapter 2: Vectors and Vector Spaces
2.1: Elementary Analytical Geometry
2.2: Vector Spaces

Chapter 3: Matrices and Linear Equations
3.1: Matrices and Linear Maps
3.2: Systems of Linear Equations
3.3: Matrix Inverse

Chapter 4: Determinants

Chapter 5: Eigenvalues and Eigenvectors
5.1: Eigenvalues/vectors
5.2: Eigenspaces
5.3: Diagonalization

Chapter 6: Special Types of Matrices
6.1: Normal Matrices
6.2: Hermitian/Self-adjoint Matrices
6.3: Real Symmetric Matrices
6.4: Unitary and Orthogonal Matrices

Chapter 7: Matrix Decompositions
7.1: LU Decomposition
7.2: QR Decomposition
7.3: Singular Value Decomposition
7.4: Principal Component Analysis and Best Low-Rank Approximation

Grading

The grade is only based on the final exam. Moodle-quizzes and bi-weekly homework submissions can each provide up to 5% bonus points (i.e., up to 10% bonus points in total can be achieved) according to the following table:

HW percentage solved	Bonus percentage
80 or more	5
60 - 79	4
40 - 59	3
20 - 39	2
5 - 19	1
less than 5	0

Exams

There will be one final exam (centrally scheduled in December) and one make-up final exam (centrally scheduled in January).

Here are links to some previous exams for Calculus and Linear Algebra. These exams cover mostly Calculus and only a bit less than the first half of our class.

Fall 2022 Mock Exam. Solutions. Only Exercises 3, 10, 11, 12, and 14 are relevant.
Fall 2022 Final Exam. Solutions. Only Exercises 2, 10, 11, 12, and 14 are relevant.
Fall 2020 Final Exam. Solutions. Only Exercises 8, 9, and 10 are relevant.

Here are links to some previous exams for Calculus and Linear Algebra II. These exams cover mostly Calculus II and most of the second half of our class.

Spring 2020 Practice Exam 1. Only Exercises 5 and 6 are relevant. (Sorry, I have no solutions available.)
Spring 2020 Practice Exam 2. Only Exercises 5 and 6 are relevant. (Sorry, I have no solutions available.)

Practice, Practice, Practice

An essential component for doing well in this class is to work on practice exercises. Math is about problem solving (as are almost all sciences)! During this course lots of possibilities for solving exercises are provided on moodle, in the example sessions, and in the tutorial, see below.

Moodle Exercises

Please go to moodle, login, and select the Elements of Linear Algebra class to view the exercises, and the solutions (after the due date). Each week on Monday a new quiz is released, and this is due the following week before the tutorial.

Homework Exercises

These are released bi-weekly, and scans of handwritten solutions are to be uploaded to moodle before the due date.

Extra Material

• Learn the Greek alphabet with this wikipedia article.

Class Schedule

Will be updated while class is progressing.

Below, please click on the date to download the lecture notes of this day.

(Note that the book references given below offer only a rough orientation. Sometimes, only parts of a particular chapter are covered in class.)

Date	Topics
Week 1 (Sep. 2 - 8, 2024)
Session 1 Notes Video	Topic: Review of natural, rational, real, and complex numbers You will learn about the following topics: Natural numbers, integers, rational numbers, real numbers, complex numbers Polynomials and their roots Irrationality of square root of 2 Fundamental Theorem of Algebra Literature: any Calculus textbook, RHB 1.1
Session 2 Notes Video	Topic: Functions, their inverses, and their graphs You will learn about the following topics: Definition of function, domain and range Discussion of standard functions: absolute value, parabola, hyperbola, sin, cos, tan, exponential function Inverse of a function Literature: any Calculus textbook
Example Session	More on set notation, Complex numbers, Roots of Polynomials, Roots of quadratic equations, Logarithm
pdf of moodle quiz	Please submit on moodle
pdf of homework sheet	Covering Weeks 1 and 2. Please submit on moodle
Week 2 (Sep. 9 - 15, 2024)
Session 3 Notes Video	Topic: Vectors in Euclidean space, vector operations, scalar product, cross product You will learn about the following topics: Vectors and their basic operations Length of a vector Unit vectors Scalar product and its geometrical interpretation Cross product and its geometrical interpretation Literature: selected topics from RHB Chapter 7
Session 4 Notes Video	Topic: Lines and planes You will learn about the following topics: Lines and their parametrizations Lines defined by linear equations Planes and their parametrizations Planes defined by normal vectors Planes defined by linear equations First encounter with systems of linear equations Literature: RHB 7.7
Example Session	Scalar and cross products, Vector application: centroid of a triangle, Lines and planes
pdf of moodle quiz	Please submit on moodle
Week 3 (Sep. 16 - 22, 2024)
Session 5 Notes Video	Topic: Definition of vector spaces and fields, examples, linear independence basis You will learn about the following topics: Definition of a vector space Examples and non-examples of vector spaces Definition of a field Examples of fields, e.g., finite fields Concept of linear independence Definition of basis of a vector space Dimension of a vector space Literature: RHB 8.1, 8.1.1
Session 6 Notes Video	Topic: Definition and correspondence of linear maps and matrices, basic matrix operations You will learn about the following topics: Definition of linear maps Linear maps in a chosen basis Definition of a matrix Summary of matrix operations: matrix times vector, matrix times matrix, transpose, Hermitian conjugate Literature: RHB 8.4, 8.6, 8.7. Strang 2.4
Example Session	Basis and linear independence, an example of a linear map
pdf of moodle quiz	Please submit on moodle
pdf of homework sheet	Covering Weeks 3 and 4. Please submit on moodle
Week 4 (Sep. 23 - 29, 2024)
Session 7 Notes Video	Topic: Homogeneous and inhomogeneous equations, Gaussian elimination You will learn about the following topics: Geometric intuition for systems of linear equations Relation of the solutions of homogeneous and inhomogenous equations Elementary row operations Augmented matrix notation A first example of Gaussian elimination Literature: The example in the wikipedia article is good. Strang Ch. 2.2. See also these old class notes of Marcel Oliver about Gaussian Elimination.
Session 8 Notes Video	Topic: Gaussian elimination, general case You will learn about the following topics: More examples of Gaussian elimination How to read off the general solution after Gaussian elimination Literature: same as previous session.
Example Session	Two lines in R^2, another exmaple of Gaussian elimination
pdf of moodle quiz	Please submit on moodle
Week 5 (Sep. 30 - Oct. 6, 2024)
Session 9 Notes Video	Topic: Pivots, kernel, range, rank-nullity theorem You will learn about the following topics: Reduced row-echelon form Pivots Kernel and nullity, and range/image and rank Rank-nullity theorem Relevance of these concepts for solutions to linear equations Literature: Strang Ch. 3.2, 3.3, also parts of 3.4, RHB 8.18.1
Session 10 Notes Video	Topic: Matrix inverse and its computation via Gaussian elimination, basis change You will learn about the following topics: Inverse of a matrix How to compute the inverse with Gaussian elimination Properties of the inverse Change of basis as application of the matrix inverse Literature: Strang Ch. 2.5, RHB 8.15 first half
Example Session	Matrix inverse, basis change
pdf of moodle quiz	Please submit on moodle
pdf of homework sheet	Covering Weeks 5 and 6. Please submit on moodle
Week 6 (Oct. 7 - 13, 2024)
Session 11 Notes Video	Topic: Determinant motivation, definition, and properties You will learn about the following topics: Geometric motivation for considering determinants Definition of the determinant Properties of the determinant Computing a determinant by bringing it into upper triangular form Literature: Leduc Chapter "The Determinant; Method 2 for defining the determinant". Also Strang Chapter 5 and RHB Chapter 8.9 (the presentation and order of topics is slightly different than in class).
Session 12 Notes Video	Topic: Determinants: properties, linear independence, Laplace expansion You will learn about the following topics: Relation between determinant and rank Relation between determinant and linear independence More properties of the determinant Minors, cofactors, and the Laplace expansion Literature: Leduc Chapter "The Determinant; Method 2 for defining the determinant" and "Laplace Expansions for the Determinant". Also Strang Chapter 5 and RHB Chapter 8.9 (the presentation and order of topics is slightly different than in class).
Example Session	Laplace expansion of a 4x4 matrix, Rule of Sarrus, Many Ways to compute a determinant
pdf of moodle quiz	Please submit on moodle
Week 7 (Oct. 14 - 20, 2024)
Session 13 Notes Video Summary Video	Topic: Cramer's rule, matrix inverse, Leibniz formula, summaries You will learn about the following topics: Camer's rule for solving systems of linear equations Computing the matrix inverse using cofactors / the classical adjoint The Leibniz formula to compute determinants Summary: Determinants Summary: Methods to compute determinants Summary: Matrix inverse Summary: A list of equivalences for invertible matrices Literature: Cramer's rule: in RHB Ch. 8.18.2 "N simultaneous linear equations in N unknowns" there is a small section on Cramer's rule, see also the chapter "Cramer's Rule" in Leduc's book. Invertibility: Leduc, "The classical adjoint of a square matrix", RHB Ch. 8.10 "The inverse of a matrix". Leibniz formula: Leduc "Definitions of the determinant"
Session 14 Notes Video	Topic: Motivation and definition of eigenvalues and eigenvectors You will learn about the following topics: Reflection across the diagonal as a motivating example Definition of eigenvalues and eigenvectors Characteristic polynomial and characteristic equation How to compute eigenvalues and eigenvectors Literature: Leduc: "Definition and Illustration of an Eigenvalue and an Eigenvector", "Determining the Eigenvalues of a Matrix", parts of "Determining the Eigenvectors of a Matrix". RHB: parts of Ch.s 8.13 and 8.14 (the material in that book is organized a bit differently than the lecture notes).
Example Session	Cramer's rule, matrix inverse, eigenvalues and eigenvectors of a 3x3 matrix
pdf of moodle quiz	Please submit on moodle
pdf of homework sheet	Covering Weeks 7 and 8. Please submit on moodle
Week 8 (Oct. 21 - 27, 2024)
Session 15 Notes Video	Topic: Properties of eigenvalues You will learn about the following topics: Algebraic multiplicity of eigenvalues A formula for the determinant in terms of the eigenvalues A formula for the trace in terms of the eigenvalues Eigenvalues of Hermitian and real symmetric matrices are real Eigenvalues of powers of a matrix Eigenvalues of the matrix inverse Cayley-Hamilton theorem Literature: Leduc: "Determining the Eigenvectors of a Matrix". Also RHB: parts of Ch.s 8.13 and 8.14 (the material in that book is organized a bit differently than the lecture notes). Strang Ch. 6.1.
Session 16 Notes Video	Topic: Eigenspaces, and geometric and algebraic multiplicities You will learn about the following topics: Eigenspaces Geometric multiplicity of eigenvalues A theorem about distinct eigenvalues Literature: Leduc: parts of "Eigenspaces" and "Diagonalization". Strang: Some parts of Ch. 6.2: "Matrix Powers Ak", and "Nondiagonalizable Matrices (Optional)".
Example Session	Properties of eigenvalues and eigenvectors, Google's PageRank part I
pdf of moodle quiz	Please submit on moodle
Week 9 (Oct. 28 - Nov. 03, 2024)
Session 17 Notes Video	Topic: Diagonalization and brief discussion of Jordan normal form You will learn about the following topics: Linearly independent eigenvectors Diagonalizable matrices Powers of diagonalizable matrices The exponential of a diagonalizable matrix The Jordan normal form Literature: Leduc: parts of "Diagonalization". RHB Ch. 8.16 Diagonalisation of matrices. Strang: parts of Ch. 6.2. (Strang Ch. 8.3 discusses the Jordan normal form in more detail for those who are interested.)
Session 18 Notes Video	Topic: Normal Matrices You will learn about the following topics: Definition of Hermitian conjugate Definition of normal matrix A matrix is normal if and only if it is diagonalizable with orthonormal eigenvectors Literature: RHB Ch. 8.7 The complex and Hermitian conjugates of a matrix, Ch. 8.12.7 Normal matrices, Ch. 8.13.1 Eigenvectors and eigenvalues of a normal matrix
Example Session	Normal Matrices, Diagonalization, Google's PageRank part II
pdf of moodle quiz	Please submit on moodle
pdf of homework sheet	Covering Weeks 9 and 10. Please submit on moodle
Week 10 (Nov. 04 - Nov. 10, 2024)
Session 19 Notes Video	Topic: Hermitian and real symmetric matrices, their properties, and applications You will learn about the following topics: Definition of Hermitian/self-adjoint and real symmetric matrices Definition of anti-Hermitian/skew-Hermitian matrices Eigenvalues of Hermitian and real symmetric matrices Applications of Hermitian and real symmetric matrices Literature: RHB Ch. 8.12.3 Symmetric and antisymmetric matrices, Ch. 8.12.5 Hermitian and anti-Hermitian matrices, Ch. 8.13.2 Eigenvectors and eigenvalues of Hermitian and anti-Hermitian matrices, (second half of Ch. 5.8 Stationary values of many-variable functions)
Session 20 Notes Video	Topic: Unitary and orthogonal matrices, their properties, and applications You will learn about the following topics: Definition of unitary and orthogonal matrices Properties and eigenvalues of unitary and orthogonal matrices Applications of unitary and orthogonal matrices Literature: RHB Ch. 8.12.4 Orthogonal matrices, Ch. 8.12.6 Unitary matrices, Ch. 8.13.3 Eigenvectors and eigenvalues of a unitary matrix, see also some parts of Ch. 8.16 Diagonalisation of matrices.
Example Session	Eigenvalues and eigenvectors of a Hermitian matrix, Unitary Matrix, Orthogonal Matrix
pdf of moodle quiz	Please submit on moodle
Week 11 (Nov. 11 - Nov. 17, 2024)
Session 21 Notes Video	Topic: Elementary row operations and the LU decomposition You will learn about the following topics: Matrix representations of elementary row operations How to obtain and prove the LU(P) decomposition Literature: Strang Ch. 2.6; RHB CH. 8.18.2 under "LU decomposition"
Session 22 Notes Video	Topic: Examples and applications of the LU decomposition, Cholesky decomposition You will learn about the following topics: Definition of Cholesky decomposition Examples of LU(P) decompositions How to use LU(P) decompositions to solve systems of linear equations How to use LU(P) decompositions to compute determinants How to use LU(P) decompositions to invert matrices Literature: Strang Ch. 2.6; RHB CH. 8.18.2 under "LU decomposition"
Example Session	Examples of (non-)existence and (non-)uniqueness of LU(P) decompositions, Cholesky decomposition
pdf of moodle quiz	Please submit on moodle
pdf of homework sheet	Covering Weeks 11, 12, and 13. Please submit on moodle
Week 12 (Nov. 18 - Nov. 24, 2024)
Session 23 Notes Video	Topic: Gram-Schmidt, orthogonal projections, QR decomposition for square matrices You will learn about the following topics: Gram-Schmidt orthonormalization procedure Definition of projectors and orthogonal projectors Using Gram-Schmidt to obtain a QR decomposition QR decompositions for invertible and singular matrices Using QR decompositions to compute determinants and inverses Literature: Strang Ch. 4.4 (explains things a bit differently than in class)
Session 24 Notes Video	Topic: Householder reflections, QR decomposition for non-square matrices, least-square method You will learn about the following topics: Method of Householder reflections to obtain a QR decomposition QR decompositions for matrices with more rows than columns Using QR decompositions to solve least-square problems Literature: Strang Ch. 4.4 (explains things a bit differently than in class); also Strang Ch. 11.1 "Fast Orthogonalization". The least-square problem is explained before QR decompositions a bit differently in Strang Ch. 4.3.
Example Session	QR decomposition, QR decomposition of a singular matrix, least-square method
pdf of moodle quiz	Please submit on moodle
Week 13 (Nov. 25 - Dec. 1, 2024)
Session 25 Notes Video	Topic: The singular value decomposition, its properties, and applications You will learn about the following topics: Derivation of the singular value decomposition (SVD) Precise statement of the SVD SVD and the four fundamental subspaces Using the SVD to solve homogeneous systems of linear equations Using the SVD to solve inhomogeneous systems of linear equations Literature: Strang Ch. 7; RHB Ch. 8.18.3 "Singular value decomposition"
Session 26 Notes Video	Topic: Image compression and Principal Component Analysis You will learn about the following topics: Basic idea behind using the SVD in applications Applying the SVD for image compression Using the SVD for Principal Component Analysis (PCA) Covariance matrix Perpendicular least-square problem Literature: Strang Ch. 7; RHB Ch. 8.18.3 "Singular value decomposition"
Example Session	Example of a Singular Value Decomposition, SVD and Least-square Problems
pdf of moodle quiz	Please submit on moodle
Week 14 (Dec. 2 - Dec. 8, 2024)
Review Week
Dec. 17, 2024
Final Exam
TBA
Final Exam (make-up)