Math 220H Notes
                                                        Matrices
                                                                    1
                                                      Joel C. Miller
                                   Spring 2013 — last updated Tuesday 21st May, 2013
                 1 c
                  
Joel C. Miller — Only for use by students of the Spring 2013 M220H course or by permission of author. If you
               ask for permission to use these notes, you will probably receive it. But please ask.
Things to add:Theorem
stating that null space of any I began writing these notes while teaching M220H (Honors Matrices) during the spring 2012 semester at
linear operator is a vector Penn State University. It was a much larger endeavor than I had anticipated, but now that I have these
space           notes, I hope to refine them and produce a better course for 2013.
                Please be aware that the examples I use in these notes will be different from the examples presented in
                lecture. The goal of this is to give you a few more examples to look at.
                Anunfortunate consequence of having a course taught out of lecture notes designed by the instructor is that
                you may end up with a narrow perspective of the subject. I will tend to teach how to solve problems the
                way I think about them, and this same emphasis will show up in my notes. There are other ways to teach
                many of these concepts, so when things aren’t clear, come talk to me and I’ll do my best to help.
                When I taught the course in 2012, the official text was Lay 4th edition. In these notes, when I refer to “the
                text”, this is what I’m talking about. You can buy it if you’d like, or borrow it from someone, or go without.
                I started writing these notes because I didn’t think it was a good textbook for an honors course. I’m not
                going to use it or expect you to have it.
                Unlike most textbooks, you’ll notice that the exercises are scattered through the notes rather than at the
                end of a chapter. Generally the exercise is put somewhere to illustrate a point that is being made, or because
                the method to solve it has just been introduced. If you’re having trouble with an exercise, the first step is
                to look at what appears just before it.
                                                                  st
                                                       2–Tuesday 21 May, 2013, 15:07
                   Contents
                   1 Introduction                                                                                                        7
                       1.1   Important Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        7
                       1.2   Introductory comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .         7
                             1.2.1   Comments on proof technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .         8
                             1.2.2   Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .    9
                       1.3   Linearity (sec 1.8 of text) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .    10
                       1.4   Matrices and Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .     13
                             1.4.1   Vectors (sec 1.3 of text — up to pg 30)       . . . . . . . . . . . . . . . . . . . . . . . . . .  13
                             1.4.2   Matrices (sec 1.9 of text)    . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  20
                             1.4.3   Matrix Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .      21
                             1.4.4   Matrix Transposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .     28
                       1.5   Linearity and Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .     30
                       1.6   Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   31
                             1.6.1   Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .      33
                             1.6.2   Iterative time stepping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .    34
                       1.7   Matlab/Octave/Python comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .            37
                       1.8   Ashort review of linearity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .     38
                   2 Vector Spaces                                                                                                     39
                       2.1   Important concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .     39
                       2.2   Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .    39
                       2.3   Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .    43
                             2.3.1   Special Subspaces     . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  44
                       2.4   Span . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   45
                       2.5   Bases (sections 2.8-2.9 and 4.3-4.4 of text) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .     48
                       2.6   Showing something is a vector space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        53
                                                                              3
                   3 Systems of Linear Equations                                                                                       57
                       3.1   Important concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .     57
                       3.2   Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   57
                       3.3   Gaussian Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .     59
                             3.3.1   Pivot positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .    65
                       3.4   Column space and Null Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        69
                             3.4.1   Column Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .       69
                             3.4.2   Null Space    . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  72
                       3.5   Onto and one-to-one functions       . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  73
                             3.5.1   The “Onto” game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        76
                             3.5.2   The one-to-one game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .      79
                       3.6   Column Space and Null Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        82
                             3.6.1   Column Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .       82
                   4 Invertible Matrices                                                                                               83
                       4.1   Important Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .       83
                       4.2   Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   83
                       4.3   Finding inverses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   84
                             4.3.1   Matrix calculations with invertible matrices . . . . . . . . . . . . . . . . . . . . . . . .       89
                       4.4   The Invertible Matrix Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .        90
                       4.5   Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .     93
                   5 Inner Products and Orthogonality                                                                                 101
                       5.1   Important Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
                       5.2   Inner Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
                             5.2.1   Vector Inner Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
                             5.2.2   General Inner Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
                       5.3   Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
                             5.3.1   Change of Basis     . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
                             5.3.2   Orthogonal basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
                             5.3.3   Magnitude of a vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
                             5.3.4   Approximation/Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
                             5.3.5   Creating an orthogonal basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
                   6 Eigenvalues and Eigenvectors                                                                                     117
                       6.1   Important Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
                       6.2   Other resource . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
                       6.3   Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
                                                                            st
                                                               4–Tuesday 21   May, 2013, 15:07