Back

Trace of Matrix

If (X) is a square matrix, the sum of its diagonal elements is called the trace.

⚠️**:** Trace is defined only for square matrices.

X=[161250340](3×3)\mathbf{X} = \begin{bmatrix} 1 & 6 & 1 \\ 2 & 5 & 0 \\ 3 & 4 & 0 \end{bmatrix}_{(3\times3)} Diag⁡(X)=[150]\operatorname{Diag}(\mathbf{X})= \begin{bmatrix} 1\\ 5\\ 0 \end{bmatrix} Tr⁡(X)=1+5+0=6\operatorname{Tr}(\mathbf{X}) = 1+5+0 = 6

Matrix Inner Product

If (X,Y) are two matrices of the same size, then

⚠️ Correction: must be same dimensions.

⟨X,Y⟩=Tr⁡(XYT)\langle X,Y\rangle = \operatorname{Tr}(X Y^T)

Also,

⟨Y,X⟩=Tr⁡(YXT)\langle Y,X\rangle = \operatorname{Tr}(Y X^T)

And

✅ Important property added

⟨X,Y⟩=⟨Y,X⟩\langle X,Y\rangle = \langle Y,X\rangle

Example

X=[102214331]Y=[02111032−1]X= \begin{bmatrix} 1&0&2\\ 2&1&4\\ 3&3&1 \end{bmatrix} \qquad Y= \begin{bmatrix} 0&2&1\\ 1&1&0\\ 3&2&-1 \end{bmatrix} YT=[01321210−1]Y^T= \begin{bmatrix} 0&1&3\\ 2&1&2\\ 1&0&-1 \end{bmatrix} XYT=[211 634 7614]XY^T = \begin{bmatrix} 2&1&1\ 6&3&4\ 7&6&14 \end{bmatrix}

Tr⁡(XYT)=2+3+14=19\operatorname{Tr}(XY^T)=2+3+14=19

Transpose Properties

Property 1

(XT)T=X(X^T)^T = X

Example

X=[142536]X= \begin{bmatrix} 1&4\\ 2&5\\ 3&6 \end{bmatrix} XT=[123456]X^T= \begin{bmatrix} 1&2&3\\ 4&5&6 \end{bmatrix}

(XT)T=X(X^T)^T = X

Property 2

⚠️ Correction (important):

(XY)T=YTXT(XY)^T = Y^T X^T

NOT

XTYTX^T Y^T

Example

X=[1213]Y=[3501]X= \begin{bmatrix} 1&2\\ 1&3 \end{bmatrix} \qquad Y= \begin{bmatrix} 3&5\\ 0&1 \end{bmatrix}

XY=

[3738]\begin{bmatrix} 3&7\\ 3&8 \end{bmatrix} (XY)T=[3378](XY)^T= \begin{bmatrix} 3&3\\ 7&8 \end{bmatrix}

Now compute

YTXTY^T X^T

YT=[3051]XT=[1123]Y^T= \begin{bmatrix} 3&0\\ 5&1 \end{bmatrix} \qquad X^T= \begin{bmatrix} 1&1\\ 2&3 \end{bmatrix} YTXT=[3378]Y^T X^T= \begin{bmatrix} 3&3\\ 7&8 \end{bmatrix}

Matches ✔️

Example 2 (Inner Product)

X=[1021]Y=[1120]X= \begin{bmatrix} 1&0\\ 2&1 \end{bmatrix} \qquad Y= \begin{bmatrix} 1&1\\ 2&0 \end{bmatrix} YT=[1210]Y^T= \begin{bmatrix} 1&2\\ 1&0 \end{bmatrix} ⟨X,Y⟩=Tr⁡(XYT)\langle X,Y\rangle = \operatorname{Tr}(XY^T) XYT=[1234]XY^T= \begin{bmatrix} 1&2\\ 3&4 \end{bmatrix} =1+4=5=1+4=5

Also

⟨Y,X⟩=Tr⁡(YXT)=5\langle Y,X\rangle =\operatorname{Tr}(YX^T)=5

Classwork

1. (YTx)(Y^T x)

YT=[12 33 01](3×2)Y^T= \begin{bmatrix} 1&2\ 3&3\ 0&1 \end{bmatrix}{(3\times2)}\qquad x=[12](2×1)x=\begin{bmatrix}1\\2\end{bmatrix}{(2\times1)} YTx=[5 9 2]Y^T x = \begin{bmatrix} 5\ 9\ 2 \end{bmatrix}

2. Hadamard Product

Z⊙Z=[100021111]⊙[100021111]Z\odot Z= \begin{bmatrix} 1&0&0\\ 0&2&1\\ 1&1&1 \end{bmatrix} \odot \begin{bmatrix} 1&0&0\\ 0&2&1\\ 1&1&1 \end{bmatrix} [100041111]\begin{bmatrix} 1&0&0\\ 0&4&1\\ 1&1&1 \end{bmatrix}

3. (2Zy)(2Zy)

2[100021111][313]2\begin{bmatrix}1&0&0\\ 0&2&1\\ 1&1&1 \end{bmatrix}\begin{bmatrix}3\\ 1\\ 3\end{bmatrix} [61014]\begin{bmatrix} 6\\ 10\\ 14 \end{bmatrix}

4. (<x,x>)(<x,x>)

⟨x,x⟩=Tr⁡(xxT)\langle x,x\rangle=\operatorname{Tr}(xx^T)

Tr⁡[1001]=2\operatorname{Tr} \begin{bmatrix} 1&0\\ 0&1 \end{bmatrix} =2

5. Outer Product

x⊗y=xyTx\otimes y = xy^T

[12][313]=[313626]\begin{bmatrix}1\\ 2\end{bmatrix}\begin{bmatrix}3 & 1 &3\end{bmatrix} = \begin{bmatrix} 3&1&3\\ 6&2&6 \end{bmatrix}

6. $Diag(x

)$

Diag⁡([1001])\operatorname{Diag}(\begin{bmatrix}1&0\\ 0&1\end{bmatrix}) =[11]= \begin{bmatrix} 1\\ 1 \end{bmatrix}

7. Trace

Tr⁡(Z)=1+2+1=4\operatorname{Tr}(Z)=1+2+1=4

8. (xTYz)(x^T Y z)

xTYzx^T Y z

[12]\begin{bmatrix} 1&2 \end{bmatrix} [130231]\begin{bmatrix} 1&3&0\\ 2&3&1 \end{bmatrix} [130]\begin{bmatrix} 1\\ 3\\ 0 \end{bmatrix} [592]\begin{bmatrix} 5&9&2 \end{bmatrix} [130]=32\begin{bmatrix} 1\\ 3\\ 0 \end{bmatrix} =32

Vector Outer Product

v⊗v=vvTv\otimes v = vv^T

[xy][xy]=[x2xyxyy2]\begin{bmatrix}x \\ y\end{bmatrix}\begin{bmatrix}x&y\end{bmatrix} = \begin{bmatrix} x^2&xy\\ xy&y^2 \end{bmatrix}

Trace of Outer Product

⚠️**:** result must be scalar

Tr⁡(v⊗v)=x2+y2\operatorname{Tr}(v\otimes v) = x^2+y^2

Linear Transformation

AX=BAX=B

[1234][56]=[1739]\begin{bmatrix}1&2 \\ 3&4\end{bmatrix}\begin{bmatrix}5\\ 6\end{bmatrix} = \begin{bmatrix} 17\\ 39 \end{bmatrix}

This represents a linear transformation

R2→R2\mathbb{R}^2 \to \mathbb{R}^2

System:

x+2y=17x+2y=17

3x+4y=393x+4y=39

Solution

y=6x=5y=6 \qquad x=5

Row Operation Rules

You can:

  1. Multiply equation by constant
  2. Swap equations
  3. Add/subtract equations

These are elementary row operations.