EigenDecomp

EigenDecomp( A, I, J )

Computes the Eigenvalues and Eigenvectors of a square symmetric matrix A indexed by I and J. Eigenvalues and Eigenvectors are also called characteristic values and characteristic vectors.

In matrix notation, when x is a row vector, then x is an Eigenvector and r the associated Eigenvalue when

A x = r x

or equivalently, when

(A - r I) x = 0

A must be symmetric to use EigenDecomp. When A is not symmetric, some or all of the Eigenvectors and Eigenvalues may be complex numbers.

The pairs of Eigenvalues and Eigenvectors returned are indexed by J. If B is the result of evaluating EigenDecomp(A,I,J), then the Eigenvalues are given by

B[.item='value']

and the Eigenvectors are given by

#B[.item='vector']

Each Eigenvector is indexed by I.

Library

Matrix

Testing for Definiteness

A square matrix A is positive definite if for for every non-zero vector x, the matrix product x'Ax > 0. The matrix A is positive definite if and only if all eigenvalues are positive. Also, a symmetric or non-symmetric square matrix A is positive definite if and only if the symmetric matrix (A+A') is positive definite. Thus the following expression tests for positive-definiteness:

 Function IsPosDefinite(A:Array[I,J] ; I,J : Index) :=
   Min(EigenDecomp(A+Transpose(A,I,J),I,J)[.item='value']>0,J)

Square matrix A is positive semi-definite when x'Ax >= 0 for every vector x, which implies that the eigenvectors are all non-negative.

 Function IsPosSemiDefinite(A:Array[I,J] ; I,J : Index) :=
   Min(EigenDecomp(A+Transpose(A,I,J),I,J)[.item='value']>=0,J)
 

Negative definite and Negative-semidefinite are tested for similarly:

 Function IsNegDefinite(A:Array[I,J] ; I,J : Index) :=
   Min(EigenDecomp(A+Transpose(A,I,J),I,J)[.item='value']<0,J)

 Function IsNegSemiDefinite(A:Array[I,J] ; I,J : Index) :=
   Min(EigenDecomp(A+Transpose(A,I,J),I,J)[.item='value']<=0,J)

Principle Components

Principle components analysis is a data analysis technique that finds the rotation of your data so that the maximum variance is oriented along the first axis, the second largest component of variance is along the second axis, etc. In the rotated data, there is zero covariance along each dimension. The principle components and the accompanying variances provide a convenient description of the statistical properties of a data set. In large very high-dimensional data sets, principle component analysis can provide a means to reduce the data to a small number of dimensions by ignoring the principle components with small variance.

EigenDecomp computes the principle components from a covariance matrix. Suppose you have a dataset X[I,R], with components indexed by I, and each data point indexed by R. (Often in Analytica, R will be the Run index). To represent the Covariance matrix, another index, I2, that is a copy of I is required. I2 can be defined as CopyIndex(I). Then the Covariance matrix of the data is obtained using:

Covariance( X,X[I=J],R )

Call this matrix CV_X. From the covariance matrix, the variances of the principle components is obtained using:

EigenDecomp(CV_X,I,J)[.item='value',J=I]

and the rotation matrix is given by

#EigenDecomp(CV_X,I,J)[.item='vector']

Define B to be this rotation matrix. The original data can be "rotated" to its principle components, and recentered at the origin, using:

MatrixMultiply( B, J, I, X-Mean(X,M), I, R )[J=I]

If we call this rotated data Xr, the the covariance of Xr, call it CV_Xr, computed as:

Covariance(Xr,Xr[I=J],R)

is a diagonal matrix (i.e., zeroes everywhere except the diagonal), where the trace of the matrix, computed as

CV_Xr[J=I]

is equal to the principle component variances returned from

EigenDecomp(CV,I,J)[.item='value',J=I]

Example

Here is a scatter plot of some data, X, with indexes I and R:

Covariance(X,X[I=J],R) →

The Eigenvalues of the covariance matrix, EigenDecomp(CV_X,I,J)[.item='value'], are 1613 and 188.7. You can compare these to the original variances along each axis seen in the diagonal of the covariance matrix. The largest eigenvalue has captured most of the variance.

The rotation matrix, #EigenDecomp(CV_X,I,J)[.item='vector'], denoted B, is

Using this to rotation the original data, using MatrixMultiply(B,J,I,X-mean(X,R),I,R)[J=I], we have

The covariance of this data, Covariance(Xr,Xr[I=J],I), is

See Also

• SingularValueDecomp
• Decompose
• Invert