İki kovaryans matrisi arasındaki benzerlik veya mesafenin ölçülmesi

28

İki simetrik kovaryans matrisi arasında benzerlik veya mesafe ölçütleri var mı (ikisi de aynı boyutlara sahip)?

Burada, iki olasılık dağılımının KL farklılığına veya matrislere uygulanmadıkları sürece vektörler arasındaki Öklid mesafesine analogları düşünüyorum. Bir kaç benzerlik ölçümü olacağını hayal ediyorum.

İdeal olarak, iki kovaryans matrisinin aynı olduğu gibi boş hipotezi test etmek istiyorum.

— Ram Ahluwalia
kaynak

3

the answers to this question: quant.stackexchange.com/q/121/108 may be of some use.

— shabbychef

2

excellent question and answer on the link - thanks - yes this is where I was going :)

— Ram Ahluwalia

2

Related question: Diagnostic plot for assessing homogeneity of variance-covariance matrices. Related paper: A simple procedure for the comparison of covariance matrices.

— amoeba says Reinstate Monica

21

You can use any of the norms $\| A-B \|_p$ (see Wikipedia on a variety of norms; note that the square-root of the sum of squared distances, $\sqrt{\sum_{i,j} (a_{ij}-b_{ij})^2}$ , is called Frobenius norm, and is different from $L_2$ norm, which is the square root of the largest eigenvalue of $(A-B)^2$ , although of course they would generate the same topology). The K-L distance between the two normal distributions with the same means (say zero) and the two specific covariance matrices is also available in Wikipedia as $\frac12 [ \mbox{tr} (A^{-1}B) - \mbox{ln}( |B|/|A| ) ]$ .

Edit: if one of the matrices is a model-implied matrix, and the other is the sample covariance matrix, then of course you can form a likelihood ratio test between the two. My personal favorite collection of such tests for simple structures is given in Rencher (2002) Methods of Multivariate Analysis. More advanced cases are covered in covariance structure modeling, on which a reasonable starting point is Bollen (1989) Structural Equations with Latent Variables.

— StasK
kaynak

i have a problem with

1 / 2 (tr (A^{- 1} B) - \log (| B | / | A |))

$1/2(\verb+tr+(A^{-1}B)-\log(|B|/|A|))$ : it doesn't give the same value if you permute

A

$A$ and

B

$B$ ( a real distance should be symmetric).

— user603

i have a problem with

(A - B)^{2}

$(A-B)^2$ : it is not affine equivariant (if you rotate the matrices, there distance changes!). Furthermore, you should somehow scale your matrices (they might be measured in very different units), also, it is only natural to require that the distance between two covariance matrices be the same as the distance between the corresponding correlation matrices: so I suggest

(A det (A)^{- 1 / p} - B det (B)^{- 1 / p})^{2}

$(A\det(A)^{-1/p}-B\det(B)^{-1/p})^2$ .

— user603

2

First, K-L is not a real distance, and that's a well known fact. Second, if the matrices are measured in different units, they cannot be equal.

— StasK

Is K-L distance similar to likelihood ratio, or are they related?

— hashmuke

7

Denote $\varSigma_1$ and $\varSigma_2$ your matrices both of dimension $p$ .

Cond number: $\log(\lambda_1)-\log(\lambda_p)$ where $\lambda_1$ ( $\lambda_p$ ) is the largest (smallest) eigenvalue of $\varSigma^*$ , where $\varSigma^*$ is defined as: $\varSigma^*:=\varSigma_1^{-1/2}\varSigma_2\varSigma_1^{-1/2}$

Edit: I edited out the second of the two proposals. I think I had misunderstood the question. The proposal based on condition numbers is used in robust statistics a lot to assess quality of fit. An old source I could find for it is:

Yohai, V.J. and Maronna, R.A. (1990). The Maximum Bias of Robust Covariances. Communications in Statistics–Theory and Methods, 19, 3925–2933.

I had originally included the Det ratio measure:

Det ratio: $\log(\det(\varSigma^{**})/\sqrt{\det(\varSigma_2)*\det(\varSigma_1)})$ where $\varSigma^{**}=(\varSigma_1+\varSigma_2)/2$ .

which would be the Bhattacharyya distance between two Gaussian distributions having the same location vector. I must have originally read the question as pertaining to a setting where the two covariances were coming from samples from populations assumed to have equal means.

— user603
kaynak

7

A measure introduced by Herdin (2005) Correlation Matrix Distance, a Meaningful Measure for Evaluation of Non-Stationary MIMO Channels is

d = 1 - \frac{tr (R_{1} \cdot R_{2})}{‖ R_{1} ‖ \cdot ‖ R_{2} ‖},

$d = 1 - \frac{\text{tr}(R_1 \cdot R_2)}{\|R_1\| \cdot \|R_2\|},$ where the norm is the Frobenius norm.

— davidc
kaynak

+1. Thanks a lot for this answer, it was very helpful to me.

— amoeba says Reinstate Monica

1

This is one minus cosine similarity, right?

— Firebug

4

The covariance matrix distance is used for tracking objects in Computer Vision.

The currently used metric is described in the article: "A metric for covariance matrices", by Förstner and Moonen.

— Andres Romero
kaynak