3 değişken için Pearson korelasyonunun analojisi

Üç değişkenin bir “korelasyonunun” bir şey olup olmadığıyla ilgileniyorum ve ne olursa olsun, bu ne olurdu?

Pearson ürün moment korelasyon katsayısı

Şimdi 3 değişken için soru:

herhangi bir şey?

R'de yorumlanabilir bir şey gibi görünüyor:

> a <- rnorm(100); b <- rnorm(100); c <- rnorm(100)
> mean((a-mean(a)) * (b-mean(b)) * (c-mean(c))) / (sd(a) * sd(b) * sd(c))
[1] -0.3476942

Normalde sabit bir üçüncü değişkenin değeri verilen 2 değişken arasındaki korelasyona bakarız. Birisi açıklığa kavuşturabilir mi?

O ise gerçekten bir şey. Öğrenmek için korelasyonun kendisi hakkında bildiklerimizi incelememiz gerekir.

1. Bir vektör değerli rastgele değişkenin ilişki matrisi $\mathbf{X}=(X_1,X_2,\ldots,X_p)$ arasında standart versiyonun bir varyans-kovaryans matrisi veya sadece "varyans," $\mathbf{X}$ . Diğer bir deyişle, her $X_i$ yerine yeniden girilmiş, yeniden ölçeklendirilmiş versiyonu gelir.

2. Kovaryans $X_i$ ve $X_j$ kendi merkezli alternatifler ürünü beklentisi olan. Yani, $X^\prime_i = X_i - E[X_i]$ ve X ′ j = X j - E [ X j ] yazdığımızda $X^\prime_j = X_j - E[X_j]$,

   $$\operatorname{Cov}(X_i,X_j) = E[X^\prime_i X^\prime_j].$$

3. Var ( X ) yazacağım X'in varyansı tek bir sayı değil. Var ( X ) i j = Cov ( X i , X j değer dizisidir. $\mathbf{X}$$\operatorname{Var}(\mathbf{X})$

   $$\operatorname{Var}(\mathbf{X})_{ij}=\operatorname{Cov}(X_i,X_j).$$

4. Amaçlanan genelleme için kovaryansı düşünmenin yolu, bunu bir tensör olarak düşünmektir . Bu , X ve doğrusal bir dönüşüm geçirdiğinde değerleri özellikle basit bir öngörülebilir şekilde değişen, 1 ile p arasında değişen ve i ve j ile endekslenen v i j miktarlarının bir koleksiyonudur . Spesifik olarak, izin Y = ( Y 1 , Y, 2 , ... , Y, q ) ile tanımlanan bir vektör değerli rastgele değişken$v_{ij}$$i$$j$$1$$p$$\mathbf{X}$$\mathbf{Y}=(Y_1,Y_2,\ldots,Y_q)$

   $$Y_i = \sum_{j=1}^p a_i^{\,j}X_j.$$

   The constants $a_i^{\,j}$ ($i$ and $j$ are indexes--$j$ is not a power) form a $q\times p$ array $\mathbb{A} = (a_i^{\,j})$, $j=1,\ldots, p$ and $i=1,\ldots, q$. The linearity of expectation implies

   $$\operatorname{Var}(\mathbf Y)_{ij} = \sum a_i^{\,k}a_j^{\,l}\operatorname{Var}(\mathbf X)_{kl} .$$

   In matrix notation,

   $$\operatorname{Var}(\mathbf Y) = \mathbb{A}\operatorname{Var}(\mathbf X) \mathbb{A}^\prime .$$

5. All the components of $\operatorname{Var}(\mathbf{X})$ actually are univariate variances, due to the Polarization Identity

   $$4\operatorname{Cov}(X_i,X_j) = \operatorname{Var}(X_i+X_j) - \operatorname{Var}(X_i-X_j).$$

   This tells us that if you understand variances of univariate random variables, you already understand covariances of bivariate variables: they are "just" linear combinations of variances.

The expression in the question is perfectly analogous: the variables $X_i$ have been standardized as in $(1)$. We can understand what it represents by considering what it means for any variable, standardized or not. We would replaced each $X_i$ by its centered version, as in $(2)$, and form quantities having three indexes,

These are the central (multivariate) moments of degree $3$. As in $(4)$, they form a tensor: when $\mathbf{Y} = \mathbb{A}\mathbf{X}$, then

The indexes in this triple sum range over all combinations of integers from $1$ through $p$.

The analog of the Polarization Identity is

On the right hand side, $\mu_3$ refers to the (univariate) central third moment: the expected value of the cube of the centered variable. When the variables are standardized, this moment is usually called the skewness. Accordingly, we may think of $\mu_3(\mathbf{X})$ as being the multivariate skewness of $\mathbf{X}$. It is a tensor of rank three (that is, with three indices) whose values are linear combinations of the skewnesses of various sums and differences of the $X_i$. If we were to seek interpretations, then, we would think of these components as measuring in $p$ dimensions whatever the skewness is measuring in one dimension. In many cases,

• The first moments measure the location of a distribution;

• The second moments (the variance-covariance matrix) measure its spread;

• The standardized second moments (the correlations) indicate how the spread varies in $p$-dimensional space; and

• The standardized third and fourth moments are taken to measure the shape of a distribution relative to its spread.

To elaborate on what a multidimensional "shape" might mean, observed that we can understand PCA as a mechanism to reduce any multivariate distribution to a standard version located at the origin and equal spreads in all directions. After PCA is performed, then, $\mu_3$ would provide the simplest indicators of the multidimensional shape of the distribution. These ideas apply equally well to data as to random variables, because data can always be analyzed in terms of their empirical distribution.

Reference

Alan Stuart & J. Keith Ord, Kendall's Advanced Theory of Statistics Fifth Edition, Volume 1: Distribution Theory; Chapter 3, Moments and Cumulants. Oxford University Press (1987).

Appendix: Proof of the Polarization Identity

Let $x_1,\ldots, x_n$ be algebraic variables. There are $2^n$ ways to add and subtract all $n$ of them. When we raise each of these sums-and-differences to the $n^\text{th}$ power, pick a suitable sign for each of those results, and add them up, we will get a multiple of $x_1x_2\cdots x_n$.

More formally, let $S=\{1,-1\}^n$ be the set of all $n$-tuples of $\pm 1$, so that any element $s\in S$ is a vector $s=(s_1,s_2,\ldots,s_n)$ whose coefficients are all $\pm 1$. The claim is

Indeed, the Multinomial Theorem states that the coefficient of the monomial $x_1^{i_1}x_2^{i_2}\cdots x_n^{i_n}$ (where the $i_j$ are nonnegative integers summing to $n$) in the expansion of any term on the right hand side is

In the sum $(1)$, the coefficients involving $x_1^{i_1}$ appear in pairs where one of each pair involves the case $s_1=1$, with coefficient proportional to $\color{red}{s_1}$ times $s_1^{i_1}$, equal to $1$, and the other of each pair involves the case $s_1=-1$, with coefficient proportional to $\color{red}{-1}$ times $(-1)^{i_1}$, equal to $(-1)^{i_1+1}$. They cancel in the sum whenever $i_1+1$ is odd. The same argument applies to $i_2, \ldots, i_n$. Consequently, the only monomials that occur with nonzero coefficients must have odd powers of all the $x_i$. The only such monomial is $x_1x_2\cdots x_n$. It appears with coefficient $\binom{n}{1,1,\ldots,1}=n!$ in all $2^n$ terms of the sum. Consequently its coefficient is $2^nn!$, QED.

We need take only half of each pair associated with $x_1$: that is, we can restrict the right hand side of $(1)$ to the terms with $s_1=1$ and halve the coefficient on the left hand side to $2^{n-1}n!$ . That gives precisely the two versions of the Polarization Identity quoted in this answer for the cases $n=2$ and $n=3$: $2^{2-1}2! = 4$ and $2^{3-1}3!=24$.

Of course the Polarization Identity for algebraic variables immediately implies it for random variables: let each $x_i$ be a random variable $X_i$. Take expectations of both sides. The result follows by linearity of expectation.

Hmmm. If we run...

a <- rnorm(100);
b <- rnorm(100);
c <- rnorm(100)
mean((a-mean(a))*(b-mean(b))*(c-mean(c)))/
  (sd(a) * sd(b) * sd(c))

it does seem to center on 0 (I haven't done a real simulation), but as @ttnphns alludes, running this (all variables the same)

a <- rnorm(100)
mean((a-mean(a))*(a-mean(a))*(a-mean(a)))/
  (sd(a) * sd(a) * sd(a))

also seems to center on 0, which certainly makes me wonder what use this could be.

If You need to calculate "correlation" between three or more variables, you could not use Pearson, as in this case it will be different for different order of variables have a look here. If you are interesting in linear dependency, or how well they are fitted by 3D line, you may use PCA, obtain explained variance for first PC, permute your data and find probability, that this value may be to to random reasons. I've discuss something similar here (see Technical details below).

Matlab code

% Simulate our experimental data
x=normrnd(0,1,100,1);
y=2*x.*normrnd(1,0.1,100,1);
z=(-3*x+1.5*y).*normrnd(1,2,100,1);
% perform pca
[loadings, scores,variance]=pca([x,y,z]);
% Observed Explained Variance for first principal component
OEV1=variance(1)/sum(variance)
% perform permutations
permOEV1=[];
for iPermutation=1:1000
    permX=datasample(x,numel(x),'replace',false);
    permY=datasample(y,numel(y),'replace',false);
    permZ=datasample(z,numel(z),'replace',false);
    [loadings, scores,variance]=pca([permX,permY,permZ]);
    permOEV1(end+1)=variance(1)/sum(variance);
end

% Calculate p-value
p_value=sum(permOEV1>=OEV1)/(numel(permOEV1)+1)