Friday 14 October 2016

matlab - ICA - Statistical Independence & Eigenvalues of Covariance Matrix


I am currently creating different signals using Matlab, mixing them by multiplying them by a mixing matrix A, and then trying to get back the original signals using FastICA.


So far, the recovered signals are really bad when compared to the original ones, which was not what I expected.


I'm trying to see whether I'm doing anything wrong. The signals I'm generating are the following:


s1 = (-x.^2 + 100*x + 500) / 3000; % quadratic
s2 = exp(-x / 10); % -ve exponential
s3 = (sin(x)+ 1) * 0.5; % sine
s4 = 0.5 + 0.1 * randn(size(x, 2), 1); % gaussian
s5 = (sawtooth(x, 0.75)+ 1) * 0.5; % sawtooth


Original Signals


One condition for ICA to be successful is that at most one signal is Gaussian, and I've observed this in my signal generation.


However, another condition is that all signals are statistically independent.


All I know is that this means that, given two signals A & B, knowing one signal does not give any information with regards to the other, i.e.: P(A|B) = P(A) where P is the probability.


Now my question is this: Are my signals statistically independent? Is there any way I can determine this? Perhaps some property that must be observed?


Another thing I've noticed is that when I calculate the eigenvalues of the covariance matrix (calculated for the matrix containing the mixed signals), the eigenspectrum seems to show that there is only one (main) principal component. What does this really mean? Shouldn't there be 5, since I have 5 (supposedly) independent signals?


For example, when using the following mixing matrix:


A =

0.2000 0.4267 0.2133 0.1067 0.0533

0.2909 0.2000 0.2909 0.1455 0.0727
0.1333 0.2667 0.2000 0.2667 0.1333
0.0727 0.1455 0.2909 0.2000 0.2909
0.0533 0.1067 0.2133 0.4267 0.2000

The eigenvalues are: 0.0000 0.0005 0.0022 0.0042 0.0345 (only 4!)


When using the identity matrix as the mixing matrix (i.e. the mixed signals are the same as the original ones), the eigenspectrum is: 0.0103 0.0199 0.0330 0.0811 0.1762. There still is one value much larger than the rest..


Thank you for your help.


I apologise if the answers to my questions are painfully obvious, but I'm really new to statistics, ICA and Matlab. Thanks again.


EDIT



I have 500 samples of each signal, in the range [0.2, 100], in steps of 0.2, i.e. x = 0:0.1:100.


Also, given the ICA Model: X = As + n (I'm not adding any noise at the moment), I am referring to the eigenspectrum of the transpose of X, i.e. eig(cov(X')).


UPDATE


As suggested (refer to comments), I tried FastICA on only 2 signals. The results were quite good (see pic below). The mixing matrix used was A = [0.75 0.25; 0.25 0.75]. However, the eigenspectrum 0.1657 0.7732 still showed only one main principal component.


My question therefore boils down to the following: What function/equation/property can I use to check whether a number of signal vectors are statistically independent?


Sine & Gaussian - FastICA




No comments:

Post a Comment

readings - Appending 内 to a company name is read ない or うち?

For example, if I say マイクロソフト内のパートナーシップは強いです, is the 内 here read as うち or ない? Answer 「内」 in the form: 「Proper Noun + 内」 is always read 「ない...