Solved

Projection of High Dimensional Data onto a subspace.

Posted on 2013-01-21
11
537 Views
Last Modified: 2016-03-02
Hi Experts,
Does anyone have a nice and easy tutorials on Projecting High dimensional Gaussian Data seating somewhere in space to a subspace? I would also like to see a Matlab implementation of this so that i can see how the Gaussian will look like after the projection.

Regards.
0
Comment
Question by:wish_C
  • 6
  • 4
11 Comments
 
LVL 15

Expert Comment

by:yuk99
ID: 38805670
I believe what you need is Principal Component Analysis (PCA).

It's implemented in MATLAB in Statistical Toolbox as PRINCOMP function and PCA function. Check the documentation (follow the links ) for code examples. Many examples you can find with google.

You can also try a good PCA example on FileExchange:
http://www.mathworks.com/matlabcentral/fileexchange/16487-pca-demonstration-example
0
 

Author Comment

by:wish_C
ID: 38805765
Thanks Yuk99 for the quick response. I am trying to open this link you sent me: http://www.mathworks.com/matlabcentral/fileexchange/16487-pca-demonstration-example

but the link does not exist anymore. Any other sample code that i can read to understand how it is coded?

Cheers.
0
 

Author Comment

by:wish_C
ID: 38805999
Thank modus_operandi for increasing the zones of my post.

Cheers.
0
Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

 
LVL 15

Expert Comment

by:yuk99
ID: 38806500
I can open the link to FileExchange right now.
Open the documentation pages for PRINCOMP and PCA. You will find links to examples and demos there.
0
 

Author Comment

by:wish_C
ID: 38809687
The example you sent is confusing to me. I just need a simple one that follow this algorithm.

Given n observations x_1, x_2, . . . , x_n of m-dimensional  column vectors
1. Compute  the mean vector µ = (x_1+x_2+. . . +x_n )/n
2. Compute the covariance matrix by MLE
     C = (1/n) Si=1n (x_i - µ)(x_i - µ)^t      - Si=1n(sigma from i=1 to n)
3. Compute the eigenvalue/eigenvector pairs (lamda_i, u_i) of C
     with lamda_1 = lamda_2 = . . . = lamda_m   ( =0 )
4. Compute the first d principal components y_i^(j)  = x_i^t u_j, for each observation x_i, 1 = i = n,  along the direction u_j , j = 1, 2, · · · , d.

Thanks.
0
 
LVL 15

Expert Comment

by:yuk99
ID: 38810957
Ah, you want to understand algorithm on the low level. The functions I gave you do this inside.

Let me help you to start. And please remember we cannot do your homework for you.

0. X = rand(n,m);

1. M = mean(X);

2. C = cov(X);
You can also check MLECOV function, if it's more suitable for you.
If you want the details how the COV function works, type "edit cov" in matlab and go through the code, it's not so difficult to understand.

3. [U L] = eig(C);

I hope you can continue to finish the last part. If you cannot, let me know what is exactly the problem, or what exactly you don't understand.
0
 

Author Comment

by:wish_C
ID: 38811902
Here is the code i came up with for the PCA implementation.
K=1;
X = rand(5,2);
M = mean(X);
C = cov(X);
[U D]=eig(C);
L=diag(D);
[sorted index]=sort(L,'descend');
Xproj=zeros(d,K);   

for j=1:K
   Xproj(:,j)=U(:,index(j));
end

Y=X*Xproj; 
plot(Y1,'d');  
axis([4 24 -2 18]);

Open in new window


I basically generated 5 X 2 matrix and apply PCA on it. I want to project the data on to a subspace (one dimensional subspace) and plot the reduced dimensions on a graph to see the behavior of the data. I also want a plot of a line that shows the direction of the transformed data.

Is my code OK? How can i see the plot of the data on the subspace with a line showing its direction?
0
 
LVL 15

Accepted Solution

by:
yuk99 earned 500 total points
ID: 38812240
I only briefly looked at your code. Here are few issues.
Looks like you don't need line 2 to calculate mean. Also d is not defines, but you can get it as d = size(U,1);
You can avoid the for-loop with Xproj = U(:,index(1:K)); replacing lines 8-12.

I haven't check if the last transformation is correct.
Y1 probably should be Y or you missed a line.
To plot 1D data it's better to use, for example, plot(Y,zeros(size(Y),'d'). In your case the x axis values just represent how the points ordered in Y.

I found a very good PCA tutorial (starting from very basics):
http://www.ce.yildiz.edu.tr/personal/songul/file/1097/principal_components.pdf
Hope it's exactly what you need.
0
 

Author Comment

by:wish_C
ID: 38815123
I have actually got this tutorials in the link. This is what i am even trying to follow by using the information in the tutorials to follow the steps and implement it in matlab. Here is what i have:
% Find the first K Principal Components of data X (n rows, d columns)
% X contains n pattern vectors with d features
X= [2.5,0.5,2.2,1.9,3.1,2.3,2.0,1.0,1.5,1.1];
Y= [2.4,0.7,2.9,2.2,3.0,2.7,1.6,1.1,1.6,0.9];
Data = [X;Y]';
mx = mean(X);
my = mean(Y);
Xadj = (X-mean(X));
Yadj = (Y-mean(Y));
DataAdj= [Xadj;Yadj];
Cadj = cov(Xadj,Yadj);
C = cov(X,Y);
[Uadj Dadj]=eig(Cadj);
[U D]=eig(C);
L=diag(D);
[sorted index]=sort(L,'descend');
[FVector index]=sort(U,'descend');

%Final data using both eigenvectors in U
ColAdjData = DataAdj';
FData = FVector * ColAdjData';
OriginalAdjustedData = FVector' * FData;

figure;
plot(X,Y, 'd');
axis equal;
axis([-2 5 -2 5])

figure;
plot(Xadj, Yadj, 'x');
axis([-2 2 -2 2])

figure;
plot(D,'*')
axis([-2 2 -2 2])

figure;
plot(Xadj, Yadj, 'x');
hold on
plot(D,'*')
axis([-2 2 -2 2])


figure;
plot(FData,'d')
axis([-2 2 -2 2])

figure;
plot(OriginalAdjustedData,'r*')
axis([-2 2 -2 2])

Open in new window


How every i have this attached plot which is difference from expected as in the Tutorials.

Even though i have the adjusted data back, but i don't know how i can add the means back to the X and Y to get the original data back.

I have use both eigenvectors in the above code to get the feature and as a results have a 2 dimensions, however i would like to know if possible what modification can be done in my code to reduce the the dimensions from 2 to 1 by selecting taking only the eigenvector with the largest eigenvalue to transform the data from 2d to 1d. Any idea on this please?
Thanks
Plot.bmp
0
 

Author Comment

by:wish_C
ID: 38823223
Any Idea on this issue on the above code please?
0

Featured Post

Free Tool: Postgres Monitoring System

A PHP and Perl based system to collect and display usage statistics from PostgreSQL databases.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Displaying an arrayList in a listView using the default adapter is rarely the best solution. To get full control of your display data, and to be able to refresh it after editing, requires the use of a custom adapter.
A short article about problems I had with the new location API and permissions in Marshmallow
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…
Finds all prime numbers in a range requested and places them in a public primes() array. I've demostrated a template size of 30 (2 * 3 * 5) but larger templates can be built such 210  (2 * 3 * 5 * 7) or 2310  (2 * 3 * 5 * 7 * 11). The larger templa…

830 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question