Solved

Projection of High Dimensional Data onto a subspace.

Posted on 2013-01-21
11
516 Views
Last Modified: 2016-03-02
Hi Experts,
Does anyone have a nice and easy tutorials on Projecting High dimensional Gaussian Data seating somewhere in space to a subspace? I would also like to see a Matlab implementation of this so that i can see how the Gaussian will look like after the projection.

Regards.
0
Comment
Question by:wish_C
  • 6
  • 4
11 Comments
 
LVL 15

Expert Comment

by:yuk99
Comment Utility
I believe what you need is Principal Component Analysis (PCA).

It's implemented in MATLAB in Statistical Toolbox as PRINCOMP function and PCA function. Check the documentation (follow the links ) for code examples. Many examples you can find with google.

You can also try a good PCA example on FileExchange:
http://www.mathworks.com/matlabcentral/fileexchange/16487-pca-demonstration-example
0
 

Author Comment

by:wish_C
Comment Utility
Thanks Yuk99 for the quick response. I am trying to open this link you sent me: http://www.mathworks.com/matlabcentral/fileexchange/16487-pca-demonstration-example

but the link does not exist anymore. Any other sample code that i can read to understand how it is coded?

Cheers.
0
 

Author Comment

by:wish_C
Comment Utility
Thank modus_operandi for increasing the zones of my post.

Cheers.
0
 
LVL 15

Expert Comment

by:yuk99
Comment Utility
I can open the link to FileExchange right now.
Open the documentation pages for PRINCOMP and PCA. You will find links to examples and demos there.
0
 

Author Comment

by:wish_C
Comment Utility
The example you sent is confusing to me. I just need a simple one that follow this algorithm.

Given n observations x_1, x_2, . . . , x_n of m-dimensional  column vectors
1. Compute  the mean vector µ = (x_1+x_2+. . . +x_n )/n
2. Compute the covariance matrix by MLE
     C = (1/n) Si=1n (x_i - µ)(x_i - µ)^t      - Si=1n(sigma from i=1 to n)
3. Compute the eigenvalue/eigenvector pairs (lamda_i, u_i) of C
     with lamda_1 = lamda_2 = . . . = lamda_m   ( =0 )
4. Compute the first d principal components y_i^(j)  = x_i^t u_j, for each observation x_i, 1 = i = n,  along the direction u_j , j = 1, 2, · · · , d.

Thanks.
0
Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 15

Expert Comment

by:yuk99
Comment Utility
Ah, you want to understand algorithm on the low level. The functions I gave you do this inside.

Let me help you to start. And please remember we cannot do your homework for you.

0. X = rand(n,m);

1. M = mean(X);

2. C = cov(X);
You can also check MLECOV function, if it's more suitable for you.
If you want the details how the COV function works, type "edit cov" in matlab and go through the code, it's not so difficult to understand.

3. [U L] = eig(C);

I hope you can continue to finish the last part. If you cannot, let me know what is exactly the problem, or what exactly you don't understand.
0
 

Author Comment

by:wish_C
Comment Utility
Here is the code i came up with for the PCA implementation.
K=1;
X = rand(5,2);
M = mean(X);
C = cov(X);
[U D]=eig(C);
L=diag(D);
[sorted index]=sort(L,'descend');
Xproj=zeros(d,K);   

for j=1:K
   Xproj(:,j)=U(:,index(j));
end

Y=X*Xproj; 
plot(Y1,'d');  
axis([4 24 -2 18]);

Open in new window


I basically generated 5 X 2 matrix and apply PCA on it. I want to project the data on to a subspace (one dimensional subspace) and plot the reduced dimensions on a graph to see the behavior of the data. I also want a plot of a line that shows the direction of the transformed data.

Is my code OK? How can i see the plot of the data on the subspace with a line showing its direction?
0
 
LVL 15

Accepted Solution

by:
yuk99 earned 500 total points
Comment Utility
I only briefly looked at your code. Here are few issues.
Looks like you don't need line 2 to calculate mean. Also d is not defines, but you can get it as d = size(U,1);
You can avoid the for-loop with Xproj = U(:,index(1:K)); replacing lines 8-12.

I haven't check if the last transformation is correct.
Y1 probably should be Y or you missed a line.
To plot 1D data it's better to use, for example, plot(Y,zeros(size(Y),'d'). In your case the x axis values just represent how the points ordered in Y.

I found a very good PCA tutorial (starting from very basics):
http://www.ce.yildiz.edu.tr/personal/songul/file/1097/principal_components.pdf
Hope it's exactly what you need.
0
 

Author Comment

by:wish_C
Comment Utility
I have actually got this tutorials in the link. This is what i am even trying to follow by using the information in the tutorials to follow the steps and implement it in matlab. Here is what i have:
% Find the first K Principal Components of data X (n rows, d columns)
% X contains n pattern vectors with d features
X= [2.5,0.5,2.2,1.9,3.1,2.3,2.0,1.0,1.5,1.1];
Y= [2.4,0.7,2.9,2.2,3.0,2.7,1.6,1.1,1.6,0.9];
Data = [X;Y]';
mx = mean(X);
my = mean(Y);
Xadj = (X-mean(X));
Yadj = (Y-mean(Y));
DataAdj= [Xadj;Yadj];
Cadj = cov(Xadj,Yadj);
C = cov(X,Y);
[Uadj Dadj]=eig(Cadj);
[U D]=eig(C);
L=diag(D);
[sorted index]=sort(L,'descend');
[FVector index]=sort(U,'descend');

%Final data using both eigenvectors in U
ColAdjData = DataAdj';
FData = FVector * ColAdjData';
OriginalAdjustedData = FVector' * FData;

figure;
plot(X,Y, 'd');
axis equal;
axis([-2 5 -2 5])

figure;
plot(Xadj, Yadj, 'x');
axis([-2 2 -2 2])

figure;
plot(D,'*')
axis([-2 2 -2 2])

figure;
plot(Xadj, Yadj, 'x');
hold on
plot(D,'*')
axis([-2 2 -2 2])


figure;
plot(FData,'d')
axis([-2 2 -2 2])

figure;
plot(OriginalAdjustedData,'r*')
axis([-2 2 -2 2])

Open in new window


How every i have this attached plot which is difference from expected as in the Tutorials.

Even though i have the adjusted data back, but i don't know how i can add the means back to the X and Y to get the original data back.

I have use both eigenvectors in the above code to get the feature and as a results have a 2 dimensions, however i would like to know if possible what modification can be done in my code to reduce the the dimensions from 2 to 1 by selecting taking only the eigenvector with the largest eigenvalue to transform the data from 2d to 1d. Any idea on this please?
Thanks
Plot.bmp
0
 

Author Comment

by:wish_C
Comment Utility
Any Idea on this issue on the above code please?
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
topping3 challenge 14 48
recursion example 16 68
C# DataGridView_RowsAdded event not firing 1 37
Re-position the objects 7 47
This article will show, step by step, how to integrate R code into a R Sweave document
Lithium-ion batteries area cornerstone of today's portable electronic devices, and even though they are relied upon heavily, their chemistry and origin are not of common knowledge. This article is about a device on which every smartphone, laptop, an…
This video teaches viewers about errors in exception handling.
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now