Solved

Parallel Processing in C Using Unix Commands

Posted on 2003-12-04
16
427 Views
Last Modified: 2010-04-15
I'm trying to write a program in C using Unix commands that will perform parallel processing of matrices for subtraction, multiplication, and transpose purposes. It will look like this when called:
matrix_proc rp cp <pathname1> <a/s/m/T> [pathname2] <pathename3>
pathname 2 is only optional if the operation is transposing. The operation to be performed will end as such:
a: m3=m1+m2
s: m3=m1-m2
m: m3=m1*m2
T: m3= transpose of m1

The program should spawn rp*cp processes to calculate the elements of the resulting matrix. rcp and cp indicate the number of processes working in parallel on the rows and columns of the resulting matrix respecitvely. The elements of the resulting matrix should be evenly split among the rp*cp processes with no less than one element being assigned to each process. (Assume that the matrices will have valid sizes for the operation being preformed, as I already made the error checking for this part.)

Also, passing of information from parents to child processes should be done through shared memory variables, whereas passing from child to parents processes should be done through pipes.
0
Comment
Question by:JeebusFnChrist
  • 6
  • 6
  • 4
16 Comments
 
LVL 45

Expert Comment

by:sunnycoder
ID: 9880056
course assignment ?
0
 

Author Comment

by:JeebusFnChrist
ID: 9880150
No, it's related to a class I'm taking but it's an outside project more for my own benefit since, clearly, I haven't been able to learn much from the teacher of that course.
0
 
LVL 84

Expert Comment

by:ozo
ID: 9880193
int p[rp][cp][3];

     if( !pipe(p[i][j]) ){ perror("pipe")'; exit(1); }
     shared1 = m1[i][j];
     shared2 = m2[i][j];
     while( (f = fork()) < 0 ){  perror("fork"); sleep(1); }
     if( f==0 ){
         close(p[i][j][0];
         shared1 *= shared2;;
         write(p[i][j][1], &shared1,sizeof(shared1));
         exit(0);
     }else{
           close(p[i][j][1])';
           p[i][j][2] = f;
     }
  }
}
for( i=0;i<rp;i++ ){
  for(j=0;j<cp;j++ ){
    read(p[i][j][0],&m0[i][j]);
    close(p[i][j][0]);
    waitpid(p[i][j][2],&staatus,WNOHANG);
   }
}
0
 
LVL 84

Expert Comment

by:ozo
ID: 9880242
int p[rp][cp][3];
 int i,j;
for( i=0;i<rp;i++ ){
  for(j=0;j<cp;j++ ){
    int shared1,shared2,f;
     if( pipe(p[i][j]) ){ perror("pipe"); exit(1); }
     shared1 = m1[i][j];
     shared2 = m2[i][j];
     while( (f = fork()) < 0 ){  perror("fork"); sleep(1); }
     if( f==0 ){
         close(p[i][j][0]);
         shared1 *= shared2;
         write(p[i][j][1], &shared1,sizeof(shared1));
         exit(0);
     }else{
           close(p[i][j][1]);
           p[i][j][2] = f;
     }
  }
}
for( i=0;i<rp;i++ ){
  for(j=0;j<cp;j++ ){
    int status;
    read(p[i][j][0],&m0[i][j],sizeof(m0[i][j]));
    close(p[i][j][0]);
    waitpid(p[i][j][2],&status,WNOHANG);
   }
}
0
 
LVL 45

Expert Comment

by:sunnycoder
ID: 9880435
ozo ... If it were in my hands, I would give you the most compact (cryptic?) coder award ;o)

JeebusFnChrist,
Do you need more help ... If you really are leaning then I guess you would !!
0
 

Author Comment

by:JeebusFnChrist
ID: 9880530
Yes, sunnycoder I could use more help. We've only had two coding assignments in this class and they were very simple so I'm not prepared for the final in any way. I found this problem in the book (In which the information provided is more cryptic than both ozo and my teacher combined) and if I could find an answer to this, preferably with a few comments though I'm good at interpretting code, it would be an excellent tool to help me learn and prepare for this class. Like I said though, this isn't an assignment I'm being graded on and is for my benefit. I've taken a lot of programming classes and unix is very unlike anything I've seen before.

Ozo, if it's possible it would be better to use the commands included in unistd.h for things like opening and creating files as it is using unistd.h that is the main focus of what I'm trying to learn. (Though the parallel processing has so far greatly eluded me which I attribute to the large emphasis being put on learning the syntax for unistd.h.)

So in conclusion to this long winded comment, thank you both very much and yes I need more help. As soon as I have an understanding of how to solve a problem like this I will gladly award your points.
0
 
LVL 84

Accepted Solution

by:
ozo earned 500 total points
ID: 9880594
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
main(int argc,char *argv[]){
  int **m1,**m2,**m3,***p;
  int rp,cp,i,j;
  FILE *F;
  rp = atoi(argv[1]);
  cp = atoi(argv[2]);
  F = fopen(argv[3],"r");
  m1 = malloc(rp*sizeof(*m1));
  m2 = malloc(rp*sizeof(*m2));
  m3 = malloc(rp*sizeof(*m3));
  p = malloc(rp*sizeof(*p));
  if( !F ){ perror(argv[3]); exit(1); }
  for( i=0;i<rp; i++ ){
    m1[i] = malloc(cp*sizeof(*m1[i]));
    m2[i] = malloc(cp*sizeof(*m2[i]));
    m3[i] = malloc(cp*sizeof(*m3[i]));
    p[i] = malloc(cp*sizeof(*p[i]));
    for( j=0;j<cp;j++ ){
      p[i][j] = malloc(3*sizeof(p[i][j]));
      fscanf(F,"%d",&m1[i][j]);
    }
    /*fscanf(F,"\n")*/
  }

  fclose(F);
  if( argv[4][0] != 'T' ){
    F = fopen(argv[5],"r");
    if( !F ){ perror(argv[5]); exit(1); }
    for( i=0;i<rp; i++ ){
      for( j=0;j<cp;j++ ){
        p[i][j] = malloc(3*sizeof(p[i][j]));
        fscanf(F,"%d",&m2[i][j]);
      }
      /*fscanf(F,"\n")*/
    }
    fclose(F);
  }
  for( i=0;i<rp;i++ ){
    for( j=0;j<cp;j++ ){
      int shared1,shared2,f;
      if( pipe(p[i][j]) ){ perror("pipe"); exit(1); }
      shared1 = m1[i][j];
      shared2 = m2[i][j];
      while( (f = fork()) < 0 ){  perror("fork"); sleep(1); }
      if( f==0 ){
         close(p[i][j][0]);
         switch( argv[4][0] ){
         case 'm': shared1 *= shared2; break;
         case 'a': shared1 += shared2; break;
         case 's': shared1 -= shared2; break;
         default: break;
         }
         write(p[i][j][1], &shared1,sizeof(shared1));
         exit(0);
      }else{
        close(p[i][j][1]);
        p[i][j][2] = f;
      }
    }
  }
  if( argv[4][0] == 'T' ){
    F = fopen(argv[5],"w");
    if( !F ){ perror(argv[5]); exit(1); }
  }else{
    F = fopen(argv[6],"w");
    if( !F ){ perror(argv[6]); exit(1); }
  }

  for( i=0;i<rp;i++ ){
    for( j=0;j<cp;j++ ){
      int status;
      if( argv[4][0] == 'T' ){
         read(p[i][j][0],&m3[j][i],sizeof(m3[j][i]));
      }else{
         read(p[i][j][0],&m3[i][j],sizeof(m3[i][j]));
      }
      close(p[i][j][0]);
      waitpid(p[i][j][2],&status,WNOHANG);
     }

  }
  for( i=0;i<rp;i++ ){
    for( j=0;j<cp;j++ ){
      fprintf(F,"%d ",m3[i][j]);
      printf("%d ",m3[i][j]);
    }
    fprintf(F,"\n");
    printf("\n");
  }
  fclose(F);
  exit(0);
}
0
 
LVL 45

Expert Comment

by:sunnycoder
ID: 9880623
> parallel processing
there are two common ways of parallel processing, threads and forking ... I feel threads suit better for this application... Do you have pthreads or some other library installed at you end ?

Another thing, are you familiar with programming with pthreads and fork/exec system calls

>shared memory variables, whereas passing from child to parents processes should be done through pipes.
Are you familiar with IPC programming ... pipes, shared memory etc...

If not, then we can begin by programming with these system calls first ... once you are confortable with them, we can move on to bigger problem like the one you posted above
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 

Author Comment

by:JeebusFnChrist
ID: 9880803
Ozo, thanks again. I'll work on figuring out what all this means. Looks much more understandable than the previous code so shouldn't be too hard.

Sunnycoder, I do not have access to any thread libraries, so it will have to be done with forking. I'm not familiar with pthreads, but I have some understanding of forking. I'm not sure what IPC programming consists of besides what you mentioned; however, I am familiar with pipes and shared memory, though shared memory only in theory, not practice. I think I'm comfortable enough with the theory behind the necessary tools to handle something like this, it's just more a matter of syntax and usage. I learn best from anaylzing actual code (Which I think would be the most effective way in general), so the most useful tool would probably be code for the above problem with comments in the places that focus on these ideas that I have theoretical and not practical knowledge of.

And I'd like to thank you both again for all this help. I came to this site by recommendation of my father who has been a member for some time now and I must say you two have made my first experience here a very positive one. I only hope that I can be as helpful to others in the future as well.
0
 
LVL 45

Expert Comment

by:sunnycoder
ID: 9880929
>I'm not sure what IPC programming consists of besides what you mentioned
message queues , signals ... loosely we also include sockets

>I think I'm comfortable enough with the theory behind the necessary tools to handle something like this
> learn best from anaylzing actual code
Ok, then spend sometime with ozo's code
He generally writes excellent code, so you will get a lot to learn from it ... Incase you need some more help post back.

>And I'd like to thank you both again for all this help. I came to this site by recommendation of my father who has been a
>member for some time now and I must say you two have made my first experience here a very positive one. I only hope that I
>can be as helpful to others in the future as well.
Glad to know that :o)
0
 

Author Comment

by:JeebusFnChrist
ID: 9880972
I agree that ozo writes excellent code as this makes perfect sense to me. However, unless I'm reading it wrong it doesn't seem to create a new file named arg[6] (or arg[5] in the event of arg[4]=='T') and write the resulting matrix m3 to this file, though I realize now I hadn't posted that in the original reply. I have some code in my notes pertaining to creating and writing to files using the Unix commands in C, so in the morning I'll look that up and modify the code to include that and either of you can check to see that it still makes sense. As for giving out the points, since this is the first time I've posted a question I'm not sure what to do. You've both been very helpful and I'd like to give each of you the full 500 points since this question was posted in 2 different areas, but I'm not sure if that's frowned upon since it's the same question. Am I just supposed to close/delete the other post and only award points for this one?
0
 
LVL 45

Expert Comment

by:sunnycoder
ID: 9881012
I normally try to avoid reading code (esp. ozo's) ... but anycase it should be a good start ... If you get stuck we are there to help anyway.

>As for giving out the points, since this is the first time I've posted a question I'm not sure what to do. You've both been very
>helpful and I'd like to give each of you the full 500 points since this question was posted in 2 different areas, but I'm not sure
>if that's frowned upon since it's the same question. Am I just supposed to close/delete the other post and only award points
>for this one?

1. You do not have to close a question until you feel your problem has been solved

2. If you ever have a question that you feel deserves attention from two topic areas, it is best to post it in one topic area and post a pointer question (with link to original question) in the other topic area...

3. Moreover, I do not think I deserve points for this question ... it was all ozo's work... If you wish to close the question right now (that would amount to saying that your problem has been solved), accept ozo's answer in any one of the questions (one which you feel is better) and post a request in community support to delete the other question.
0
 

Author Comment

by:JeebusFnChrist
ID: 9893454
I compiled this and changed the couple things I needed to so it works fine. One quick question before I close this though:
How would I go about doing a processing speed check for the program? The only way I've ever done this before was just by printing the system time to the screen before and after running a program. Is there a more elegant way to do this?
0
 
LVL 84

Expert Comment

by:ozo
ID: 9893691
man times
0
 
LVL 45

Expert Comment

by:sunnycoder
ID: 9894932
also check these

man clock
http:/Q_20618373.html#8544359
0
 

Author Comment

by:JeebusFnChrist
ID: 9894982
Thanks a lot, ozo. I appreciate all the help
0

Featured Post

What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

Have you thought about creating an iPhone application (app), but didn't even know where to get started? Here's how: ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ Important pre-programming comments: I’ve never tri…
This tutorial is posted by Aaron Wojnowski, administrator at SDKExpert.net.  To view more iPhone tutorials, visit www.sdkexpert.net. This is a very simple tutorial on finding the user's current location easily. In this tutorial, you will learn ho…
The goal of this video is to provide viewers with basic examples to understand how to create, access, and change arrays in the C programming language.
The goal of this video is to provide viewers with basic examples to understand and use conditional statements in the C programming language.

760 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

22 Experts available now in Live!

Get 1:1 Help Now