?
Solved

multi thread writing

Posted on 2003-02-27
14
Medium Priority
?
438 Views
Last Modified: 2010-04-15
hi there,
I need some help with the multi threading, and i am sweatering with it right now. Thanks for your help.

Say, i everytime i retrieve 1000 records and i would want to split it to 2-3 threads, each thread would write to a seperate file, finally to combine the files into one. Any detailed working/example codes to show how do i create a thread, pass this array to thread, and wrting to files?
0
Comment
Question by:pigyc
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 6
  • 5
  • 2
  • +1
14 Comments
 

Expert Comment

by:posternb
ID: 8036145
What compiler/platform?  Any particular thread library in mind/available?
0
 

Author Comment

by:pigyc
ID: 8037029
on hp11, compiler? what command would i be able to check? I heard around that pthread might be the one unless there are other thread libs.
0
 
LVL 1

Expert Comment

by:dnataraj
ID: 8040307
# include <pthread.h>
# include <unistd.h>
# include <stdio.h>

void * WriteToFile1(void *arg){
     while(1)
        {
               //Read Record Function()
               //Write Record Function()

        }
}

void * WriteToFile2(void *arg){
     while(1)
        {
          //Read Record Function()
               //Write Record Function()
        }
}

main(){
     pthread_t pt1,pt2;
     pthread_create(&pt1,NULL,WriteToFile1,NULL);    
     pthread_create(&pt2,NULL,WriteToFile2,NULL);    
     
     while(1)
         {

}
0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 1

Expert Comment

by:dnataraj
ID: 8040308
# include <pthread.h>
# include <unistd.h>
# include <stdio.h>

void * WriteToFile1(void *arg){
     while(1)
        {
               //Read Record Function()
               //Write Record Function()

        }
}

void * WriteToFile2(void *arg){
     while(1)
        {
          //Read Record Function()
               //Write Record Function()
        }
}

main(){
     pthread_t pt1,pt2;
     pthread_create(&pt1,NULL,WriteToFile1,NULL);    
     pthread_create(&pt2,NULL,WriteToFile2,NULL);    
     
     while(1)
         {
                usleep(100);
         }

}


compile with -lpthread option
0
 
LVL 10

Expert Comment

by:makerp
ID: 8041045
are you sure using more than 1 thread willl yield any real performance benifit. people often think that if they use more than one thread the task will be completed quicker; this is often not the case. the only real problems that can be solved with multithreading is getting over the delays with blocking calls; for example socket work etc. even this can be solved with async IO instead of blocking IO....

if you want to learn multi-threading then write an application that makes good use of it...

0
 

Author Comment

by:pigyc
ID: 8050352
thanks for emailing my questions.
after reading 'makerp' email, i choose a path which is  different than my original ways:
launching two threads, each thread would retrieve a block of data from database and do some calculation and put the result into an array; finally each thread wrting the array result to a seperate file. Now everything has its merried way with one single thread and i saw each single file has correct records when running the program with single thread, until i run two threads together, then the results in two seperate files are wired, and it is hard to trace/tell what's happening inside.
i have the following pseudo code, anyone could light me that what is the  problem with this? why i don't have anything written out for these two files while single thread would write out correctly? Does one thread disturb/blocking/interfere the other thread's writing? BTW: if you have trick ways to trace, that would be very helpful. thanks

void *get_history_all( void *argv )
{
  /*get data and put them in a array*/
write_history_records(result, fp,  num_rows);
destroy_array(&result);
}

void write_history_records(char *result, FILE *fp, int num){
for(int i=0; i<num, i++)
fprintf(fp, "%s", ...);
}


int main(){
char temp1_file[256], temp2_file[256];
FILE *t1_fp, *t2_fp;
strcpy(temp1_file, "thread1.txt");
  strcpy(temp2_file, "thread2.txt");

  if(!(t1_fp = fopen(temp1_file, "w"))){
    printf("temp1_file open failed");
  }
  if(!(t2_fp = fopen(temp2_file, "w"))){
    printf("temp2_file open failed");
  }

 argval[0].fp = t1_fp;
 argval[0].fp = t1_fp;

ret_code = pthread_create(&first_thread, NULL, (void *(*)(void *))get_history_all, (void *)&(argval[0]) )
;
if (ret_code != 0)
  {
    printf("thread_create failed, error number: %d\n", ret_code);
    exit(1);
  }

ret_code = pthread_create(&second_thread, NULL, (void *(*)(void *))get_history_all, (void *)&(argval[1]))
;
if (ret_code != 0)
  {
    printf("arb_thread_create failed, error number: %d\n", ret_code);
    exit(1);
  }
/* wait for the first thread to terminate */
  ret_code = pthread_join(first_thread, NULL);
  if (ret_code != 0)
  {
    printf("first thread_create failed, error number: %d\n", ret_code);
  }
  fprintf(t1_fp,"%s","f1 finished");
  fclose(t1_fp);
  /* wait for the second thread to terminate */
  ret_code = pthread_join(second_thread, NULL);
  if (ret_code != 0)
  {
    printf("second thread_create failed, error number: %d\n", ret_code);
  }
  fclose(t2_fp);
  exit(0);
}

 
0
 

Author Comment

by:pigyc
ID: 8050550
thanks for emailing my questions.
after reading 'makerp' email, i choose a path which is  different than my original ways:
launching two threads, each thread would retrieve a block of data from database and do some calculation and put the result into an array; finally each thread wrting the array result to a seperate file. Now everything has its merried way with one single thread and i saw each single file has correct records when running the program with single thread, until i run two threads together, then the results in two seperate files are wired, and it is hard to trace/tell what's happening inside.
i have the following pseudo code, anyone could light me that what is the  problem with this? why i don't have anything written out for these two files while single thread would write out correctly? Does one thread disturb/blocking/interfere the other thread's writing? BTW: if you have trick ways to trace, that would be very helpful. thanks

void *get_history_all( void *argv )
{
  /*get data and put them in a array*/
write_history_records(result, fp,  num_rows);
destroy_array(&result);
}

void write_history_records(char *result, FILE *fp, int num){
for(int i=0; i<num, i++)
fprintf(fp, "%s", ...);
}


int main(){
char temp1_file[256], temp2_file[256];
FILE *t1_fp, *t2_fp;
strcpy(temp1_file, "thread1.txt");
  strcpy(temp2_file, "thread2.txt");

  if(!(t1_fp = fopen(temp1_file, "w"))){
    printf("temp1_file open failed");
  }
  if(!(t2_fp = fopen(temp2_file, "w"))){
    printf("temp2_file open failed");
  }

 argval[0].fp = t1_fp;
 argval[0].fp = t1_fp;

ret_code = pthread_create(&first_thread, NULL, (void *(*)(void *))get_history_all, (void *)&(argval[0]) )
;
if (ret_code != 0)
  {
    printf("thread_create failed, error number: %d\n", ret_code);
    exit(1);
  }

ret_code = pthread_create(&second_thread, NULL, (void *(*)(void *))get_history_all, (void *)&(argval[1]))
;
if (ret_code != 0)
  {
    printf("arb_thread_create failed, error number: %d\n", ret_code);
    exit(1);
  }
/* wait for the first thread to terminate */
  ret_code = pthread_join(first_thread, NULL);
  if (ret_code != 0)
  {
    printf("first thread_create failed, error number: %d\n", ret_code);
  }
  fprintf(t1_fp,"%s","f1 finished");
  fclose(t1_fp);
  /* wait for the second thread to terminate */
  ret_code = pthread_join(second_thread, NULL);
  if (ret_code != 0)
  {
    printf("second thread_create failed, error number: %d\n", ret_code);
  }
  fclose(t2_fp);
  exit(0);
}

 
0
 
LVL 10

Accepted Solution

by:
makerp earned 1000 total points
ID: 8051311
have you omitted some global variables.... if so these will need protecting with mutexes. when you have data that is used by more than one thread it is often the case that you will need to restrict access to one thread at a time other wise the data will get corrupt. look up synchronisation...

now if you are going to use synchonisation to ensure only one thread is writing to the array at any one time then what on earth is the point of having two threads doing it. all they will do is end up taking it in turns to write to the array, in actual fact it will probably take longer for two threads than one because if you are only using one you not need to worry about synchronsation and therefore wont have the overhead of it (which can be quite considerable)..

Paul
0
 
LVL 10

Expert Comment

by:makerp
ID: 8051321
pthread_mutex_init(&mutex, NULL);     //initialize the mutex object
0
 

Author Comment

by:pigyc
ID: 8051915
paul,
You have the points. yes if only thread could write to array, then there is no point to use two threads. I have something unclear--i don't think i have gloables unless the resizable array dynamically created in the get_history_all func would considered 'global' to two threads---I would imagine each thread would have their own access to get_history_all func and wouldn't share the array they each created in this function; now after they each created the array and fill in with data, it comes to write_history_record and passed in the array thread created to write to a different file. Would the array passed into the write_history_recored share by two theads? I wasnot quite sure on this--in single instance/thread, this wouldn't consider as global.

void *get_history_all( void *argv )
{
   /*first create resizeable array locally called */

 /*get data and put them in a array*/
write_history_records(result, fp,  num_rows);
destroy_array(&result);
}

void write_history_records(char *result, FILE *fp, int num){
for(int i=0; i<num, i++)
fprintf(fp, "%s", ...);
}


I define a struc to pass into the createthread function.
typedef struct loaderarg
{
  db_connection          *conn;
  int                           account_no_small;
  int                           account_no_big;
  FILE  *fp;
} LoaderArg;
0
 

Author Comment

by:pigyc
ID: 8051949
i accidently hit the enter key.


in writing_history_record(   ){
/*first locally create resizeable array called*/
/*   RA    result;*/
/*filled in the array with data from db to 'result*/
/*pass this result to write_history_record   */
}

would this result array considered global to two threads?

I found anther thing which might add the reason why i don't get correct records to two output files: seems my db connection to fetech data from db is not thread safe--i need to figure out a way to do that. Ultimately,it is a learning process for me to touch multi threading. thanks
0
 
LVL 10

Expert Comment

by:makerp
ID: 8058765
if the array is local to the function then each thread has its own array, if this is the case then forget my previous comment about synchronisation...

as far as your db_connection goes, i would wrap this with a mutex
0
 

Author Comment

by:pigyc
ID: 8064382
Paul,
Thanks for your email. Yes i tested using the array not created dynamically using and it wrokds. I tested
db connection with mutex it works, and aslo i tested to open my db connection inside each thread(inside get_history_all function, and it works too). Things are narrowing down to what are globals(locals) to threading. As long as i don't mess this up, it works. And the box running my program has multi processors which is a good news. thanks for your help
0
 
LVL 10

Expert Comment

by:makerp
ID: 8074062
glad to help....
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

An Outlet in Cocoa is a persistent reference to a GUI control; it connects a property (a variable) to a control.  For example, it is common to create an Outlet for the text field GUI control and change the text that appears in this field via that Ou…
Summary: This tutorial covers some basics of pointer, pointer arithmetic and function pointer. What is a pointer: A pointer is a variable which holds an address. This address might be address of another variable/address of devices/address of fu…
The goal of this video is to provide viewers with basic examples to understand and use pointers in the C programming language.
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use for-loops in the C programming language.

765 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question