create multiple shared memory

What's your operating system?

In windows you use different names. In Linux you use different keys (typically using ftok for the key).

I am using Linux and developing a Pro*C program. I defined a variable for that using ftok but it can only generate 1 key. how do i generate different keys so that i can create multiple shared memory?

#define TRANSITION_SHM_KEY ftok(".", '2')

main() {
  create_shm (TRANSITION_SHM_KEY, 1000000, &shm_id);
  load_data (shm_id, &line_count);
}

void create_shm (key_t key, size_t size, int *id)
{
   int shm_id;

   shm_id = shmget (key, size, IPC_CREAT|IPC_EXCL|SHM_MODE);
   if (shm_id == -1)
   {
      perror ("Can't create share memory");
      return (1);
   }
   else
  {
      printf("\nSHM_ID:%x\n",key);
  }
  *id = shm_id;
}

load_data() {
   trans_param = (TRANS_PARAM *) shmat (shm_id, (void *) NULL, 0);
   if (trans_param == (TRANS_PARAM *) NULL)
   {
       perror ("shmat");
       return (1);
   }
 /* and so on..*/
}

elsia,

You should also be aware that shmem is similar to files, that is, a block survives when a process calls exit(). Cehck with the "ipcs" command if a block already exists.

Your program will fail if the shared memory block already exists. You probably don't want that. So, you should first try to get a block with a key via shmget(), and if you fail (check proper errno), create a new block:

int shm_open(char* key, size_t size){
    int shmid;
    key_t num_key = ftok(".",key);

    shmid = shmget(num_key, size, IPC_EXCL | SHM_MODE);

    if(shmid < 0 && errno == ENOENT)
        shmid = shmget(num_key, size, IPC_EXCL | SHM_MODE | IPC_CREAT);
       
   
    if(shmid < 0 ){
        perror("shmget");
        fail();
    }
           
    return shmid;
}

With shmat and a NULL address, you cannot use pointers in your shared memory, as the address your memory block will map to is arbitrary. If you want to use pointers, you must use an explicit address. But then shmat can fail when some other block is already at this address.

Tell me, Stefan, what's the benefit of using do {...} while (0) in your macro? I vaguely remember seeing this used by AN Other, but I can't remember finding out what the rationale was.

rstaveley,
The macro behaves just like a single C instruction.

When you either define a macro as
#define something(val) { step1; step2; }
#define something step1; step2

and then do

if(conditon)
    something(123);

The 2nd case will give you a nasty surprise. And you can't do

if(conditon)
    something(123);
else
    something_else();

in both cases. Using a do{ }while(0) wrapper works. Unfortunately, you cannot generate values this way; but there is a nice GCC extension for this:
http://gcc.gnu.org/onlinedocs/gcc-3.4.2/gcc/Statement-Exprs.html#Statement-Exprs

Stefan

Thanks Stefan! That's a good trick. I knew there would be method in your madness ;-)

hi stefan,

i tried running your script and it throws me invalid argument error:
SIZE = 0x01000000 (16MB)
Got shmid = 77935
Attached    0 blocks (   0MB) addr=0xe320c000 delta = 0x00000000 ( 0MB) sdelta = 0x00000000 (   0MB)
Failure in shmem_multimap.c:195 - shmat failure
addr != (void*)-1
22=Invalid argument

can you explain more about what is shmem_multimap.c doing and why do you need rand()?

hi rstaveley,

I tried creating multiple key from the same path, using a different project identifier but it doesnt work.. it is still refering to the same key althought the identifier is different. why is that so?

When you provide a fully qualified path with filename to the following, do you not get two different keys?

--------8<--------
#include <stdio.h>
#include <sys/types.h>
#include <sys/ipc.h>

int main(int argc,const char *argv[])
{
key_t key1,key2;

        key1 = ftok(*++argv,'1');
        key2 = ftok(*argv,'2');
        printf("key1 %x\n",key1);
        printf("key2 %x\n",key2);
}
--------8<--------

e.g.

   ./a.out /home/.

#define KEYPATHNAME "/share_mem/loadshm.dat"
#define KEYPATHNAME2 "/share_mem/loadshm.pc"
#define TRANSITION_SHM_KEY_2 ftok(KEYPATHNAME, '1')
#define TRANSITION_SHM_KEY_3 ftok("KEYPATHNAME2,'2')

void create_shm (key_t key, size_t size, int *id)
{
   int shm_id;

   shm_id = shmget (key, size, IPC_CREAT|IPC_EXCL|SHM_MODE);
   if (shm_id == -1)
   {
      perror ("Can't create share memory");
      return (1);
   }
   else
  {
      printf("Shared memory successfully created. SHM_ID:%x\n",key);
  }
  *id = shm_id;
}

void loadshm()
{
   char temp_key[25]="TRANSITION_SHM_KEY";
   key_t tmp;
   int r2;

   for (r2=2;r2<10;r2++)
   {
      sprintf(temp_key,"%s_%d",temp_key,r2);
      printf("\ntemp_key:%s\n",temp_key);
      tmp = (key_t)temp_key;
      result = create_share_mem (tmp, 1000000, &shm_id);

      if (result != 0)
      {
         printf ("%s Can't create share memory key = [%x]\n", generate_log_time (dtm, sizeof(dtm)), tmp);
         return (result);
      }

      strncpy(temp_key,NULL,strlen(temp_key));
      strcpy(temp_key,"TRANSITION_SHM_KEY");
   }       
}

main() {
  loadshm ();
}

the OUTPUT on screen is:
-------------------------------
temp_key:TRANSITION_SHM_KEY_2
Shared memory successfully created. SHM_ID:7b041f70

temp_key:TRANSITION_SHM_KEY_3
Can't create share memory : File exists
15/10/2004 15:02:02 Can't create share memory key = [7b041f70]

when i do a ipcs, i found that the mode of my shared memory is set to C.. which means that it needs to be cleared when the first attach is run

m  114799 0x7b041f70 -Cr--------   arbor13  arboradm

how do i change that mode?

Hmmm, maybe only Solaris supports multiple shmats of the same block.

But you can still mount a block read-only?

elsia,
> it needs to be cleared when the first attach is run

That makes sense - not to clear a newly created shmem block is a security problem. You could get memory previously released from another process.

I think you just need to shmat it for writing (shmat()'ing it for reading before writing something there doesn't make much sense.)

Consider populating the start of the block with a structure like:

struct shmem_header{
    char initialized;
    ...
}

then your read processes will know that it's virgin memory when initialized is 0.

elsia,
> can you explain more about what is shmem_multimap.c doing and why do
> you need rand()?

rand() was just to see that a write of a number into one attached block was reflected in all the other blocks, too.

About the invalid argument:
     EINVAL
           The shmid argument is not a valid shared memory  iden-
           tifier.

...that's probably the reason. You first need to run the other program to create a block with the 4711 key.

ok.. creating multiple key from different path, using a different project identifier is working now.. the reason why it wasnt working prevoiusly is because i created the shared mem in a loop. why cant i do that? it is redundant to manually create 9 shared memory and populate each with different data. how can i loop the shared memory creation and data loading using for loop?

i tried this but failed to create the 2nd memory:
transition.h
-------------
#define TRANSITION_ARRAY_SIZE (1000000)
#define KEYPATHNAME "/share_mem/mykey.dat"
#define KEYPATHNAME2 "."
:
#define TRANSITION_SHM_KEY_1 ftok(KEYPATHNAME, '1')
#define TRANSITION_SHM_KEY_2 ftok(KEYPATHNAME2, '2')
:

loadsharemem.pc
---------------------
int create_share_mem (key_t key, size_t size, int *id)
{
   int shm_id;
   shm_id = shmget (key, size, IPC_CREAT|IPC_EXCL|SHM_MODE);
   if (shm_id == -1)
   {
      perror ("Can't create share memory : shmget : ");
      return (1);
   }
   else
  {
      printf("Shared memory created successfully! shm key:%x\n",key);
  }
  *id = shm_id;
   return (0);
}

int main() {
for (i=1;i<10;i++)
{
   sprintf(temp_key,"%s_%d",temp_key,i);
   printf("\ntemp_key:%s\n",temp_key);
   tmp_r[i] = create_share_mem ((key_t)temp_key, 1000000, &shmid[i]);
   if (tmp_r[i] != 0)
   {
      printf ("Can't create share memory key = [%x] \n"(key_t)temp_key);
   }

   printf (" Now loading data for shmid [%d] ...\n",shmid[i]);
   tmp_r[i] = load_data (shmid[i], &line_count);
   if (tmp_r[i] != 0)
   {
       printf ("Can't load tabfile to share memory.\n");
   }
   printf ("Load data success, total records = [%d] share memory key = [%x]\n",line_count,(key_t)temp_key);

   strncpy(temp_key,NULL,strlen(temp_key));
   strcpy(temp_key,"TRANSITION_SHM_KEY");
}
}

the OUTPUT was:
---------------------
temp_key:TRANSITION_SHM_KEY_1
Shared memory created successfully! shm key:7b041f80
Now loading data for shmid [249967] ...
****
ORIGIN=[100], int_flag=[0]
sms_flg=[#], target=[40000]
key combination=[1000#]
****
ORIGIN=[100], int_flag=[1]
sms_flg=[#], target=[40010]
key combination=[1001#]
Load data success, total records = [2] share memory key = [7b041f80]

temp_key:TRANSITION_SHM_KEY_2
Can't create share memory : File exists
Can't create share memory key = [7b041f80] size = [20000264]

Try the following:

server.c
--------8<--------
#include <stdio.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/stat.h>
#include <sys/shm.h>
#include <unistd.h>
#include <errno.h>

int main(int argc,const char *argv[])
{
const char *filename;
key_t key1,key2;
struct stat statbuf;
int shmid1,shmid2;
#define SIZE 512
volatile char *buf1 = NULL,*buf2 = NULL;

        if (argc != 2)
                return (fprintf(stderr,"Usage: %s {keyfile}\n",argv[0]),1);
        filename = argv[1];
        if (stat(filename,&statbuf) != 0)
                return (fprintf(stderr,"Error: Unable to stat keyfile %s\n",filename),1);
        fprintf(stderr,"Using keyfile %s (inode %x)\n",filename,statbuf.st_ino);
        key1 = ftok(filename,'1');key2 = ftok(filename,'2');
        fprintf(stderr
                ,"Shared memory key:\n"
                 "\tkeyfile: %s\n"
                 "\tkey '1': %x (project %x)\n"
                 "\tkey '2': %x (project %x)\n"
                ,filename,key1,'1',key2,'2');
        if ((shmid1 = shmget(key1,SIZE,IPC_CREAT|IPC_EXCL|0666)) == -1) {
                if (errno == EEXIST) {
                        if ((shmid1 = shmget(key1,SIZE,0666)) == -1)
                                return (perror("shmget existing key1"),1);
                        fprintf(stderr,"Got existing shmid1 %x\n",shmid1);
                }
                else
                        return (perror("shmget create key1"),1);
        }
        else
                fprintf(stderr,"Created shmid1 %x\n",shmid1);
        if ((shmid2 = shmget(key2,SIZE,IPC_CREAT|IPC_EXCL|0666)) == -1) {
                if (errno == EEXIST) {
                        if ((shmid2 = shmget(key2,SIZE,0666)) == -1)
                                return (perror("shmget existing key2"),1);
                        fprintf(stderr,"Got existing shmid2 %x\n",shmid2);
                }
                else
                        return (perror("shmget create key2"),1);
        }
        else
                fprintf(stderr,"Created shmid2 %x\n",shmid2);
        if ((buf1 = (char*)shmat(shmid1,NULL,0)) == NULL)
                return (perror("shmat shmid1"),1);
        if ((buf2 = (char*)shmat(shmid2,NULL,0)) == NULL)
                return (perror("shmat shmid2"),1);
        strcpy((char*)buf1,"Server has written into project '1'");
        strcpy((char*)buf2,"Server has written into project '2'");
        printf("The buffer for project '1' contains: %s\n",buf1);
        printf("The buffer for project '2' contains: %s\n",buf2);
        sleep(2);
        printf("The buffer for project '1' contains: %s\n",buf1);
        printf("The buffer for project '2' contains: %s\n",buf2);
}
--------8<--------

client.c
--------8<--------
#include <stdio.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/stat.h>
#include <sys/shm.h>
#include <unistd.h>
#include <errno.h>

int main(int argc,const char *argv[])
{
const char *filename;
key_t key1,key2;
struct stat statbuf;
int shmid1,shmid2;
#define SIZE 512
volatile char *buf1 = NULL,*buf2 = NULL;

        if (argc != 2)
                return (fprintf(stderr,"Usage: %s {keyfile}\n",argv[0]),1);
        filename = argv[1];
        if (stat(filename,&statbuf) != 0)
                return (fprintf(stderr,"Error: Unable to stat keyfile %s\n",filename),1);
        fprintf(stderr,"Using keyfile %s (inode %x)\n",filename,statbuf.st_ino);
        key1 = ftok(filename,'1');key2 = ftok(filename,'2');
        fprintf(stderr
                ,"Shared memory key:\n"
                 "\tkeyfile: %s\n"
                 "\tkey '1': %x (project %x)\n"
                 "\tkey '2': %x (project %x)\n"
                ,filename,key1,'1',key2,'2');
        if ((shmid1 = shmget(key1,SIZE,0666)) == -1)
                return (perror("shmget key1"),1);
        fprintf(stderr,"Got existing shmid1 %x\n",shmid1);
        if ((shmid2 = shmget(key2,SIZE,0666)) == -1)
                return (perror("shmget key2"),1);
        fprintf(stderr,"Got existing shmid1 %x\n",shmid1);
        if ((buf1 = (char*)shmat(shmid1,NULL,0)) == NULL)
                return (perror("shmat shmid1"),1);
        if ((buf2 = (char*)shmat(shmid2,NULL,0)) == NULL)
                return (perror("shmat shmid2"),1);
        strcpy((char*)buf1,"Client has written into project '1'");
        strcpy((char*)buf2,"Client has written into project '2'");
}
--------8<--------

Build:
   gcc server.c -o server
   gcc client.c -o client

Execute:
    ./server . & sleep 1;./client .

(i.e. run the server in background mode, so you can run the client while the server is running, I am using the current directory as the keyfile in this example - you can of course use another file)

This leaves the shared memory in the system. The server is responsible for creating it if it isn't already created. It gives read+write permissions to the server and client. In this exmaple bothe the server and the client write, but only the server reads.

BTW... I've not bothered illustrating how to use shmctl to remove the shared memory (with IPC_RMID). You can, however, and should use the command line tool ipcrm to remove the shmids, when you've finished with this.

Does this help?

BTW... if you are wondering what those 0666 (octal numbers are), they provide read+write permissions to user, group and all. You can of course change these to 0600, if you want to restrict read+write permissions to the user only (good idea if you are on a shared server).

I think that you were bitten by permissions and/or failing to remove the shared memory. The IPC_EXCL flag makes shmget fail if shared memory with the provided key already exists. You could use IPC_CREAT on its own in the server, but you probably ought to be aware of whether the shared memory is being reused or not, which is why my illustration uses IPC_CREAT|IPC_EXECL and then tests for errno EEXISTS, with a message that lets you know if the memory is newly created or being reused.

Here's how to get rid of the stderr stuff when you run that test.

./server . 2> /dev/null & sleep 1;./client . 2> /dev/null

You should get an output like the following:

--------8<--------
[rstaveley@mims shm]$ ./server . 2> /dev/null & sleep 1;./client . 2> /dev/null
[1] 26362
The buffer for project '1' contains: Server has written into project '1'
The buffer for project '2' contains: Server has written into project '2'
[rstaveley@mims shm]$ The buffer for project '1' contains: Client has written into project '1'
The buffer for project '2' contains: Client has written into project '2'

[1]+  Exit 73                 ./server . 2>/dev/null
--------8<--------

The output can be confusing if you are not used to running applications in background mode, but in essence you see the shared memory displayed by the server twice; once before the client has written to it and once afterwards.

ok.. finally got my shared memory running.. but i am facing some problem with the memory size. i tried to load about 15000 records into 5 of the 9 shared memory created but the program throw a core dump error.

When i tried to increase the shared memory size from 1000000 to 1500000, i got an Invalid Argument error.
shmget (key, 1000000, IPC_CREAT|IPC_EXCL|SHM_MODE);


I tried to increase the number of my array too and i end up getting:
error 1809: size of aggregate exceeds compiler limits.

Have checked the user limit in unix and it's unlimited.. What is wrong? How do i fix it?