Improve company productivity with a Business Account.Sign Up

x
  • Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 177
  • Last Modified:

Fastest Method of Searching

Hi !

I have a text file about 3 Meg in size.  I have to search for a string of about 7 characters in length at the beginning of each line.  Each line can be well over 300 characters.  Currently I am opening the file and reading the whole line with fgets and comparing the first 7 characters.  What would be the best method for this as I need to increase the speed.

Marvin.
0
checkin
Asked:
checkin
1 Solution
 
newexpertCommented:
For 32bit apps obtain 3MB of memory

char *membuf = (char *)malloc(sizeof(char) * 3145728);
int filelen = fread(membuf, sizeof(char), 3145728, fileptr);

and search through membuf will be a lot faster.
0
 
ozoCommented:
Build a hash index, and jump directly to the line.
0
 
checkinAuthor Commented:
Hi !

I do not want to play with malloc as the system we use could possible swap allot if under pressure.  This could cause problems.

I am interested in building a hash index, where can I get some info about this as I have never touched these before.

Regards,

Marvin.
0
 
rbrCommented:
Short example for a hash


typedef struct hash_entry {
      unsigned char key[8];
      struct hash_entry *pnext;
      long file_pos;
} HASH_ENTRY;

void init_hash (HASH_ENTRY ***ppphash, long anz)
{
      *ppphash= (HASH_ENTRY **)calloc (sizeof(HASH_ENTRY *),anz);
}
int hash_function (unsigned char *pkey)
{
      unsigned long hash=0;
      
      for (i=0;i<strlen(pkey);i++)
            hash+=pkey[i];
      return (hash);
}

void add_to_hash (HASH_ENTRY **phash, long anz, unsigned char *pkey,unsigned long file_pos)
{
      
      HASH_ENTRY *p,*plast=NULL,*pcurr;

      p= (HASH_ENTRY *)calloc(sizeof(HASH_ENTRY),1);

      pcurr = phash[hash_function (pkey)%anz];
      while (pcurr != NULL) {
            plast=pcurr;
            pcurr=pcurr->pnext;
      }

      if (plast==NULL) {
            phash[pos]=p;
      } else {
            plast->pnext=p;
      }
      p->filepos=filepos;
      strcpy (p->pkey,pkey);
}      

int find_in_hash (HASH_ENTRY **pphash,long anz,unsigned char *pkey, unsigned long *pfilepos)      
{

      HASH_ENTRY *pcurr;      

      pcurr = phash[hash_function (pkey)%anz];
                                    
      while (pcurr!=NULL) {
            if (strcmp (pcurr->key,pkey)==0) {
                  *pfilepos=pcurr->filepos;
                  return (0); /* Found */
            }
            pcurr=pcurr->pnext;
      }
      return(1); /* not found*/
}

/* Main routine */

HASH_ENTRY **pphash;
long anz=1000L; /* The greater the faster */

init_hash (&pphash,anz);

/* loop over all lines and for each line use */
add_to_hash (pphash,anz,key,filepos); /* Where key is your search key and filepos the line in the file or position of ftell */

/* Now you can search a key with find_in_hash */

            
0
 
rbrCommented:
int hash_function should be unsigned long hash_function
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Get your problem seen by more experts

Be seen. Boost your question’s priority for more expert views and faster solutions

Tackle projects and never again get stuck behind a technical roadblock.
Join Now