Solved

File record sort

Posted on 2000-03-18
15
300 Views
Last Modified: 2010-04-02
If you have a file containing line sequential records (structures) and there are three different structure types with a common ‘item code’ field, how do you sort these structures in ascending order of item code and write the sorted structures to a new file? Iam trying to use malloc but not sure if this is correct.
Thank you for any help.
0
Comment
Question by:spark60
  • 4
  • 4
  • 2
  • +4
15 Comments
 
LVL 1

Expert Comment

by:ntdragon
ID: 2631398
send so code and i"ll help you
i don't understand what execttly is you problem

you should open the file using fopen(...)
as i remember
then to read all the data in to an array of structure the sort it and put it back if you have a spesific problem tell me and i"ll try to help you
0
 

Expert Comment

by:sylphe
ID: 2631461
Could you be more precise about the different structures, and the item code ? What about the size of the 3 structures, are they equals ?
0
 
LVL 1

Expert Comment

by:ntdragon
ID: 2632055
about the size now two are three opetions

1)all the structures have only static datamembers then you should have in the file some sort of indecator that will tell you which structure is it and you"ll know the size by it

2)if the structure have a dynamic datamembers then execpt the indecator put the stucture size

i hope you understood this one

now about the sorting you should in put the structure first there is two thing that you can do

1)use the idea of containers from c++
put all the structures in one array and sort then and put it in the file

2)use three arrays one for each structure and then you the merge algorithm and merge the three arrays into the file

i recommend the second becouse you are useing structures and not class's

hope i helped more this time
0
 
LVL 22

Expert Comment

by:cookre
ID: 2632389
I'd make one pass thru the file storing just the item code and the record number it came from, sort that much smaller, fixed length array, then re-build the file based on the sorted array.

The sort will be easier and faster, although at the expense of the file re-build - if you can't load the whole thing into RAM.  If you can, so much the better.
0
 
LVL 10

Expert Comment

by:rbr
ID: 2635522
Pls post more info or better a code what you have done already.
0
 
LVL 22

Expert Comment

by:cookre
ID: 2636058
If you can load the entire file into RAM, there's a real quick way to rebuild the file after the sort.

As you build the array to be sorted, instead of the record number, save the offset of the beginning of the record.  The rebuild will be right peppy.

Note that even if the whole file won't fit into RAM, you can still save the offset of the record within the file so that you can go directly to it later without having to count lines.

0
 
LVL 9

Expert Comment

by:Pacman
ID: 2636344
I agree with cookre.

I did something like that in earlier DOS-times. When the it run out of memory then you can split it into two or more destination files.
Then you must du a merge at the end.

spark60: you have many possibilities. From easy to pretty complicated. I would say it depends on size of data and size of memory which one you choose.
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 

Expert Comment

by:jaicon
ID: 2640467
I think that you must to use one array of UNIONs(Using realloc for redim the size of this array), each UNION with the three structures, in this way you can read any register and then you can use qsort for example to sort the array by item Code; and then write it into the file.

I wait for any comments .

0
 
LVL 22

Expert Comment

by:cookre
ID: 2640640
jaicon: I see you're new to EE, so you probably don't know that it's a common convention to post comments only.  

Submitting an answer moves the question to the 'locked' category.  Few experts look at locked questions, thereby denying the questioner the additional audience.

Don't worry about it - lots of first-timers do it - and welcome to EE!
0
 

Expert Comment

by:jaicon
ID: 2645998
I am sorry, I didn't know this before. Thanks.
0
 

Author Comment

by:spark60
ID: 2650603
Thanks for the response.This seems to be along the lines I am thinking(I am trying to use malloc) but need more expansion.
Here is a better explaination of what I am trying to do.

struct rec1{
           char id;
           char item_code[4];
           char amount[6];
           }

struct rec2{
           char id;
           char item_code[4];
           }

struct rec3{
           char id;
           char item_code[4];
           char company[10];
           char address[30];
           }

These are the records and the unsorted binary file, unsort.dat,contains say 200 records.These have to be read from this file, sorted in ascending item code order,and then written to a new binary file, sorted.dat, in this order. My main problem is how to manipulate the records(structures).
Thanks for any further help.
0
 
LVL 22

Accepted Solution

by:
cookre earned 75 total points
ID: 2651561
Shoot, with only a couple hundred, you can loadem all into RAM, build a table of the item_codes with pointers to the record offsets, and rebuild real quick.

1) Get size of file
2) malloc space for file, e.g.:

struct thingy
{
char ItemCode[4];
long RecOffset;
} RecPtrs[500];

char *FileBuff;
char  FileName[256];
int   FileHandle;
long  FileSize;
long  CurrOff;
long  CharOff;
int   RecCtr;
....
strcpy(FileName,"blahblah");
FileHandle=open(FileName,_O_BINARY|_O_RDWR);
if (FileHandle==<ErrorValueForYourSys>)
   {
   oopsie stuff
   }
FileSize=_filelength(FileHandle);
FileBuff=malloc((size_t) FileSize);
CharOff=0;
CurrOff=0;
RecCtr=0;
....
3) Step through FileBuff one record at a time.  Use CurrOff as the offset into FileBuff of the 'next' record to look at and identify the end of the record by CR/LF or LF (as appropriate for your machine).  Keep CurrOff at the beginning of the record and move CharOff, looking for the end of line marker.  Use RecCtr to count records.  On any given record, save its' item_code into the RecCtr'th instance of RecPtrs.ItemCode and save CurrOff into the RecCtr'th instance of RecPtrs.RecOffset.

(In case you've never use pointers like FileBuff before, the k'th character in the area pointed to by FileBuf is *FileBuf+k.

Step to the next record by adding 2 (or 1, depending on the size of you end of line marker) to CharOff and putting that into CurrOff.  Don't forget to increment RecCtr.

4) Be lazy and use qsort to sort RecPtrs.

5) Re-write the file one record at a time, getting the offset in FileBuff to the k'th record from the k'th RecOffset.

Although scanning for the end of line markers during the rebuild is simple enough, you could avoid that in several ways:
a) add an end of record offset in RecPtrs
b) add a record length to RecPtrs
Both of these could be implemented easily by using CharOff as an offset from CurrOff.


Are we getting closer?
0
 

Author Comment

by:spark60
ID: 2654698
Hi cookre,
   I think this is what I am looking for. Let me try to interpret to see if I am understanding your explanation.

Open my file and determine the size.
Allocate block of memory which is sizeof file and let Filebuff point to it.
Loop
{
   CurrOff = Filebuff
   RecPtrs[RecCtr].RecOffset = CurrOff
   
   read the record from file.
   Strcpy(RecPtrs[RecCtr].ItemCode,item_code)
   Put the record in memory.(Filebuff has now moved to start of next record)
   
   RecCtr ++
}
until end of file.

This should of transferred the records into memory and set up an array of pointers to point to them.

Please let me know if this is right or wrong. For the moment I will try to code up to this point.
Thank you for your help.
0
 

Author Comment

by:spark60
ID: 2659394
ntdragon:Thanks for the comment.I have tried linked lists but only as one and not three.This may be another option.

cookre:I think I understand your idea now.I will try to code this coming week
to see if I can solve problem and will post comment.

Thanks for help.
0
 

Author Comment

by:spark60
ID: 2670260
Hi cookre,
  Have solved provlem - the key was constructing an array of pointers to the records in the buffer.
 Many thanks for your help.
0

Featured Post

What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

Join & Write a Comment

Preface I don't like visual development tools that are supposed to write a program for me. Even if it is Xcode and I can use Interface Builder. Yes, it is a perfect tool and has helped me a lot, mainly, in the beginning, when my programs were small…
This tutorial is posted by Aaron Wojnowski, administrator at SDKExpert.net.  To view more iPhone tutorials, visit www.sdkexpert.net. This is a very simple tutorial on finding the user's current location easily. In this tutorial, you will learn ho…
Video by: Grant
The goal of this video is to provide viewers with basic examples to understand and use while-loops in the C programming language.
The goal of this video is to provide viewers with basic examples to understand opening and reading files in the C programming language.

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now