Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17

x
?
Solved

Compress DBF files ?

Posted on 2010-08-14
5
Medium Priority
?
1,478 Views
Last Modified: 2013-12-01
Hello gurus,

I am trying to make a backup on the web of my application...|Unfortunately the data is quite big...one of the tables I have is around 2GB. Is there a way to compress it "more" than by using winrar or winzip ?

I could on the other hand keep track of the "changed/new" records and take only those...
Unfortunately the data files are under foxpro 2.5 DOS and there is no way to know what records where changed/added...
any hint there ?
0
Comment
Question by:ammounpierre
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
5 Comments
 
LVL 14

Expert Comment

by:tusharkanvinde
ID: 33436925
No. That should be the best compression you will be able to get. Foxpro database do not have any recording of changes made. If one of the tables is 2GB is size, it is very near to the maximum size a table can be. It would be better to do something regarding that like changing your backend database or splitting the big file.
0
 
LVL 33

Expert Comment

by:pgnatyuk
ID: 33437003
How I remember, in FoxPro there is a special command: pack. It will pack the table. FoxPro does not delete the record in the DBF but marks it as deleted. So this pack command removes the deleted records.
Another procedure that will reduce the size is re-indexing. It is a long operation, but fully re-created indexes will reduce the database. So you will have less data to compress.
Ignore my comment, if you already do all these things before the compressing.


0
 

Author Comment

by:ammounpierre
ID: 33437267
Yep. All is done...
Coming to my question n#2 is there a "simple" way to know the changed/deleted/added records ?
In that case the procedure would take only the new data...
0
 
LVL 33

Accepted Solution

by:
pgnatyuk earned 1000 total points
ID: 33437323
Before the pack command call you know about the deleted records.
Added records are in the end of the table.
Nothing about edited.

In ForPro you have count command.

Actually, all depends on your database: maybe new records have a field with the timestamp, same about the edited records. Maybe you have an operational journal, transaction table, etc. There is no a built-in tool controlling that in the DBF-files.

0
 
LVL 30

Assisted Solution

by:Olaf Doschke
Olaf Doschke earned 1000 total points
ID: 33438071
I'd be more worried about the 2GB file anyway. You may save some bytes, even if the table is packed, if it has a memo and you SET BLOCKSIZE TO 1, if that was available in FPD. It minimizes fpt file blocks for memos. So it can shorten the fpt file, if you have memos.

Like pgnatyuk said deleted records are marked deleted before a pack, so first saving ids of all deleted records, then doing a pack and doing that at each backup, you know the changes in that respect, storing the max id for each backup you know the new records.

That is, if your records have an id, an integer primary key field. I know free fox tables and win or dos 2.x tables have no primaray key index, still you can have such a unique integer.

You can use checksums to find out changes. You do a checksum per record after a backup and at the next backup compute the checksum of each record to find changed ones when their checksum changes. Either you store checksums in a separate table or a new checksum field.

In vfp DBC you couls use insert, update and delete triggers happening in these events to log changes when they are made, update a checksum or even mirror data to a secondary database.

Bsides all that a complete backup is a safer and easier to restore backup anyway. Even if GB is quite large.

Bye, Olaf.
0

Featured Post

Will your db performance match your db growth?

In Percona’s white paper “Performance at Scale: Keeping Your Database on Its Toes,” we take a high-level approach to what you need to think about when planning for database scalability.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

In its Critical Driver Installation Order document, Intel stresses that this sequence be followed:   1       Fresh Operating System Installation 2       Latest Service Pack or Patch 3       Intel® Chipset Software Installation Utility (INF Utili…
1. Start | Settings | Printers and Faxes. 2. Go to the File Menu. 3. Select Server Properties. 4. Select the Drivers tab. 5. Select the printer you need information about. 6. Once again select Properties. You should get a list of the files requ…
Finding and deleting duplicate (picture) files can be a time consuming task. My wife and I, our three kids and their families all share one dilemma: Managing our pictures. Between desktops, laptops, phones, tablets, and cameras; over the last decade…
Monitoring a network: how to monitor network services and why? Michael Kulchisky, MCSE, MCSA, MCP, VTSP, VSP, CCSP outlines the philosophy behind service monitoring and why a handshake validation is critical in network monitoring. Software utilized …

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question