oddszone
asked on
Compressing a simple Serial message
I have an application that needs to send a serial message that represents a colour grid of say 50*50 cells
Each cell in the grid can be 0 (black) ....1 (green)... etc.... upto 7 (red) but most cells are black.
Is there a simple and quick way to compress the string. I thought possibly detecting a black cell then sending number of consecutive back cells after that.
Sample ASCII data:
"3740000000000000000000000 0000000000 0000000000 0167674001 1000000000 0000000000 0000000000 0000000000 1127511300 0000000000 0000000000 0000000000 0000000120 2303100000 0000000000 0000000000 0000000000 0000012213 2210000000 0000000000 0000000000 0000000000 0001233211 0000000000 0000000000 0000000000 0000000000 0003131100 0000000000 0000000000 0000000000 0000000000 4442100000 0000000000 0000000000 0000000000 0000000022 4110000000 0000000000 0000000000 0000000000 0000022411 0000000000 0000000000 0000000000 0000000000 0011144200 0000000000 0000000000 0000000000 0000000000 1303300000 0000000000 0000000000 0000000000 0000000003 1330000000 0000000000 0000000000 0000000000 0000002243 1000000000 0000000000 0000000000 0000000000 0000124510 0000000000 0000000000 0000000000 0000000000 0013630000 0000000000 0000000000 0000000000 0000000000 1152000000 0000000000 0000000000 0000000000 0000000123 1000000000 0000000000 0000000000 0000000000 0000001200 0000000000 0000000000 0000000000 0000000000 0000100000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 00"
So
1st cell is colour 3,
2nd cell is colour 7
3rd cell is colour 4
The next 43 cells are black.
I have to do this as serial comms is at it's limits and messages are too long to send. Increasing baud rate is not an option.
Each cell in the grid can be 0 (black) ....1 (green)... etc.... upto 7 (red) but most cells are black.
Is there a simple and quick way to compress the string. I thought possibly detecting a black cell then sending number of consecutive back cells after that.
Sample ASCII data:
"3740000000000000000000000
So
1st cell is colour 3,
2nd cell is colour 7
3rd cell is colour 4
The next 43 cells are black.
I have to do this as serial comms is at it's limits and messages are too long to send. Increasing baud rate is not an option.
ASKER
That doesn't seem to correctly decompress as some of message is chopped. It might be me though I'll check.
BUT, I also need to make the same routine in native "C" code for the embedded side of the project so a library is not the ideal solution as I don't know compression algoryhm in your example.
BUT, I also need to make the same routine in native "C" code for the embedded side of the project so a library is not the ideal solution as I don't know compression algoryhm in your example.
I sent the link to you because you posted your question in C# zone and the code in the link is C#.
The algorythm basically uses a GZipStream object (http://msdn.microsoft.com/en-us/library/system.io.compression.gzipstream.aspx) that is a .Net framework native object that interally uses the Zip algorythm to compress Streams (in this case Streams containing text strings).
The algorythm basically uses a GZipStream object (http://msdn.microsoft.com/en-us/library/system.io.compression.gzipstream.aspx) that is a .Net framework native object that interally uses the Zip algorythm to compress Streams (in this case Streams containing text strings).
I've tested the code with your data (in "Sample ASCII Data") with the following statistics:
Original data string (the one in your example): 1927 characters.
Compressed data string: 360 characters (82% compression aprox).
Once decompressed, the string data is exactly the same than the original.
Hope that helps.
Original data string (the one in your example): 1927 characters.
Compressed data string: 360 characters (82% compression aprox).
Once decompressed, the string data is exactly the same than the original.
Hope that helps.
ASKER
Yes I should have mentioned the C thing sorry.
I don't think the code in the link is working 100% either.
+++++++++++++
UPDATE
======
I need a solution that could easily be converted to 'C' for the embedded side
Thanks
I don't think the code in the link is working 100% either.
+++++++++++++
UPDATE
======
I need a solution that could easily be converted to 'C' for the embedded side
Thanks
ASKER
Just checked and C# code DOES work but my serial couldn't cope with the data so I thought it was wrong :(
If you only have 8 colors, 0-7, then you really only need 4 bits to represent those potential choices - using ASCII characters is sending 4 bits than are really needed. For example, to send the four colors 3, 7, 4, and 0 you can bit-shift 3 left by 4 bits and OR it with 7. Then do the same thing for 4 and 0. Send the 2-byte string "w@" instead of the 4-byte string "3740". Will cut your data size in half.
...if you ever need to send an odd number of colors, you'd need to implement some kind of header to the message to indicate how many colors follow (in the case of an odd number of cells, you'd need to indicate that only the first 4 bits of the last byte should be used, and discard the least significant 4).
Actually, you only need 3 bits. You could take the three least snignificant bits from 7 color values, and re-combine them to make 3 7-bit bytes. This would reduce data size by less than half, and keep each resulting value under 128 so it can still be represented by ASCII.
ASKER
I've worked out if I replace
"00000000000000000000" with "9"
and
"000000000" with "8"
I get
37498000167674001190000000 0011275113 9000000000 1202303198 1221322198 1233211980 0031311980 0444219800 0224119800 2241198001 1144298001 3033980000 3133980002 2431980001 2451980001 3639800001 1529800012 3198000001 2980000001 9999999999 9999999999 9999999999 9998800000 00
Which is 249 characters.
"00000000000000000000" with "9"
and
"000000000" with "8"
I get
37498000167674001190000000
Which is 249 characters.
ASKER
Or perhaps
replace consecutive "0"s with "8xx8" where xx is 0 count.
This gives
37484381676740011839811275 1138398120 2303184081 2213221840 8123321184 3831311842 8444218438 2241184282 2411842811 1442842813 0338448313 3843822431 8438124518 4381363844 8115284381 2318458128 4681810178
which is only 187 chars.
Would this pack/unpack code be easy write in C anc C#?
replace consecutive "0"s with "8xx8" where xx is 0 count.
This gives
37484381676740011839811275
which is only 187 chars.
Would this pack/unpack code be easy write in C anc C#?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
http://madskristensen.net/post/Compress-and-decompress-strings-in-C.aspx
Hope that helps.