I have an existing system that stores a measurement as an integer of 32th of an inch (1 in = 32). The numbers are only valid for these increments. I need to be able to convert this value to its millimeter equivalent and back. The only allowed values for millimeters are whole numbers. If I use a simple factor of 32/25.4 and then round the result I get a consistent mm. But then I need to be able to convert back to inches and get the same value. The problem is that due to rounding, the result may have been rounded up or down to the next valid integer. And then if I convert back it may not have the original value. The factor of 32/25.4 = 1.25984252.

We could live with a change, except for specific common values (mainly whole inch or at least foot values). For example 3 feet or 36 inches = 2688, which when converted to mm = 914 (913.6 rounded up). The problem is that when converting 914 back (914 * 1.25984252) we get 2689 (3' 1/32"), not the original 2688. While I can see this is inevitable, given that there are more 32th per inch then millimeters. I am looking for a way to adjust the result to avoid some common values.

When I graph the rounding issue, I see a pattern that makes me think I can recreate it on the reverse conversion enough to handle common values.

Is it possible to devise a fudge factor that can keep the whole inch values from being effect by the rounding issue? At least for the first 20'? As the issue is predictable, it should be possible to devise a way to compensate enough to always be able to convert from whole inches to mm and back to there original stored values. I just can't seem to get something to work in all cases.

Here are some standard measurements that I need to have reproducible results for:

Measurement Stored mm rounded Refactored Rounded Delta

1 3/8" 44 35 35 44.0945 44 0

1 3/4" 56 44.45 44 55.4331 55 1

2" 64 51 51 64.2520 64 0

3' 1152 914.40 914 1151.4961 1151 1

7' 2688 2133.60 2134 2688.5039 2689 -1

3'6" 1344 1066.80 1067 1344.2520 1344 0

4' 1536 1219.20 1219 1535.7480 1536 0

4'6" 1728 1371.60 1372 1728.5039 1729 -1

5' 1920 1524.00 1524 1920.0000 1920 0

6' 2304 1828.80 1829 2304.2520 2304 0

6'8" 2560 2032.00 2032 2560.0000 2560 0

6'10" 2624 2082.80 2083 2624.2520 2624 0

7' 2688 2133.60 2134 2688.5039 2689 -1

7'2" 2752 2184.40 2184 2751.4961 2751 1

7'6" 2880 2286.00 2286 2880.0000 2880 0

7'10" 3008 2387.60 2388 3008.5039 3009 -1

8' 3072 2438.40 2438 3071.4961 3071 1

8'6" 3264 2590.80 2591 3264.2520 3264 0

9' 3456 2743.20 2743 3455.7480 3456 0

10' 3840 3048.00 3048 3840.0000 3840 0

I need the delta to be zero for these values.