Does the idea of dealing with bits scare or confuse you? Does it seem like a waste of time in an age where we all have terabytes of storage? If so, you're missing out on one of the core tools in every professional programmer's toolbox. Learn how to do amazing, modern things with bits!
In the past year, I've had two different developers come to me for help with problems that were actually quite simple. The underlying issue in both cases was that the developers had either never learned or had forgotten everything about working with the most basic unit of computing - the bit.
A lot of programming languages have gotten so high-level and easy to use that it's become easy to forget about things like bits. Why worry about those when your boss is asking you to fix a bug with e-mail address validation?
So you fix the bug and then your boss asks you to enforce IP filters so that only people with IP addresses within 220.127.116.11/24 can hit certain web pages, which prompts you to adopt a confused look. You've seen that kind of IP address notation before and you know it has something to do with masking, so you start looking for an explanation. When you find that people are talking about bitwise operators you get that dread in your stomach because dealing with bits is like a foreign language to you.
Here are three pieces of good news:
#1. Bits are a lot easier to understand than a foreign language.
#2. You probably already know a lot of the concepts but don't realize
you know them.
#3. Understanding bits will make you a better programmer.
Let's dive in!
Imagine you're watching your favorite sport on TV and you look at the scoreboard to see what your team's score is. It looks like your team has 122 points. Then you realize that the number "122" is split onto 3 separate, large LCD screens: "1" "2" and "2". Wow, you think - what a waste of money. Each one of those LCD screens probably costs several hundred dollars, and they could easily just show the full number "122" on a single LCD screen.
Yet, this is precisely the kind of wasteful thing that many programmers do every day, and I'll give a more relevant example. Let's say that an average programmer named Matt is storing the ages of people in a database. Matt creates his database and creates a column that holds up to 3 characters (since most people will not live beyond 999 years old). All of his data seems to be stored perfectly well in the database and he can get it back out easily later on. He thinks everything is fine.
What he doesn't realize is that he's using up to 3 bytes of data for information that can be stored in 1
byte of data. That's up to 2 extra bytes of wasted data. It doesn't seem like much, but let's say that 5 years later, he has 10 million contacts in his database. Now that waste has multiplied into 20,000,000 bytes of wasted data! Now let's say that he has a backup system which stores up to 30 days of full backups. He now is storing 20 million wasted bytes * 30 days = 600,000,000 bytes of wasted data altogether - all because he didn't use the proper data type at the beginning.
Hopefully, most programmers will use an integer for an age column, but I've seen many people create 15-character text fields on their databases to store IP addresses like "18.104.22.168" - and that's just IPv4 addresses. We're entering the age of IPv6 addresses like "fe80:abcd:ef01:2345:6789:0abc:def0:1234", and I can already see people creating 39-character text fields to store that data. Programmers are doing this because they don't understand bits well enough to know how an IPv4 address can actually fit into 4 bytes instead of 15.
Imagine Matt creating a database that stores access logs of a web server, and it stores the IP addresses as 15-character strings. Five years later, with 30 days of backups, we're talking about many gigabytes of wasted space and bandwidth.
Bits, Basements, and Bulbs
So let's talk about how Matt can improve his situation. The first step is understanding what a single byte can do - and you might already know some or most of this.
A single byte can hold any number between 0 and 255. This is because a single byte is made up of 8 bits, and each bit is either a 0 (off) or a 1 (on).
An easier way to think about this is to imagine going into a completely dark basement. There are eight light switches on the wall, so you flip the one on the right, and a small lamp in the corner with one tiny lightbulb turns on. It makes the room a little brighter, but you want more brightness, so you flip the next switch (the second one from the right), and this time, a different lamp turns on. This new lamp has two lightbulbs in it, so now you have three lightbulbs turned on in the room.
You flip on the third light switch from the right, and an even bigger lamp turns on - this one has 4
lightbulbs in it, so now you have a total of 7
lightbulbs turned on.
You start realizing that the number of bulbs is doubling on each lamp as you move from the right to the left. So you flip the left-most light switch and suddenly the room gets really
bright because a huge chandelier with (yes, you guessed it) 128 lightbulbs suddenly turns on. So now you have a total of 135 lightbulbs turned on.
You turn on all of the eight light switches, and you end up with every lamp and chandelier in the room turned on, with a total of 255 light bulbs (and a really
This is precisely how a computer thinks about numbers. A byte is like a room with 8 light switches that goes from a total value of 0 (no switches turned on) all the way up to 255 (all switched turned on). So a value of zero looks like 00000000 while a value of 255 looks like 11111111. Or an easier way to look at it might be:
A byte value of zero has all bits set to off:
0 0 0 0 0 0 0 0
___ ___ ___ ___ ___ ___ ___ ___
128 64 32 16 8 4 2 1
A byte value of 73 is made up of the bits for 64, 8, and 1 being turned on:
0 1 0 0 1 0 0 1
___ ___ ___ ___ ___ ___ ___ ___
128 64 32 16 8 4 2 1
A byte value of 255 has all bits turned on:
1 1 1 1 1 1 1 1
___ ___ ___ ___ ___ ___ ___ ___
128 64 32 16 8 4 2 1
Simple enough, right? So what happens if we need to store the number 256?
We're Gonna Need a Bigger Boat
Great question - I'm glad you asked it, and the simple answer is: use another byte! The harder question is how to use the two bytes together. After all, each byte can only hold a value that goes up to 255.
The answer is simply a matter of instructing the computer to treat both bytes as one, big number. If you think about it, we already do this every day with numbers. For example, the number 24 is technically two digits - 2 and 4. But when we read them together, the 2 isn't just a 2 - it represents 20. So when a computer needs to deal with numbers that are larger than a single byte, it simply needs to be told that "there are 2 bytes here, and they should be considered as one big number."
Or Maybe Just Bigger Bits....
In programming languages, a single byte has 8 bits, so it's an 8-bit number. A two byte number is a 16-bit number and is commonly referred to as a "short." When a computer is told to treat 2 bytes as a 16-bit number, it keeps the same pattern of doubling numbers, so the values of one byte suddenly become much larger. It looks like this:
_____ _____ _____ _____ _____ _____ _____ _____ ___ ___ ___ ___ ___ ___ ___ ___
32768 16384 8192 4096 2048 1024 512 256 128 64 32 16 8 4 2 1
Byte #1 Byte #2
So by being told to use two bytes, a computer can handle numbers between 0 and 65,535.
So there's really no difference in how the data is stored - every piece of data out there is just a series of 8-bit bytes. However, with a little bit of programming instructions, a computer can be told to sometimes use more than one byte in order to hold larger numbers.
This same concept extends into 32-bit numbers and 64-bit numbers. A 32-bit number, commonly known as a "long", is made up of four bytes and can hold numbers up to 4,294,967,295. A 64-bit number uses eight bytes and holds values up to 18,446,744,073,709,551,615.
Bazillions: An IP Address Love Story
Now we're entering an age of 128-bit computing, and the first most common appplication of 128-bit numbers is the IPv6 standard. While IPv4, which is actually a 32-bit number (again, a "long"), allowed up to 4.2 billion IP addresses in the address, IPv6 addresses are 128-bit numbers. I'll allow you to think about how many IP addresses that are allowed with IPv6.
Some people might look at an IP address and say, "That's not a 32-bit number." What they are missing is that the whole "###.###.###.###" syntax is simply a visual representation of a long number. Each ### is simply the value of each byte. So "22.214.171.124" is actually:
Byte #1 = 72
Byte #2 = 123
Byte #3 = 45
Byte #4 = 67
If we actually take those 4 bytes and tell the computer to treat them as one big number, we get 1,216,032,067. Likewise, you can "convert" 1,216,032,067 back into that IP address at any time. So in reality, Average Matt could store all his IPv4 IP addresses as "long" numbers, taking up only 4 bytes of space each time, instead of up to whopping 15 bytes of text per each one.
An additional benefit to storing them as long numbers is that you can use math with numbers. Need to find out if 126.96.36.199 is between 188.8.131.52 and 184.108.40.206? You can either break up the strings into pieces and compare each set of numbers, or you can simply convert all three to long numbers and do simple less-than and greater-than comparisons.
Just a Little Bit of Negativity
One point of interest - a computer often needs to work with negative numbers. So pretty much any number can be told to be treated as signed or unsigned. So far, we've been talking about unsigned numbers (or numbers that don't have any possibility of having a negative "sign" in front of them), so an 8-bit number goes from 0 to 255, a 16-bit number goes from 0 to 65,535, and so on.
When the computer is told to treat a number as a signed number, the maximum value is basically split in half, and you get half of it as negative, and half as positive. So a signed 8-bit number can go from -128 to 127. A signed 16-bit number can go from -32,768 to 32,767. One bit is used to indicate whether the number is negative or positive.
Some languages, like PHP, will default to using signed numbers all the time. So when you try to tell it to deal with a really large 32-bit number, like 3,000,000,000, it can only go up to 2,147,483,647 before it has nowhere else to go. So it will then "wrap around". So if you tell PHP to convert "127.255.255.255" to a long, it'll correctly tell you the result is 2147483647, which is the maximum value for a signed 32-bit number. However, if you just add 1 and tell it to convert "220.127.116.11" to a long, it will come back with -2147483648.
So it can be helpful to understand what your programming language is doing for you whenever you ask it to deal with different types of numbers.
Bit Maps and Other Creative Uses for Bits
Moving on, it can often be useful to work with bits. There are numerous applications and creative ways you can use all 8 bits of a byte, and they don't always have to be math-related problems. For example, let's say that you wanted to log a complex, daily schedule of times that you were working, in 5-minute blocks, like this:
My Billable Time
12:05 AM - 12:15 AM
12:30 AM - 12:35 AM
1:00 AM - 1:35 AM
Imagine you had a hundred entries in your log. Now, you could
store the start and stop time for every entry. Or you could simply use bits to represent 5-minute blocks throughout the day:
Bit #1 = 12:00 AM - 12:05 AM
Bit #2 = 12:05 AM - 12:10 AM
Bit #3 = 12:10 AM - 12:15 AM
So in the above example's first three entries, you could have a "bit map" that looks like this:
The first bit is 0 because you didn't work from 12:00 AM to 12:05 AM. The next two bits are 1, indicating that you worked from 12:05 AM to 12:10 AM and from 12:10 AM to 12:15 AM, and so on.
So in 24 bits (3 bytes), we've described someone's activity in 5-minute increments for the first 2 hours of the day! You could do the same thing for an entire 24-hour period in only 36 bytes. For the purposes of comparison, that's the same amount of bytes it takes to store the following part of this paragraph: "So in 24 bits (3 bytes), we've descr". So in the same few bits that would be used to store that fragment of a sentence, you could store the entirety of someone’s billable time with 5-minute accuracy!
Of course, the results wouldn't really make any sense as numbers. You'd have all sorts of "random" numbers, but the total values of each byte would have no purpose here. You'd be using each byte to simply record eight 5-minute blocks of time.
This, This, and This, But Not That
Another common way of using bits is to allow someone to mix and match permissions. File permissions on a UNIX system can be pretty simple. You can read a file, write to a file, and/or execute a file. So what if you want someone to read and execute a file, but not
write a file? Bits to the rescue!
The common permission set looks like this:
Bit value of 1 = Execute
Bit value of 2 = Write
Bit value of 4 = Read
So if you want full permissions, you end up with 7 (1 + 2 + 4). If you want just read and execute permissions, you'd use a permission of 5 (1 + 4). If you want read and write, but not execute, then you'd have a permission of 6 (2 + 4). You can store any combination of up to 8 different permissions in a single byte!
Because of all the unique applications here, most programming languages provide a set of functions and tools for dealing specifically with bits. These are usually called "bitwise operators" and they tend to have similar syntax between different languages. Usually you'll have tools for moving bits left or right (e.g. shifting a bit to the left look like going from 00000001 to 00000010, which is actually the same thing as multiplying a number by two). You'll also often have tools for flipping the bits (all 0s become 1s and all 1s become 0s), or comparing two different sets of bits (e.g. "show me all the 1 bits from value A that are also set in value B").
Being Bitwise with IP Address Masks
The operators are simple and there's only a handful of them, but they DO take a little bit of extra thinking to use efficiently. For example, bitwise operators turn a complex concept like an IP address netmask (like our earlier CIDR example of "18.104.22.168/24") into a very simple matter of comparing a bunch of 1s and 0s. It's simply a matter of finding out how someone else decided how they wanted to use the bits. In the CIDR example, the "/24" simply means "of the 32-bits that make up this IP address, keep the first 24 bits as they are, and then the remaining 8-bits define the range, from 0 to 255." So "22.214.171.124/24" literally means "keep the 71.23.45" part. Or in other words, "126.96.36.199 to 188.8.131.52".
Was this Helpful? Click a Button!
Hopefully this opens up some new ways of thinking about data and how you can make more effective use of every single byte of data. Thanks for reading and be sure to vote this article as helpful if you liked it.Copyright © 2015 - Jonathan Hilgeman. All Rights Reserved.