What is bitwise, and where does its purpose lie?

Also take a look at http://www.cprogramming.com/tutorial/bitwise_operators.html ("Bitwise Operators in C and C++: A Tutorial") - the purpose should become clear.

A bit more in-depth: http://www.codeproject.com/KB/cpp/bitbashing.aspx ("An introduction to bitwise operators"), which portraits the purpose as "The beauty of having bitwise operators is that you can use a BYTE, WORD or DWORD as a small array or structure. Using bitwise operators you can check or set the values of individual bits or even a group of bits.".

>> I'm interested in looking into the encryption side of programming.  I still
>> don't quite understand what shifting a byte stands to accomplish.

Well, a maybe-not-too-complex example on how bitwise operations come in handy is the MD5 Algorithm: http://en.wikipedia.org/wiki/MD5#Algorithm

I'll just give you a quick rundown on bits and why we use these strange "bitwise" operators. If I repeat anything the other experts have said, I apologize, as I don't have time to read all of their responses and adjust mine accordingly.
 
Quite often, we need to pass 'messages' from one object to another, and those 'messages' contain values that can only mean a set amount of things. For example, when we want to open a file, there are only a few set ways we can do this: write-only, read-only, or read-write (with some more optional stuff, such as whether or not to lock the file, etc). We could make an object that contained 4 boolean variables (one for read, write, read-write, and lock), but remember that a Boolean doesn't actually take up a single bit, as RAM doesn't allow such a layout of bits.
 
Also quite often, we need to drill messages from one place to another hundreds of times a second, and every bit counts, so we want to compact these messages to make them as lightweight as possible. This is when bit flags come in. We can combine the above 4 bits into a numeric value, and treat them as binary instead of numbers. For example, suppose you are the method who is acting on opening files, and you receive a message like this:
 
00000110
 
As you also know, we usually work in bytes (8 bits), so I padded on 5 extra zeros to the left. The 6th bit indicates read access, the 7th indicates write, and the last indicates lock. 0 means false, 1 means true, of course. So, if the 6th bit is on, we need read access. If the 6th AND 7th bits are on, we need read-write, and if the 8th bit is on, we need to lock the file. Easy, right? The above equates to 6 if you convert it from binary into a numeric value (which is what we, as humans, work with usually). There are six bitwise operators in most (if not all) langauges:
  Bitwise AND ( & )
Bitwise OR ( | )
Bitwise XOR ( ^ )
Bitwise Left Shift ( << )
Bitwise Right Shift ( >> )
Bitwise COMPLEMENT (Or NOT) ( ~ )

 
Each of these operators have unique properties, but they all have one thing in common: they operate on binary values. The first five operate on two binary values, while the last has a single operand, and thus only operates on one value. The first operator, Bitwise AND ( & ), will return true, or 1, if two corresponding bits are both true. For example, look at the following:
 
   1
& 1
1
 
Because those two bits were both true, the AND operation returned another true. However, the operators don't have to work one bit at a time. You can operate on a whole byte, or a whole short (16 bits), a whole 32-bit int, and so on. Usually, you only use this with booleans and non-floating point values. Take this for example (I'm just using 8 bits because I'm lazy, heh):
 
    0 1 1 0 0 1 1 0
& 1 0 1 1 0 1 1 1
    0 0 1 0 0 1 1 0
 
Now that you have an idea as to how bitwise operators work, I'll just explain the rest in general.
 
Bitwise OR returns true if either of the two bits are true. So, the same two binary values (above) OR'ed instead of AND'ed looks like this:
 

   0 1 1 0 0 1 1 0
|  1 0 1 1 0 1 1 1  
  1 1 1 1 0 1 1 1
 
The last of the three in this 'group' (that act using two like binary values) is XOR (Exclusive OR). It's the strangest; it returns true if one of corresponding bits are true, but the other is false. If both are true or both are false, it returns false.
 
    0 1 1 0 0 1 1 0
^  1 0 1 1 0 1 1 1  
    1 1 0 1 0 0 0 1
 
The next two related functions are left shift and right shift. It literally shifts bits to the left or right according to the number of bits. These two act strangely on signed/unsigned numbers, so I suggest searching this on Bing or Google or whatever you like to use.
 
Complement is the easiest. It just reverses the bits on each bit, so 0 becomes 1 and 1 becomes 0.
 
So how does this even converge into your programming career?! Well, these operators transform binary values according to certain 'algorithms'. So, let's say you have a set of flags and they mean seperate things. A certain bit being on means one thing, while the next means something else, etc. etc. How do we 'extract' these values and tell if the bits are on or off?! And how do we turn certain bits on and off? Usually, we use AND, OR, and XOR for manipulating flags.
 
To turn a certain flag off, you can take the value you wish to turn the flag off on and & it by the bit you want to turn off. You have to think in binary, but write code in decimal, so consider the following boolean expression:
 
(someFlags | 2) > someFlags
 
Assuming someFlags is a byte (remember, binary is laid out as 128 64 32 16 8 4 2 1 for a byte), when we OR it by 2, if the value gets larger, we know that the 2 bit was off in the original value. If OR'ing it keeps the value the same, we know that the 2 bit was already on.
 
If you need more help on this, I suggest going through the links other experts have posted. I think my post is quite long enough. However, if you'd like me to elaborate further, I would be happy to :)
 
Hope it helps,
Nate