It works like this:

Party A encrypts using his/her private key, and then using Party B's public key. The fact that Party A uses his/her private key, means that only Party A's public key can decrypt the message. Party B's public key used by Party A, means that only Party B can read that message.

When party B receives the message, they use Party A's public key and then Party B's private key to read the message. The chances of cracking this algorithm by bruteforce are quite slim (unless you are a surveillance agency and its your job to do this), but as computing power progresses, it will become possible.

There are different algorithms. RSA is used mostly for this purpose.

A hashing algo is a one-way function that generates a checksum of a message/file. This checksum is wrapped with the message and sent to the recipient. If the sender encrypts the hash using his private key, then the message is said to have been digitally signed by the sender using his private key. This can be verified by the receiver using the public key of sender to get to the hash. After that, the receiver generates a hash of the message and compares both. If they match, the message was received unaltered. There are many hashing algorithms such as MD5, SHA1, SHA-256.

You can read more about cryptography in Bruce Schneier's excellent book "Applied Cryptography". There is also a presentation on this at http://www.mirrors.wiretapped.net/security/info/books/crypto-tutorial/