RS232 connection

No. Without hardware, software would have to handle the encryption. And even then, the data would have to be encrypted BEFORE transmission over the wired. So unless both sides have everything needed to handle such a task, it's not going to be possible. Hardware solutions at least will ensure you of that, because the burden of encryption is removed from the actual endpoints.

Certainly.  For example, believe it or not, some people still have modems connected to their PC's. When those folks establish a VPN connection to work or whatever, their data is being encrypted... without hardware.

So yes, it is possible.

Thank you for your answers. I would like to apologize for not elaborating the scenario. Actually, they want to have a software solution to eliminate the use of the hardware (SE-6600R Serial Data Encrypter) that is being used for encryption. But, I don't have any idea which software application to use or could make this work.

What devices are communicating and what is being used to communicate now? I am going to assume 2 PCs, that I hope both have Ethernet ports on them. However, if your requirement is solely around a serial cable itself, you are probably not really in luck. Encrypting data before transmission and decrypting afterward provides some protection in terms of the data itself, but does not secure the actually communication channel.

Here is an application that you can take a look at: Virtual Serial Port Redirector