for ssh without password, are both ways correct

That's how you do it. It's not two different ways, you just shared each machines public keys with each other. Instead of one way, you setup bi-directional.

technically the private keys should stay on the system that generates them... Hence the term "private keys"

Hi mugojava, first of all, I only need one-way, that is, from machine A to machine B. Also, please pay attention to that for way 1, it is sending the public key while for way 2, it is sending the private key. Would you please give me your thought? Thanks!

@scott_silva, so are you saying only the first way is the right way? I only need one direction, that is from Machine A to Machine B.

There are two ways to facilitate access from server a to server B' services One option is tge direct path connecting from server a to server b including requisite local tunnels to the services on b that need to accessed from A. The other option is the connection is initiated from server b to server A with remote tunnels which provide user/s on server A access to services on server b.

The ssh connection itself is a one way connection meaning the user initiating the connection can do things within that session it can not be overtaken/used........ By others.

Then why for AWS, it gives you a key (private key?) in order to connect to their server? @scott_silva

I misread your #2.  I would use method one. I have not tried method 2. Generally in rsa based access you generate and share the public key with machines you want to access. Not the other way around. I'm not even sure that works. I'll have to try it.

Both of those ways will work. It doesn't matter where you generate the private/public key pair. As long as a machine is set up with the public key, you can use the private key to gain access to it. The main thing to remember is after generating the pair, and distributing the public key, you will want to limit access to your private key.

"Then why for AWS, it gives you a key (private key?) in order to connect to their server? @scott_silva "

Just because someone does it doesn't make it right... The original premise of public/private key encryption was that you never shared the private key because then someone can pretend to be you...

Why it is called "public/private"

In the case of AWS, you get the private key because you are the controlling entity... The key is generated for you, but technically you would be generating the keys from your access system to administer your server... Therefore you get the private key.

The private key belongs on the client side(Machine A)

The public key belongs on the server(Machine B)

As to where you generate your public/private keys it can be done with either Machine. If  you generate the public/private keys on Machine A then copy the public key to Machine B. If you generate your private/public keys on Machine B then copy the private key to Machine A. You don't need to generate pairs on both machines.

If you have several client machines it's advisable that each have its own private/public pair for security reasons. For example, if one of your client devices was stolen you can pull the corresponding public key from the server without affecting other client machines. In the end this would mean each client machine has a unique private key and the corresponding public keys would be added to the server.

AWS likely provides you the private key to the public key they setup on the VM server's ssh authorized_keys. Not sure they do not have password authentication.

Here is a paragraph that I got from Amazon: "A key pair consists of a public key that AWS stores, and a private key file that you store. Together, they allow you to connect to your instance securely. For Windows AMIs, the private key file is required to obtain the password used to log into your instance. For Linux AMIs, the private key file allows you to securely SSH into your instance." So it looks like we are using the private key to SSH into the AWS instance. So it is method 2. Am I right?

Arnold, when you setup the server, it will let you download the private key. So does it mean that instead of using your own private key generated on your own machine, AWS generated the key pair for you, put the public key on the server and let you use the private key to login?

Yes.
I think it is a measure of security; with this setup they do not allow password based authentication. Without the key, no access, you can not guess the key. you can not brute force the key. etc.