vsftpd under Centos - how to jail some user to their home dirs, let others see some dirs, let others see all

I am building a LINUX box to replace and old legacy FTP server. I am using Centos and vsftpd.  The box is dedicated to FTP with only SSH and webmin running on it and only those ports open in the firewall. Webmin and SSH will be IP limited to in house or other trusted IPs. vsftpd will have to be able to accept connections from any IP, unless blocked. No anonymous access allowed.  At some point, we may add FTPS (FTP with SSL) to the box for users (the vsftpd is ready for that). I will be using SFTP (FTP over SSH) to move files back and forth, but that is just for me - not the users.

What I need to do is to allow some users access to just their home folders (and sub-folders under that):

These would be like:

user1  /home/user1
user2 /home/user2
user3 /home/user3

Then I need some users to have their own homes, but be able to navigate to some of the other users folders. Like:

user4 /home/user4  but can download, upload, rename, and delete in the folders of user1 and user3 but not user2 .

The I need some user that can see all the /home ftp folders, but can't back out to the OS or / folders

user5 /home/user5 but can see /home/user1-4 and delete, rename, upload, and download from those.

Finally I need a user (me) that can access any of those folders and back out to do whatever I need on the rest of the box.

I have it set up to chroot the users and have the user list on so I can exclude some users (like me) so I can go elsewhere.  I am just wondering what is best way to set this up. Do I just make the folks like "user4" a member of the user1 and user3 groups?  That is OK, but when I add more users and I have to back and add that group to all the new user groups. Since I don't want the lowest level users to see each others stuff, I can't make then a part of the same group.

Another though would be to put the homes of user1-3 in a sub folder:

user1 /home/lowlevelusers/user1
user2 /home/lowlevelusers/user2
user3 /home/lowlevelusers/user3

Then I could make the home of user4 /home/lowlevelusers and then they could see the user 1-4 folders. I think I would still have to make them part of groups or they could not delete or download the files. Maybe that is the key - I DO make a lowleveluser group and make users 1-3 members but jail them to their own folders, then make user4 a member of their own group AND lowleverusers.  Then when we add more low level users we just make them part of the lowlevelusers group (only) and then user 4 automatically gets to see the new folder and access what is placed there?

Then user5's home could be just /home and make them part of user4's group and also lowlevelusers group?  Then they could see and do it all, but could not back out into the rest of the box?

I saw mention of a user config file (/etc/vsftp_user/conf/user4) and that would seem to work to be a help, but I need to do it as simple as possible so someone using webmin could do this and not have to SSH in and mess with files at a low level.

My config file so far is pasted below. If anything related OR unrelated to the user levels needs to be added, changed, or deleted then please let me know.

anonymous_enable=NO
local_enable=YES
local_umask=022
anon_upload_enable=NO
#anon_mkdir_write_enable=YES
dirmessage_enable=YES
xferlog_enable=YES
connect_from_port_20=YES
xferlog_std_format=YES
idle_session_timeout=900
#data_connection_timeout=120
#nopriv_user=ftpsecure
#async_abor_enable=YES
ftpd_banner=Welcome to our FTP server.
#deny_email_enable=YES
#banned_email_file=/etc/vsftpd.banned_emails
chroot_list_enable=YES
chroot_list_file=/etc/vsftpd.chroot_list
chroot_local_user=YES
#ls_recurse_enable=YES
pam_service_name=vsftpd
userlist_enable=YES
#enable for standalone mode
listen=YES
tcp_wrappers=YES
ascii_download_enable=YES
#When enabled, ASCII mode data transfers will be honoured on downloads.
ascii_upload_enable=YES