cPanel migration + rsync to complete the migration

Using CPanel increases likelihood new server will be broken in subtle ways.

The first thing I do when working with a project which uses CPanel is to move to a non-CPanel setup. If you take this approach your migration will be faster + problem free.

The big problem moving CPanel system, especially between CPanel versions, is there's no clear way to verify anything is working.

You just have to attempt (note I said attempt) to issue CPanel actions which cause all CPanel config files to recompile + see what works + what's broken, then try to fix what's broken.

Doing this with CPanel, verses a non-Panel system, will take a very long time + you or whoever you hire will almost always miss crucial problems, because that's just the nature of CPanel. No way around this problem.

Aside: Here's my sequence for escaping CPanel or any other Panel system, I've used many times.

1) On source CPanel machine, install lxd-p2c which converts an entire physical machine to an LXD container.

2) On target machine setup an Ubuntu Bionic machine running latest SNAP version of LXD, as the SNAP version is the only version receiving updates at this point.

I use Ubuntu Bionic as it's an LTS version with 5 years of updates + a very recent Kernel version, unlike RedHat/CentOS Distros running very old Kernels.

3) Run lxd-p2c cloning source machine to LXD container on target machine.

4) Verify container is working + change IPs of all sites to new container IP + retire source machine.

5) At this point, begin setting up additional LXD containers all running Ubuntu Bionic which will house either single sites or collections of sites or possibly any API custom code.

For APIs, I always run these in their own container, as poorly written custom code is very sensitive to upgrades, hence should live alone in a partitioned ecosystem.

6) After all sites + custom code are migrated out of original container, archive the original container + destroy the runtime version of container.

Hmmm

I don't know where to start with my comment but either way, I was looking for a solution for my EXACT situation, not for an opinion on using or not using cPanel.

I need and have to use cPanel for the purpose that I want. There is no other way around it, and from a non-linux expert I have to say that I love cPanel. It hardly causes any problems and it works.

I am still looking for an opinion from someone who can help me with a rsync operation after the migration is completed.

Thank you

Just keep in mind, anytime you move from CPanel to CPanel, be sure to set aside enough budget/time to handle problems.

Then just rsync, you CPanel files from one machine to another + start fixing problems.

Nothing easy about CPanel migration. No short cuts. Just many sleepless nights.

Just follow the rsync instructions provide by CPanel support.

Big Note: The only direction about what rsync to run should come from CPanel support. Only CPanel support really understands how to do this correctly. They will provide you with exact rsync syntax for your migration.

Note: If all this geek speak sounds like gibberish, likely best to hire someone to do this migration for you.

Be prepared for false starts + a potentially long timeline for fixing problems that arise during this process.

Hi David,

Appreciate all your efforts but I guess that all your comments so far are going in the opposite direction of my initial expectations.

I work with cPanel for 10+ years and I was using my hosting company's technical staff to do the migrations for me or to instruct me how to do it. Across the years I became used to these tasks and many others. I have conducted at least 40-50 server migrations in the past, all from cPanel to cPanel and only had 1 major issue (in an account that was about 1TB), but this was not an issue from cPanel, it's all I can say.

cPanel is amazingly good for my needs. I do not intend to go away from it, at all.

Right now, on this specific server, I do not have a support team to give me a rsync command and I would prefer someone externally to cPanel as they are a bit reluctant to give these sensitive commands. However, based on the answers I got so far, I think this will be the only possible way.

I honestly do not understand what you are trying to achieve with your answers. It seems that you are overcomplicating things (lxd-p2c, Ubuntu Bionic and so on). The task I am asking is possible with rsync. This is sufficient and what I am asking for. I will do a test in a temporary environment server to see if it works obviously.

FYI, there are no false starts: the migration was successful and I got 50 GB migrated from one server to another within 24 hours. The logs say it all went well and the disk space used is as well.
There are no long timelines as well. All we need is to run a command and let the rsync work and send the remaining data (which I assume it is less than 100 MB of data).
Fixing problems? No need. cPanel is here exactly to avoid any kind of problems. Once the account is migrated, what problems can there be? It even changed the IP addresses from the DNS entries automatically and it is all 100% correct, as it should be.

Awesome!

Sounds like you've had good experience with this process, so just follow the steps you normally follow to do your migration.

The basic rsync command is:
rsync <option> <from location> <to location>

The normal set of options is just "-avz", which is:
a = archive mode (preserve everything possible, like timestamps, permissions, etc)
v = verbose
z = use compression (good for server-to-server transfers unless you're transferring files that are already compressed, like JPEGs or ZIPs, in which case it's wasted CPU)

So it's now:
rsync -avz <from> <to>

You'd identify the folder you want to rsync FROM locally (you'd run this on the original/source machine), let's say it's /home/data:

rsync -avz /home/data <to>

Finally, the <to> can be a remote SSH target, so it can be broken down like this:
rsync -avz /home/data <remote_username@remote_server:remote_path>

Final example:

rsync -avz /home/data root@newserver01:/home/data

Select allOpen in new window

When you run that, rsync will do some checking on both sides to figure out what files are different / missing, and it will push changes from the local server (the old server) to the remote (new) server. Just update the example to use the paths you want to sync up, with the right username and hostname.

Also, if you only want to push MISSING files (files that only exist on the local server) and NOT files that exist on both sides but are different, then there's a switch for that, too. There's actually quite a few switches/options, so check rsync --help in case you have specific requirements.

Thank you, gr8gonzo

This is exactly the help that I was looking for.
Your explanation was clear and to the point.

I will have to sort out the paths now because I think that /home/data/ is just a generic example. I only need to have the email folders synced basically so I will look into those rsync switches and will try to come up with the command that I need.

Highly appreciated for the info.

I agree to gr8gonzo, be aware that part of the data in cPanel is in a database and will not get copied this way.
(If you need all data an export/import MAY work if table structures are still the same... this part of copy should be done with due diligence though).
And you can run rsync several times if there are still changes during the copy, the resulting time to copy should decrease as fewer data needs to be transfered.

At the risk of stating the obvious here is how we do our CPanel migrations (albeit with far less data than you are working with)

1. We setup the DNS on the new server to point to the current server as well as setting up the MX records to do the same
2. We do the nameserver change (if that is required)
3. We wait for propagation of the nameserver change to happen
4. When we are ready to migrate we put a maintenance page up on the old site and prevent any further data changes
5. We synchronise files and database data to the new site.
6. We update the DNS to point back to the new server and set a TTL to 60
7. After a reasonable amount of time we set the TTL back to 1800 or 14400 or whatever the default was.

This works for our CPanel migrations - putting it out there in case it is useful.

Hi Julian,

We prefer the other way around because 40-50 GB of data may take between 24-48 hours depending on the servers' locations.
So... we usually transfer everything first and the DNS are changed only when the migration is completed. The TTL changes are valid and usefull, I changed them to 3600 for my case.

Right now, I only need to run the rsync command to make sure that all email folders are properly synced. I am contacting cPanel support now since I didn't have any luck here.

Thank you.

And what exactly didn't work from gr8gonzo's  comment?

I dont think you understood what I was saying.

The DNS change is in two parts - the first is to mitigate the propagation delay by switching nameservers (assuming you are doing this).

By pointing the new domain to the old and then switching the NS - you can transfer the files and then when done put up the Migration page and do a final synch of anything that has changed. Changing the DNS to point back to the new domain with a short TTL means that the switch to the new domain is very quick.

However, I understand how the answers are not inline with your requirements. A bit of digging did find these references

https://bobcares.com/blog/cpanel-to-cpanel-migration/

Referred from here

https://forums.cpanel.net/threads/upgrading-migrating-to-a-new-cpanel-server.293282/#post1234792

Hopefully these will be useful.