Moving files from one ubuntu server to another
Hello Experts!
Novice, maker-of-rookie-mistakes here! I implemented a wiki using VirtualBox (Ubuntu server 14.04) and BitNami MediaWiki (using Rubystack). However, I did not notice that my virtual machine was running out of space. Now, at 99%+ capacity, the machine's capabilities are limited. I created a new VBox with twice as much storage and now I know where to keep an eye on the capacity.
I have succeeded in using scp to copy the entire rubystack folder to the new machine. However, it does not preserve the permissions. Rsync has an option to preserve permissions but I get permission denied when I try to write to the correct location on the new machine, presumably because sudo on the source machine does not have any effect on the destination machine. When I rsync the rubystack folder into the home directory, even when I use -a, it makes the admin user the owner, recursively, instead of root, etc.
Short of going through both machines to fix the permissions, what are my options to move this wiki over to the new machine?
I am also open to expanding the size of the existing machine, but my initial research made it seem much more complicated than "simply" copying the files to a new machine. Maybe someone else has some feedback regarding this process?
Novice, maker-of-rookie-mistakes here! I implemented a wiki using VirtualBox (Ubuntu server 14.04) and BitNami MediaWiki (using Rubystack). However, I did not notice that my virtual machine was running out of space. Now, at 99%+ capacity, the machine's capabilities are limited. I created a new VBox with twice as much storage and now I know where to keep an eye on the capacity.
I have succeeded in using scp to copy the entire rubystack folder to the new machine. However, it does not preserve the permissions. Rsync has an option to preserve permissions but I get permission denied when I try to write to the correct location on the new machine, presumably because sudo on the source machine does not have any effect on the destination machine. When I rsync the rubystack folder into the home directory, even when I use -a, it makes the admin user the owner, recursively, instead of root, etc.
Short of going through both machines to fix the permissions, what are my options to move this wiki over to the new machine?
I am also open to expanding the size of the existing machine, but my initial research made it seem much more complicated than "simply" copying the files to a new machine. Maybe someone else has some feedback regarding this process?
ASKER
Yes, well, my question lacks detail because I have no idea what I'm doing. Yet. I will have to do some research just to figure out what you are saying. ;-)
I'm certain that the users do not have identical uid/gid, just the name is the same, lets say, "admin". Is it the uid/gid that the machine really uses to identify the user and it's permissions? So, if the uid/gid is the same, a machine might think the user is the same?
With the BitNami setup, the permissions are generally not something on which you need to focus, it's kind of like linux-light. The users and permissions are set automatically, so you don't have to directly give apache permission to access mysql, etc. For me it's been a good way to delve into ubuntu and other technologies that I don't get to touch very often (professionally I am a database developer).
My original machine is so full that I can no longer start the services needed for the wiki to run.
When I use rsync to copy the files, which is everything under and including the rubyscript folder, I can't copy them directly into /opt which is where they have to go - using BitNami, the only way to restore is to put everything in the same exact path. So I put them into the /home/admin folder and then can copy them to /opt, but then admin becomes the owner of everything. When I use scp, I can copy into the correct location, but the owner and group, and/or the permissions, are not always correct. For example, instead of mysql, root ends up as the owner of the mysql folder and everything under it. Scp does not seem to have an option to preserve the permissions. I have not looked into SFTP yet.
I'm certain that the users do not have identical uid/gid, just the name is the same, lets say, "admin". Is it the uid/gid that the machine really uses to identify the user and it's permissions? So, if the uid/gid is the same, a machine might think the user is the same?
With the BitNami setup, the permissions are generally not something on which you need to focus, it's kind of like linux-light. The users and permissions are set automatically, so you don't have to directly give apache permission to access mysql, etc. For me it's been a good way to delve into ubuntu and other technologies that I don't get to touch very often (professionally I am a database developer).
My original machine is so full that I can no longer start the services needed for the wiki to run.
When I use rsync to copy the files, which is everything under and including the rubyscript folder, I can't copy them directly into /opt which is where they have to go - using BitNami, the only way to restore is to put everything in the same exact path. So I put them into the /home/admin folder and then can copy them to /opt, but then admin becomes the owner of everything. When I use scp, I can copy into the correct location, but the owner and group, and/or the permissions, are not always correct. For example, instead of mysql, root ends up as the owner of the mysql folder and everything under it. Scp does not seem to have an option to preserve the permissions. I have not looked into SFTP yet.
chown -RH username:group path
This will recursively reset the ownership user and group the -H should be used it will prevent the ownership change from following symlinks.
I.e. Making changes to unintended files. Within the file path you have a reference (symbolic link) to /somefolder without the -H option, the ownership of /somedir and its contents will also have their ownership changed, and if there are any symlinks there ..........
It will go on and on.
You should be able to boot the VM using single user directive
When you say you no longer have space to boot are you including the host system in that?
This will recursively reset the ownership user and group the -H should be used it will prevent the ownership change from following symlinks.
I.e. Making changes to unintended files. Within the file path you have a reference (symbolic link) to /somefolder without the -H option, the ownership of /somedir and its contents will also have their ownership changed, and if there are any symlinks there ..........
It will go on and on.
You should be able to boot the VM using single user directive
When you say you no longer have space to boot are you including the host system in that?
ASKER
Ah, ok, I did not know about the -H option; that's very good to know! No, the host is fine, it's just the vm that is full, so the new machine is on the same host. I can boot it ok, just can't start the services to run the wiki, specifically the service that accesses mysql won't start.
It turns out that the uid/gid of the admin users that I created when I first created the machines is the same on both machines. Does this make it any easier to transfer the files?
It turns out that the uid/gid of the admin users that I created when I first created the machines is the same on both machines. Does this make it any easier to transfer the files?
ASKER
i tried sftp, which apparently puts files only in the home directory of the user that is logging into the destination server. If root is the owner/group of my destination folder, I get permission denied on the source server; if my admin user is the owner/group of my destination folder, I can put the files on the destination server but the owner/group for all of them is my admin user instead of the existing owner/group, even when using the -p option.
This makes one think they should log into the destination server as root, but of course enabling root as a user is a bad idea. (I also tried to sftp into the source server from the destination server but either there isn't enough room to establish the connection or sftp is not installed, because I just get a connection time-out.)
Short of enabling root as a user, how do I get files over to another server without the ownership or permissions changing? I can't be the only person having this problem; what do people do when they migrate web stuff to their production environment?
This makes one think they should log into the destination server as root, but of course enabling root as a user is a bad idea. (I also tried to sftp into the source server from the destination server but either there isn't enough room to establish the connection or sftp is not installed, because I just get a connection time-out.)
Short of enabling root as a user, how do I get files over to another server without the ownership or permissions changing? I can't be the only person having this problem; what do people do when they migrate web stuff to their production environment?
You can Change both ownership and group ownership if files when you gave elevated rights using chown user:group.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Using tar might just work, because there is a shared folder attached to the vm but since it's stored on the host, it won't take up any space in the vm. I'll let you know!
ASKER
Oh my gosh, using tar totally worked. It's been over a month that I have been trying to recover this thing and I am sooooo relieved. Thank you so much!!!
I had tried to zip everything previously but zip was not installed and the installation crapped out (of course, because there was no more room on the machine), so that's when I started looking at copying the files without zipping them, but moving them to the shared folder on windows first completely screwed up the permissions and ownership. That's when I tried copying them directly. But it appears the only way to maintain all of the permissions and ownership is to zip or tar.
Now I get to go move my husband's wiki at his work; easy peasy now!
I had tried to zip everything previously but zip was not installed and the installation crapped out (of course, because there was no more room on the machine), so that's when I started looking at copying the files without zipping them, but moving them to the shared folder on windows first completely screwed up the permissions and ownership. That's when I tried copying them directly. But it appears the only way to maintain all of the permissions and ownership is to zip or tar.
Now I get to go move my husband's wiki at his work; easy peasy now!
On linux/Unix gzip and bzip2 are some of the compression tools.
tar often has an argument that can be used that will tar and gzip the archive.
Tar cfz myarchiv.tgz directory_to_be_archived/
Glad I could help.
tar often has an argument that can be used that will tar and gzip the archive.
Tar cfz myarchiv.tgz directory_to_be_archived/
Glad I could help.
Getfacl can be used to get the current permissions written to a file and than use that to reapply it on the new server.
The permissions of which you talk are these relate to Apache being able to read them?
Depending on what the issue is, I.e. What capacity issues you have.
An option could be to allocate another disk to the VM which will then be used to copy data that expands and overlay the new drive where the data .....
I.e. /var/www/hrml needs to now be 30Gb to allow for 50% growth.
Your question lacks much of the detail to understand what your issue is.
In linux/unix space can be added on path references.
/ 10 GB 90% use with 50% in /somedir
Allocating a new drive of size 20GB will show up as /dev/sdc
Partition this drive as a single partition /dev/sdc1
mount /dev/sdc1 /mnt
cd /somedir
find . | cpio -pdvmu /mnt #this will clone the data to make sure no changes are made during the copy, you might as well stop services.
mv /somedir /somedir.old
mkdir /somedir
umount /mnt
mount /dev/sdc1 /somedir
ls -l / | grep somedir
Make sure to use chown/chmod to match ownership and permissions on the two. Once you confirm they work.
/somedir.old can be deleted to free up the space on /.