This article will cover how to manually take image files in Cloud Files and build them to a new Cloud Server. This will ONLY work for Linux. I don’t have a clue how to make this work on Windows :-p
There can be several reasons why you would want to do this. Maybe you want to manually move from a US datacenter to a UK datacenter. Maybe you have an account and your boss/co-worker/friend has an account and you want to share images. Whatever the reason, these are the steps to make it work.
Credit where credit is due: The idea for this was originally published at http://failverse.com/manually-creating-a-cloud-server-from-a-cloud-files-image/ (Thanks Jordan and Dewey). My article will just cover doing it using curl instead of wget, and some of the potential pitfalls not covered in their article.
So here goes! First things first, you will need to start up a new stock server in the receiving account that is the EXACT same image as the server that the image was built from. For example, if the image in Cloud Files was originally taken from a server running CentOS 5.5, you will build a stock image that is running CentOS 5.5 in the receiving account. Login to the new server you built.
Make a backup of the new server’s /etc directory. You will need this later:
cp -a /etc /etc.bak
If necessary, install curl (Some distros of linux come with it, others don’t).
Authenticate to the Cloud Files Account where the image is stored:
curl -D - -H "X-Auth-User: YourUserName" -H "X-Auth-Key: YourAPIKey" https://auth.api.rackspacecloud.com/v1.0
After you run that, it will spit out a list of names and values, like this:
HTTP/1.1 204 No Content Date: Wed, 30 Mar 2011 04:15:28 GMT Server: Apache/2.2.3 (Mosso Engineering) X-Storage-Url: https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_6f597497-4986-44ea-9081-1234567890 X-Storage-Token: 63ea9670-c80f-402d-9657-1234567890 X-CDN-Management-Url: https://cdn1.clouddrive.com/v1/MossoCloudFS_6f597497-4986-44ea-9081-68b8ee123456 X-Auth-Token: 63ea9670-c80f-402d-9657-c59bdb123456 X-Server-Management-Url: https://servers.api.rackspacecloud.com/v1.0/123456 Content-Length: 0 Connection: close Content-Type: application/octet-stream
We care about X-Storage-URL (That is where the image files are stored) and X-Storage-Token (This is your authentication token that lets you actually download stuff). Now let’s see a list of all of the image files in the account. Replace your Storage Token and your URL below. Don’t forget the /cloudservers at the end of the URL.
curl -H "X-Storage-Token: 63ea9670-c80f-402d-9657-1234567890" https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_6f597497-4986-44ea-9081-1234567890/cloudservers SpotImage_20110307_202634_cloudserver657824.tar.gz.0 SpotImage_20110307_202634_cloudserver657824.yml TestCopy1_20100910_193142_cloudserver330148.tar.gz.0 TestCopy1_20100910_193142_cloudserver330148.yml delweb1ssl_20110326_054323_cloudserver710521.tar.gz.0 delweb1ssl_20110326_054323_cloudserver710521.tar.gz.1 delweb1ssl_20110326_054323_cloudserver710521.yml
As you can see above, there are several files associated with each image. All of the data is stored in the .tar.gz files. The .yml file is a configuration file that for this article we don’t care about. You will see that some of the images have more than one .tar.gz file. This happens when the image is larger than 5GB and it gets chunked into multiple objects in Cloud Files. We will assume that we are working with a chunked image because that will make it just a little bit harder.
Let’s grab the delweb1ssl image. Grab the first chunk like this:
cd / curl -H "X-Storage-Token: 63ea9670-c80f-402d-9657-1234567890" https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_6f597497-4986-44ea-9081-1234567890/cloudservers/delweb1ssl_20110326_054323_cloudserver710521.tar.gz.0 > dlimage.tar.gz.0
This can be up to 5GB, so it may take a little while. Next up, download the 2nd chunk (and then third, fourth, etc)
curl -H "X-Storage-Token: 63ea9670-c80f-402d-9657-1234567890" https://storage101.dfw1.clouddrive.com/v1/MossoCloudFS_6f597497-4986-44ea-9081-1234567890/cloudservers/delweb1ssl_20110326_054323_cloudserver710521.tar.gz.1 > dlimage.tar.gz.1
Note that we are just changing what file we are getting and what we are calling it locally. Do this for as many .tar.gz files as there are in the account.
Now that we have all of the images downloaded, cat them together to make one big image
cat dlimage.tar.gz* > myimage.tar.gz rm -f dlimage.tar.gz*
If the new server isn’t large enough to hold the stock image, the downloaded images from Cloud Files, AND the concatenated image, you may run out of disk space. For this reason, you might want to just start with a huge 8GB (320GB Hard Drive) or 16GB (640GB Hard Drive) server and downsize after you are done with this.
Now that we have the one big image, we need to extract that out onto the filesystem. More than likely, you will need the newest version of tar to have the –hard-dereference option available. Your choices are to either download tar and install it from source, or grab a fully compiled version of tar here. (Thanks again Jordan). We’ll use the compiled version because it is just easier.
wget http://joshprewitt.com/scripts/static_tar chmod +x static_tar ./static_tar --strip-components=2 --hard-dereference -xpf myimage.tar.gz -C /
This can take awhile.
Remember when we backed up /etc above? (You did that, right?) Now we will want to bring that back in. However, is we just completely overwrite the /etc directory that we just extracted we will lose things like our users, groups, iptables, etc because they will be overwritten with the default values. To make sure we always have the /etc directory from the tar available, save that as another backup directory:
cp -a /etc /etc.tar
Ok, now we have 3 etc directories:
/etc = The version off of the backup
/etc.bak = The stock image /etc directory with all defaults
/etc.tar = A backup copy of the etc directory from the tarball
From here you can manually bring over your network config files and anything else necessary from the default image, but I prefer to just replace the entire /etc directory with stock data and bring over what I need from the /etc.tar directory later.
cp -a /etc.bak/* /etc/ cp /etc.tar/passwd /etc.tar/shadow /etc.tar/group /etc/.
Depending on what distro you are running, you will also want to grab your iptables rules from /etc.tar In anything RHEL based it would be:
cp /etc.tar/sysconfig/iptables /etc/sysconfig/iptables
That’s pretty much it. Cross your fingers, reboot and see if it comes back up!