MigrateToNewServer

Example 3.x Migration

One server running:

• RT 3.2.1
• Apache 1.x
• mod_perl 1.x
• Mysql 4.x

to this

Mysql replication pair running:

• RT 3.4.5
• AT 1.2.3
• mod_perl 2.x
• apache 2.x
• Mysql 5.x

Note that I had no custom overlays to worry about - all of the customisation was in Scrips and therefore in the database. Filesystem-based changes need scrutinising closely to see if they still make sense against the new version of the file they are overlaying.

There isn't anything very interesting in the non-RT software upgrades. I built the replication pair, installed all the software, installed RT 3.4.5 and AT 1.2.3 etc. and tested it worked. The task was then to migrate the RT database from the old server to the new server. This is the interesting bit and should be of use to anyone having to do a migration upgrade to new servers, replicated or not and with or without any software upgrades (other that RT ...). This is basically very easy:

• Stop users connecting to the RT system (stop apache and sendmail)
• Take a mysqldump of the rt3 database. I used this:

/usr/local/mysql/bin/mysqldump --user=rt_user –-password=<password> --opt --skip-lock-tables
--single-transaction --databases rt3 > /home/mysql/rt.sql

• /Note:/ I found that with the above command my binary attachments would get corrupted with this mysqldump command. Instead I used this one with the default-character-set binary setting:

/usr/local/mysql/bin/mysqldump --user=rt_user –-password=<password> --opt --skip-lock-tables
 --single-transaction --default-character-set=binary --databases rt3 > /home/mysql/rt.sql

I was moving from a MySQL 4 to a MySQL 5 database so I couldn't use the "--add-drop-database" option with the mysqldump command (doesn't exist in the version 4 command). This just means an extra step below if you already have an rt3 database on your new boxe(s) (if, for example, you've been testing etc.)

• Stop Mysql on the old server - you don't want any updates to the db at this point since it's now about to be moved ...
• Copy this dump file to your new server (or master server in case of a replication pair)
• You will have to drop any existing rt3 database on the new box manually if you used the command above. The command for this is:

mysqladmin --user=root --password=<password> drop rt3

• Import the db onto the new server:

mysql --user=root --password=<password> < /home/mysql/rt.sql

This can take a while. For a 1.3Gb dump file, it took about 8 minutes for me. It was nice to see that MySQL replication was never more than a few seconds behind during this. Not bad for such a heavy INSERT database load.

• Update the database schema. This is the only really interesting part of the whole thing. Note I've already installed RT 3.4.5 at this point but I'm going to run the schema upgrade scripts that come with the distribution to update the database for use with RT 3.4.5. This is nice and safe because this is a copy - you already have a backup on your old machine. This is very simple - first check the RT README that comes with the distribution to see which schema updates you need - how to find out is in the upgrading section. Here, I'm upgrading from 3.2.1 to 3.4.5 so have to do two schema updates. Assume RT is in /usr/local/rt. You'll need to be inside the RT source installer tree for this part (if you dont' have it around any more, get it again, make sure it's the same version you installed!)

for action in schema acl insert; do /usr/local/rt/sbin/rt-setup-database --dba root
--dba-password=<password> --action $action --datadir etc/upgrade/3.3.0; done

for action in schema acl insert; do /usr/local/rt/sbin/rt-setup-database --dba root
--dba-password=<password> --action $action --datadir etc/upgrade/3.3.11; done

The very first schema update takes a little while and if you are using replication, you will see your slave get up to several minutes behind the master. This is normal for a schema update.

Later, I updated to 3.6.x - there is one more schema update you need which came with 3.5.1. So, from inside the 3.6.x install tree, do:

/usr/local/rt/sbin/rt-setup-database --dba root --dba-password <password> --action insert --datadir etc/upgrade/3.5.1

I also wanted to update the database for the AT Asset Tracking add-on. Again, AT is already installed, I'm just needing to run the schema update scripts that comes with it so, cd into the AT distribution tree and do:

/usr/local/rt/sbin/at-setup-database --action init --datadir etc/AssetTracker/
--dba root --dba-password=<password>

That's pretty much it. You should be able to start up apache and sendmail on the new server and test it. All we are really doing here is importing the old database to a new server which already has the new RT installed and then running a subset the RT installer routine again to upgrade the database schema.

Two issues I found doing this:

• Permissions

The only thing I had a little trouble with afterwards going from 3.2.1 to 3.4.5 were the global permissions:

SeeCustomField
ModifyCustomField
ShowSavedSearches
ShowConfigTab

When then going to 3.6 later, I found that the permissions on the AT main directory (if you're using AT) were changed and I had to pt them back to 755.

Some users/groups who were able to do these things in 3.2.1 were no longer able to do them after the migration - ten seconds in the Global Config tab fixes it - just add these permissions to the appropriate groups etc.

• glibc problem

I noticed that /tmp on the new server was filling up with temporary directories and that apache error logs contained many lines like the following:

Can't remove directory /tmp/oOhmXQuGk2 (Directory not empty) at
/opt/ActivePerl-5.8/lib/5.8.7/File/Temp.pm line 898

After investigation, this turned out to be the readdir() perl function interface to the system call of the same name returning all filenames as empty strings. This meant that the File::Temp module would fail to remove the temp files in these temp directotories so that when rmdir()was called, the directories were not empty and would not be removed.

After more investigation, this was found to be a known bug when mod_perl and perl are linked against different versions of the glibc library which have incompatible struct dirent structures.

Make sure perl and mod_perl are linked against the same glibc version.

Upgrades / Migration from RT2.x to RT3.x (RT2 to RT3)

• Download from the latest version of RT-Extension-RT2toRT3-xxx.xx.tar.gz from http://www.cpan.org/authors/id/F/FA/FALCONE/
• Copy to both the OLD and NEW RT servers, if they are different machines. Otherwise just edit the lib statements as below
• Untar/gz into a directory and run perl Makefile.PL, make, make install
• View the README to see the latest data for your install
• Edit the use lib statements in both scripts to use the relevant RT libraries for both the old and new server
  • rt-2.0-to-dumpfile (Old server)
  • dumpfile-to-rt-3.0 (New server)
• Create a directory OLDTIX on a drive with enough space to hold the entire ticket database in raw text format (estimate 1.5x binary database size for worst case)
• Run rt-2.0-to-dumpfile OLDTIX and the script will export all your tickets. This took about 2 seconds per ticket on an old server, so prepare for an overnight run on a slow machine or large database.
• Mount/scp OLDTIX onto the new RT server
• Edit dumpfile-to-rt-3.0 and note the following variables which you may want to change (as usual in Perl, 0 == false, 1 == true):

my $import_global_metadata = 1;
my $import_users           = 1;
my $import_groups          = 1;
my $import_queues          = 1;
my $import_tickets         = 1;
my $import_links           = 1;
my $merge_on_import        = 1;

• If using MySQL:
  • Find the largest attachment in the OLD database. If using Mysql on the NEW instance, change max_allowed_packet in my.cnf to be larger than the size of the largest attachment.

[mysqld]
max_allowed_packet=50M

• • Restart mysql
• Cross your fingers and run dumpfile-to-rt-3.0 OLDTIX. Other options are detailed in the README.
• This worked to migrate an ancient Solaris 8 / PostgreSQL / RT2.0.15 install to RHEL4/MySQL/RT3.6.3.

Another Migration Script

Read about it here: http://www.gossamer-threads.com/lists/rt/users/87539