Upgrading RT 3.6.5 on Ubuntu to 3.8.7

Hello everyone! This might not help many people, but hopefully it'll keep someone from having to go through the rigemerole I went through in order to upgrade my system.

Here's the scenario. Our original RT server was running on Ubuntu 8.04 and RT was installed from the repositories as 3.6.5 for ease of setup. Also the documentation on setting it up for that specific environment was really well put together.

The issue is that now we're up to Ubuntu 10.10 and 3.8 is in the repositories. You can't upgrade the Ubuntu 8.04 system via APT because only 3.6 exists in the repositories. Thus the problem I ran into.

Assumptions

1. You have an existing 8.04 server (or earlier) with a working 3.6.5 (or earlier) RT install
2. You have (or can create) a Ubuntu 10.04 server
3. You have an internet connection and some time
4. You have a username and password for the RT Database (or root)
5. You have shell access to the server we're working with

Overview of Process

This document will be broken up into sections by step to give you a general overview of the process before we delve into the specifics. I would like to make sure you understand the steps we're taking as well as perform them. Below is the list of steps we will take from start to finish:

1. Setup 10.04 Server (Assuming one does not Exist)
2. Install Request Tracker 3.8 on 10.04 Server
3. Configure the RTDB database manually
4. Configure RT and Apache
5. Configure fetchmail for RT
6. Dump 3.6.5 database
7. Move 3.6.5 database to new server
8. import 3.6.5 database and perform upgrades
9. Upgrade database schema
10. Restart Apache and Get Working
11. Additional information

Setup a 10.04 Server

If you already have a 10.04 server setup somewhere, then you can skip this step, but if you're just starting the process you'll have to build one. Obviously you'll require the ISO which you can download from http://www.ubuntu.com. I always recommend building with a 64-bit image for scaleability, but you can use a x86 ISO if you so choose.

During your installation, you can choose most of the defaults or specify your partitions as you desire, but the big thing to remember is that you will want to select the LAMP Stack Packages as well as OpenSSH (Primarily so you can access it remotely)

Once the server is up and running, you'll want to test MySQL to make sure it's working. Only recommend doing this because I had a terrible time with my first few installations. For some reason, the daemon wouldn't start. Gave it a fresh install and voila everything worked.

To test this, you'll have to have the root password you setup during the install. Issue the following command:

mysql -u root -p

Enter the password you setup for the root user and if you see the following you're good to go.

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql>

Install Request Tracker 3.8 on 10.04 Server

Now that you have the LAMP stack installed and verified, you have mostly everything you'll need in order to get moving. First, we'll go ahead and run an update to make sure the packages we have are up-to-date

sudo apt-get update

Once that is complete, it can't hurt to go ahead and performa a system upgrade (Not a distro upgrade). This will make sure all of the installed code is up to date.

sudo apt-get upgrade

This may take a little bit, so you can feel free to go grab a cup of coffee or a sandwich, as it can take several minutes depending on your internet connection. Once it's complete, however, we will go ahead and install RT 3.8 with the following command

sudo apt-get install request-tracker3.8

request-tracker3.8 is a meta package containing all of the dependant child packages you will need for the base setup. (Note that this does not include RTFM. I will go over how to install RTFM at the end of the document)

You will notice that it will download a literal crap-load of packages, but when it asks you whether you want the packages or not, say Yes. If I remember correctly, it may be close to 100 megabytes of data, so it might take a little bit.

Once the download is complete, it will begin installing the packages and you will be prompted for two things:

1. Allow the system to handle RT_SiteConfig.pm permissions ---- Say Yes
2. Perform database setup with dbconfig-common --- SAY NO

It is very important to say no to the second question. The default installation for 3.8 is diferent than it was in 3.6 which defaulted with a mysql connection. If you allow it to install with dbconfig-common, it will setup a sqlite database and will render mysql somewhat useless.

For this reason we will setup the database (the basics) ourselves.

Configure the RT Database Manually

Don't worry, even if you've never touched a database before, I'll walk you through this so it won't be too painful. in fact, you won't really be doing much directly on the database unless you really want to.

Once you've got RT installed, you're going to want to navigate to the /usr/sbin directory and use a tool there called rt-setup-database.

cd /usr/sbin

./rt-setup-database --action init --dba root --prompt-for-dba-password

I want to explain a little about what this does. The rt-setup-database function can be used to create the database or initialize it. Here, the nomenclature is a little misleading because all "create" does is create an empty database called rtdb. The init function is what actually populates it with all of the required data. If you start with the init function, it will create the database and then fill it performing everything we need at one step.

I have bolded the root option in the code because you can use any other user that has access to the database if it exists. At this point though, honestly, it's just as easy to use root.

Once this process completes, your database will be completely setup, but we want to do one more thing where we will have to actually touch the database. RT will require a username and password with which to access the database, so we will create one.

First, let's get into the mysql command line the same as before:

mysql -u root -p

Now that we're in mysql we're going to create a user called rtuser with password wibble to access the database. Of course you can always change these out for whatever you would like. I would avoid using root here though, because if anyone was able to exploit security in the RT web interface, they would have access to all databases on your server.

grant all privileges on rtdb.* to 'rtuser'@'localhost' identified by 'wibble';

flush privileges;

As stated earlier, the bold sections can be changed out for your needs, rtuser being the username and wibble being the password. Run these commands one at a time (and don't forget the ";" and once done, you can type "quit" to exit mysql.

Now we're ready to start configuring RT itself.

Configuring RT and Apache

The only thing we need to directly touch / configure for RT here is the RT_SiteConfig.pm file which you may or may not remember from your last setup. There are two very important things to keep in mind here.

1. Your RT Name MUST be the same as the one you will import later (Your old 3.6.5 system)
2. Your Organization name MUST be the same as the one you will import later (Your old 3.6.5 system)

Let's say in your old RT_SiteConfig.pm file your RT name was "Request Tracker 3.6" and your Organization was "IT Support Co." The new system you're setting up now MUST also have the exact same RT names and organization names or your import of the database will not work.

To keep you from worrying about all of the crazy info you might need in you config.pm file, here is a sample one:

# This file was generated by running "update-rt-siteconfig-3.8".
#
# While local modifications will not be overwritten without permission,
# it is recommended the they are instead placed in
# /etc/request-tracker3.8/RT_SiteConfig.d
#
# Note that modifications to the RT_SiteConfig.d directory won't
# take effect until the update command mentioned above is run again.

# start /etc/request-tracker3.8/RT_SiteConfig.d/40-timezone
# dynamically find out the current timezone
my $zone = "UTC";
$zone=`/bin/cat /etc/timezone`
    if -f "/etc/timezone";
chomp $zone;
Set($Timezone, $zone);
# end   /etc/request-tracker3.8/RT_SiteConfig.d/40-timezone
# start /etc/request-tracker3.8/RT_SiteConfig.d/50-debconf
# THE BASICS:

Set($rtname, 'RT Name');
Set($Organization, 'Organization Name');

Set($CorrespondAddress , 'correspond@youraddress.com');
Set($CommentAddress , 'comment@youraddress.com');

# THE WEBSERVER:

Set($WebPath , "/rt");
Set($WebBaseURL , "http://localhost");

# end   /etc/request-tracker3.8/RT_SiteConfig.d/50-debconf
# start /etc/request-tracker3.8/RT_SiteConfig.d/51-dbconfig-common
# THE DATABASE:
# generated by dbconfig-common

# map from dbconfig-common database types to their names as known by RT
Set($DatabaseType, 'mysql');
Set($DatabaseHost, 'localhost');
Set($DatabasePort, );

Set($DatabaseUser , 'rtuser');
Set($DatabasePassword , 'wibble');
Set($DatabaseName, 'rtdb');
Set($PreferRichText, 1);

# end   /etc/request-tracker3.8/RT_SiteConfig.d/51-dbconfig-common
1;

As mentioned before you will substitute your RT Name and Organization name into the config file in the noted locations. If you're new to RT and are for some reason using this document as a "Getting started" reference, the Correspond Address will be whatever email address you want the user to see whenever an agent responds to a ticket. The Comment address is the one users will see when someone comments on a ticket. These are generally only seen on the inside.

I have bolded localhost because you can use localhost or the FQDN (Fully Qualified Domain Name) of your server, or the IP Address of your server, whichever is more convenient. I always use the FQDN, but that's because I have an internal domain.

Lastly you will designate the username and password RT will use to access the database. we have used rtuser and wibble in this case, but these can be substituted if you chose another username and password.

Now, we have to tell Apache to use RT. You will do this by pasting the following into /etc/apache2/sites-available/default

<VirtualHost *:80>

  ServerAdmin webmaster@localhost

        DocumentRoot /var/www/
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /var/www/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>

        ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
        <Directory "/usr/lib/cgi-bin">
                AllowOverride None
                Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
                Order allow,deny
                Allow from all
        </Directory>

        ErrorLog /var/log/apache2/error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog /var/log/apache2/access.log combined
        ServerSignature On

    Alias /doc/ "/usr/share/doc/"
    <Directory "/usr/share/doc/">
        Options Indexes MultiViews FollowSymLinks
        AllowOverride None
        Order deny,allow
        Deny from all
        Allow from 127.0.0.0/255.0.0.0 ::1/128
    </Directory>

Include "/etc/request-tracker3.8/apache2-modperl2.conf"
RedirectMatch ^/$ /rt/

</VirtualHost>

Now, before you restart apache you'll have to enable mod_re-write or you'll get an error from modperl2.conf. Do this with the following:

sudo a2enmod rewrite

You'll get a notice that it has been enabled and you can force-reload of Apache to enable it, but we're going to go a step farther and restart apache altogether.

sudo /etc/init.d/apache2 restart

Now, you should actually be able to browse to http://your.ip.address.here and you'll be redirected to the RT3.8 login page. The default is username root and password is password :)

If you just wanted to get a base RT3.8 install, you're there now, but have a few more steps to go to import our databases and get fetchmail up and running, but before we could do that, we had to get the base system for 3.8 up and running.

Configure Fetchmail for RT

This is the big deal, because honestly, a ticketing system is kind of broken if a user has to login to log a ticket. The purpose of fetchmail is to poll against a mail server to check for new emails and, when they come in, create a ticket out of them.

Now I had a crapton of problems getting this to work on 10.04, but was finally able to get it to work and hopefully it'll save you a lot of headaches.

First though, let's go ahead and install fetchmail

sudo apt-get install fetchmail

Once that's done, you'll want to create the file /etc/fetchmailrc with your preferred editor and enter the following:

set daemon 60
set invisible
set no bouncemail
set no syslog
set logfile /var/log/fetchmail.log

#support update - General
poll mail.yourdomain.com
protocol pop3
username "username" password "secret"
mda "/usr/bin/rt-mailgate --queue GENERAL --action correspond --url http://localhost/rt/"
no keep

#support comment - General
poll mail.yourdomain.com
protocol pop3
username "username" password "secret"
mda "/usr/bin/rt-mailgate --queue GENERAL --action comment --url http://localhost/rt/"
no keep

Here, you will want to replace mail.yourdomain.com with the address of the mailserver you will be polling and insert the username and password it will be using to access the account. Here, we have indicated that anything it finds in the update or comment sections will be applied to the GENERAL queue, but if you have other queues, you can substitute accordingly.

Now, by default, fetchmail will generate an error if you don't create the log file and change it's permissions, so we will do the following:

sudo touch /var/log/fetchmail.log

chown fetchmail /var/log/fetchmail.log

After that, you can start it with the following:

/etc/init.d/fetchmail restart

Dump your 3.6.5 Database

In this step, we leave the newly functional 10.04 server to go back to your older 8.04 ubuntu system running RT 3.6.5. Here, we are going to do a dump of the entire RT database, which we will be moving to the new device. We will do the dump with the following

mysqldump -u root --password="secret" rtdb > ~/rtbackup.sql

Note that here we are specifying the root user in the database and supplying the root pasword in place of secret. If you have another user and pass that have access to the database, you can susbtitute those. Since the default database name of the 3.6.5 implementation was rtdb we are specifying that database.

This will generate a sequel file in your home directory called rtbackup.sql. You may have to be patient depending on how large your database is. Ours was approimately 1.8G and took close to a half hour to backup. During this period, the web interface will not be accessible.

Move the Database to your New (10.04) Server

Once it is complete, we will SCP the file to the newer Ubuntu 10.04 system

scp ~/rtbackup.sql user@ip.address.of.system:/home/user

Here you must designate the user who will be logging in via ssh as user. The IP Address of the new system should be inserted here. If you've never used SCP before, here is a sample if copying the code to a machine on 192.168.2.4 with username timmy.

scp ~/rtdbackup.sql timmy@192.168.100.15:/home/timmy

This may take a while, depending on the size of the database, but SCP will give you an estimation of the time required and the status during the transfer.

Once it's complete we'll have to hop back onto the 10.04 machine.

Import 3.6.5 Database and Perform Upgrades

Now, based on the previous step, we have a database backup of our old system in our home directory. So now we're going to import tha data, over-writing the data created by the rt-setup-database section earlier.

mysql -u root --password="secret" rtdb < ~/rtbackup.sql

At this point you're probably used to the drill and know to insert your root password for the database in place of secret. This may take about as long as it did to perform the previous mysql dump, so you might want to take a break at this point.

Once the import is done, we will have to upgrade the database to the newest version, and we will return to a function we have used once before.

cd /usr/sbin

./rt-setup-database --action upgrade --dba root --prompt-for-dba-password

Once you have issued the upgrade command, it will ask you for the root database password as it did before and then the version of the database you are upgrading from. Put in 3.6.5 and hit enter. When it asks you for the version to upgrade to, just hit enter to upgrade it completely

This process will complete remarkably fast, leaving your last step to upgrade the mysql schema.

Upgrading the MySQL Schema

To do this navigate to the specified directory and run the specified command

cd /usr/share/request-tracker3.8/etc/upgrade/

./upgrade-mysql-schema.pl rtdb root secret > ~/queries.sql

Here we are specifying that we would like to upgrade the rtdb database and specifying the root username and its password in place of secret. This will create the sql we need and we are writing it to a file called queries.sql in our home directory.

Now, the last step is to apply the changes to the database

mysql -u root --password="secret" rtdb < ~/queries.sql

Same drill as before. Here you'll provide the username (root) and it's password (secret) for the database you want (rtdb).

Restart Apache and Get Working

Once this is done, it wouldn't hurt to go ahead and restart apache

sudo /etc/init.d/apache2 restart

But now you should be able to browse to http://ip.address.of.server and you will be forwarded to RT. Since the database has been imported, you can login with the same username and password that you would have used on your old 3.6.5 system. Once you've logged in, you'll find that everything is in good working order, but you'll want to double check all of your custom stuff (such as scrips, templates, Custom Fields, etc) to make sure they work. Also do some thorough testing on the fetchmail setup to make sure tickets are getting created correctly.

Additional Information

This isn't really a step, just some information that I wanted to add about RTFM and mail. First, RT will be unable to actually send any mail unless you have an MTA you can relay through. Setting that up is far outside of the scope of this function, but I wanted to at least show you how to get postfix on the box to relay to an existing mail server (assuming it will allow it).

MAIL SETUP

The first thing you'll have to do is, once RT is running the way you want it, is to install postfix with the following command

sudo apt-get install postfix

Once it's done installing, you'll want to edit /etc/postfix/main.cf with your editor of choice and set the following:

relayhost = ip.address.of.server

The relayhost setting will already be in the config file, but nothing will be set for it. You'll specify the IP address of your relay mail server here and then restart postfix.

sudo /etc/init.d/postfix restart

Once this is done, RT will be able to send email without an issue

RTFM

If you had RTFM up and running on your previous install, don't worry, because getting it running here will be the easiest part of this whole process. First, install RTFM with the following:

sudo apt-get install rt3.8-rtfm

Then restart apache

sudo /etc/init.d/apache2 restart

Once you log back in after the Apache restart, you'll notice that RTFM now shows up in the toolbar!

Conclusion

Well hopefully this will get you up and running. I know I went through a little trouble getting it up and running, so hopefully you won't have to go through the same.