NCSA CyberSecurity

Author:	Terry Fleury (tfleury@ncsa.uiuc.edu)
Last Update:	April 13, 2006

Implementation

Server Setup

In each step below, a 'general' instruction is given first, followed by a link for specific instructions. These specific instructions are initially hidden and can be viewed by clicking on the appropriate link.

1. Install the operating system For your server.
   Instructions For Fedora Core 4...
   
   Download and install Fedora Core 4. You should be able to install as few or as many packages as you want. Using the "workstation" installation profile might be a good selection. After you have a working installation, update your software to the latest versions using "yum update".

2. Install and configure Java and Tomcat.
   Instructions For Installing Java 1.5.0...
   
   1. Remove limited version of Java 1.4.2 (if installed).
      Depending on your installation, you may or may not have installed a limited runtime Java 1.4.2 environment. While this Java installation may be adequate for running Java applets in your web browser, it is lacking in many areas. For Tomcat, it is better to use a fuller Java installation such as one provided by Sun. Currently, the only yum-based solution for Tomcat is available from the JPackage Project. However, even with this repository, you can only get Tomcat 5.0.x, not 5.5.x. Additionally, the Java available in this repository is 1.4.2.

      So, I feel it's best to abandon the yum approach in favor of downloading tarballs and untarring them to your hard drive. Using this approach, all files are confined to two directories (one for Java, one for Tomcat), and installation could even be accomplished by a non-root user.

      But first you need to get rid of the barebones Java package which may have been installed before. As root, run the following yum command:

      yum -y remove java-1.4.2-gcj-compat
      

      The "-y" option will answer yes to all questions so that the package and any dependencies (such as jessie) are removed without further input from the user. You may wonder why bother removing this particular Java since we'll be installing a better Java "over it". The problem is the strange symbolic linking strategy used by java-1.4.2-gcj-compat. If you look at /usr/bin/java, you will see that it's a symbolic link to /etc/alternatives/java, which in turn is a symbolic link to /usr/lib/jvm/jre-1.4.2-gcj/bin/java. It can get very confusing which version of java and corresponding libraries are being used when you run a Java application. Better to simply remove the potential problem.
   2. Download Java 1.5.0, a.k.a. Java 2 Platform Standard Edition 5.0.
      Now that we have a "clean slate", we can downlaod and install the latest version of Java. As of March 10,2006, this is Java 1.5.0_06. Older versions, such as J2SE 1.4.2, should work as well. When downloading Java, make sure to get the JDK (J2SE Development Kit) 5.0 Update 6. You can get either the "Linux self-extracting file" which is basically a tarball, or you can get the "Linux RPM in self-extracting file", which will give you an RPM you can install as root. The approach you use is up to you.
   3. Install the Java files.
      The self-extracting file you downloadeded has a .bin extension. You need to "run" the file to extract the Java files. If you downloaded the RPM version, the resulting file will be named jdk-1_5_0_06-linux-i586.rpm. Use a standard rpm -Uvh command to install this file. If you downloaded the tarball version, the result is a directory named jdk1.5.0_06 in the current directory. In either case, here's how to extract the Java files:

      chmod +x jdk-1_5_0_06-linux-i586.bin
      ./jdk-1_5_0_06-linux-i586.bin
      

      The RPM installs all files to /usr/java/jdk1.5.0_06/. You can install the "self-extracting file" anywhere you like, but for the purposes of these instructions, we will assume that you installed it in /usr/java/ so that the directory referenced in the following instuctions is the same regardless of which method you chose.
   4. Set the environment variables.
      The /usr/java/jdk1.5.0_06/bin directory is probably not in your path, so you need to add it. Edit the file /etc/profile and add the following lines near the bottom of the file, but before the unset entries.

      export JAVA_HOME=/usr/java/jdk1.5.0_06
      pathmunge $JAVA_HOME/bin after
      

      Now not only are the java binaries in your path, but you can easily refer to the Java installation directory using the $JAVA_HOME environment varilable.
   Instructions For Installing Tomcat 5.5.17...
   
   1. Download Tomcat 5.5.17.
      Note that though these instructions are specific to Tomcat 5.5.x, you should be able to use 5.0.x if you prefer, modifying the path names where appropriate. Tomcat uses random mirrors for downloading. At a minimum, you will need to get the Core Package. View the Package README to see if you want or need any other packages.
   2. Install the Tomcat files.
      The package you downloaded is a tarball. You can install it anywhere you like. For the sake of consistency, let's create a directory /usr/tomcat and install the tarball there. The resulting files will be extracted to the directory /usr/tomcat/apache-tomcat-5.5.17/.
   3. Set the environment variables.
      The /usr/tomcat/apache-tomcat-5.5.17/bin directory is probably not in your path, so you need to add it. Edit the file /etc/profile and add the following lines near the bottom of the file, but before the unset entries.

      export CATALINA_HOME=/usr/tomcat/apache-tomcat-5.5.17
      pathmunge $CATALINA_HOME/bin after
      

      Notice that the environment variable here is $CATALINA_HOME. There are numerous references to this environment variable in both the Tomcat documentation and in other software packages, so make sure the name is correct.

3. Install Globus/CoG Kit libraries to support proxy certificates in Tomcat.
   While Tomcat comes with libraries to allow for https connections, these libraries do not support proxy certificates. So you need to download a few Java libraries from the Globus CVS site and copy them to the Tomcat library directories.
   Instructions For Fedora Core 4...
   
   You need to have cvs installed to get the Globus libraries. If you don't have it installed, you can use yum as "yum install cvs". Then run the following commands:

   > cd /tmp
   > cvs -d:pserver:anonymous@cvs.globus.org:/home/globdev/CVS/globus-packages login
   Logging in to :pserver:anonymous@cvs.globus.org:2401/home/globdev/CVS/globus-packages
   CVS password:   (here, type <ENTER> for an empty password)
   > cvs -d:pserver:anonymous@cvs.globus.org:/home/globdev/CVS/globus-packages \
         co wsrf/java/common/source/lib
   cvs checkout: Updating wsrf/java/common/source/lib
   U wsrf/java/common/source/lib/LICENSE-DOM.html
        ... and a bunch of other files ...
   

   Next, copy the required libraries to the appropriate locations.

   > cd /tmp/wsrf/java/common/source/lib
   > cp cog-axis.jar cog-jglobus.jar cog-url.jar cryptix32.jar \
        cryptix-asn1.jar jce-jdk13-131.jar log4j-1.2.13.jar puretls.jar \
        $CATALINA_HOME/common/lib
   > cp cog-tomcat.jar $CATALINA_HOME/server/lib
   

4. Configure Tomcat to use these Globus Java libraries.
   Next you need to configure Tomcat to actually use these Globus Java libraries you just copied over to the Tomcat library directories. The configuration is nearly the same for both Tomcat 5.0.x and Tomcat 5.5.x.
   Instructions For Configuring Tomcat...
   
   Edit the file $CATALINA_HOME/conf/server.xml and make the following changes:

   • In the <Service name="Catalina"> section, look for a <Connector ...> XML block with the comment <!-- Define a SSL HTTP/1.1 Connector on port 8443 -->. This Connector, which provides Tomcat's built-in SSL support, should already be commented out, which is exactly what we want. Just below that section, add the following:

     <!-- Define a GSI HTTPS/1.1 Connector on port 8443 
              Supported parameters include:
              proxy="/path/to/file" // proxy file for server to use
                 - or -
              cert="/path/to/file"  // server certificate file in PEM format 
              key="/path/to/file"   // server key file in PEM format
     
              cacertdir="/path/to/dir" // directory containing trusted CA certs
     
              mode="ssl"            // use 'standard' SSL via "https://" - default
                 - or -     
              mode="gsi"            // use 'GSI' SSL via "httpg://" to delegate 
                                    // a (proxy) credential over TLS
                   
     -->
     <Connector
         className="org.globus.tomcat.coyote.net.HTTPSConnector"
         port="8443"
         maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
         enableLookups="false" disableUploadTimeout="true"
         acceptCount="100" debug="0" scheme="https" autoFlush="true"
         protocolHandlerClassName="org.apache.coyote.http11.Http11Protocol"
         socketFactory="org.globus.tomcat.catalina.net.BaseHTTPSServerSocketFactory"
         cert="/etc/grid-security/hostcert.pem"
         key="/etc/grid-security/hostkey.pem"
         cacertdir="/etc/grid-security/certificates" 
         mode="ssl"
     />
     

     If you are using Tomcat 5.0.x, change the socketFactory line to:

         socketFactory="org.globus.tomcat.catalina.net.HTTPSServerSocketFactory"
     

   • In the <Engine name="Catalina" defaultHost="localhost"> section, look for a <Valve ...> XML block, which should already be commented out. Just below that section, add the following if you are using Tomcat 5.5.x:

     <!-- Define a GSI HTTPS/1.1 Valve for Connector on port 8443 -->
     <Valve className="org.globus.tomcat.coyote.valves.HTTPSValve55" />
     

     If you are using Tomcat 5.0.x, use the following className line instead:

     <Valve className="org.globus.tomcat.coyote.valves.HTTPSValve" />
     

   
   
5. Add Globus libraries to Tomcat's CLASSPATH.
   If you are using certain Globus libraries in your web service code, you may need to add them to Tomcat's CLASSPATH at startup time.
   Instructions For Configuring Tomcat...
   
   Edit the file "$CATALINA_HOME/bin/setclasspath.sh" and add the following to the end of the file:

   CLASSPATH="$CLASSPATH":"$CATALINA_HOME"/common/lib/cog-url.jar
   

   
   
6. (Optional) Configure Tomcat for authorization via grid-mapfile.
   If you want gridmap-style authorization (in addition to the authentication you configured above), you need to install another .jar file and add another <Valve ...> section to your Tomcat's server.xml configuration file.
   Instructions For Configuring Tomcat...
   
   • Download this cog-gridmap.jar file and copy it to $CATALINA_HOME/server/lib/.
   • Edit the file $CATALINA_HOME/conf/server.xml and make the following change. In the <Engine name="Catalina" defaultHost="localhost"> section, look for the <Valve className="org.globus.tomcat.coyote.valves.HTTPSValve55"/> you added in the previous step. Just below that XML block, add the following XML block for Tomcat 5.5.x:

     <!-- Require that Distinguished Names (DNs) exist in the gridmap file
              Supported parameters include:
              gridmapfile="/path/to/file"  // location of the grid-mapfile;
                                           // defaults to standard Globus location
              allowUnsecured="true"        // allow connections without a DN entry
              meaningfulErrors="true"      // send meaningful 403 error messages
              passRequest="true"           // pass the request on to the next valve 
              // all boolean options above default to "false" if unspecified
     -->
     <Valve 
         className="org.globus.tomcat.coyote.valves.GridmapFilterValve55"
         gridmapfile="/etc/grid-security/grid-mapfile"
         allowUnescured="false"
         meaningfulErrors="true"
         passRequest="true" 
     />
     

     If you are using Tomcat 5.0.x, use the following className line instead:

         className="org.globus.tomcat.coyote.valves.GridmapFilterValve"
     

   • For more information on grid-mapfiles, including formatting and tools for managing, see the Globus documentation in the section entitled Specifying Identity Mapping Information.
   
   
7. Obtain a certificate to secure your Tomcat server.
   There are several ways to secure your Tomcat server. You can obtain an SSL certificate from one of the many commercial certificate providers, for example VeriSign. This will allow users to connect to your website via an https:// connection without any annoying warning messages. You can obtain a free SSL certificate from CAcert.org. You can create a self-signed certificate which will generate a warning message for users, but otherwise provides a secure https connection. Finally, if you are at an organization which can act as a Certificate Authority (CA) such as NCSA, you can obtain a host certificate authorized by that organization.
   Instructions For NCSA-Signed Certificate...
   
   Log onto tungsten.ncsa.uiuc.edu or public-linux.ncsa.uiuc.edu and run "/usr/local/NCSA-CA/bin/ncsa-cert-request -host HOSTNAME" where HOSTNAME is the FQDN (fully-qualified domain name) of your Tomcat server. Follow the directions filling in the appropriate information. The host certificate and key will be stored in ~/.globus/hostcert.pem and ~/.globus/hostkey.pem respectively. For more information on NCSA certificates and policies regarding their issuance, see http://www.ncsa.uiuc.edu/UserInfo/Grid/Security/.
   
   Instructions For CAcert.org...
   
   Go to CAcert.org, create a free account, and request a server certificate. Basically, you will need to generate a Certificate Signing Request (CSR), paste the contents of the resulting text file into their web-based submission form, and install the resulting server certificate into Tomcat.
   
   Instructions For Commercial SSL Certificate Provider...
   
   You need to generate a Certificate Signing Request (CSR) for your commercial SSL certificate provider. Typically, you can find instructions for this procedure on the commercial provider's web site. If not, you can generate a CSR with OpenSSL.
   Instructions For Self-Signed Certificate...
   
   A self-signed certificate does not provide the security guarantees provided by a certificate signed by a commercial Certificate Authority and thus is NOT recommended. But it will allow you to provide a secure https connection to your site. The biggest difference clients will see is a warning message stating that your site's identity cannot be verified and thus is not a "trusted site". Clients then have the option of accepting the certificate for the session. OpenSSL has facilities for generating a self-signed certificate.

8. Install your host certificate and private key for the Tomcat Server.
   Instructions For Fedora Core 4...
   
   Strictly speaking, you can install your host certificate and key anywhere you like on your server as long as the Tomcat server has read access to that location. The instructions here assume that you have obtained an NCSA-signed certificate and thus use a configuration typical for a globus installation. This configuration will work equally well with a self-signed or commercially-signed certificate.

   Install hostcert.pem and hostkey.pem in /etc/grid-security
   Set permissions appropriately for these directories and files:
   chmod 755 /etc/grid-security
   chmod 644 /etc/grid-security/hostcert.pem
   chmod 600 /etc/grid-security/hostkey.pem

9. If necessary, install Certificate Authority files.
   If you obtained your host certificate from a commercial provider, you do not need to install any CA files since clients will have information on the commercial provider and accept their root CA as trusted. If you generated a self-signed certificate, you do not need to install any CA files since the certificate is not signed by any Certificate Authority.
   For Other Certificate Authorities...
   
   You will need to download and install the CA files for the CA that issued your host certificate. CAcert.org has its own set of root certificates that you can use. You may need to rename them using their hash values.

   For NCSA-issued certificates, download the following two certificate authority files by right-clicking on them and selecting "Save As...":
   http://ca.ncsa.uiuc.edu/4a6cd8b1.0
   http://ca.ncsa.uiuc.edu/4a6cd8b1.signing_policy

   Move the files you downloaded to /etc/grid-security/certificates/ and set appropriate permissions:
   chmod 755 /etc/grid-security/certificates
   chmod 644 /etc/grid-security/certificates/*

10. If you are running a firewall, allow access for http/https to Tomcat.
    By default, Tomcat uses port 8080 for http connections and port 8443 for https connections. (However, you can change these values in the $CATALINA_HOME/conf/server.xml file.)
    Instructions For Fedora Core 4...
    
    For the firewall provided with Fedora, run "system-config-securitylevel". Select the "Customize" button. Then on the "Other ports" line, enter "8080 8443" (without the quotes).

11. Restart the Tomcat server.
    Instructions For Fedora Core 4...
    
    If the Tomcat server is running, you can run the following commands to "restart" the server:

    > shutdown.sh ; startup.sh
    Using CATALINA_BASE:   /usr/tomcat/apache-tomcat-5.5.17
    Using CATALINA_HOME:   /usr/tomcat/apache-tomcat-5.5.17
    Using CATALINA_TMPDIR: /usr/tomcat/apache-tomcat-5.5.17/temp
    Using JRE_HOME:       /usr/java/jdk1.5.0_06
    Using CATALINA_BASE:   /usr/tomcat/apache-tomcat-5.5.17
    Using CATALINA_HOME:   /usr/tomcat/apache-tomcat-5.5.17
    Using CATALINA_TMPDIR: /usr/tomcat/apache-tomcat-5.5.17/temp
    Using JRE_HOME:       /usr/java/jdk1.5.0_06
    

    If the server isn't running, you may see error messages like this after the first set of four lines (above):

    Mar 10, 2006 2:59:54 PM org.apache.catalina.startup.Catalina stopServer
    SEVERE: Catalina.stop: 
    java.net.ConnectException: Connection refused
            at java.net.PlainSocketImpl.socketConnect(Native Method)
            at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
            at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
            at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
            at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
            at java.net.Socket.connect(Socket.java:507)
            at java.net.Socket.connect(Socket.java:457)
            at java.net.Socket.<init>(Socket.java:365)
            at java.net.Socket.<init>(Socket.java:178)
            at org.apache.catalina.startup.Catalina.stopServer(Catalina.java:394)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:585)
            at org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:324)
            at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:415)
    

    Don't worry about it. It's just Java's way of saying that there was nothing to shut down. You may also notice that there isn't a lot of output when starting up. That's because all output (other than the 4 lines above) is written to files in the $CATALINA_HOME/logs/ directory. It's a good practice when first configuring Tomcat to look at the contents of $CATALINA_HOME/logs/catalina.out to make sure that there are no error messages when starting up. One way to do this is to change your "restart" command to something like this:

    shutdown.sh ; startup.sh ; tail -f $CATALINA_HOME/logs/catalina.out
    

    The tail -f command will "follow" the output of the catalina.out file as it is being written.