Restricting Community Accounts using the Communty Shell (commsh)

Author: Terry Fleury (tfleury@ncsa.illinois.edu)

Last Update: June 25, 2009

Introduction

Overview

The Community Shell application (commsh) represents a means of securing shared, community accounts at the resource level. This document serves as a guide for resource administrators seeking to implement the Community Shell on their resources.

The first section, 'Installation', details basic installation and configuration of the Community Shell.

The second section, 'Administation', details usage of administrative accounts to manage each community.

Background

A growing trend in supercomputing is the use of community gateways that provide supercomputing resources to a wide audience. These gateways typically maintain a user database so that each user can log in to an individual account at the gateway. However, the community of users may share a single community account credential which is utilized when performing computations on supercomputing resources. Thus, while the gateway/portal may be able to track individuals, the supercomputing resources can track only the single community account user. Since all gateway users (theoretically) have access to the community account credential, a gateway user could conceivably execute unauthorized code on a supercomputing resource and do so with a high degree of anonymity, which may result in all users of that gateway losing access to the resource. The goal of the Community Shell project is to mitigate this potential for abuse by placing restrictions on the applications which may be executed by a community account.

To this end, a configuration file is created for each shared account that explicitly lists which applications (commands) the account can run. It can further specity the form of the parameters (arguments) passed to those applications. In a typical scenario, this configuration file will establish a directory for applications, known as a Community Software Area, used by the community. Any applications within this single directory will be allowed, and applications outside of this directory will be restricted. Applications can then be added to or removed from this directory as needed.

In a more advanced/secure scenario, the executables in the Community Software Area are not called directly by the gateway. Rather, a set of static (i.e. unchanging) scripts is written by a community developer. These scripts are placed in a protected location on the supercomputing resources with the restriction that they are run with security provided by the Community Shell (i.e. commsh restricts the community account to run only the approved scripts rather than the binaries in the Community Software Area). The scripts then call the binaries in the Community Software Area. This allows for the binaries to be updated as needed (i.e. to upgrade to new software package versions) without involvement of the system administrator, while keeping the security restrictions placed on the scripts (so that only a small set of pre-approved binaries can be executed). The system administrator is responsible for approving the scripts and installing them in an appropriate directory.

In either scenario, it is important that the community account itself does not have access to modify the contents of the Community Software Area. If the community account credential can be utilized to upload scripts into the Community Software Area, arbitrary code can be executed on the system thereby circumventing security precautions. In order to prevent this form of abuse, it is recommended that a second, community administrator account, be established for each community.

The community administrator account would be used by the community administrators to manage which applications the community account can exeucte. The community administrator account should have its own set of credentials (password or private key) that are separate from the community account's. The community administrator account credentials should not be stored on the same server as the community account credentials, so that a compromise of the community gateway server will not compromise the administrative account.

It should be noted that this community administrator method is only our suggested method of managing community accounts. There are other feasible methods, including group-based access or having the resource administrator manage each account directly. The method which makes the most sense for a given resource should be determined by the resource administrator.

Installation

This section explains how to install and configure the Community Shell (commsh) to restrict community accounts under a UNIX-like system.

Prerequisites

In order to build and install commsh, you will need the following software installed:

GNU Make
A standards-compliant C++ compiler (gcc3 preferred)
Sed

In order to make use of the GRAM functionality, you will need:

a fully configured GRAM installation or a GT4.x installation
GNU Patch (or compatible)

Installation of commsh

A source tarball of the most recent version of commsh can be downloaded from:

http://security.ncsa.illinois.edu/research/commaccts/downloads/commsh-latest.tar.gz

You will need at least version 0.1.3b for GRAM support (see below).

After downloading and unpacking the tarball, the package can be built as follows:

./configure
gmake
gmake install     # Note: you must be 'root' to do this

You can control where the package is installed by passing arguments to ./configure. For a full list of options, run:

./configure --help

It is especially important to configure the final location of the configuration file, as this location is hard-coded into the executable binary and cannot be changed without recompiling. By default, the commsh binary will be installed in /usr/local/bin/ and the associated configuration file will be read from /etc/commsh.conf. You can change these locations by using the "--prefix=/alternate/binary/path" and "--sysconfdir=/alternate/config/path" command line parameters when running ./configure.

Proposed Configuration

For a full list of configuration directives, see the commsh.conf man page.

Directories / Files

Here is the proposed layout for the various directories and files utilized by commsh, a Community Software Area, and a Community Account User. We assume that the request for a Community Software Area was made by "jzsmith", who is the primary Community Developer. The Community Software Area is named "ntroport". The Community Account User is named "ntrouser". Note that while the Community Software Area name can differ from the Community Account User name (as shown here), it is often the case that the two are the same. They are different in this example to illustrate the ownership of files and directoreis (i.e. uid and gid).

    Ownership         Perm Directory / File        Usage
    ---------         ---- ----------------        -----
    root:root         0755 /usr/local/bin/commsh   Location of commsh binary [1]
    root:root         0644 /etc/commsh.conf        Configuration file for commsh [2]
    root:root         0755 /etc/commsh.d/          Directory for per-community configurations [3]
    root:root         0755 /etc/commsh.d/ntroport/ Location of config file and optional scripts for Community Software Area [4]
    ntrouser:ntroport 2770 ~ntrouser               Community Account User's home directory [5]
    jzsmith:ntroport  2775 $TG_COMMUNITY/ntroport/ Location of Community Software Area binaries and associated files [6]

Explanatory Notes

The default configuration of commsh installs the binary into /usr/local/bin. While this is what we will use in the proposed configuration, you can change this with the "--prefix=..." option when running configure.
When commsh is first installed, a sample configuration file is installed in /etc/commsh.conf.sample. The system administrator can look at this file for some example uses of commsh. Ultimately, the system administrator must create /etc/commsh.conf either by copying the sample configuration file and editing it, or by creating it from scratch. If you want commsh to read the configuration file from some place other than /etc/commsh.conf, you can set the "--sysconfdir=..." option when running configure.
This directory is the storage location for all Community Software Area configuration directories (as shown in [4] below). In other words, this directory should contain only directories, no files.
The configuration files for commsh specific to the ntroport Community Software Area / Community Account User reside here. The /etc/commsh.conf file contains references to the various Community Software Areas' configuration files. For example, the /etc/commsh.conf file will have entries like this, one for each Community Software Area:
```
    # /etc/commsh.conf
    CheckUser ntrouser
    ReadUserConfig ntrouser /etc/commsh.d/ntroport/commsh.conf
```
This new configuration file /etc/commsh.d/ntroport/commsh.conf has the entries for binaries (for a simple setup) in the Community Software Area directory or scripts (for a complex setup) in the /etc/commsh.d/ntroport directory which will be called by commsh. Here are two basic configuration files showing a simple configuration and an advanced configuration. The first configuration file shows a very simple setup where a Community Account User is allowed to run any executable in the Community Softare Account $TG_COMMUNITY/bin directory, using any number of command line parameters.
```
    # /etc/commsh.d/ntroport/commsh.conf
    # Simple configuration - direct access to Community Software Area binaries
    # Allow the Community Account User to run any command in the Community Software Area bin directory with any parameters
    DirectAccess $TG_COMMUNITY/ntroport/bin/* **
```
The second example is more complex and involves a bit of indirection to deter unauthorized modification of allowable executables. In this setup, a Community Account Developer has created two scripts with the intention that they not be modified. They are placed in the read-only /etc/commsh.d/ntroport directory by the system administrator. Within these two scripts are calls to executables stored in the Community Software Area $TG_COMMUNITY/bin directory. (Note that the scripts are highly specific to a particular Community Software Area and are not shown here.) The configuration file below allows a Community Account User to call only these two scripts with two command line parameters (for input and output). The scripts call executables in the Community Software Area $TG_COMMUNITY/bin directory with the two command line parameters. The idea here is to give a Community Account User a limited number of commands that can be called, but still allow a Community Developer to update the underlying executables stored in the Community Software Area $TG_COMMUNITY/bin directory.
```
    # /etc/commsh.d/ntroport/commsh.conf
    # Advanced configuration - indirect access to Community Software Area binaries via protected scripts
    # Allow the Community Acccount User to run two scripts in the protected directory with two parameters
    # script1 and script2 in turn call executables in $TG_COMMUNITY/ntroport/bin
    DirectAccess /etc/commsh.d/ntroport/script1 -input * -output *
    DirectAccess /etc/commsh.d/ntroport/script2 -input * -output *
```
When the request for a Community Account User has been approved, its home directory is automatically created. In this case, we deliberately chose the Community Software Area name to be different from the Community Account User name. You will probably want to choose them to be the same. The Community Account User home directory will be utilized by the various binary files for input and output since the binaries will be executed with the Community Account User's credential via GRAM. In other words, all input files should be transferred to the Community Account User's home directory, for example via GridFTP. The Community Software Area binaries, which are often stored in $TG_COMMUNITY/ntroport, read the input files from the Community Acccount User's home directory. Any output generated by the binaries will be written to the Community Acccount User's home directory. This output can then be fetched using GridFTP. Note that the group permission (gid) on the Community Acccount User's home directory allows for easy access by the Community Developers for debugging purposes.
When the request for a Community Software Area has been approved, the Community Developers's Community Software Area directory is automatically created. Note again that we deliberately chose the Community Software Area name to be different from the Community Account User name, but you should choose them to be the same. The directory $TG_COMMUNITY/ntroport is where the binaries (and other associated files) for the Community Software Area will be stored. These binaries are called either directly or by the scripts located in /etc/commsh.d/ntroport/. This allows for the binaries to be updated frequently while keeping the scripts secure (since the scripts should not require frequent updating).

Here we show a basic configuration for two Community Account Users named commuser1 and commuser2. Each Community Account User will have an entry in the global /etc/commsh.conf file to read in the per-community configuration files.

/etc/commsh.conf:

# Allow any user but root to run commands through commsh
AllowUser *
DenyUser root

# Community Acccount User 1 (commuser1)
# Check GRAM job submissions by this user, assuming GRAM support is enabled
CheckUser commuser1
# Load an external configuration file for this user
ReadUserConfig commuser1 /etc/commsh.d/commuser1/commuser1.conf

# Community Acccount User 2 (commuser2)
# Check GRAM job submissions by this user, assuming GRAM support is enabled
CheckUser commuser2
# Load an external configuration file for this user
ReadUserConfig commuser2 /etc/commsh.d/commuser2/commuser2.conf

/etc/commsh.d/commuser1/commuser1.conf:

# Allow commuser1 to run any command in /opt/communities/commuser1/bin with any parameters
DirectAccess /opt/communities/commuser1/bin/* **
# Allow commuser1 to create new directories and delete any files in /home/commuser1/shared/
DirectAccess /bin/mkdir /home/commuser1/shared/*
DirectAccess /bin/rm -rf /home/commuser1/shared/*

/etc/commsh.d/commuser2/commuser2.conf:

Allow commuser2 to run three scripts with partially specified parameters.
DirectAccess /opt/communities/commuser2/bin/script1 -input * -output *
DirectAccess /opt/communities/commuser2/bin/script2 -input * -output *
DirectAccess /opt/communities/commuser2/bin/script3 -input * -output *

The most important thing to understand when creating command restrictions is the difference between * and **, and what each of them will match.

A single asterisk (*) will match any character in a single argument. In general, this means it will not match a space unless the space is enclosed in quotation marks or escaped with a back-slash. Additionally, an asterisk in the command itself will not match a backslash (/). For example:

DirectAccess /opt/communities/*/script < *

Will match:

/opt/communities/one/script < input
/opt/communities/two/script < "input file"
/opt/communities/three/script < input\ file


But will not match:

/opt/communities/one/two/script < input
/opt/communities/one/script < input file

In contrast, a double asterisk (**) should only appear 
at the end of a command restriction specification, and indicates that 
any additional parameters will be 
accepted.

Activation / Shell Support

In order to activate commsh parsing for a given user, the user's shell 
must be changed to commsh. Exact details on how to change a user's shell 
vary from system to system, but is often set in /etc/passwd. The Community Account User's shell should be changed 
to:

/usr/local/bin/commsh




Or to the location where you installed commsh, if you specified a 
different installation location.


Typically, the shell must be added to /etc/shells (or a
similar file) for the system to accept it as a valid shell.


GRAM support (optional)

GRAM support of commsh is currently part of the latest
standard GRAM 
installation. However, older versions of GRAM require a simple patch to make it possible to use
commsh 
to implement command-based restrictions on GRAM jobs.  This patch can 
be downloaded from:


http://security.ncsa.illinois.edu/research/commaccts/downloads/globus-job-manager-script-pl.diff


To install the patch, run the following as user globus:

cp globus-job-manager-script-pl.diff $GLOBUS_LOCATION/libexec
cd $GLOBUS_LOCATION/libexec
patch -p0 < globus-job-manager-script-pl.diff


If the patch complains about 'fuzz', don't worry. The patch is designed 
to work with multiple versions of GRAM. As long as both hunks succeed, 
the patch has been successfully applied.


Once the patch is applied, edit 
$GLOBUS_LOCATION/libexec/globus-job-manager-script.pl and 
uncomment the $FILTER_COMMAND line. Also change this line to 
point to your 
installation of commsh if you changed the installation location.

NOTE: Under some installations, such as TG CTSS, there are separate 
trees for each service. Under these installations, the patch must be 
applied to all directories serving GRAM, both WSGRAM (GRAM4) and 
pre-WSGRAM (GRAM2) services. Under TG CTSS, this should be the 
prews-gram directory as well as the globus-wsrf 
directory.




Other Links
Man page for commsh binary


Man page for commsh.conf configuration file


Community
Shell Installation / Configuration Instructions for TeraGrid

NCSA CyberSecurity

Projects and Software