Tech Note: How to Grid-Enable An Existing DataStage Installation

Recently we have tried to follow the official IBM Documentation about Grid deployment: http://pic.dhe.ibm.com/infocenter/iisinfsv/v9r1/index.jsp?topic=%2Fcom.ibm.swg.im.iis.ds.parjob.dev.doc%2Ftopics%2Fgrid_intro.html. Unfortunately it turns out that it does not work.

 

The correct way to do it is to use the Grid deployment toolkit provided by IBM. Here are the steps we followed to grid-enable an existing DataStage 9.1 installation:

1. Install IBM Tivoli Workload Scheduler LoadLeveler

This is one of the many supported resource managers but we selected it since it comes from IBM and we love IBM software :). The installation procedure could be found here: http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=%2Fcom.ibm.cluster.loadl.doc%2Fllbooks.html. In addition there is an IBM Redbook: "Workload Management with LoadLeveler", rather outdated but still a valuable source.

Please mind that Grid Toolkit 4.2.6+ is not compatible with LoadLevler 4.1.1.0-0, so consider installing 4.1.1.14 or later. LoadLeveler 5.x would also work, but requires Redhat / Centos 6.

2. Install IBM Grid Toolkit  

• Source the dsenv: . /opt/IBM/InformationServer/Server/DSEngine/dsenv
• Copy the grid toolkit files under $DSHOME/../PXEngine
• Execute install.sh and provide the required responses
• Server type: Information Server Install
• Resource Manager: LoadLeveler
• Path to Resource Manager commands: /home/loadl/bin
• Grid JOBDIR: /home/dsadm/grid_jobdir
• Maximum number of nodes per job (global): 3
• List of valid LoadLeveler Classes: small medium large No_Class
• Enter a default Class for running jobs: medium
• Source once again the dsenv
• Restart DataStage

3. Configure ssh

• Replace rsh with ssh in $APT_ORCHHOME/etc/remsh
• Enable password-less ssh login for the unix user used to run the jobs

Now you should be able to add the Grid parameters to your job and successfully execute them using the grid environment. More details about the grid environment installation could be found in another IBM Redbook: "Deploying a Grid Solution with the IBM InfoSphere Information Server".