Deploying MATLAB Executables with Release R2007a

Deploying MATLAB Executables under MATLAB R2007a

Assuming you have figured out how to compile MATLAB, the following shows you how to deploy MATLAB executables for release R2007a on LSC grid computers, and the Tier 2 centers at AEI (Atlas) and UWM (Nemo).
If you haven't setup up MATLAB compiling, go to this web page on MATLAB compilation.
The technique for deploying MATLAB executables on stand-alone grid computers with independent disk systems (such as AEI's Atlas and UWM's Nemo) can differ from that on grid computers (such as those on the main LSC grids CIT, LHO and LLO) that share disks with the development nodes where a MATLAB installation resides
Deploying executables on stand-alone grids (AEI,UWM) or
Deploying executables on main LSC grids

Deploying executables on stand-alone grids (AEI,UWM)

To run MATLAB executables on grid computers without MATLAB installed (or without access to the MATLAB installation files), we need to install the MATLAB Component Runtime (MCR) files on that system. Such systems include Atlas at AEI and Nemo at UWM.

1) Check if MCR is already set up

Often, the Grid computer sys-admin has already installed this. If that is the case, obtain the location of the MCR installation from them and jump to step 4). On the Atlas cluster in June, 2008, the MCR is installed at /home/thorne/MCR/glnx86/v76 (32-bit) and /home/thorne/MCR/glnxa64/v76 (64-bit) On the Nemo cluster, the MCR would be installed at /home/kthorne/MCR/glnx86/v76 (32-bit) and /home/kthorne/MCR/glnxa64/v76 (64-bit).

2) Get MCR Installer file (MCRInstaller.zip)

This can be created using the buildmcr command within MATLAB. Because the user may not be allowed to write into the default MATLAB directory, we have it create that file locally. Use the following at the MATLAB prompt on the system (such as CIT cluster) where you are compiling your executables
% buildmcr('.');
This will take some time. Here are copies of the 32-bit buildmcr log and 64-bit buildmcr log. A copy of the glnx86 installer is stored on the CIT grid computer at ~thorne/no-backup/Matlab_MCR/R2007a/glnx86. A copy of the glnxa64 installer is stored on the CIT grid computer at ~thorne/no-backup/Matlab_MCR/R2007a/glnxa64.

3) Transfer and install on Grid computer

In the following, it is assumed that you can access the Grid computer using grid-enabled ssh and sftp. For this example, I will use the Nemo cluster.
A) Access the grid computer head node with 'sftp' and create a directory for the MCR files
[thorne@ldas-pcdev1 ~]$ cd ~thorne/no-backup/Matlab_MCR/R2007a/glnx86
[thorne@ldas-pcdev1 glnx86]$ sftp hydra.phys.uwm.edu
Connecting to hydra.phys.uwm.edu...
sftp> mkdir MCR
sftp> cd MCR      
sftp> mkdir R2007a
sftp> mkdir R2007a/glnxa64
sftp> cd R2007a/glnxa64
B) Copy the MCRInstaller.zip for v7.6 to the grid computer and quit sftp
sftp> put MCRInstaller.zip
Uploading MCRInstaller.zip to /home/kthorne/MCR/R2007b/glnxa64/MCRInstaller.zip
MCRInstaller.zip                              100%  138MB   9.8MB/s   00:14
sftp> quit
[thorne@ldas-pcdev1 glnx86]$
C) Access the grid computer head node with 'ssh' and unzip the MCR Installer on the grid computer 
[thorne@auriga ~]$ ssh hydra.phys.uwm.edu
------------------------------------------------------------------------------
Welcome to Nemo at UWM
[kthorne@hydra MCR]$ cd MCR
[kthorne@hydra MCR]$ cd glnx86
[kthorne@hydra MCR]$ unzip R2007a/glnx86/MCRInstaller.zip -d glnx86
[kthorne@hydra MCR]$ logout
[thorne@ldas-pcdev1 glnx86]$
** this creates a v76 sub-directory under the MCR/glnx86 directory

4) Put script with MCR in LD_LIBRARY_PATH on Grid computer

To use MCR on the grid computer, the MCR needs to be added to the LD_LIBRARY_PATH.
This is similar to the setup above, but for MCR directories. Here it is for glnx86 in csh:
# -- the MCR_ROOT location is system specific
setenv MCR_ROOT /home/kthorne/MCR/glnx86/v76
setenv ARCH glnx86
if ( $?LD_LIBRARY_PATH ) then
    setenv LD_LIBRARY_PATH ${MCR_ROOT}/sys/opengl/lib/glnx86:${LD_LIBRARY_PATH}
else
    setenv LD_LIBRARY_PATH ${MCR_ROOT}/sys/opengl/lib/glnx86
endif
setenv LD_LIBRARY_PATH ${MCR_ROOT}/sys/java/jre/glnx86/jre1.5.0/lib/i386:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MCR_ROOT}/sys/java/jre/glnx86/jre1.5.0/lib/i386/server:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MCR_ROOT}/sys/java/jre/glnx86/jre1.5.0/lib/i386/native_threads:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MCR_ROOT}/bin/glnx86:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MCR_ROOT}/sys/os/glnx86:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MCR_ROOT}/runtime/glnx86:${LD_LIBRARY_PATH}
setenv XAPPLRESDIR ${MCR_ROOT}/X11/app-defaults
This setup has some changes from the MathWorks instructions: Here it is for glnxa64 in csh:
# -- the MCR_ROOT location is system specific
setenv MCR_ROOT /home/kthorne/MCR/glnxa64/v76
setenv ARCH glnxa64
if ( $?LD_LIBRARY_PATH ) then
    setenv LD_LIBRARY_PATH ${MCR_ROOT}/sys/opengl/lib/glnxa64:${LD_LIBRARY_PATH}
else
    setenv LD_LIBRARY_PATH ${MCR_ROOT}/sys/opengl/lib/glnxa64
endif
setenv LD_LIBRARY_PATH ${MCR_ROOT}/sys/java/jre/glnxa64/jre1.5.0/lib/amd64:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MCR_ROOT}/sys/java/jre/glnxa64/jre1.5.0/lib/amd64/server:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MCR_ROOT}/sys/java/jre/glnxa64/jre1.5.0/lib/amd64/native_threads:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MCR_ROOT}/bin/glnxa64:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MCR_ROOT}/sys/os/glnxa64:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MCR_ROOT}/runtime/glnxa64:${LD_LIBRARY_PATH}
setenv XAPPLRESDIR ${MCR_ROOT}/X11/app-defaults
This has additions similar to the glnx86 case. I have prepared MCR setup scripts (both csh and bash) for the following: You should be able to modify these as needed. Transfer this MCR Setup file to the Grid computer using sftp. Here is an example for Nemo. 
[thorne@auriga ~]$ sftp hydra.phys.uwm.edu
Connecting to hydra.phys.uwm.edu...
sftp> put MCRSetup_R2007a_glnx86.csh

5) Transfer MATLAB executable to Grid computer

You must move both the compiled executable and the .ctf file to the Grid computer. If doing it statically and assuming 'matlabprog' is the executable, then use
sftp> mkdir Exectuables
sftp> cd Executables
sftp> put matlabprog
sftp> put matlabprog.ctf
sftp> chmod 755 matlabprog
sftp> quit
An alternative is to transfer these within the Condor submission, but remember that you still need to do the 'chmod 755' on the transferred MATLAB executable.

6) Create script to run MATLAB executable on grid

I have found it easier to use a shell script as the 'executable' under condor. Since you can place the MATLAB compiled executable in directories visible from the condor grid nodes, your script can concentrate on setup concerns. Assuming
a) your home directory is /usr1/home/griduser
b) your MATLAB exectuable is named 'matlabprog' and is in sub-directory 'Executables'
c) you have put the MCRSetup_x.csh script in your home directory on the grid computer
then the following is needed in a script to run your executable: (the #'s denote optional comment lines)
# define the home directory
setenv HOME /home/griduser
# add MATLAB MCR to your LD_LIBRARY_PATH
source ${HOME}/MCRSetup_R2007a_glnx86.csh
# add directory with executable to LD_LIBRARY_PATH
setenv LD_LIBRARY_PATH ${HOME}/Executables:${LD_LIBRARY_PATH}
# define variable to point at your executable
setenv MATBIN ${HOME}/Executables/matlabprog
#  run your executable
${MATBIN} ..command-line arguments..

Deploying executables on LSG grid computers

On the main LSC grid computers (CIT, LHO, LLO), the processing nodes can see (through NFS) the same directories as the head and development nodes. This means that MATLAB executables do not have to be tranferred to the "grid computer". Also, since the installed MATLAB libraries are visible, there is no need to build and deploy a MATLAB Common Runtime (MCR).

1) Make sure HOME and PATH will be defined for grid jobs

When MATLAB or the MCR expands a *.ctf file, it wants to save a preferences file to the location ${HOME}/.matlab/mcr_v76. If HOME is not a defined environment variable, it will fail. Also, it requires that the LD_LIBRARY_PATH and PATH environment variables be defined.

One way to ensure this with Condor grid jobs is to include 'getenv = True' in the condor submission. Add the following in your Condor submission script
getenv = True

2) Fix your startup.m file to running executables

When running a stand-alone build on a system where the grid nodes can see the development disks (such as CIT,LLO,LHO), you need to make sure it sees the local encrypted m-files instead of ones from your CVS sandbox. We wrap any addpath commands (which let us see the CVS sandbox files) with a conditional to prevent their use when running an executable. For Release 2007a, the version('-release') function returns a text string 2007a that can not be converted to a numeric value. This complicates things if you wish to preserve backward compatibility with Release 13. A complete syntax of the startup.m file should now be:
verNum = str2num(version('-release'));
if(~isempty(verNum))
    verOld = verNum < 14;
else
    verOld = false;
end
if( (verOld == true) || (~isdeployed))
 ...
  addpath commands
 ...
end
clear verNum;
clear verOld;

The startup.m file should be in your ~/matlab directory.

**NOTE** You may have already done this if following the compilation info page

3A) Create 32-bit MATLAB setup script

If using 32-bit executables, a script is needed to to add the 32-bit 'glnx86' MATLAB libraries to LD_LIBRARY_PATH when running the MATLAB executables. This script can be the same one used when testing your compiled MATLAB. If you don't already have such a script, create a script with the following:
setenv MATLAB_ROOT /ldcg/matlab_R2007a
setenv ARCH glnx86
#
if ( $?LD_LIBRARY_PATH ) then
    setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/sys/opengl/lib/glnx86:${LD_LIBRARY_PATH}
else
    setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/sys/opengl/lib/glnx86
endif
setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/sys/java/jre/glnx86/jre1.5.0/lib/i386:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/sys/java/jre/glnx86/jre1.5.0/lib/i386/server:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/sys/java/jre/glnx86/jre1.5.0/lib/i386/native_threads:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/bin/glnx86:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/sys/os/glnx86:${LD_LIBRARY_PATH}
setenv XAPPLRESDIR ${MATLAB_ROOT}/X11/app-defaults
#
unsetenv MATLAB
Note that this is in 'reverse' order from the Compiler list to ensure correct order in LD_LIBRARY_PATH. Also, I have added a library (/sys/opengl/lib/glnx86) which previous experience with Compiler 4.1 showed was often necessary. I have provided examples in csh (MatlabSetup_R2007a_glnx86.csh) and bash (MatlabSetup_R2007a_glnx86.sh) that handle a non-existent LD_LIBRARY_PATH and checks if MATLAB was already added to it. They are the same as the ones used in testing executables on the development node.

3B) Create 64-bit MATLAB setup script

If using 64-bit executables at CIT, a script is needed to to add the 64-bit 'glnxa64' MATLAB libraries to LD_LIBRARY_PATH when running the MATLAB executables. This script can be the same one used when testing your compiled MATLAB. If you don't already have such a script, create a script with the following:
setenv MATLAB_ROOT /ldcg/matlab_R2007a
setenv ARCH glnxa64
#
if ( $?LD_LIBRARY_PATH ) then
    setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/sys/opengl/lib/glnxa64:${LD_LIBRARY_PATH}
else
    setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/sys/opengl/lib/glnxa64
endif
setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/sys/java/jre/glnxa64/jre1.5.0/lib/amd64:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/sys/java/jre/glnxa64/jre1.5.0/lib/amd64/server:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/sys/java/jre/glnxa64/jre1.5.0/lib/amd64/native_threads:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/extern/lib/glnxa64:${LD_LIBRARY_PATH} 
setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/sys/os/glnxa64:${LD_LIBRARY_PATH}
setenv LD_LIBRARY_PATH ${MATLAB_ROOT}/bin/glnxa64:${LD_LIBRARY_PATH}
setenv XAPPLRESDIR ${MATLAB_ROOT}/X11/app-defaults
#
unsetenv MATLAB
Note that this is in 'reverse' order from the Compiler list to ensure correct order in LD_LIBRARY_PATH. This setup does not match that in the MATLAB documentation. I have added a library (/sys/opengl/lib/glnx86) which previous experience with Compiler 4.1 showed was often necessary. I have provided examples in csh (MatlabSetup_R2007a_glnxa64.csh) and bash (MatlabSetup_R2007a_glnxa64.sh) that handle a non-existent LD_LIBRARY_PATH and checks if MATLAB was already added to it.

4A) Create script to run 32-bit MATLAB executable on grid

I have found it easier to use a shell script as the 'executable' under condor. Since you can place the MATLAB compiled executable in directories visible from the condor grid nodes, your script can concentrate on setup concerns. Assuming
a) your home directory is /archive/home/griduser on ldas-grid.ligo-*.caltech.edu
b) your MATLAB exectuable is named 'matlabprog' and is in directory /archive/home/griduser/Executables
c) you have put the MatlabSetup_R2007a_glnx86.csh script in your home directory
then the following is needed in a script to run your 32-bit executable: (the #'s denote optional comment lines)
# define the home directory (this is not set automatically)
#  MATLAB needs HOME defined.
setenv HOME /archive/home/griduser
# add 32-bit MATLAB MCR to your LD_LIBRARY_PATH
source ${HOME}/MatlabSetup_R2007a_glnx86.csh
# add directory with executable to LD_LIBRARY_PATH
setenv LD_LIBRARY_PATH ${HOME}/Executables:${LD_LIBRARY_PATH}
# define variable to point at your executable
setenv MATBIN ${HOME}/Executables/matlabprog
#  run your executable
${MATBIN} ..command-line arguments..

4B) Create script to run 64-bit MATLAB executable on grid

I have found it easier to use a shell script as the 'executable' under condor. Since you can place the MATLAB compiled executable in directories visible from the condor grid nodes, your script can concentrate on setup concerns. Assuming
a) your home directory is /archive/home/griduser on ldas-grid.ligo.caltech.edu
b) your MATLAB exectuable is named 'matlabprog' and is in directory /archive/home/griduser/Executables
c) you have put the MatlabSetup_R2007a_glnxa64.csh script in your home directory
then the following is needed in a script to run your 64-bit executable: (the #'s denote optional comment lines)
# define the home directory (this is not set automatically)
#  MATLAB needs HOME defined.
setenv HOME /archive/home/griduser
# add 64-bit MATLAB MCR to your LD_LIBRARY_PATH
source ${HOME}/MatlabSetup_R2007a_glnxa64.csh
# add directory with executable to LD_LIBRARY_PATH
setenv LD_LIBRARY_PATH ${HOME}/Executables:${LD_LIBRARY_PATH}
# define variable to point at your executable
setenv MATBIN ${HOME}/Executables/matlabprog
#  run your executable
${MATBIN} ..command-line arguments..

Keith Thorne 
t h o r n e @ g r a v i t y . p s u . e d u
Last modified: Fri, June 13, 2008