PURPOSE Here I outline how to compile Matlab .m files into a stand-alone C application. The primary benefit of using a stand-alone is that it *does not* consume Matlab licenses at run-time, so you can run hundreds of parallel applications on a grid without upsetting the license manager at all. To this end, I'll also show how to get that C app to run on the Sun Grid Engine (SGE). WHY THIS TUTORIAL? Unfortunately, other tutorials I found, while helpful, lack realism. The sample applications built are usually the super simple "generate a magic square matrix" variety. These don't help anybody deal with important concerns like:
ACKNOWLEDGMENTS Thanks to Soumya Ghosh for a good intro conversation about using mcc and discussion about dealing with addpath and related issues. RELATED LINKS For other tutorials on using mcc, check out: http://helix.nih.gov/Applications/matlab_compiler.html http://pages.stern.nyu.edu/~nwhite/scrc/CompilingMatlab.htm DEMO APPLICATION OVERVIEW I've chosen as a demo application a Gibbs Sampler for Latent Dirichlet Allocation, which I use for my research. The application reads in a bag-of-words representation about a document collection from a .mat file, runs a Gibbs Sampler to discover the latent topic structure, and saves the results of each run of the sampler to a .mat file for long-term safe keeping. You can find a compressed Unix archive of all the code for this project at this link or as an attachment at the bottom of this tutorial. Windows users will probably need to do some extra legwork to make it work on their machines. Here's a view of the key files and folders involved in the project:  The main matlab script is driver_LDA.m. The bash script to build the MCC app is buildMCC.sh. The bash scripts to run a batch of grid jobs are submit_LDA_jobs.sh and grid_driver_LDA.sh. The data directory contains the bag-of-words data itself and the script to load it. The mex directory contains the C++ code for an LDA Gibbs Sampler, as well as compiled mex files for 32 and 64 bit Unix. I compiled under Matlab R2010a and gcc C++ compiler version 4.2, so you may need to recompile if your setup is different. The LDAutil directory contains utility scripts that initializing the sampler, calculate log likelihood, etc. We'll store the output and error messages from the grid in the logs directory, and the results of the sampler in the results directory. GATEWAY MATLAB SCRIPT For the purposes of this tutorial, you don't need to know anything specific about the files buried in different subdirectories. We concentrate on the main Matlab script driver_LDA.m. The first section of the code looks like this: function [] = driver_LDA( D, K )% Main Function to Execute LDA Gibbs Sampler% on abstracts from Psych Review journal% ...% Author: Mike Hughes (mhughes@cs.brown.edu)% Date: 3 March 2011% **** END USER MUST MODIFY THESE TWO LINES! ****datafilepath = '/home/mhughes/mex/mcc/data/bagofwords_psychreview.mat';resultsDir = '/home/mhughes/research/results/mccExp/';if ( ~isdeployed && ~ismcc ) addpath( genpath( '.' ) );endif strcmp( class(D), 'char' ) Params.D = str2double(D);elseif strcmp( class(D), 'double' ) Params.D = D;endif strcmp( class(K), 'char' ) Params.K = str2double(K);elseif strcmp( class(K), 'double' ) Params.K = K;end... We notice a few important lessons that can generalize to any Matlab application we wish to deploy.
RUNNING THE LDA GIBBS SAMPLER IN MATLAB It's probably best to make sure the code runs fine in Matlab first before we try to compile it to C. As a demo, we call the function driver_LDA from the Matlab command line, passing arguments indicating to run the sampler on D=500 documents in the dataset and to discover K=10 topics. >> driver_LDA( 500, 10)Params = D: 500 K: 10 V: 9244============= Run = 1/2 =================Begin MEX LDA Sampler :10 iters completed iter 1/10 after 0.11 sec completed iter 2/10 after 0.23 sec completed iter 3/10 after 0.36 sec completed iter 4/10 after 0.49 sec completed iter 5/10 after 0.62 sec completed iter 6/10 after 0.75 sec completed iter 7/10 after 0.88 sec completed iter 8/10 after 1.00 sec completed iter 9/10 after 1.14 sec completed iter 10/10 after 1.26 sec .................................... saved to file /home/mhughes/mex/mcc/results/K10/Samples_01.mat============= Run = 2/2 =================Begin MEX LDA Sampler :10 iters completed iter 1/10 after 0.11 sec completed iter 2/10 after 0.24 sec completed iter 3/10 after 0.37 sec completed iter 4/10 after 0.50 sec completed iter 5/10 after 0.62 sec completed iter 6/10 after 0.75 sec completed iter 7/10 after 0.88 sec completed iter 8/10 after 1.01 sec completed iter 9/10 after 1.13 sec completed iter 10/10 after 1.25 sec .................................... saved to file /home/mhughes/mex/mcc/results/K10/Samples_02.matThe results were saved to the results folder. COMPILING VIA MCC Our goal is to build -- in a separate directory -- a standalone C application that implements the functionality of driver_LDA.m. We'll call this standalone directory mccLDA/. Make sure this directory exists and is empty. We use the shell script buildMCC.sh to compile the application, since we need to give mcc lots of custom arguments. This script looks as follows: (don't worry, all but one line are comments) #!/bin/bash# Script to build Matlab stand-alone application for LDA Gibbs Sampler# NB: Make sure to run the following before trying to execute this script# >> chmod +x buildMCC.shmcc -m -v -I './data/' -I './LDAutil/' -I './mex/' -R '-nojvm, -nodesktop, -nosplash' -d 'mccLDA' driver_LDA.m# Explanation of option flags for mcc# -v = verbose mode # -d <outdir> = create all output files in specific directory <outdir>. must exist beforehand.# -m = create stand-alone C application # -I <mdir> = add the directly <mdir> to search path when looking for necessary .m files# -R <matlabopt> = set the C application to use the matlab startup option <matlabopt> by default# in particular, the following are helpful options:# -nojvm = disables java virtual machine (which shouldn't be necessary for grid jobs# -nodesktop = tells matlab to not be a desktop GUI application# -nosplash = disables startup "splash" screen with matlab logo on startup # -? = displays mcc helpOne key observation is that we need to tell mcc explicitly where all the dependent .m files and .mex files are via the -I option flag. This could get complicated for huge applications, but a good solution is just to use Matlab's built in genpath command at the root directory of the project. >> genpath('.')ans =.:./LDAutil:./data:./mccLDA:./mex:Feeding the resulting string to the -I flag should be just fine. RUNNING THE C APPLICATION THROUGH THE TERMINAL MCC generates a shell script called "run_driver_LDA.sh" that is all we need to run our app. Its first argument is the location of the Matlab default libraries. The remaining args are the arguments D,K for our original Matlab function driver_LDA.m. Below is an example interactive terminal session running this app on a single 64-bit machine. bench6 ~/mex/mcc $ cd mccLDAbench6 ~/mex/mcc/mccLDA $ lsdriver_LDA* driver_LDA_main.c mccExcludedFiles.log run_driver_LDA.sh*driver_LDA.prj driver_LDA_mcc_component_data.c readme.txt submit_LDA_jobs.shbench6 ~/mex/mcc/mccLDA $ ./run_driver_LDA.sh /local/projects/matlab/R2010a/ 500 10------------------------------------------Setting up environment variables---LD_LIBRARY_PATH is .:/local/projects/matlab/R2010a//runtime/glnxa64:/local/projects/matlab/R2010a//bin/glnxa64:/local/projects/matlab/R2010a//sys/os/glnxa64:/local/projects/matlab/R2010a//sys/java/jre/glnxa64/jre/lib/amd64/native_threads:/local/projects/matlab/R2010a//sys/java/jre/glnxa64/jre/lib/amd64/server:/local/projects/matlab/R2010a//sys/java/jre/glnxa64/jre/lib/amd64/client:/local/projects/matlab/R2010a//sys/java/jre/glnxa64/jre/lib/amd64Params = D: 500 K: 10 V: 9244============= Run = 1/2 =================Begin MEX LDA Sampler :10 iters completed iter 1/10 after 0.57 sec completed iter 2/10 after 1.19 sec completed iter 3/10 after 1.78 sec completed iter 4/10 after 2.37 sec completed iter 5/10 after 2.96 sec completed iter 6/10 after 3.54 sec completed iter 7/10 after 4.13 sec completed iter 8/10 after 4.71 sec completed iter 9/10 after 5.30 sec completed iter 10/10 after 5.88 sec .................................... saved to file /home/mhughes/mex/mcc/results/K10/Samples_01.mat============= Run = 2/2 =================Begin MEX LDA Sampler :10 iters completed iter 1/10 after 0.56 sec completed iter 2/10 after 1.16 sec completed iter 3/10 after 1.72 sec completed iter 4/10 after 2.33 sec completed iter 5/10 after 2.89 sec completed iter 6/10 after 3.46 sec completed iter 7/10 after 4.05 sec completed iter 8/10 after 4.64 sec completed iter 9/10 after 5.24 sec completed iter 10/10 after 5.81 sec .................................... saved to file /home/mhughes/mex/mcc/results/K10/Samples_02.matRUNNING THE C APPLICATION ON THE GRID To get this C application to run on the Sun Grid Engine (SGE), we must have some helper scripts to (1) submit a bunch of jobs to the grid and (2) instruct the grid what commands to execute for each job. 1) Script that submits a bunch of jobs to the grid. The script submit_LDA_jobs.sh contains the following code: #! /bin/sh# Bash script for sumbitting matlab jobs to an SGE cluster queue.# This is the script you run from the command line to submit the jobs.# Original script skeleton from David Black-Schaffer# http://cva.stanford.edu/people/davidbbs/cluster/matlab/# Use the default settings configured for Brown University's grid # (non-Brown people definitely don't need this)source /com/sge/default/common/settings.sh# Submit 3 jobs to the gridqsub -m bea -l arch=lx24-amd64 -t 1-3 grid_driver_LDA.sh 500# Option explanations:# Mail Option (-m bea): trigger email alerts at beginning, end, and aborting of jobs# Arch Option: (-l arch=...): make sure 64 bit machines only# Batch option: (-t 1-3): submit array of jobs, each numbered 1,2, or 3This script submits (via qsub command) 3 jobs labeled ("1","2","3") that each execute the job script grid_driver_LDA.sh with input argument D=500. Here at Brown we run this script on the terminal after ssh-ing into the head grid node sge, but other users will need to consult with their grid system admin to figure out what to do. 2) Script that defines what each job should execute The script grid_driver_LDA.sh sets up each grid job and launches our standalone application with the appropriate arguments. The script assigns a different number of topics to each of the 3 jobs initialized by the above code. We denote K as the number of topics parameter. It should be 10, 20, and the 30 for jobs 1, 2, and 3 respectively. Note that it's better to assign input parameters based on SGE_TASK_ID in this wrapper shell script than in the Matlab code, since we don't want to recompile the entire MCC application just to change the range of input parameters. The full code contents of grid_driver_LDA.sh are listed here: #!/bin/bash #$ -S /bin/sh #$ -cwd #$ -r y #$ -q long.q -l arch=lx24-amd64 # put stdout and stderr files in the right place for your system.# NOTE that $TASK_ID is the correct var here, but not in rest of script (need SGE_TASK_ID)#$ -o logs/$JOB_ID.$TASK_ID.out#$ -e logs/$JOB_ID.$TASK_ID.errecho "JOB ID: " $JOB_IDecho "SGE TASK ID: "$SGE_TASK_ID# Assign number of topics K based on task number assigned by SGE# remember that "=" as assignment operator CANNOT be surrounded by spacesKs=(10 20 30 40)K=${Ks[$SGE_TASK_ID-1]}echo "K: " $K# Run the MCC main script # make sure the first arg is the path to the matlab libraries# remaining arguments are:# the D value passed to this script# the K value we computed based on the task numbermccLDA/run_driver_LDA.sh /local/projects/matlab/R2010a $1 $KexitThe first section of this script sets some options for SGE. Most of these aren't too interesting. The most important thing is that it directs each job to send its standard output and standard error feeds to files with names of the form: path/to/working/dir/logs/<JOB_ID>.<TASK_ID>.out path/to/working/dir/logs/<JOB_ID>.<TASK_ID>.err I find this much easier to use than the default log files created by SGE, which get dumped directly in the current working directory en masse. 3) Actually running a batch of jobs on the grid With the above scripts and a fully compiled MCC application, we can finally use the grid! Here's a snapshot of my terminal session on the head grid node sge to launch the jobs. sge ~/mex/mcc $ pwd/home/mhughes/mex/mccsge ~/mex/mcc $ lsLDAutil/ data/ logs/ mex/ submit_LDA_jobs.sh*buildMCC.sh* driver_LDA.m grid_driver_LDA.sh* mccLDA/ results/sge ~/mex/mcc $ ./submit_LDA_jobs.shYour job-array 8385129.1-3:1 ("grid_driver_LDA.sh") has been submittedThat's all the feedback you get from the grid launching. The log files contain much more information about the progress of each job. We can check that our tasks were run with the correct parameters by examining the first three lines of each log file. We should see that K is set to 10, 20, and 30 on tasks 1, 2, and 3. sge ~/mex/mcc $ head -n3 ./logs/8385129.1.outJOB ID: 8385129SGE TASK ID: 1K: 10sge ~/mex/mcc $ head -n3 ./logs/8385129.2.outJOB ID: 8385129SGE TASK ID: 2K: 20sge ~/mex/mcc $ head -n3 ./logs/8385129.3.outJOB ID: 8385129SGE TASK ID: 3K: 30Finally, we can examine the complete standard output of a job to make sure its working properly. sge ~/mex/mcc $ cat ./logs/8385129.3.outJOB ID: 8385129SGE TASK ID: 3K: 30------------------------------------------Setting up environment variables---LD_LIBRARY_PATH is .:/local/projects/matlab/R2010a/runtime/glnxa64:/local/projects/matlab/R2010a/bin/glnxa64:/local/projects/matlab/R2010a/sys/os/glnxa64:/local/projects/matlab/R2010a/sys/java/jre/glnxa64/jre/lib/amd64/native_threads:/local/projects/matlab/R2010a/sys/java/jre/glnxa64/jre/lib/amd64/server:/local/projects/matlab/R2010a/sys/java/jre/glnxa64/jre/lib/amd64/client:/local/projects/matlab/R2010a/sys/java/jre/glnxa64/jre/lib/amd64Warning: No display specified. You will not be able to display graphics on the screen.SGE_TASK_ID = 3Params = D: 500 K: 30 V: 9244============= Run = 1/2 =================Begin MEX LDA Sampler :10 iters completed iter 1/10 after 0.70 sec completed iter 2/10 after 1.46 sec completed iter 3/10 after 2.19 sec completed iter 4/10 after 2.93 sec completed iter 5/10 after 3.65 sec completed iter 6/10 after 4.38 sec completed iter 7/10 after 5.12 sec completed iter 8/10 after 5.89 sec completed iter 9/10 after 6.62 sec completed iter 10/10 after 7.33 sec .................................... saved to file /home/mhughes/mex/mcc/results/K30/Samples_01.mat============= Run = 2/2 =================Begin MEX LDA Sampler :10 iters completed iter 1/10 after 0.67 sec completed iter 2/10 after 1.42 sec completed iter 3/10 after 2.12 sec completed iter 4/10 after 2.87 sec completed iter 5/10 after 3.61 sec completed iter 6/10 after 4.34 sec completed iter 7/10 after 5.08 sec completed iter 8/10 after 5.81 sec completed iter 9/10 after 6.57 sec completed iter 10/10 after 7.28 sec .................................... saved to file /home/mhughes/mex/mcc/results/K30/Samples_02.matAlternatively, we can inspect the contents of all subdirectories of our results directory: sge ~/mex/mcc $ ls -R resultsresults:K10/ K20/ K30/results/K10:Samples_01.mat Samples_02.matresults/K20:Samples_01.mat Samples_02.matresults/K30:Samples_01.mat Samples_02.matLooks like everything's there. We have 2 sampler runs saved for each value of our parameter K. That's it! We've successfully deployed a standalone Matlab application on the grid. |
Tutorials / How To >
