User Tools

Site Tools


matlab_usage

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
matlab_usage [2016/03/04 22:16]
mgstauff [Parallel Toolbox Considerations]
matlab_usage [2018/02/06 20:44] (current)
mgstauff [Matlab Search Path]
Line 5: Line 5:
 If you need to replace the default CfN settings in your startup.m file, see /share/admin/matlab-startup-common.m on chead. If you need to replace the default CfN settings in your startup.m file, see /share/admin/matlab-startup-common.m on chead.
  
 +==== Matlab Search Path ====
 +To customize the Matlab search path, change your ~/matlab/startup.m file, or use the MATLABPATH environment variable:
 +
 +https://www.mathworks.com/help/matlab/matlab_env/add-folders-to-matlab-search-path-at-startup.html
 +
 +Add the definition to your ~/.bash_profile file to have it load whenever you login.
 ==== Matlab via qsub batch jobs==== ==== Matlab via qsub batch jobs====
 You may need to run Matlab scripts in a qsub job. It's fairly easy. This command will run a series of Matlab commands from the command line (shell), which we can use in a qsub job script: You may need to run Matlab scripts in a qsub job. It's fairly easy. This command will run a series of Matlab commands from the command line (shell), which we can use in a qsub job script:
Line 38: Line 44:
 Matlab is known to be greedy about how many CPU cores it uses. When it sees, for example, 16 cores on the machine where it's running, it will generally use 16 threads for its own code that is multi-threaded. The cluster will automatically set the number of threads for matlab to use according to how many cores/slots you requested for your job. This is done in two ways: Matlab is known to be greedy about how many CPU cores it uses. When it sees, for example, 16 cores on the machine where it's running, it will generally use 16 threads for its own code that is multi-threaded. The cluster will automatically set the number of threads for matlab to use according to how many cores/slots you requested for your job. This is done in two ways:
  
-* Single core/slot: a wrapper around the 'matlab' routine will pass the ''-singleCompThread'' option to matlab at startup. This is a reliable way to ensure you only use one thread in matlab.+  * Single core/slot: a wrapper around the 'matlab' routine will pass the ''-singleCompThread'' option to matlab at startup. This is a reliable way to ensure you only use one thread in matlab.
  
-* Multiple cores/slots: your startup.m file calls a script that uses ''maxNumCompThreads'' to set the number of threads to match the number of cores/slots for your job. *However*, this is unreliable because matlab will ignore this in some cases.+  * Multiple cores/slots: your startup.m file calls a script that uses ''maxNumCompThreads'' to set the number of threads to match the number of cores/slots for your job. **However**, this is unreliable because matlab will ignore this in some cases - see below.
  
-*WARNING*+**WARNING**
 If you notice matlab running very slowly during heavy computation (e.g. much slower than when you prototyped on your own machine), it may be because matlab is using more threads than you have cores/slots assigned to your job.  If you notice matlab running very slowly during heavy computation (e.g. much slower than when you prototyped on your own machine), it may be because matlab is using more threads than you have cores/slots assigned to your job. 
  
 In this case, the slowdown happens because SGE 'binds' your job to a set number of cores on the compute node so it doesn't take more computing resources than it's been allocated. But matlab sees more cores on the node so allocates more computational threads. These threads then are all competing on a smaller number of cores for processing time and memory access, slowing things down.  In this case, the slowdown happens because SGE 'binds' your job to a set number of cores on the compute node so it doesn't take more computing resources than it's been allocated. But matlab sees more cores on the node so allocates more computational threads. These threads then are all competing on a smaller number of cores for processing time and memory access, slowing things down. 
  
-What to do: you should request more cores to lessen the computational bottleneck. Anecdotally, 4-8 cores may relieve the problem. *Or*, you can request a single core for your job, and matlab will be put into single-thread mode and will not try to use multiple computational threads. This latter approach is particularly good for large batch jobs.+What to do: you should request more cores to lessen the computational bottleneck. Anecdotally, 4-8 cores may relieve the problem. **Or**, you can request a single core for your job, and matlab will be put into single-thread mode and will not try to use multiple computational threads. This latter approach is particularly good for large batch jobs.
  
 +See more discussion in [[using_ogs_sge#parallel-processing_multi-threading|details of parallel processing]].
  
 ==== Parallel Toolbox Considerations ==== ==== Parallel Toolbox Considerations ====
Line 54: Line 61:
 === Number of Workers === === Number of Workers ===
  
-When using Matlab's Parallel Toolbox, you generally must specify the number of workers to match the number of OGS/SGE slots requested by your job. To programmatically get the number of slots you requested, you can use the ''NSLOTS'' shell environment variable, or the matlab 'maxNumCompThreads' command.+When using Matlab's Parallel Toolbox, you generally must specify the number of workers to match the number of SGE slots requested by your job. To programmatically retrieve the number of slots/cores you requested for the current job, you can use the ''NSLOTS'' shell environment variable, or the matlab 'maxNumCompThreads' command.
  
 NOTE: Your jobs (qsub or qlogin) run on a single compute node for easier parallel processing, and each compute node has only 16 cores. So you don't want to ask for more than 16 slots (and thus matlab workers) for a job. But also note that asking for 16 slots meaning waiting for a compute node to be completely free before you can start your job. NOTE: Your jobs (qsub or qlogin) run on a single compute node for easier parallel processing, and each compute node has only 16 cores. So you don't want to ask for more than 16 slots (and thus matlab workers) for a job. But also note that asking for 16 slots meaning waiting for a compute node to be completely free before you can start your job.
matlab_usage.1457129808.txt.gz ยท Last modified: 2016/03/04 22:16 by mgstauff