Execution of mig scripts on cheaha

This section details the execution of mig script, the R-Methodological Analysis script given by Tapan Mehta of SSG to John-Paul on cheaha

  • Following the instructions on http://me.eng.uab.edu/wiki/index.php?title=R-userinfo to run example R-scripts, copied the .bashrc onto my .bashrc in cheaha
  • Copied the mig-scripts on cheaha (~/mig)
  • The shell script that actually submits the R-job to SGE is arrayjobsge in ~/mig/scripts directory and the R-scripts were located in the ~/mig/R directory. So copied the arrayjobsge to ~/mig/R directory.
  • Since, execution of the actual mig script takes usually 1 day to complete, I worked with a minimal version of the scripts.
    The minimal version of the R-script MigAnalysis?.R consisted of
         #Move toward a production version that will call the appropriate
         #functions in a loop.
         library(nlme)
         #source("SimBayes.R")
         #source("ExtractData.R")
         #source("EMFull.R")
         #source("MIFull.R")
    
         allArgs <- commandArgs(FALSE)
         migArgs <- unlist(strsplit(allArgs[9],split=":"))
         print(migArgs)
    
    The input data file ~/mig/R/mig-pars-10.txt contains just one line when compared with the original
         10:0.1:1000                10:0.1:15
         10:0.1:2000
         10:0.1:3000
         10:0.1:4000
         10:0.1:5000
         10:0.1:6000
         10:0.1:7000
         10:0.1:8000
         10:0.1:9000
         10:0.1:10000
         10:0.2:1000
         10:0.2:2000
         10:0.2:3000
         10:0.2:4000
         10:0.2:5000
         10:0.2:6000
         10:0.2:7000
         10:0.2:8000
         10:0.2:9000
         10:0.2:10000
         10:0.3:1000
         10:0.3:2000
         10:0.3:3000
         10:0.3:4000
         10:0.3:5000
         10:0.3:6000
         10:0.3:7000
         10:0.3:8000
         10:0.3:9000
         10:0.3:10000
    
    and I slightly modified the job submission script, arrayjobsge to contain a single task numbered 2-2 as opposed to 2-30 in the original script
          #!/bin/bash                                                                                                #!/bin/bash
          #$ -S /bin/bash                                                                                            #$ -S /bin/bash
          #$ -t 2-30                                                                                                 #$ -t 2-2
          #$ -V                                                                                                      #$ -V
          #$ -N mig10                                                                                                #$ -N mig10
          #$ -cwd                                                                                                    #$ -cwd 
          # -m beas                                                                                                  # -m beas
          # -M ppreddy@uab.edu                                                                                       # -M ppreddy@uab.edu                                                                                    
          #$ -l h_rt=1000:00:00                                                                                      #$ -l h_rt=1000:00:00
          #$ -e ./sge-error/mig-err.$TASK_ID                                                                         #$ -e ./sge-error/mig-err.$TASK_ID                   
          #$ -o ./sge-output/mig-out.$TASK_ID                                                                        #$ -o ./sge-output/mig-out.$TASK_ID
    
          SEEDFILE=mig-pars-10.txt                                                                                   SEEDFILE=mig-pars-10.txt
          SEED=$(cat $SEEDFILE | head -n $SGE_TASK_ID | tail -n 1)                                                   SEED=$(cat $SEEDFILE | head -n $SGE_TASK_ID | tail -n 1) 
          echo $SEED                                                                                                 echo $SEED 
          R CMD BATCH --no-save --no-restore --$SEED MigAnalysis.R MigStdOut.$SGE_TASK_ID                            R CMD BATCH --no-save --no-restore --$SEED MigAnalysis.R MigStdOut.$SGE_TASK_ID
    
  • So with the above changes, submitted the R-job to cheaha through SGE command
    qsub arrayjobsge
  • The result is an Error
    cat :10:0.1:15: No such file or directory
  • Tried a few things (unsuccessfully) to overcome the above error. The mail describing this in r-group is as follows:
         1. I successfully ran the R CMD BATCH --no-save --no-restore mig-pars-10.txt MigAnalysis.R MigStdOut.$SGE_TASK_ID from the OS-Shell directly rather than submitting it to SGE in a shell script
         2. I just executed lines 13 to 15 of the "arrayjobsge" on the shell prompt, it runs fine and I get no "cat 10:0.1:15 No such file or directory error"
         3. I tried changing the bash path and no success.
         4. I tried not using the variable $SEEDFILE and just putting the mig-params... filename directly into the line 14 of "arrayjobsge"
    
            13. SEEDFILE=mig-pars-10.txt 
            14. SEED=$(cat mig-pars-10.txt | head -n $SGE_TASK_ID | tail -n 1)
            15. echo $SEED
            16. R CMD BATCH --no-save --no-restore $SEED MigAnalysis.R MigStdOut.$SGE_TASK_ID
    
            but the same response: cat 10:0.1:15 No such file or directory  
         5. John Paul's suggestion
            "try replacing the whole of the subcommand on line 14 with just the value from the params file you intend to use, ie. a static assignment. Do this to get rid of this cat command entirely to make sure this is the 
             line causing the problem and not some subtle other error"
    
             I did this, and to my surprise, the same error: "cat 10:0.1:15: No such file or directory. So , it wasn't this assignment and pipelining "$(cat $SEEDFILE | head -n $SGE_TASK_ID | tail -n 1)" that was causing the   
             problem? ~/mig/R/arrayjobsge
             {{{
             #!/bin/bash
             #$ -S /bin/bash
             #$ -t 2-2
             #$ -V
             #$ -N mig10
             #$ -cwd
             # -m beas
             # -M ppreddy@uab.edu
             #$ -l h_rt=1000:00:00
             #$ -v PATH,R_HOME,R_LIBS,LD_LIBRARY_PATH,CWD
    
             SEED='10:0.1:15'
             echo $SEED
             R CMD BATCH --no-save --no-restore $SEED MigAnalysis.R MigStdOut.$SGE_TASK_ID
             }}} 
             I did a grep -w 'cat' * in the ~/mig/R directory and I get:
             {{{ 
             EMFull.R:##cat("Missing data pattern:\n")
             EMFull.R:##cat("Basic descriptive statistics\nMean\n")
             EMFull.R:##cat("Standard deviations\n")
             EMFull.R:##cat("\n\nCovariance matrix\n")
             EMFull.R:##cat("Correlation matrix\n")
             EMFull.R:if(convrg==0){cat("Iteration limit exceeded!!!!\n")}
             EMFull.R:#cat("Number of iterations: ",iter,"\n")
             EMFull.R:##cat("Summary statistics:\n")
             EMFull.R:##cat("EM Covariance:\n")
             EMFull.R:##cat("EM Correlation:\n")
             mig10.e59004.2:cat: 10:0.1:15: No such file or directory
             }}}
             I tried SEED="10:0.1:15" (in double-quotes) the same error
           6. I read the bash man page and command substitution had two options $(command ) or `command`. The command substitution $(cat file) can be replaced by the equivalent but faster $(< file)
    
              I tried `command` - no difference
              I tried replacing cat with $(< file) --- I definitely do not get the cat error, but the data in the file is not getting parsed either
    
  • Finally, Tapan Mehta replied through r-group that the ~/.bashrc that I might be using may not be the current one as used by the SSG to run the same script.
  • So, copied tapan's .bashrc. The key difference was in the R_VERSION in addition to JAVA_PATH and CLASS_PATH. The .bashrc that I had had an R_VERSION=2.3.1 and tapan's was 2.6.2
  • So, after copying tapan's .bashrc and submitting the job, the cat error was gone
  • Following John-Paul's suggestion to test the source of the cat error, I did two things
  • First, I reverted the R_VERSION to 2.3.1 and executed the job script arrayjobsge and I get the cat error.
  • but when I commented out the "R CMD BATCH.........." and submitted the script again, I get no cat error (with the R_VERSION = 2.3.1)
  • So, doing the above two steps, it was concluded that the R Version had somehow got to do with the cat error.

Execution of mig scripts on stage with Gridway

This section details the execution of mig script, the R-Methodological Analysis script given by Tapan Mehta of SSG to John-Paul on stage