Scratch Space
Scratch is a dedicated file system used as a temporary working directory for active jobs. If your jobs create large temporary files, or need a lot of space while they run, this is what you'll want to use.
By default, all jobs are automatically provisioned space that is accessible through the environment variable $TMPDIR. Once the job completes, any files that are created will be purged.
Local Scratch
This is the default option. While the most performant, it is limited on available space that is shared amongst all jobs on the node. Review the hardware specs on BOSE and BGSC for amounts available on each machine.
[myuser@bose ~]$ sinteract --ntasks=4
# -- interactive session is launched -- #
[myuser@cn09 ~]$ echo $TMPDIR
/local/scratch/job_66575
Network Scratch
If you start your sbatch, sinteract, or sdevelop commands with USE_NETWORK_SCRATCH=1, it'll tell the automatic scratch system to use network scratch instead. This is a shared space that provisions a single job folder accessible by every node and is not limited on available space. This is recommended when working with MPI-based jobs that use multiple nodes.
Batch ("sbatch") Mode
Add the line #SBATCH --export=ALL,USE_NETWORK_SCRIPT=1 to your sbatch script.
#SBATCH --job-name="My Job"
#SBATCH --ntasks=1
#SBATCH --mem=5G
#SBATCH --export=ALL,USE_NETWORK_SCRATCH=1 # Add This Line
echo "My scratch directory is $TMPDIR"
Results in: My scratch directory is /network/scratch/job_12345
You can also temporarily change it at any time when you call 'sbatch', which will only apply for that single job submission.
Interactive Mode
[myuser@bose ~]$ USE_NETWORK_SCRATCH=1 sinteract --ntasks=4
# -- interactive session is launched -- #
[myuser@cn09 ~]$ echo $TMPDIR
/network/scratch/job_66575
Using Scratch
To use the scratch space, start off by copying any files (such as input or data files) to your temporary working space. Then, if needed, you can change your directory to that space and work out of it.
cp my-files.input $TMPDIR/ # Copy input file to scratch
cd $TMPDIR # Change location to the scratch space
myprogram < my-files.input # Run your program
After your main program runs, make sure to copy back any files to your personal or group folder that you'd like to keep. This is important as once the job finishes, it'll automatically purge all files that were available under $TMPDIR.
Using Gaussian or Q-Chem?
Gaussian and Q-Chem are set up to automatically use scratch space for any of their temporary working files. By default they use local scratch, unless you start your job submissions with USE_NETWORK_SCRATCH=1.