Difference between revisions of "Cray"

From Tau Wiki
Jump to: navigation, search
 
(→‎GPU performance tracking)
 
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
This document describes how to configure TAU and PDT on Cray XT3/RedStorm. It assumes that the backend nodes are running Catamount OS.
+
<!--
 +
 
 +
This document describes how to configure TAU and PDT on Cray XT3/RedStorm or Cray CNL. Choose the subsection that applies to you. Both cases assumes that the backend nodes are running Catamount OS.
 +
 
 +
==Cray XT3==
  
 
1. Configuring PDT
 
1. Configuring PDT
Line 29: Line 33:
  
 
Which builds binaries in the x86_64/bin/ directory which may be executed on the front-end.
 
Which builds binaries in the x86_64/bin/ directory which may be executed on the front-end.
 +
 +
==Cray CNL==
 +
 +
===with PGI compilers===
 +
 +
 +
1. Configuring PDT
 +
 +
<pre> % ./configure -GNU
 +
% make clean install </pre>
 +
 +
Installs PDT binaries in x86_64/bin directory.
 +
 +
2. Installing TAU
 +
 +
It is important to note that executables that are created for the backend nodes do not run on the XT3 front-end. Hence, it is important to compile TAU with -pdt_c++=g++ so that tau_instrumentoris built for the front-end as it is required for compilation. To install TAU:
 +
 +
<pre> % ./configure -arch=craycnl -pdt=<dir> -pdt_c++=g++ -mpi
 +
% make clean install </pre>
 +
 +
Builds xt3/lib/Makefile.tau-mpi-pdt-pgi stub Makefile. Configuring in this manner builds the TAUlibraries in xt3/lib directory.
 +
 +
To use SHMEM on XT3, we use:
 +
 +
<pre> % ./configure -arch=craycnl -pdt=<dir> -pdt_c++=g++ -shmem
 +
% make clean install </pre>
 +
 +
To build other tools (such as pprof, tau_merge, tau2vtf) that execute on the front-end, we install TAU using:
 +
 +
<pre> % ./configure
 +
% make clean install
 +
</pre>
 +
 +
Which builds binaries in the x86_64/bin/ directory which may be executed on the front-end.
 +
 +
===with PathScale compilers===
 +
 +
1. Configuring PDT
 +
 +
<pre> % ./configure -GNU
 +
% make clean install </pre>
 +
 +
Installs PDT binaries in x86_64/bin directory.
 +
 +
2. Installing TAU
 +
 +
It is important to note that executables that are created for the backend nodes do not run on the XT3 front-end. Hence, it is important to compile TAU with -pdt_c++=g++ so that tau_instrumentoris built for the front-end as it is required for compilation. To install TAU:
 +
<pre>
 +
module unload PrgEnv-pgi
 +
module load pathscale/3.0
 +
module load xt-pe/2.0.33
 +
module load xt-mpt-pathscale/2.0.33
 +
module load xt-libsci/10.2.0
 +
module load PrgEnv-pathscale/2.0.33
 +
 +
./configure -fortran=pathscale -c++=CC -cc=pathcc -arch=craycnl -pdt=/spin/proj/perc/TOOLS/pdt/pdtoolkit-3.12 -pdt_c++=g++ -mpi
 +
</pre>
 +
-->
 +
== Cray XK6/XK7 ==
 +
 +
=== GPU performance tracking ===
 +
 +
1. Configuring TAU:
 +
module load cudatoolkit
 +
./configure -arch=craycnl -cuda="$CRAY_CUDATOOLKIT_DIR" -cudalibrary="$CRAY_CUDATOOLKIT_POST_LINK_OPTS" -bfd=download
 +
 +
Setup your environment:
 +
 +
export PATH=<path to tau2>/craycnl/bin:$PATH
 +
export LD_LIBRARY_PATH=<path to tau2>/craycnl/lib:$LD_LIBRARY_PATH
 +
 +
2. CUDA
 +
 +
Build as normally would, and modify your run command to be:
 +
 +
aprun -N 1 tau_exec -T serial,cupti -cupti ./matmult
 +
 +
3. OpenACC
 +
 +
Remember to load the cray acc module:
 +
 +
module load  craype-accel-nvidia20
 +
 +
Both PGI and Cray compilers uses the CUDA driver API to interact with the GPU, so to setup TAU to collect those calls:
 +
 +
export TAU_CUPTI_API=driver
 +
 +
Compile as normally would and run with tau_exec:
 +
 +
aprun -N 1 tau_exec -T serial,cupti -cupti ./matmult
 +
 +
4. Viewing profiles
 +
 +
TAU profile are written to disk as '''profile.*''' (you may have several files.) You can view TAU profiles either through '''pprof''' (text-basied) or '''paraprof''' (GUI).
 +
 +
5. Tracing
 +
 +
Traces can be capture by setting:
 +
 +
export TAU_TRACE=1
 +
 +
before running your application. The traces need to be post-processed as well, issue these commands:
 +
 +
aprun -N 1 tau_exec -T serial,cupti -cupti ./matmult
 +
tau_multimerge
 +
tau2slog2 tau.trc tau.edf -o matmult.slog2
 +
jumpshot matmult.slog2
 +
 +
Jumpshot is a common Trace visualizer bundled with TAU.

Latest revision as of 18:21, 25 October 2012

Cray XK6/XK7

GPU performance tracking

1. Configuring TAU: 

module load cudatoolkit
./configure -arch=craycnl -cuda="$CRAY_CUDATOOLKIT_DIR" -cudalibrary="$CRAY_CUDATOOLKIT_POST_LINK_OPTS" -bfd=download

Setup your environment: 

export PATH=<path to tau2>/craycnl/bin:$PATH
export LD_LIBRARY_PATH=<path to tau2>/craycnl/lib:$LD_LIBRARY_PATH

2. CUDA

Build as normally would, and modify your run command to be: 

aprun -N 1 tau_exec -T serial,cupti -cupti ./matmult

3. OpenACC

Remember to load the cray acc module: 

module load  craype-accel-nvidia20

Both PGI and Cray compilers uses the CUDA driver API to interact with the GPU, so to setup TAU to collect those calls: 

export TAU_CUPTI_API=driver

Compile as normally would and run with tau_exec: 

aprun -N 1 tau_exec -T serial,cupti -cupti ./matmult

4. Viewing profiles

TAU profile are written to disk as profile.* (you may have several files.) You can view TAU profiles either through pprof (text-basied) or paraprof (GUI).

5. Tracing

Traces can be capture by setting: 

export TAU_TRACE=1

before running your application. The traces need to be post-processed as well, issue these commands: 

aprun -N 1 tau_exec -T serial,cupti -cupti ./matmult
tau_multimerge
tau2slog2 tau.trc tau.edf -o matmult.slog2
jumpshot matmult.slog2

Jumpshot is a common Trace visualizer bundled with TAU.

Retrieved from "http://tau.uoregon.edu/mediawiki-tau/index.php?title=Cray&oldid=2040"