Difference between revisions of "NEK"
| (3 intermediate revisions by the same user not shown) | |||
| Line 25: | Line 25: | ||
For TAU: | For TAU: | ||
| − | ./configure -cc=pgcc -c++=pgCC -fortran=pgi -mpi -pdt=/lustre/medusa/biersdor/pdtoolkit-3.17/ -pdt_c++=g++ | + | ./configure -cc=pgcc -c++=pgCC -fortran=pgi -mpi -mpilibrary=-lpgbind -pdt=/lustre/medusa/biersdor/pdtoolkit-3.17/ -pdt_c++=g++ |
make install | make install | ||
| Line 56: | Line 56: | ||
export TAU_OPTIONS="-optPreProcess -optVerbose -optRevert -optTauSelectFile=`pwd`/select.tau" | export TAU_OPTIONS="-optPreProcess -optVerbose -optRevert -optTauSelectFile=`pwd`/select.tau" | ||
| − | '''select.tau''' | + | '''select.tau''' |
BEGIN_FILE_EXCLUDE_LIST | BEGIN_FILE_EXCLUDE_LIST | ||
| Line 69: | Line 69: | ||
./makenek r1854a | ./makenek r1854a | ||
make -f makefile | make -f makefile | ||
| − | There is a auto-generated '''makefile''', any further changes can be made to this file. | + | |
| + | There is a auto-generated '''makefile''', any further changes can be made to this file. For example make sure that '''-r8''' option is passed to the compiler. | ||
| + | |||
| + | == Running NEK == | ||
| + | |||
| + | If compiling was successful you should have '''nek5000' executable. | ||
| + | |||
| + | Use the '''nekbmpi''' script to run the example: | ||
| + | |||
| + | ./nekbmpi r1854a <# of procs> | ||
| + | |||
| + | On 8 processors this example takes just less than one hour. | ||
| + | |||
| + | == Performance Data == | ||
| + | |||
| + | Mean breakdown: | ||
| + | |||
| + | [[Image:nek-profile.png|750px]] | ||
| + | |||
| + | Major MPI routines across each node: | ||
| + | |||
| + | [[Image:nek-mpi-profile.png|750px]] | ||
| + | |||
| + | Profile: | ||
| + | |||
| + | [[Image:nek_vortex_8p.ppk]] | ||
Latest revision as of 19:06, 29 February 2012
Background
| Link | Code Version | Machine | Date |
|---|---|---|---|
| SVN repo | svn February 2012 | Keeneland | February 2012 |
Build TAU/PDT
For PDT:
./configure ; make ; make install
For TAU:
./configure -cc=pgcc -c++=pgCC -fortran=pgi -mpi -mpilibrary=-lpgbind -pdt=/lustre/medusa/biersdor/pdtoolkit-3.17/ -pdt_c++=g++ make install
Building NEK
use the pgi modules:
module load PE-pgi openmpi/1.5.1-pgi
As an example we will build and run for the 3D vortex example:
cd examples/vortex cp ../../trunk/nek/makenek . cp ../../trunk/tools/scripts/nekbmpi .
This gives us the scripts needed to build and run NEK.
First edit makenek and change the compiler to the TAU wrapper scripts:
# source path SOURCE_ROOT="<path to trunk/nek>" # Fortran compiler F77="tau_f90.sh" # C compiler CC="tau_cc.sh"
Setup your environment:
export TAU_MAKEFILE=<path to TAU>/x86_64/lib/Makefile.tau-mpi-pdt-pgi export TAU_OPTIONS="-optPreProcess -optVerbose -optRevert -optTauSelectFile=`pwd`/select.tau"
select.tau
BEGIN_FILE_EXCLUDE_LIST comm_mpi.* END_FILE_EXCLUDE_LIST
Compile
./makenek r1854a make -f makefile
There is a auto-generated makefile, any further changes can be made to this file. For example make sure that -r8 option is passed to the compiler.
Running NEK
If compiling was successful you should have nek5000' executable.
Use the nekbmpi script to run the example:
./nekbmpi r1854a <# of procs>
On 8 processors this example takes just less than one hour.
Performance Data
Mean breakdown:
Major MPI routines across each node:
Profile:
