Difference between revisions of "LAMMPS"
(→Building with TAU) |
|||
| (7 intermediate revisions by the same user not shown) | |||
| Line 11: | Line 11: | ||
|[http://lammps.sandia.gov/download.html#git git repo] | |[http://lammps.sandia.gov/download.html#git git repo] | ||
|Keeneland | |Keeneland | ||
| − | | | + | |May 2012 |
|} | |} | ||
| Line 18: | Line 18: | ||
To start copy over the file [[Media:Makefile.keeneland | Makefile.keeneland]] to the src/MAKE directory. Then do, | To start copy over the file [[Media:Makefile.keeneland | Makefile.keeneland]] to the src/MAKE directory. Then do, | ||
| + | |||
| + | configure TAU: | ||
| + | |||
| + | ./configure -mpi -cuda=<cuda dir> -pdt=<pdt dir> | ||
| + | make install | ||
| + | |||
| + | build LAMMPS: | ||
cd src | cd src | ||
| Line 26: | Line 33: | ||
export TAU_MAKEFILE=<path to TAU>/x86_64/lib/Makefile.tau-icpc-mpi-pdt | export TAU_MAKEFILE=<path to TAU>/x86_64/lib/Makefile.tau-icpc-mpi-pdt | ||
export TAU_OPTIONS='-optVerbose -optShared' | export TAU_OPTIONS='-optVerbose -optShared' | ||
| − | make keeneland CC=tau_cc.sh CXX=tau_cxx.sh F90=tau_f90.sh | + | make keeneland CC=tau_cc.sh CXX=tau_cxx.sh F90=tau_f90.sh LINK=tau_cc.sh |
| − | + | running: | |
| + | |||
| + | cd examples/USER/cuda | ||
| + | |||
| + | Then choose either method depending on your choice of MPI implementation: | ||
| + | |||
| + | |||
| + | === OPENMPI === | ||
| + | |||
| + | Running with MPI on Keeneland by doing: | ||
| + | qlogin -V -l nodes=2:ppn=3 | ||
| + | mpirun --mca btl_openib_flags 1 --mca mpi_leave_pinned 0 --mca \ | ||
| + | btl_openib_warn_default_gid_prefix 0 -npernode 3 -np $NPROCS_3 --hostfile ${PBS_NODEFILE} \ | ||
| + | tau_exec -T mpi,cupti,icpc -cupti \ | ||
| + | /lustre/medusa/biersdor/mylammps/src/lmp_keeneland_tau_wgpu -cuda on -sf cuda < in.melt_2.5.cuda | ||
| + | |||
| + | |||
| + | === MPICH2 === | ||
| + | As of this writing the USER/CUDA package requires mpich2 to run on more than one node. Here are the commands to get started: | ||
| + | |||
| + | qlogin -V -l nodes=2:ppn=3 | ||
| + | mpdboot --totalnum=2 --ncpus=3 -f mpich_machines.txt | ||
| + | cat mpich_machines.txt | ||
| + | kid008.nics.utk.edu:3 | ||
| + | kid009.nics.utk.edu:3 | ||
| + | mpirun -np 6 tau_exec -T icpc,mpi,pdt,cupti,mpich2 -cupti /lustre/medusa/biersdor/mylammps/src/lmp_keeneland_tau_mpich2 -sf cuda -cuda on < in.melt_2.5.cuda | ||
| − | + | == Performance results == | |
| − | |||
| − | + | [[Image:lammps_profile.png|750px]] | |
| + | [[Image:lammps_3gpu.ppk]] | ||
| − | + | [[Image:lammps_trace.png|750px]] | |
| − | |||
Latest revision as of 19:33, 21 May 2012
| Link | Code Version | Machine | Date |
|---|---|---|---|
| Sandia website | git repo | Keeneland | May 2012 |
Building with TAU
To start copy over the file Makefile.keeneland to the src/MAKE directory. Then do,
configure TAU:
./configure -mpi -cuda=<cuda dir> -pdt=<pdt dir> make install
build LAMMPS:
cd src make yes-user-cuda cd ../lib/cuda make lib cd ../../src export TAU_MAKEFILE=<path to TAU>/x86_64/lib/Makefile.tau-icpc-mpi-pdt export TAU_OPTIONS='-optVerbose -optShared' make keeneland CC=tau_cc.sh CXX=tau_cxx.sh F90=tau_f90.sh LINK=tau_cc.sh
running:
cd examples/USER/cuda
Then choose either method depending on your choice of MPI implementation:
OPENMPI
Running with MPI on Keeneland by doing:
qlogin -V -l nodes=2:ppn=3
mpirun --mca btl_openib_flags 1 --mca mpi_leave_pinned 0 --mca \
btl_openib_warn_default_gid_prefix 0 -npernode 3 -np $NPROCS_3 --hostfile ${PBS_NODEFILE} \
tau_exec -T mpi,cupti,icpc -cupti \
/lustre/medusa/biersdor/mylammps/src/lmp_keeneland_tau_wgpu -cuda on -sf cuda < in.melt_2.5.cuda
MPICH2
As of this writing the USER/CUDA package requires mpich2 to run on more than one node. Here are the commands to get started:
qlogin -V -l nodes=2:ppn=3 mpdboot --totalnum=2 --ncpus=3 -f mpich_machines.txt cat mpich_machines.txt kid008.nics.utk.edu:3 kid009.nics.utk.edu:3 mpirun -np 6 tau_exec -T icpc,mpi,pdt,cupti,mpich2 -cupti /lustre/medusa/biersdor/mylammps/src/lmp_keeneland_tau_mpich2 -sf cuda -cuda on < in.melt_2.5.cuda
Performance results
