[ Home | Overview | Publications ] • [ Examples ] • [ Software/Downloads | Documentation/Questions ] • [ People | Acks ]
[Contents]
To reduce our maintenance efforts, we have opted to distribute HPCToolkit primarily in source form. The following prerequisites apply to our default source code distribution.
HPCToolkit can be installed on the following platforms:
Platform | Notes |
---|---|
Linux + x86_64 | |
Linux + x86 | |
Linux + ppc64 | |
Cray XT/XE/XK (CNL + x86_64) | Compute Node Linux ≥ 2.1 |
IBM Blue Gene/Q (BG/Q kernel + ppc64) | |
IBM Blue Gene/P (BG/P kernel + ppc32) | BG/P kernel ≥ V1R3M0 |
Linux + IA64 | testing (uses libunwind) |
Linux + MIPS-le (64 and n32) | for SiCortex clusters; little endian MIPS (no longer tested) |
Compiler: HPCToolkit requires an ANSI/ISO C/C++ compliant compiler that supports a few GNU extensions. Generally speaking, any recent GCC (4.x, 3.3+) should be fine. Some vendor compilers that support GNU extensions also work.
Additionally, HPCToolkit requires a basic version of GNU make, not a vendor supplied make. We have successfully used versions 3.70, 3.76, 3.77, 3.78, 3.79, 3.80.
To use hpcviewer and hpctraceviewer (HPCToolkit's interactive presentation tools) on MacOS, Windows, or Linux, a Java runtime environment (JRE) ≥ 1.5 is required. Use the Oracle or IBM version; we have had problems with OpenJDK.
To use hpcviewer and hpctraceviewer on Linux, GTK+ is also required.
Note: Since HPCToolkit's databases are portable, we often install hpcviewer and hpctraceviewer on local laptops/desktops to avoid sending display traffic over relatively slow network connections.
Highly recommended: To collect measurements based on hardware performance monitoring unit (PMU) information, PAPI ≥ 3.5 is required. Without PAPI, HPCToolkit can only collect measurements based on an operating system timer. (Exception: On IBM Blue Gene/P, PAPI is not required. On this system, HPCToolkit uses IBM's native Universal Performance Counters directly.)
There are currently three ways to support PAPI on Linux-based OSs:
Use a kernel with perf_events (see also: this), which was introduced with kernel 2.6.32 (née PCL in kernel 2.6.31) and which is now becoming part of standard distributions. With perf_events, use PAPI ≥ 4.1.2.
Apply the perfctr kernel patch (x86_64-based systems only). One benefit of perfctr is that it includes out-of-the-box support for many Red Hat kernels. As of this writing, perfctr supports vanilla kernels through 2.6.32.
Apply the perfmon2 kernel patch. As of this writing, perfmon2 supports vanilla kernels through 2.6.30. (Unfortunately, development has stopped because of perf_events.)
Minor requirement: To use hpcsummary (a relatively minor tool), Python 2.x is required.
Minor requirement: To build HPCToolkit's man pages, a variant of Perl 5 is required. We have successfully used versions 5.6.x - 5.10.x.
[Contents]
Download HPCToolkit components as follows:
Download hpctoolkit from our subversion repository.
Download hpctoolkit-externals from our subversion repository.
Think of hpctoolkit-externals as a "package manager" that automatically configures and builds support libraries that hpctoolkit needs. It contains several external libraries. (Some of these libraries, like libelf and libxml2 are commonly found on Linux systems, but not always. Others, like OpenAnalysis and SymtabAPI are almost never available. Finally, a few packages, like GNU binutils have been heavily patched.) In some cases it may be possible to use versions of these packages that are already installed on your system. However, we do not support such configurations.
Download binary releases of hpcviewer and hpctraceviewer for the platform on which your are building HPCToolkit.
Note: It is perfectly reasonable to install hpcviewer and hpctraceviewer in multiple locations. Since HPCToolkit's databases are portable, we often install hpcviewer and hpctraceviewer on local laptops/desktops to avoid sending display traffic over relatively slow network connections from systems where we collect measurement data.
[Contents]
The build and install follows autotools conventions (./configure && make && make install
).
In each case, more options are available by executing ./configure --help
.
See the README file for more details.
The typical build is as follows.
Please note that a list of examples below provides configure
commands for selected platforms.
Build and install HPCToolkit's Externals, which typically does not require any special options; we recommend using the GCC compilers:
cd hpctoolkit-externals
mkdir BUILD && cd BUILD
../configure [CC=<c-compiler>]
[CXX=<c++-compiler>] \
[--prefix=<hpctoolkit-externals-install>
] make install
make clean
Externals is not persistent as HPCToolkit will copy everything it needs from the installation.
Thus, the installation path <hpctoolkit-externals-install>
should be a local path and not something like /usr
.
If --prefix
is not supplied it will automatically be set to ./<Autoconf-host>
.
To specify a C and C++ compiler other than gcc
and g++
, use CC
and CXX
.
Please be sure the Externals build completes before configuring HPCToolkit. (Note: After building Externals once, HPCToolkit can be configured and built several times using the same Externals installation.)
Build and install HPCToolkit using commands similar to the following; we recommend using the GCC compilers:
cd hpctoolkit
mkdir BUILD && cd BUILD
../configure [CC=<c-compiler>]
[CXX=<c++-compiler>]
[MPICXX=<mpi-c++-compiler>] \
--prefix=<hpctoolkit-install> \
--with-externals=<hpctoolkit-externals-install> \
[--with-papi=<papi-install>]
make install
The above commands will build HPCToolkit and install it into <hpctoolkit-install>
.
To specify a C and C++ compiler other than gcc
and g++
, use CC
and CXX
.
To enable MPI support, (1) ensure a working mpicxx
or mpiCC
is in your shell's path; or (2) use MPICXX
.
(Naturally, CXX
and MPICXX
should be compatible; typically MPICXX
is a wrapper for CXX
.)
To enable PAPI-based measurements, use the --with-papi
option; it takes as an argument the path of your PAPI installation, <papi-install>
.
Because the HPCToolkit installation is self-contained, it may be renamed or moved to another location. All HPCToolkit Externals source code, build files and installations may be removed; all HPCToolkit source code and build files may be removed.
Install hpcviewer and hpctraceviewer .
cd <hpcviewer> && ./install [ -j <path-to-java> ] <hpctoolkit-install>
cd <hpctraceviewer> && ./install [ -j <path-to-java> ] <hpctoolkit-install>
<hpcviewer>
and <hpctraceviewer>
are the directories created by unpacking the respective tarballs; and <hpctoolkit-install>
is the above HPCToolkit installation directory.The list below gives example HPCToolkit Externals and HPCToolkit configure
commands (respectively) for selected platforms.
N.B.: The commands are designed to replace the respective commands in the above build instructions.
Blue Gene/Q:
../configure --prefix=`pwd`/../powerpc64-linux
../configure \
MPICXX=/bgsys/drivers/ppcfloor/comm/gcc/bin/mpicxx \
HPCPROFMPI_LT_LDFLAGS="-all-static" \
--prefix=<install-path> \
--with-externals=<hpctoolkit-externals>/powerpc64-linux \
--with-papi=/soft/perftools/papi
MPICXX
and PAPI may be different on your system.Blue Gene/P:
../configure --prefix=`pwd`/../powerpc32-linux
../configure \
HPC_LT_LDFLAGS="-all-static -L../../lib/stubs-gcc_s" \
HPCPROFMPI_LT_LDFLAGS="-L/bgsys/drivers/ppcfloor/gnu-linux/powerpc-bgp-linux/lib -lstdc++" \
--prefix=<hpctoolkit-install> \
--with-externals=<hpctoolkit-externals>/powerpc32-linux
Cray XT/XE/XK:
module swap PrgEnv-pgi PrgEnv-gnu
../configure --prefix=`pwd`/../x86_64-linux
../configure \
MPICXX="CC" \
HPC_LT_LDFLAGS="-all-static -L../../lib/stubs-gcc_s" \
--prefix=<hpctoolkit-install> \
--with-externals=<hpctoolkit-externals>/x86_64-linux \
--with-papi=/opt/cray/papi/4.3.0.1/perf_events/no-cuda
--enable-xop
'.
The AMD Interlagos CPUs (6200 series) use these instructions.
Currently, Jaguar at ORNL uses these CPUs, but Hopper at NERSC does not.
Intel MIC (early and experimental):
The solution is to build all of externals and hpctoolkit twice:
once for the main cores and once for the MIC cores.
Install the builds into two separate install trees (prefixes).
Both externals and hpctoolkit support VPATH builds, so you only
need to checkout one copy of each source tree.
Let's say that you use BUILD-x86_64
and BUILD-k1om
for the build directories and install-x86_64
and
install-k1om
for the install directories.
cd BUILD-x86_64
../configure --prefix=/path/to/externals/install-x86_64
PATH="/usr/linux-k1om-4.7/bin/:$PATH"
cd BUILD-k1om
../configure --prefix=/path/to/externals/install-k1om \
--host=x86_64-k1om-linux \
CC=x86_64-k1om-linux-gcc \
CXX=x86_64-k1om-linux-g++
cd BUILD-x86_64
../configure --prefix=/path/to/hpctoolkit/install-x86_64 \
--with-externals=/path/to/externals/install-x86_64 \
--with-papi=/path/to/papi-install
PATH="/usr/linux-k1om-4.7/bin/:$PATH"
cd BUILD-k1om
../configure --prefix=/path/to/hpctoolkit/install-k1om \
--with-externals=/path/to/externals/install-k1om \
--host=x86_64-k1om-linux \
CC=x86_64-k1om-linux-gcc \
CXX=x86_64-k1om-linux-g++
--with-papi=/path/to/papi-k1om
to the hpctoolkit configure
for MIC.
Be sure that this version of PAPI is built for the MIC cores.
hpcprof-mpi
is untested for the MIC cores.
You can build a restricted set of tools with the
--enable-back-end
option (experimental).
This option builds only the tools, libraries and prerequisites needed for
running hpcrun
and hpclink
and collecting data
on the back-end compute nodes.
In that case, run hpcstruct
and hpcprof
on the main cores.
On Stampede, you can launch a mixed-mode job (symmetric computing) with
ibrun.symm
and give separate command lines for the main
and MIC cores.
ibrun.symm -m "command line for main cores" \
-c "command line for MIC cores"
Linux-x86 (32-bit) on Linux-x86_64:
../configure CC="gcc -m32" CXX="g++ -m32" \
--host=i686-linux --prefix=`pwd`/../i686-linux \
--without-libunwind
../configure CC="gcc -m32" CXX="g++ -m32" MPICXX="mpicxx -m32" \
--host=i686-linux --prefix=<hpctoolkit-install> \
--with-externals=<hpctoolkit-externals>/i686-linux \
--with-papi=<papi-install> \
--disable-hpcrun-static
[Contents]
The following options may be useful in exceptional circumstances.
To enable/disable HPCToolkit features, the following options may be used.
Option | Meaning |
---|---|
--enable-mpi |
enable/disable build of hpcprof-mpi [default: yes] |
--with-upc=PATH |
path to IBM BlueGene/P UPC installation |
--with-upc-lib=LIBRARY |
path to IBM BlueGene/P UPC library(s), absolute or relative to WITH_UPC/runtime/SPI |
--enable-develop |
enable/disable build for development (sane debugging) [default: no] |
To request that HPCToolkit Externals use an alternate user-supplied installation for a given package, the following options may be used.
Option | Meaning |
---|---|
--with-binutils=PATH |
path to binutils install directory |
--with-libdwarf=PATH |
path to libdwarf install directory |
--with-libelf=PATH |
path to libelf install directory |
--with-libmonitor=PATH |
path to libmonitor install directory |
--with-libunwind=PATH |
path to libunwind install directory |
--with-libxml2=PATH |
path to libxml2 install directory |
--with-old-monitor=PATH |
path to old monitor install directory |
--with-open-analysis=PATH |
path to open analysis install directory |
--with-symtabAPI=PATH |
path to symtabAPI install directory |
--with-xed2=PATH |
path to xed2 install directory |
--with-xerces=PATH |
path to xerces/c install directory |
In all cases, the default is for HPCToolkit externals build the package itself, depending on platform.
To explicitly request that a package x
not be built, use --without-x
.
[Page last updated: 2013/05/24]