The CSC Benchmarks (Spring 1993)
by Sami Saarinen
Center for Scientific Computing
P.O. Box 405
Tietotie 6
FIN-02101 Espoo
Finland
e-mail: sbs@csc.fi
April 9, 1993
Table of contents
1. Introduction .................................................
2. Contents of the benchmarks ...................................
2.1 A subset of the SPEC-benchmarks .............................
2.1.1 Reporting the SPEC-results ................................
2.1.2 Accessing the SPEC-benchmarks .............................
2.2 The CSC Benchmark Suite, version 2 ..........................
2.2.1 Benchmark program characteristics .........................
2.2.2 Run rules .................................................
2.2.3 Reporting the CSC Benchmark Suite results .................
2.3 Parallel benchmarks .........................................
2.4 Miscellaneous benchmarks ....................................
3. Performance metrics ..........................................
3.1 Wall clock time .............................................
3.1.2 Sum of running times ......................................
3.2 Ratio of wall clock times ...................................
3.3 Nominal flop count ..........................................
3.4 Application performance (nominal Mflop/s) ...................
3.5 Benchmark performance .......................................
3.6 Benchmark averages ..........................................
3.6.1 Geometric mean ............................................
3.6.2 Arithmetic mean ...........................................
3.6.3 Harmonic mean .............................................
3.7 Benchmark instability .......................................
3.8 Parallel program measures ...................................
3.8.1 Parallel speedup ..........................................
3.8.2 Parallel efficiency .......................................
3.9 Measuring communication overhead ............................
3.9.1 Computation to communication ratio ........................
3.9.2 Modeling communication time ...............................
4. Execution instructions .......................................
4.1 Reading the benchmark tape ..................................
4.2 Directories, files and utility functions ....................
4.2.1 CSCSUITE_2 ................................................
4.2.2 PARALLEL ..................................................
4.2.3 MISC ......................................................
4.2.4 Timer functions ...........................................
4.2.5 Controlling the number of processors ......................
4.3 Compiling and linking the applications ......................
4.3.1 The CSC Benchmark Suite ...................................
4.3.2 Parallel benchmarks .......................................
4.3.3 Miscellaneous benchmarks ..................................
4.4 Running the benchmarks ......................................
4.4.1 The CSC Benchmark Suite ...................................
4.4.2 Parallel benchmarks .......................................
4.4.3 Miscellaneous benchmarks ..................................
5. Contact information ..........................................
Appendices:
A: Configuration of the CSC reference computer system ...........
B: A short description of the SPEC-benchmarks ...................
C: Guidelines to report system configuration ....................
D: Description of the CSC Benchmark Suite, version 2 ............
E: PARMACS macro calls ..........................................
1. Introduction
This document describes the CSC Benchmarks to be run by selected vendors in
Spring 1993 during computer evaluation project.
First, I will go through the contents of the benchmarks. After that I will
talk about some benchmark metrics that we are interested in. Finally, the
detailed benchmark instructions are followed.
This benchmark has been very carefully planned and should be easy port to
any UNIX-based systems having proper Fortran-77 and C-compilers. I have
included things like 'run script' and 'makefile' -generators to facilitate
the installation, executable building and running the benchmarks. Also,
detailed information about the programs' characteristics is included. This
contains description of each individual program, the reference results on
our reference computer system, the nominal megaflop-count collected by help
of Cray Hardware Performance Monitor (hpm) and some additional information
that would categorize the benchmark programs.
Our reference computer system is Silicon Graphics Indigo R4000. More
detailed information about the system is found in appendix A.
2. Contents of the benchmarks
The benchmark is divided into the four parts:
(1) A subset of the Standard Perfomance Evaluation Cooperative (SPEC)
benchmarks
(2) The CSC benchmark suite, version 2
(3) Parallel benchmarks
(4) Miscellaneous benchmarks
Most of the benchmarks are applied to all vendors, if not otherwise stated.
The following subsections will explain these benchmarks. Where
appropriate, the appendices are used to list more detailed information.
2.1 A subset of the SPEC-benchmarks
The aim of this benchmark is to report the per processor performance of the
proposed system by following the run rules specified by Standard Perfomance
Evaluation Cooperative (SPEC) cooperative.
Because most of the computer manufacturers are currently members of the SPEC
organization (or have access to their benchmarks), this benchmark does not
(in average) require any explicit running of the benchmarks since the results
are reported in SPEC's newsletter quarterly.
We are specifically interested in SPEC Floating Point Suite 92 (SPECfp92 or
CFP92) and SPEC Integer Suite 92 (SPECint92 or CINT92) break down or per
program results.
The SPEC CFP92 benchmark suite consists of CPU intensive benchmarks that
are intended to be meaningful samples of applications which perform
floating point logic and computations in a technical computing environment.
The SPEC CINT92 benchmark suite consists of CPU intensive benchmarks that
are intended to be meaningful samples of applications which perform
non-floating point logic and computations in a technical computing
environment.
Note that neither CFP92 nor CINT92 does not assess the ability of a system
under test to handle disk, graphics or any form of networking or
communication.
Many of the SPEC benchmarks have been derived from publicly-available
application programs, and they are intended to be portable to as many
current and future hardware platforms as possible.
Appendix B contains a brief description of each SPEC-benchmark program.
2.1.1 Reporting the SPEC-results
We are interested in individual turn-around times for each program.
Note however, that unlike SPEC, we will compare these times to our
reference computer.
The vendors should report the SPEC-results in the following form
(wall clock times in the table indicate reference computer results reported
by the Silicon Graphics Inc. in the SPEC-newsletter Vol.4, No.3, September
1992, page 28 and 18):
SPECfp92:
____________________________________
| Program | Wall clock time |
| | (seconds) |
|_______________|____________________|
| 013.spice2g6 | 541.5 |
| 015.doduc | 36.3 |
| 034.mdljdp2 | 88.0 |
| 039.wave5 | 89.1 |
| 047.tomcatv | 40.8 |
| 048.ora | 101.6 |
| 052.alvinn | 101.5 |
| 056.ear | 272.7 |
| 077.mdljsp2 | 76.8 |
| 078.swm256 | 359.2 |
| 089.su2cor | 176.7 |
| 090.hydro2d | 182.0 |
| 093.nasa7 | 230.8 |
| 094.fpppp | 171.5 |
|_______________|____________________|
SPECint92:
____________________________________
| Program | Wall clock time |
| | (seconds) |
|_______________|____________________|
| 008.espresso | 41.1 |
| 022.li | 95.0 |
| 023.eqntott | 14.4 |
| 026.compress | 66.9 |
| 072.sc | 61.9 |
| 085.gcc | 124.8 |
|_______________|____________________|
In addition to this, the vendor should report the configuration of the
benchmarked system as specified in appendix C.
2.1.2 Accessing the SPEC-benchmarks
In case some vendor(s) have currently no access to the benchmarks source
codes, they can be purchased from the following address (attn. Dianne Dean):
SPEC [Standard Performance Evaluation Corporation]
c/o NCGA [National Computer Graphics Association]
2722 Merrilee Drive
Suite 200
Fairfax, VA 22031
USA
Phone: +1-703-698-9600 Ext. 318
FAX: +1-703-560-2752
E-Mail: spec-ncga@cup.portal.com
The prices for CINT92 and CFP92 release 1.1 QIC 24 -tapes are 425USD and
575USD, respectively.
2.2 The CSC Benchmark Suite, version 2
This benchmark comprises of a set of programs that represents the average
load of the current computer systems at CSC. The benchmark consists of 14
floating point intensive programs. All except one C-code are written in
Fortran-77. A description of each program is found in appendix D.
2.2.1 Benchmark program characteristics
Each program in this benchmark set has been carefully analyzed by Cray's
Hardware Performance Monitor (hpm) and using some general tools available
under Unix operating system. In addition to that reference timings are
provided for the reference system, Silicon Graphics Indigo R4000.
Following table provides some static information about the programs:
______________________________________________________________________
| Program Prec. Source | Text Data+Bss TotalSize | DiskSpace |
| (bits) lines | (KB) + (KB) = (KB) | (KB) |
|____________________________|_____________________________|___________|
| ARCTWOD 64 3759 | 344 2788 3132 | 942 |
| CASTEP 64 13337 | 548 3680 4228 | 2642 |
| FREQUENCY 32 276 | 212 25777 25989 | 628 |
| GRSOS 32 316 | 196 19613 19809 | 457 |
| INVPOW93 64 486 | 220 2930 3150 | 6408 |
| MOPAC 64 22093 | 676 26824 27500 | 1294 |
| NASKER 64 1101 | 200 2877 3077 | 351 |
| NBODYOPT 32 1459 | 152 1224 1376 | 299 |
| RIEMANN (*) 64 439 | 48 74 122 | 86 |
| SIMCZO 64 2069 | 252 35960 36212 | 837 |
| WHY12M 64 996 | 220 23742 23962 | 18201 |
| MD1 32 1129 | 168 9230 9398 | 327 |
| PDE1 64 207 | 144 37041 37185 | 287 |
| QCD1 64 2641 | 252 7763 8015 | 448 |
|____________________________|_____________________________|___________|
(*) Written in C-language.
All programs should confirm the precision shown above. Note, that GRSOS
contains some REAL*8 (DOUBLE PRECISION) operations that should not be
removed, but the data-arrays are still INTEGERs and REALs. Note also, that
all 64-bit codes, except CASTEP contain explicit DOUBLE PRECISION
definition (either by IMPLICIT-statement or variable-wise). This means, the
vendor has to activate DOUBLE PRECISION via appropriate compiler flag or,
if such does not exist, manually include IMPLICIT DOUBLE PRECISION
-statements at the beginning of each routine in CASTEP. Furthermore, CASTEP
seems to treat all variables that begin with letter 'C' as COMPLEX.
CASTEP may also require some intrinsic functions (CEXP, AIMAG, AMOD) to be
changed (EXP, IMAG, MOD). This change can be made by hand or running CASTEP
sources through the '/lib/cpp -P' with appropriate definitions (-DCEXP=EXP
-DAIMAG=IMAG -DAMOD=MOD). If the C-preprocessor is invoked by the
Fortran-compiler as default, include these definitions as 'FFLAGS' (see
chapter 4.2).
Some of the codes also require automatic SAVE-statements to be activated.
I rely on the fact that all vendors have such a compiler flag available.
I recommend to use "auto-SAVE" for all programs, but especially CASTEP,
MOPAC, SIMCZO, MD1, PDE1 and QCD1.
Some of the programs are suitable for fine grain parallelism. We have
found that for example ARCTWOD, FREQUENCY, GRSOS, INVPOW93, NASKER may
benefit from parallel processors.
The programs can also be divided by the application area. Following table
tries to give to each program a typical application area(s):
________________________________________________________
| Program Application area(s) |
|________________________________________________________|
| ARCTWOD Fluid Dynamics, Engineering |
| CASTEP Chemistry, Physics |
| FREQUENCY Engineering, Mathematics |
| GRSOS Physics |
| INVPOW93 Engineering, Mathematics, Eigenvalues |
| MOPAC Chemistry |
| NASKER Fluid Dynamics, Engineering, Mathematics |
| NBODYOPT Astrophysics, Mathematics |
| RIEMANN Mathematics |
| SIMCZO Structural Mechanics, Fluid Dynamics |
| WHY12M Mathematics, Sparse Matrices |
| MD1 Chemistry |
| PDE1 Mathematics, Partial Differential Equations |
| QCD1 Quantum Mechanics |
|________________________________________________________|
In the next table approximate nominal megaflop-counts are derived from
Cray's 'hpm' Group-0 and Group-3 executions and reference timings for the
reference computer system. Nominal megaflop-count and performance are
described more detailly in the 'Performance metrics' chapter later on.
Table also provides additional information gathered from 'hpm':
o Vectorization percentage, which is calculated from the ratio of
vector floating point operations to total number of floating
point operations (vector & scalar) on Cray X-MPEA.
o Average vector length (Avg.VL), which is expressed in modulo 64
since Cray X-MPEA's vector registers length is 64.
o Memory references tell how many accesses to 64-bit precision words
have been occurred during the execution of the program.
The application performance for the reference system is calculated by
dividing the nominal megaflop-count on Cray X-MPEA by the wall clock time
on the reference system. This is NOT the REAL Mflop/s, only a fairly good
approximation in most cases. However, very high values (reaching or
exceeding the theoretical peak) may indicate that the compiler and/or the
pre-processor has done a good job or the program takes advantage of the
32-bit arithmetic that is not available on Cray X-MPEA.
_______________________________________________________________________
|Program Floating point Memory | Wall clock | Application |
| Nom.Ops. Vector Avg.VL Refs. | time (sec) | Performance |
|__________________________________________|______________|_____________|
| ARCTWOD 555M 100% 53 692M | 81.2 | 6.8 |
| CASTEP 3211M 69% 43 3880M | 450.7 | 7.1 |
| FREQUENCY 517M 100% 63 1405M | 84.2 | 6.1 |
| GRSOS 9162M 97% 59 3192M | 446.3 | 20.5 |
| INVPOW93 1014M 100% 57 1521M | 143.8 | 7.0 |
| MOPAC 1256M 58% 11 1573M | 162.1 | 7.7 |
| NASKER 2149M 100% 53 2169M | 294.5 | 7.3 |
| NBODYOPT 824M 86% 10 547M | 64.2 | 12.8 |
| RIEMANN 248M 0% 0 2527M | 72.2 | 3.4 |
| SIMCZO 1356M 94% 32 2311M | 659.4 | 2.1 |
| WHY12M 164M 3% 0 1017M | 124.7 | 1.3 |
| MD1 837M 22% 63 576M | 92.4 | 9.1 |
| PDE1 494M 100% 60 522M | 92.3 | 5.4 |
| QCD1 1314M 94% 64 1792M | 268.7 | 4.9 |
|__________________________________________|______________|_____________|
| Total 23101M 66% 41 23724M | 3036.7 | 7.6 |
|__________________________________________|______________|_____________|
From these reference results following additional information is derived:
______________________________ ____________________
| Statistic | Reference system |
| | value |
|______________________________|____________________|
| Benchmark performance | 7.6 |
| Geometric mean performance | 5.9 |
| Arithmetic mean performance | 7.2 |
| Harmonic mean performance | 4.6 |
| Benchmark instability | 15.8 |
|______________________________|____________________|
Please refer chapter 3 to understand these metrics.
2.2.2 Run rules
Performance results should be given for two versions of the codes:
o Baseline
o Optimized
CSC provides the Baseline source codes. It is up to vendor to optimize
Baseline source codes in order produce the Optimized versions. We will
consider this as a credit to the vendor.
In addition to this, varying number of processors must be used to accomplish
the tasks. For the Baseline and Optimized runs we following number of
processors must be used:
o single processor results
o P-processor results
o P/2-processor results [P/2+1 is P is odd]
o P_opt-processor results
where P = maximum number of proposed processors.
P_opt = number of processors that will give optimal performance
each individual program.
This means that in principle maximum of 2x4 executions of each benchmark
program is required in order to run complete benchmark suite. However, it
is left to vendor to decide whether to optimal number of processors differs
from P-processor results.
When running any of the benchmark programs, the following general run rules
must be followed:
o All times reported must be for runs that produce correct results.
o All information necessary for replication of the results should
be disclosed and available at request.
o Single-user mode is allowed, but must be reported.
o Use of benchmark specific software (preprocessors etc.) is not
allowed. Note, that this does NOT prevent use regular
preprocessors provided by vendors nor use of KAP, VAST etc.
However, all performance improvers must be included in the
proposed system, too.
To obtain the Baseline results, the vendor must obey following additional
rules:
o Source code may not be modified unless it is required for
portability. This includes manual insertion of compiler
directives. All changes must be reported.
o No use of scientific libraries that would replace the original
code is allowed unless this is done automatically by the compiler
and does not increase the compilation/linking time dramatically.
Thus, the general rule for obtaining the Baseline results is to give
compiler and preprocessors full freedom to optimize as much as possible
without need of any making source code modifications ny hand.
To obtain the Optimized results, the vendor CAN do the following things:
o Modify by hand the source as required, but still solving the same
problem and providing the same output (final and intermediate)
using the same input files.
o Insert compiler directives.
o Insert calls to the scientific libraries
2.2.3 Reporting the CSC Benchmark results
Before reporting the actual run times, the vendor should report the
compilation and linking times (wall clock time) for each application and
case. We will consider it as a credit if the compiler and linker do not
spend too much time in optimizing codes.
For the Baseline and the Optimized results, the vendor should report the
CSC Benchmark Suite results in the following form:
___________________________________ ___________________ ___________________
| | # of CPUs = 1 | # of CPUs = P/2 | # of CPUs = P |
| Program |___________________|___________________|___________________|
| | Wall clock time | Wall clock time | Wall clock time |
| | (seconds) | (seconds) | (seconds) |
|_______________|___________________|___________________|___________________|
| ARCTWOD | | | |
| CASTEP | | | |
| FREQUENCY | | | |
| GRSOS | | | |
| INVPOW93 | | | |
| MOPAC | | | |
| NASKER | | | |
| NBODYOPT | | | |
| RIEMANN | | | |
| SIMCZO | | | |
| WHY12M | | | |
| MD1 | | | |
| PDE1 | | | |
| QCD1 | | | |
|_______________|___________________|___________________|___________________|
To report the "P_opt-processor" results, create similar table than above,
but specify also the number of processors used to run each of the programs.
This means that the actual value of "P_opt" may vary from program to program.
2.3 Parallel benchmarks
2.3.1 CSCSUITE_2
This item was already covered in 2.2.3.
2.3.2 DM
DM or distributed memory benchmarks consists of 4 programs, of which one
belong to miscellaneous tests (COMMS1). The rest three are DM-versions of
MD1, PDE1 and QCD1. They are coded in Fortran, but contain system
dependent PARMACS message passing macros (see Appendix E) to facilitate
process to process communication.
The following cases should be at least run (2, 4, 8, 16 processors):
o MD1 - NC=9 i.e. 4*9^3 (2916) & NC=11 i.e. 4*11^3 (5324) atoms
o PDE1 - NN=6 & NN=7
o QCD1 - 16*4^3 and 8*8^3 systems
The brief instructions to these tests are found in chapter 4.
2.4 Miscellaneous benchmarks
The brief instructions to these tests are found in chapter 4.
3. Performance metrics
In this chapter some crucial performance metrics used in this report are
described.
3.1 Wall clock time
This is known also as real elapsed time or turn-around time or
time-to-solution or running time. It is measured from the beginning of
program to the end of application. It is the only relevant measure while
comparing parallel applications with each other.
In any benchmark context CPU-times are often mentioned. During this
benchmark we are not very much interested in these figures.
3.1.2 Sum of running times
Given a set of wall clock times for different benchmark applications, the
sum of running times is a simply sum of wall clock times. During this
benchmark it has no special meaning, but is included for for the sake of
completeness.
3.2 Ratio of wall clock times
Assuming that wall clock times have been measured for a particular
application. The ratio of wall clock times is application wall clock time
over reference computer system's wall clock time.
3.3 Nominal flop count
Nominal flop count are gathered from Cray's Hardware Performance Monitor
(hpm) by counting the Cray hardware multiplies, adds and reciprocals for a
particular application and converting them to nominal megaflop count using
following formula:
Nominal flop count = Multiplies + Adds - 2 * Reciprocals
The formulr stems from the fact that one divide on Cray hardware requires
1 reciprocal and 3 multiplies (or 2 multiplies and one add).
Although the REAL flop count varies from system to system, this value is
probably the best value that we can reliably get. And it stays constant
once obtained.
3.4 Application performance (nominal Mflop/s)
Dividing the program Nominal flop count (expressed normally in megaflops)
by the wall clock time in a particular machine, we get a fairly good
measure for application performance that resembles the famous Mflop/s
metric:
Cray's Nominal flop count
Application performance = _________________________________________
Application wall clock time on any machine
This can also be misleading, if interpreted incorrectly. Namely smart
compiler may optimize away a lot of floating point operations that current
Cray compiler was not able to. Also 32-bit codes may perform "unexpectedly"
fast.
We recommend to call it as an "application performance", NOT a Mflop/s.
3.5 Benchmark performance
Benchmark performance is defined by the ratio of sum of all applications'
nominal flop counts to sum of all applications' wall clock times:
Sum of applications' nominal flop counts
Benchmark performance = ________________________________________
Sum of applications' wall clock times
3.6 Benchmark averages
Various summaries can be drawn from the results of several individual
program executions. These summaries are sometimes characterized by
different averages.
Although we don't rely on averages as much as individual program results,
it is still worth to mention them. One of the most popular and less
misleading average is the Geometric mean. It is not very sensitive to large
variations in data values. Sometimes very misleading ones are the
Arithmetic and Harmonic means. The former one will give unexpectly good
averages if only one data value gets high. The latter one in turn works in
the other way around: if there is one bad result (low value), it will
destroy the whole data set average.
3.6.1 Geometric mean
Geometric mean is defined as a Nth root of a product of N data values:
1/N
Geometric mean = ( Product {data_i} )
i=[1..N]
3.6.2 Arithmetic mean
Arithmetic mean is defined as an average of N data values:
Arithmetic mean = ( Sum {data_i} ) / N
i=[1..N]
3.6.3 Harmonic mean
Harmonic mean is defined as a reciprocal average of N reciprocal data
values:
N
Harmonic mean = ___________________________
Sum {1 / data_i}
i=[1..N]
3.7 Benchmark instability
Benchmark instability of a tested machine is defined by taking the ratio of
maximum attained application performance to minimum attained application
performance of the programs included in the benchmark set:
Maximum attained application performance
Benchmark instability = __________________________________________
Minimum attained application performance
3.8 Parallel program measures
Parallel program measures stem from the fact that run time for any
application is essentially composed of two parts: sequential and parallel
portion's time, T_s and T_par, respectively. Thus, one-processor wall clock
time is expressed in form:
T(1) = T_s + T_par
Running the same application in parallel with P-processors will result the
run time to be:
T(P) = T_s + T_par / P
This is, however, only an approximation, since increasing the number of
processors will also require synchronization and communication to be
included. Thus, in reality the run time for parallel application is higher:
T(P) = T_s + T_par / P + T_sc(P)
where T_sc is the time spent in synchronization and communication between
processors and is typically a monotonically increasing function of the number
of processors P.
The latter formula is the reason why we want the vendor to run certain
applications with "P_opt-processors". In reality, if we exclude so called
embarrasingly parallel applications, no continuous performance increase is
gained by increasing the number of processors. This is because typical
parallel application suffers from synchronization and communication
bottlenecks and after certain number of processors this throuphput time
becomes larger, thus making application to run slower eventhough more
processors are put to work together.
In the distributed memory benchmarks we would also like to see the size
effect to the solution time. If the problem size is N (defined
appropriately), then solution time varies according to following formula:
T(P,N) = T_s(N) + T_par(N) / P + T_sc(P,N)
3.8.1 Parallel speedup
In order to measure parallel speedup, it is currently recommended by
the benchmark authorities, that the speedup itself should not be used to
compare results between different architectures. However, using is to
test application speedup within a single architecture is not a bad idea.
Parallel speedup is defined:
One processor wall clock time
Parallel speedup = _____________________________
P-processor wall clock time
Note, that if the code has been optimized, then comparing the speedups of
baseline results to optimized is illegal. This is because the one processor
version performs now much better than in baseline resulting lower parallel
speedups for optimized results. A general rule is: the better attained
single processor performance the lower parallel speedup.
Note also, that parallel speedup may exceed actual number of processors if
the problem does not fit properly into the memory of one-processor system.
This results too high one-processor execution than what would be expected
if there were enough memory available.
3.8.2 Parallel efficiency
This is defined by the following formula and expressed in percentages:
One processor wall clock time
Parallel efficiency = ___________________________________ x 100%
(P-processor wall clock time) * P
3.9 Measuring communication overhead
In distributed memory applications which use mainly message passing to
exchange data between processors, a communication becomes an important
aspect. During data exchange processes are normally busy with sendign or
receiving the data, thus creating serious bottlenecks to parallel code.
3.9.1 Computation to communication ratio
One way to measure the goodness of message passing application is to keep
track of the time spent in communication. The computation to communication
ratio is thus defined:
Wall clock time - Communication time
Comp.Comm. ratio = _______________________________________
Communication time
The larger the ratio, the better performance and less communication.
3.9.2 Modeling communication time
Communication itself can be modeled by the following linearized formula:
Communication time = Latency_time + Transfer_Speed * Number_of_Bytes
The Latency_time is essentially the time spent while sending zero length
message. The Transfer_Speed is the speed of communication network. This is
usually also a weak function of number of bytes transferred.
4. Execution instructions
4.1 Reading the benchmark tape
The benchmark streamer tape (cartridge) is written on Sun Sparcstation's
tape drive using 'tar'-command. Before reading it to disk, specify
benchmark root directory. I will refer it here through the environment
variable BENCH:
% setenv BENCH /my/benchmark/root/directory
% mkdir $BENCH
% cd $BENCH
To unload the tape, type:
% tar xv
In case of alternate tape drive, use command like this:
% tar xvf /dev/other_tape_drive
Some systems may not recognize the tape format. In such case you may have to
use 'dd'-command with bytes swaps:
% dd if=/dev/tape_drive ibs=obs conv=swab | tar xvf -
After this run following command to install the benchmark files properly:
% install
This will do the rest; for example create additional directories,
uncompress few compressed tar-files found in tape, create some useful
symbolic links.
In case of serious problems in unloading the tape, I will put the tar-file
also into our anonymous ftp nic.funet.fi (128.214.6.100), under directory
pub/csc/benchmark. Use binary transfer more to retrieve the benchmark
tar-file named 'spring93.tar'.
4.2 Directories, files and utility functions
Once you have unloaded the tape, you will find following subdirectories
under your benchmark root directory:
o CSCSUITE_2
o PARALLEL
o MISC
These refer to the test sets described in the chapter 2. Naturally there
are no files for the SPEC-benchmarks.
In the following subsections I will use next two abbreaviations quite
often:
o <applic> - refers to application name (ARCTWOD, CASTEP, etc.)
o <arch> - refers to computer architecture (SGI, CRAY, etc.)
Also, there are two important files that appear time to time during use of
command scripts:
o program.list - contains list of application codes
o <arch>.make_flags - contains default modifications to standard
makefile settings
For example, file 'CRAY.make_flags' may look like this:
ARCTWOD - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
CASTEP - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
FREQUENCY - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
GRSOS - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
INVPOW93 - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
MOPAC - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
NASKER - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
NBODYOPT - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
RIEMANN - C # CC=cc
SIMCZO - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
WHY12M - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
MD1 - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
PDE1 - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
QCD1 - Fortran # FC=cf77 ;LD=cf77 ;FFLAGS=-Zv -Wf'-dp -a static'
A brief explanation to this. Take for example ARCTWOD:
o ARCTWOD is written in Fortran
o override default FC-setting (FC=f77) in makefile with cf77
o override default LD-setting (LD=f77) with cf77
o use new flags for Fortran compilation: -Zv -Wf'-dp -a static'
The format is simply the following (one program per line):
o Program name
o dash ('-')
o language (keywords 'Fortran' or 'C')
o hash ('#')
o overrides: NAME=new_value
o separator between overrides is semicolon (';')
This file is parsed by using 'awk' during generation of application
specific makefiles.
4.2.1 CSCSUITE_2
CSCSUITE_2 contains several shell scripts and also subdirectories for each
application to run the tests described in section 2.2. In addition to that
a subdirectory called "lib" is found. This is a depository for timer-function
routines and library.
Each application directory contains files and directories organized in the
following way:
Directories:
o <applic>/src - fsplit'ted Fortran or C-source codes
o <applic>/results - reference results directory (<applic>.out)
Files:
o <applic>/<applic>.exe - the executable for the application
o <applic>/<applic>.in - standard input file (if any)
o <applic>/<applic>.out - standard output file (if any)
o <applic>/* - other files: miscellaneous input
o <applic>/src/Makefile.<arch> - Architecture dependent makefile
Following shell scripts are found directly under CSCSUITE_2:
o Makefile - A driver makefile for some scripts
o program.list - List of application programs. Used by scripts.
o <arch>.make_flags - Default compiler/linker etc. flags used upon
generation of architecture dependent makefile
o build_script - to build Bourne-shell run script for a single
application. See also 4.2.5 for limiting CPU-number.
o build_all_scripts - to build all scripts in one shot
o build_make - to build architecture dependent makefile for
specific application
o build_all_makes - to build all architecture dependent makefiles
o make_program - to invoke architecture dependent makefile for
specific application
o make_all_programs - to make all programs, including libraries
o run_program - to run specific program
o run_all_programs - to run all programs after each other
o get_times - a utility to collect times from <applic>/<applic>.out's
o get_diffs - a utility to compare results with reference system
4.2.2 PARALLEL
PARALLEL directory contains two subdirectories CSCSUITE_2 and DM.
CSCSUITE_2 is essentially a duplicate of CSCSUITE_2-directory that has been
created during installation of the tape. Its role is to provide similar
environment than CSCSUITE_2, but for running 14 application codes now in
parallel, P > 1. I added also few new scripts to facilitate this:
o run_parallel - to run application in parallel with varying
number of processors
o get_parallel_times - to get tabulated list of parallel times
for a single application
DM is a directory for distributed memory benchmarks COMMS1, MD1, PDE1 and
QCD1 as well as depository for the PARMACS - PVM 2.4.2 -interface.
Interface calls are documented briefly in appendix E.
The test program COMMS1 does not actually belong to DM-applications, but
miscellaneous tests. As it contains PARMACS macros, it should be run under
this directory. It is explained briefly under 4.2.3, MISC.
4.2.3 MISC
MISC contains subdirectories and files for miscellaneous tests:
o MEMTEST - Several memory tests:
- CACHEMISS to test hardware behaviour during cache conflicts
- LARGEMEM to test how large single array can be allocated
- MEMSCAN to scan arrays with stride one or ramdomly
o IOTEST - I/O-subsystem -tests
- IOZONE to write/read file with several block/filesizes
o KERNEL - Kernel operations test
- BRE to run 15 different kernels tests
Although MISC directory does not contain communication test (COMMS1), such
is included under PARALLEL/DM -directory. Purpose of this test is to test
communication speed between two nodes when message size varies from 1 to
40000 bytes.
4.2.4 Timer functions
Throughout the benchmark subdirectory 'lib/' contains timer-functions.
As we are not interested in CPU-times in this benchmark, but wall clock
times, a utility function to get this has been coded. Times are obtained by
using C-routines that are called by Fortran-routines. Fortran-routines in
turn are called by user program.
Some minor (or none) modficications are needed to link properly the
timer-interface.
Fortran-routines ('lib/src/timer.f') are:
- SUBROUTINE INITIM(IDUMMY) to initialize timer (once a run).
Done implicitly by the library when any of the Fortran-routines
found in 'lib/src/timer.f' are called.
- SUBROUTINE SHOTIM(IDUMMY) to print out current wall clock time
since initialization of timer.
- SUBROUTINE TIMER(T) , DOUBLE PRECISION T
to store wall clock time since initialization to the variable 'T'
- DOUBLE PRECISION FUNCTION CPUTIM()
returns actually the wall clock time since initialization.
Among the C-routines ('lib/src/times.c') that Fortran-routines above call,
the most important is 'waltim'. As this is called by the Fortran-routines,
some computer systems may need the underscore to be appended after routine
name ('waltim_') or capitalized expression be used ('WALTIM').
Not all benchmarks use these timer routines. In case the first time any
benchmark program is executed and the execution fails, I recommend to the
'lib/' directory.
4.2.5 Controlling the number of processors
In some systems, especially shared memory multiprocessors, it is possible
explicitly to specify the number of processors to be used in runtime. In
order to be sure that a proper number of processors is in use, modify
'build_script' command procedure before trying to generate run scripts at
any time. For example, Silicon Graphics requires environment variable
'MP_SET_NUMTHREADS' to be set to one (1) when one-processor results are
needed.
4.3 Compiling and linking the applications
4.3.1 CSCSUITE_2
In order to successfully compile and link applications, change to CSCSUITE_2
directory and go through following steps:
(1) Run 'build_all_scripts' to create Bourne-shell run-scripts.
Modify, if necessary, files 'build_script' or
'build_all_scripts'.
NOTE: Be sure that your application really uses ONE processor
in this context. Modify 'build_script' accordingly.
(2) Create file <arch>.make_flags to contain default settings
for your system <arch> for subsequent makefile-generation step.
(Hint: Use file 'SGI.make_flags' as example)
(3) Generate makefile's under <applic>/src -directories using
command 'build_all_makes'.
(4) Check validity of just created makefiles,
<applic>/src/Makefile.<arch>.
(5) Make the executables and record the compilation & linking time
for each application. Use command 'make_all_programs'.
If make fails, lower the optimization level for particular
source by modifying corresponding <applic>/src/Makefile.<arch>,
and re-run make. Change the source code only in extreme cases.
Report the changes.
Here is what I did when I built the executables for SGI:
% build_all_scripts
% emacs SGI.make_flags # See beginning of section 4.2
% build_all_makes
% make_all_programs SGI clean # Clean all possible junk
% make_program lib SGI # Be sure 'timer.a' exist. See 4.2.4
% make_all_programs SGI -n # Trial run; don't make anything yet
% make_all_programs SGI # Make really now!
After that I had to lower the optimization level in one of the CASTEP
routine. So I modified CASTEP's makefile manually and re-ran the make:
% emacs CASTEP/src/Makefile.SGI
% make_program SGI CASTEP clean # Clean first to remove garbage
% make_program SGI CASTEP
To record the compilation & linking time for each application, you can make
each program in the following manner:
% time make_program <arch> <applic>
and write down the time.
4.3.2 Parallel benchmarks
4.3.2.1 CSCSUITE_2
Follow the same rules as in 4.3.1, but make changes to <arch>.make_flags to
activate parallel compilation. Be sure that while running any of the
applications, the number of processors is set either to 'P/2' or 'P' (or
P_opt !). Thus you must check 'build_script' before building scripts and
refer the result table in the chapter 2.2.3
4.3.2.2 DM
Generally, follow the same rules as in 4.3.1, but make changes to
<arch>.make_flags. As this benchmark contains 4 tests for distributed
memory computing that use PARMACS macros presented in Appendix E, some site
depended changes are needed.
Source files for this tests are written in Fortran (extension .f) or using
m4-macro processors language (.m4 files). Before actual Fortran compilation
the m4-macro files containing PARMACS-macros must be preprocessed to get
the Fortran equivalents. To facilitate this "preprocessing", we provide the
vendor with CSC-developed PARMACS -- PVM 2.4.2 interface.
In order to successfully link the DM-application with our PVM-interface,
following steps must be performed:
o Go to 'pvm2.4.2/' directory, read the Postscipt-instructions
how install pvm for your machine (in case you are not familiar
with it), and replace libraries 'libpvm.a', 'libfpvm.a' and
PVM-daemon executable 'pvmd' with your equivalents.
o Go to 'macrolib/' and check the Fortran-interface that is
called by the application program once the PARMACS calls have been
substituted. The source files under 'macrolib/src' use some
C-routines, like 'getarg()', 'iargc()', 'getcwd()', sleep().
Check that your system accepts these to be called from Fortran.
In case you wish to use vendor specific PARMACS-routines, some other path
must be followed.
As these, rather difficult, steps have been completed, you may have to
modify following files:
o build_make - to re-organize libraries
o <arch>.make_flags - to provide additions/substitutes to compiler flags
Once all seems to work, do as in 4.3.1.
Note that each application will has two executables: 'host' and 'node'.
They are also fixed size master and slave process executables.
4.3.3 Miscellaneous benchmarks
4.3.3.1 Memory tests
This contains three separate tests: CACHEMISS, LARGEMEM and MEMSCAN. These
are located in the directories MISC/MEMTEST/CACHEMISS,
MISC/MEMTEST/LARGEMEM and MISC/MEMTEST/MEMSCAN, respectively. Please refer
the 'Readme' files there for detailed information.
The CACHEMISS is a C-program that performs a full matrix multiply with
increasing stride. We plot a curve where X-axis is the problem size (matrix
dimension or effective stride) and Y-axis contains nominal Mflop/s rate
calculated by the program. The vendor is encouraged to provide equivalent
curve for an optimized version, that probably is written in Fortran or uses
library routines to perform the task.
To create CACHEMISS, go to its directory, modify 'Makefile' and create
executable.
LARGEMEM is a small program that is used to test how large single array can
be allocated in Fortran and in what circumstances. It contains four cases,
in each case the array to be allocated is in DOUBLE PRECISION:
o mem1.m4 - array is to be allocated (probably) from stack
o mem2.m4 - array is put to a named COMMON-block
o mem3.m4 - array is put to a unnamed COMMON-block
o mem4.m4 - array is SAVE'd ("static")
To create LARGEMEM, go to its directory, and run 'generate.csh'. This
actually executes the actual programs, too.
MEMSCAN is a "memory scanner" that performs one dimensional array
operations, like summing, scaling, saxpy, assigment and so on. Four
different versions exist:
o sequential and random scan (gather/scatter -like)
o single and double precision for both above
To create MEMSCAN, go to its directory, modify 'Makefile.SGI' and run it.
4.3.3.2 I/O-tests
This consists of modified IOZONE-test. Purpose of this test is to check
file buffer caching effect. Program writes sequentially N byteblocks of data
into a file with size X megabytes. Then file is closed and opened again,
but this time for reading.
A version that explicitely does 'fsync()' before 'close()' is run against
version that does not 'fsync()'. The purpose of 'fsync()' is enforce write
to disk and then return back to application. I hope all vendor can provide
similar functionality, if 'fsync()' itself does not exist.
To create IOZONE test programs, IOTEST/IOZONE directory, modify
'Makefile.SGI' and run it.
4.3.3.3 Computational kernel test
This test contains 15 BLAS-1 or -2 operations that are supposed to test
single processor performance in kernel operations.
The test is located in KERNELS/BRE directory. To create the 'BRE.exe',
modify makefile and run it.
4.4 Running the benchmarks
In order to run any of the benchmarks, be sure that there's enough disk
space available in the planned run directory. This directory can be located
in another disk than the benchmark directory. Also, the root directory for
the runs must exist.
The run scripts do not refer to any particular run directory, other than
specified when program was invoked. In the following sections we refer the
run directory as <rundir>.
4.4.1 The CSC Benchmark Suite
To run the CSC Benchmark Suite, you can use the following scripts:
o run_program - to run a single program
o run_all_programs - to run all programs after each other
Consider 'run_program'. It will run a single program in the following
manner (assume working directory CSCSUITE_2):
o Checks that <rundir> exist. If not, tries to create it.
o Creates application run directory <rundir>/<applic>.
If this already exist (from previous trial run, for example),
rename it to <rundir>/old.<applic>. And if in turn
<rundir>/old.<applic> already exist, delete all files under it.
o Copies all regular files (not directories or files under
directories) found under <applic> to <rundir>/<applic>.
o Changes directory ('cd') to <rundir>/<applic> and start running
the application.
o Upon completion, file <rundir>/<applic>/<applic>.out is copied
back to <applic>/<applic>.out.
Use 'get_times' and 'get_diffs' to collect timing information and
differences compared to reference results. 'get_times' applies directly
to <applic>/<applic>.out files for each application creating a tabulated
timing summary for application. 'get_diffs' similarly compares
differences on output files <applic>/<applic>.out and reference result
files <applic>/results/<applic>.out.
A sample execution:
% run_program RIEMANN /tmp/bench # Runs RIEMANN under /tmp/bench
% run_all_programs /tmp/bench # Runs all program in sequence
% get_times > <arch>.summary # A tabulated list of timings
% get_diffs > <arch>.diff # A list of differences
4.4.2 Parallel benchmarks
4.4.2.1 CSCSUITE_2
Generally, follow the rules in sequential CSCSUITE_2. Check also that the
processor number is correct. Following scripts are new or modified since
reading the sequential CSCSUITE_2:
o build_script
o get_parallel_times
o get_parallel_diffs
o run_parallel
A sample execution:
% run_parallel ARCTWOD /tmp/paral 2 4 # Run ARCTWOD using 2 and 4 procs
under /tmp/paral
% get_parallel_times > <arch>.summary # A tabulated list of timings
% get_parallel_diffs > <arch>.diff # A list of differences
4.4.2.2 DM
All DM-benchmarks consists of several cases to be run per application. The
input files are found in <applic>/<applic>.cases* files. Each case label
can be recognized in the input file name. For example, the case '11_8x1x1'
refers to file 'MD1/MD1.case11_8x1x1'. The case label means: run MD1 with 4
times 11 cubed atoms, use processor topology 8 by 1 by 1 processors (8
processors in ring).
It is up to vendor to choose the to choose the topology. Only number of
processors matters. For instance, the case above can be replaced with
'11_2x2x2' if the vendor think this provide faster turn-around time. But
also input file 'MD1/MD1.case11_2x2x2' with the corresponding changes must
be embedded.
If the PVM-interface is acceptable, following scripts may found helpful:
o run_dm - runs one DM-application with variable number of "cases".
o run_all_dms - runs all required DM-applications in sequence.
o run_pvm - invoked by 'run_dm'. Check whether PVM-daemon is
already running and prevents accidental starting of additional
PVM-daemon(s).
o kill_pvm - kills currently active PVM-daemon which in turn kills
all processes that communicate with this daemon.
A sample execution:
% run_dm MD1 /tmp/dm 11_2x1x1 11_8x1x1 # Runs two cases of MD1
under /tmp/dm
% kill_pvm # Be sure that PVM-daemon is down
% run_all_dms /tmp/bench # Runs all DM-programs in sequence
% kill_pvm # Be sure that PVM-daemon is down
% get_dm_times > <arch>.summary # A tabulated list of timings
4.4.3 Miscellaneous benchmarks
All the following run scripts are found under corresponding program directory.
Run CACHEMISS by typing 'cachemiss.csh'. Apply 'matlabgen.csh' to get
Matlab-suitable data file for curve plotting.
Run LARGEMEM by typing 'generate.csh'.
Run MEMSCAN by typing 'run_memscan'.
Run IOZONE by typing 'run_iozone'. And 'get_times' applied to logfile.
Run BRE kernel test by typing 'run_bre'. Modify BRE.dat -file before that.
Run communication test 'COMMS1' as a part of DM-application.
5. Contact information
Any questions about the benchmark should be directed to me or Klaus
Lindberg. I will be in holiday between April 13 and 25. Here is more
information:
Sami Saarinen (or Klaus Lindberg)
Center for Scientific Computing (CSC)
Tietotie 6
P.O.Box 405
FIN-02101 Espoo
Finland
Tel: Int + 358 - 0 - 457 2713 (Sami, direct)
Int + 358 - 0 - 457 4050 (Klaus, direct)
Int + 358 - 0 - 457 1 (switch board)
Fax: Int + 358 - 0 - 457 2302
And as stated in section 4.1, the benchmark file is obtainable via
anonymous ftp in case of tape reading problems.
The results should be send to us in printed format and, if possible, in a
tar-file using streamer tapes, reels or so. We can even provide you a ftp
access to some specific place at our center to send us the result data.
Appendix A
Configuration of the CSC reference computer system:
(1) Hardware Configuration:
Manufacturer: Silicon Graphics Inc.
Model number: INDIGO R4000
CPU: MIPS R4000 Processor Chip Revision: 2.2
FPU: MIPS R4010 Floating Point Chip Revision: 0.0
Speed: 50 MHZ IP20 Processor
Peak performance: 50 Mflop/s (64-bit)
Number of CPUs: 1
Data cache size: 8 Kbytes
Instruction cache size: 8 Kbytes
Secondary unified instruction/data cache size: 1 Mbyte
Main memory size: 96 Mbytes
Disk subsystem: 1.6GB + 1.2GB + 0.4GB SCSI
Other Hardware: None
Network Interface: Integral Ethernet: ec0, version 1
(2) Software Configuration:
O/S & Version: IRIX 4.0.5F
Compilers & Version: SGI Fortran 77, 3.4.1
SGI Ansi C, 1.1
Compiler flags: -O2 -static -sopt,-so=3,-r=3,-ur=8 -jmpopt -lfastm
(except in programs CASTEP, GRSOS, MOPAC, NBODYOPT,
RIEMANN, SIMCZO, WHY12M and MD1 where
"-O2 -static" was used)
Other Software: Fortran 77 Fopt Scalar Optimizer (KAP)
File system type: SGI efs
(3) System Environment:
System state: Multi-user
Tuning Parameters: None
Background load: None
Appendix B
CFP92, current release: Rel. 1.1:
This suite contains 14 benchmarks performing floating-point
computations. 12 of them are written in Fortran, 2 in C. The
individual programs are:
013.spice2g6 Simulates analog circuits (double precision).
015.doduc Performs Monte-Carlo simulation of the time evolution
of a thermo-hydraulic model for a nuclear reactor's
component (double precision).
034.mdljdp2 Solves motion equations for a model of 500 atoms
interacting through the idealized Lennard-Jones
potential (double precision).
039.wave5 Solves particle and Maxwell's equations on a
Cartesian mesh (single precision).
047.tomcatv Generates two-dimensional, boundary-fitted coordinate
systems around general geometric domains
(vectorizable, double precision).
048 ora Traces rays through an optical surface containing
spherical and planar surfaces (double precision).
052.alvinn Trains a neural network using back propagation
(single precision).
056.ear Simulates the human ear by converting a sound file to
a cochleogram using Fast Fourier Transforms and other
math library functions (single precision).
077.mdljsp2 Similar to 034.mdljdp2, solves motion equations for a
model of 500 atoms (single precision).
078.swm256 Solves the system of shallow water equations using
finite difference approximations (single precision).
089.su2cor Calculates masses of elementary particles in the
framework of the Quark Gluon theory (vectorizable,
double precision).
090.hydro2d Uses hydrodynamical Navier Stokes equations to
calculate galactical jets (vectorizable, double
precision).
093.nasa7 Executes seven program kernels of operations used
frequently in NASA applications, such as Fourier
transforms and matrix manipulations (double
precision).
094.fpppp Calculates multi-electron integral derivatives
(double precision).
CINT92, current release: Rel. 1.1:
This suite contains 6 benchmarks performing integer computations,
all of them are written in C. The individual programs are:
008.espresso Generates and optimizes Programmable Logic Arrays.
022.li Uses a LISP interpreter to solve the nine queens
problem, using a recursive backtracking algorithm.
023.eqntott Translates a logical representation of a Boolean
equation to a truth table.
026.compress Reduces the size of input files by using Lempel-Ziv
coding.
072.sc Calculates budgets, SPEC metrics and amortization
schedules in a spreadsheet based on the UNIX cursor-
controlled package "curses".
085.gcc Translates preprocessed C source files into optimized
Sun-3 assembly language output.
Appendix C
Use following guidelines to report system configuration under test:
(1) Hardware Configuration:
A description of the system (cpu/clock, fpu/clock), number of processors,
relevant peripherals, etc. are included in this space. The amount of
information supplied should be sufficient to allow duplication of results
by another party.
The following checklist is provided to show examples of hardware features
which may affect the performance of benchmarks. The checklist is not
intended to be all-inclusive, nor is each feature in the list required to
be described. The rule of thumb is: "if it affects performance or the
feature is required to duplicate the results, describe it":
o Manufacturer
o Model number
o CPU component ID and clock speed
o Floating Point Unit (FPU) ID and clock speed
o Theoretical peak performance in Mflop/s (32- and 64-bit
floating point operations) per processor
o Number of CPUs, FPUs and vector units
o Cache Size (per CPU), description and organization
o Memory (Amount and description)
o Disk subsystem configuration
- Number of active connected disks
- Disk controllers: ID, number and type
- Disk: ID, number and type
o Other Hardware
o Network Interface
(2) Software Configuration:
The description of the software configuration used should be detailed
enough so that another party could duplicate the results reported. The
following is a list of items that should be documented:
o Operating System Type and Release level (with
revision/update level if appropriate)
o Compiler release levels used. If multiple compilers
are available, specify which ones were used and when
o Compiler flags used per test program/source file
o Other Software required to reproduce results
o File system type used
o Firmware level
(3) System Environment:
This section should document the overall systems environment used to run
the benchmark. The following is a list of possible items that might be
documented:
o Single or multi-user state
o System tuning parameters
o Process tuning parameters (e.g. stack size, time slice)
o Background load, if any
Appendix D
CSC Benchmark Suite: Version 2
______________________________
ARCTWOD Program solves the Euler equations in generalized curvelinear
coordinates using implicit finite-difference method with
approximated factorizations and a diagonalization of the
implicit operators. The test case involves with 0.8-Mach flow
over upper of a biconvex airfoil in a stretched 165x48
rectangular mesh.
The original code (ARC2D) was developed by Thomas Pulliam at
the NASA Ames Research Center.
CASTEP Program stands for CAmbridge Serial Total Energy Package and
is an Ab Initio Molecular Dynamics program with Conjugate
Gradient minimization for the electronic energy.
The Al-Si-Al trimer, arranged originally in linear geometry,
is calculated using Car-Parrinello-type first principles
density functional method. The wave functions are expanded
using plane waves, and the Fourier coefficients are solved
using Conjugate Gradient technique. The atomic positions are
solved by minimizing the total energy with respect to the
atomic coordinates.
The example program does 25 iterations.
CASTEP is maintained and partially developed by Victor Milman
at the Cambridge University, UK. The input file for the test
runs was prepared by Ari Seitsonen at the Helsinki University
of Technology.
FREQUENCY Program creates frequency dependent complex transfer function
matrix and calculates magnitude and phase angle for a given
excitation in a given degree of freedom. To accomplish the
task, the program uses 50 unevenly distributed excitation
frequencies in a range of (0,24] Hz and ten lowest
eigenvectors, that represent dynamic behaviour of a given
structure in that range.
The original code was written as a part of Sami Saarinen's
Master Thesis on "A Numerical Study of the Harmonical
Vibrations on the Off-Machine Coater".
GRSOS Program is a restricted version of the solid-on-solid (SOS)
growth model. It is basically an algorithm that models
the deposition of particles on a substrate. The restriction
is that the deposition occurs only of the difference in height
between the site and its nearest neighbors is less than one.
The SOS model itself does not allow overhangs or bubbles.
The model can be used to study the temporal development of
growth processes.
The problem size is set to 1000 and program makes 5 iterations
through the Monte-Carlo loop.
GRSOS was developed by Tuomo Ala-Nissila at the Helsinki
University of Technology while visiting the Brown University.
INVPOW93 Program extracts 4 lowest eigenvalues and eigenvectors of a
given symmetric 605x605 matrix. It uses Inverse Power method
with Deflation after each converged (tolerance is 1.0E-6)
eigenvalue. The Deflation process is carried out by using
temporary disk storage and contains relatively large amount of
sequential binary I/O.
INVPOW93 was developed by Sami Saarinen at the CSC.
MOPAC Program is a general purpose semi-empirical molecular orbital
package for the study of chemical structures and reactions.
In the test case the thermodynamic quantities of anisole
(internal energy, heat capacity, partition function, and
entropy) are calculated by using the AM1-method for
translation rotation and vibrational degrees of freedom for
the default temperature range 200-400 K.
Version of the program is 5.0. MOPAC was developed by James J.
P. Stewart at the US Air Force Academy.
The input file was prepared by Raimo Uusvuori at the CSC.
NASKER Program executes 7 program kernels used frequently in NASA
and other scientific computing applications. These kernel
operations are: MXM to perform matrix product on two input
matrices, CFFT2D to perform radix 2 FFT on a two dimensional
input array, CHOLSKY to perform a Cholesky decomposition in
parallel on a set of input matrices, BTRIX to perform a block
tridiagonal matrix solution, GMTRY to set up arrays for a
vortex method solution and to perform Gaussian elimination on
the resulting array, EMIT to create new vortices according to
certain boundary conditions, and VPENTA to simultaneously
invert three matrix pentadiagonals in a highly parallel
fashion.
NASKER was collected from other NASA programs by David
Bailey and John Barton at the NASA Ames Research Center.
NBODYOPT This program computes the relative locations of ten planets by
integrating equations of motions with the 7th order
Adams-Stoermer -difference method. Program contains a lot of
trigonometric and other intrinsic function calls.
The original code was written in Algol and converted to
Fortran-77 by Hannu Karttunen at the CSC. The current version
contains some obvious code optimization constructs that may
remove unnecessary cache memory conflicts.
RIEMANN Program calculates period matrices of real algebraic curves.
Given a set of generators and fixed points of the generators,
the algorithm calculates the period matrices using a partial
sum approximation (using elements of the group up to a given
word length). The algorithm is recursive and uses
complex-number arithmetic programmed in C. Numerical under
and overflow situations are not checked for, so one should use
normalized matrices as input and limit the word length to a
small enough number.
RIEMANN was developed and input file prepared by Juha Haataja
at the CSC.
SIMCZO Program is a development version of a numerical simulation for
the Czochralski crystal growth. Based on the Finite Element
Method, program simulates the crystal growth in the
axisymmetric case including the free boundaries between the
solid and liquid and liquid and gas, respectively. The fluid
flow in the liquid is governed by the coupled Navier-Stokes
and diffusion-convection equations. The temperature
distribution in the crystal is governed by the
diffusion-convection equation. SIMCZO was developed and input
file prepared by Jari Jarvinen at the CSC.
WHY12M Program solves a system of linear equations by Gaussian
elimination using certain sparse matrix techniques. The
program is designed to solve efficiently problems which
contain only one system with a single right hand side.
The order of the matrix is set to 4500 and the population of
non-zero elements is less than 1%. Program writes a
checkpoint-file to make an efficient restarts possible.
The original code (Y12M) was developed at the Regional
Computing Center at the University of Copenhagen.
The input file was created by Sami Saarinen at the CSC.
MD1 This Molecular dynamics program essentially involves solution
of the equations of motion for a system of a large number of
interacting particles. The problem is solved numerically by
calculating approximate solutions of the second order
differential equations at a large number of time steps in
a given interval 'dt'. The particles are considered to
interact through an effective pair potential which is adjusted
to reproduce experimental results and is used to model the
exact many-body potential. For monatomic neutral atoms the
form of pair potential most often employed is the
Lennard-Jones potential.
A typical MD calculation for N particles involves for each
time step calculation of the forces on the N particles and
their new positions, and calculation of energies and radial
distribution functions.
In this test the number of atoms is 4 times NC cubed, where
NC is set to 11 in a non-distibuted memory version, thus
simulation is carried out with 5324 atoms. The number of time
steps is set to 600.
MD1 was developed by Mark Pinches at the University of
Southampton, UK. Sequential and distributed memory versions
are part of the European parallel benchmarking effort, Genesis.
PDE1 Program solves the Poisson-Equation on a 3-dimensional
grid by parallel red-black relaxation. It is an extreme
example of the class of PDE solvers, as due to the simplicity
of the discretization of Poisson's equation the number of
floating point operations per gridpoint is quite small
relative to more complex PDEs. The ratio of computation
to communication is thus rather low.
The parallelization is performed by grid splitting. A part of
the computational grid is assigned to each processor. After
each computational step, values at the boundary of the
subgrids are exchanged with nearest neighbours.
The sequential version automatically produces results for a
range of problem sizes. The problem size is determined by the
grid size, which is related to the parameter N. The number of
grid points in each direction is 2**N + 1.
For the non-distibuted memory version N is set to 7 and
standard implementation of the red-black relaxation is used.
PDE1 was developed by J. Klose and M. Lemke at the PALLAS
GmbH. Sequential and distributed memory versions
are part of the European parallel benchmarking effort, Genesis.
QCD1 Program is based on a 'pure gluon' SU(3) lattice gauge theory
simulation, using the Monte-Carlo heatbath technique. It
uses the 'quenched' approximation which neglects dynamical
fermions. The simulation is defined on a four-dimensional
lattice which is a discrete approximations to continuum
space-time. The basic variables are 3 by 3 complex matrices.
Four such matrices are associated with every lattice site. The
lattice update is performed using a multi-hit Metropolis
algorithm.
In the parallel version of the program, the lattice can be
distributed in any one or more of the four lattice directions.
The lattice size is based on a 4-dimensional space-time
lattice of size: N = NS **3 * NT, where NT & NS are even
integers.
For the non-distibuted memory version NT & NS are set to 8 and
start configuration is disordered.
QCD1 was developed by Eckardt Kehl at PALLAS GmbH.
Sequential and distributed memory versions are part of the
European parallel benchmarking effort, Genesis.
Appendix E
PARMACS macro calls used in this benchmarks:
[Ref: Parallel Computing 15 (1990) 199-132, North-Holland]
ENVHOST Declaration macros for host and node program. Put before the
ENVNODE first data or executable statement in Fortran-77 source.
INITHOST The first executable statement in host and node main program.
INITNODE
TORUS(nx,ny,nz,slave,process_file)
Sets up a (nx,ny,nz) grid of (logical) processors and defines
the executable to be named 'slave'. Information to be used by
'REMOTE_CREATE' is passed to the temporary file 'process_file'
(not used by the PVM-interface).
REMOTE_CREATE(process_file,process_id_vector)
A macro for process creation. Read information from
'process_file' that was written by TORUS-macro. In return
'process_id_vector' contains instance numbers (thread/process
id's) of just created 'slave's.
MYPROC A variable to identify own process id.
HOSTID A variable to identify host process id.
SEND(target_id,buffer,buffer_len,msgtype)
Asyncroniously send 'buffer_len' amount of bytes pointed by
'buffer' to the process id 'target_id'. The message
identification type is 'msgtype'.
(Note: My PVM-interface encodes the 'msgtype' to be internally
(during send) 'msgtype * MAXNODES + target_id'.
This made the PARMACS RECV easier to implement)
SENDR(target_id,buffer,buffer_len,msgtype)
Same as SEND(...), but syncronious. (PVM-interface: SEND).
RECV(buffer,buffer_len,actual_len,sender,msgtype,condition)
Receives 'buffer' of data which length cannot exceed
'buffer_len' bytes. The actual length will be stored in
'actual_len'. The sender instance was 'sender' and the message
type was actually 'msgtype'. Receive is made after 'condition'
is fulfilled:
o MATCH_ID(select_sender) : Receives any message that comes
next from the selected sender.
o MATCH_TYPE(select_type) : Receives any message that has
the selected type.
o MATCH_ID_AND_TYPE(select_sender,select_type) : Both of the
previous conditions must be true.
o MATCH_ANY : Receive any message that arrives.
(PVM-interface: RECV polls receive buffer until respective
UDP-message that fulfills the 'condition' has been arrived)
RECVR(buffer,buffer_len,actual_len,sender,msgtype,condition)
Same as RECV(...), but syncronious. (PVM-interface: RECV).
CLEAN_UP This macro MUST BE the last statement to be executed before
leaving the host or node program.
(PVM-interface: Nodes stay in 'barrier()' until all nodes have
reached the barrier. When host exits, it tries
to 'terminate()' possible hanging nodes.)