Abstract: Special challenges exist in writing reliable numerical library software for heterogeneous
computing environments. Although a lot of software for distributed-memory parallel computers
has been written, porting this software to a network of workstations requires careful
consideration. The symptoms of heterogeneous computing failures can range from erroneous
results without warning to deadlock. Some of the problems are straightforward to solve, but
for others the solutions are not so obvious, or incur an unacceptable overhead. Making
software robust on heterogeneous systems often requires additional communication. We
describe and illustrate the problems encountered during the development of ScaLAPACK and
the NAG Numerical PVM Library. Where possible, we suggest ways to avoid potential pitfalls,
or if that is not possible, we recommend that the software not be used on heterogeneous
networks.
Download: pdf
Citation
- Practical Experience in the Numerical Dangers of Heterogeneous Computing (pdf, software)
L. Blackford, A. Cleary, J. Demmel, I. Dhillon, J. Dongarra, S. Hammarling, A. Petitet, H. Ren, K. Stanley, R. Whaley.
ACM Transactions on Mathematical Software 23(2), pp. 133-147, June 1997.
Bibtex: