Implementation of implicit finite element methods for incompressible flows on the CM-5

A parallel implementation of an implicit finite element formulation for incompressible fluids on a distributed-memory massively parallel computer is presented. The dominant issue that distinguishes the implementation of finite element problems on distributed-memory computers from that on traditional shared-memory scalar or vector computers is the distribution of data (and hence workload) to the processors and the non-uniform memory hierarchy associated with the processors, particularly the non-uniform costs associated with on-processor and off-processor memory references. Accessing data stored in a remote processor requires computing resources an order of magnitude greater than accessing data locally in a processor. This distribution of data motivates the development of alternatives to traditional algorithms and data structures designed for shared-memory computers, which must now account for distributed-memory architectures. Data structures as well as data decomposition and data communication algorithms designed for distributed-memory computers are presented in the context of high level language constructs from High Performance Fortran. The discussion relies primarily on abstract features of the hardware and software environment and should be applicable, in principle, to a variety of distributed-memory system. The actual implementation is carried out on a Connection Machine CM-5 system with high performance communication functions.