Towards compiler-aided correctness checking of adjoint MPI applications

Algorithmic Differentiation (AD) is a set of techniques to calculate derivatives of a computer program. In C++, AD typically requires (i) a type change of the built-in double, and (ii) a replacement of all MPI calls with AD-specific implementations. This poses challenges on MPI correctness tools, such as MUST, a dynamic checker, and TypeART, its memory sanitizer extension. In particular, AD impacts (i) memory layouts of the whole code, (ii) requires more memory allocations tracking by TypeART, and (iii) approximately doubles the MPI type checks of MUST due to an AD-specific communication reversal. To address these challenges, we propose a new callback interface for MUST to reduce the number of intercepted MPI calls, and, also, improve the filtering capabilities of TypeART to reduce tracking of temporary allocations for the derivative computation. We evaluate our approach on an AD-enhanced version of CORAL LULESH. In particular, we reduce stack variable tracking from 32 million to 13 thousand. MUST with TypeART and the callback interface reduces the runtime overhead to that of vanilla MUST.