TY - JOUR T1 - EMPIRE-PIC: A Performance Portable Unstructured Particle-in-Cell Code AU - T. Bettencourt , Matthew AU - A. S. Brown , Dominic AU - L. Cartwright , Keith AU - C. Cyr , Eric AU - A. Glusa , Christian AU - T. Lin , Paul AU - G. Moore , Stan AU - A. O. McGregor , Duncan AU - Pawlowski , Roger P. AU - Phillips , Edward G. AU - Roberts , Nathan V. AU - Wright , Steven A. AU - Maheswaran , Satheesh AU - Jones , John P. AU - A. Jarvis , Stephen JO - Communications in Computational Physics VL - 4 SP - 1232 EP - 1268 PY - 2021 DA - 2021/08 SN - 30 DO - http://doi.org/10.4208/cicp.OA-2020-0261 UR - https://global-sci.org/intro/article_detail/cicp/19400.html KW - PIC, electrostatics, electromagnetics, HPC, performance portability. AB -

In this paper we introduce EMPIRE-PIC, a finite element method particle-in-cell (FEM-PIC) application developed at Sandia National Laboratories. The code has been developed in C++ using the Trilinos library and the Kokkos Performance Portability Framework to enable running on multiple modern compute architectures while only requiring maintenance of a single codebase. EMPIRE-PIC is capable of solving both electrostatic and electromagnetic problems in two- and three-dimensions to second-order accuracy in space and time. In this paper we validate the code against three benchmark problems — a simple electron orbit, an electrostatic Langmuir wave, and a transverse electromagnetic wave propagating through a plasma. We demonstrate the performance of EMPIRE-PIC on four different architectures: Intel Haswell CPUs, Intel's Xeon Phi Knights Landing, ARM Thunder-X2 CPUs, and NVIDIA Tesla V100 GPUs attached to IBM POWER9 processors. This analysis demonstrates scalability of the code up to more than two thousand GPUs, and greater than one hundred thousand CPUs.