arrow
Volume 30, Issue 4
EMPIRE-PIC: A Performance Portable Unstructured Particle-in-Cell Code

Matthew T. Bettencourt, Dominic A. S. Brown, Keith L. Cartwright, Eric C. Cyr, Christian A. Glusa, Paul T. Lin, Stan G. Moore, Duncan A. O. McGregor, Roger P. Pawlowski, Edward G. Phillips, Nathan V. Roberts, Steven A. Wright, Satheesh Maheswaran, John P. Jones & Stephen A. Jarvis

Commun. Comput. Phys., 30 (2021), pp. 1232-1268.

Published online: 2021-08

Export citation
  • Abstract

In this paper we introduce EMPIRE-PIC, a finite element method particle-in-cell (FEM-PIC) application developed at Sandia National Laboratories. The code has been developed in C++ using the Trilinos library and the Kokkos Performance Portability Framework to enable running on multiple modern compute architectures while only requiring maintenance of a single codebase. EMPIRE-PIC is capable of solving both electrostatic and electromagnetic problems in two- and three-dimensions to second-order accuracy in space and time. In this paper we validate the code against three benchmark problems — a simple electron orbit, an electrostatic Langmuir wave, and a transverse electromagnetic wave propagating through a plasma. We demonstrate the performance of EMPIRE-PIC on four different architectures: Intel Haswell CPUs, Intel's Xeon Phi Knights Landing, ARM Thunder-X2 CPUs, and NVIDIA Tesla V100 GPUs attached to IBM POWER9 processors. This analysis demonstrates scalability of the code up to more than two thousand GPUs, and greater than one hundred thousand CPUs.

  • Keywords

PIC, electrostatics, electromagnetics, HPC, performance portability.

  • AMS Subject Headings

  • Copyright

COPYRIGHT: © Global Science Press

  • Email address
  • BibTex
  • RIS
  • TXT
@Article{CiCP-30-1232, author = {Matthew and T. Bettencourt and and 17867 and and Matthew T. Bettencourt and Dominic and A. S. Brown and and 17868 and and Dominic A. S. Brown and Keith and L. Cartwright and and 17869 and and Keith L. Cartwright and Eric and C. Cyr and and 17870 and and Eric C. Cyr and Christian and A. Glusa and and 17871 and and Christian A. Glusa and Paul and T. Lin and and 17872 and and Paul T. Lin and Stan and G. Moore and and 17873 and and Stan G. Moore and Duncan and A. O. McGregor and and 17874 and and Duncan A. O. McGregor and Roger P. and Pawlowski and and 17875 and and Roger P. Pawlowski and Edward G. and Phillips and and 17876 and and Edward G. Phillips and Nathan V. and Roberts and and 17883 and and Nathan V. Roberts and Steven A. and Wright and and 17884 and and Steven A. Wright and Satheesh and Maheswaran and and 17885 and and Satheesh Maheswaran and John P. and Jones and and 17886 and and John P. Jones and Stephen and A. Jarvis and and 17887 and and Stephen A. Jarvis}, title = {EMPIRE-PIC: A Performance Portable Unstructured Particle-in-Cell Code}, journal = {Communications in Computational Physics}, year = {2021}, volume = {30}, number = {4}, pages = {1232--1268}, abstract = {

In this paper we introduce EMPIRE-PIC, a finite element method particle-in-cell (FEM-PIC) application developed at Sandia National Laboratories. The code has been developed in C++ using the Trilinos library and the Kokkos Performance Portability Framework to enable running on multiple modern compute architectures while only requiring maintenance of a single codebase. EMPIRE-PIC is capable of solving both electrostatic and electromagnetic problems in two- and three-dimensions to second-order accuracy in space and time. In this paper we validate the code against three benchmark problems — a simple electron orbit, an electrostatic Langmuir wave, and a transverse electromagnetic wave propagating through a plasma. We demonstrate the performance of EMPIRE-PIC on four different architectures: Intel Haswell CPUs, Intel's Xeon Phi Knights Landing, ARM Thunder-X2 CPUs, and NVIDIA Tesla V100 GPUs attached to IBM POWER9 processors. This analysis demonstrates scalability of the code up to more than two thousand GPUs, and greater than one hundred thousand CPUs.

}, issn = {1991-7120}, doi = {https://doi.org/10.4208/cicp.OA-2020-0261}, url = {http://global-sci.org/intro/article_detail/cicp/19400.html} }
TY - JOUR T1 - EMPIRE-PIC: A Performance Portable Unstructured Particle-in-Cell Code AU - T. Bettencourt , Matthew AU - A. S. Brown , Dominic AU - L. Cartwright , Keith AU - C. Cyr , Eric AU - A. Glusa , Christian AU - T. Lin , Paul AU - G. Moore , Stan AU - A. O. McGregor , Duncan AU - Pawlowski , Roger P. AU - Phillips , Edward G. AU - Roberts , Nathan V. AU - Wright , Steven A. AU - Maheswaran , Satheesh AU - Jones , John P. AU - A. Jarvis , Stephen JO - Communications in Computational Physics VL - 4 SP - 1232 EP - 1268 PY - 2021 DA - 2021/08 SN - 30 DO - http://doi.org/10.4208/cicp.OA-2020-0261 UR - https://global-sci.org/intro/article_detail/cicp/19400.html KW - PIC, electrostatics, electromagnetics, HPC, performance portability. AB -

In this paper we introduce EMPIRE-PIC, a finite element method particle-in-cell (FEM-PIC) application developed at Sandia National Laboratories. The code has been developed in C++ using the Trilinos library and the Kokkos Performance Portability Framework to enable running on multiple modern compute architectures while only requiring maintenance of a single codebase. EMPIRE-PIC is capable of solving both electrostatic and electromagnetic problems in two- and three-dimensions to second-order accuracy in space and time. In this paper we validate the code against three benchmark problems — a simple electron orbit, an electrostatic Langmuir wave, and a transverse electromagnetic wave propagating through a plasma. We demonstrate the performance of EMPIRE-PIC on four different architectures: Intel Haswell CPUs, Intel's Xeon Phi Knights Landing, ARM Thunder-X2 CPUs, and NVIDIA Tesla V100 GPUs attached to IBM POWER9 processors. This analysis demonstrates scalability of the code up to more than two thousand GPUs, and greater than one hundred thousand CPUs.

Matthew T. Bettencourt, Dominic A. S. Brown, Keith L. Cartwright, Eric C. Cyr, Christian A. Glusa, Paul T. Lin, Stan G. Moore, Duncan A. O. McGregor, Roger P. Pawlowski, Edward G. Phillips, Nathan V. Roberts, Steven A. Wright, Satheesh Maheswaran, John P.. (2021). EMPIRE-PIC: A Performance Portable Unstructured Particle-in-Cell Code. Communications in Computational Physics. 30 (4). 1232-1268. doi:10.4208/cicp.OA-2020-0261
Copy to clipboard
The citation has been copied to your clipboard