Advancing electron beam applications require pushing toward the quantum degeneracy limit. Nanoscale structured cathodes are a promising electron source for this regime, but the numerical tools for studying these designs remain limited. A previous paper detailed the implemented of a flat-panel fast-multipole-accelerated boundary element method, which solves the relevant Poisson problem. However, flat panels are inadequate and inefficient for representing curved surfaces at the high precision necessary for many applications. Additionally, the boundary element method has an established numerical instability when evaluated near the domain boundary. To resolve this, a general high-order curvilinear element interpolation and modified quadrature method is developed utilizing a differential algebraic mapping for greater accuracy in the boundary surface representation. The boundary instability effect is mitigated by devising local corrections to the quadrature scheme in the form of Cartesian Taylor expansions. This approach is suitably general, requiring only small modifications for application to other kernels, and can easily be incorporated into a fast multipole accelerated framework. The refined algorithm is evaluated with respect to both accuracy and efficiency using several analytic structures and the performance capacity is highlighted by the capability of accurately determining the field enhancement factor for a single nanotip electron cathode.