In this work, an inverse averaging finite element method (IAFEM) is developed for solving the size-modified Poisson-Nernst-Planck (SMPNP) equations. Comparing with the classical Poisson-Nernst-Planck (PNP) equations, the SMPNP equations add a nonlinear term to each of the Nernst-Planck (NP) fluxes to describe the steric repulsion which can treat multiple nonuniform particle sizes in simulations. Since the new terms include sums and gradients of ion concentrations, the nonlinear coupling of SMPNP equations is much stronger than that of PNP equations. By introducing a generalized Slotboom transform, each of the size-modified NP equation is transformed into a self-adjoint equation with exponentially behaved coefficient, which has similar simple form to the standard NP equation with the Slotboom transformation. This treatment enables employing our recently developed inverse averaging technique to deal with the exponential coefficients of the reformulated formulations, featured with advantages of numerical stability and flux conservation especially in strong nonlinear and convection-dominated cases. Comparing with previous stabilization methods, the IAFEM developed in this paper can still possess the numerical stability when dealing with convection-dominated problems. And it is more concise and easier to be numerically implemented. Numerical experiments about a model problem with analytic solutions are presented to verify the accuracy and order of IAFEM for SMPNP equations. Studies about the size-effects of a sphere model and an ion channel system are presented to show that our IAFEM is more effective and robust than the traditional finite element method (FEM) when solving SMPNP equations in simulations of biological systems.