This study proposes a new formulation of singular boundary method (SBM) to solve the 2D potential problems, while retaining its original merits being free of integration and mesh, easy-to-program, accurate and mathematically simple without the requirement of a fictitious boundary as in the method of fundamental solutions (MFS). The key idea of the SBM is to introduce the concept of the origin intensity factor to isolate the singularity of fundamental solution so that the source points can be placed directly on the physical boundary. This paper presents a new approach to derive the analytical solution of the origin intensity factor based on the proposed subtracting and adding-back techniques. And the troublesome sample nodes in the ordinary SBM are avoided and the sample solution is also not necessary for the Neumann boundary condition. Three benchmark problems are tested to demonstrate the feasibility and accuracy of the new formulation through detailed comparisons with the boundary element method (BEM), MFS, regularized meshless method (RMM) and boundary distributed source (BDS) method.