An efficient p-multigrid method is developed to solve the algebraic systems which result from the approximation of elliptic problems with the so-called Fekete-Gauss Spectral Element Method, which makes use of the Fekete points of the triangle as interpolation points and of the Gauss points as quadrature points. A multigrid strategy is defined by comparison of different prolongation/restriction operators and coarse grid algebraic systems. The efficiency and robustness of the approach, with respect to the type of boundary condition and to the structured/unstructured nature of the mesh, are highlighted through numerical examples.