The concrete aggregate model is considered as a type of weakly discontinuous problem consisting of three phases: aggregates which randomly distributed in different shapes, cement paste and internal transition zone (ITZ). Because of different shapes of aggregate and thin ITZs, a huge number of elements are often used in the finite element method (FEM) analysis. In order to ensure the accuracy of the numerical solutions near the interfaces, we need to use higher-order elements. The widely used FEM softwares such as ANSYS and ABAQUS all provide the option of quadratic elements. However, they have much higher computational complexity than the linear elements. The corresponding coefficient matrix of the system of equations is a highly ill-conditioned matrix due to the large difference between three phase materials, and the convergence rate of the commonly used solving methods will deteriorate. In this paper, two types of simple and efficient preconditioners are proposed for the system of equations of the concrete aggregate models on unstructured triangle meshes by using the resulting hierarchical structure and the properties of the diagonal block matrices. The main computational cost of these preconditioners is how to efficiently solve the system of equations by using linear elements, and thus we can provide some efficient and robust solvers by calling the existing geometric-based algebraic multigrid (GAMG) methods. Since the hierarchical basis functions are used, we need not present those algebraic criterions to judge the relationships between the unknown variables and the geometric node types, and the grid transfer operators are also trivial. This makes it easy to find the linear element matrix derived directly from the fine level matrix, and thus the overall efficiency is greatly improved. The numerical results have verified the efficiency of the resulting preconditioned conjugate gradient (PCG) methods which are applied to the solution of several typical aggregate models.