Minimax optimization problems are an important class of optimization problems arising from modern machine learning and traditional research areas. While there have been many numerical algorithms for solving smooth convex-concave minimax problems, numerical algorithms for nonsmooth convex-concave minimax problems are rare. This paper aims to develop an efficient numerical algorithm for a structured nonsmooth convex-concave minimax problem. A semi-proximal point method (SPP) is proposed, in which a quadratic convex-concave function is adopted for approximating the smooth part of the objective function and semi-proximal terms are added in each subproblem. This construction enables the subproblems at each iteration are solvable and even easily solved when the semiproximal terms are cleverly chosen. We prove the global convergence of our algorithm under mild assumptions, without requiring strong convexity-concavity condition. Under the locally metrical subregularity of the solution mapping, we prove that our algorithm has the linear rate of convergence. Preliminary numerical results are reported to verify the efficiency of our algorithm.