This paper proposes a stable and efficient implicit block Lower-Upper Symmetric-Gauss-Seidel (LU-SGS) algorithm-based lattice Boltzmann flux solver (LBFS) for simulation of hypersonic flows. In this method, the finite volume method (FVM) is applied to discretize the Navier-Stokes equations, and the LBFS is utilized to evaluate the numerical flux at the cell interface. In LBFS, the local solution of discrete velocity Boltzmann equation (DVBE) with the non-free parameter D1Q4 lattice Boltzmann model is adopted to reconstruct the inviscid flux across the cell interface, and the viscous flux is approximated by conventional smooth function approach. In order to improve the robustness and convergence rate of the simulation for hypersonic flows, especially for problems with complex geometry, the implicit block LU-SGS algorithm is introduced to solve resultant discrete governing equations. A double cone model at Mach number of $Ma=9.86$ is firstly simulated to validate the proposed scheme, and a hypersonic flight vehicle with wings and rudders at Mach number of $Ma=5.56$ is then calculated to extend the application in practical engineering problems. Numerical results show that the proposed scheme could offer a more accurate and effective prediction for hypersonic flows.