In this paper, the electromagnetic scattering from a rectangular large open cavity embedded in an infinite ground plane is studied. By introducing a nonlocal artificial boundary condition, the scattering problem from the open cavity is reduced to a bounded domain problem. A compact fourth order finite difference scheme is then proposed to discrete the cavity scattering model in the rectangular domain, and a special treatment is enforced to approximate the boundary condition, which makes truncation errors reach $\mathcal{O}(h^4)$ in the whole computational domain. A fast algorithm, exploiting the discrete Fourier transformation in the horizontal and a Gaussian elimination in the vertical direction, is employed, which reduces the discrete system to a much smaller interface system. An effective preconditioner is presented for the BICGstab iterative solver to solve this interface system. Numerical results demonstrate the remarkable accuracy and efficiency of the proposed method. In particular, it can be used to solve the cavity model for the large wave number up to $600\pi$.