The dyadic Green's function in multi-layer structures for Maxwell equations is a key component for the integral equation method, but time consuming to calculate. A novel algorithm, the Fast Interpolation and Filtering Algorithm (FIFA), for the calculation of the dyadic Green's function in multi-layer structures is proposed in this paper. We discuss in specific details, ready for use in practical calculations of scattering in layer media, how to apply FIFA to calculate various components of the dyadic Green's function. The algorithm is based on two techniques: interpolation of Green's function both in the spectral domain and spatial domain, and low pass filter window based acceleration. Compared to the popular Complex Image Method (CIM), FIFA provides the same speed and overcomes several difficulties associated with CIM while being more general and robust. Specifically, there are no limitations on the frequency range, the number of layers in the structure and the type of Green's functions to be calculated, and moreover, no need to extract surface wave poles from the spectral form of the Green's function. Numerical results are given to demonstrate the efficiency and robustness of the proposed method.