Synaptic strengths between neurons are plastic and modified by spontaneous activity and information from the outside. There is increasing interest in the impact of correlated neuron activity and learning rules on global network structure. Here the networks of exponential integrate-and-fire neurons with spike timing-dependent plasticity (STDP) learning rules are considered, by providing the theoretical approximation of spiking cross-covariance between connected neurons and the theory for the evolution of synaptic weights. Background input mean and variance highly affect the spiking covariance, even for the fixed baseline firing rate and connection. Through analyzing the effects of covariance and STDP on vector fields for pairwise correlated neurons under fixed baseline firing rate, we show that the connections from a neuron with lower input mean to that with higher one will strengthen for balanced Hebbian STDP. However, this situation is reversed for Anti-Hebbian cases. Moreover, for potentiation dominated STDP, the synaptic weights for the networks of neurons with lower input mean are more likely to be enhanced. In addition, these properties found from coupled neurons also hold for large recurrent networks in both theories and simulations. This study provides a self-consistent theoretical method for understanding how correlated spiking activity and STDP shape the network structure and an approach for predicting structures of large networks through the analysis of simple neural circuits.