fMRI studies have suggested that there are two different attention networks: the dorsal attention network (DAN) and the ventral attention network (VAN). The emotion network has also been discovered by some researches. The dorsal attention network controls goal-oriented top-down deployment of attention; the ventral attention network controls stimulus-driven bottom-up deployment of attention; the emotion network will feed back the stimulus especially fearful expressions from the environment. The interaction within these networks has been noticed but few studies have been carried out. The purpose of this study is to explore the interaction within these networks. The regions of interest were acquired by using the GLM analysis after which the granger causality among these ROIs was calculated. Connections among ROIs were considered as causal when their respective granger causality value is greater than the mean value of all granger causalities. According to the results, there is interaction within the three networks, which suggested that the ventral attention network may distract the dorsal attention network and that the emotion network may influence both attention networks.