This paper investigates the performance of the downlink hybrid access network with millimeter wave (mmWave) capability by evaluating the joint effect of user association, radio resource allocation as well as interference mitigation. Our hybrid access network consists of multiple macro-cells operating at sub-6 GHz ultra high frequency (UHF) frequency bands, and multiple small-cells operating at mmWave frequency bands laid on cell-edge of each macro-cell. On the basis of the user association, we propose the distributed maximizing worst (DMW) subcarrier allocation algorithm for the macro-cell transmissions based on OFDMA scheme. Finally, to combat interference among macro-cells, the proposed heuristic interference mitigation (HIM) algorithm is carried out to enhance the network throughput performance. Our simulation results show that, with the aid of proposed DMW and HIM algorithms, the mmWave hybrid access network significantly outperforms the traditional UHF only access network.