In two-tier heterogeneous cellular networks (HCN) where small cells are deployed overlaid with macro cells, base station (BS) association is one of the fundamental problems, which becomes even more challenging when the small cell BSs are connected to the gateways via imperfect backhaul links, e.g. using wireless technologies. In this paper, we first analyze the mean packet delay in radio access and backhaul networks via modeling the HCN using spatial point process. With the proposed model, we formulate a BS association optimization problem based on cell range expansion so as to minimize the mean network packet delay. Adopting an intuitive solution for the spectrum allocation, the optimal association policy is obtained, which can be directly translated to optimal bias factor configurations. Numerical results are presented to compare the proposed and traditional association policies, which show the significant effect of backhaul on network performance and demonstrate the urgency of a joint system design approach for radio access and backhaul networks.