This paper considers the problem of joint beamformer design for a two-tier wireless network, whereby a set of cache-enabled access points (APs) are connected to the base station (BS) via wireless backhaul links. The APs can prefetch and store the files requested by users, to serve users directly in the access links. Thus low-latency transmissions are enabled as the transmission in the backhaul links is saved. However, due to the limited cache capacity, not all requested files can be stored in the APs, some of the non-cached APs then should be utilized as long as their transmission delays in the access and backhaul links can be well addressed. Two delay optimal beamformer design (DOBD) problems are formulated to minimize the overall delay incurred in the backhaul and access link transmissions via a joint optimization of the beamformer at the BS and APs. We consider the DOBD problem under non-fragment and fragment caching policies, both involving nonconvex link rate constraints. The semi-definite relaxation (SDR) and sequential convex approximation (SCA) schemes are adopted to approximate the nonconvex problems into convex ones, which are then iteratively solved. Numerical results demonstrate the convergence and improved transmission delay performance of the proposed scheme under various network settings.