Compressive channel estimation (CCE) schemes have been proposed in millimeter-wave (mmWave) systems. The design of the analog combining matrix plays an important role to provide high channel recovery quality. In this paper, we propose a fixed-loading-based flexible hybrid beamforming (HBF) architecture (FL-FHA), which enables an arbitrary number of phase shifters (PSs) and a flexible connection strategy for the analog network. With the proposed flexible HBF architecture, we develop a variational-Bayesian-inference (VBI)-based algorithm to estimate the mmWave channel exploiting the channel support side information (CSSI) at the base station. Furthermore, we propose a parallel configuration optimization (CO) algorithm to optimize the FL-FHA according to CSSI, aiming at improving the overall CE performance. Simulation results show that the proposed CSSI-assisted joint CO and CCE outperforms the baselines with a small number of phase shifters.