An asymmetrical fully-digital transceiver architecture has recently demonstrated enhanced energy efficiency for millimeter-wave (mmWave) massive MIMO systems. It achieves high beamforming gain by utilizing all transmit antennas and RF chains in downlink transmission, while reducing the number of receive RF chains and antennas for uplink reception to lower hardware costs. However, this asymmetry poses challenges in downlink channel estimation due to the discrepancy in uplink and downlink channel dimensions. To address this issue, we propose a Sparse Bayesian Learning-based Joint Uplink and Downlink Channel Estimation (SBL-JUDCE) approach. SBL-JUDCE directly estimates the downlink channel by padding the uplink training signal vector to account for deactivated receive antennas. Leveraging the sparsity of the mmWave channel, we formulate the downlink channel estimation as a compressive sensing problem using a 0-1 masking matrix. We develop an SBL framework to solve this problem effectively. Simulation results demonstrate the superiority of SBL-JUDCE over a two-step approach, where the uplink channel is estimated first and then used to infer the downlink channel, in terms of channel estimation error and spectral efficiency for downlink transmission with the estimated channel.