Optimizing beamforming is crucial in mitigating pronounced propagation loss and ensuring reliable communication at millimeter-wave (mmWave) frequencies. Traditional beam optimization methods rely on either precise channel estimation or extensive beam training in the mm Wave band, both of which entail substantial pilot overhead. To alleviate this overhead, we leverage the spatial congruence between sub-6 GHz (sub-6G) and mm Wave channels and propose a sub-6G information and few pilots aided beam prediction network (SPBPNet) through deep learning. Specifically, the proposed SPBPNet comprises two cascaded modules: i) the angular information extraction module, which extracts angular features from the available sub-6G channel estimate and maps them to a minimal set of narrow beam directions to be measured in the mm Wave band; and ii) the beam prediction module, which takes limited beam training along the selected directions and then fuses measurements in the mm Wave band with the sub-6G channel information to generate mmWave beam predictions. Numerical results demonstrate that SPBPNet efficiently maps sub-6G channel estimates to mmWave beams and achieves a superior balance between performance and pilot overhead compared to state-of-the-art benchmarks. Moreover, SPBPNet exhibits resilience to varying sub-6G channel estimates at different signal-to-noise ratio levels.