Millimeter-wave system is promising to enhance the data rate of wireless communications, due to abundant frequency spectrum available, while it suffers from large path loss. The codebook-based beamforming is widely used to compensate the severe channel attenuation. However, the large size of codebook makes it difficult to directly search the optimal precoder and combiner in practice. In this paper, we consider a general setup which has no a priori information about the channel, and propose an iterative scheme to optimize the precoder and combiner jointly. The proposed scheme uses the warm start strategy to perform initialization. By treating each observation of spectrum efficiency as a random variable, the proposed scheme utilizes the alternate discrete stochastic optimization which uses probabilistic forecasting, to find the optimum with reduced computational complexity. Moreover, we prove the convergence of the discrete stochastic optimization in our scenario. Simulation results show that the proposed algorithm outperforms the conventional algorithm in terms of higher spectrum efficiency and lower complexity.