With the development of deep learning, data-driven approaches have shown great success in the semiconductor manufacturing. For example, wafer bin maps (WBM) recognition is a critical application to identify failure modes and finding root-cause to reduce yield loss. However, the WBM studies provide static classification results without tracing the temporal patterns. This study develops a spatio-temporal strip map prediction system for the flip-chip bonding process in the assembly house. The proposed strip bin map (SBM) prognostic system including prediction model and recognition model can provide pre-alarm of defect and predict the defect mode for the final process. In practice, not all processes are followed by a functional test (FT) and thus this proposed system can simulate the defect generation based on the Bayes' theorem for tracking the changes of the spatio-temporal patterns. An empirical study of the semiconductor assembly manufacturer in Taiwan was conducted to validate the proposed framework, and the results show that the proposed SBM system successfully predict the SBM and defect mode for the final test and provide an effective pre-alarm.