The solar power variability is due to the variability of solar irradiance. Several factors are involved in the situation, such as cloud thickness and air pollution. In this paper, we attempt to find a novel way to predict the amount of solar irradiance. A image-based forecasting method is developed, and Long Short-Term Memory (LSTM) neural network is applied for data training. Daily solar irradiance and sky images are record by the record system, and uploaded to the MySQL database for storage. Feature values obtained by analyzing sky images are used as the input of neural network with solar irradiance. After some performance evaluation indicators were demonstrated, we found that the proposed method has good predictive performance with 5 to 60 minutes in present.