In recent years, face recognition technologies develop rapidly especially in those systems based on deep learning. However, the models of the general face recognition system are fixed in use after trained, which is difficult to adapt to the new data collected during use. In this paper, we design a face recognition system based on OpenFace. Different from other systems, we propose an intelligent model training method S-DDL(self detection, decision and learning) using incremental SVM algorithm which makes our system able to update the classification model in real time during execution. With this method, the accuracy of our system will increase on specific human groups and the time consumption of training new models can be limited with the incremental SVM algorithm. The results show that our face recognition system has a good real time performance and accuracy. The S-DDL method can obviously improve the accuracy within little time during the execution of system and the incremental SVM algorithm has a better performance than the traditional one.