Due to the COVID-19 pandemic, wearing a mask (masking) has become common in daily life, which also affects the face recognition system. The recognition system has to consider the face as input first to memorize the face of a person. Normally, the memorization process usually and only requires a clear facial face. However, when the face is masked, the system might be confused. In this work, we design a framework for a face recognition system to increase the accuracy of recognition of masked face, which can also classify the face into masked or not masked.