Today, in semiconductor manufacturing, wafer failures are frequent problems with the production lines. To increase the production yield, images are the most important pieces of data used to detect defect-free wafers. However, there are many tools that can be installed specifically to monitor production lines, inspect mapped defects and detect the main causes of die failures by using wafers images during semiconductor manufacturing process. The underlying objective is to overcome the need for a physical check on the wafer which are in most cases too long. Thus, the need to have a design that will measure and detect these visual faults in an automated fashion is a big challenge for the industry. Recently, deep learning approaches have proven to be a suitable way to overcome this issue. However, they rely on the availability of sufficiently representative datasets which is not our case: data on anomalies is scarce. The goal of this paper is to evaluate state of the art deep learning methodology such as GAN and VAE to overcome this challenge. Implementation results show that the GAN architecture achieves a convincing image generation in a limited sample setting, while the VAE architecture fails to converge at training time.