Deep learning for image denoising has recently attracted considerable attentions due to its excellent performance. Since most of current deep learning-based denoising models require a large number of clean images for training, it is difficult to extend them to the denoising problems when the reference clean images are hard to acquire (e.g., optical coherence tomography (OCT) images). In this article, we propose a novel unsupervised deep learning model called as nonlocal-generative adversarial network (nonlocal-GAN) for OCT image denoising, where the deep model can be trained without reference clean images. Specifically, considering that the background areas of OCT images mainly contain pure real noise samples, we creatively train a discriminator to distinguish background real noise samples from the fake noise samples generated by the denoiser, that is the generator, and then the discriminator will guide the generator for denoising. To further enhance denoising performance, we introduce a nonlocal means layer into the generator of the nonlocal-GAN model. Furthermore, since nearby several OCT B-scans have strong correlations, we also propose a nonlocal-GAN-M model to utilize the high correlations within nearby B-scans. Extensive experimental results on clinical retinal OCT images demonstrate the effectiveness and efficiency of the proposed method.