Recently, few-shot learning (FSL) has been introduced for hyperspectral image (HSI) classification with few labeled samples. However, existing FSL-based HSI classification methods mainly focus on the meta-knowledge transfer between HSIs. Compared with HSIs, natural images have sufficient annotated data. To utilize natural images (base class data) to achieve accurate classification of HSIs (novel class data), we propose a novel few-shot classification framework with SSL (FSCF-SSL) for HSIs in this article. The orientation of objects in natural images is relatively unitary, whereas the objects of image patches for each pixel in HSIs have diverse orientations in the spatial domain. To make better use of base classes, we design an SSL with geometric transformations (SSLGTs), which sets rotation labels as supervision to extract low-level features that can better represent diverse orientations, and then conduct SSLGT and FSL on base classes to learn transferable spatial meta-knowledge. Next, a spectral-spatial feature extraction network is carefully designed to better utilize the spatial and spectral information of HSIs, where the weights of the first seven layers of the spatial part are initialized by the weights of the corresponding layers trained on base classes. Finally, to fully explore the few annotated data from novel classes, we design an SSL with contrastive learning (SSLCL) that can mine the category-invariant features contained in the novel class data itself, and then perform SSLCL and FSL on novel classes to learn more discriminative individual knowledge. Experimental results on four HSI datasets show that FSCF-SSL offers a significant improvement over state-of-the-art methods. The code is available at https://github.com/Li-ZK/FSCF-SSL-2023.