Extending CLIP for Text-to-font Retrieval

This study addresses the challenge of font retrieval in design by proposing a novel approach utilizing contrastive learning to establish a shared embedding space for texts and fonts. In contrast to previous methods limited to word-level queries, our method enables text-font retrieval at the sentence level. We collected text-font pair data from web pages and design templates on the Internet, finetuned the CLIP model on these pairs, and obtained text and font encoders for our application. The top-k fonts were then retrieved using the cosine distance between input text and font embeddings. Our approach offers three key advantages: (1) retrieving fonts with sentence-level text as input, which is intuitively consistent with design behaviors; (2) leveraging text-font pair data available on the Internet without manual annotation; and (3) scalability, the trained font encoder can encode new font candidates without retraining the model. We introduced an evaluation metric for font retrieval results. The results indicate that the retrieved fonts in the top 3 score better than those from baseline methods, and the top 1 retrieved font is competitive with the fonts selected by experienced graphic designers.