Ubiquitous deployment of networked cameras has boosted the prevalence of video analytics. Leveraging machines to automatically analyze captured videos has become the driving force behind a variety of contemporary applications. However, it is known to be resource-hungry with highly dynamic demands, which mismatches the existing monolithic cloud service deployment with coarse-grained resource allocation. Recent advances in serverless computing, which offers ultra-fast and fine-grained autoscaling, would become a game-changer. This article closely examines the potential and challenges of serverless computing in building modern video analytics applications. We accordingly present an integrated framework with geo-distributed resources, and identify the critical design issues toward its implementation. We further discuss a series of promising research directions in this field.