The table-based fact verification task is important yet challenging as it requires models to have both linguistic reasoning and symbolic reasoning ability. Existing studies mainly established two kinds of methods, program-enhanced methods and table-based pre-trained models, to excel at different reasoning aspects. In this paper, we propose a Program-enhanced Table Parser (Prog-TAPAS) to effectively combine these two methods. Leveraging a program enhanced module with program tables and synthesize program statements, we combine the strengths of symbolic reasoning into language-based inference models to support our fact verification model. Experiments show that our proposed framework outperforms all baseline systems with a considerable margin and achieves competitive results, on the TABFACT benchmark.