The surveillance of air pollution is becoming a highly concerned issue for city residents and urban administrators. Fixed air quality stations as well as mobile gas sensors have been deployed for air quality monitoring but with sparse observations over the entire temporal-spatial space. Therefore, an inference algorithm is essential for comprehensive fine-grained air pollution sensing. Conventional physically-based models can hardly be applied to all the scenarios, while pure data-driven methods suffer from sampling bias and overfitting problems. This paper presents a hybrid algorithm for air pollution inference by guiding the data learning process with physical model. The quantitative combination of knowledge from observed dataset and a discretized convective-diffusion model is performed within a multi-task learning scheme. Evaluations show that, benefited from physical guidance, our hybrid method obtains higher extrapolation ability and more robustness, achieving the same performance with 1/8 sample amount and obtaining 31.9% less error in noisy synthesized environment. In a real-world deployment in Tianjin, our algorithm outperforms the pure data-driven model with 9.69% less inference error over a 9-day PM_2.5 data collection.