Mobile crowdsensing (MCS) leverages crowd intelligence, i.e., smart devices and their owners, to collect data in an intelligent and cost-efficient manner. One of the fundamental research problems in MCS is task allocation, where a group of smart device owners are recruited as workers to reach and sense specified targets. In task allocation, task publishers submit their data collection tasks with the constraints and budgets, while workers report their estimated costs and possible constraints associated with data collection. The task allocation problem aims at allocating tasks to workers to maximize profit from the gap between the compensation to workers and the available budget while satisfying constraints from both sides. As task allocation problems are often NP-hard, heuristic schemes are widely used to obtain time-efficient results. However, the performance of heuristic methods may vary significantly in different environments, especially for NP-hard problems. To address task allocation problems in MCS, in this paper, we integrate a carefully designed graph attention network (GAT) into deep reinforcement learning (DRL) and develop a GAT-based DRL method (GDRL) to solve an NP-hard task allocation problem. Compared with manually crafted heuristics, our approach features the flexibility and self-adaptability of DRL, enabling the solver to interact with and adjust to new environments and generalize its experience to different situations. Extensive numerical results show that our proposed method can achieve significantly better results than the reference schemes in various experiment settings.