Recent improvements in communication and artificial intelligence (AI) are accelerating the realization of roadside unit (RSU) assisted automated vehicle networks. It is expected to greatly boost the level of automated driving by introducing distributed RSUs to Everything (R2X) networks, in which RSUs not only behave as data forwarders but also actively and collectively perceive and analyze the environment to assist automated vehicles in making driving decisions. However, there are still many challenges for offloading computation-intensive and delay-sensitive tasks from RSUs to other network nodes in a distributed way, including tradeoffs among multiple objectives, substantial computation complexity for the problem, stringent autonomy requirements, and so forth. In this article, we present the characteristics and enabling technologies of a distributed Deep Reinforcement Learning (DRL) framework for R2X in the AI landscape. As case studies, we evaluate the performance of task offloading in the distributed learning framework from three perspectives of QoS, caching, and battery lifetime of RSUs, which demonstrates the effectiveness of our framework. Finally, we discuss the open challenges for future researches.