Unmanned Aerial Vehicle (UAV)-assisted mobile edge computing (MEC) has emerged as a promising solution to support the computation-intensive tasks in the Internet of Things (IoT) networks. As for the operation of UAV-assisted MEC, jointly design of the UAV trajectory control and computation offloading strategies becomes the key for achieving high offloading efficiency, which is extremely challenging due to the uncertain and dynamic demands in the network. In this paper, aiming at maximizing the offloading task amount, we propose an Multi-Agent joint TrAjectory and Computation Offloading (MA-TACO) scheme, where all related factors including task type variety, quality of service (QoS) guarantee, and service fairness are taken into account. To facilitate each UAV to obtain the best joint strategy under dynamic network environment, considering the complex decisions with both continuous and discrete variables, we develop an Optimization-oriented Multi-Agent Deep Reinforcement Learning approach (OMADRL), where each UAV could autonomously learn the trajectory decision to adapt to the dynamic demands, and the offloading decision would be made by solving a mixed-integer programming problem based on the observations, which would be utilized to guide the trajectory learning. Comparing with solely relying on learning, such an optimization-oriented way could reduce the action space dimension and make each UAV achieve the best strategy faster. The simulation results indicate the effectiveness of the proposed scheme.