This letter focuses on the constraint-driven optimal control for multiple Autonomous Underwater Vehicles (AUVs). The different tasks are formulated as multiple constraints on optimization. First, the task goals are formulated by the time-varying control barrier functions. Then, a novel cost-to-constraint transformation method is proposed for time-varying tasks to achieve the high-order constraint-driven task execution under AUV dynamics. The priority of each task is adjusted by the maximum allowed limit of the slack variable, thus relaxing the task constraints and avoiding potential constraint conflicts. Finally, the collision-free optimal energy-aware task execution is achieved by a minimum input norm objective that is incorporated into computationally efficient quadratic programming, which is scalable to different task behaviors. Comparative numerical simulations considering time-varying trajectory tracking and cooperative coverage tasks validate the effectiveness of the proposed method.