This research proposes an innovative optimal trajectory tracking scheme for uncertain linear discrete-time (DT) systems, leveraging trajectory-dependent Q-learning. Unlike conventional optimal tracking control approaches, the proposed method eliminates the need for a desired trajectory generator function, typically modeled as the dynamics of an autonomous system. Instead, we tackle the tracking problem by learning a $Q$ -function that depends on a horizon of reference trajectory points in the future, which enables the computation of optimal feedback gains and time-varying feedforward control inputs without prior knowledge of system parameters or access to the complete reference trajectory. To enhance the effectiveness of the controller in multitask scenarios, we use the Efficient Lifelong Learning Algorithm (ELLA) to generate a shared knowledge base and use online adaptive control methods to directly learn parameters for each task, enabling information transfer between tasks. Simulation results using a power system demonstrate the efficacy of our approach.