In this paper, a novel online approximate dynamic programming (ADP) technique for completely unknown continuous-time linear systems is proposed to solve the infinite horizon linear quadratic (LQ) optimal control problems. For relaxing the assumption of the known input coupling matrix, the conventional LQ optimal control problem is converted into the proposed cheap control problem. Then, the ADP agent iteratively solves this cheap optimal control problem in online fashion to obtain the near-optimal solution of the conventional LQ optimal control problem. In addition, we mathematically prove the approximation property of the cheap optimal control problem with respect to the conventional LQ optimal control problem. The numerical simulation for ideal DC motor shows the applicability of the proposed ADP algorithm.