We propose an interleaved method for robotic task and motion planning (TAMP) problems, which involves optimizing both continuous and discrete dynamic behaviors. The coupling between the task planning and motion planning results in a large search space, causing challenge for computing the optimal solution. To address this challenge, we develop a novel bi-level algorithm leveraging the Depth First Search (DFS) algorithm and the Monte Carlo Tree Search (MCTS) algorithm to solve the TAMP. Incorporating task completion cost estimation from the motion planning level, we solve the task planning problem in a computationally efficient manner. We prove that our proposed TAMP algorithm is complete, i.e., it always finds the optimal solution if there exists one. Finally, we present simulation results to demonstrate that the proposed algorithm can find the optimal solution of the TAMP problem with lower computation cost than existing algorithms.