With the increasing complexity of modern power system, conventional dynamic load modeling with ZIP and induction motors (ZIP + IM) is no longer adequate to address the current load characteristic transitions. In recent years, the Western Electricity Coordinating Council Composite Load Model (WECC CLM) has shown to effectively capture the dynamic load responses over traditional load models in various stability studies and contingency analyses. However, a detailed WECC CLM model typically has a high degree of complexity, with over one hundred parameters, and no systematic approach to identifying and calibrating these parameters. Enabled by the wide deployment of PMUs and advanced deep learning algorithms, proposed here is a double deep Q-learning network (DDQN)-based, two-stage load modeling framework for the WECC CLM. This two-stage method decomposes the complicated WECC CLM for more efficient identification and does not require explicit model details. In the stage one, the DDQN agent determines a proper load composition that can approximate the true transient dynamics. In the second stage, the remaining parameters of the WECC CLM are selected with Monte-Carlo simulations. The proposed method shows that the identified load model is capable of accurately simulating the given dynamics of the reference load model. In addition, the identified load model has strong robustness to represent the reference load model under a wide range of contingencies. The proposed framework is verified using an IEEE 39-bus test system on commercial simulation platforms.