Drug design and optimization are complex tasks that require strategically efficient exploration of the extremely vast search space. Various fragmentation strategies have been presented in the literature to reduce the complexity of the molecular search space. From the optimization perspective, drug design can be viewed as a multi-objective optimization process. Deep reinforcement learning (DRL) frameworks have displayed promising performances in this field. However, lengthy training periods and inefficient use of sample data limit the scalability of the current frameworks. In this paper, we (1) review the fundamental concepts of deep or multi-objective RL methods and their applications in molecular design, (2) investigate the performance of a recent multi-objective DRL-based and fragment-based drug design framework, named DeepFMPO, in a real application by integrating protein-ligand docking affinity score, and (3) compare this method with a single-objective variant. Through experiments, we find that the DeepFMPO framework (with docking score) can achieve limited success, however, it is incredibly unstable. Our findings encourage further exploration and improvement. Possible sources of the framework's instability and suggestions of further modifications to stabilize the framework are examined.