In this paper, we consider the problem of full-duplex covert millimeter wave (mmWave) communications, where a mmWave transmitter (Alice) sends information signals to its intended receiver (Bob) covertly in the presence of a watchful warden (Willie). For covering the presence of Alice, Bob operates in the full-duplex mode and generates jamming signals with a time-varying power. We investigate the covert rate optimization for both the single data stream case and the multiple data streams case under the constraints of the detection error probability at Willie. Specifically, for the single data stream case, we analytically characterize the minimum detection error probability at Willie and establish a framework for optimizing the analog beamforming, transmit power, and analog jamming jointly. As for the case of multiple data streams, we derive a tractable lower bound of the minimum detection error probability at Willie and formulate a joint optimization of the hybrid precoder and analog jamming design problem for the maximization of the achievable covert rate. Although the joint design problem is nonconvex, we adopt the penalty decomposition technique to handle the effect of the coupling between the analog precoder and digital precoder paving the way for the development of an iterative algorithm to locate its Karush-Kuhn-Tucker (KKT) solution. Finally, we show that our proposed joint design algorithm can be adapted to handle the multi-antenna Willie scenario and simulation results show that our proposed joint design algorithms can achieve significantly better performance as compared with some benchmark schemes.