Due to the demands for high data rate and high energy efficiency, Millimeter-Wave (mmWave) communication systems have drawn a lot of attention recently. The mmWave transceivers are usually highly integrated but have severe RF impairments. Thus, to design pre-distortion schemes, it is difficult to pre-distort each RF component separately. In this paper, we consider joint IQ (In-phase and Quadrature) imbalance and PA (Power Amplifier) nonlinearity pre-distortion designs for mmWave transmitters. In the designs, we treat the transmitter as a whole and only process the baseband input and PA output. We propose two methods, one is the two-step design, where the IQ imbalance and PA nonlinearity are pre-distorted in two-steps with a unified training structure. The other is the one-step design, where we first derive the model that can jointly characterize IQ imbalance and PA nonlinearity, then one block is used to pre-distort the both. We also show that decreasing the number of coefficients properly can decrease the computational complexity without significant loss in performance.