In recent years, a large variety of short-term load forecasting (STLF) methodologies have been proposed, but a common drawback of them is that a large amount of historical data is required to train the model. However, in reality there is often limited historical data, for example a new house or substation is built. To tackle this issue, a combining forecasting method of deep Residual Neural Network (ResNet)-based transfer learning for residential loads is proposed in this paper. To improve the accuracy and reliability of transfer learning, a novel deep ResNet with dual skip connections (ResNet-DSC) is proposed as the base-model. Then with sparse training data, a Bayesian Probability Weighted Averaging (BPWA) approach is proposed, to address the model combination parameters estimation problem. In addition, Probability Density Function (PDF) forecasting is also delivered by the aforementioned method. For demonstration, series of experiments, with actual residential load data, was performed. With the comparison with other transfer learning and non-transfer learning approaches, the effectiveness and improvement of the proposed method has been validated.