This work investigated two statistical properties, namely stationarity, and entropy, of a real-world publicly available battery dataset considering features like Current, Charge Capacity, Discharge Capacity, Temperature, and Voltage, with the objective of providing insights for the development of models that would be best suited for early End-of-life (EOL) prediction. From the characteristics of the data, we hypothesized that the lack of stationarity and higher entropy would deteriorate the performance of LSTM models while having less of an impact on CNNs. To fortify this hypothesis we developed 4 types of models and investigated their performances. The results for this case study indicate that CNN-based models are more robust to these properties of the data, while the LSTM-based ones are more sensible and therefore have worse performance. We discuss this sensibility by analyzing the correlation of these statistics with model performance. The paper presents a detailed process for preprocessing, model generation, and comparison. Our best LSTM-based model had 18.3% error while the best CNN-based model presented 3.5% error when considering unseen test data, using only the first 100 cycles of the batteries.