Physiological signals obtained from wearable sensors has the potential to capture pedestrians' distress caused by physical disorders in the built environment, such as litter, abandoned houses, poorly maintained sidewalks, and graffiti. A limited amount of prior work has demonstrated the feasibility of deep learning models using physiological response data for tracking pedestrians' distress. Yet, the explain ability of such models estimating the distress in a time-continuous manner is not widely investigated. In this context, the objective of this paper is to examine an explainable machine learning algorithm for reliably identifying segments of physiological response data that contribute to the detection of physical disorders in the built environment. The data used in this study include a set of time series of electrodermal activity, electrocardiogram, heart rate, and skin temperature, collected from 67 participants walking on a pre-defined route in College Station, Texas, USA. Time continuous self-reports were recorded retrospectively for each participant over the span of the route. A long short-term memory (LSTM) neural network predicts reported stimuli of distress from participants. Next, the local interpretable model-agnostic explanations (LIME) algorithm was applied to the input time series of the LSTM model yielding an explanation of the corresponding distress label. Results from the model of the LIME algorithm are qualitatively evaluated, which indicate the presence of meaningful segments in the time series that have the greatest influence on the final distress prediction. Findings obtained by this research have the potential to advance our understanding about sources of pedestrians' distress, contributing to promoting urban walkability.