Deep learning has achieved outstanding success in the hyperspectral image (HSI) classification task. Almost all the current deep learning methods are used to conduct classification predictions by leveraging the output features from the deepest layer, which generally ignore the attention to multilayer outputs, so that the capability of hierarchical representation is limited. To remedy such deficiency, in this article, we propose to build a novel deep network form, called tensored deep autoencoder network (TDAE), for HSI classification. For this method, the tensor decomposition constraint item is built and introduced into a deep autoencoders network with a fully connection layer. It not only achieves the integration of multilayer output features but also captures the structure information among outputs. By such way, the network’s ability for hierarchical representation is significantly enhanced. Furthermore, to solve such built model, we further design an alternating update optimization scheme and obtain the desired feature forms. The features are further input into the fully connection layer to generate the label of the given HSI. Extensive experiments have been conducted to validate that the proposed TDAE method achieves more competitive performance compared with several state-of-the-art approaches.