The surface classification of heterogeneous urban areas can be refined using the integration of spectral and 3D information. However, pixel-classification based fusion requires semi-pixel geo-registration accuracy. In this paper the 3D information is obtained from the hyperspectral data set itself. This study presents an adaptation of optimized MRF based stereo matching for the creation of 3D scenes using hyperspectral data. The obtained 3D information is integrated into a SVM classifier procedure. The results obtained in this study show the potential in the creation of 3D scenes using hyperspectral data and the benefit of combining this data with spectral information for better classification of the urban materials.