Analysis and classification of digital modulation signals is an important feature of spectrum management and intelligence gathering. Various methods focus on classification according to modulation type with low accuracy at low signal-to-noise ratio (SNR). The presence of fading makes it more difficult due to the instantaneous variation in the SNR. Thus, the adaptive smooth-windowed Wigner Ville bispectrum (SWWVB) is proposed that combines the advantage of time-frequency analysis and higher order statistics (HOS). The estimated signal parameters from digital modulation signal such as ASK, FSK and M-ary FSK are then used as inputs to a rule based classifier. Compared to the bilinear time-frequency distribution, the system based on the adaptive SWWVB are able to maintain consistent classification accuracy in similar fading condition.