Audio amplifiers, including the contemporary class D amplifiers (CDAs), are typically qualified by their several nonlinearities, including total harmonic distortion and intermodulation distortion (IMD). In the case of filterless CDAs, IMD remains largely unexplored, and the mechanism thereof is largely unknown. This paper presents an analytical modeling of IMD for the prevalent first- and second-order filterless CDAs and shows that the dominant mechanisms of the former is phase error while of the latter is duty-cycle error. By means of multidimensional Fourier series analysis, analytical expressions for the IMD components and thereafter, the IMD expression for these CDAs, are derived. The derived expressions depict that the IMDs of these CDAs are significant, and the IMD of the first-order filterless CDA, in spite of its lower noise-suppression attribute, is somewhat unexpectedly superior to the second-order filterless CDA. Furthermore, the derived expressions delineate the parameters that affect IMD and are insightful to designers to optimize/vary pertinent parameters to reduce IMD. The derived IMD expressions are verified against HSPICE simulations and on the basis of measurements on a prototype CDA IC and other CDAs realized discretely.