In this paper, the theory of the unified discrete Fourier-Hartley transform (UDFHT) is developed. The UDFHT includes the discrete Fourier and Hartley transforms of types I-IV as special cases, and with little modification, it also includes the discrete cosine and sine transforms of types II-IV. A unified efficient structure using fast Fourier transform is proposed. The UDFHT is then generalized (GDFHT) to the case of non-constant coefficients. Sufficient conditions for orthogonality of the transform are presented. The GDFHT can be used to compute the modified discrete cosine transform for both non-window and window modes as illustrated in the paper.