This paper presents a sparse factorization for the delay Vandermonde matrix (DVM) along with fast, exact, radix-2, and recursive algorithms to compute the DVM-vector product for wideband multi-beam antenna arrays. The proposed algorithms enable low-complexity wideband beamformers in emerging millimeter-wave wireless communication networks by reducing the complexity of $N$-beam wideband beamforming from $\mathcal {O}(N^{2})$ to $\mathcal {O}(N \mathrm{\: log\:} N)$, where $N=2^{r}(r \geq 1)$. As a result, the algorithms are faster than the brute-force computation of the DVM-vector product and more efficient than the direct realization of true-time-delay-based multi-beam beamformers. The proposed low-complexity algorithms' signal flow graph (SFG) is also presented to highlight their suitability for hardware implementations. The 2-D frequency responses of DVM-based beamformers are explained through an array signal processing example. Simulation results suggest that integrated circuit (IC) implementations of the SFG significantly reduce chip area and power consumption.