This paper presents discrete stochastic approximation (DSA) algorithms for time synchronization in OFDM systems. It is shown that the discrete stochastic approximation algorithms can be effectively used to achieve a significant reduction in computational complexity compared to brute force maximum-likelihood (ML) methods for OFDM synchronization. The most important property of the proposed algorithms is their recursive self-learning capability - most of the computational effort is spent at the global or a local optimizer of the objective function. An adaptive version of the discrete stochastic approximation algorithm is also presented for tracking time varying time delays and frequency offsets in time selective fading channels. Detailed numerical examples illustrate the performance gains of these DSA based synchronization algorithms.