In this paper, we propose a novel gate delay time mismatch tolerant time-mode signal accumulator whose input and output are represented by a time difference of two digital signal transitions. Within the proposed accumulator, the accumulated value is stored as the time difference between the two pulses running around the same ring of a delay line, so that there is no mismatch between the periods of the two pulses, thus the output drift of the accumulator is suppressed in principle without calibrating mismatch of two rings, which is used to store the accumulated value in the conventional one. A prototype of the proposed accumulator was fabricated in 180nm CMOS. The accumulating operation is confirmed by both time and frequency domain experiments. The standard deviation of the error of the accumulating operation is 9.8ps, and compared with the previous work, the peak error over full-scale is reduced by 46% without calibrating the output drift.