The objective of low-delay codes is to protect communication streams from erasure bursts by minimizing the time between the packet erasure and its reconstruction. Previous work has concentrated on the constant-delay scenario, where all erased packets need to exhibit the same decoding delay. We consider the case of heterogeneous delay, where the objective is to minimize the average delay across the erased packets in a burst. We derive delay lower bounds for the average case, and show that they match the constant-delay bounds only at a single rate point 0.5. We then construct codes with optimal average delays for the entire range of code rates. The construction for rates under 0.5 achieves optimality for every erasure instance, while the construction for rates above 0.5 is optimal for an infinite number, but not all, of the erasure instances.