Deep learning (DL) algorithms have been deployed on an increasing number of end devices to enable various smart applications. Energy harvesting becomes the most promising energy supply, considering the enormous energy consumption of those algorithms. Frequent over-the-air (OTA) code programming is required to update the new model incrementally learned on the nearby edge server, adapt to the environmental changes over time, and learn new knowledge. However, it is a grand challenge to update the DL code on devices due to the constrained resources and low harvested energy. This paper proposes a novel intermittent OTA approach to update incremental DL algorithms on energy harvesting IoT devices to address those challenges. Specifically, we propose a delta encoding strategy to reduce the update code size, a data transmission optimization strategy to reduce the communication energy consumption, and runtime support to enable efficient intermittent updates. The experimental results demonstrate that the proposed approach can achieve reliable and efficient intermittent updates.