Recent development in network technology can realize the control of a remote plant by a digital controller. However, there is a delay caused by data transmission of control inputs and outputs. The delay degrades the control performance without taking it into consideration. In general, it is a difficult problem to identify the delay beforehand. We also assume that the plant's parameters have uncertainty. To solve the problem, we use reinforcement learning to achieve optimal digital control. First, we consider state feedback control. Next, we consider the case where the plant's outputs are observed, and apply reinforcement learning to output feedback control. Finally, we demonstrate by simulation that the proposed control method can search for the optimal gain and that it can adapt to the change of the delay.