Wireless communications can be improved by employing free space optical (FSO) channels. Since optical signals can only be transmitted via line-of-sight paths, UAVs are employed to forward data from a base station (BS) to remote users for urgent tasks using multi-hop mixed FSO/RF links. The UAVs employ the decode and forward protocol to relay data. The last UAV decodes and forwards the data to multiple users through RF links using non-orthogonal multiple access (NOMA). To improve fairness, a modified deep reinforcement learning (DRL) algorithm is used to optimize the transmit power allocation in real-time to minimize the maximum user decoding outage probability. Numerical results are presented to illustrate the system design tradeoffs. In addition, the validity of the proposed approach are verified by comparing it with exhaustive search algorithm.