Semi-grant-free non-orthogonal multiple access (SGF-NOMA) is a potential paradigm to support massive connec-tivity for the short packets Internet of things (IoT) applications while satisfying the undistracted transmission requirements of primary IoT users. However, resource allocation in SGF-NOMA is more challenging due to the sporadic traffic of grant-free (GF) users and the need to satisfy the quality of service (QoS) requirements of grant-based (GB) users. The GF users access and choose resources at random, resulting in frequent power collisions and decoding failures at the base station (BS). This paper develops a general learning framework that enables GF users to learn from historical information to avoid power collisions. We utilize a hybrid multi-agent deep reinforcement learning (hMA-DRL) framework to maximize the connectivity and enhance the number of successful decoded users at the BS. The numerical results show that the proposed scheme achieves a solution near to the optimal one and increases the successful decoded users by 42.38% as compared to the benchmark scheme. The considered algorithm performs well with an increasing number of users as compared to the competitive and cooperative MA-DRL algorithms.