In this paper, we present an adaptive multi-agent system with duplex control laws for non-smooth resource allocation problem, where the decisions are subjected to local constraints and network resource constraints. The multi-agent system based on the distance penalty function method is developed in three sets of coupled differential inclusions or equations, where the last set of differential equations are designated to learn an adaptive penalty vector. In the multi-agent system, proportional and integral controls can be performed from two different layers of the multiplex control network with an independent communication topology at each layer. The existence of equilibrium points and convergence of the multi-agent system are proven for achieving optimal resource allocation starting from any initial resource allocation. Finally, the simulation results of two illustrative examples are discussed to substantiate the theoretical results.