In this paper design and implementation of a modular mixed-signal feed-forward neural network is presented. The network is implemented based on the Continuous Valued Number System (CVNS) arithmetic with neurons distributed in the network. Synapse weights are implemented on the chip using capacitive analog memories. Weight values are stored as the CVNS values and are refreshed and updated using the overlap between the CVNS digits. Current-mode logic is used for implementation in order to simplify the circuit design, and especially addition, which resulted in reduced power and area consumption. The distributed nature of the neurons allows for expansion of the network into larger networks. Individual modular layers are fabricated in TSMC CMOS 180nm, and are used to form different network sizes. The module is used to configure two proof of concept examples, a 2 - 2 - 1 and a 3 - 2 - 1 network to solve the XOR problem. Results of test and verification presented in this paper show the network flexibility of the proposed design to form various network configurations.