This paper introduces a new hybrid CMOS-Nano circuit for efficient implementation of spiking neurons and spike-timing dependent plasticity (STDP) rule. In our spiking neural architecture, the STDP rule has been implemented by using neuron circuits which generate two-part spikes and send them in both forward and backward directions along their axons and dendrites, simultaneously. The two-part spikes form STDP windows and also they carry temporal information relating to neuronal activities. However, to reduce power consumption, we take the circuitry of two-part spike generation out of the neuron circuit and use the regular shaped pulses, after the training has been performed. Furthermore, the performance of the rule as spike-timing correlation learning and character recognition in a two layer winner-take-all (WTA) network of integrate-and-fire neurons and memristive synapses is demonstrated as a case example.