This technical note proposes to study the activity invariant sets and exponentially stable attractors of linear threshold discrete-time recurrent neural networks. The concept of activity invariant sets deeply describes the property of an invariant set by that the activity of some neurons keeps invariant all the time. Conditions are obtained for locating activity invariant sets. Under some conditions, it shows that an activity invariant set can have one equilibrium point which attracts exponentially all trajectories starting in the set. Since the attractors are located in activity invariant sets, each attractor has binary pattern and also carries analog information. Such results can provide new perspective to apply attractor networks for applications such as group winner-take-all, associative memory, etc.