In general, codes for defective memories require complex Gaussian elimination (GE) operations in their encoding. This paper proposes a low complexity sparse coding scheme such that the encoding can be accomplished efficiently by a graph-based elimination algorithm (GBE). The proposed algorithm takes advantage of the sparse structure of sparse codes. The corresponding graph of the sparse code is divided into layers according to the degree of variable nodes. Then, a message passing process is performed from higher to lower layers of the graph. The GBE algorithm has only linear complexity and performs closely to GE.