Eliminating the bias term of the Support Vector Machine (SVM) classifier permits substancial simplification to training algorithms. Using this elimination, the optimization invloved in training can be decomposed to update as low as one coordinate at a time. This paper explores two directions of improvements which stem from this simplification. The first one is about the options available for choosing the coordinate to optimize during each optimization iteration. The second one is about the parallelization schemes which the simplified optimization facilitates.