In this paper, a centralized two-block separable convex optimization with equality constraint is considered. The first fully parallel primal-dual discrete-time algorithm called Parallel Alternating Direction Primal-Dual (PADPD) is proposed. In the algorithm, the primal variables are updated in an alternating fashion like Alternating Direction Method of Multipliers (ADMM). The algorithm can handle non-smoothness of objective functions with strong convergence. Unlike existing discrete-time algorithms such as Method of Multipliers (MM), ADMM, Bi-Alternating Direction Method of Multipliers (BiADMM), and Primal-Dual Fixed Point (PDFP) algorithms, all primal and dual variables in the proposed algorithm are updated independent of each other. The algorithm can be directly extended to any finite multi-block optimization without further assumptions while preserving its convergence. It is shown that the rate of convergence of the algorithm for Quadratic/Linear cost functions is exponential or linear under suitable assumptions. Finally, a numerical example is given to show that PADPD not only can compute more iterations (since it is fully parallel) for the same time-step but also can have faster convergence rate than that of ADMM.