We consider multi-group multicast beamforming in large-scale systems to minimize the transmit power subject to the signal-to-interference-plus-noise ratio (SINR) requirements. Based on the optimal multicast beamforming structure, we propose a fast first-order algorithm to obtain the beamforming solution. The algorithm utilizes the successive convex approximation (SCA) method and solves each SCA subproblem by dual reformulation along with the extra-gradient method for fast closed-form updates. Initialization methods are also explored, including an extragradient-based fast initialization approach that is proposed to generate initial feasible points for SCA. Simulations show that the proposed algorithm provides a near-optimal performance with substantially lower computational complexity for large-scale systems than the existing algorithm.