This paper investigates the weighted least squares (WLS) and minimax design problems for variable fractional delay (VFD) FIR filters. For the WLS design, the general case of incorporating arbitrary nonnegative weighting functions is considered. The optimal solution is characterized by two matrix equations. An efficient algorithm using conjugate gradient (CG) techniques is proposed to solve the WLS solution. The proposed algorithm is guaranteed to converge to the optimal solution in a finite number of iterations. Moreover, an iterative reweighted least squares (IRLS) algorithm that uses the proposed CG algorithm as its iteration core is developed for the minimax design problem. In both of the algorithms, the filter coefficients are arranged as matrices, achieving a great saving in computations and memory space. The associated computational complexity is analyzed. Some design examples are provided and comparisons with existing methods show that the proposed ones are either computationally more efficient or can obtain the better filter performance, or both.