Weiszfeld’s Method: Old and New Results

Abstract

In 1937, the 16-years-old Hungarian mathematician Endre Weiszfeld, in a seminal paper, devised a method for solving the Fermat–Weber location problem—a problem whose origins can be traced back to the seventeenth century. Weiszfeld’s method stirred up an enormous amount of research in the optimization and location communities, and is also being discussed and used till these days. In this paper, we review both the past and the ongoing research on Weiszfed’s method. The existing results are presented in a self-contained and concise manner—some are derived by new and simplified techniques. We also establish two new results using modern tools of optimization. First, we establish a non-asymptotic sublinear rate of convergence of Weiszfeld’s method, and second, using an exact smoothing technique, we present a modification of the method with a proven better rate of convergence.

Appendices

Appendix A: Notations

Following is a list of notations that are used throughout the paper.

• ${\mathcal {A}}= \left\{ \mathbf{a}_{1} , \mathbf{a}_{2} , \ldots , \mathbf{a}_{m} \right\}$—the set of anchors.

• $\omega _{1} , \omega _{2} , \ldots , \omega _{m}$—given positive weights.

• $\omega = \sum _{i = 1}^{m} \omega _{i}$—sum of weights.

• $f\left( \mathbf{x}\right) = \sum _{i = 1}^{m} \omega _{i}\left\| {\mathbf{x}-\mathbf{a}_{i}} \right\|$—the Fermat–Weber objective function.

• $\mathbf{x}^{*}$—an optimal solution of the Fermat–Weber problem. If the anchors are not collinear, then $\mathbf{x}^{*}$ is the unique optimal solution.

• $X^{*}$—the optimal solution set of the Fermat–Weber problem. When the anchors are not collinear, $X^{*}$ is the singleton $\{ \mathbf{x}^{*} \}.$

• $f^{*}$—the optimal value of the Fermat–Weber problem.

• $T(\mathbf{x}) = \frac{1}{\sum _{i = 1}^{m} \frac{\omega _{i}}{\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| }}\sum _{i = 1}^{m} \frac{\omega _{i}\mathbf{a}_{i}}{\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| }$—the operator defining Weiszfeld’s method.

• $h(\mathbf{x}, \mathbf{y}) := \sum _{i = 1}^{m} \omega _{i}\frac{\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| ^{2}}{\left\| {\mathbf{y}- \mathbf{a}_{i}} \right\| }$—an auxiliary function used to analyze Weiszfeld’s method.

• $L(\mathbf{x}) = \sum _{i = 1}^{m} \frac{\omega _{i}}{\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| }$—serves as a kind of “Lipschitz” constant, and the operator $T$ can be written as taking a gradient step with stepsize $1/L(\mathbf{x})$: $T(\mathbf{x}) = \mathbf{x}- 1/L(\mathbf{x})\nabla f(\mathbf{x})$.

• $\mathbf{R}_{j} = \sum _{i = 1 , i \ne j}^{m} \omega _{i} \left( \mathbf{a}_{j} - \mathbf{a}_{i}\right) /\left\| {\mathbf{a}_{i} - \mathbf{a}_{j}} \right\|$, $j = 1 , 2 , \ldots , m$. An important property related to $\mathbf{R}_{j}$ is that $\mathbf{a}_{j}$ is optimal if and only if $\left\| {\mathbf{R}_{j}} \right\| \le w_{j}$.

• $\mathbf{d}_{j} = -\mathbf{R}_{j}/\left\| {\mathbf{R}_{j}} \right\|$—the steepest descent direction of $f$ at $\mathbf{a}_{j}$.

Appendix B: Proof of Lemma 7.1

Let $\mathbf{x}:= \mathbf{a}_{j} + t_{j}\mathbf{d}_{j}$. Then, from the definition of $\mathbf{x}$, we have that $\mathbf{d}_{j} = (1/t_{j})\left( \mathbf{x}- \mathbf{a}_{j}\right)$ and hence

$$\begin{aligned} -\frac{1}{t_{j}}\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2}&= \frac{1}{t_{j}}\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2} - 2\left\langle {\mathbf{x}- \mathbf{a}_{j} , \frac{\mathbf{x}- \mathbf{a}_{j}}{t_{j}}} \right\rangle \nonumber \\&= \frac{1}{t_{j}}\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2} - 2\left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{d}_{j}} \right\rangle \nonumber \\&= \frac{1}{t_{j}}\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2} - 2\left\langle {\mathbf{x}- \mathbf{a}_{j} , -\frac{\mathbf{R}_{j}}{\left\| {\mathbf{R}_{j}} \right\| }} \right\rangle \nonumber \\&= \frac{1}{t_{j}}\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2} + \frac{2}{\left\| {\mathbf{R}_{j}} \right\| }\left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{R}_j} \right\rangle . \end{aligned}$$

(33)

Now, we will expand the first term of the right-hand side of (33)

$$\begin{aligned} \frac{1}{t_{j}}\left\| {\mathbf{x}- \mathbf{a}_j} \right\| ^{2}&= \frac{L\left( \mathbf{a}_{j}\right) }{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2} \\&= \frac{1}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( \sum _{i = 1 , i \ne j}^{m} \frac{\omega _{i}}{\left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| }\right) \left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2} \\&= \frac{1}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\sum _{i = 1 , i \ne j}^{m} \omega _{i}\frac{\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2}}{\left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| } \\&= \frac{1}{\left\| {\mathbf{R}_{j}} \right\| \!-\! \omega _{j}}\sum _{i = 1 , i \ne j}^{m} \omega _{i}\left( \frac{\left\| {\mathbf{x}\!-\! \mathbf{a}_{i}} \right\| ^{2}}{\left\| {\mathbf{a}_{j} \!-\! \mathbf{a}_{i}} \right\| } \!+\! 2\frac{\left\langle {\mathbf{x}\!-\! \mathbf{a}_{i} , \mathbf{a}_{i} \!-\! \mathbf{a}_{j}} \right\rangle }{\left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| } \!+\! \frac{\left\| {\mathbf{a}_{i} \!-\! \mathbf{a}_{j}} \right\| ^{2}}{\left\| {\mathbf{a}_{j} \!-\! \mathbf{a}_{i}} \right\| }\right) \\&\!=\! \frac{1}{\left\| {\mathbf{R}_{j}} \right\| \!-\! \omega _{j}}\sum _{i = 1 , i \ne j}^{m} \omega _{i}\left( \frac{\left\| {\mathbf{x}\!-\! \mathbf{a}_{i}} \right\| ^{2}}{\left\| {\mathbf{a}_{j} \!-\! \mathbf{a}_{i}} \right\| } \!+\! 2\frac{\left\langle {\mathbf{x}\!-\! \mathbf{a}_{i} , \mathbf{a}_{i} \!-\! \mathbf{a}_{j}} \right\rangle }{\left\| {\mathbf{a}_{j} \!-\! \mathbf{a}_{i}} \right\| } \!+\! \left\| {\mathbf{a}_{j} \!-\! \mathbf{a}_{i}} \right\| \right) . \end{aligned}$$

The middle term can be also written as follows

$$\begin{aligned} 2\frac{\left\langle {\mathbf{x}- \mathbf{a}_{i} , \mathbf{a}_{i} - \mathbf{a}_{j}} \right\rangle }{\left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| }&= 2\frac{\left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{a}_{i} - \mathbf{a}_{j}} \right\rangle }{\left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| } + 2\frac{\left\langle {\mathbf{a}_{j} - \mathbf{a}_{i} , \mathbf{a}_{i} - \mathbf{a}_{j}} \right\rangle }{\left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| } \\&= 2\frac{\left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{a}_{i} - \mathbf{a}_{j}} \right\rangle }{\left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| } - 2\left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| . \end{aligned}$$

Thus, from the definition of $\mathbf{R}_{j}$, we have

$$\begin{aligned} \frac{1}{t_{j}}\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2}&\!=\! \frac{1}{\left\| {\mathbf{R}_{j}} \right\| \!-\! \omega _{j}}\sum _{i \!=\! 1 , i \ne j}^{m} \omega _{i}\left( \frac{\left\| {\mathbf{x}\!-\! \mathbf{a}_{i}} \right\| ^{2}}{\left\| {\mathbf{a}_{j} \!-\! \mathbf{a}_{i}} \right\| } \!+\! 2\frac{\left\langle {\mathbf{x}\!-\! \mathbf{a}_{j} , \mathbf{a}_{i} \!-\! \mathbf{a}_{j}} \right\rangle }{\left\| {\mathbf{a}_{j} \!-\! \mathbf{a}_{i}} \right\| } \!-\! \left\| {\mathbf{a}_{j} \!-\! \mathbf{a}_{i}} \right\| \right) \\&= \frac{1}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( \sum _{i = 1 , i \ne j}^{m} \omega _{i}\frac{\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| ^{2}}{\left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| } \right. \\&\quad +\left. 2\left\langle {\mathbf{x}- \mathbf{a}_{j} , \sum _{i = 1 , i \ne j}^{m} \omega _{i}\frac{\mathbf{a}_{i} - \mathbf{a}_{j}}{\left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| }} \right\rangle - f\left( \mathbf{a}_{j}\right) \right) \\&= \frac{1}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( \sum _{i = 1 , i \ne j}^{m} \omega _{i}\frac{\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| ^{2}}{\left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| } - 2\left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{R}_{j}} \right\rangle - f\left( \mathbf{a}_{j}\right) \right) . \end{aligned}$$

Note that, by the fact that $\frac{a^{2}}{b} \ge 2a - b$ for any $a \in \mathbb {R}$ and $b \in \mathbb {R}_{++}$ we have

$$\begin{aligned} \frac{\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| ^{2}}{\left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| } \ge 2\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| - \left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| . \end{aligned}$$

Hence

$$\begin{aligned} \frac{1}{t_{j}}\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2}&\ge \frac{1}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( \sum _{i = 1 , i \ne j}^{m} \omega _{i}\left( 2\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| - \left\| {\mathbf{a}_{j} - \mathbf{a}_{i}} \right\| \right) \right. \\&\quad -\left. 2\left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{R}_{j}} \right\rangle - f\left( \mathbf{a}_{j}\right) \right) \\&\quad = \frac{1}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( 2\sum _{i = 1 , i \ne j}^{m} \omega _{i}\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| - 2\left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{R}_{j}} \right\rangle - 2f\left( \mathbf{a}_{j}\right) \right) \\&\quad = \frac{2}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( \sum _{i = 1 , i \ne j}^{m} \omega _{i}\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| - \left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{R}_{j}} \right\rangle - f\left( \mathbf{a}_{j}\right) \right) . \end{aligned}$$

Plugging the last inequality in (33) yields

$$\begin{aligned} -\frac{1}{t_{j}}\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2}&= \frac{1}{t_{j}}\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2} + \frac{2}{\left\| {\mathbf{R}_{j}} \right\| }\left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{R}_{j}} \right\rangle \\&\ge \frac{2}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( \sum _{i = 1 , i \ne j}^{m} \omega _{i}\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| - \left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{R}_{j}} \right\rangle - f\left( \mathbf{a}_{j}\right) \right) \\&\quad + \frac{2}{\left\| {\mathbf{R}_{j}} \right\| }\left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{R}_{j}} \right\rangle \\&= \frac{2}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( \sum _{i = 1 , i \ne j}^{m} \omega _{i}\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| - f\left( \mathbf{a}_{j}\right) \right) \\&\quad - \left( \frac{2}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}} - \frac{2}{\left\| {\mathbf{R}_{j}} \right\| }\right) \left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{R}_{j}} \right\rangle \\&= \frac{2}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( \sum _{i = 1 , i \ne j}^{m} \omega _{i}\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| - f\left( \mathbf{a}_{j}\right) \right) \\&- \frac{2\omega _{j}}{\left\| {\mathbf{R}_{j}} \right\| \left( \left\| {\mathbf{R}_{j}} \right\| - \omega _{j}\right) }\left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{R}_{j}} \right\rangle \\&= \frac{2}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( \sum _{i = 1 , i \ne j}^{m} \omega _{i}\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| - f\left( \mathbf{a}_{j}\right) \right) \\&+ \frac{2\omega _{j}}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{d}_{j}} \right\rangle \\&= \frac{2}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( \sum _{i = 1 , i \ne j}^{m} \omega _{i}\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| - f\left( \mathbf{a}_{j}\right) + \omega _{j}\left\langle {\mathbf{x}- \mathbf{a}_{j} , \mathbf{d}_{j}} \right\rangle \right) \\&= \frac{2}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( \sum _{i = 1 , i \ne j}^{m} \omega _{i}\left\| {\mathbf{x}- \mathbf{a}_{i}} \right\| - f\left( \mathbf{a}_{j}\right) + \omega _{j}\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| \right) \\&= \frac{2}{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}\left( f\left( \mathbf{x}\right) - f\left( \mathbf{a}_{j}\right) \right) , \end{aligned}$$

where the second equality from below follows from the fact that $1 = \left\| {\mathbf{d}_{j}} \right\| = \left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| /t_{j}$. Hence,

$$\begin{aligned} f\left( \mathbf{a}_j\right) - f\left( \mathbf{x}\right) \ge \frac{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}{2t_{j}}\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| ^{2} = t_{j}\frac{\left\| {\mathbf{R}_{j}} \right\| - \omega _{j}}{2} = \frac{\left( \left\| {\mathbf{R}_{j}} \right\| - \omega _{j}\right) ^{2}}{2L\left( \mathbf{a}_{j}\right) }, \end{aligned}$$

where the first equality follows from the fact that $\left\| {\mathbf{x}- \mathbf{a}_{j}} \right\| = t_{j}$, and the last equality follows from the definition of $t_{j}$. $\square$