Learning-augmented algorithms for online subset sum

Abstract

As one of Karp’s 21 NP-complete problems, the subset sum problem, as well as its generalization, has been well studied. Among the rich literature, there is little work on the online version, where items arrive over list and irrevocable decisions on packing them or not must be made immediately. Under the online setting, no deterministic algorithms are competitive, while for randomized algorithms the best competitive ratio is 1/2. It is thus of great interest to improve the performance bounds for both deterministic and randomized algorithms, assuming predicted information is available in the learning-augmented model. Along this line, we revisit online subset sum by showing that, with learnable predictions, there exist learning-augmented algorithms to break through the worst-case bounds on competitive ratio. The theoretical results are also experimentally verified, where we come up with a new idea in designing experiments. Namely, we design neural networks to serve as adversaries, verifying the robustness of online algorithms. Under this framework, several networks are trained to select adversarial instances and the results show that our algorithms are competitive and robust.

Appendices

A The learnability of predictions

In this section, we show that our predictions are efficiently learnable by proving the following theorem.

Theorem 8

Assume that each online subset sum instance I is sampled from an unknown distribution $\mathcal {I}$ and each item size belongs to a finite set $\mathcal {F}$. For each proposed algorithm, any $\epsilon >0$ and $\delta \in (0,1)$, there exists a learning algorithm which $(\epsilon ,\delta )$-learns the prediction from $O\left( \frac{1}{\epsilon ^2}\left( \log (|\mathcal {F}|) \right) + \ln \left( \frac{1}{\delta }\right) \right)$ training instances.

We start by introducing several definitions used here.

Definition 1

[11] Assume each instance $\mathcal {I}$ is sampled from a distribution $\mathcal {D}$. For an augmenting Algorithm A, let $s^*$ be the prediction with maximum $\mathrm {E}_{\mathcal {I}\sim \mathcal {D}}(A(\mathcal {I},s^*))$, where $A(\mathcal {I},s^*)$ is the objective value of Algorithm A with prediction $s^*$ on instance $\mathcal {I}$. A learning Algorithm L $(\epsilon ,\delta )$-learns the prediction from m samples if for any distribution $\mathcal {D}$, with probability at least $1-\delta$ over m samples $\mathcal {I}_1,...,\mathcal {I}_m \sim \mathcal {D}$, L outputs a prediction $\hat{s}$ such that

$$\begin{aligned} |\mathrm {E}_{\mathcal {I}\sim \mathcal {D}}(A(\mathcal {I},\hat{s}))-\mathrm {E}_{\mathcal {I}\sim \mathcal {D}}(A(\mathcal {I},s^*))| \le \epsilon .\end{aligned}$$

Definition 2

[11] Let $\mathcal {H}$ denote a set of real-valued functions defined on the set X. A finite subset $S = {x_1,...,x_m}\subseteq X$ is (pseudo-)shattered by $\mathcal {H}$ if there exist real-valued witnesses $r_1,...,r_m$ such that for each of the $2^m$ subsets T of S, there exists a function $h\in \mathcal {H}$ such that $h(x_i)>r_i$ if and only if $i\in T$ (for $i=1,2,..m$). The pseudo-dimension $d_{\mathcal {H}}$ of $\mathcal {H}$ is the cardinality of the largest subset shattered by $\mathcal {H}$.

From the definition of pseudo-dimension, we have the following observation.

Observation 1

If $\mathcal {H}$ is a finite set, its pseudo-dimension $d_{\mathcal {H}}$ is at most $\log (|\mathcal {H}|)$.

Proof

Consider a subset $S\subseteq X$. If $\mathcal {H}$ shatters it, there exists a unique function $h\in \mathcal {H}$ corresponding to each subset $T\subseteq S$. Thus, if $|\mathcal {H}| < 2^{|S|}$, $\mathcal {H}$ cannot shatter S because there is no bijection between $\mathcal {H}$ and the power set of S. In other words, if S is shattered by $\mathcal {H}$, we have $|S| \le \log (|\mathcal {H}|)$. Certainly, the cardinality of the largest subset shattered by $\mathcal {H}$ is also at most $\log (|\mathcal {H}|)$, completing this proof. $\square$

Definition 3

[11] Given a finite set of training instances $\{\mathcal {I}_1,...,\mathcal {I}_m\}$, say an algorithm is an empirical risk minimization (ERM) algorithm of an augmenting algorithm A, if it always returns a prediction $\hat{s}$ with minimum $\sum _{j=1}^{m}A(\mathcal {I}_j,\hat{s})$, where $A(\mathcal {I}_j,\hat{s})$ is the objective value of Algorithm A with prediction $\hat{s}$ on instance $\mathcal {I}_j$.

Rishi Gupta and Tim Roughgarden [11] give a theorem connecting the above three definitions. Putting that theorem into our augmenting algorithm context, we have

Theorem 9

[11] If any prediction $\hat{s}$ belongs to a finite set $\mathcal {H}$ and the objective value of the problem (e.g. subset sum) ranges in [0, H], for an augmenting algorithm, its any ERM algorithm $(\epsilon ,\delta )$-learns the optimal prediction from $O\left( \left( \frac{H}{\epsilon }\right) ^2\left( d_{\mathcal {H}} + \ln \left( \frac{1}{\delta }\right) \right) \right)$ training instances given any $\epsilon > 0$ and $\delta \in (0,1)$.

Proof of Theorem 8

For the online subset sum problem, its objective value is at most 1 since we assume that size of knapsack is 1. If any item size is in a finite set $\mathcal {F}$, the largest item size (in an optimal solution) also belongs to $\mathcal {F}$, meaning that any prediction $\hat{s}\in \mathcal {F}$. According to Observation 1, the pseudo-dimension of $\mathcal {F}$ is at most $\log (|\mathcal {F}|)$. Thus, applying Theorem 9 to our model completes the proof directly and the claimed learning algorithm is an ERM algorithm, which is directly picking $\hat{s}\in \mathcal {F}$ with the best average performance in the training instances. $\square$

B Worst-case bounds for learning-augmented algorithms

We construct a set of subset sum instances and a probability distribution, and show that the expected ratio of any deterministic algorithm will not be better than a certain value. According to Yao’s principle [27], this value is a worst-case bound for any (randomized) algorithms. Finally, we connect this value to predictions and prediction errors. We see that if the largest item size is predicted, no learning-augmented algorithm has a competitive ratio better than $1/2+\epsilon$ for any $\epsilon$ even when $\eta =0$; if we predict the largest item size in an optimal solution and the prediction error is larger than 0, the competitive ratio of any learning-augmented algorithm is at most $1/2+\epsilon -\eta$ (Theorem 5).

Inspired by the instance given in [12], we construct a distribution of n input sequences as follows, where $n=\frac{1}{\epsilon }$. The i-th input sequence is

$$\begin{aligned}\frac{1}{2}+\delta , \frac{1}{2}+\frac{\delta }{2},... \frac{1}{2}+\frac{\delta }{i},\frac{1}{2}-\frac{\delta }{i},\end{aligned}$$

where $\delta =\epsilon /4$. Obviously, its optimal objective value is 1 by packing the last two items $\frac{1}{2}+\frac{\delta }{i}$ and $\frac{1}{2}-\frac{\delta }{i}$. We claim the following lemma.

Lemma 1

If each item sequence is given with probability 1/n, the competitive ratio of any online deterministic algorithm is at most $\frac{1}{2} + \epsilon - \frac{1}{n}\sum _{i=1}^{n}\frac{\delta }{i}.$

Proof

The technique in this proof is similar to that in [12]. Say A is a deterministic algorithm, meaning that A has a deterministic policy to deal with large items with size $>1/2$. Due to the capacity constraint, A has to accept at most one of these large items and reject all the others. If A rejects all items with size $>1/2$, its competitive ratio is definitely less than $1/2\le \frac{1}{2}+\epsilon -\eta$. If A chooses a large item to accept, say this item is $\frac{1}{2} + \frac{\delta }{j}$. Because A is deterministic, it should always reject other large items.

For the input sequence j, A reaches on the optimal solution. For each input sequence $i<j$, the objective value is at most $\frac{1}{2}-\frac{\delta }{i}$ since A rejects all large items in these sequence. For each input sequence $i>j$, the competitive ratio is always $\frac{1}{2} + \frac{\delta }{j}$. Thus, we have

$$\begin{aligned} \begin{aligned} \frac{E(A)}{\mathrm {OPT}}&= \frac{1}{n}\left( \sum _{i=1}^{j-1} \left( \frac{1}{2} -\frac{\delta }{i}) + 1+\sum _{i=j+1}^{n}( \frac{1}{2} + \frac{\delta }{j}\right) \right) \\&\le \frac{1}{n}\left( \frac{1}{2}(j-1) -\sum _{i=1}^{j-1} \frac{\delta }{i} + (1+ \delta -\frac{\delta }{j})\right. \\&\left. \quad +\sum _{i=j+1}^{n}( \frac{1}{2} +2\delta - \frac{\delta }{i})\right) \\&\le \frac{1}{n}(\frac{1}{2}n + \frac{1}{2}+ 2\delta (n-j+1)) - \frac{1}{n}\sum _{i=1}^{n}\frac{\delta }{i} \\&\le \frac{1}{2} + \frac{1}{n}(\frac{1}{2} +2\delta n)- \frac{1}{n}\sum _{i=1}^{n}\frac{\delta }{i} \\&= \frac{1}{2} + \frac{1}{n}(\frac{1}{2} +2 (\frac{\epsilon }{4}) (\frac{1}{\epsilon }) )- \frac{1}{n}\sum _{i=1}^{n}\frac{\delta }{i} \\&= \frac{1}{2} + \epsilon - \frac{1}{n}\sum _{i=1}^{n}\frac{\delta }{i} . \end{aligned} \end{aligned}$$

$\square$

Predicting the largest item. Let the prediction of any constructed instance be $\frac{1}{2}+\delta$. Clearly, even if given such a prediction for each instance, $E(A)/\mathrm {OPT}$ is still at most $1/2+\epsilon$ for any deterministic algorithm A. For the error-free case, we see that knowing the accurate largest item size cannot help to bypass the bound 1/2.

Predicting the largest item in an optimal solution. Let the prediction of any constructed instance be 1/2. Similarly, the bound $\frac{1}{2} + \epsilon - \frac{1}{n}\sum _{i=1}^{n}\frac{\delta }{i}$ still holds. We can compute the expected prediction error $E(\eta )=\frac{1}{n}\sum _{i=1}^{n}\frac{\delta }{i}$. Thus, $E(A) \le 1/2+\epsilon -E(\eta )$ for any deterministic algorithm A. This implies that no learning-augmented algorithm has a competitive ratio better than $1/2+\epsilon -\eta$ and completes the proof of Theorem 5.

C Training details

This section presents training details of the experiments. An adversary network is a one-layer GRU network with hidden size 32. The input batch size of this network is 5. Each adversary is trained for 50000 epochs by Adam optimizer. The learning rate is 0.001.

Additionally, we use two tricks when sampling instances. One is that when the adversary outputs $s_i$ in step i, replace $s_i$ by a random number with probability p, helping avoid local optima. We initialize p by 0.04 and decrease it linearly to 0 as the number of training epoch grows. The other trick is that we always let the last instance of one batch be the instance with highest reward so far, in order to guide the network to a right direction. Without this trick, the adversary will encounter gradients vanishing frequently during its training.