Linear Time Maximum Segmentation Problems in Column Stream Model

Abstract

We study a lossy compression scheme linked to the biological problem of founder reconstruction: The goal in founder reconstruction is to replace a set of strings with a smaller set of founders such that the original connections are maintained as well as possible. A general formulation of this problem is NP-hard, but when limiting to reconstructions that form a segmentation of the input strings, polynomial time solutions exist. We proposed in our earlier work (WABI 2018) a linear time solution to a formulation where minimum segment length was bounded, but it was left open if the same running time can be obtained when the targeted compression level (number of founders) is bounded and lossyness is minimized. This optimization is captured by the Maximum Segmentation problem: Given a threshold M and a set $\mathcal {R} = \{\mathcal {R}_1,\ldots ,\mathcal {R}_m\}$ of strings of the same length n, find a minimum cost partition P where for each segment $[i,j] \in P$, the compression level $\vert \{\mathcal {R}_k[i,j]: 1\le k \le m\} \vert$ is bounded from above by M. We give linear time algorithms to solve the problem for two different (compression quality) measures on P: the average length of the intervals of the partition and the length of the minimal interval of the partition. These algorithms make use of positional Burrows–Wheeler transform and the range maximum queue, an extension of range maximum queries to the case where the input string can be operated as a queue. For the latter, we present a new solution that may be of independent interest. The solutions work in a streaming model where one column of the input strings is introduced at a time.

This work was partially supported by the Academy of Finland (grant 309048).

Appendix

About the Column Stream Model 

Given an algorithm for a problem with an input $\mathcal {I}$ and an output $\mathcal {O}$, the space complexity of this algorithm corresponds to the space used by $\mathcal {I}$ and by $\mathcal {O}$ and the auxiliary space which is the temporary space used by this algorithm. Therefore the space complexity is in $\varOmega (\vert \mathcal {I} \vert +\vert \mathcal {O} \vert )$. In the case of problems of Maximum Segmentation, all algorithms have a space complexity of $\varOmega (nm)$ where the input is a set of m strings of size n. As we want to avoid an auxiliary space of $\varTheta (nm)$ (this could be too big for a computer), we cannot use the random access model. Indeed the random access model corresponds to open all the file in input in the temporary memory. We suggest a specific streaming data model where the set of strings of the same length is seen column by column: the Column Stream Model. In this model, the size of the input is in $\varTheta (m)$ which is acceptable.

To prove the realism of this model, we implemented a streaming way to read a file and we tested this implementation with files of different sizes (see Fig. 2). The experiments were run on a machine with an Intel Xeon E5-2680 v4 2.4GHz CPU, which has a 35 MB Intel SmartCache. The machine has 256 gigabytes of memory at a speed of 2400MT/s. The code was compiled with g++ using the -Ofast optimization flag.

Fig. 2.

Time and space complexity to read a set of recombinants depending of the buffer size (256, 1024 and 4096).