Wavelet-based separation of nonlinear show-through and bleed-through image mixtures

Abstract

This work addresses the separation of the nonlinear real-life mixture of images that occurs when a page of a document is scanned or photographed and the back page shows through. This effect can be due to partial paper transparency (show-through) and/or to bleeding of the ink through the paper (bleed-through). These two causes usually lead to mixtures with different characteristics.

We propose a separation method based on the fact that the high-frequency components of the images are sparse and are stronger on one side of the paper than on the other one. The same properties were already used in nonlinear denoising source separation (DSS). However, we developed significant improvements that allow us to achieve a competitive separation quality by means of a one-shot processing, with no iteration. The method does not require the sources to be independent or the mixture to be invariant, and is suitable for separating mixtures such as those produced by bleed-through, for which we do not have an adequate physical model.

Introduction

This paper focuses on the separation of two-image mixtures that occur in a well known practical situation: when we scan or photograph a document and the back page shows through. This effect is often due to partial transparency of the paper (which we designate by show-through). Another possible cause is bleeding of ink through the paper, a phenomenon that is more common in old documents, in which the ink has had more time to bleed. The latter phenomenon is commonly designated by bleed-through. The two phenomena may be simultaneously present in the same document.

In this work we use, as test examples, three different kinds of mixtures. The first kind essentially only contains the show-through effect: five pairs of images were printed on the two sides of five sheets of tracing paper¹ which, due to its high transparency, creates very strong mixtures. The second type corresponds to an old manuscript letter written in very thin “air-mail” paper (also called onion skin paper, which is rather transparent, causing show-through to occur). This document also has some areas in which bleed-through appears to have occurred. The third kind of mixture corresponds to images of old manual transcripts of music (partitures), which mostly contain the bleed-through effect. For each document, scanning or photographing both sides allowed us to obtain two different mixtures of the contents of the two pages. In this paper we address the source separation problem whose aim is to recover, form the two acquired images of each document, the original page images.

Show-through is known to lead to nonlinear mixtures [2], [5], [3]. A physical model of the show-through mixture of gray-level images printed with halftoning has been presented in [3]. Bleed-through probably is a much more complex phenomenon, which is much harder to model.

Source separation is often performed by assuming that the sources are statistically independent from each other, an assumption which leads to the use of independent component analysis (ICA) techniques. While linear ICA is a well studied problem for which several efficient solutions exist [6], [7], [27], nonlinear ICA is a much less studied problem [1], [13], [10], [8]. Nonlinear ICA has the additional difficulty of being ill-posed, having an infinite number of solutions without any simple relationship with one another [14], [11]. The mixtures addressed in this work are nonlinear and noisy, and the letter and partiture mixtures are spatially variant. Besides these challenging properties, most of the sources studied in this work do not completely obey the independence assumption, a fact which affects the quality of the results obtained through ICA-based methods [2], [5].

Instead of assuming independence of the source images, we propose a solution that uses other properties of images and of the mixture process. We use the well known fact that high-frequency components of images are sparse (and that high-frequency wavelet coefficients are also sparse), and we formulate a competition based on the observation that each source is more strongly represented in one of the mixture components than in the other one. Making assumptions that are suited to the present problem, our method achieves a good perceptual separation quality even when the sources are non-independent and the mixture is spatially variant. The separation method that we propose is similar to the denoising step used by nonlinear denoising source separation (DSS) [5]. However, we use an improved form of competition, and also a wavelet transform that is more suited to the problem at hand. These improvements lead to a method that performs the separation in a single step, without the iterative procedure required by nonlinear DSS.

Both the old manuscript letter and the old partitures with the bleed-through effect are addressed for the first time in this paper. On the other hand, the tracing paper mixtures have already been studied in other works [2], [5], [3]. Contrasting with the method proposed here, which, due to its use of wavelets, performs a non-point-wise transformation, all the other mentioned methods performed point-wise separation. One of them [2] used the MISEP method of nonlinear ICA [1] to train a regularized MLP which performed the separation. In another one [3], MISEP was used to train a nonlinear physical model of the mixture process. Nonlinear DSS has also been applied to some of the tracing paper mixtures [5]. Nonlinear DSS does not assume independence of the sources, but assumes spatial invariance of the mixture. It uses the same basic ideas that are used in this paper, albeit in a less efficient manner. Show-through and/or bleed-through mixtures have also been addressed in [19], [22], [23], [24], [9], but in different settings from the one considered here. In [19], [24] separation is archived through linear models, which were shown to be too restrictive to separate tracing paper mixtures [2]. Refs. [22], [9] focus only on the restoration of text documents, for which linear separation yields relatively good results (see [2]). In [23], the contents of both pages are assumed to consist of text, and separation is linear and is based on a single color image from one side of the document.

This manuscript is structured as follows: Section 2 describes the three kinds of mixtures that were studied, as well as the processes of image acquisition and alignment. Section 3 describes the proposed separation method. Section 4 presents experimental results, and Section 5 concludes.

The Matlab separation routines and the images used in this work are available at http://www.lx.it.pt/∼mscla/. The routines for performing image alignment are available at http://www.lx.it.pt/∼lbalmeida/ica/seethrough/.