RCAM-Transformer: A Novel Approach to Table Reconstruction Using Row-Column Attention Mechanism

Abstract

Table reconstruction, a critical task in the field of table structure recognition (TSR), plays a vital role in various domains, such as data mining, machine learning, and information retrieval. While many existing TSR methods employ transformer-based models with generally impressive performance, a gap remains in transformer models specifically designed to handle the distinct attributes of table rows and columns. Moreover, there is a lack of robust table reconstruction strategies based on object detection models. To address these issues, we introduce the Row-Column Attention Mechanism (RCAM). When combined with a transformer model and integrated with partial global attention, it forms the RCAM-Transformer. This model is tailored to effectively process the unique properties of tabular data. In addition, we have developed a novel table reconstruction strategy that leverages object detection models, which improves the recognition and treatment of tabular data. Our experiments, conducted using the PubTables-1M and FinTabNet dataset, along with our self-constructed Annual Report TableSet, not only validated the effectiveness of the RCAM but also demonstrated the improved accuracy of table reconstruction with the use of our RCAM-Transformer. Such outcomes highlight the potential of the RCAM-Transformer to advance table extraction in various fields.

A Appendix

This appendix presents details of table reconstruction. Generally, we require the results of the object detection model, that is, accurate coordinates and attributes of the cells. We then proceed to the post-processing steps for table reconstruction. Several details are worth discussing further. One such detail is that the object detection model often outputs overlapping rows or columns, even if the confidence limit of the output is increased. Addressing this necessitates the design of a reasonable IoU to filter these overlaps, because failing to do so would result in redundant table lines in the final table reconstruction results. Another detail pertains to the results of the Hough Line Transform. Interference is often encountered, such as small illustrations, colored backgrounds, or closely connected characters, all of which generate noise. This necessitates the design of more reasonable image morphological operations and subsequent filtering to remove horizontal and vertical lines detected via Hough Line Transform that account for less than a certain proportion of the total length and width of the table. Moreover, blank rows may appear after the results from the model recognition are aligned with Hough Line Transform. Although these factors do not affect the table structure, they can influence the downstream tasks. Therefore, to remove blank rows, it is necessary to lock the results of Hough Line Transform and remove the horizontal lines below them. All the examples shown in Fig. 7 have undergone blank-row processing. During the merging of cross-cells, headers or non-headers can be selectively merged according to the cell attributes. However, we cannot effectively merge areas without text, such as the top-left corners of Figs. 4d, 7a, 7c, 7e, 7g, 7i, and 7k. If we let the model recognize a blank area as a spanning cell, this category of recognition would become unstable.

Fig. 7.

Table reconstruction results.