Digital halftoning, referring to convert a continuous-tone image into a bi-level halftone image, has been applicable to several bi-level output devices. However, inverse halftoning as a classic image restoration problem is still challenging to reconstruct the continuous tone and image details from halftone images. In this paper, a transformer-based deep inverse halftoning network with attention mechanism is proposed for halftone image restoration. The key is to design an encoder-decoder architecture consisting of Swin Transformer, channel attention, and global/local attention modules, for image feature learning and reconstruction. As a result, the proposed network effectively learns features hierarchically from the input halftone image and well reconstruct the corresponding continuous-tone image. The proposed deep model has been shown to outperform the state-of-the-art (SOTA) deep halftone image restoration networks quantitatively and qualitatively.