Single wall carbon nanotube (SWCNT) is a promising one dimensional (1D) material to fabricate high performance infrared (IR) detectors owing to its unique electrical and physical properties. The 1D Schottky barrier between metal and CNT can separate the photon-generated electron-hole pairs so as to produce photocurrent for quantification and detection. However, the theory developed for the planar metal-semiconductor contact is not compatible with the 1D Schottky barrier within the CNT, thus the optimized structure for a CNT detector is unknown. Our understanding can be improved by using the capacitance-coupled electrostatic doping from a gate of a CNT transistor, which will find out the role of the CNT energy level. A standard back gate CNT transistor based photodetector was fabricated, which showed that positive gate voltages could improve the performance by widening the Schottky barriers. However, the back gate geometry will modulate two Schottky barriers simultaneously with applied bias, severely degrading the detector performance. In order to optimize gate structure for the CNT IR detector, we propose a detector integrated with three different gate structures: side gates for source and drain, and middle gates for the bulk of CNT. The side gates next to the source and drain control the carrier injection at the junctions independently, while the middle gates can block the fringing field from the other gates, and modulate the Fermi level of the CNT channel. We found that opposite gate voltages at source and drain terminals can optimize the performance of the detector by widening one barrier, but eliminating the other. The optimized structure can lead to a high performance nano-scale photon harvest device. This will pave the way for the CNT as a significant building block for future nano-optoelectronics.