Abstract: Due to the low illumination in the night, it is difficult to observe and identify the nighttime images exactly. To address this problem, we try to map the feature colors of different scenery to the nighttime images of visible and long-wave infrared bands such that the observers can analyze the images and speed up the reaction time. To this end, this paper presents a category-oriented color lookup tables night-vision colorization method by combining deep learning. Firstly, it utilizes the semantic segmentation via parallel fully convolutional network and the feature of two-way network at the end of the network to get the segmentation categories. And then, it colorizes the nighttime images by each category-oriented color lookup table to avoid the same color scheme and unnatural colors resulted from the global colorization like using single lookup table. In the post-processing stage, both the image normalization and the multispectral bilateral filtering will be adopted to smooth the colors and preserve the edges. The proposed method can fix the dual-band information, which is different from the other methods. By constructing the category-oriented color look-up table, the proposed method can obtain more stable results and much better material resolution. Moreover, the image contrast is enhanced and the thermal targets is highlighted by red such that it is much easier to detect. Besides, the proposed method can also be extended according to different environments and applications via the fixed category-oriented color lookup tables. Finally, by the comparison with other methods, such as global colorization by single lookup table, it is shown that the proposed method in this work can attain significant improvements in this area of night-vision colorization.