Abstract: For the CPU serial operation mode, it is a very time-consuming process to obtain the inverse of large-scale matrix. Aiming at the above shortcoming, this paper proposes a new programming method based on the common platform CUDA for GPU designed by NVIDIA. By using the multi-threaded parallel processing technology of GPU, a large scale of data during solving the inverse matrix are parallelly computed such that a higher speedup may be obtained. Moreover, both the single-precision and the double-precision FLOPS of GPU are analyzed according to the results of this program. Finally, some characteristics of the proposed algorithms are summarized by analyzing the effect of the data transmission time on the performance of GPU.