Abstract: Due to the outstanding data efficiency and learning rapidly, many real-world problems have begun to apply reinforcement learning to learn complex strategies. However, reinforcement learning in high-dimensional environments is often limited by the curse of dimensionality or catastrophic interference, leading to bad performance or even failure. This paper proposes a sparse convolutional deep reinforcement learning method around representation learning, which conforming to Lasso-type optimization. First, the theory and advantages of sparse representation are reviewed, and the sparse convolution method is innovatively introduced into deep reinforcement learning, proposing a new sparse representation method. Second, this paper derives the differentiable optimization layer defined by sparse convolutional encoding mathematically and give an optimization algorithm. In addition, in order to verify the effectiveness of the new sparse representation method, we tested it in benchmark environments commonly considered within the related literature. Experimental results show that the algorithm using sparse convolutional encoding has better performance and robustness, achieving comparable or even superior performance with more than 50% reduction in model cost. In addition, we also studied the impact of sparsity on algorithm performance, and the results show that an appropriate degree of sparsity can achieve better performance.