Abstract: Due to the outstanding data efficiency and learning rapidly, the reinforcement learning method is applied in many real-world problems to learn complex strategies. However, reinforcement learning in high-dimensional environments is often limited by the curse of dimensionality or catastrophic interference, resulting in poor performance or even learning failure. Aiming at representation learning, this paper proposes a sparse convolutional deep reinforcement learning method based on Lasso-type optimization. First, the theory and advantages of sparse representation are reviewed, and the sparse convolution method is innovatively introduced into deep reinforcement learning to obtain a new sparse representation method. Secondly, a mathematical derivation is conducted on the differentiable optimization layer defined by sparse convolutional encoding, based on which an optimization algorithm is proposed. In addition, in order to verify the effectiveness of the new sparse representation method, we tested this method in common benchmark environments. It is shown from experimental results that the sparse convolutional encoding-based algorithm has better performance and robustness, achieving comparable or even superior performance while reducing model overhead by more than 50%. Furthermore, the impact of sparsity on algorithm performance is also investigated, which shows that appropriate sparsity can achieve better performance.