The ciphertext only attack algorithm (COA) is an effective attack algorithm for the double random-phase encoding (DRPE) encryption system using random phase mask (RPM) as the key. In order to improve the ability of DRPE encryption system to resist ciphertext only attack algorithm attack, a DRPE encryption system based on random Radon transform is proposed. The system performs random Radon transform on the image to be encrypted and then encrypts it using the DRPE encryption system. In addition to RPM, a set of random parameters selected by the random Radon transform is used as a new key for successfully recovery the original image. The simulation results show that the DRPE encryption system based on Radon transform can effectively resist the COA algorithm and improve the security of the DRPE encryption system compared to the DRPE encryption system using only RPM as the key.