For semantic segmentation of optoelectric images, it proposes a segmentation algorithm based on encoder-decoder structure and local image enhancement. Firstly, the algorithm adopts an atrous spatial pyramid pooling (ASPP) module based on coprime factors to reduce the grid effect caused by multiscale convolution and improve the capability of representing pixel adjacent information of convolution kernels. Secondly, in order to improve pixel weight of areas that are difficult to segment, the algorithm combines loss function consisting of mean intersection and cross entropy with weight decay strategy. Experimental results show that the proposed algorithm can effectively improve the accuracy of image semantic segmentation.