Stereo matching under dramatic illumination changes is a big challenge in imbalanced binocular vision, self-driving cars, and the remote sensing image field. A novel, to the best of our knowledge, multi-brightness layer mechanism with a genetic optimization algorithm is proposed in this paper. The mechanism of multi-brightness layers transforms the two images with dramatic illumination changes into a series of matched pairs with similar brightness by the stretching function and histogram matching principle. Therefore, the large illumination variations are reduced greatly. Moreover, the initial disparities as first generation of genetic optimization approach are generated from matched pairs using fast segmentation local stereo matching to increase the efficiency and accuracy. For further improving the accuracy of disparity, an enhanced genetic optimization algorithm for stereo matching is designed to have more inliers and continuity. The experimental results comparing with state-of-the-art stereo matching methods demonstrate that the proposed method has better performance in accuracy and stability. (C) 2021 Optical Society of America