To overcome such defects as poor environmental adaptability of traditional machine vision, a kind of environmental perception method based on binocular infrared ultra-wide field of view (FOV) camera was proposed. Horizontal and vertical models for ultra-wide FOV binocular infrared imaging were established by using high order polynomial model. Stereo vision range and threshold were researched on the basis of parallax model. Infrared ultra-wide FOV binocular stereo vision system with the vision field of 170°×128° was established. Stereo vision experiments were carried out under different conditions, such as uneven illumination, smog, etc. Subjective and objective evaluation shows that infrared ultra-wide FOV binocular stereo vision system has good applicability to complex scenes and is able to realize 3D depth perception of the infrared ultra-wide FOV, which basically meets the requirements of small robot vision system.