In this paper, we propose methods to improve image-based auto-focus that can compensate for drawbacks of traditional auto-focus control. When adjusting the focus, there is a problem that the focus window cannot be set to the same position if the camera's LOS is not directed at the same location and flow or shake. To address this issue, we applied image tracking techniques to improve optimal focus localization accuracy. And also, although the same focus value should be calculated at the same focus step, but different values can be calculated by camera's fine shaking or image disturbance due to atmospheric scattering. To tackle this problem a SAFS (Stable Adjacency Frame Selection) has been proposed. As a result of this study, our proposed methodology shows more accurate than traditional methods in terms of finding best focus position.