High definition (HD) camera is widely used in surveillance systems. An HD camera with optical zoom is useful for monitoring a large area. However, it is inconvenient for a user to manually control the optical zoom for a long time. To exploit the functionality and extend the application domains of a HD camera, the zooming should be controlled automatically. Therefore, an automatic zooming mechanism is proposed in this paper. When the number of an object is small in the field of view (FOV) of the camera and an object is moving through the FOV, the zoom is controlled for capturing the object as clear as possible. A clear object image is useful for related image-based services, such as face recognition. In order to achieve the above goal, a Gaussian Mixture Model (GMM), temporal image differencing, a CamShift tracking method, and a Kalman filter are utilized for object detection and tracking. Then, an adaptive neuro-fuzzy inference system (ANFIS) is used to learn and determine a suitable value for adjusting the zoom. According to the experimental study of the prototype, the results show that the proposed mechanism is useful to capture the clear images of moving objects in a practical environment. A face detection algorithm is also used to demonstrate the feasibility of the captured clear images.