Ben Caunt, from Warm Automation, has showcased his new open-source project that allows robots to detect and track objects in real time using Moondream AI and OpenCV. The project, called MoondreamObjectTracking, combines AI-based object detection with optical flow tracking to follow objects efficiently. It works by capturing video from a webcam, identifying objects using Moondream’s vision model, and tracking their movement with OpenCV’s Lucas-Kanade algorithm. A Kalman filter is also used to smooth the tracking data, making the results more stable.
The system is designed to run across multiple devices on the same network. A webcam publisher streams video frames, which are then processed by the tracking module to detect and follow the target object. The tracking data is shared in real-time, allowing a robot to respond accordingly. A separate visual servoing component can use this data to control the robot’s movement, adjusting its speed and direction based on the object's position. The project is lightweight and can be set up on a laptop or directly on a robot. Unlike traditional approaches that rely solely on neural networks, this system optimizes detection frequency by combining AI-powered identification with optical flow tracking. When new detections arrive, the system intelligently rewinds and reapplies them, ensuring consistent object localization.
By open-sourcing MoondreamObjectTracking, Ben has made it easy for others to experiment with real-time object tracking for robotics. The system is useful for applications like autonomous navigation and interactive robotics. Since it does not require powerful hardware, it is accessible to hobbyists and researchers alike. The project shows how AI and traditional computer vision methods can be combined to improve real-time tracking performance in a simple and efficient way.
