Instructional Material: Track ArUco Marker

This example program demonstrates how to program an autonomous behavior for the Tello, where it automatically acts on what it "sees". This program is a modification of the FindMarker program. In that program, we processed the video stream from the drone looking for the presence of an ArUco Marker in the camera field of view.

In this program when we enter marker detection mode with the X button, the program will take over flight control of the drone. It will rotate slowly until it detects an ArUco Marker in the camera field of view. The program will process the information returned by the marker detection class and compute a new flight command to center the marker image in the field of view. We do this by using the target rectangle. We determine the X axis location of the target center line and compare it to the image (field of view) center line. The difference tells us how far and in which direction the target rectangle is offset from the center of the field of view. We use this information to turn the drone as needed to point directly at the marker. As long as the marker is detected, the program will work to keep the marker centered and so cause the drone to track the marker. If marker detection is lost, the drone will resume rotating to acquire a new marker to track.

Additionally, when a  marker is first detected, the program records the size of the marker target rectangle. As it processes subsequent images it compares the new size of marker to the original size and uses the difference as a measure of distance. It then adds a forward or backward component to the flight command to make the drone maintain the original distance to the marker. If the target rectangle area is smaller than the original area, the target has moved away from the drone so we fly forward to get the target areas to match again. If the target area is larger than the original size, the target has moved closer to the drone so we fly backward to restore the original distance.

A key part of such schemes is to adjust (or scale) the offset and distance values to values that make sense as flyRC commands so that the drone moves at a speed that is compatible with the field of view and size of the marker. When trying to center the target, moving the drone too fast will cause the target to go out of view and stop the tracking. In the same vein, if you move the target too fast the drone will not be able to track it. All of this can be a challenge due to the Tello's small field of view.

Note: in order to improve tracking accuracy and responsiveness, we have shortened the delay of the main loop to 25ms.

Modify the Main class to execute the TrackMarker class and test autonomous marker tracking.

A great exercise for you would be to modify the program to adjust the drone's height to center the target vertically as well horizontally.