WorksComputer Science
Tutorial: Vision-Based Formation Control with Differential-Drive Robots
Abstract
This tutorial presents a step-by-step introduction to multi-robot formation control using differential- drive robots, an overhead camera, and Python. ArUco markers are used to estimate robot positions and orientations, while desired pairwise displacement vectors define a circular formation. By comparing measured and desired relative positions, the controller computes movement vectors that are converted into differential-drive wheel commands, with basic proximity-sensor-based obstacle avoidance. The tutorial guides readers through hardware and software setup, robot pose estimation, formation geometry, feedback control, system integration, and formation-error evaluation. Motivated by the need to bridge the gap between theoretical understanding and practical implementation in robotics education, we emphasize how mathematical concepts are translated into coordinated behaviors on physical robots. Although the implementation uses a specific robotic platform, the underlying perception and formation-control principles can be adapted to other differential-drive robots and simulation environments. The accompanying open-source GitHub repository provides the Python implementation, setup instructions, and demonstrations to support hands-on learning and further exploration of multi-robot systems.
GitHub Repository: https://github.com/SMARTlab-IU/formation-control-tutorial
Introduction
Formation control enables a team of robots to achieve and maintain a desired spatial arrangement through coordinated motion. Unlike classical point-convergence and rendezvous problems, in which robots move toward a common location [1, 2], formation control seeks to establish specified spatial relationships among robots. This capability is fundamental to multi-robot coordination and provides a useful setting for studying how individual robot behaviors contribute to a collective objective. However, translating formation-control principles into physical robot motion requires more than understanding the underlying mathematical models. It also involves integrating sensing, coordinate transformations, feedback calculations, and motion control into a functioning robotic system.
Formation control has been extensively studied through various approaches, including behavior-based coordination [3], control of nonholonomic mobile robots [4], and formation-constrained multi-agent control [5]. A comprehensive survey by Oh et al. [6] categorizes major formation-control strategies and their underlying principles. Related research has also explored beacon-based distributed structure formation [7], multipoint rendezvous [8], bearing-aided rendezvous using network topology [9], and communication-aware robot coordination, including directional-antenna-based leader–follower relaying [10] and consensus using the direction of arrival of wireless signals [11]. These studies illustrate different approaches to multi-robot coordination, sensing, and communication. While such methods provide important theoretical and algorithmic foundations, implementing even a relatively simple formation controller on physical robots introduces practical challenges that are valuable for students to understand.
A primary motivation for developing this tutorial is to make these fundamental concepts more accessible through hands-on learning. In robotics education, students often encounter mathematical models and control algorithms before having opportunities to observe how they behave on physical systems. We believe that connecting theory with implementation is an essential part of learning robotics, particularly in multi-robot systems, where sensing, coordination, and individual robot motion must work together. Accordingly, this tutorial is designed to guide students from the basic geometric principles of formation control to their implementation and demonstration on physical robots, helping them develop both conceptual understanding and practical experience.
This tutorial presents a step-by-step implementation of vision-based formation control using a team of differential-drive robots (e.g., Robomation’s HamsterS), an overhead camera, and Python. ArUco markers are used to identify individual robots and estimate their positions and orientations in image coordinates. Each robot is assigned a fixed slot among equally spaced points on a circle, and the desired formation is represented through pairwise displacement vectors. By comparing the measured relative positions with the desired displacements, the controller computes a movement vector for each robot, which is subsequently translated into turning and forward wheel commands. Because the formation is defined through relative displacements rather than fixed global destinations, the robots can establish the desired arrangement at different locations within the camera’s field of view. Figure 1 illustrates the formation acquisition process using four physical robots.
Figure 1. Formation acquisition by four robots in one experimental run at seconds. Red arrows show control vectors; gray lines connect robots in consecutive formation slots. The sequence provides a qualitative illustration of the process. The frame annotations label the four robots 0–3; the example configuration uses marker IDs 0, 1, 2, and 3.
Throughout the tutorial, we emphasize the relationship between mathematical formulations and their practical implementation. Readers will learn how to estimate robot poses from camera images, construct desired formation geometries, calculate displacement errors, and convert control vectors into differential-drive motion. We also address practical considerations such as consistent coordinate conventions, robot identification, controller parameters, obstacle responses, and formation-error evaluation. The accompanying GitHub repository provides the Python implementation, setup instructions, and demonstrations [12], enabling readers to reproduce the experiments and further explore the concepts presented here. We hope this tutorial provides an accessible starting point for students and researchers interested in robotics, particularly multi-robot systems, and encourages further learning and experimentation in robot coordination and control.
System Overview
Our system consists of a team of differential-drive robots, a stationary overhead USB camera, and a central computer running Python. Each robot carries a unique ArUco marker, which OpenCV uses to estimate its position and heading in image coordinates. The computer compares the measured relative positions with the desired pairwise displacements of a circular formation and computes a control vector for each robot. These vectors are converted into differential-drive wheel commands and transmitted through RoboID. The process repeats with each new camera frame, forming a continuous perception-and-control loop.
The implementation is centralized: a single computer processes all observations and computes commands for the entire team. If any required marker is not detected, all robots stop until valid observations are restored. Onboard proximity sensors also provide basic reactive obstacle avoidance. Figure 2 illustrates the overall system architecture, while Figure 4 summarizes the control decisions within each iteration.
Figure 2. System architecture. A fixed overhead camera observes the marked robots; the central computer estimates poses, computes pairwise formation errors, and sends wheel commands through RoboID. The configuration supplies marker IDs, slot geometry, gains, and thresholds. The dashed display block is optional.
Figure 4. Frame-by-frame control flow. The host computer updates marker-based poses and stops all robots if a required pose is missing. Otherwise, it computes formation-control vectors and selects wheel commands, with the proximity response taking priority over formation motion. The pairwise error is current displacement minus desired displacement, .
Unlike our centralized implementation, Fredslund and Matarić [13] investigate formation control using local sensing and minimal communication, while Lawton et al. [14] study decentralized formation maneuvers.
Project structure
The program is organized into main.py, settings.py, and four modules in utils/. Table 1 summarizes their roles.
| File | Role |
main.py | Initializes robots and executes the main control loop. |
settings.py | Stores marker IDs, formation parameters, gains, and thresholds. |
utils/camera_supervisor.py | Detects markers and updates robot positions and orientations. |
utils/formation_control.py | Constructs formation slots and desired relative displacements. |
utils/actuation.py | Converts a control vector into wheel commands and checks proximity sensors. |
utils/img_overlay.py | Provides optional pose, vector, and formation-error visualization. |
Table 1. Organization of the Python implementation.
Hardware Setup
Requirements
The example uses two or more HamsterS differential-drive robots, a fixed overhead USB camera, one distinct printed ArUco marker per robot, and a computer capable of running Python. We recommend a camera capable of 1080p resolution. Other differential-drive robots can use the formation calculations and marker-based pose-estimation approach, but their connection libraries, sensor calls, and wheel-command units may differ.
Generating and mounting markers
ArUco markers are square fiducial markers that can be identified from their encoded patterns [15]. Use the same dictionary during generation and detection. The example configuration uses DICT_7X7_100. The following excerpt generates marker ID 0, the first marker in that configuration:
import cv2
dictionary = cv2.aruco.getPredefinedDictionary(
cv2.aruco.DICT_7X7_100
)
marker = cv2.aruco.generateImageMarker(
dictionary, 0, 500
)
cv2.imwrite("marker_0.png", marker)
Generate a different ID for each robot. Print the markers flat, retain a clear white margin, and attach them securely so that they cannot rotate relative to the robot. See the OpenCV documentation for the marker-generation and detection APIs [16].
Mounting direction matters. The top edge of the generated marker image must point toward the robot’s front. The orientation calculation in § 5 uses this convention. The marker center represents the robot center only approximately unless the marker is mounted centrally.
Camera placement
Secure the camera above the workspace, approximately perpendicular to the floor. Keep every marker within view and maintain adequate, consistent lighting. Motion blur and autofocus changes can reduce detection reliability; fixed focus can help. The camera should remain stationary during operation.
Software Setup and Configuration
Our implementation uses Python 3.10 or newer, NumPy [17] for array calculations, OpenCV [18] with the ArUco module for image processing, and RoboID for communication with HamsterS robots. The RobotAgent annotations use the | None syntax to indicate that a value may be temporarily unavailable, such as a robot position before its marker is detected. Install the dependencies with:
python -m pip install numpy opencv-contrib-python roboid
The controller imports the robot interface with from roboid import *. RoboID is distributed through its maintainer’s Python package page [19]. Matplotlib is used only for the optional error plot described in § 10; it is not part of the installation command above.
Controller settings
The example uses the following values in settings.py:
target_ids = [0, 1, 2, 3]
ROBOTS_N = 4
SHAPE_R = 100
KP_DISTANCE = 0.7
KP_ANGLE = 0.5
DISTANCE_THRESHOLD = 10 # Control-vector magnitude, pixels
ANGLE_THRESHOLD = 30 # Degrees
OBS_THRESHOLD = 70
OBS_SPEED = 50
Here, SHAPE_R is the formation radius in image pixels. The gains determine turning and forward commands, while the thresholds determine when those commands or the proximity response are selected. Their roles are explained in § 7.
The order of target_ids defines the mapping between marker IDs and robot connections. In this example, marker ID 0 corresponds to robot index 0, ID 1 to index 1, ID 2 to index 2, and ID 3 to index 3. Marker IDs do not need to match robot indices, as long as the mapping remains consistent. Connect the physical robots in the specified order, and ensure that ROBOTS_N equals the number of entries in target_ids. Note that marker IDs are distinct from RoboID connection IDs. During initialization, slot=i assigns each robot to the formation slot corresponding to its index.
Robot Pose Estimation
ArUco detection and corner ordering
Create a detector using the dictionary selected when printing the markers:
aruco_dict = cv2.aruco.getPredefinedDictionary(
cv2.aruco.DICT_7X7_100
)
detector = cv2.aruco.ArucoDetector(aruco_dict)
corners, ids, rejected = detector.detectMarkers(frame)
The result corners contains four image coordinates for each detected marker, and ids contains the corresponding marker IDs. If no markers are identified, ids is None. The third result contains rejected marker candidates; it is not a Boolean success flag.
OpenCV orders the corners as top-left, top-right, bottom-right, and bottom-left relative to the marker’s encoded orientation [16]. Denote them by , , , , with each a two-dimensional image coordinate. Figure 3 shows the geometry used below.
Figure 3. Position and heading from the ordered marker corners. The image -axis points right and the -axis points down. The direction from the bottom-edge midpoint to the top-edge midpoint is the robot’s forward direction when the marker is mounted as specified. The marker pattern is schematic, not a printable ArUco marker.
Finding robot position
The robot position is estimated by averaging the four marker corners:
The averaging is applied separately to the horizontal and vertical coordinates. In Python:
marker_corners = corners[index][0]
center = marker_corners.mean(axis=0)
x = center[0]
y = center[1]
Here, index selects one detected marker from the corners output. It is the marker’s position in the returned list, not its encoded ID. Positions remain in camera pixel coordinates: the origin is at the top-left of the image, increases to the right, and increases downward. The same convention must be used in all subsequent calculations.
Finding robot orientation
Compute the midpoints of the top and bottom marker edges:
The forward vector and heading are
The implementation converts the angle to degrees and normalizes it to :
topLeft, topRight, bottomRight, bottomLeft = marker_corners
top_center = (topLeft + topRight) / 2
bottom_center = (bottomLeft + bottomRight) / 2
heading_vector = top_center - bottom_center
theta = math.degrees(
math.atan2(heading_vector[1], heading_vector[0])
)
theta %= 360
Because image increases downward, positive angles increase clockwise. The pose can be described as , although the RobotAgent object stores position and orientation in separate fields.
Updating poses safely
The update_poses() function resets every robot’s position and orientation before processing a new frame. A robot receives a valid pose only if its marker is detected in that frame.
The function belongs in utils/camera_supervisor.py and uses the detector initialized above:
def update_poses(ROBOTS, frame):
for robot in ROBOTS.values():
robot.position = None
robot.orientation = None
corners, ids, _ = detector.detectMarkers(frame)
if ids is None:
return
for marker_corner, marker_id in zip(corners, ids.flatten()):
marker_id = int(marker_id)
if marker_id not in ROBOTS:
continue
c = marker_corner.reshape((4, 2))
topLeft, topRight, bottomRight, bottomLeft = c
heading = ((topLeft + topRight)
- (bottomLeft + bottomRight)) / 2
ROBOTS[marker_id].position = c.mean(axis=0)
ROBOTS[marker_id].orientation = (
math.degrees(math.atan2(heading[1], heading[0])) % 360
)
The main loop stops the team when a required pose is missing. Without this reset, an early return when no markers are found would leave the previous frame’s poses in place.
Optional visual check. To draw the detected marker outlines and IDs, add:
if ids is not None:
cv2.aruco.drawDetectedMarkers(frame, corners, ids)
Formation Geometry and Control Vectors
The formation is defined in two steps. First, place fixed slots on a circle. Second, calculate the displacement from every slot to every other slot. The controller then compares these desired displacements with the measured robot-to-robot displacements.
Circular formation
For robots and radius , consecutive slots are separated by radians. Slot has desired coordinates
The collection of these positions is (“P star”). The configuration uses ROBOTS_N for and SHAPE_R for . The corresponding function is
def p_star():
desired_positions = np.zeros((settings.ROBOTS_N, 2))
for k in range(settings.ROBOTS_N):
th = 2 * np.pi * k / settings.ROBOTS_N
desired_positions[k] = [
settings.SHAPE_R * np.cos(th),
settings.SHAPE_R * np.sin(th),
]
return desired_positions
Figure 5 shows the same construction for different robot counts. All four circles use the same radius and display scale. The panel illustrates slot geometry; the multi-robot tutorial uses two or more robots.
Figure 5. Fixed slots on circles of equal radius, pixels. Identical axis bounds and aspect ratios give all four circles the same displayed diameter. Slot coordinates follow Eq. (4), using image axes with increasing downward. For four robots, the slots form a square.
The slot coordinates are defined relative to ; they describe the formation shape, not a fixed destination in the camera image. The distance between adjacent slots is
For ROBOTS_N = 4 and SHAPE_R = 100, the adjacent center-to-center spacing is approximately 141.4 pixels. At a fixed radius, increasing the number of robots reduces the adjacent spacing. Select a radius that leaves room for the robots’ physical size and movement.
Desired pairwise displacements
Rather than commanding robot to move to the image coordinate , define the desired vector from its slot to slot :
The collection of these vectors is (“D star”). Each vector specifies both direction and distance. For example, means that robot should be 100 pixels to the right of robot . That relationship does not change when the whole formation is moved to another part of the camera image.
Figure 6 illustrates the desired vectors for three robots. In each panel, the filled square is the source slot, and the arrows point toward the other slots.
Figure 6. Desired pairwise displacement vectors in a three-robot circular formation. Each arrow is calculated as target-slot position minus source-slot position, . The three panels use the same geometry and scale.
The function d_star() builds an array with shape , using robot indices for array entries and slots for desired positions:
def d_star(ROBOTS, desired_positions):
desired_offset = np.zeros(
(settings.ROBOTS_N, settings.ROBOTS_N, 2)
)
for robot_a in ROBOTS.values():
for robot_b in ROBOTS.values():
desired_offset[robot_a.robot_i, robot_b.robot_i] = (
desired_positions[robot_b.slot]
- desired_positions[robot_a.slot]
)
return desired_offset
Formation error and control vector
The measured displacement from robot to robot is
Subtract the desired displacement to obtain the pairwise error:
If the current displacement equals the desired displacement, the error is zero. The subtraction order is important: the current displacement comes first.
For example, let , , and . Then
The horizontal and vertical components are calculated in the same image frame used for pose estimation. The positive horizontal and negative vertical components contribute a rightward and upward direction, respectively, to robot ’s control vector. Its final movement direction also depends on the errors relative to the other robots.
Each robot’s control vector is the sum of its pairwise displacement errors:
The result gives a desired movement direction and a magnitude used by the wheel-command routine. The sum excludes because a robot has zero displacement relative to itself. It is a sum of displacement errors, not a sum of uncorrected robot-to-robot distances.
NumPy broadcasting calculates pairwise displacements for all robots without explicitly looping over each pair. The None entries introduce new array dimensions, allowing NumPy to subtract each robot’s position from every other robot’s position:
actual_offset = positions[None, :, :] - positions[:, None, :]
displacement_error = actual_offset - desired_offset
u = np.sum(displacement_error, axis=1)
Here, actual_offset[i, j] equals . The displacement arrays have shape , and u has shape , one movement vector per robot.
The use of pairwise differences connects this calculation conceptually to consensus-based coordination; Olfati-Saber et al. [20] provide an overview. Analyses of switching topologies and communication delays [21], and of dynamically changing interaction topologies [22], provide further theoretical background. Those results concern their respective models and assumptions; they do not establish convergence or delay tolerance for this camera-based turn-then-drive implementation.
Differential-Drive Motion Control
A differential-drive robot cannot directly execute an arbitrary planar displacement. It changes heading and moves forward by controlling its two wheels. This motion constraint is standard in mobile robotics [23]; we use a turn-then-drive rule.
Target heading and angle error
The target direction of is
Convert this angle to degrees, subtract the measured heading, and wrap the result to :
In Python:
target_angle = math.degrees(math.atan2(uy, ux))
angle_error = target_angle - theta
angle_error = (angle_error + 180) % 360 - 180
Both headings use the downward- image convention and degrees. Wrapping selects the smaller angular difference instead of requesting an unnecessary long turn.
Turning
When exceeds ANGLE_THRESHOLD, the robot turns in place by driving its wheels in opposite directions. We cap the turning command at 40:
where corresponds to KP_ANGLE. The wheel commands are
turn_speed = min(settings.KP_ANGLE * abs(angle_error), 40)
if angle_error > 0:
robot.hamster.wheels(turn_speed, -turn_speed)
else:
robot.hamster.wheels(-turn_speed, turn_speed)
For another robot platform, check its wheel-command signs rather than changing the formation-error equations.
Forward motion and stopping
The movement-vector magnitude is
When this magnitude is at or below DISTANCE_THRESHOLD, the formation-control branch stops the robot. Otherwise, once the heading error is within ANGLE_THRESHOLD, both wheels receive the same forward command:
where corresponds to KP_DISTANCE.
distance = np.linalg.norm(u)
if distance <= settings.DISTANCE_THRESHOLD:
robot.hamster.wheels(0, 0)
elif abs(angle_error) <= settings.ANGLE_THRESHOLD:
forward_speed = min(settings.KP_DISTANCE * distance, 40)
robot.hamster.wheels(forward_speed, forward_speed)
Meaning of the stopping threshold. Despite its name, DISTANCE_THRESHOLD is applied to the control-vector magnitude. It is not a direct measurement of inter-robot clearance, and it does not state that every pairwise error is below the same value. The normal turning and forward command caps are both 40; these are separate from OBS_SPEED.
Reactive obstacle avoidance
Before the formation-motion decisions above, apply_control() checks the left and right proximity sensors. Readings above OBS_THRESHOLD trigger the following reactive response:
left = robot.hamster.left_proximity()
right = robot.hamster.right_proximity()
if left > settings.OBS_THRESHOLD and right > settings.OBS_THRESHOLD:
robot.hamster.wheels(-settings.OBS_SPEED, -settings.OBS_SPEED // 2)
return
if left > settings.OBS_THRESHOLD:
robot.hamster.wheels(settings.OBS_SPEED, -settings.OBS_SPEED)
return
if right > settings.OBS_THRESHOLD:
robot.hamster.wheels(-settings.OBS_SPEED, settings.OBS_SPEED)
return
The response temporarily overrides formation motion. It is basic reactive obstacle avoidance, not collision-free path planning. Once the sensor readings no longer trigger the response, the robot resumes the formation-motion decisions. The missing-marker stop remains outside this routine in the main loop.
Tuning the controller
SHAPE_R controls target spacing, DISTANCE_THRESHOLD controls the stopping condition for , and ANGLE_THRESHOLD controls how closely a robot must face its movement direction before moving forward. A smaller angle threshold requires closer alignment but can cause repeated turns. Begin with low wheel speeds; higher speeds can produce overshooting, collisions, or wheel slippage. Near the formation, reducing speed or increasing the stopping threshold can reduce oscillations.
Main Control Loop
The complete system repeatedly updates robot poses, calculates formation-control vectors, and sends wheel commands. The following excerpts show the main components of main.py.
Robot initialization
The RobotAgent class stores a marker ID, array index, robot connection, fixed formation slot, and the current position and orientation:
from dataclasses import dataclass
import numpy as np
from roboid import *
@dataclass
class RobotAgent:
marker_id: int
robot_i: int
hamster: HamsterS
slot: int
position: np.ndarray | None = None
orientation: float | None = None
ROBOTS = {
marker_id: RobotAgent(
marker_id=marker_id,
robot_i=i,
hamster=HamsterS(i),
slot=i,
)
for i, marker_id in enumerate(settings.target_ids)
}
The dictionary is keyed by marker ID. robot_i indexes the position and control-vector arrays, while slot selects a desired formation position. Position and orientation can be None before the first successful observation.
The main program also imports cv2, settings, update_poses, p_star, d_star, and apply_control from their respective modules in Table 1.
Image capture and the missing-marker branch
The program opens camera index 0 and requests 30 frames per second:
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FPS, 30)
The actual frame rate depends on the camera and computer. Each iteration calls ret, frame = cap.read(). Here, ret indicates whether the frame was read successfully. A failed read exits the loop. After a successful read, update_poses(ROBOTS, frame) updates the observations.
If any required position is missing, the loop stops every robot, displays the camera frame, checks whether the user pressed q, and otherwise skips the formation update:
if any(robot.position is None for robot in ROBOTS.values()):
for robot in ROBOTS.values():
robot.hamster.wheels(0, 0)
cv2.imshow("Camera", frame)
if cv2.waitKey(1) & 0xFF == ord("q"):
break
continue
This behavior depends on clearing poses before processing each frame, as explained in § 5.
Formation calculation and actuation
For a frame with all required poses, populate the position array, construct the desired displacements, calculate the errors, and send each robot its own vector:
positions = np.zeros((settings.ROBOTS_N, 2))
for robot in ROBOTS.values():
positions[robot.robot_i] = robot.position
desired_positions = p_star()
desired_offset = d_star(ROBOTS, desired_positions)
actual_offset = positions[None, :, :] - positions[:, None, :]
displacement_error = actual_offset - desired_offset
u = np.sum(displacement_error, axis=1)
for robot in ROBOTS.values():
apply_control(robot, u[robot.robot_i], robot.orientation)
These calculations implement Eqs. (4)–(11). The wheel-command routine implements the turning, forward-motion, stopping, and proximity decisions in § 7.
Display and cleanup
The main loop displays the camera frame in the OpenCV window and uses cv2.waitKey(1) to detect the quit key. The camera image is not automatically annotated unless marker drawing or the optional overlay is enabled.
Place the loop inside a try block and use a finally block to stop the wheels and release the camera and display resources:
finally:
for robot in ROBOTS.values():
robot.hamster.wheels(0, 0)
cap.release()
cv2.destroyAllWindows()
Running the System
Step 1: Prepare the robots. Attach one unique ArUco marker to each robot. Turn on and connect the HamsterS robots individually in the order corresponding to target_ids, then place them inside the camera view.
Step 2: Position and check the camera. Keep the camera stationary above the operating area. Confirm that all markers are visible, face the camera, and appear in focus under adequate lighting. Before starting formation control, verify that each detected marker ID matches its robot, the reported heading points toward the robot’s front, and a short turn command rotates in the intended direction.
Step 3: Start the program. Run:
python main.py
The camera window should display the live image. Marker outlines, IDs, and vector overlays appear only when the relevant optional drawing calls are enabled. The robots should turn toward their control vectors and move to reduce their formation errors.
Step 4: Stop the system. Press q in the OpenCV window. The cleanup block stops every robot before closing the camera and display.
Expected Behavior and Error Measurement
The robots can begin at different positions and orientations within the operating area. As the loop runs, the camera estimates their poses, the controller computes displacement errors, and the robots turn and drive to reduce those errors. The formation-motion branch stops a robot when its control-vector magnitude is within the threshold, unless the proximity response takes precedence.
With four robots, the expected arrangement is a square around a common formation center, not a single shared pixel. Robot size, camera noise, wheel-speed differences, and the stopping threshold can cause the physical formation to differ from the ideal geometry. Figure 1 shows one experimental run as a qualitative illustration.
Measuring formation error
We measure the mean pairwise formation error as
There are ordered pairs of distinct robots. The metric averages the magnitude of the displacement mismatch over those pairs; a smaller value means a closer match to the desired relative arrangement.
The function global_error() in utils/img_overlay.py calculates this metric, while draw_overlay() provides optional visualization. To record the error, initialize an elapsed-time clock and two lists before the main loop:
import time
from utils.img_overlay import global_error, draw_overlay
error_times, errors = [], []
start_time = time.perf_counter()
After calculating the displacement errors and control vectors in a valid frame, add:
error = global_error(positions, desired_offset)
error_times.append(time.perf_counter() - start_time)
errors.append(error)
draw_overlay(frame, ROBOTS, u, error) # Before cv2.imshow(...)
Only frames in which all required robots are detected contribute samples. With Matplotlib installed, plot the recorded values after the run:
import matplotlib.pyplot as plt
plt.plot(error_times, errors)
plt.xlabel("Time (s)")
plt.ylabel("Mean formation error (px)")
plt.savefig("formation_error_over_time.png", dpi=200)
plt.close()
These lines are optional and are not part of the main loop by default. The measured trace can fluctuate because of motion and sensing noise.
Troubleshooting
Table 2 summarizes common issues and the corresponding checks. Use the same coordinate convention and angular units throughout; changing one sign in isolation can introduce a new inconsistency.
| Symptom | Checks and adjustments |
| Camera does not open | Try another camera index, such as cv2.VideoCapture(1), and check that another program is not using the camera. |
| No markers detected | Check the dictionary, marker IDs, lighting, focus, marker size, and visibility of the complete marker border. |
| Robot moves incorrectly | Check image coordinates, marker mounting direction, wheel signs, and the angular-error calculation. Both measured and target headings must use degrees and the same image frame. |
| Robot turns the wrong way | Check the platform’s left- and right-wheel sign conventions. Adapt the wheel mapping in apply_control() rather than changing the formation-error equations. |
| Robots oscillate near the formation | Reduce wheel speed or increase the stopping threshold. Very tight heading tolerance can also produce repeated turning. |
Table 2. Common implementation issues and troubleshooting checks.
Limitations
Although this tutorial provides a practical introduction to multi-robot formation control, the implementation has several limitations. First, robot positions and formation dimensions are expressed in image pixel coordinates rather than physical units. Without camera calibration and image-to-world coordinate conversion, pixel distances do not necessarily correspond to uniform physical distances across the workspace. Pose estimation also depends on reliable ArUco marker detection and proper marker placement.
Second, the system uses a centralized architecture with a fixed overhead camera. All robot poses are estimated and control commands are computed on a single computer. Consequently, the system requires all robot markers to remain visible; if any required marker is lost, the robots stop until valid observations are restored. Furthermore, the formation geometry and robot-to-slot assignments are predefined, without dynamic reassignment or adaptation of the desired formation.
Finally, the controller uses a simple turn-then-drive strategy with fixed gains and thresholds. Its stopping condition is based on the magnitude of the combined formation-control vector, rather than directly enforcing individual pairwise error bounds. Although proximity sensors provide reactive obstacle avoidance, the implementation does not include collision-free path planning or explicit inter-robot separation constraints. Therefore, reducing formation error does not guarantee collision-free motion or convergence under all operating conditions.
Conclusion
This tutorial presented a step-by-step implementation of vision-based multi-robot formation control using differential-drive robots, an overhead camera, and Python. We introduced the key components of a multi-robot control system, from ArUco-based robot identification and pose estimation to desired formation geometry, relative displacement errors, and differential-drive motion control. By integrating these components into a continuous perception-and-control loop, we demonstrated how mathematical formation-control principles can be translated into coordinated robot behavior.
A primary motivation for developing this tutorial is to help bridge the gap between theoretical concepts and practical implementation in robotics education. While mathematical models and control algorithms provide the foundation for understanding multi-robot coordination, implementing them on physical robots introduces additional challenges involving sensing, coordinate systems, actuation, and system integration. Through an accessible, hands-on example, we aim to help students not only understand how formation control works, but also gain the confidence and practical skills to implement, experiment with, and further explore multi-robot systems.
We hope this tutorial serves as a useful educational resource for students, educators, and researchers who wish to gain hands-on experience in robotics, particularly in multi-robot systems. One of our primary goals is to make practical multi-robot experimentation accessible without requiring expensive robotic platforms or specialized laboratory facilities. The HamsterS robots used in our implementation are relatively affordable, costing approximately US$100 per robot at the time of writing. Moreover, the underlying formation-control principles can be adapted to other differential-drive robots by modifying the platform-specific sensing, communication, and actuation interfaces. We hope this accessibility encourages readers to build their own multi-robot systems, experiment with real robots, and deepen their understanding of robotics through practical implementation.
For readers who do not have access to physical robots, the Robotarium at the Georgia Institute of Technology provides an alternative platform for remotely conducting experiments with real multi-robot systems [24, 25]. Although adapting the implementation to the Robotarium requires using its supported interfaces, it offers an opportunity to explore multi-robot coordination without owning robotic hardware. The accompanying GitHub repository provides the Python implementation, setup instructions, and demonstrations to support independent learning and experimentation. Ultimately, we hope this tutorial helps bridge the gap between theory and practice, inspires readers to explore more advanced problems in multi-robot systems, and encourages them to develop their own ideas and research in robotics.
Acknowledgments
The authors would like to thank the students enrolled in Multi-Robot Systems (CSCI-B 490/590 and ENGR-E 399/599) at Indiana University Bloomington in Spring 2026 for their insightful discussions and valuable feedback. We also thank SMART Lab alumni Dr. Vishnunanand Venkatesh (Astemo Americas, Inc.) and Prof. Wonse Jo (Incheon National University) for their contributions to developing the Hamster robot control system and SMART Table used in SMART Lab technical seminars.
Generative AI tools were used to assist with language editing and figure preparation. All AI-assisted content was carefully reviewed and verified by the authors.
References
- [1]H. Ando, Y. Oasa, I. Suzuki, and M. Yamashita, “Distributed memoryless point convergence algorithm for mobile robots with limited visibility,” IEEE Transactions on Robotics and Automation, vol. 15, no. 5, pp. 818–828, 1999.DOI
- [2]J. Lin, A. S. Morse, and B. D. O. Anderson, “The multi-agent rendezvous problem. Part 1: The synchronous case,” SIAM Journal on Control and Optimization, vol. 46, no. 6, pp. 2096–2119, 2007.DOI
- [3]T. Balch and R. C. Arkin, “Behavior-based formation control for multirobot teams,” IEEE Transactions on Robotics and Automation, vol. 14, no. 6, pp. 926–939, 1998.DOI
- [4]J. P. Desai, J. P. Ostrowski, and V. Kumar, “Modeling and control of formations of nonholonomic mobile robots,” IEEE Transactions on Robotics and Automation, vol. 17, no. 6, pp. 905–908, 2001.DOI
- [5]M. Egerstedt and X. Hu, “Formation constrained multi-agent control,” IEEE Transactions on Robotics and Automation, vol. 17, no. 6, pp. 947–951, 2001.
- [6]K.-K. Oh, M.-C. Park, and H.-S. Ahn, “A survey of multi-agent formation control,” Automatica, vol. 53, pp. 424–440, 2015.DOI
- [7]T. Mina, W. Jo, S. S. Kannan, and B.-C. Min, “Beacon-based distributed structure formation in multi-agent systems,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023, pp. 9277–9284.arxiv.org/abs/2303.00920
- [8]R. Parasuraman, J. Kim, S. Luo, and B.-C. Min, “Multipoint rendezvous in multirobot systems,” IEEE Transactions on Cybernetics, vol. 50, no. 1, pp. 310–323, 2020.
- [9]S. Luo, J. Kim, R. Parasuraman, J. H. Bae, E. T. Matson, and B.-C. Min, “Multi-robot rendezvous based on bearing-aided hierarchical tracking of network topology,” Ad Hoc Networks, vol. 86, pp. 131–143, 2019.DOI
- [10]B.-C. Min, R. Parasuraman, S. Lee, J.-W. Jung, and E. T. Matson, “A directional antenna based leader-follower relay system for end-to-end robot communications,” Robotics and Autonomous Systems, vol. 101, pp. 57–73, 2018.arxiv.org/abs/1711.08007
- [11]R. Parasuraman and B.-C. Min, “Consensus control of distributed robots using direction of arrival of wireless signals,” in Distributed Autonomous Robotic Systems: The 14th International Symposium, ser. Springer Proceedings in Advanced Robotics, N. Correll, M. Schwager, and M. Otte, Eds. Springer, 2019, vol. 9, pp. 17–34, presented at DARS 2018.DOI
- [12]C. Marowski and B.-C. Min, “Formation control tutorial,” GitHub repository, 2026. [Online]. Available: https://github.com/SMARTLab-IU/formation-control-tutorial
- [13]J. Fredslund and M. J. Matarić, “A general algorithm for robot formations using local sensing and minimal communication,” IEEE Transactions on Robotics and Automation, vol. 18, no. 5, pp. 837–846, 2002.DOI
- [14]J. R. T. Lawton, R. W. Beard, and B. J. Young, “A decentralized approach to formation maneuvers,” IEEE Transactions on Robotics and Automation, vol. 19, no. 6, pp. 933–941, 2003.DOI
- [15]S. Garrido-Jurado, R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez, “Automatic generation and detection of highly reliable fiducial markers under occlusion,” Pattern Recognition, vol. 47, no. 6, pp. 2280–2292, 2014.
- [16]OpenCV, “Detection of ArUco markers,” OpenCV 4.x documentation, accessed November 6, 2026. [Online]. Available: https://docs.opencv.org/4.x/d5/dae/tutorial_aruco_detection.html
- [17]C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. Fernández del Río, M. Wiebe, P. Peterson, P. Gérard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, H. Abbasi, C. Gohlke, and T. E. Oliphant, “Array programming with NumPy,” Nature, vol. 585, no. 7825, pp. 357–362, Sept. 2020.
- [18]G. Bradski, “The OpenCV library,” Dr. Dobb’s Journal, 2000.
- [19]K.-H. Park, “roboid: Python package for the ROBOID project,” PyPI package, accessed November 6, 2026. [Online]. Available: https://pypi.org/project/roboid/
- [20]R. Olfati-Saber, J. A. Fax, and R. M. Murray, “Consensus and cooperation in networked multi-agent systems,” Proceedings of the IEEE, vol. 95, no. 1, pp. 215–233, 2007.
- [21]R. Olfati-Saber and R. M. Murray, “Consensus problems in networks of agents with switching topology and time-delays,” IEEE Transactions on Automatic Control, vol. 49, no. 9, pp. 1520–1533, 2004.
- [22]W. Ren and R. W. Beard, “Consensus seeking in multiagent systems under dynamically changing interaction topologies,” IEEE Transactions on Automatic Control, vol. 50, no. 5, pp. 655–661, 2005.
- [23]R. Siegwart, I. R. Nourbakhsh, and D. Scaramuzza, Introduction to Autonomous Mobile Robots, 2nd ed. Cambridge, MA: MIT Press, 2011.
- [24]D. Picken, P. Glotfelter, L. Wang, M. Mote, A. D. Ames, E. Feron, and M. Egerstedt, “The robotarium: A remotely accessible swarm robotics research testbed,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017, pp. 1699–1706.
- [25]S. Wilson, P. Glotfelter, L. Wang, S. Mayya, G. Notomista, M. Mote, and M. Egerstedt, “The robotarium: Globally impactful opportunities, challenges, and lessons learned in remote-access, distributed control of multirobot systems,” IEEE Control Systems Magazine, vol. 40, no. 1, pp. 26–44, Feb. 2020.DOI





