WorksComputer Science
Technical Report of Auriga's 1:10 Scale Autonomous Vehicle Robot
Abstract
This report presents the design and implementation of Auriga Robotics' 1:10 scale autonomous vehicle, developed at Shahid Beheshti University's Robotics and Intelligent Automation Laboratory. The platform integrates a custom mechanical chassis, a distributed embedded system based on ESP32 microcontrollers, and a Jetson Orin NX module for high-level perception and control. The embedded layer manages sensor interfacing, real-time data acquisition, and actuator control, while the Jetson handles environment perception through deep-learning–based lane, sign, and traffic light detection, as well as decision-making and trajectory control. System communication is achieved via structured UART links, ensuring synchronized operation between perception and control subsystems. The vehicle's software stack incorporates ROS, modular firmware design, and multiple control algorithms—Stanley, PID, and Pure Pursuit—to achieve stable autonomous navigation in structured driving scenarios. The overall system demonstrates robust performance and reliability in urban-like environments, serving as a modular research platform for future autonomous driving studies.
Introduction
Autonomous driving represents one of the most advanced integrations of control theory and computer vision, and is of high importance in self-driving cars in the current age. To explore these concepts in a manageable and scalable way, Auriga Robotics at Shahid Beheshti University developed a 1:10 scale autonomous vehicle designed to perform perception, decision-making, and control in real time. The project was carried out as part of the team’s ongoing research in intelligent robotics and autonomous navigation systems, with the goal of building a modular platform suitable for academic experimentation. An image of the robot is attached in Figure 1.

Figure 1. Auriga’s 1:10 scale autonomous vehicle.
The primary objective of the project was to design a complete autonomous driving stack, that includes hardware, embedded firmware, and high-level perception algorithms. The vehicle was required to detect and follow lanes, recognize traffic signs and lights, and maintain stable longitudinal and lateral control under various driving scenarios. To achieve this, the system was divided into three processing layers: low-level sensor integration and actuation was handled by two ESP32 microcontrollers (called CONTROL and ZERO), and high-level perception and decision-making executed on an NVIDIA Jetson Orin NX mini-computer, in which all three units maintained stable communication with each other. On the software side, implemented on the Jetson Orin NX, the architecture was built around ROS (Robot Operating System) [4] to allow modular algorithm development, parallel debugging, and integration of computer vision and control pipelines.
The remainder of this report is organized as follows. § 2 presents the hardware design, including the mechanical structure, sensors, and electronics. § 3 details the embedded and high-level software architecture, covering sensor integration, communication between the processing units, and the Robot Operating System. § 4 discusses the perception and computer-vision algorithms, together with the training strategy and experimental validation of the lane-detection model. § 5 covers vehicle localization, navigation, and the control algorithms used for lane-keeping, decision-making, parking, and intersection handling.
This report describes the 2025 platform of Team Auriga, which builds and improves on our earlier 2024 vehicle [1].
Hardware Description
Mechanical Design
The mechanical structure of the autonomous vehicle was fully custom-designed to balance robustness, modularity, and scalability. It consists of three main parts: the chassis, the steering and drivetrain system, and the vehicle body. Each component was designed and fabricated to ensure reliable operation, efficient integration with electronic systems, and easy maintenance.
Car Chassis. The chassis of the vehicle is designed at a 1:10 scale relative to a full-sized car. It consists of an aluminum base plate and a 3D-printed plastic upper layer. The chassis layout was designed to accommodate the DC motor, servo motor, rotary encoder, electronic control boards, and other system components. The primary motivation behind developing a custom chassis, rather than using a prebuilt model, was to integrate specific mechanical features required for improved performance. These include the addition of a more powerful DC motor for greater torque and speed, an increased ground clearance to allow traversal over small obstacles, and custom mounting points for sensors and electronics. The vehicle dimensions are mentioned in Table 1.
| Dimension | Value (cm) |
| Wheelbase (front axle to rear axle) | |
| Vehicle width | |
| Overall length (front bumper to rear bumper) | |
| Height |
Table 1. Vehicle dimensions.
Steering and Drivetrain System. The vehicle employs a single differential rear-wheel drive system, where propulsion is provided by a DC motor mounted on the rear axle. The steering mechanism follows an Ackermann configuration [25] and is actuated by a servo motor. This arrangement enables the vehicle to achieve appropriate turning angles and efficiently cover a wide range of trajectories.
Car Body. The robot’s body consists of two main parts: the front bumper, which houses the headlights and front distance sensors, and the rear section, which includes the taillights and the camera mount along with the rear distance sensors. Both parts were designed and fabricated using 3D printing to ensure precision fitting and lightweight structure.
Electrical Design
The electrical system of Auriga’s autonomous vehicle is centered around a main control board and three auxiliary side boards. Two of these side boards are positioned at the front and roof of the vehicle, while the third board is dedicated to servo motor and DC motor data acquisition. This separation minimizes high-current noise coupling from the motor driver into sensitive digital communication lines. The side boards facilitate organized routing of sensor wires, ensuring efficient connections and reduced signal interference. Power is supplied by a 5000 mAh 4-cell lithium-polymer (Li-Po) battery, which delivers energy to both the drive and electronic subsystems, maintaining stable operation during extended runs. Voltage regulation is achieved through a combination of switching and linear regulators that maintain all subsystem voltages within safe operational limits. Over-voltage and over-current protection circuits are included to enhance reliability and prevent component damage. Several indicator LEDs are placed around the board to provide visual confirmation of correct voltage levels and regulator status, simplifying debugging and system validation.
Control Board. As shown in Figure 2, the main control board integrates two ESP32 microcontrollers [2]—ZERO and CONTROL—that communicate through a UART link. ZERO MCU handles sensor data acquisition and low-level control tasks, while CONTROL MCU manages high-level communication with the onboard computer (NVIDIA Jetson Orin NX [3]) and actuation, enabling synchronized data exchange and command handling.

(a) Top layer of the control board.

(b) Bottom layer of the control board.
Figure 2. Control board PCB layout showing (a) the top layer and (b) the bottom layer.
Power for the control board and peripherals is provided by both switching and linear regulators: two XL4016 switching regulators generate 12 V and 5 V outputs for the Jetson Orin NX, servo motor, encoder, and sensors, while two AMS1117-3.3 V and one AMS1117-5 V linear regulators provide stable voltage for logic-level components.
DC Motor Driver Board. For DC motor actuation, an L298 dual H-bridge driver [9] is employed, delivering sufficient current capacity for precise speed control. The printed circuit board (PCB) layout and assembly were designed and fabricated by the team, resulting in a compact and reliable architecture that integrates all power and control subsystems efficiently. The PCB design for the driver board is shown in Figure 3.

Figure 3. DC motor driver board PCB layout showing the top (left) and bottom (right) layers.
Sensors. Range measurement is handled by VL53L1X time-of-flight (ToF) laser-ranging sensors [6] mounted on the front, rear, and sides of the vehicle, with a total of 12 ToF sensors. To avoid I²C [10] interference across these sensors, they are all multiplexed and connected to the ZERO MCU. These sensors provide high-speed and accurate distance measurements for obstacle detection, complementing camera-based perception for enhanced situational awareness.
Orientation is measured by a single compass module—the CMPS14 [8]—which provides a tilt-compensated heading for real-time localization. An MPU-9250 [7] was developed and tested separately on a breadboard and was not used on the robot. The CMPS14 is read by the ZERO MCU via the I²C protocol.
The DC motor includes an optical rotary encoder that provides wheel speed and position feedback, enabling precise closed-loop control of longitudinal motion.
To enhance the visual feedback of the autonomous vehicle and provide a more intuitive representation of its actions, a string of WS2812B Neopixel LEDs is integrated into the system. These LEDs are used to indicate various behaviors of the vehicle, such as turning left or right, scanning the surrounding environment, and detecting obstacles.
Embedded Design, High-Level Software Architecture, and Robot Operating System (ROS)
To achieve the required computational performance and maintain synchronized control across all subsystems, the vehicle’s architecture incorporates three distinct processing units: two ESP32-based embedded controllers, named ZERO and CONTROL, and an NVIDIA Jetson Orin NX module. Each unit is assigned specific responsibilities within the perception–control hierarchy to ensure modularity, determinism, and real-time responsiveness. The Jetson Orin NX serves as the high-level computing platform, running perception algorithms, decision-making routines, and overall control logic. The CONTROL ESP32 functions as the intermediary layer, managing actuator commands, multiplexing, and structured communication with both the Jetson and the ZERO module. The ZERO ESP32 handles all low-level sensor acquisition tasks, including ToF, compass, and encoder data collection, before transmitting the processed data to the CONTROL unit for integration and forwarding to the Jetson.
| Unit | Processor | Primary Function |
| Jetson Orin NX | ARM Cortex-A78AE + GPU | Perception and control logic |
| CONTROL ESP32 | Xtensa LX6 Dual-Core | Intermediary communication layer, actuator interfacing |
| ZERO ESP32 | Xtensa LX6 Dual-Core | Sensor acquisition, low-level actuation |
Table 2. Overview of the three processing units and their primary responsibilities.
Embedded System Software Architecture
The embedded subsystem forms the low-level operational layer of Auriga’s autonomous vehicle. It is responsible for acquiring sensor data, executing actuator commands, and maintaining synchronized communication between all processing units. The subsystem consists of two ESP32 microcontrollers, referred to as CONTROL and ZERO, that cooperate with the Jetson Orin NX computer. The CONTROL unit interfaces directly with the Jetson, relaying processed sensor data from ZERO and applying actuator commands such as steering and motor control. The ZERO unit manages all time-critical sensor measurements and DC motor control, providing a continuous data stream to CONTROL over UART. Together, these two microcontrollers form a distributed real-time embedded network bridging the vehicle’s physical layer and the Jetson’s high-level control layer.
Firmware Architecture
Both ESP32 units run a FreeRTOS-based firmware [5] divided into concurrent tasks, each handling an independent subsystem. On the ZERO module, tasks include ToF sensor polling, compass and encoder sampling, and UART data transmission. CONTROL executes complementary tasks for UART reception, servo actuation, lighting and signaling control (NeoPixels and buzzer), and I²C multiplexer channel selection. Inter-task communication relies on FreeRTOS queues to ensure deterministic timing and non-blocking operation. Both cores of each ESP32 are utilized, dedicating time-critical acquisition and actuation tasks to one core while communication and housekeeping tasks run on the other. Figure 4 illustrates the steady-state organization of dataflow and task scheduling among CONTROL, ZERO, and Jetson. Task core numbers have been labeled in the form TASK-{core#}/task_name, and every arrow from one node to the other represents the data being transmitted, and the direction of transmission. The Robot Operating System architecture for perception and control has been abstracted away as perception pipeline and control in the figure, and will be covered in depth in § 3.2.

Figure 4. Embedded firmware steady-state architecture.
| MCU | Task | Core | Prio. | Function / period |
| ZERO | Sensor read and transmit | 0 | 10 | Reads all sensor data, builds and sends packet p1. |
| ZERO | Instruction receiver | 0 | 10 | Parses p3 (motor command, ToF bitmask); 1-tick polling |
| ZERO | ToF reader | 1 | 13 | Sweeps the 12 ToF sensors every ms; pushes results to a queue |
| ZERO | Button monitor | 1 | 15 | Debounces the motor cut-off button (ISR-flagged) (this task was not included in Figure 4 but is implemented). |
| CONTROL | ZERO→Jetson forwarder | 0 | 10 | Receives p1 from ZERO, checks its integrity and forwards it to Jetson |
| CONTROL | NeoPixel handler | 1 | 10 | Applies the light mode; 5-tick polling, 500 ms blink toggle |
| CONTROL | Jetson receiver | 1 | 14 | Validates p2, drives the servo and queues NeoPixel data to NeoPixel handler task, relays p3 to ZERO |
Table 3. FreeRTOS tasks on the two ESP32 units (priority: higher runs first).
Three queues carry data between tasks (ToF values, ToF bitmask, light mode), and a mutex serialises access to the shared I²C bus between the ToF reader and the compass read. Time-critical events are handled in interrupt service routines placed in IRAM: encoder edges (ZERO), the Jetson over-current signal (ZERO), the motor cut-off button (ZERO), and the ToF multiplexer step (CONTROL).
Sensor and Actuator Integration
As shown in Figure 4, during steady-state operation the embedded layer continuously acquires and distributes data from fifteen data channels: twelve VL53L1X time-of-flight (ToF) sensors arranged around the vehicle perimeter, one rotary encoder for wheel speed feedback, a CMPS14 tilt-compensated magnetic compass, and the time elapsed since Jetson handshake was made.
To prevent IC address conflicts, the VL53L1X modules are multiplexed through a hardware selector controlled by the CONTROL ESP32. The ZERO ESP32 requests channel selection and sequentially samples the active sensor, packaging the range data for transmission. The CMPS14 provides a direct heading measurement, while the encoder delivers rotational velocity used for odometry and closed-loop motor control.
All twelve VL53L1X sensors share the default IC address, so only one is connected to the bus at a time. After each reading, ZERO sends a 100 µs pulse on a dedicated line; an interrupt on CONTROL advances a channel counter (modulo 12) and drives the four multiplexer select lines. The sensors run in long-distance mode with a 33 ms timing budget and a 40 ms inter-measurement period, giving a full-sweep of about half a second. Because packets are sent faster than sweeps complete, the last ToF values are repeated until a new sweep arrives. The Jetson can disable individual sensors through a 12-bit bitmask in p2; disabled or failed sensors are reported as −1.
The rotary encoder is decoded in x4 mode: an interrupt on every edge of both channels reads both pin levels and updates a signed position counter, giving 1600 counts per revolution. Speed is obtained from the count difference between consecutive packets and the measured time step.
The steering servo is driven at 50 Hz with a pulse width of 1000–1815 µs, mapped linearly from the 8-bit command (3.2 µs per step, using a 16-bit PWM timer). The DC motor driver receives an enable line, a direction line, and a 500 Hz, 10-bit PWM signal; the sign of the motor command sets the direction.
All sensor outputs are collected and structured into a unified serial data packet, transmitted from ZERO to CONTROL via UART. CONTROL verifies packet integrity, forwards the data to the Jetson Orin NX, and awaits actuator commands. These commands are returned to CONTROL, containing servo steering angles, motor drive values, and LED states. CONTROL applies its own actuator commands locally and relays motor control commands to ZERO, which drives the DC motor accordingly. The details of packet structure and synchronization protocol are discussed in section 3.1.5.
Sensor Fusion for Heading Estimation
Note: the MPU-9250 was not used on the 2025 vehicle. Due to integration deadlines, the robot used only the CMPS14 heading during live track testing. The sensor fusion described below was developed and validated offline on a breadboard after the laboratory closures, is independent of the robot platform, and is the intended architecture for future platforms.
Accurate heading estimation is required for stable vehicle control and localization. The test platform was designed to use two complementary modules: an MPU-9250 IMU (3-axis accelerometer, 3-axis gyroscope, 3-axis magnetometer) and a CMPS14 tilt-compensated compass (BNO080-based) which provides a fused, tilt-compensated heading output. The relevant module specifications and internal fusion capability are documented in the manufacturers’ datasheets.
A two-stage fusion pipeline was implemented: (1) internal Kalman filtering [11, 12] on MPU-9250 measurements to produce a drift-compensated yaw estimate and (2) a rate-dependent blend between and the CMPS14 heading .
Stage 1 — Heading estimation using the MPU-9250 data. The MPU-9250 sensor only provides raw vector readings for the relative acceleration , magnetic field strength , and the angular velocity . Here, we combine the three to get an estimate on , the heading angle.
Definitions. Define the world orthonormal vector basis , where points towards the north magnetic pole, points to the magnetic west, and points upwards from the horizontal plane. This basis is universal and fixed. Parallel to the fixed basis, define the time-varying body orthonormal basis , where points to the front of the vehicle, points to the left of the vehicle, and points upwards with respect to the vehicle. The basis system is shown in Figure 5.

Figure 5. World and body coordinate systems.
For any pure vector , let the representation in the world and body coordinates be and respectively. These two representations are related to each other with a rotation matrix , such that
Where encodes the full orientation of the robot. Here, since the robot only moves in the horizontal plane, we only care about the heading angle and a full orientation state of the robot is not computed. The heading angle is defined as the angle between the horizontal projection (projection onto the plane spanned by ) of and the magnetic north axis , measured clockwise about :
Additionally, we define the operators for projecting any vector to the horizontal plane in both coordinate systems:
Where is the identity matrix. Therefore for any vector , let be the vector projected onto the horizontal plane.
Accelerometer. The accelerometer on the MPU-9250 measures the relative acceleration vector in the body coordinate (the subscript denotes the measured result: the final value the sensor gives), where is the gravitational acceleration. Therefore, the ideal readings are of the form
For the stationary robot, from Eq. (4) we get . The real acceleration vector is fast-changing in time, hence for the discretized model of the accelerometer , a low-pass filter is implemented to arrive at an estimate of in the form
The value was chosen for this platform. From Eq. (5), we get an estimate of the universal up direction , expressed in the body coordinates (Eq. (6)).
From this, we can find the horizontal plane in the body coordinates. Define the forward direction unit vector , which is the projection of the body front unit vector onto the horizontal plane, calculated as
and the left direction unit vector in the horizontal plane is then simply obtained by
Gyroscope. The gyroscope measures the angular velocity vector in the body coordinates. The heading rate change is
Projecting onto the estimated vertical rather than the board’s up axis () keeps the yaw rate correct when the vehicle pitches or rolls.
The chip gives a 16-bit integer per direction. The conversion factor, specified in the datasheet, is LSB/(/s). Discretizing Eq. (9), the gyroscope output is the noisy, biased measurement
with slowly varying bias and white noise . A startup average of 200 stationary samples is subtracted, giving as the measured angular velocity. The measured heading rate is therefore
for small near the robot startup (the update rule for is discussed in Stage 2). Here, comes from Eq. (6). Integrating with the measured step (about 10 ms, calculated in each iteration) gives
Where maps any angle into the principal interval . Hence, the gyroscope gives a final noisy and biased estimate .
Magnetometer. The magnetometer sensor is useful for measuring Earth’s magnetic field, which provides a universally fixed coordinate system. The magnetometer measurement model is
where is the Earth’s magnetic field, is the magnetic offset vector, and is the magnetic field distortion matrix. The methods required to extract from are discussed in Stage 4. For the ideal magnetometer with , we can get a direct estimate on the heading angle.
Decomposing Earth’s magnetic field as (Eq. (3)), we have
and since , we get
This is the equation for tilt-compensation. Since, by definition,
and by definition for the heading angle, since
we conclude that the heading angle is given by
where . Hence, we get a direct estimate on the heading angle .
Stage 2 — Kalman filtering. Integrating the measured heading rate alone (Eq. (12)) is dead reckoning: the white noise accumulates as a random walk in , and any bias left over after subtracting accumulates as a linear drift. The magnetometer heading (Eq. (18)) has the opposite property: it does not drift, but it is noisy and is disturbed by steel and motor currents. Since the two errors are complementary, the two heading sources are fused with a Kalman filter [11, 12, 13], which is closely related to a complementary filter, except that the gain is computed from the noise statistics instead of being fixed.
State and model. The state is
where is the residual heading-rate bias that remains after the startup average has been removed in Eq. (11). The true heading rate is therefore , where is the measured heading rate of Eq. (11). The measured acts as a known input to the model, and the residual bias is modeled as a random walk:
These correspond to the continuous-time update rules of and . The bias is only affected by noise in this model. The first row of Eq. (20) reads , which is Eq. (12) with the bias removed, and the second row says stays constant between steps. The process noise is , where accounts for the gyroscope noise and integration error in , and is the rate at which the bias wanders. The magnetometer heading is the measurement:
Here is Eq. (18) evaluated with the calibrated magnetic vector of Stage 4 in place of . The accelerometer is neither an input nor a measurement of the filter. It only provides (Eq. (6)), which is used in and in the tilt compensation of .
From here, for any quantity (vector or not) let be the quantity estimated at step using the data up to (the prediction section), and let be the updated prediction after acquiring the magnetometer heading estimate, . Furthermore, if is the true reading, define as the error in the quantity .
Prediction. Let be the covariance matrix. The prediction runs once per gyroscope sample, whether or not a magnetometer sample is available:
followed by . Here, . Without magnetometer samples this reduces to dead reckoning, and grows. This is fixed when calculating in the update section.
Update. When a valid magnetometer heading is available, the innovation
is calculated as the difference between the predicted value and the directly measured value. Then, the update algorithm updates the estimated values as below.
The innovation uses the signed wrap so that 359° 1° counts as +2° instead of −358°. The gain splits the correction between the two states according to their uncertainties: a large makes the filter trust the gyroscope, and a large makes it trust the magnetometer. Afterwards, is wrapped to and is symmetrized, to avoid asymmetry in single-precision arithmetic.
observes only , but the pair is observable, since
has rank two. In practice, if the gyroscope has a residual bias, the innovation keeps the same sign. The cross-covariance links this to , so the second gain component moves until the innovation averages to zero. The bias is therefore observable only through the magnetometer.
Adaptive measurement noise. The Earth’s field strength is constant at a fixed location, so a measured strength that deviates from indicates a disturbance (motors, nearby steel). Define
where is the horizontal part of the calibrated field (in the 3D calibration mode of Stage 4, the full vector is used instead). A disturbed field therefore makes the magnetometer less trusted, and if the sample is discarded and only the prediction step is run.
Initialization. The first call of the update loop only stores a time reference, and is measured from a hardware timer at every iteration and limited to 0.2 s. Before the first accepted magnetometer sample, the heading is unknown. At that sample, the filter is initialized as , , , and . The same re-initialization occurs if the magnetometer calibration becomes valid for the first time or becomes invalid. If the startup average was rejected because the vehicle moved during the measurement (spread larger than 3°/s), it is set to zero, and then has to absorb the full gyroscope bias.
| Parameter | Meaning | Value |
| heading process noise | ||
| residual-bias random-walk noise | ||
| magnetometer heading noise | (≈ 3°) | |
| field-deviation scale in Eq. (29) | 0.1 | |
| before first update | initial heading variance | |
| initial residual-bias variance |
Table 4. Kalman filter parameters.
Approximating the steady state with the continuous-time algebraic Riccati equation, with the measurement noise intensity for ms, gives a heading standard deviation of about 0.26°, a residual-bias standard deviation of about 0.06°/s and a heading gain per update. The heading therefore converges to the magnetometer with a time constant of about s, while the gyroscope carries the heading between updates.
Stage 3 — Rate-dependent CMPS14/MPU blend. The CMPS14 provides a direct tilt-compensated heading . We expected this sensor to drift considerably more at standstill than the Kalman-filtered MPU-9250; our tests did not confirm this (§ 3.1.4). To exploit the fast reaction of the onboard fusion while retaining the low-noise, bias-corrected estimate from the MPU Kalman filter at low dynamics, we blend the two headings with a scalar weight that is a monotonic function of the yaw rate magnitude:
The weight is implemented as a smooth piecewise linear function in our baseline:
Thresholds are design parameters and must be tuned to the vehicle dynamics; the angular velocity thresholds were empirically selected for our platform. The two sensors do not share the same zero reference, so a constant offset measured with the board at rest is subtracted from , and the difference is wrapped to before blending. In our tests the blend did not measurably reduce stationary drift compared with using the CMPS14 alone (§ 3.1.4).
Stage 4 — Magnetometer calibration and startup. So far the magnetometer was assumed ideal. In reality the measured vector follows Eq. (13), , where (soft iron: scaling, cross-axis coupling) and (hard iron: constant offset) depend on the vehicle’s wiring, batteries and mounting. They are therefore estimated on the vehicle itself, from the sensor readings only [14, 15].
Raw data. The AK8963 magnetometer is read at 100 Hz in 16-bit continuous mode through the IC bypass of the MPU-9250. Each chip axis is scaled as (0.15 T), where is a factory sensitivity adjustment read once from the chip’s fuse ROM [18]. The chip axes are not aligned with the gyroscope and accelerometer (its and are swapped and is inverted), so .
Earth’s magnetic field. At a fixed location is constant, so if the vehicle is rotated, traces a sphere of radius in the body frame. Eq. (13) turns this sphere into an ellipsoid: the measured vectors satisfy
so is the centre of the ellipsoid, and describes its shape. The calibration fits this ellipsoid to recorded samples and then maps it back to a sphere:
where is the symmetric square root of . The product is orthogonal, so is the Earth’s field up to a rotation, with . This is the vector used in Eq. (18) and Eq. (29). Two limits follow. First, is the mean size of the fitted ellipsoid (the geometric mean of its semi-axes), not an independent measurement of the local field. Second, an ellipsoid fit cannot observe a rotation of the magnetometer axes relative to the body frame, which would appear as a constant heading error.
Least-squares fit. Let , , be the stored samples with centroid . They are centred and scaled, with T. Centring puts the origin inside the ellipsoid, and scaling keeps the equations well conditioned. Writing the ellipsoid as , the unknowns enter linearly,
and are obtained from the regularized normal equations with , solved by Gaussian elimination with partial pivoting [17]. Completing the square gives the centre and the shape matrix with . The hard-iron vector follows from . With the eigendecomposition (Jacobi method [17]),
This is an unconstrained algebraic fit, as opposed to the ellipse-specific constrained fit of [16], so the validity of the result is enforced by the acceptance tests below. The computation is done in double precision (software-emulated on the ESP32), which is why it runs in a separate low-priority task.
2D and 3D modes. A ground vehicle only yaws, so the component of the field is not excited and cannot be calibrated. By default, only the – components are fitted, with the five unknowns , , , , and . In this mode, has no coupling with and , the component of keeps its previous value, and the field strength in Eq. (29) is evaluated on the horizontal part only. The full 3D fit takes over when at least 80 samples are stored and their span is at least half of the mean – span, which only happens if the vehicle has been tilted. In the 2D mode, an error in the offset leaks into the tilt-compensated heading and grows with the tilt angle, so it is small as long as the vehicle stays level.
On-the-fly operation.
Every valid sample is stored in a 128-sample circular buffer only if it lies farther than from all stored samples. A stationary vehicle therefore adds nothing, and the oldest samples are replaced when the buffer is full.
Once per second a background task copies the buffer and attempts a fit, provided there are at least 30 samples and they cover at least 11 of 12 equal angular sectors of the – plane around their centroid (the vehicle has turned nearly a full circle).
The fit is rejected if is not positive definite, if the ratio of the largest to the smallest semi-axis exceeds 2, if , or if the RMS relative error
exceeds 6% (horizontal norm in the 2D mode).
A new fit replaces the current calibration only if its on the current buffer is lower than that of the current one. It is then blended in, , so that the heading does not jump. The result is handed to the filter through a mutex-protected buffer and is picked up at the next iteration.
Startup. At power-up, the MPU-9250 is reset and identified, the AK8963 is enabled through the bypass mode and its ASAI values are read, the gyroscope bias is averaged over 200 samples (about 1 s, with the vehicle standing still), and the calibration task is started. The vehicle must then be rotated in place by about one full turn, so that the sector-coverage test is passed. Until the first valid calibration exists, no magnetometer update can be made and the heading is flagged invalid. At that moment the Kalman filter is initialized from the first accepted (Stage 2). From then on, the calibration keeps refining itself while the vehicle drives. The firmware can also be seeded with a stored calibration.
Limitations. The calibration is static: field changes that depend on the motor current are not modeled and are only handled by the field-strength gate of Eq. (29). The heading is relative to magnetic north, which is distorted by indoor steel and varies between locations. If the vehicle is relocated, the calibration re-adapts as old samples are replaced in the buffer. Furthermore, in preliminary unlogged tests the 3D mode appeared to output much higher bias than the 2D mode, which we suspect is contributed to by the inaccuracy of the up direction estimation in Eq. (6).
Experimental Validation of the Heading Estimator
The estimator was tested on a breadboard rather than on the vehicle, with an MPU-9250 and a CMPS14 taped side by side on one board and logged over UART at 20 Hz. All processing was done offline. No external angle reference was available, so the experiments measure agreement between the two sensors and stability at rest, not absolute accuracy. The alignment offset between the two sensors was not captured on-line in the reported run. It was instead computed offline as the circular mean of the CMPS14–MPU difference over the first 23 min of stationary data (about , with a peak-to-peak variation of about ), and the aligned CMPS14 heading and the blended heading were recomputed from it. This is exact for the blended heading, because the blend weight depends only on the MPU yaw rate.
Figure 6 shows the magnetometer samples held in the calibration buffer for the three conditions tested, reconstructed offline from the logged raw samples with the same storage rule as the firmware (128-sample circular buffer, minimum spacing of ). The fit error and field radius of the last accepted calibration of each segment are given in the panel titles. Without magnets the raw samples form an offset ring, and the corrected samples are centred on the origin. The two magnet conditions are shown in Figure 7. The weak magnets leave the horizontal field close to the undisturbed value of about , whereas the strong magnets raise it to about . The corrected weak-magnet samples did not form a circle, while those of the other two cases did. The reason is not yet known.

Figure 6. Magnetometer calibration buffers. Blue: raw samples; orange: samples corrected with the last accepted calibration of each segment; the black cross marks the fitted hard-iron centre.

Figure 7. (a) Weak magnets. (b) Strong magnets. Breadboard setup of the two magnetic disturbance tests.
Figure 8 shows 23 min with the board untouched (the last minutes of the recording, when the operator returned to the desk, are excluded). The MPU Kalman heading had a slope of /min, a standard deviation of and a peak-to-peak variation of . The CMPS14 had a slope of /min and a peak-to-peak variation of , which is its output resolution. At rest the blend weight is zero, so the blended heading is identical to the MPU heading. Our design assumed that the CMPS14 drifts considerably more than the filtered MPU at standstill, and the blend gives the MPU full weight at low rates for that reason. This run does not support the assumption. In a second 14 min run (during which a cable was briefly touched) the CMPS14 again varied by against about for the MPU. In both recordings the CMPS14 was at least as stable as the MPU, so the blend gave no measurable benefit at rest. A further 30 min run showed about of CMPS14 drift, so CMPS14 stability seems to vary between runs. The measured MPU standard deviation is of the same order as the predicted in Stage 2, although the two quantities are not strictly the same (the prediction is an estimation error with respect to the true heading, which was not measured here).

Figure 8. Stationary heading change of the CMPS14, the MPU Kalman estimate and the blended heading over 23 min. The blended heading coincides with the MPU estimate because the blend weight is zero at rest.
Figure 9 shows a 29 s rotation by hand with yaw rates up to about /s, using blend thresholds of /s and /s. After alignment the two headings stay within about of each other and mostly within , and the difference drifts slowly toward . The blend weight follows the rate as designed, so the blended heading follows the MPU at low rates and the CMPS14 at high rates. Without a ground-truth angle we cannot say which sensor is closer to the true heading, only that they agree. The experiment therefore does not show that the blend improves on either sensor alone.

Figure 9. Manual rotation: aligned CMPS14 and MPU headings, blended heading, their difference, the yaw rate, and the blend weight .
The magnet tests are summarised in Figure 10. Placing the magnets near the board leaves the MPU heading uncorrected until the calibration is refitted: in both cases the CMPS14–MPU difference grew almost linearly, by about – in 35–45 s () while the board stood still, and returned to a few degrees as soon as a new calibration became valid after a manual reset. We believe the magnetometer samples were being rejected by the field-strength gate during these intervals, so the filter was integrating the gyroscope alone, but we did not verify this directly. With the strong magnets and a new calibration the MPU held its heading within about over 22 s at rest and followed the CMPS14 within a few degrees during rotation. A plausible reason is that magnets fixed to the board act as an additional constant hard-iron offset, which the calibration removes. The calibration does not recover by itself, since the old buffer still holds samples from the undisturbed field and a new fit is accepted only after a full rotation. In the present implementation a lasting disturbance therefore requires a manual reset.

Figure 10. Horizontal field strength (top) and CMPS14–MPU difference (bottom) across the magnet tests. The vertical lines mark the start of the weak-magnet test, the start of the strong-magnet phase (the magnets were introduced about 45 s later), the strong-magnet stationary test and the strong-magnet rotation.
These results come from a single board and a small number of runs, without an external reference, and only the planar (2D) calibration mode was exercised. They should be read as a first check of the implementation and not as a characterisation of its accuracy.
Communication Protocol and Data Packet Structure
Data transfer between processing units is implemented over two UART channels, following a structured packet-based protocol. ZERO periodically sends data packets (p1) containing ToF, compass, and encoder readings to CONTROL, which checks packet integrity (using checksum methods) and forwards them as packet p1 to Jetson. CONTROL also receives control and lighting commands (p2) from Jetson, which upon arrival extracts data, actuates NeoPixel and Servo Motor directly, and sends DC motor speed as packet p3 to ZERO.
Each communication packet follows a fixed structure designed for robustness and low latency. The format is summarized as:
where $ is the start byte (synchronization marker), are the data fields separated by !, chk8 is an 8-bit additive checksum for data integrity, # is the end byte (termination marker), and . denotes padding characters used to maintain fixed payload length.
This structure ensures byte-level synchronization and error detection even under variable serial latency conditions. Message acknowledgment is intentionally minimized to reduce round-trip delay, while periodic redundancy provides fault tolerance.
Both UART links use 115200 baud, 8N1. Table 5 lists the packets. A receiver waits for the start byte, reads a fixed-length frame, and then flushes its RX buffer, so only the most recent frame is processed. The integrity field is an 8-bit additive checksum (sum of the numeric fields modulo 256). The sensor packet p1 is verified on the Jetson; p2 and p3 are verified by their receivers (CONTROL and ZERO).
| Packet | Direction | Length | Wire time | Fields |
| p1 | ZERO→CONTROL→Jetson | 85 B | 7.4 ms/hop | 12 ToF distances (mm), compass bearing (0.1°), encoder count, speed, motor-enable flag, time elapsed, checksum |
| p2 | Jetson→CONTROL | 25 B | 2.2 ms | steering command (0–255), buzzer, motor command (), light mode, ToF bitmask, checksum |
| p3 | CONTROL→ZERO | 18 B | 1.6 ms | motor command, ToF bitmask, checksum |
Table 5. Packet formats. Wire time is for a fixed-length frame at 115200 baud, 8N1 (10 bits per byte).
Forward latency (time passed between a ZERO packet transmission and a Jetson packet reception) was measured to have mean 18.2 ms with standard deviation of a 4.0 ms. The backward latency (time passed between a Jetson packet transmission and a ZERO packet reception) was shorter, owing to the smaller packet size. The mean was measured to be around 9.2 ms, with standard deviation of 3.4 ms. These measurements were performed by simultaneously hardware interrupting all three devices, and setting the interrupted time to zero, and continuously polling data transmission and reception time across the three devices.
Startup Sequences, Edge Cases, and Fault Protection. At startup, CONTROL initiates a handshake with ZERO to verify connectivity and sensor readiness before enabling motor actuation. All sensors are sequentially initialized and validated before control commands are accepted. Emergency stop and fail-safe routines are integrated into CONTROL’s firmware: if serial communication from Jetson is lost for more than 500 ms, or if a fault flag is raised by ZERO, CONTROL immediately halts motor drive signals and centers the steering servo. These measures ensure safe degradation of operation during transient or fault conditions.
Robot Operating System (ROS) Architecture
The autonomous vehicle software stack is implemented using the Robot Operating System (ROS) Noetic [4], running on Ubuntu 20.04 LTS on the Jetson Orin NX onboard computer. ROS provides a modular, distributed middleware that enables perception, decision-making, and actuation to run in parallel while exchanging data through topics, shared memory, and global parameters.
System Overview
The system consists of seven ROS nodes: Camera, Sign, Network, Postprocess, Crosswalk, Control, and Listener. Each node executes as an independent process but participates in a unified computation graph, visualized in Figure 11. Large data objects such as RGB and depth frames are communicated through shared memory to minimize copying overhead, while status, events, and processed features are sent through ROS topics and parameters.

Figure 11. Graph of ROS nodes and topic-level publish–subscribe relationships.
| Node | Inputs | Primary Function / Outputs |
| Camera | Intel RealSense D455 | Captures synchronized RGB and depth frames at and writes them to shared memory. Publishes a startup-ready parameter after the first valid frame. |
| Sign | Shared memory frames | Performs traffic sign detection. Publishes sign_topic (sign type and distance) and cross_input to trigger crosswalk processing. |
| Network | Shared memory RGB frame | Runs lane segmentation model and writes a tensor to shared memory. |
| Postprocess | Network output + RGB frame | Extracts lane offset and heading angle, published on line_topic. |
| Crosswalk | RGB frame + cross_input | Computes crosswalk angle and position, and receives the intersection parameter from Control. |
| Control | Perception topics + uart_topic | Computes steering, throttle, and mode, publishing serial data to the Listener. |
| Listener | chat topic | Converts control outputs to serial packets and republishes hardware feedback on uart_topic. |
Table 6. Summary of ROS nodes and their functions
Node-Level Data Flow
The Camera node initializes the Intel D455 at , the operating point providing the highest depth accuracy. It writes RGB and depth frames directly to shared memory, and after the first valid capture sets a ROS parameter indicating that dependent nodes may start processing. This prevents premature access to uninitialized memory.
The Sign node reads the current frames from shared memory and generates two outputs: (1) sign type and distance on sign_topic, and (2) a binary crosswalk trigger on cross_input.
The Network node receives the latest RGB frame and performs neural inference, producing a feature tensor stored in shared memory for downstream use.
The Postprocess node extracts lane geometry from the tensor: lateral offset and heading angle. These are published to line_topic, consumed by the Control layer.
The Crosswalk node uses shared memory frames and cross_input to compute crosswalk angle and position. Additionally, it receives a runtime parameter named intersection from the Control node indicating whether the vehicle is navigating a junction.
The Control node subscribes to line_topic, sign_topic, crosswalk outputs, and hardware feedback from uart_topic. It computes high-level actuation commands, which are published as chat messages.
The Listener node interfaces directly with the microcontrollers, translating chat into serial commands and publishing returning sensor feedback back onto uart_topic. The Listener node waits for the camera-ready parameter, opens the Jetson UART (/dev/ttyTHS0, 115200 baud), and sends the latest command packet to CONTROL on every loop iteration after clamping each field to its valid range. Incoming p1 packets (§ 3.1.5) are checked against their checksum and republished as 17 integers on uart_topic; if corrupted bytes are detected, the port is closed, reopened, and flushed.
The shared-memory relationships are shown in Figure 12.

Figure 12. Shared memory and data exchange structure among ROS nodes.
Launch Configuration and Fault Handling
All nodes are executed through a unified launch file in launch/. The configuration defines output behavior, restart rules, and global parameter initialization.
Because the Camera node allocates the shared memory region, if it restarts, the memory pointer changes. To avoid inconsistent pointers, the system is configured so that a Camera crash triggers a complete system restart, while crashes in other nodes cause only those nodes to restart individually.
Global shared parameters (calibration constants, visualization toggles, and processing modes) are propagated through the same launch configuration and are accessible system-wide, as shown in Figure 13.

Figure 13. Global parameter exchange for cross-node synchronization.
Timing, Synchronization, and Performance
Real-time operation requires matching node execution rates. The lane network (Network) runs at approximately 13 Hz and is therefore the bottleneck for downstream modules. Nodes such as Postprocess and Control can reach 30 Hz, but first verify whether new data is available before computing. If the network output has not changed, they defer processing, preventing redundant work and improving CPU efficiency.
Conversely, the Camera must operate faster than all dependent nodes to minimize data age and reduce perception-to-actuation latency.
This combined scheduling strategy produces deterministic timing, low redundant computation, and predictable behavior across the full perception and control pipeline.
Computer Vision and Perception
The perception stack serves as the primary sensory input for the autonomous vehicle, mapping the drivable area and enabling adherence to traffic rules. We designed a custom vision pipeline centered around a modified Point Instance Network (PINet) [20], optimized specifically for the sharp curvature and unique lighting conditions of the competition track. The stack is entirely vision-driven: a single Intel RealSense D455 provides synchronized RGB and depth, and three cooperating perception paths—lane detection, traffic sign and signal recognition, and crosswalk detection—run concurrently and publish their state to the controller through a zero-copy shared-memory bus.
This section is organized to follow a single camera frame through the full pipeline. We first describe acquisition, then the lane-detection network architecture and its mathematical training objective, the deterministic geometric post-processing that converts network outputs into physical control variables, the sign and crosswalk perception paths, the inter-process communication layer that binds them, and finally the experimental validation of the trained model. Throughout, we emphasize the engineering rationale behind each design decision, since the dominant constraint—real-time inference on an embedded Jetson Orin at 30 FPS—shaped nearly every choice in the stack.
Acquisition System
Visual data is acquired via an Intel RealSense D455 depth camera [19], selected for its global-shutter RGB sensor and its wide-baseline active stereo depth module. The global shutter is a deliberate choice for a moving platform: at competition speeds a rolling-shutter sensor introduces skew artifacts on vertical lane markings during yaw motion, which would corrupt the sub-pixel key point regression discussed in § 4.3. The camera streams synchronized RGB and depth frames at 30 FPS, with the depth stream hardware-aligned to the color optical frame so that any RGB pixel can be dereferenced directly into a metric depth value. This alignment is what later enables the depth-gated sign validation of § 4.7 without a separate extrinsic calibration step.
Unlike traditional pipelines that crop the input to a Region of Interest (ROI), we process the full sensor resolution (resized to the network input of ). We deliberately avoided ROI cropping to maintain the maximum possible field of view. This ensures the network can utilize peripheral visual cues—essential for detecting the entry points of sharp hairpin turns—which would otherwise be discarded by aggressive cropping. The trade-off is a larger input tensor and therefore a heavier front-end; we absorb this cost in the aggressive stride-2 stem described in Table 7 rather than by discarding field of view.
| Block Name | Type | Input Res. | Out Ch. | Engineering Rationale |
| Stem | Conv2D + PReLU | 64 | Aggressive downsampling to reduce memory footprint. | |
| HG-Block 1 | Standard Hourglass | 128 | Initial coarse feature extraction. | |
| HG-Block 2 | Standard Hourglass | 128 | Refinement of features with skip connections. | |
| HG-Block 3 | Dilated Hourglass | 128 | Custom: Uses dilation rates (2, 4) to expand receptive field for sharp turns. | |
| Heads | Conv | 7 | Projects features into: 1x Confidence, 2x Offsets, 4x Embeddings. |
Table 7. Layer-wise Complexity Analysis.
Figure 14 shows a representative raw RGB frame captured on the competition track. The scene is intentionally cluttered—lane markings, a pedestrian crosswalk, a mandatory-direction sign, and a bank of distant traffic signs are all simultaneously visible—illustrating the range of cues the perception stack must disentangle from a single monocular frame. The lower portion of the frame is partially occluded by the vehicle’s own chassis and wiring harness; because we do not crop, the network must (and does) learn to ignore this static foreground rather than having it removed by a hand-tuned mask.

Figure 14. Raw RGB frame captured from the RealSense D455 on the competition track. The full, uncropped field of view retains peripheral cues (distant signs, crosswalk, hairpin markings) at the cost of a static foreground occlusion from the chassis, which the network learns to suppress.
Lane Detection Architecture
We utilized a deep learning approach based on the Stacked Hourglass architecture. Standard lane detectors often struggle with the tight turning radii ( m) found in 1:10 scale tracks. To address this, we engineered specific modifications into the PINet backbone.
Layer-wise Complexity Analysis
The summary below details the structural evolution of the feature maps as they propagate through the network. The Stem layer performs aggressive downsampling to reduce the memory footprint. The architecture then transitions into feature extraction, where HG-Block 3 introduces specific dilated convolutions. Finally, the Heads project these deep features into the specific 7-channel output.
Network Design: The ``Dilated'' 3-Stack Hourglass
Our model consists of three stacked hourglass modules. While the original PINet implementation uses standard convolutions, we engineered the bottleneck layers to utilize dilated convolutions [21]. The overall data flow is shown in Figure 15.

Figure 15. Top-Down Network Data Flow. The high-resolution input () is first processed by a Resizing Network to reduce spatial dimensions to , optimizing inference speed on the Jetson Orin. The core backbone consists of three Stacked Hourglass modules. The final module (Hourglass 3) is engineered with a Dilated Bottleneck.
The Engineering Challenge: Standard convolutions in a compact network often lack the receptive field necessary to “see” the curvature of a sharp turn before the vehicle enters it.
The Solution: By applying dilation rates of 2 and 4 in the bottleneck layers, we expanded the network’s receptive field without increasing the parameter count. This allows the model to capture the global context of the lane geometry (e.g., a sharp U-turn) while retaining local spatial resolution for precise localization.
Reliability Strategy: We opted to train a 3-Hourglass network from scratch rather than distilling a larger 4-Hourglass teacher network. This “direct training” approach ensured that the validation metrics observed during training perfectly matched the inference behavior on the embedded hardware.
Mathematical Formulation of the Loss Function
To ensure precise key point estimation and instance segmentation, we implemented a multi-task loss function. This function is applied to the three output branches (Confidence, Offset, and Embedding) of each hourglass module.
Confidence Loss. The confidence branch predicts the probability of a key point existing in a given grid cell. We utilize a split loss formulation to handle the class imbalance. The Existence Loss penalizes errors only in cells where key points actually exist ():
The Non-Existence Loss () minimizes false positives in background cells with a regularization term to prevent overfitting to zero:
Offset Loss. To mitigate quantization error, the network predicts normalized offsets in the range 0–1. The loss is calculated only for cells containing key points:
Embedding Feature Loss. To distinguish between multiple lane instances, we employ a discriminative loss function [22]. This pulls embeddings of the same instance () closer together and pushes embeddings of different instances () apart by a margin :
Where the pairwise distance term is defined as:
Total Optimization Objective. The final objective function is a weighted sum of these components:
Data Curation and Augmentation Pipeline
We developed a proprietary semi-automated data pipeline to solve the lack of labeled data.
Manual Annotation: We manually annotated 5,000 key frames.
The Augmentation Engine: We developed custom software to procedurally generate a dataset of 33,000 frames. The software applied randomized perturbations including geometric (random zooming, cropping, and horizontal flipping), photometric (HSV shifting to simulate different time-of-day lighting), and noise injection (Gaussian noise addition to simulate low-light sensor grain) transformations.
Training Strategy
Training was performed in two distinct stages.
Transfer Learning: Pre-trained for 20 epochs on the TuSimple public dataset [23].
Domain Adaptation with Manual Scheduling: Fine-tuned on our custom augmented dataset. We stepped the learning rate down (from to ) specifically when the loss value dropped below 1.00.
Figure 16 summarizes the two training stages.

Figure 16. Phase 1 and Phase 2 Training Diagram. A public dataset provided the base feature extraction weights, while aggressive domain adaptation using synthetic expansion tuned the model to the physical constraints of the competition track.
Lane Geometry Extraction and Post-Processing
The raw outputs of the PINet network exist in perspective image space, which are non-linear and unsuitable for direct lateral control. To bridge the gap between perception and path planning, we engineered a deterministic post-processing pipeline that extracts physical state variables: Cross-Track Error (distance), Heading Error (degree), and Curvature.
Inverse Perspective Mapping (IPM)
To accurately measure distances and angles, the raw clustered lane points are projected into a Bird’s Eye View (BEV) using a highly calibrated Homography matrix () [24]. Let represent a point in image space. The corresponding physical ground plane point is obtained via:
By executing this affine transformation after network inference rather than before, we preserve the far-field visual horizon for the neural network while achieving geometrically accurate measurements in the near-field for the controller. The pipeline in fact maintains two parallel realizations of this transform: a full-frame cv2.warpPerspective used only when a rasterized BEV image is required for visualization or the debug stream, and a lightweight point-domain realization that applies the homogeneous matrix product directly to the sparse cluster centroids. The latter avoids the cost of interpolated image resampling entirely, which matters when the transform is evaluated every frame at 30 FPS on embedded hardware.
Polynomial Path Fitting
Filtered BEV points are isolated into left and right lane boundaries. We model the lane geometry by fitting a second-degree polynomial to the and coordinates of the lane boundaries:
If both lane lines are visible, the algorithm computes a dynamic centerline trajectory by averaging the polynomial coefficients: .
Control Variable Generation
With the centerline modeled mathematically, critical control variables are extracted dynamically relative to a look-ahead point (). The Heading Error () is computed from the derivative of the polynomial:
The Curvature () is extracted using the standard formula for the radius of curvature:
To maximize stability, the final reported Cross-Track Error and Heading Error are not sampled from a single look-ahead point, but rather derived from a weighted spatial average spanning , heavily penalizing variations close to the vehicle chassis.
Embedding-Guided Instance Clustering
Raw detections emerging from the confidence, offset, and embedding heads arrive as an unordered point cloud; before any geometric fitting can occur, points must first be grouped into discrete lane instances. Rather than committing to a fixed-count clustering algorithm such as -means, we implemented a greedy, similarity-threshold clustering scheme that mirrors the training-time discriminative loss (Eq. (42)) directly at inference time: each candidate point is compared, in embedding space, against every cluster centroid formed so far, and is merged into the nearest cluster whenever the distance falls under a fixed affinity threshold ; otherwise it seeds a new cluster, bounded to a maximum of 12 concurrent instances to guarantee bounded worst-case runtime. Cluster centroids are updated incrementally as a running mean rather than recomputed from scratch, keeping the per-frame cost linear in the number of detected points.
Algorithm 1 Greedy Embedding-Affinity Clustering
Require: Points passing the confidence threshold; embeddings ; affinity threshold ; instance cap
Ensure: Instance-labeled point sets
1: 2: for each point do 3: if then 4: create a new cluster with centroid 5: else 6: 7: if then 8: assign point to cluster 9: ▹ incremental running mean
10: else if then
11: create a new cluster with centroid
12: end if
13: end if
14: end for
Two further filters run around this clustering stage. First, any cluster retaining three points or fewer is discarded as sensor noise prior to fitting, since a two-point set cannot support a statistically meaningful regression. Second, a region-of-interest mask, enforced via a calibrated polygon test against the drivable corridor, rejects points that fall outside the physically relevant track area both before and after clustering — suppressing off-track clutter and adjacent-lane bleed-through at the image periphery. A dedicated far-field suppression rule additionally discards any cluster whose points lie overwhelmingly () above a fixed BEV row threshold, since such clusters typically correspond to unstable horizon-line artifacts rather than the near-field lane segment relevant to control.
Adaptive-Order Fitting and Nearest-Boundary Selection
Rather than committing every detected cluster to the fixed second-degree model of Eq. (46), the pipeline selects the polynomial order according to point count: sparse clusters (6–16 points) receive a first-degree (linear) fit, while well-populated clusters (> 16 points) receive the full second-degree fit. This prevents an under-constrained quadratic from chasing detector noise on short, partially-occluded lane segments — a frequent occurrence on a 1:10 scale track where motion blur and occlusion routinely truncate the visible lane length.
Because the network can, and does, emit several simultaneous line-cluster candidates per frame — occluded segments, opposing-straightaway bleed-through on hairpin exits, or spurious far-field fragments that survive the filters above — the pipeline does not naively take the two most confident clusters as “left” and “right.” Instead, every surviving fit is evaluated at the vehicle’s projected center row, and the pipeline independently selects the fit whose lateral position is nearest to the center line on the left and on the right, discarding all remaining candidates as scene clutter. This center-adjacency heuristic is what keeps the controller stable even in frames where the network reports four or five simultaneous line candidates.
When only one boundary survives this selection — for example, at a hairpin apex where the outer line briefly leaves the field of view — the centerline is reconstructed by offsetting the single available fit using a lane-width estimate. This width is tracked dynamically from the most recent frames in which both boundaries were visible, clamped to a physically plausible band of 60–100 px in BEV space, and falls back to a fixed default of 70 px whenever the running estimate leaves this band. This graceful-degradation path is what prevents a momentary single-line detection from producing a discontinuous jump in the reported centerline.
Finally, the pixel-domain lateral offset (Eq. (46)) and heading angle (Eq. (47)) are not physically meaningful on their own: both are converted to real-world units by calibration constants obtained from the same homography used for the IPM warp in § 4.6, before being packed and transmitted over the 30 Hz ROS control topic served by the shared-memory bus described in § 4.9.
Figure 17 shows a representative output of the full post-processing chain. After Inverse Perspective Mapping rectifies the perspective distortion, the greedy embedding-affinity clustering (Alg. 1) resolves the scene into discrete instances, and the adaptive-order polynomial fitter produces the three curves used by the controller: the left boundary (blue), the right boundary (red, dashed), and the synthesized centerline (green) computed by averaging the boundary coefficients. In the rectified BEV domain the physically parallel lane boundaries appear vertical and near-parallel, confirming that the homography is well calibrated; residual convergence at the top of the frame is the expected far-field noise that the weighted spatial averaging over is designed to attenuate.

Figure 17. Bird’s Eye View output of the post-processing pipeline. Clustered lane points are rectified via the homography and fitted with adaptive-order polynomials: left boundary (blue), right boundary (red, dashed), and the synthesized centerline (green). The near-vertical, parallel boundaries validate the IPM calibration.
Traffic Sign and Signal Perception
Due to the scarcity of high-quality annotated data for 1:10 scale traffic signs within the target environment, relying strictly on single-frame neural network classification yielded unacceptable variance (flickering). To guarantee robust perception, we implemented a hybrid depth-gated approach coupled with an aggressive temporal smoothing architecture.
Depth-Gated Validation. The RealSense D455 provides aligned depth and RGB maps. Instead of relying purely on 2D bounding box dimensions to estimate sign proximity, we sample the raw depth tensor (scaled by 0.001 meters/unit) at the centroid of the detected sign. Sign bounding boxes reporting distances beyond a physical limit of 0.85 meters are immediately rejected. This depth-gating prevents the vehicle from reacting prematurely to signs in adjacent lanes or far down the track.
Temporal Voting Filter (FIFO Buffer). To overcome classification instability from the sparse training data, we treat the detection stream as a noisy time-series signal. We engineered a sliding-window First-In-First-Out (FIFO) buffer of length .
Let be the raw classification at time . The final published classification is dictated by a strict majority voting threshold:
This hysteresis loop guarantees that temporary occlusions or motion blur do not trigger an erroneous state change, dramatically smoothing the behavioral transitions of the vehicle. Figure 18 illustrates the filter.

Figure 18. The FIFO Temporal Filter mitigates single-frame detection errors, translating noisy, data-starved classifier outputs into robust, state-machine-ready signals.
Crosswalk Detection
Pedestrian crosswalks require a fundamentally different treatment from the parametric lane model. A crosswalk is not a continuous boundary to be tracked but a discrete, safety-critical region that must trigger a deterministic stop-and-yield behavior. We therefore run crosswalk detection as an independent perception path rather than folding it into the PINet lane head, so that a crosswalk can be flagged even in frames where the lane geometry is momentarily ambiguous.
Detection and Geometric Validation. The detector operates on the rectified Bird’s Eye View, where the characteristic ladder pattern of a crosswalk becomes a set of near-rectangular, regularly spaced stripes of consistent orientation and area. Candidate stripes are extracted and then subjected to a geometric consistency test: an isolated bright rectangle is rejected, whereas a group of stripes sharing a common orientation, comparable area, and uniform inter-stripe spacing is accepted as a crosswalk. This structural voting scheme is deliberately conservative—demanding a repeating pattern rather than a single marking—so that the isolated dashes of a lane boundary or a specular reflection on the floor cannot masquerade as a crossing. Figure 19 shows a representative detection, in which each validated stripe is bounded by a green box in the BEV frame.

Figure 19. Crosswalk detection in the rectified BEV frame. Each validated stripe of the repeating ladder pattern is bounded by a green box; the isolated marking at the bottom of the frame is correctly excluded by the group-consistency test, preventing a single stripe or floor reflection from triggering a false stop.
Behavioral Signaling. Upon a validated detection, the crosswalk node updates a boolean status flag in shared memory and immediately notifies the control node, which allows the vehicle to execute a controlled deceleration before the marked region rather than reacting only once the stripes are directly beneath the chassis. Because the crosswalk flag is transmitted over the same zero-copy shared-memory channel described in § 4.9, this safety-critical signal incurs no serialization latency and remains synchronized with the lane and sign outputs on a per-frame basis.
High-Throughput IPC Architecture
A critical bottleneck in complex ROS 1 environments is the latency and CPU overhead caused by TCP/IP serialization of large image payloads. To resolve this, we bypassed the standard sensor_msgs/Image pipelines. Both the Lane Perception and Sign Perception nodes access raw camera tensors directly via POSIX Shared Memory (multiprocessing.shared_memory). This zero-copy Inter-Process Communication (IPC) strategy reduces frame distribution latency to sub-millisecond levels, ensuring the entire perception stack operates synchronously at 30 FPS.
Experimental Results and Validation
Training Convergence Analysis
The model’s learning trajectory validates our two-stage training strategy.
Rapid Adaptation (Epochs 0-8): Sharp initial decline in loss using pre-trained weights.
Optimal Performance Window (Epochs 8-21): Loss minimized to its lowest sustainable point. Our custom dataset prevented overfitting.
Stability: Post-epoch 21, the manual learning rate scheduler prevented “exploding gradient” issues.
Figure 20 shows the training loss over epochs.

Figure 20. Training loss over epochs. The period highlighted between Epochs 8 and 21 demonstrates the model achieving maximum accuracy without succumbing to overfitting constraints.
Qualitative Domain Adaptation
Figure 21a shows the model inference on the public TuSimple dataset. Figure 21b demonstrates the model operating in our custom lab environment. Despite differences in lighting and texture, the network successfully generalized its lane instance extraction capabilities. The most demanding test of this transfer is not the dataset the network was pre-trained on, but the raw competition track under laboratory lighting, where reflective flooring, hard shadows, and the vehicle’s own chassis occlusion are all present in a single frame.
End-to-End Lab Inference and Instance Separation
The clearest evidence that the trained network operates correctly on the physical platform is its in-lab output on live frames. Figure 22 overlays the network’s key point predictions—after the greedy embedding-affinity clustering of Algorithm 1—directly onto a raw track frame captured in the lab. The two lane boundaries are cleanly resolved into distinct instances, rendered here in blue (left boundary) and green (right boundary). Critically, the discriminative embedding loss (Eq. (42)) has done exactly what it was trained to do: even though the two point sets converge toward a common vanishing region near the crosswalk, the network assigns them to separate instances rather than merging them, and it correctly suppresses spurious detections on the reflective foreground and the static chassis in the lower frame. Read together with the domain-adaptation panels of Figure 21, this confirms that both the feature extractor and the instance-separation head transferred successfully from the public pre-training data to the competition environment.

(a) TuSimple Dataset

(b) Custom Lab Track
Figure 21. Domain Adaptation Results. (a) Successful instance segmentation on the TuSimple pre-training dataset. (b) Successful transfer to the custom competition environment, handling distinct texture and lighting changes dynamically without ROI cropping.

Figure 22. PINet output in the lab environment. Clustered key points are overlaid on a raw competition-track frame, with the two lane boundaries separated into distinct instances (blue: left, green: right) by the embedding-guided clustering stage. Foreground reflections and the chassis occlusion are correctly rejected.
This image-space clustered output is precisely the input consumed by the post-processing pipeline of § 4.6: the same blue/green instances, once passed through Inverse Perspective Mapping and adaptive-order polynomial fitting, yield the rectified Bird’s Eye View trajectory of Figure 17. The two figures therefore represent consecutive stages of a single frame’s journey from raw pixels to controller-ready state variables.
Control and Localization
Accurate localization is essential for enabling the autonomous vehicle to determine its position, orientation, and motion relative to the environment. In this system, vehicle dynamics are modeled using the Ackermann steering mechanism and the standard Bicycle Kinematic Model [25], which provides a mathematically tractable representation of motion for ground vehicles with small steering angles. This model is used both for localization estimates and as a foundation for motion planning and feedback-based correction.
Vehicle Kinematic Modeling
The bicycle model treats the vehicle as a single-track system with a front and rear wheel aligned along a common centerline. The steering input is applied at the front wheel, and the resulting motion is governed by geometric constraints imposed by the fixed wheelbase.
Figure 23 illustrates the kinematic abstraction of the vehicle used in this project.

Figure 23. Bicycle model representation of the vehicle kinematics.
Following the Ackermann steering relationship, the steering geometry for the equivalent bicycle model may be expressed as:
Since is the turning radius of the rear-axle centre, this is the mean of the two wheel radii measured on the rear-axle line, and it is exact when the wheel axes meet at a common point on that line.
Given the front steering angle and linear velocity , the kinematic state of the vehicle is represented by three parameters: (longitudinal position), (lateral position), and (vehicle heading).
Depending on the reference point used (front axle or rear axle), the equations of motion are expressed as follows:
Front-Axle Model
Rear-Axle Model
Here, denotes the vehicle wheelbase, is the linear velocity of the equivalent rear wheel (at the rear-axle center), is the linear velocity of the equivalent front wheel (at the front-axle centre), is the vehicle orientation, and is the front steering angle. These expressions provide a motion-level representation for predicting the vehicle’s pose based on its current control inputs.
Steering-Limited Path Design
Due to the mechanical constraints of the Ackermann steering layout, the vehicle cannot arbitrarily translate between arbitrary pose configurations with a single steering command. Instead, its feasible motion is limited by the wheelbase and maximum steering angle . As a result, transitioning between two configurations typically requires a sequence of bounded curvature maneuvers [26].
Figure 24 illustrates an example of repositioning the vehicle using up to three maximum-steering arcs, a common strategy when a single steering motion is insufficient.

Figure 24. Vehicle relocation between two poses using up to three steering segments with maximum steering angle.
The minimum turning radius imposed by the vehicle geometry is:
To represent motion between an initial pose and target pose , the path may be decomposed into three sequential curvature segments of angular change , , and , where:
Let the cumulative displacement vector be:
where , , and are rotation matrices corresponding to the vehicle orientation at the start of each segment:
Figure 25 provides a geometric representation of the three-segment motion formulation.

Figure 25. Geometric and mathematical representation of three-segment steering motion.
This three-segment maneuver is specifically required for repositioning the vehicle into a region on its right side that cannot be reached through a single ordinary steering command, due to the geometric constraints imposed by the vehicle’s minimum turning radius. Repositioning into the mirror-symmetric region on the left side does not require re-deriving the maneuver: it suffices to mirror the target displacement about the vehicle’s longitudinal axis, solve Eq. (59) for that mirrored displacement to obtain , , , and then negate all three resulting angular changes (i.e., use , , ) to obtain the angular changes that actually produce the desired displacement into the left-side region. See Figure 26.

Figure 26. Example three-segment steering maneuvers (tangent circular arcs) for different , , combinations, illustrating repositioning into regions unreachable by a single steering command.
A special case frequently encountered in structured environments (e.g., four-way intersections) is the 90° right turn with constant curvature. In this maneuver:
This symmetry simplifies analytic evaluation and enables computation of the turn using closed-form expressions for geometry and displacement. In practical use, this model is executed with steering saturated at and verified using gyroscope feedback.
Asymmetric steering limits. Eq. (59) uses the same minimum radius for all three arcs. This is appropriate when the vehicle moves according to the bicycle model with Ackermann-type steering and the maximum steering angle is the same in both directions. If the maximum wheel angle differs between turning to the right (positive limit) and turning to the left (negative limit), the minimum radius takes a different value for each direction, and , where denotes the reference wheel: the right wheel for repositioning into a region on the right side of the vehicle, and the left wheel for repositioning into a region on the left side of the vehicle. Two cases are distinguished. Case 1: The difference is not negligible. Each arc uses the radius of its own steering direction:
For a region on the left side of the vehicle, the corresponding relation follows the same mirroring procedure described earlier: mirror the target displacement about the vehicle’s longitudinal axis, solve as in Eq. (64), then negate the resulting , , , only now using the left-side radii , , in place of , , .
Case 2: The difference is small enough that the two radii can be averaged.
For a region on the right side of the vehicle, Eq. (59) is retained with replaced by . For a region on the left side of the vehicle, following the same mirroring procedure described earlier, Eq. (59) is retained with replaced by .
Navigation and Steering Control Algorithms
To maintain stable, lane-centered navigation under varying road topologies and speeds, multiple feedback controllers [27] are implemented in the vehicle’s decision and actuation loop. Each controller is selected for its specific performance characteristics and switching between them is performed according to the vehicle state and operational mode.
Figure 27 illustrates the two primary geometric feedback quantities used in lane-keeping control: the Cross-Track Error (CE), the lateral deviation of the vehicle center from the lane reference centerline, and the Heading Error (HE), the angular misalignment between the vehicle’s orientation and the tangent direction of the lane.

Figure 27. Geometric interpretation of cross-track and heading errors used in lane-keeping control.
PID Controller. A PID controller [28] is used primarily as a backup steering mechanism and for regulating the forward velocity. The steering command is computed as:
In final deployment, PID also regulates speed: when heavy curvature is detected, the proportional component automatically reduces forward velocity, improving cornering stability. A higher-priority obstacle-response P-regulator supersedes normal lane PID when vision or onboard sensors detect objects obstructing the path.
Stanley Controller. The Stanley controller [29, 30] is the primary steering law used for continuous lane following due to its kinematic and speed-adaptive formulation. The steering angle is computed as:
where is the forward velocity and prevents singular behavior near zero speed.
This model-based formulation provides smoother transitions during acceleration and deceleration, reducing oscillation at higher speeds. Its direct grounding in the vehicle geometry makes it superior to a pure PID controller for continuous closed-loop lane tracking.
Pure Pursuit Controller. The Pure Pursuit controller [31, 32] computes the steering angle based on a geometric look-ahead point projected onto the lane reference, determined from the rear-axle position (Figure 28). The steering angle is calculated as

Figure 28. Pure Pursuit geometry illustrating the look-ahead point, look-ahead distance, and heading error angle.
where is the vehicle wheelbase, is the look-ahead distance, is the look-ahead gain, and is the forward velocity. The angle is defined as the angle between the vehicle’s longitudinal heading direction and the line connecting the rear-axle position to the look-ahead point on the lane reference.
This method is particularly effective for tracking curved roads and complex trajectories, as it determines the steering command from the geometric relationship between the vehicle and the look-ahead point. The speed-dependent look-ahead distance allows the controller to adapt its tracking behavior to the vehicle’s forward speed.
Gain Calibration. Controller parameters for PID, Stanley, and Pure Pursuit were tuned empirically through iterative testing. Real-time visualization of control variables was used while adjusting responsiveness, lateral settling time, overshoot, and high-speed stability. The final gains delivered fast, non-oscillatory velocity control, minimal steady-state lane deviation at moderate/high speeds, and smooth cornering without abrupt steering.
Decision Making, Parking, and Intersection Handling
The decision layer of the vehicle converts high-level perception outputs into discrete driving actions. All logic is structured as a Finite State Machine (FSM) that evaluates lane geometry, traffic signs, crosswalk signals, and feedback from onboard sensors, then transitions between states such as lane following, parking, and intersection traversal.
Parking Behavior
During normal navigation, the vehicle continues lane following until a parking sign is detected by the vision subsystem. Once detected, the FSM transitions into the parking sequence shown in Figure 29. The vehicle searches for a valid free space using onboard sensors and begins the maneuver as soon as a suitable location is detected.

Figure 29. Finite State Machine for autonomous parking behavior.
The parking process consists of two main phases.
Open-Loop Geometric Maneuver: An open-loop geometric maneuver was developed to achieve the required vehicle alignment after turning. The maneuver consists of three sequential phases: maximum steering for trajectory insertion, straight displacement for position adjustment, and counter-steering for final heading correction. The motion parameters were calculated using the kinematic bicycle model and further refined through empirical calibration to compensate for practical effects such as tire slip, actuator delay, and mechanical backlash, resulting in repeatable and accurate maneuver execution.
Feedback-Based Correction: Once the initial maneuver completes, sensor data (vision and IMU) are used to refine the final position. The negative feedback loop corrects residual lateral and angular error to ensure precise alignment within the parking space.
Figure 30 illustrates example end states for parallel and perpendicular (cross) parking.

Figure 30. Example parking results: left – parallel parking; right – perpendicular (cross) parking.
Sign-Approach Distance Estimation on Curved Paths
Correctly estimating the remaining distance to a sign matters beyond the intersection maneuver of § 5.4.3: the same estimate also drives the initial approach of the parking maneuver (§ 5.4.1). Once the park sign is detected, the vehicle must advance by the corresponding encoder distance so that its side-facing distance sensor reaches approximately the correct position; specifically, the free-space search algorithm for parking is initiated about 10 cm past the point where the park sign was first sighted. In this case, as in the intersection approach, the straight-line distance reported by the vision pipeline is not, in general, the distance the vehicle will actually travel: whenever the road curves during the approach, the vehicle follows a curved path rather than a straight line toward the sign, which the procedure below accounts for.
As long as the vision pipeline reports a valid sign distance, the vehicle drives directly on that live reading. From the moment this data is lost — or is deliberately no longer used — the required travel distance is instead estimated from three quantities available onboard: the yaw rate obtained from the compass (CMPS), the vehicle speed , and the last valid sign distance reported by vision before tracking was lost. Three cases are distinguished, summarized in Table 8 and detailed below.
| Case | Condition | Action |
| 1 | Path assumed locally straight: advance the last known sign distance directly, commanded by the encoder. | |
| 2 | and | Curve detected, but the arcsine relation of Case 3 is outside its valid domain (); fall back to the same straight-line action as Case 1. |
| 3 | and | Curve detected and the sign lies within the estimated turn radius: use the arc-length correction of Eqs. (70)–(71) instead of the raw distance . |
Table 8. Sign-approach distance estimation: case selection based on yaw rate and the vision-reported distance.
Case 1 — straight path. If the magnitude of the yaw rate falls below a minimum threshold , the road is assumed to be locally straight for the remainder of the approach. The vehicle then simply continues toward the sign using the last known distance , commanding the encoder to cover exactly that distance.
Cases 2–3 — curved path. Otherwise the vehicle is in a curve, and the local radius of curvature is estimated from the current motion state as
If (Case 2), the sign lies beyond the estimated turn radius; the correction below is outside its valid range, so the vehicle falls back to the same straight-line action as Case 1, using directly.
Case 3 — within-radius correction. If , the sign lies within the estimated turn radius, and the raw distance increasingly underestimates the true travel distance as the path curves more sharply. Assuming the curvature does not change appreciably before the sign is reached (Figure 31), the subtended arc angle is

Figure 31. Geometry relating the estimated turn radius , the vision-reported forward distance , and the subtended angle .
and the encoder is commanded to travel the corresponding arc length rather than the chord distance :
which gives a more accurate stopping distance than driving the raw vision distance whenever the path curvature is significant relative to the remaining distance to the sign.
Intersection Control and Turning Logic (Version 4)
When approaching an intersection, the correct maneuver is determined based on the recognized sign. The three supported actions (turn right, turn left, and go straight) are shown in Figure 32.

Figure 32. Available intersection actions: right turn, left turn, forward crossing.
Once sign data is first acquired, the vehicle enters the “Intersection” state, and the maneuver proceeds through three stages: (1) approach and crosswalk-data acquisition, (2) pose correction relative to the crosswalk, and (3) turn execution. Each stage is described below and illustrated in Figure 33 and Figure 35.

Figure 33. Stage 1 — sign-distance approach and crosswalk-data acquisition at the crosswalk inner edge, with a single probing retry.

Figure 35. Stage 2 — iterative crosswalk pose correction, and Stage 3 — turn execution (unchanged).
Stage 1 — Approach and crosswalk-data acquisition (Figure 33). Because the probability of losing the sign from the camera’s field of view increases as the vehicle gets closer to it, the approach to the sign is not driven by live vision data alone: at each control step the vehicle checks whether current sign-distance vision data is valid, advancing on that live reading if so, and otherwise falling back to the curvature-based estimate of § 5.4.2. As the vehicle nears the hood-to-sign offset ( cm), it begins polling for crosswalk data as well.
When the vehicle subsequently reaches cm from the crosswalk’s inner edge, it checks whether crosswalk vision data is currently valid. If so, it advances the remaining distance to the inner edge on that data directly. If not — whether because crosswalk data had been in use and was suddenly lost, or because no crosswalk data had been received at all since the 10 cm point — the vehicle instead completes the approach by dead reckoning: in the first case, from the encoder progress accumulated since the 10 cm point; in the second, from the full curvature-based estimate of § 5.4.2.
Once at the crosswalk inner edge, the vehicle queries crosswalk data once more. If it is valid, the vehicle proceeds directly to Stage 2. If not, a single probing feedback maneuver is executed and the query is repeated exactly once; if crosswalk data is still unavailable after this one retry, the vehicle is considered to be in an unsafe state, since the direction and magnitude of any approach skew remain unknown.
Stage 2 — Pose correction relative to the crosswalk (Figure 35). With valid crosswalk data available, the crosswalk-approach angle is measured. If it exceeds the allowed tolerance, a corrective feedback maneuver shifts the vehicle a modest amount to the left, away from the right-hand corner of the intersection, and crosswalk data is queried again to re-measure the angle; this correction loop repeats until the angle falls within tolerance (see Figure 34). Rather than requiring a fresh valid vision reading at every iteration, the angle change produced by each feedback step can instead be estimated from the commanded CMPS rotation and fused with the most recent vision reading, reducing dependence on repeated crosswalk detections.

Figure 34. Feedback maneuver design for crosswalk alignment and lateral correction. The illustrated maneuver positions the vehicle behind the crosswalk while shifting it slightly to the left. Alternative feedback maneuvers can achieve the same positioning and lateral shift without necessarily aligning the vehicle perpendicular to the crosswalk.
Stage 3 — Turn execution (Figure 35). Once the crosswalk-approach angle is within tolerance, the vehicle first advances straight, by encoder count, until it is aligned just behind the crosswalk. The subsequent action depends on the turn direction: for a right turn, the measured crosswalk-approach angle is substituted into the three-steering-wheel geometry of § 5.2 to solve for the three required encoder (CMPS) values, which are then applied to the vehicle in sequence; for a left turn, the vehicle instead performs a single rotation of fixed magnitude, counter-clockwise, corresponding to an encoder travel of , where accounts for the observed approach angle.
Evolution of the Intersection-Handling Strategy
The intersection-handling logic was developed iteratively over four versions, moving from a fixed, open-loop maneuver to a closed-loop, kinematically grounded approach. Table 9 summarizes the progression; the following subsections describe each version in more detail.
| Version | Core Approach | Key Strengths | Key Limitations |
| Ver. 1 | Open-loop, hardcoded actuation triggered at a fixed 10 cm distance from the sign (e.g., advance 30 cm, then turn until a preset heading change is sensed by the compass). | Minimal development time. | Only IMU feedback; almost entirely dependent on a fixed approach geometry. |
| Ver. 2 | Added intersection-approach angle. A PID loop first corrects heading error, then the same hardcoded turning actions as Ver. 1 are executed. | Introduces road-relative feedback. Simpler algorithm; fewer tuneable parameters. | High perception noise from a low-quality, poorly mounted camera; the vision data was only reliable in a narrow crosswalk viewing position, so the feedback loop had to re-sample it 2–3 times. Failed when the vehicle approached skewed toward the right edge of the road. Could not handle intersections on curved roads. Two-lane validation was added but was not accurate enough to enable feedback after the crosswalk or inside the intersection. |
| Ver. 3 | Generalized further: road curvature () was introduced to estimate the distance the vehicle must travel through a bend to reach the sign, with turn direction (CW/CCW) taken from perception. | Broader applicability across road geometries. | More parameters to tune and longer development time than earlier versions. Increased generality reduced reliability: at plain, straight intersections the algorithm could still unnecessarily factor in crosswalk angle and curvature, causing failures over part of the intersection design space. |
| Ver. 4 | Adopted the vehicle’s bicycle-model kinematics to reach the target lane more reliably and directly, motivated by a reduced maximum steering angle that had been increasing the distance travelled in the opposing lane. Repositioning between an initial and target pose is solved as a three-segment steering maneuver (§ 5.2), with the required encoder (CMPS) travel computed for each segment. The same model-based CMPS estimation is used to judge how far the vehicle must travel through a curve to reach a set distance from the sign, replacing noisy curvature-based perception. A dedicated feedback behavior biases the vehicle slightly left to counteract drift toward the right edge; once the vehicle reaches the crosswalk at a shallow, safe angle with no risk of edge proximity, the three-segment maneuver is executed using the measured crosswalk-approach angle. | More reliable: grounded in the actual vehicle kinematic model rather than hardcoded actions. More accurate sign approach in sharp intersection turns, by relying on the vehicle model and CMPS changes instead of noisy curvature perception. | Highly sensitive to the mechanical design, steering calibration (§ 5.5) and measurement accuracy (§ 3–§ 5); imprecise geometry directly degrades the open-loop maneuver. Tends to look mechanically stiff, and a reverse segment is less visually natural — a potential drawback for real-world user acceptance. |
Table 9. Summary of the four versions of the intersection-handling algorithm.
Version 1 — Open-Loop Baseline. The first version implemented the simplest possible behavior: once the vehicle was within 10 cm of the sign, a hardcoded sequence of actions executed the turn open-loop, with no feedback — for example, advancing 30 cm and then turning until the onboard compass (CMPS) sensed a preset change in heading, at which point the maneuver was considered complete. Its only advantage was a very short development time; it could not compensate for any deviation from the assumed approach geometry.
Version 2 — Heading-Error Feedback. The second version added the intersection-approach angle to the pipeline and began handling the heading error caused by a skewed arrival at the intersection. A PID controller first corrected this heading error, after which the vehicle executed the same type of hardcoded turning actions as version 1. This version incorporated road-relative feedback for the first time and kept the algorithm simple, with few design parameters. However, the camera in use was of low quality and poorly positioned on the chassis, so the processed vision data carried substantial error; because valid-looking data was only obtained from one
specific viewing position relative to the crosswalk, the motion-feedback loop had to re-acquire vision data two to three times per approach. The version performed poorly whenever the vehicle’s pose relative to the road was already skewed toward the right edge, and it could not handle intersections located on curved roads. Two-lane detection was added to verify that the target lane had been reached, but its accuracy was insufficient, so feedback after the crosswalk and inside the intersection itself remained unavailable.
Version 3 — Curvature Generalization. [1] The third version generalized the approach further by introducing an initial implementation of road curvature () to estimate the distance the vehicle needed to travel through a bend to reach the sign, with the turn direction (clockwise or counter-clockwise) taken from the perception pipeline. This broadened the range of road geometries the algorithm could address, but at the cost of more tunable parameters and a longer development time than the previous versions. More importantly, the added generality reduced reliability: because the logic could still weight crosswalk angle and curvature even at plain, straight intersections, it occasionally failed on a subset of intersection types that the earlier, simpler logic had handled correctly.
Version 4 — Bicycle-Model-Based Maneuvering. The fourth version targeted more reliable motion: reaching the target lane faster and minimizing time spent in the opposing lane. A reduction in the vehicle’s maximum steering angle had made this worse, forcing a long excursion in the wrong lane before the vehicle could correct itself. To address this, the version adopted the vehicle’s motion geometry — the bicycle model (§ 5.1) — based on the observation that the vehicle’s motion lies on a circular arc. This raised the geometric question of how to move between two poses with defined initial and target headings when a single steering action cannot satisfy the constraint; the resulting solution was the three-segment steering maneuver described in § 5.2, in which the required encoder (CMPS) travel for each of the three segments is computed and applied to the vehicle. The same model-based CMPS estimation was reused to judge how far the vehicle should travel through a curve to reach a specified distance from the sign, replacing the noisy curvature-based perception of version 3. A dedicated feedback behavior was added to counter drift toward the right edge of the road: it biases the vehicle slightly to the left and is shaped so that its behavior does not itself tend toward the right edge. Once the vehicle reaches the crosswalk at a shallow, safe angle with no risk of edge proximity, the three-segment maneuver is executed using the measured crosswalk-approach angle.
Grounding the maneuver in the bicycle model made the behavior more reliable overall, since it replaced hardcoded actions with the actual vehicle kinematic model. It also improved accuracy when approaching the sign through sharp intersection turns, since the vehicle no longer depended on curvature data derived from noisy road-curve perception, relying instead on the vehicle model and CMPS changes. The main drawback is that the three-segment maneuver is strongly dependent on the vehicle’s mechanical design and on measurement accuracy (§ 2–§ 5): imprecise geometry directly reduces maneuver accuracy. In addition, the maneuver tends to look mechanically stiff, and when a reverse segment is required it can appear visually unnatural — a consideration for acceptance in real-world use.
Experimental Results and Validation
Steering Calibration
To calibrate the steering mechanism fabricated by the mechanical engineering team and to relate the commanded input to the wheel angles, an empirical data collection procedure was conducted in coordination with the embedded control subsystem.
At every setpoint, the combined (equivalent single-track) steering angle reported below is computed directly from Eq. (50), i.e. as applied to each individual trial, rather than as the simple arithmetic mean of the two measured wheel angles, so that it is consistent with the bicycle model of § 5.1.
The embedded controller maps control commands to the steering actuator (servo motor). The servo hardware is configured to operate across a nominal angular range of to with an estimated positioning variance of . A total of 13 discrete control input setpoints were evaluated, spanning from 255 down to 0. At each setpoint, 3 independent measurements were collected for both the left wheel angle () and the right wheel angle ().
At extreme steering inputs, such as and , the two front wheels turn by different angles (Figure 36), so the steering is not parallel. The equivalent angle of Eq. (50) is defined for any pair of wheel angles; it therefore calibrates the input of the bicycle model but does not by itself verify the Ackermann condition.

Figure 36. Front-left and front-right wheel steering angles, alongside the equivalent steering angle derived from the Ackermann relationship.
Across the 13 tested setpoints, a one-sample t-test with a significance level of showed that the combined steering angle (), computed from the per-trial wheel angles via the cot-average relation of Eq. (50) rather than a simple arithmetic average, does not have a statistically significant deviation from the desired servo setpoint in 92.3% (12 of 13 setpoints) of the operating range, with P-values greater than 0.05 at every setpoint except for . This indicates a high level of agreement between the commanded and measured steering angles. Table 10 and Figure 37 show this result.
| Control Input | Commanded Angle (°) | Combined Mean (°) | Combined StdDev (°) | Combined Var (°) | Mean Diff. (°) | T-Statistic | P-Value | Significant () |
| 0 | −40 | −40.198 | 2.204 | 4.857 | −0.198 | −0.156 | 0.8904 | No |
| 25 | −32 | −35.988 | 1.068 | 1.140 | −3.988 | −6.469 | 0.0231 | Yes |
| 50 | −24 | −24.917 | 1.013 | 1.027 | −0.917 | −1.567 | 0.2577 | No |
| 75 | −16 | −18.249 | 1.374 | 1.887 | −2.249 | −2.835 | 0.1051 | No |
| 90 | −11 | −12.321 | 1.888 | 3.565 | −1.321 | −1.211 | 0.3494 | No |
| 115 | −3 | −3.552 | 2.854 | 8.146 | −0.552 | −0.335 | 0.7696 | No |
| 128 | 0 | 0 | 0 | 0 | 0 | 0 | 1.0000 | No |
| 150 | 7 | 6.382 | 2.197 | 4.825 | −0.618 | −0.488 | 0.6741 | No |
| 175 | 14 | 15.217 | 2.791 | 7.792 | 1.217 | 0.755 | 0.5290 | No |
| 190 | 19 | 18.991 | 1.501 | 2.253 | −0.009 | −0.010 | 0.9931 | No |
| 215 | 27 | 27.825 | 1.528 | 2.335 | 0.825 | 0.935 | 0.4484 | No |
| 240 | 35 | 33.480 | 1.815 | 3.293 | −1.520 | −1.451 | 0.2839 | No |
| 255 | 40 | 36.515 | 2.158 | 4.657 | −3.485 | −2.797 | 0.1076 | No |
Table 10. Commanded steering angle versus the combined measured steering angle , computed from the per-trial wheel angles via the cot-average relation of Eq. (50). Statistics are over 3 trials per setpoint; the one-sample -test uses .

Figure 37. Commanded (input) steering angle versus the combined measured steering angle (mean_steer), computed solely from the cot-average of Eq. (50) — the individual per-wheel means are not used in this comparison. The dashed line marks perfect agreement (measured = commanded); error bars show ±1 StdDev across the 3 trials.
Additionally, the alignment of the steering system around the neutral position was checked. The left wheel crosses at and the right wheel at ; at the right wheel reads and the left wheel . Larger angular offsets between the calibrated and the actual wheel angles appear at large as residuals outside the ±2 cm band in Figure 40 and as the systematic trends of Figure 41 (fitted offsets in Table 12). For closed-loop path tracking and high-level trajectory planning the relationship between the 8-bit integer control input and the equivalent bicycle steering angle is modeled by Eq. (72). This fit is obtained by a direct linear regression of against the combined (equivalent single-track) steering angle of Eq. (50), evaluated from the per-trial measurements at each setpoint (as shown in Figure 38) — not by averaging the two individual wheel calibrations below, since a simple arithmetic average of and is not equivalent to Eq. (50). The separate per-wheel calibrations and are also present in Eq. (72), but are not used to derive .

Figure 38. Linear regression model for control input vs. steering angle .
| Statistic | Right wheel | Left wheel | Pooled |
| Sample size | 13 | 13 | 26 |
| Mean residual (cm) | |||
| Std. deviation (cm) | 2.18 | 1.13 | 1.70 |
| Std. error of mean (cm) | 0.60 | 0.31 | 0.33 |
| 95% CI of mean (cm) | |||
| -test vs. zero, -value | 0.941 | 0.776 | 0.839 |
| 90% CI of mean (cm) | |||
| Equivalence to cm (TOST [36] ) | 0.004 | ||
| Bootstrap [34] 95% CI of mean (cm) | |||
| RMSE (cm) | 2.09 | 1.09 | 1.67 |
| MAE (cm) | 1.51 | 0.66 | 1.09 |
| Max (cm) | 5.27 | 2.70 | 5.27 |
| Limits of agreement [35] (cm) | |||
| Points within cm [95% CI, %] | 10/13 = 77% [46, 95] | 11/13 = 85% [55, 98] | 21/26 = 81% [61, 93] |
| Slope of vs. (cm/deg) | — | ||
| Correlation (-value) | 0.86 () | (0.009) | — |
| Shapiro–Wilk [33] (normality of ) | 0.440 | 0.020 | 0.020 |
| (uncertainty-weighted) | 12/13 | 13/13 | 25/26 |
| (-value) | 0.77 (0.689) | 0.22 (0.999) | 0.49 (0.985) |
| Best-fit angular offset (deg) | — | ||
| RMSE after offset (cm) | 1.21 | 0.80 | — |
Table 12. Summary statistics of the residuals (all lengths in cm unless noted).

Figure 40. Model prediction (line) and measured lengths with cm error bars (points).

Figure 41. Residual versus wheel steering angle for the right (left panel) and left (right panel) front wheels. The shaded band marks the cm measurement tolerance and the dashed line is the linear trend of the residuals (right wheel: cm/deg, ; left wheel: cm/deg, ).
Experimental Validation of the Bicycle-Model Geometry
Reliable maneuvering in confined spaces (parking and intersection alignment, § 5.4.3) relies on the vehicle’s kinematic geometry. Our planner assumes the bicycle model of § 5.1, in which the motion is fully determined by the wheelbase and the front steering angle (for example , Eq. (57)). Before building maneuver planning on this assumption, we tested whether the relation actually holds on our physical vehicle, how large the deviations are, and whether they are random or systematic. In short: how does the real vehicle behave compared with the model? The dimensions of the vehicle used in the tests are listed in Table 1. The wheelbase cm is the only parameter entering the model prediction.
The steering command was swept from 0 to 255 at 13 setpoints. The steering angle of each front wheel follows the linear calibration of the previous section (§ 5.5.1). At every setpoint and for each front wheel separately, the rolling direction of the wheel was extended on the ground until it met the line through the two rear wheels (the rear-axle line), as illustrated in Figure 39. The length of this segment, from the wheel to the intersection point, was measured with an estimated uncertainty of cm. One measurement was taken per wheel and setpoint ( per wheel).

Figure 39. Measured quantity: the segment from the front wheel, along its rolling direction, to the rear-axle line.
In the bicycle model the wheelbase , the rear-axle line and the wheel’s heading form a right triangle, so the predicted length is as Eq. (73), which equals cm at and grows as the steering angle increases. The measured value was compared with this prediction at the calibrated angle of each wheel. Table 11 lists the model prediction, the measured length and the residual at every setpoint, and Figure 40 compares them graphically.
| Right wheel | Left wheel | |||||||
| (°) | Model (cm) | Meas. (cm) | (cm) | (°) | Model (cm) | Meas. (cm) | (cm) | |
| 0 | 35.50 | 36.7 | 32.49 | 30.0 | ||||
| 25 | 31.43 | 36.7 | 29.70 | 29.5 | ||||
| 50 | 28.74 | 29.3 | 27.79 | 27.5 | ||||
| 75 | 26.99 | 28.7 | 26.52 | 26.3 | ||||
| 90 | 26.30 | 27.2 | 26.02 | 25.8 | ||||
| 115 | 25.63 | 26.1 | 25.56 | 25.6 | ||||
| 128 | 25.51 | 25.5 | 25.50 | 25.4 | ||||
| 150 | 5.80 | 25.63 | 25.5 | 6.36 | 25.66 | 25.8 | ||
| 175 | 14.12 | 26.29 | 25.7 | 13.80 | 26.26 | 26.6 | ||
| 190 | 19.11 | 26.99 | 26.2 | 18.27 | 26.85 | 27.5 | ||
| 215 | 27.42 | 28.73 | 26.8 | 25.71 | 28.30 | 31.0 | ||
| 240 | 35.73 | 31.41 | 28.0 | 33.15 | 30.46 | 31.5 | ||
| 255 | 40.72 | 33.65 | 31.0 | 37.62 | 32.19 | 32.0 |
Table 11. Model prediction, measured length and residual per steering command . Values with cm are in bold.
The results show that the mean model residual was only cm for the right wheel and cm for the left wheel, with no statistically significant mean bias ( and , respectively). The RMSE was 2.09 cm for the right wheel and 1.09 cm for the left wheel, while 21 of the 26 measurements (81%) were within the cm measurement tolerance (Table 12).
Although the average error was small, the residuals showed a systematic dependence on steering angle, as shown in Figure 41. The right-wheel error increased with steering angle (), while the left-wheel error showed an opposite trend (); the fitted slopes and correlations are listed in Table 12. This indicates that the main deviation is not purely random and is likely related to a small offset between the calibrated and actual wheel angles. Consistently, fitting a single angular offset per wheel reduces the RMSE from 2.09 to 1.21 cm (right wheel) and from 1.09 to 0.80 cm (left wheel), with best-fit offsets of and , respectively (Table 12).
Overall, the measured wheel directions agree with the bicycle-model geometry within the measurement tolerance for most setpoints, with a systematic residual consistent with small angular calibration offsets. The bicycle model is therefore used as an approximate model. This experiment checks each wheel separately; whether the vehicle as a whole follows circular arcs is examined in the next subsection through the Ackermann condition.
Implication for control. Since , the sensitivity
is about 0.9–1.2 cm per degree over the calibrated extremes of Table 14, i.e. by roughly 3.5% of . This holds in both steering directions: at the negative limit (, right wheel) cm, while at the positive limit () cm. An angular offset of 3–5°, as suggested by Figure 41, therefore changes by roughly 3–6 cm.
| Wheel | Extreme angle (deg) | Setpoint | (cm) |
| Right | |||
| Right | |||
| Left | |||
| Left |
Table 14. Estimated minimum turning radius for each front wheel at both extreme calibrated steering angles ( at and at ) over the tested command range ().
Experimental Validation of the Ackermann Model
Although the vehicle was designed and controlled under the assumption of an Ackermann steering model, experimental testing revealed significant deviations between the predicted and observed vehicle motion. Therefore, the validity of the Ackermann model for the present vehicle configuration was investigated experimentally.
The calibration in this section validates the steering mechanism through the equivalent single-track angle of Eq. (50). This quantity can be computed for any pair of wheel angles, including parallel (non-Ackermann) steering, so agreement between and the commanded angle shows that the equivalent angle is calibrated. It does not show that the two wheels obey the Ackermann condition. Likewise, the observation that the inner and outer wheels turn by different angles shows only that the wheels are not parallel; it does not show that the difference is the correct one. The geometric Ackermann condition relates the two angles to the vehicle dimensions:
With the front track width cm and the wheelbase cm, the required value is .
The calibrated mean wheel angles of the 13 setpoints () were used, with an angular uncertainty of about per wheel (see the calibration above). Each setpoint is plotted against the servo commanded angle, which spans to ( corresponds to , corresponds to and corresponds to . The quantity plotted is the absolute value (Eq. 75) of the difference between the cotangents of the left and right wheel angles, which equals . Ideal Ackermann steering gives the constant value on both sides of neutral and parallel steering gives 0. The absolute value does not show which wheel turns more; this sign is kept in Table 13, where a negative value means that the outer wheel turns more than the inner wheel. The inner wheel was taken as the right wheel for positive commands and the left wheel for negative commands, consistent with the smaller right-wheel radius at in Table 14. For each setpoint the measured value was compared with , and the outer-wheel angle required by the Ackermann condition was computed from the measured inner-wheel angle. The Ackermann ratio is the measured value divided by : 100% is ideal Ackermann and 0% is parallel steering. Setpoints with a wheel angle below (servo commands from about to ) are excluded from the chart and the table, because the difference of two cotangents is dominated by noise near straight-ahead. Table 13 and Figure 42 summarise the comparison. Figure 42 is the main result: it shows the measured against the servo command together with the ideal Ackermann line (0.706) and the parallel-steering line (0).
| (°) | (°) | Required (°) | Measured (°) | Ackermann (%) | ||
| 0 | −38.3 | −44.1 | −0.234 ± 0.06 | 26.9 | 44.1 | −33 |
| 25 | −30.9 | −35.8 | −0.286 ± 0.08 | 22.8 | 35.8 | −41 |
| 50 | −23.4 | −27.5 | −0.385 ± 0.14 | 18.3 | 27.5 | −55 |
| 75 | −16.0 | −19.1 | −0.614 ± 0.28 | 13.4 | 19.1 | −87 |
| 190 | +18.3 | +19.1 | +0.142 ± 0.24 | 15.6 | 18.3 | +20 |
| 215 | +25.7 | +27.4 | +0.149 ± 0.12 | 20.8 | 25.7 | +21 |
| 240 | +33.2 | +35.7 | +0.141 ± 0.08 | 25.5 | 33.2 | +20 |
| 255 | +37.6 | +40.7 | +0.136 ± 0.06 | 28.2 | 37.6 | +19 |
Table 13. Measured versus required Ackermann geometry. A negative sign means the outer wheel turns more than the inner wheel.

Figure 42. Measured (blue, with uncertainty bars) against the servo commanded angle from to . The green line is the ideal Ackermann value and the dashed red line is parallel steering (0). The measured points lie below the ideal line at every command. On the negative side the outer wheel turns more than the inner wheel (negative sign in Table 13), which the absolute value does not show.
At the steering limits the right wheel turns about 8% more than the left wheel at ( against ) and about 15% more at ( against ), and the difference changes linearly with the servo command and crosses zero near (), not at (Figure 43). This pattern is typical of two wheels driven with unequal effective gain plus a small neutral offset (the right wheel reads at ). It is not the pattern produced by a steering trapezoid. The equivalent-angle result of Eq. (50) is therefore still valid for the bicycle model, but the physical mechanism does not implement Ackermann geometry. Consequence. The two front wheels do not roll about a common centre on the rear-axle line, so the outer wheel is partly dragged sideways, most strongly near the steering limits. This is expected to cause tyre slip, a turning radius different from , and different behaviour in left and right turns. These effects were not measured here. Because errors of this kind accumulate in open-loop maneuvers, the feedback correction of § 5.4.3 remains necessary.

Figure 43. Difference between the right and left wheel angles, , against the servo commanded angle (points) with a linear fit. The difference changes linearly with the command () and crosses zero near (), not at .
Turning-Radius Estimate
For the maneuver planning of § 5.2 we convert the validated wheel geometry into a turning radius using the standard bicycle-model relation
applied directly to each front wheel’s own calibrated angle , rather than to the single equivalent bicycle angle of Eq. (50). This gives the physical radius traced by that wheel, which is what determines the space the vehicle occupies while turning.
Why the minimum matters. The space required for a turn is set by the smallest achievable radius, which occurs at the largest steering angle the linkage can reach. Because each front wheel has its own calibrated angle, and the two ends of the steering range do not give the same angle, we evaluate at both extreme calibrated angles of each wheel in Table 11: the minimum angle (, at ) and the maximum angle (, at ), consistent with Eq. (57). The wheelbase tolerance ( cm) and the angular calibration uncertainty (, § 5.5.1) are propagated through Eq. (76) to obtain . The result is reported per wheel and per steering end, giving four radii, because the two wheels and the two steering directions are not interchangeable for this purpose. The results are listed in Table 14 and plotted in Figure 44.

Figure 44. Turning radius over the wheel steering angle –, with the propagated uncertainty band () and the four extreme operating points ( and for each wheel, plotted at ).
Future Work
While the current sensor fusion and adaptive Kalman filter architecture is designed to isolate and reject transient magnetic anomalies, several limitations remain regarding long-term calibration stability and dynamic environmental changes. Future development of the heading estimation and magnetometer calibration pipeline will focus on the following key areas:
Dynamic Motor-Current Compensation. The current magnetometer calibration (Stage 4) is entirely static; it estimates the hard- and soft-iron distortions caused by the physical chassis but does not account for the dynamic magnetic field generated by the vehicle’s own DC motor. In future iterations, we plan to model this motor-induced field. By correlating the commanded motor PWM signal—or data from a dedicated current sensor—with the induced magnetic deviation, this dynamic hard-iron offset can be actively subtracted from the raw AK8963 readings before they enter the calibration buffer.
Autonomous Calibration Recovery. As demonstrated in the magnetic disturbance tests, a persistent, lasting anomaly (such as a strong magnet attached to the chassis or transitioning into a structurally different indoor environment) pollutes the active sliding window. Currently, this forces the Kalman filter into an extended dead-reckoning state and necessitates a manual reset and a full rotation to re-establish the baseline. Future work will investigate dual-buffer or multiple-hypothesis calibration schemes. By maintaining a shadow buffer of the disturbed field, the system could autonomously detect when a permanent shift has occurred and seamlessly transition to the new calibration ellipsoid without requiring human intervention or halting the vehicle.
Passive 3D Ellipsoid Population for Ground Vehicles. Because a planar ground vehicle does not naturally pitch or roll, the -axis of the magnetometer experiences almost zero variance, forcing the system to rely on a constrained 2D calibration mode to avoid singular matrix inversions. This 2D mode is vulnerable to tilt errors if the vehicle encounters uneven terrain. A future improvement will involve harvesting the minor pitch and roll variations caused by longitudinal acceleration, hard braking, and suspension dynamics. By selectively filtering and storing these rare out-of-plane samples over extended driving periods, the algorithm could slowly accumulate enough -axis variance to passively solve the full 3D ellipsoid without requiring the operator to manually tumble the vehicle in 3D space.
Full Localization. The set of sensors, actuators, and perception pipeline are theoretically enough to evaluate a robot full state: . This will be developed in the future.
Additionally in the control section, because the Ackermann condition is not satisfied, the bicycle model describes the real vehicle less accurately than assumed: the path actually followed for a given steering command differs from the one predicted by , and the difference is not the same for left and right turns. Any algorithm whose result depends on a model-based prediction of the vehicle motion therefore inherits this error. For the current mechanism we recommend relying on feedback wherever possible and keeping model-based algorithms to a minimum.
If the vehicle is to keep this steering structure, a more accurate motion model should be identified from data instead of derived from geometry. We therefore propose to record the motion of the vehicle (displacement and rotation) in many experiments with different steering commands, speeds and turn directions, and to train a neural network on this supervised dataset to predict the motion from the vehicle state and command. Such a model would include the left–right asymmetry and wheel slip that the bicycle model does not contain, and could replace the bicycle model in the planner. This was not performed in this work.
Acknowledgments
The authors wish to note that the submission of this manuscript was delayed due to prolonged disruptions to research activities caused by civil unrest and subsequent military conflict in Iran. These circumstances significantly restricted access to laboratory facilities and delayed manuscript preparation. The work presented here corresponds to the robotic system as completed in early 2025.
Additionally, the authors used AI language models to assist with English-language editing, LaTeX formatting, and manuscript preparation, owing to limited fluency in English. All technical content, experiments, data, and conclusions are the authors’ own, and the authors have reviewed the text and take full responsibility for its contents.
Author Contributions
| Name(s) | Role and Contribution Summary |
| Marya Sabouri Dodaran | Design of motion-geometry-based maneuvering algorithms for sign and signal response; experimental calibration and validation of the vehicle kinematic model; contributions to lane-keeping and cruise control |
| Ali Taheri Garakani | Implementation of AI-based lane detection model, and model data post-processing |
| Mandana Shabanzadeh Nasrollah-Abadi | Electronics, hardware design and integration |
| Yousef Ashrafi | ROS architecture and system communication |
| Mohammad Mahdi Azimi | Embedded firmware design, sensor data handling, heading estimation algorithm, actuator interfacing and control, and inter-processor communication system design |
| Erfan Nekouei Ghahfarokhi | Design and training of AI-based lane detection model |
| Amirhossein Kheiri Holighi | Sign and traffic light detection (vision algorithms) |
| Seyyed Sobhan Hosseini Hajibakandeh | Implementation of control algorithms to maintain lane position and prevent vehicle drift |
| Matin Jamshidi | Mechanical chassis, car body and drivetrain system design |
| Amirhossein Gholizadeh Behbahani | Implementation of an adaptive cruise control system for vehicle speed regulation under various conditions and collision prevention |
| Mohammad Hossein Moaiyeri | Team supervision |
Table 15.
References
- [1]A. Kheiri Holighi, S. S. Hosseini Hajibekandeh, A. Gholizadeh Behbahani, K. Khatibi, S. Najafi Shabestari, G. Ghoreshi, A. Dadnavi, S. Sadeghi, S. Karimi Maksous, M. Jamshidi, M. Shabanzadeh Nasrolah Abadi, and M. H. Moayeri, “Technical report of 1:10 scale autonomous vehicle robot,” arXiv preprint arXiv:2410.11746, 2024.arxiv.org/abs/2410.11746
- [2]Espressif Systems, ESP32 Technical Reference Manual, 2023.
- [3]NVIDIA, Jetson Orin NX Developer Kit User Guide.
- [4]M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng, “ROS: an open-source Robot Operating System,” in ICRA Workshop on Open Source Software, vol. 3, no. 3.2, 2009.
- [5]R. Barry, Mastering the FreeRTOS Real Time Kernel: A Hands-On Tutorial Guide. Real Time Engineers Ltd., 2016.
- [6]STMicroelectronics, VL53L1X: Time-of-Flight Long Distance Ranging Sensor with Advanced Ranging Technology, Datasheet.
- [7]InvenSense, MPU-9250 Product Specification, Rev. 1.1, 2016.
- [8]Devantech Ltd., CMPS14 Tilt Compensated Compass Module, Technical Documentation.
- [9]STMicroelectronics, L298: Dual Full-Bridge Driver, Datasheet.
- [10]NXP Semiconductors, UM10204: I²C-bus Specification and User Manual, Rev. 7.0, 2021.
- [11]R. E. Kalman, “A new approach to linear filtering and prediction problems,” Journal of Basic Engineering, vol. 82, no. 1, pp. 35–45, 1960.DOI
- [12]G. Welch and G. Bishop, “An introduction to the Kalman filter,” Dept. Comput. Sci., Univ. North Carolina at Chapel Hill, Tech. Rep. TR 95-041, 1995.
- [13]D. Simon, Optimal State Estimation: Kalman, H∞, and Nonlinear Approaches. Hoboken, NJ, USA: Wiley, 2006.
- [14]J. F. Vasconcelos, G. Elkaim, C. Silvestre, P. Oliveira, and B. Cardeira, “Geometric approach to strapdown magnetometer calibration in sensor frame,” IEEE Trans. Aerosp. Electron. Syst., vol. 47, no. 2, pp. 1293–1306, 2011.DOI
- [15]V. Renaudin, M. H. Afzal, and G. Lachapelle, “Complete triaxis magnetometer calibration in the magnetic domain,” J. Sensors, vol. 2010, Art. no. 967245, 2010.DOI
- [16]A. Fitzgibbon, M. Pilu, and R. B. Fisher, “Direct least square fitting of ellipses,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 5, pp. 476–480, 1999.DOI
- [17]G. H. Golub and C. F. Van Loan, Matrix Computations, 4th ed. Baltimore, MD, USA: Johns Hopkins Univ. Press, 2013.
- [18]Asahi Kasei Microdevices, AK8963: 3-Axis Electronic Compass, Datasheet MS1356-E-02, Oct. 2013.
- [19]L. Keselman, J. I. Woodfill, A. Grunnet-Jepsen, and A. Bhowmik, “Intel RealSense stereoscopic depth cameras,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops (CVPRW), 2017, pp. 1267–1276.arxiv.org/abs/1705.05548
- [20]Y. Ko, Y. Lee, S. Azam, F. Munir, M. Jeon, and W. Pedrycz, “Key points estimation and point instance segmentation approach for lane detection,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 7, pp. 8949–8958, 2022.arxiv.org/abs/2002.06604
- [21]F. Yu and V. Koltun, “Multi-scale context aggregation by dilated convolutions,” in Proc. Int. Conf. Learning Representations (ICLR), 2016.arxiv.org/abs/1511.07122
- [22]B. De Brabandere, D. Neven, and L. Van Gool, “Semantic instance segmentation with a discriminative loss function,” arXiv preprint arXiv:1708.02551, 2017.arxiv.org/abs/1708.02551
- [23]TuSimple, “TuSimple lane detection benchmark,” 2017. [Online]. Available: https://github.com/TuSimple/tusimple-benchmark
- [24]R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 2004.DOI
- [25]R. Rajamani, Vehicle Dynamics and Control, 2nd ed. New York, NY, USA: Springer, 2012.
- [26]L. E. Dubins, “On curves of minimal length with a constraint on average curvature, and with prescribed initial and terminal positions and tangents,” American Journal of Mathematics, vol. 79, no. 3, pp. 497–516, 1957.DOI
- [27]B. Paden, M. Čáp, S. Z. Yong, D. Yershov, and E. Frazzoli, “A survey of motion planning and control techniques for self-driving urban vehicles,” IEEE Trans. Intell. Veh., vol. 1, no. 1, pp. 33–55, 2016.arxiv.org/abs/1604.07446
- [28]K. J. Åström and T. Hägglund, Advanced PID Control. Research Triangle Park, NC, USA: ISA, 2006.
- [29]S. Thrun et al., “Stanley: The robot that won the DARPA Grand Challenge,” Journal of Field Robotics, vol. 23, no. 9, pp. 661–692, 2006.
- [30]G. M. Hoffmann, C. J. Tomlin, M. Montemerlo, and S. Thrun, “Autonomous automobile trajectory tracking for off-road driving: Controller design, experimental validation and racing,” in Proc. American Control Conf. (ACC), 2007, pp. 2296–2301.DOI
- [31]R. C. Coulter, “Implementation of the pure pursuit path tracking algorithm,” Robotics Institute, Carnegie Mellon Univ., Pittsburgh, PA, USA, Tech. Rep. CMU-RI-TR-92-01, 1992.
- [32]J. M. Snider, “Automatic steering methods for autonomous automobile path tracking,” Robotics Institute, Carnegie Mellon Univ., Pittsburgh, PA, USA, Tech. Rep. CMU-RI-TR-09-08, 2009.
- [33]S. S. Shapiro and M. B. Wilk, “An analysis of variance test for normality (complete samples),” Biometrika, vol. 52, no. 3/4, pp. 591–611, 1965.DOI
- [34]B. Efron, “Bootstrap methods: Another look at the jackknife,” The Annals of Statistics, vol. 7, no. 1, pp. 1–26, 1979.
- [35]J. M. Bland and D. G. Altman, “Statistical methods for assessing agreement between two methods of clinical measurement,” The Lancet, vol. 327, no. 8476, pp. 307–310, 1986.pubmed.ncbi.nlm.nih.gov/2868172
- [36]D. Lakens, “Equivalence tests: A practical primer for t tests, correlations, and meta-analyses,” Social Psychological and Personality Science, vol. 8, no. 4, pp. 355–362, 2017.