Using optical sensors to resolve vehicle heading issues

A control system (100) fuses different sensor data together to determine an orientation of a vehicle (50). The control system (100) receives visual heading data for the vehicle (50) from a camera system (102), global navigation satellite system (GNSS) heading data from a GNSS system (108), and inertial measurement unit (IMU) heading data from an IMU (110). The control system (100) may assign weights to the visual, GNSS, and IMU heading data based on operating conditions of the vehicle (50) that can vary accuracy associated with the different visual, GNSS, and IMU data. The control system (100) then uses the weighted visual, GNSS, and IMU data to determine a more accurate vehicle heading.