This document describes the openpilot user interface system, which provides the visual interface for both onroad driving and offroad configuration. The UI is built on pyray (Python bindings for raylib) and uses a widget-based architecture for composable UI elements.
The UI supports two primary variants: the "big UI" (MainLayout) optimized for comma 3/3X hardware, and the "mici" UI (MiciMainLayout) designed for the comma 4.
For details on the primary UI framework and rendering orchestration, see Main UI System. For connectivity and settings, see Network and Settings Management. For map integration, see Navigation and Maps. For audio and visual alerts, see Sound and Alerts.
Sources: README.md45-50 openpilot/selfdrive/ui/mici/onroad/driver_camera_dialog.py1-11 openpilot/selfdrive/selfdrived/events.py158-160
The UI system follows a hierarchical architecture with GuiApplication at the core, managing the rendering loop, input handling, and widget rendering. The layout orchestrators manage transitions between different operational modes (home, settings, onroad), while UIState tracks vehicle state and synchronizes with the messaging system.
The following diagram bridges the high-level system concepts to the specific code entities that implement them.
Sources: openpilot/selfdrive/ui/mici/onroad/driver_camera_dialog.py6-11 docs/how-to/turn-the-speed-blue.md34-35 openpilot/selfdrive/selfdrived/events.py10-18
The GuiApplication class is the foundation of the UI system. It manages:
Touch input is handled in a dedicated thread running at a high rate to match the touch controller hardware. The MouseState class polls for touch events and buffers them for the main rendering thread to process during the next frame update.
Sources: openpilot/selfdrive/ui/mici/onroad/driver_camera_dialog.py55-56 openpilot/selfdrive/ui/mici/onroad/driver_camera_dialog.py7-9
All UI elements inherit from the Widget base class, providing a composable architecture. The system uses specialized widgets for complex interactions:
| Widget | Purpose | File |
|---|---|---|
ModelRenderer | Renders predicted path, lane lines, and lead indicators using shader polygons | selfdrive/ui/onroad/model_renderer.py |
CameraView | Handles VisionIPC connection and EGL zero-copy texture rendering | openpilot/selfdrive/ui/mici/onroad/driver_camera_dialog.py14-21 |
AlertRenderer | Displays critical and informational alerts with dynamic sizing | selfdrive/ui/onroad/alert_renderer.py |
HudRenderer | Renders onroad overlays like speed, cruise max, and buttons | docs/how-to/turn-the-speed-blue.md48-60 |
DriverCameraView | Specialized view for driver monitoring alignment | openpilot/selfdrive/ui/mici/onroad/driver_camera_dialog.py14-21 |
The UI utilizes advanced rendering techniques to draw anti-aliased paths and gradients. On supported hardware, CameraView uses VisionIPC for zero-copy frame distribution from the camera daemon.
Sources: openpilot/selfdrive/ui/mici/onroad/driver_camera_dialog.py27-30 docs/how-to/turn-the-speed-blue.md54-60
The UIState singleton subscribes to cereal messages via SubMaster. It tracks critical status such as whether the vehicle is started, the current engagement state, and device-specific parameters.
The MainLayout and MiciMainLayout classes manage the primary transitions between operational states.
Sources: openpilot/selfdrive/ui/mici/onroad/driver_camera_dialog.py6-9 openpilot/selfdrive/selfdrived/events.py20-24
AugmentedRoadView provides the primary driving interface. It combines the camera stream with model overlays. It uses calibration data to project the model's predicted path onto the 2D screen coordinates, ensuring the "augmented reality" lines align with the actual road.
The MiciMainLayout is optimized for the comma 4. It features a different layout structure compared to the comma 3/3X, including specific widgets like the DriverCameraDialog for driver monitoring setup.
Sources: openpilot/selfdrive/ui/mici/onroad/driver_camera_dialog.py23-41 openpilot/selfdrive/selfdrived/events.py158-162
The UI includes built-in tools for monitoring and optimization:
replay tool to test changes without being in a car.hud_renderer.py can be quickly verified by restarting the UI process.Sources: docs/how-to/turn-the-speed-blue.md28-35 docs/how-to/turn-the-speed-blue.md73-76
Refresh this wiki