The Sound and Alerts system in openpilot is responsible for providing real-time audio-visual feedback to the driver. This system ensures that the driver is aware of the system's state (engagement, disengagement) and critical safety warnings. It consists of micd for ambient noise monitoring, selfdrived for alert logic and event management, soundd for audio playback, and the UI for visual rendering.
The audio-visual alert pipeline follows a distributed model where events are detected in various daemons, aggregated in selfdrived, and finally rendered by soundd (audio) and the UI (visual).
The following diagram illustrates how an event transitions from detection to an audible and visual alert.
Alert Trigger and Rendering Pipeline
Sources: openpilot/selfdrive/selfdrived/events.py81-98 openpilot/selfdrive/ui/soundd.py134-143 openpilot/selfdrive/controls/controlsd.py41-43
micd)The micd process monitors the device's microphone to calculate ambient noise levels. This data is used by soundd to dynamically adjust the volume of alerts, ensuring they are audible over road noise without being unnecessarily loud in a cabin.
soundPressure messages containing weighted decibel (dB) levels openpilot/system/micd.py64-69Sources: openpilot/system/micd.py11-34 openpilot/system/micd.py72-96
soundd)soundd is the central audio playback engine. It manages a library of .wav files and coordinates playback based on the selfdriveState message.
soundd subscribes to soundPressure from micd. It uses a FirstOrderFilter to smooth the ambient noise input openpilot/selfdrive/ui/soundd.py80 openpilot/selfdrive/ui/soundd.py170-173
AMBIENT_DB (24 dB) and DB_SCALE (30 dB) openpilot/selfdrive/ui/soundd.py25-26AudibleAlert.warningImmediate) ramp from current volume to max volume over 4 seconds (ALERT_RAMP_TIME) openpilot/selfdrive/ui/soundd.py21 openpilot/selfdrive/ui/soundd.py178-181The system maps AudibleAlert enums to specific audio files in sound_list openpilot/selfdrive/ui/soundd.py36-50
| AudibleAlert | File Name | Play Count |
|---|---|---|
engage | engage.wav | 1 |
disengage | disengage.wav | 1 |
refuse | refuse.wav | 1 |
prompt | prompt.wav | 1 |
promptDistracted | prompt_distracted.wav | Infinite (None) |
warningSoft | warning_soft.wav | Infinite (None) |
warningImmediate | warning_immediate.wav | Infinite (None) |
Sources: openpilot/selfdrive/ui/soundd.py36-55 openpilot/selfdrive/ui/soundd.py134-143
Alerts are encapsulated in the Alert class and managed by the Events class in selfdrived.
Alert ClassThe Alert class defines the properties of a notification openpilot/selfdrive/selfdrived/events.py115-141:
alert_text_1 (primary) and alert_text_2 (secondary).visual_alert (HUD icons like steerRequired).audible_alert (mapped to soundd assets).IntEnum (LOWEST to HIGHEST) used to determine which alert to display when multiple events occur openpilot/selfdrive/selfdrived/events.py29-35Events are represented by the OnroadEvent message. Daemons like controlsd or dmonitoringmodeld populate these events based on vehicle state or driver state openpilot/selfdrive/controls/controlsd.py41-43
Code Entity Relationship: Events to Alerts
Sources: openpilot/selfdrive/selfdrived/events.py56-112 openpilot/selfdrive/selfdrived/events.py153-205
Standard alert templates include:
AudibleAlert.warningImmediate openpilot/selfdrive/selfdrived/events.py180-185AudibleAlert.warningSoft openpilot/selfdrive/selfdrived/events.py165-170AudibleAlert.refuse openpilot/selfdrive/selfdrived/events.py153-163The UI process renders visual alerts. Additionally, specialized components like the DriverCameraDialog can trigger audio feedback during the driver monitoring (DM) preview.
preAlert, promptDistracted, and warningImmediate respectively openpilot/selfdrive/ui/mici/onroad/driver_camera_dialog.py108-112Sources: openpilot/selfdrive/ui/mici/onroad/driver_camera_dialog.py102-116 openpilot/selfdrive/ui/mici/onroad/driver_camera_dialog.py118-150
soundd includes a safety mechanism check_selfdrive_timeout_alert. If it stops receiving selfdriveState messages for more than 5 seconds (SELFDRIVE_STATE_TIMEOUT) while the system is enabled, it triggers an immediate warning openpilot/selfdrive/ui/soundd.py57-64 openpilot/selfdrive/ui/soundd.py138-140
AMBIENT_DB of 24 openpilot/selfdrive/ui/soundd.py25NoEntryAlert and uses specific StartupAlert layouts openpilot/selfdrive/selfdrived/events.py158-160 openpilot/selfdrive/selfdrived/events.py206-210VOLUME_BASE and specific engage/disengage .wav files openpilot/selfdrive/ui/soundd.py29-31 openpilot/selfdrive/ui/soundd.py51-55Sources: openpilot/selfdrive/ui/soundd.py22-33 openpilot/selfdrive/selfdrived/events.py158-163 openpilot/selfdrive/ui/tests/test_soundd.py11-31
Refresh this wiki