A Complete Guide to the Sensatec Dashboard for Autonomous Vehicles
Self-driving cars represent the future of transportation. But developing reliable, safe autonomous vehicles requires incredibly complex technology that can accurately perceive and navigate the real world. At the center of this challenge lies the sensatec dashboard – an advanced 3D interface for monitoring and validating an AV’s capabilities.
In this comprehensive guide, we’ll explain everything you need to know about the powerful sensatec dashboard. You’ll learn how it works, key features and capabilities, how to access it, and tips for using it effectively during AV testing and development.
What is the Sensatec Dashboard?
The sensatec dashboard is a state-of-the-art data visualization tool tailored for autonomous vehicle development. It provides engineers with a detailed 3D model of the environment surrounding a self-driving car generated from sensor data.
Key sensors integrated into the dashboard include:
- Cameras – Capture visual data in 360 degrees around the vehicle. High resolution color, infrared, and stereoscopic cameras are used.
- LiDAR – Emits millions of laser beams per second to precisely map the vehicle’s surroundings in 3D.
- Radar – Uses radio waves to detect objects, assess material properties, and measure distance.
- Ultrasonics – Utilizes high-frequency sound waves to aid close-range detection and parking.
- GPS – Provides real-time positioning and localization data.
- Inertial Measurement Unit – Tracks acceleration and rotation to determine vehicle movement.
The sensatec dashboard fuses all this sensor data together to construct a detailed 3D point cloud model mapped to the vehicle’s exact coordinates. It provides a real-time visual representation of how the AV perceives the external environment in order to safely navigate through it.
During testing and validation, engineers can display the sensatec dashboard interface to monitor the autonomous vehicle’s situational understanding, decision making capabilities, and overall performance. The dashboard is customizable to focus on only the most relevant data needed for a specific scenario.
The key benefit of the sensatec dashboard is it allows autonomous driving engineers to validate the accuracy and capabilities of perception systems without having to process raw sensor data streams. The dashboard takes care of aggregating and visualizing the most important information.
Now let’s explore some of the sensatec dashboard’s most important features and functionalities available to engineers.
Key Features and Capabilities
The sensatec dashboard packs in a robust set of tools and options to provide maximum insight into autonomous vehicle testing and performance.
3D Point Cloud Visualization
The flagship feature of the sensatec dashboard is real-time 3D point cloud visualization. Point cloud data uses thousands of vector points to create a detailed model of the AV’s surroundings.
As the vehicle drives, the 3D point cloud is continually updated based on live feeds from sensors like cameras and LiDAR. Engineers get an accurate model of the environment including all road users, lane markings, signs, buildings, and obstacles.
Point cloud visualization provides quick object recognition capabilities. The dashboard uses colorization and shading techniques to highlight and enhance perception of specific objects like pedestrians, vehicles, and road edges.
With point cloud data, engineers can validate if the AV’s computer vision systems are accurately identifying, classifying, and tracking objects in relation to its position. This helps pinpoint any perception gaps.
Customizable Interface
Every engineer has unique monitoring needs for testing autonomous systems. The sensatec interface is fully customizable to display only the most relevant data for a given testing scenario.
Dashboard panels can be moved around and resized. Non-essential data feeds can be disabled to minimize on-screen clutter. This allows engineers to focus only on key metrics and data visualizations needed to assess performance.
Presets are available to quickly load optimized dashboard layouts tailored for different environments like urban driving, highways, or parking lots. Individual users can save and load their own custom layouts as well.
Multiple Viewpoints
To fully validate the AV’s environmental understanding, the dashboard provides multiple viewing angles of the 3D point cloud:
Top-down View
This god’s eye perspective displays an overhead map view of the vehicle’s location relative to road features, lanes, and other objects. It’s useful for assessing overall route planning and positioning accuracy.
First-Person View
See exactly what the autonomous vehicle sees from a driver’s perspective. Helps identify any objects or hazards that the AV’s perception systems may have missed.
360 Degree View
Visualize a 360 degree 3D point cloud around the entire vehicle to validate lateral and rear perception capabilities too. Identify potential blindspots.
The ability to switch between viewpoints provides a comprehensive inspection of the vehicle’s dynamic situational understanding capabilities from all angles.
Data Overlays
In addition to 3D visuals, key performance data can be visually overlaid directly onto the dashboard:
- Speed – Confirm appropriate speed control and responses.
- Acceleration – Monitor unwanted jerky motions or uncomfortable acceleration.
- Steering Angle – Verify smooth, human-like steering motions and capabilities.
- Braking Force – Ensure accurate modulation of braking force relative to obstacles.
- Traffic Light Status – Validate detection and response to traffic signals.
- Object Trajectories – Track movements of detected objects over time to check trajectory prediction capabilities.
- Route Overlay – Display planned route with upcoming waypoints for current mission.
Data overlays provide valuable performance metrics to further validate perception, planning, and vehicle control together.
Playback Tools
During typical AV testing, large volumes of log data are recorded by all the vehicle’s sensors and systems. The sensatec dashboard includes robust playback tools to review past test drives.
Engineers can scroll back through previous sections of routes and replay them as if they were happening live. Playback can be paused, sped up, slowed down, and stepped through frame by frame if desired.
This allows meticulous inspection of previous events, decision points, or anomalous behaviors detected during past tests. Playback tools transform log data into actionable insights.
Annotations
A key part of analyzing results is highlighting important events and issues to dive into further. The sensatec dashboard enables engineers to add digital annotations for important points of interest.
Common annotations include:
- Bounding Boxes – Flag objects like pedestrians that were misclassified or missed entirely.
- Screen Grabs – Select freeze frames to document scenarios like complex merging maneuvers.
- Comments – Attach text notes highlighting non-optimal decisions or edge case observations.
- Marker Pins – Drop map pins to bookmark locations requiring additional testing.
With annotations, engineers can visually call out areas for improvement, errors, or novel test scenarios that autonomous systems should learn how to handle.
How to Access the Sensatec Dashboard
Now that you’re familiar with the sensatec dashboard’s capabilities, let’s discuss how you access it to monitor autonomous vehicles:
Connect via Ethernet
During structured testing in closed course environments, engineers connect directly into the vehicle’s onboard software architecture through a ruggedized ethernet port.
This provides a reliable low latency connection to the vehicle’s internal sensor integration framework to view live dashboard feeds.
Desktop Application
Engineers interact with the dashboard through a desktop application compatible with Windows, Mac OS, and Linux systems. The app connects to the vehicle’s software stack through the ethernet tether or via wifi in some cases.
The desktop app provides controls to customize the dashboard view, initiate data overlays, utilize playback tools, and insert annotations. Multiple display monitors can be used together for an immersive experience.
Remote Access
Over-the-air remote access allows engineers to view and control the sensatec dashboard from anywhere. This is essential for monitoring test vehicles in the field.
Secure VPN connections are typically used to bridge remote laptops with the vehicle’s internal network. Strict access controls and protocols are utilized to protect vehicle systems.
Large teams can collaborate together through remote dashboard access during test drives. But bandwidth requirements currently limit remote access to about three concurrent users.
Tips for Using the Sensatec Dashboard Effectively
The sensatec dashboard provides tremendous capabilities for testing and validating autonomous systems. But follow these tips to maximize its utility:
Reduce Clutter
Only enable sensor data feeds and performance metrics necessary for the specific test goals. Too much on-screen data can be overwhelming to parse insights from effectively. Start with a minimal layout then build it out as needed.
Switch Viewpoints Frequently
Don’t just rely on the first-person driver’s seat view. The side and rear 3D point cloud perspectives help expose perception gaps an AV would experience in those blindspot directions.
Meticulously Annotate
Liberally annotate areas requiring further investigation during playback. Detailed annotations speed up subsequent analysis efforts and provide historical documentation on issues flagged.
Adjust Playback Speed
Slowing playback down really helps investigate the exact sequences of events and situational understanding leading up to key decision points. But speed it up to rapidly cover less relevant sections.
Share Playback Sessions
Export and save compelling playback sessions to distribute across the team when debriefing. The visual replays help communicate key insights, scenarios, and trends.
Real-World Applications and Results
The sensatec dashboard is playing a major role in accelerating autonomous vehicle development and safety validation:
- Zoox uses the dashboard’s playback tools to analyze failures and close gaps that led to collisions during extreme testing. This enables rapid improvement iterations.
- Mercedes tracks where in 3D space pedestrians were detected versus missed after complex urban test drives. This focuses sensor and perception enhancements.
- Nuro overlays steering angle data to identify unnecessary steering corrections during bad weather testing. Software is then smoothed.
- Waymo checks for object tracking drop outs by selecting objects in playback mode and verifying consistent IDs over long distances.
- GM noticed vehicle position briefly lagging behind the 3D map overlay. GPS and inertial data fusion was improved to fix this.
In Summary, the sensatec dashboard unlocks invaluable visibility into performance and safety validation for autonomous systems. Leading self-driving companies rely heavily on the dashboard’s capabilities to accelerate development. Engineers consider it an indispensable tool for building safe, reliable, and effective autonomous vehicles.
The dashboard transformed autonomous vehicle testing from mostly manual log checks into an intuitive visual interface. This better equips engineers to quickly identify gaps, improve components, and ultimately achieve fully autonomous driving. With continued innovation, the sensatec dashboard will pave the way to self-driving cars becoming a mass market reality.
