Onboard perception
Onboard perception turns camera input into observations the system can use. Detection and tracking identify objects and follow them across frames; vision-language models add descriptions and contextual interpretation. Geolocation mapping then connects those observations to positions, allowing the perception pipeline to feed control commands.
How it connects
- Camera input
- Detect objects
- Track targets
- Interpret the scene
- Map locations
- Inform control
Follow objects from image to location
The perception pipeline moves from camera input through a detection model and a multi-target tracker to scene analysis, geolocation mapping, and control commands. Detection targets include people, vehicles, boats, aircraft, buildings, infrastructure, animals, equipment, debris, and fire or smoke. Tracking maintains the connection between detections across frames so subsequent analysis can follow objects within the scene.
Give visual observations meaning
A vision-language model, or VLM, combines visual input with language to describe a scene and interpret activity. Its architecture connects a vision encoder, a projection layer, and a language decoder within an edge-optimized runtime. The resulting descriptions support recognition of construction activity, crowds, and incidents, as well as explanations of why a detection is unusual in context.
Read changes across frames
Multi-frame reasoning compares observations over time to detect changes and follow how an event develops. Structured tagging records objects, counts, environmental context, and image quality alongside natural-language descriptions. Together, these outputs let scene analysis describe an activity and its progression, rather than reducing each frame to an isolated list of detected objects.