1) Data simulation algorithm of disaster emergency situations
Figure 1. Data simulation.
•
We build algorithms using pyroom-acoustics library that simulate the multi-channel waveform acquired on the flying drone.
Figure 2. Design of drone for test environment
•
We designed drones with a multi-channel mic array for testing environmental data acquisition and testing our model on real-world scenarios.
2) Multi-channel multi-class classification and DoA estimation
Figure 2. Classification and DoA estimation
•
We build multi-channel classification models to detect and classify emergency rescue sounds (class: male, female, baby).
•
Also, we build multi-channel Diection-of-Arrivial (DoA) models to estimate the direction of the sound sources
•
The models should be robust to drone noise and the status of the drone (hovering or moving).
•
A low-flops regime for drone-integrated model scenarios is also one of the major parts of the project. We test our algorithm on NVIDIA Jetson devices considering lightweight hardware costs.