Dive in Deeper
Born with congenital hearing loss, I experienced firsthand the challenges faced by the deaf and hearing-impaired community. My ability to hear was restored through a cochlear implant — a technology that, while life-changing, can cost over $50,000 for both ears. I created AudioAware to bring an affordable, accessible alternative to those who need it most. With a total hardware cost of under $20, AudioAware delivers real-time sound localization, classification, and visual feedback, bridging the gap between cutting-edge AI research and accessible assistive technology.
How it Works
-
Detect & Localize
-
Four microphones capture sound from all directions.
-
JerryNet determines where the sound is coming from.
-
-
Identify Sound
-
CLAP model matches the sound to its most likely source, even if it’s never been trained on it before.
-
Important sounds like sirens or alarms are prioritized.
-
-
Pinpoint the Object
-
YOLOv9 detects all objects in the camera’s view.
-
SIRA-SSL picks the exact object producing the sound using audio-visual matching.
-
Performance
​AudioAware’s AI pipeline consistently outperforms baseline models in real-world testing. From accurately identifying where a sound is coming from to pinpointing the exact object responsible, each component is optimized for speed and accuracy, completing a full detection cycle in under 0.8 seconds.