Today, almost all general-purpose computing devices contain a speaker and microphone. We propose utilizing these ubiquitous sensors to bring novel sensing abilities to devices without extra or special hardware. Specifically, we emit inaudible frequency sweeps using the built in speaker, and capture the reflected waveforms using the built in microphone. These sounds are then analyzed and classified to infer some aspect of the environment.

Our work was originally inspired by the field of soundscape ecology. We learned that different environs (natural or otherwise) have significantly different acoustic properties— the composition and spatial configuration of flora and geological features act as acoustic filters, passing some sounds while attenuating others. This can introduce selective pressure, leading to e.g., animals communicating using frequencies that carry farthest in their particular habitats.

This inspired us to run experiments in our lab, sweeping through a range of frequencies emitted from a laptop, and looking at attenuation due to the environment. We noticed that changes in physical configuration could alter the signal, for example, a door being opened or the lid of our laptop being repositioned; it can even capture subtle relationships between wearable devices and the human body, such as knowing when headphone buds are removed from the ears. Such actions appear to expose different facets and/or materials, with different acoustic reflectance characteristics. Additionally, in enclosed “chambers”, such as the ear canal, cars, rooms and similar, certain frequencies may become standing waves, with amplifying effects. Finally, a sound wave reflected off multiple surfaces will return to the microphone at different phases, producing characteristic interference effects. These effects all interact to produce a “sound signature”, which can be used for classification, and in turn, power unique interactive applications. 

Research Team: Gierad Laput, Xiang ‘Anthony’ Chen, Chris Harrison