Detection of keystrokes by sound is achieved by placing a listening device close to the target
Every time I am quite impressed by the methods that are discovered and/or developed both to obtain information and to access X section, hack X device and it is that until now the ones that still fascinate me a lot are those that are based on the sounds generated by the CPU fan to obtain information, as well as the use of the to see through walls, among many others.
That is why, personally, I really like to share this type of news here on the blog and in the case of Today I will share the news about the development of a method that created a team of researchers who It is based on detecting the information entered on the keyboard 95% accurate when analyzing the sound of keystrokes recorded by a nearby smartphone or picked up by a nearby microphone.
Proven input detection accuracy outperforms all acoustic analysis methods character per symbol previously known They do not use a language model. The proposed method can be used, for example, to determine entered passwords or typed messages, in a situation where the attacker placed his smartphone next to the victim or received a sound recording while entering confidential information (for example, when during communication the victim logs in with a password for some information systems).
With recent developments in deep learning, the ubiquity of microphones, and the rise of online services via personal devices, acoustic side channel attacks present a greater threat to keyboards than ever before.
The input is recreated using a classifier based on a machine learning model that takes into account sound characteristics and volume level when different keys are pressed.
It is mentioned that to carry out an attack, preliminary training of the model is required, which requires matching the input sound with information about the keys being pressed. Under ideal conditions, the model can be trained using malware installed on the attacked computer, making it possible to simultaneously record sound from a microphone and intercept keystrokes.
In a more realistic scenario, the data needed to train the model could be collected by matching input text messages with audio from a set recorded as a result of a video conference. Input detection accuracy when training a model based on Zoom and Skype video conferencing input analysis decreases slightly to 93% and 91,7%, respectively.
In an experiment to train a machine learning model using audio from a Zoom conference, each of the 36 keys (0-9, a-z) on the keyboard was pressed 25 times in a row with different fingers and with different force.
The data about the sound of each press was transformed into an image with a spectrogram reflecting the change in frequency and amplitude of sound over time
spectrograms transferred for training to a classifier based on the CoAtNet model, used for the classification of images in artificial vision systems. That is, during training the image is compared with the spectrogram of each keystroke with the name of the key. To determine the keys pressed by the sound, the CoAtNet model returns the most likely key based on the transmitted spectrogram, similar to returning the most likely label when recognizing objects by their image.
In the future, the researchers intend to explore the possibility of recreating keyboard input by recording sound from smart speakers and, to improve the accuracy of determining input text, use a language model that classifies input. in the context of whole words.
Finally if you are interested in knowing more about it, you can check the details in the following link