The unpleasant consequences of an insect sting allergy could result in a life-threatening situation which occurs within a few minutes without medical help. The most common triggers of threatening insect venom allergies are the honey bee and wasps. However, mosquitoes and horseflies can cause severe swelling at the site of the bite, sometimes lasting for several days. Pathogenetically, they are based on allergic reactions to proteins of the insect saliva, which lead to an immediate type reaction in about 75% of the population and to a late type reaction in about 50%. Around 20 percent of Germany is allergic to insect bites caused by bees, wasps or mosquitoes. Up to 3.5% of the population (approx. 3 million people) react with systemic reactions to an insect bite, and up to 25% (21 million) experience increased local reactions (swelling larger than 10 cm in diameter for longer than 24 hours). (https://cdn.aerzteblatt.de/pdf/109/13/m238.pdf)
Different types of venom require relevant treatment. The type of venom might differ from species to species even though they belong to the same family.
- Severity I: itching, flushing, hives.
- Severity II: Cardiovascular symptoms, low blood pressure, gastrointestinal symptoms.
- Severity III: Shock, severe respiratory distress, clouding of consciousness.
- Severity IV: Respiratory arrest and circulatory arrest.
Therefore, it is crucial, not only to define the family but also the species of the insect in an effort to apply the right treatment.
Although there are vaccines and medication to protect against diseases such as malaria and yellow fever, the best protection against these diseases is to protect yourself against their bites. However, for some diseases such as dengue there is no prevention or treatment method thus the best protection against these diseases is to protect yourself against their bites.
Implementation of the Klangio Technology
Each insect species has a specific sound characteristic. This characteristic is determined by wing beat and thus by size, speed and body shape of the insect. Basic frequency wing beat of a mosquito is between 400Hz and 600Hz which can measure up to 1200Hz during mating. This can be applied for every species as there is a specific range of frequency for every type of insect family and species thus it is possible to differentiate them through audio.
Klangio offers experience in the fields of audio event, scene analysis and signal processing. The technology is used to retrain the net by using microphone recordings or publicly available data sets to detect the number and type of insects in the environment. It is not only important to build a deep learning architecture to represent the data, but also how the model is trained. In order to classify audios from insects with a noisy background, the model is trained through a series of insect audio and sounds from white noises simultaneously.
We process raw audio data with a log-mel spectrogram as input to our architecture, and it passes through the ResNeSt50 backbone. Deep Learning has shown brilliant performance in the audio domain. It can catch numerous patterns of target classes correctly in the time-series data. The more important point is the environment and data matter in the research. The environments like fields or the middle of the mountains make batches of noises interfering with the insect sounds. Consequently, it is needed to build a noise-robust, multi-label audio classifier.
For the purpose of preventing insect bites and its consequences, the deep learning technology is used as a mobile application to optimize and ease the daily usage. The user is notified simultaneosly and warned about the danger. This information consists of the number, distance and type of the insect.
- Processing directly to the cell phone → no upload to the cloud.
- Analysis of the audio stream permanent → no storage of the recording.
In principle, three deployment modes are conceivable:
- Pull-To-Refresh: when Radar is open, a refresh can be triggered manually. It will be obeyed only if desired.
- Radar mode: When the mosquito radar is open, the update is triggered automatically every 5 seconds. The microphone is only active when the radar is open.
- Background mode: The app listens in the background, even if the mosquito radar is not open. Listening can be activated and deactivated in the settings.
Audio examples: https://www.youtube.com/watch?v=ccmAp6PBdDI
In case there is an insect threat nearby, the efficient algorithm will allow you to take the necessary cautions and prevent any undesired consequences. Stay tuned for further information considering the development of the technology as we continue developing this technology.