Vybz

Overview

Our goal is to create a sound detection system that alerts its users in response to the amount of noise in the room. This is an efficient way for people to gauge the volume level of their environment and maintain the desired vibe or atmosphere in the room. A practical application for this system is for students in dorm rooms hosting social gatherings and parties, the system will alert the user when the sound level of the room has gotten too loud. This device will hopefully reduce the amount of noise complaints from neighbors and visits from RAs. This application will consist of three main components: A microphone to gauge the level of noise in the room, a Raspberry Pi to analyze data and send signals to the speaker, and a website to easily download the python file which runs the program. In addition, the Raspberry Pi will be primed with code gathered from our own knowledge and coding libraries which include an audio spectrum analyzer that works in real time. Lastly, the program will alert the user via text message when the desired noise level is reached. With music being a vital part of today's culture, the Vybz detection system would be useful for keeping the social atmosphere of social gatherings, uninterrupted by the need to check if the volume level is too loud.

Link to log: https://classes.engineering.wustl.edu/ese205/core/index.php?title=Vybz_Log

Link to Project Proposal: https://docs.google.com/presentation/d/1-aceL_Ulm3FMN4_c8DnqJogrqBLV7KN6ToSFz62DLEk/edit?usp=sharing

Proposal PDF: Media:Vybz_Project_Proposal.pdf

Link to Poster: Media: poster.png

Link to our GitHub: https://github.com/bvandersman/vybz/tree/master

Link to our Tutorial: https://classes.engineering.wustl.edu/ese205/core/index.php?title=How_to_Read_Audio_Input_from_a_USB_Microphone_on_the_Raspberry_Pi

Team Members

• Daniel Li
• Isaac Thomas-Markarian
• Benjamin van der Sman
• TA: Sam Chai
• Instructor: Dennis Mell

Objectives

Original:

Originally, our group wanted to not only create a device that detects the noise level in the room, but also create a speaker that adjusts its own output in response to the noise level in the room. This speaker would be self adjusting and would either lower its output when the atmospheric volume was high or raise its output when the volume was low. Although we ultimately ended up changing our project, many key elements of the original remain, such as using the Raspberry Pi and microphone to take in and process raw audio data, implementing an audio spectrum analyzer to find a running set of integers in real time to represent the volume level, and producing a result that changes based on the changes in the environment.

New Objectives:

• Gather resources (speaker, microphone(s), A/D converter, etc...)
• Configure the microphone with an A/D converter so the microphone's analog signal is transformed into a digital one
  • The Raspberry Pi needs a digital signal to read
• Prime the Raspberry Pi to receive and analyze the digital signal using:
  • Our own coding knowledge
  • Newfound Python skills
  • Libraries of code for the Fast Fourier Transform
• Code a running set of integers in real time to represent the volume level
  • Can be gathered through various sound libraries
• Set a volume level threshold that is able to notify the user when reached
  • This will require threading
• Create a visually appealing website that will print out notifications of the sound level
• Send the user a text notification when the desired threshold is reached
• Develop a plan for a demonstration that does not impede on other's project but effectively displays our project
  • Demonstration is in Lopata Gallery
  • There will be approximately 10 other projects in close proximity

Obstacles and Direction Shift

As we progressed through our project, we found it difficult to manipulate and analyze raw audio data, especially when attempting to code a change in output. To manipulate the output of a speaker based on the volume in the room, we would have had to code a sound equalizer that manipulates the amplitude of a sound file in real time. Although it is not hard to find sound libraries that can change the amplitude of a sound file, it is extremely difficult to do it in real time, which was necessary for the success of our project. Since our speaker was to be auto adjusting, we were not able to adjust the volume output in real time. With our limited coding experience and knowledge, by the time we had created our audio spectrum analyzer which produced the sound input, we did not have the expertise or time to build an equalizer of this extent. Thus, we decided to go another direction and create a volume detection system, which analyzes the volume of the room and warns the user if it is loud enough to cause noise complaints. We found this to be a very practical use for an audio spectrum analyzer, as this system could prevent noise complaints from neighbors in not only college dorms, but neighborhoods as well. We developed a website that made the program easy to access, used the Twilio platform to notify the user via text message, and created a project that was not only useful to others but also within our skillsets.

Challenges

• Learn how to program the Raspberry Pi using Python and skills from the tutorial videos
  • This will include learning the basics of Python through Codeacademy and using the tutorials from the ESE205 page
• Access and dissect coding libraries that may hold code that is useful but lies outside of our skill level
  • This will require some digging. We must make sure our code is compatible to our Pi and Linux
• Find and apply the audio spectrum analyzer to create sound profiles of the ambient noise and the speaker's music
  • We will need to show a running set of integers representing sound level
• Establish a communication between the Raspberry Pi and the microphone
  • We need to figure out where the audio data goes once the microphone is plugged in
• Develop website that will show notification
  • The website must be easy to navigate and visually appealing
• Prompting a notification being sent to a phone when the volume reaches a certain level
  • This will require using a cloud communication platform (Twilio)
• Work to develop an effective demo given the location, Lopata Gallery
  • We will have to practice our demo to adjust to the acoustics of Lopata Lobby
  • We will need to adjust our threshold to fit a loud time of day

Gantt Chart

Budget

• Raspberry Pi (Provided)
• Microphone ($9.99)

Total: ~$10

Design and Solutions

Overall Setup:

We used a Raspberry Pi and a USB microphone to detect the sound level in the room. Using the analog signal from the music played, we translated the sound bytes into a series of running integers that represented the volume of sound in real time. We tested that when the level falls within the 70-80 range of the integers, the sound leaks through the walls in WashU campus dorms. In response, we programmed our Raspberry Pi to print out the statement “Turn the volume down. Your RA can hear you!” when integers in the desired range are reached. Because it is unrealistic of the Vybz user to be closely monitoring the program on their computer while they are preoccupied with listening to loud music or hosting a social event, we created an alert system so that the program will send an SMS message to the user when the volume in the room has surpassed the threshold for an elongated period of time via Twilio, a cloud communications platform. To make this system more visually appealing, we designed a website with information about Vybz and a link to download the Python file which contains our executable program.

Recognizing Audio

One of the earliest problems we ran into was getting the Raspberry Pi to recognize the audio signal from the microphone. In order to analyze audio information with a computer like the Pi, we had to convert the analog signal which is the type of signal received by a microphone, and convert it into a digital signal which can be analyzed or manipulated with a computer. This signal transformation can be done with an A/D converter (analog to digital), but instead of purchasing this device, we found a USB microphone with this converter built in. Even with this microphone, it was difficult to find where the signal was being recognized on the Raspberry Pi. One of our first successes in visualizing the digital audio signal was through a program called Audacity, where we used a built in Fast Fournier Transformer (FFT) to visualize the frequency of the signal.

Audio Input through Audacity

While this was a hopeful step in analyzing the signal, we found that Audacity is only able to analyze the signal and create this visual representation after recording. This was a crucial limitation because in order for our device to function as we conceptualized it, the program needed to be able to analyze the audio data in real-time. After extensive searches for real-time audio analysis programs, we came to the conclusion that we would have to build an audio spectrum analyzer for the device to work as intended.

Audio Spectrum Analyzer

A spectrum analyzer is a device that displays signal amplitude as it varies by signal frequency. The amplitude is displayed on the vertical axis and the frequency is displayed on the horizontal axis. There are three main types of spectrum analyzers; the superheterodyne analyzer, the real-time analyzer, and the audio analyzer. A superheterodyne analyzer converts part of the input signal to the center frequency of a band-pass filter and is a common method of building a radio receiver. The conversion is done by running a voltage-controlled oscillator across a range of frequencies which makes it possible for the full frequency range of the device to be analyzed. Real-time analyzers collect real-time frequency range and sample the incoming RF (radio frequency) spectrum over a limited time duration, converting the information using the fast Fourier transform (FFT) algorithm. Because it's real-time data collection, there are no breaks in the calculated RF spectrum. Spectrum analyzers can also be used in the audio spectrum, displaying volume levels of frequency bands audible to humans. This method aims at analyzing the harmonics and characteristics of audio signals. For our project, we incorporated characteristics of the real time and audio analyzers. A fundamental part of many spectrum analyzers is the FFT (short for Fast Fournier Transformer) component. An FFT is a discrete Fournier transform algorithm that samples a signal over a period of time and divides it into its frequency components. The most elementary components these signals can be broken down to are sinusoidal oscillations. The FFT basically takes signals and breaks them down into the individual sine waves that construct them. Once these signals are in this elementary form, they can be manipulated with basic operations for many theoretical purposes. A challenging point in our project came with building a real-time audio spectrum analyzer because the FFT algorithm is used to make most real-time analyzers, but not for audio analyzers. Incorporating the technologies from both types proved difficult. Our solution was to go by a model of a real-time spectrum analyzer that didn't use the FFT component and modify it to create a spectrum analyzer that fit our needs; one that both processed audio and did so in real-time.

Audio Spectrum Analyzer Code and Plot

Integer Conversion

The raw data came out as zeros and ones when we ran our initial code for the audio spectrum analyzer. This was not optimal since our purpose for this data was to compare it to a threshold in an intuitive manner. We found that the best way to navigate this problem was to convert our raw data to integer values. We set the sound bytes into integers by finding the standard deviation of each chunk of sound data that was inputted into the microphone. We modified our code so that this transformation occurred in real-time and didn't affect the speed at which the feedback loop of our program would run. Below is a visual of the array of of integers which refreshes itself at near real-time speeds.

Running Integer Values

GUI Component To create the front-end of our program, we used tkinter, a module in the Python standard library which serves as an interface to Tk, a simple toolkit. provides us with a variety of common GUI elements which we can use to build our interface – such as buttons, menus and various kinds of entry fields and display areas. We used these buttons to allow users to control when to record whether or not they are being too loud. Also, we configured out front-end to display their relative noise level in the form of two simply messages: "You're being too loud your RA can hear you!" and "Party on!".

Twilio

Twilio is a cloud platform that creates a pathway to communicate SMS messages via applications through a globally available API. We realized that it was unrealistic for someone using our device to constantly be monitoring their computer to see if the volume was too loud. This server allowed us to send an alert message to the user's phone when the threshold in our audio spectrum analyzer had been exceeded for certain amount of "time". Our, code, however, did not use time duration to prompt the message alert. Rather, we established a counter so that when the print statement we established in our code earlier ("You're being too loud, your RA can hear you!") printed x amount of times, the SMS would be sent via Twilio, simulating the effect of an internal timer in our program. We thought it fair for this count to be 100 times. Given the refresh speed of our spectrum analyzer, this ended up being the equivalent of roughly 30 seconds.

Attached is a link to Twilio's website: https://www.twilio.com/?pdv=c&pcrid=243726082447&pmt=e&pkw=twilio&campaign=G_S_Brand_Alpha_NA&utm_source=google&utm_medium=cpc&utm_term=twilio&utm_campaign=G_S_Brand_Alpha_NA&utm_content=Brand&gclid=EAIaIQobChMIhuuRzeHn2gIVjcDACh2UiwrHEAAYASAAEgKr3vD_BwE

Website

Our website for Vybz informs viewers of what Vybz is about, some information about our team, and a downloadable file for our executable python program. While we originally intentioned for our program to run inside of the website, complications arose as we found that the only way to run the python program directly from the website was to pay for a server with this ability. Since building the website from scratch using HTML and cascading style sheets (CSS) did not resolve this problem of running our program, we chose to build the website with Weebly.com, sacrificing some of the autonomy to create the website exactly as we wanted it but saving time in the building process and reaching the same result of an elegant website with the downloadable file available on it.

Attached is the link to our website as well as pictures of the main page and the download page.

https://thevybzprogram.weebly.com/

New Skills

Going through this project, there was a lot to learn. Some key takeaways from our semester are:

• Learning to connect the Raspberry Pi to our personal computers
  • Tutorial: https://classes.engineering.wustl.edu/ese205/core/index.php?title=Autostarting_Raspberry_Pi_Using_VNC_Viewer
• Receiving an input through our USB microphone and our Pi
  • Tutorial: https://classes.engineering.wustl.edu/ese205/core/index.php?title=How_to_Read_Audio_Input_from_a_USB_Microphone_on_the_Raspberry_Pi
• Creating an Audio Spectrum Analyzer to Measure Volume
• Designing a Website and Demonstration
• Sending a text message as a result of Python code being run
  • Tutorial: https://classes.engineering.wustl.edu/ese205/core/index.php?title=How_to_send_and_receive_text_messages_using_the_raspberry_pi_and_the_Twilio_API
• General coding techniques in Python and with the Raspberry Pi
• Managing Project Time

Results and Next Steps

video demo

The final results of the Vybz Detection is a device that:

• Incorporates an audio spectrum analyzer
• Transforms individual sound bytes into a string of running integers that could easily be analyzed in real time
• Uses an algorithm that alerts the Vybz user when the volume threshold was passed with a print statement on a GUI screen
• Is accompanied by a website we built with information about the company and a link to download the Python file with our code
• Incorporates an alert system that sent an SMS message to the Vybz user when the volume threshold had been exceeded for a specific duration of time

While we met all of the goals that we set for the Vybz detection system, with additional time to enhance the device, we would:

• Add settings so that the user could calibrate the threshold of the program depending on how far away from their neighbor or whoever else they are concerned about disrupting in a simple way. This would take Vybz to a wider market than just WashU campus dorms which is where we currently have the threshold based on.
• In the future, Vybz could also be a sound detection system that logs the volume levels in any space to reduce noise complaints, and could provide additional information about atmospheric volume for people investing in residential spaces while still holding its original purpose of maintaining the ambience of a party or social event.
• Adjust the audio spectrum analyzer code to be able to run with any microphone provided, as many devices such as laptops, phones, and tablets already have built in microphones.
• Make the Python compatible for all operating systems, as ours is only built to run on Linux devices. This would make the code much more accessible to people with any type of device capable of receiving audio input.
• Redesign the messaging system and create a prompt that will allow anyone to put in their phone number to receive notifications, as right now it only works for one number.

Project Poster