Log Entry 1 - 09/28/23

Today, we got together and did research on the history of medical auscultation. We discovered that analog stethoscopes have many issues, namely sound quality, background noise leakage, comfort/fit, hygiene, and compatibility with Electronic Health Records.

We also learned that healthcare professionals can gather a lot of information from auscultation:

• Cardiac issues (pericarditis, murmurs, heart valve leakages, arrythmias, narrow valves (stenosis), regurgitation)

• Pulmonary issues (blockages in the trachea and bronchi, fluid in the lungs, rhonchi (a snoring-like sound) which can be an indication of asthma, pneumonia, chronic bronchitis, or cystic fibrosis)

• Absence of sound can tell just as much as presence of sound

• Heart Rate

• Assess pregnant women who are short of breath

From this, we determined that our project can take 1 of 2 paths:

1. Consumer device for at-home non-medical identification of heart and lung conditions
2. Medical grade device for use by medical staff to aid in diagnoses

After much discussion, we decided to go with option 1, a consumer medical device.

Log Entry 2 - 10/04/23

Today, one of our group members spoke with a Respiratory Therapist and also a retired Biomedical Device Engineer to speak about our problem space and get feedback.

One of the main things we learned was that ease-of-cleaning is a big thing for healthcare professionals. This criteria was amplified during the pandemic. The respiratory therapist told us that due to the need to clean stethoscope diaphragms repeatedly, they were needing to replace the film quite frequently. The retired biomedical engineer told us that telehealth and the ability to store auscultation data would be important.

This was useful, and will help guide our designs.

Log Entry 3 - 10/27/23

Today, we had our preliminary design presentation (PDP). We formally defined our problem statement as follows:

With the decline of auscultation skill amongst healthcare professionals, physical and auditory limitation of stethoscopes and no standard for auscultation data, we propose a digital auscultation device that utilizes computer aided auscultation to aid healthcare professionals to screen and diagnose cardiopulmonary conditions.​

We presented 4 possible designs, which are shown below. We ended up going with Design 3, based on our design selection matrix.



We chose Design 3 because it was the most ergonomic, user-friendly, and easy to clean. It ranked the highest in our design matrix.

We also established our project timeline, which you can find in detail in the sidebar.

Log Entry 4 - 11/10/23

We've been working on detailed design and component selection. We've been considering different chipsets, microphones, and specifics of mechanical design.

We have made high level signal flow block diagrams to show how we're going to take the analog ausculation sounds, convert them into digital signals, and then pass them on to our microcontroller for processing.

Additionally, our solution involves a desktop application to visualize and store auscultation data.

Log Entry 5 - 11/20/23

Today, we had our Final Design Presentation! We showcased our CAD models, component selection, and filter design.

Below is our preliminary CAD of the selected design. For context, this device will be about the size of a 24oz Nalgene bottle.

More details about the CAD and mic interface can be found in the Mechanical subsection of this website.

For our main components, we have the following.

• Microphone: Adafruit I2S MEMS digital microphone

• Microcontroller: ESP32-DEVKITC-32E

More precise reasoning for why we made these selections can be found in their respective sections.

Below, preliminary filter designs are shown. The filters were designed in MATLAB. We will design filters in MATLAB and then, using the Signal Processing Toolbox and MATLAB Coder, we will generate C code from this.

We go into depth about the filter design in its respective section.

Log Entry 6 - 01/17/24

Streaming audio from ESP32 microcontroller to desktop was surprising a challenging task. Initially, we were researching the Advanced Audio Distribution Profile (A2DP). This is a Bluetooth profile that is dedicated to streaming audio. Ideally, we want the sound producer (ESP32) to act as A2DP source and send the audio data, while the consumer (PC) to act as A2DP sink and receive the data. The diagram below shows the dataflow.

Upon further research, we shockingly found out that major operating systems, including Windows and MacOS, do not support A2DP sink! They can send out audio to other devices, but they cannot receive audio from other devices using A2DP protocol. This is intriguing because we know for fact that Bluetooth headsets can get microphone voice onto laptops.

We researched Bluetooth headsets. Most common implementation is as follows: the headset uses A2DP profile when the user is listening to music (one-way audio), but it falls back to the plain old Bluetooth serial port profile (SSP) when the user speaks into the microphone (two-way audio). This explains why the audio quality suddenly drops when you join a video call, since the audio is being sent over SSP, which is not designed for high quality audio.

However, since cardiopulmonary sounds have low frequency, SSP is suited for our purpose and is selected as our profile of use.

Log Entry 7 - 02/10/24

Cardiopulmonary sounds are maximum 3 kHz. By Nyquist’s sampling theorem, we need to sample at least twice as fast (6 kHz) to obtain good results. We did 3 optimizations to achieve this.

1. The microphone on VitalSonics uses I2S communication protocol. From the datasheet, every audio sample is 18 bits in length, which is not very convenient since a byte contains 8 bits. Nonetheless, we found out through experimentation that the sound top 2 bits of audio sample are always 0, and hence we only need to send 16 bits (2 bytes) per sample.
2. We designed the audio buffer size to be close to Bluetooth's maximum packet size. There are some overhead with each Bluetooth transmission. If maximize packet size, we can minimize the number of transmissions, which reduces the total overhead and allows us to stream faster.
3. We found out that on Windows, although Bluetooth shows up as a COM port, it is actually a virtual COM port. The baud rate does not matter. It will accept incoming bytes until its buffer fills up. We set the desktop application to read as fast as possible to prevent data loss.

In the end, we were able to achieve a sampling rate of 20 kHz, which exceeds our goal by more than 3 times. Below is a sample recording.

Log Entry 8 - 03/20/24

The frequency range that the sounds listened to for auscultation are limited. Pulmonary sounds exist in the frequency range of 100-1000Hz, and cardio sounds exist from 0-650Hz. By filtering sounds that do not exist in this frequency range, distracting ambient noise from the surrounding environment and unnecessary sounds of the patient’s body picked up by the microphone can be greatly reduced. This then enables the safe amplification of the unfiltered sounds to aid further with auscultation.

Sixth order lowpass Butterworth filters were implemented directly on the ESP32 board to cut off the upper limit of these frequency ranges. The filters were designed for both the 44.1kHz standalone mode and the 20kHz recording modes. Their bode plots can be seen below.

To ensure proper functionality of these filters, they were tested in isolation from the separate system components. Sin waves over a set frequency range were processed by the filters and the resulting gain was plotted. These plots can be seen below.

Log Entry 9 - 03/21/24

Creating an Audio Streaming and Processing Application with PySide6 and PyQtGraph

In this article, we'll dive into the implementation of an audio streaming and processing application built using PySide6, PyQtGraph, and various other libraries in Python. This application allows users to stream audio data from a connected device, visualize the audio waveform and its frequency spectrum in real-time, and process the streamed audio data.

Let's break down the main components and functionalities of the code:

Initialization and Configuration

The code starts by importing necessary libraries such as PySide6 for GUI development, PyQtGraph for plotting, and other utilities. It also defines configurable parameters such as serial port settings, audio sample rate, chunk size, and file names.

Streaming Thread

A StreamingThread class is defined to handle the streaming of audio data from a connected device. This thread continuously reads audio data from the serial port, updates global audio buffers, and emits signals to update the UI with the latest audio data.

Audio Player

The AudioPlayer class manages audio playback from a WAV file in a Qt application. It utilizes threading for playback, emits signals for UI updates, and provides methods for controlling playback state and seeking to specific positions within the audio file. This class effectively handles audio playback within the application, ensuring smooth user interaction.

Audio Processor

An AudioProcessor class processes received audio data asynchronously from the Audio Player. It processes incoming audio data, updates the global audio buffers, and emits signals to update the UI.

Main Window Class

The MainWindowClass class represents the main window of the application. It initializes the UI components, sets up the streaming and processing functionalities, and handles user interactions.

User Interface

The user interface consists of widgets for displaying the audio waveform and frequency spectrum plots using PyQtGraph. Users can interact with the application through buttons to start/stop recording, load WAV files, and control playback.

Functionality

• Users can start and stop recording audio data from a connected device.
• Audio data is streamed and displayed in real-time on the waveform and frequency spectrum plots.
• Users can load WAV files for playback and visualization.
• Playback controls allow users to pause, resume, and stop audio playback.

Conclusion

In summary, this Python application provides a comprehensive solution for streaming, processing, and visualizing audio data in real-time. It offers a user-friendly interface for controlling audio recording and playback, making it suitable for various audio analysis and processing tasks.

Log Entry 10 - 03/22/24

Making the device battery powered and USB-C rechargeable

The device must be battery powered because it must be portable and not require a wired connection to a laptop. As such, the obvious method to power the microcontroller given the device's physical constraints (must be handheld) is to power it using batteries.

Battery Selection

After comparing various types of rechargeable batteries, it was determined that a LiPo battery would best suit our needs. LiPo batteries are known for being lightweight, yet still able to provide power for a reasonable amount of time. This is evident by the extensive use of LiPo batteries in drones.

For this project, a 3.7V 600mAh battery was selected. The dimensions of the battery are 42mm x 25mm x 8mm, so this is perfectly packaged for our device.

Powering the ESP32-DEVKITC-32E

Reading datasheets for the ESP32 DevKitC revealed a heap of useful information. The ESP32 DevKitC has two voltage input pins, 3.3V and 5V. The devkit does have an LDO (low dropout) voltage regulator on board, but there is a caveat. Input voltage only goes through the LDO regulator if the input voltage is supplied through the 5V input pin.

The LDO voltage regulator is needed to buck the voltage down to 3.3V for the mic input voltage.

Due to the constraint of the LDO regulator and the size constraints of the device preventing us from using a bigger battery, a boost converter was needed.

The idea is as follows: Boost the LiPo battery voltage from 3.7V to above 5V. Feed the 5V+ voltage into the 5V pin on the ESP32 DevKitC. Then, the ESP32 DevKitC will handle the bucking of the voltage down to 3.3V.

For this purpose, a DC-DC battery boost module was purchased, shown below. This boost converter boosts the voltage up to 5V, which is the fed into the 5V input pin on the ESP32 DevKitC, and then properly bucked down to 3.3V via the on board LDO regulator.

Making it rechargable

Finally, the last step was to make the whole system rechargeable. One of the requirements of the device is to use USB-C for recharging, due to its widespread adoption worldwide. As such, a USB-C charging module was purchased.

We had to ensure that the charging module we selected had built in over-current and over-voltage protection to make the device as safe as possible. After reviewing several candidates, we went with a charging module that uses the TP4056 IC. This module has the appropriate over-current and over-voltage protections, and is USB-C compatible. See picture below.

Putting it all together

Overall, the entire power management system is as follows: 3.7V LiPo Battery -> TP4056 USB-C Charging Module -> Switch -> Boost Converter -> ESP32-DEVKITC-32E

The final power management system can be seen pictured below.

Log Entry 11 - 03/23/24

Final Mechanical Design

The physical form of the device has gone through some considerable design changes over the evolution of this project. One of the biggest concerns with earlier designs was how to actually hold and operate the device. The old design was unintuitive. A major redesign of the main device took place to address these issues.

Latest Design

Firstly, we set new design goals for the product. We wanted it to be compact, handheld, and extremely simple to operate. As such, the final device is a compact 78.5mm (3.09in) x 69.3mm (2.73in) x 76.2mm (3in) in dimension.It is very comfortable to hold in your hands due to the rounded sides, and all the buttons can be used using just your index finger.

The device weighs in at around 170g with the battery in, at full charge. This is extremely light, and comfortable to hold.

A sheet of Mylar is inserted between the I2S MEMS mic and the outer snap ring. This is because Mylar is an excellent electrical insulator and will ensure no electrical interference from the skin conducting electricity. Mylar is also an industry standard to traditional stethoscopes, so Mylar was used for our project as well.

The snap rings and mic housing are press fit, and we used cyanoacrylate glue to make sure it all stays in place. For the lid at the back, we used threaded inserts and M2 screws.

Renders and various views of the device can be seen below.

To make assembly easy, we used JST connectors for the buttons and LiPo battery. This made it easy to troubleshoot during assembly. We used a perfboard for the microcontroller and associated connections. Once the circuitry was completed and all continuity tests were completed, the perfboard was cut to size. See pictures below of the assembly process.

For more, please see the demo video. Note that this video was edited with the intention to be played during the Mechatronics Symposium, so there is limited/no sound.

Log Entry 12 - 03/23/24

Real Pulmonary Sound Collection

Below, you'll find a real-life recording of pulmonary sounds collected on one of our group members. You'll hear 2 breaths, followed by two coughs (a procedure that resembles something a doctor might ask you to do).