Earswitch
EarSwitch™ is a revolutionary health technology company on a mission to radically transform how people with neurological assistive needs regain control of their lives through the use of a transformative in-ear device. This device allows people who would otherwise be unable to communicate due to severe neurological impairments do so via the voluntary and physiological movement from the tensor tympani – a muscle within the middle ear, a physiological movement that can produce a dull rumbling sound known as "ear rumbling." This groundbreaking communication approach allows users to communicate in ways previously thought impossible, leveraging a natural body response to create powerful assistive technology.
EarSwitch™ approached The Product Partnership (TPP), a collection of four separate leading creative companies, including New Icon, that work collaboratively to deliver a wide range of product development capabilities, with a project brief that detailed what they were looking to achieve.
From here New Icon took on the software development aspect, collaborating with three other TPP companies; Realise, Cubik and Amalgam to scope the project and craft the grant application. It was this collective expertise that played a crucial role in securing the funding needed to accelerate the project.
EarSwitch™ were already working with Bath University to answer a number of research questions via online surveys with their primary goal being to explore:
Building on the foundational work in the first phase, the second phase of the project focused on the actual in-ear device creation, and integrating the software, developed previously, into tangible hardware. This technically demanding project required a versatile, experienced team to support the project and involved a collaborative effort amongst three TPP companies: Cubik, Realise, and New Icon—covering electronics, hardware, and software.
Cubik spearheaded the electronics development, using a Compute Module 4 (CM4) from a Raspberry Pi (which is essentially a Raspberry Pi without the onboard peripherals) allowing for flexibility in design. The CM4 was chosen for its compact form and adaptability, key factors for the device's intended use.
Realise were responsible for designing and producing the housing that accommodates the circuit board.
Participants who completed the initial online survey were invited to Bath University for direct assessment using an otoscope—an instrument that allows for detailed examination of the ear. This phase was critical for validating the survey data and observing the muscle's response in a controlled environment.
Tasked with the data management aspect of the project, New Icon developed a sophisticated data storage system that helped the evolution of the algorithm to analyse the video frames captured during the otoscope examinations and identify sensible default parameters. This process involved:
The software developed by New Icon enabled a significant enhancement in the efficiency and accuracy of data collection, playing a crucial role in advancing Bath University's research algorithm. This improvement was pivotal for a deeper understanding of the tensor tympani muscle’s capabilities.
New Icon's approach focused on providing a tailored software solution that streamlined the collection and initial processing of extensive data sets. This enhanced not only the efficiency of the process but also ensured the robustness and precision of the data collected.
In collaboration with Bath University, New Icon integrated the OpenCV library to interpret ear movement and register clicks onto the hardware.
The software component was critical in bringing the physical device to life. New Icons task was to adapt the algorithm previously developed with Bath University—written in Python—so it could be operational within the hardware. This involved configuring the operating system of the CM4 and writing specific system services. Upon device startup, these services would launch automatically, detecting when the algorithm triggered an action and responding accordingly.
The software was designed to perform several functions based on the algorithm's output:
The device now functions as a groundbreaking assistive technology tool, enabling users to perform tasks like typing on a virtual keyboard or navigating through smartphone apps with single-event inputs. This was made possible by the "ear rumbling" detection capability of the hardware, converting detected muscle movements into actionable commands on connected devices. The system was designed to interpret these commands according to the configuration of the connected laptop or smartphone, facilitating a wide range of accessibility options.
The second phase of the project not only transformed the theoretical advancements from the initial research phase into a practical, user-oriented device but also underscored the importance of integrating software with precise hardware specifications to achieve a truly accessible and multifunctional tool.
The device is now in use and actively improves communication for people with neurological disabilities such as motor neurone disease. However, this innovative earpiece has unveiled a host of potential applications that stretch well beyond its original purpose. Its versatility makes it well-suited not just for healthcare but also for industries like gaming and personal wellness. This wide-ranging applicability underscores the transformative potential of the device, promising not only to revolutionise daily tasks for individuals with neurological impairments but also to enhance everyday activities for the general population.
There are several standard ways devices can communicate with other devices (mobile apps in this scenario):
1) The device presents as a wireless access point (the device temporarily shows up as its own Wi-Fi signal that the app can connect to)
2) The device has a bluetooth (the device presents a bluetooth signal that the app on the phone can connect to)
3) A physical USB connection (connect the device and your phone with the app together with a cable)
In all IoT projects the first challenge with a device is usually connecting to the internet or a mobile phone that can talk to it and give it the connection and config details it needs.
The Ear Switch team required their device to seamlessly interact with a variety of equipment, so they implemented the HID (Human Interface Device) protocol. This standard enables devices such as mice, keyboards, and gaming controllers to communicate their functions and capabilities to computers.
Setting up the Ear Switch is straightforward, making it easier for caregivers assisting users. Once configured, the device can connect to a computer or mobile device in two ways: either plugged in with a USB cable or connected wirelessly via Bluetooth. This flexibility ensures that users can easily control the device using the method that best suits their needs.
To make this work, the team wrote special code that lets the Ear Switch act like a HID, both for USB connections and for Bluetooth. This means you can control the device with your ear, whether you're wired or going wireless!
Developing physical devices without screens can pose quite a challenge as you need to be able to access them to run code, test and debug. In order to make this effective during development we used the following technologies:
This allows the device to present its own Wi-Fi temporarily to get login details for the main network Wi-Fi. Once the main Wi-Fi details are collected it switches the WAP off and connects to the local Wi-Fi using the new credentials. This then means packages and updates can be installed and creates a good local test environment for development.
Universal plug-and-play allows the device to be a good citizen on the network and be easy to find, and is a great protocol and technology for network service discovery.
In order to be easily compatible with a multitude of devices the EarSwitch™ team wanted to support the HID (Human Interface Device) protocol. The HID protocol allows hardware devices to communicate to operating systems and register how they behave and what capabilities they have from a simple mouse or keyboard to a complex gaming control pad.
Once the device is successfully registering EarSwitch™’ it needs to be connected to a target device either a computer or a mobile device that can then be controlled from the user's ear!
The EarSwitch™ device can be connected to a device (that is desired to be controlled) through two methods. Wired via a USB cable or wirelessly via Bluetooth.
Therefore we produced code that could emulate the HID protocol for both USB and Bluetooth:
Send us a message for more information about how we can help you