US9271064B2 - Method and system for contact sensing using coherence analysis - Google Patents
Method and system for contact sensing using coherence analysis Download PDFInfo
- Publication number
- US9271064B2 US9271064B2 US14/079,506 US201314079506A US9271064B2 US 9271064 B2 US9271064 B2 US 9271064B2 US 201314079506 A US201314079506 A US 201314079506A US 9271064 B2 US9271064 B2 US 9271064B2
- Authority
- US
- United States
- Prior art keywords
- microphone
- contact
- coherence
- event
- coherence function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000004458 analytical method Methods 0.000 title claims abstract description 43
- 238000001514 detection method Methods 0.000 claims abstract description 53
- 238000012545 processing Methods 0.000 claims abstract description 34
- 230000004044 response Effects 0.000 claims abstract description 19
- 230000006870 function Effects 0.000 claims description 59
- 238000004891 communication Methods 0.000 claims description 25
- 230000003595 spectral effect Effects 0.000 claims description 19
- 230000008859 change Effects 0.000 claims description 9
- 230000007423 decrease Effects 0.000 claims description 8
- 230000003993 interaction Effects 0.000 claims description 7
- 230000009471 action Effects 0.000 claims description 4
- 238000009499 grossing Methods 0.000 claims description 4
- 230000035945 sensitivity Effects 0.000 claims description 4
- 238000007493 shaping process Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims 1
- 210000000613 ear canal Anatomy 0.000 description 24
- 239000007787 solid Substances 0.000 description 13
- 238000010079 rubber tapping Methods 0.000 description 12
- 238000005516 engineering process Methods 0.000 description 11
- 238000007789 sealing Methods 0.000 description 9
- 230000005236 sound signal Effects 0.000 description 7
- 230000008878 coupling Effects 0.000 description 6
- 238000010168 coupling process Methods 0.000 description 6
- 238000005859 coupling reaction Methods 0.000 description 6
- 238000012544 monitoring process Methods 0.000 description 5
- 210000003454 tympanic membrane Anatomy 0.000 description 5
- 230000002159 abnormal effect Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 4
- 230000006835 compression Effects 0.000 description 4
- 238000007906 compression Methods 0.000 description 4
- 230000005284 excitation Effects 0.000 description 4
- 239000000463 material Substances 0.000 description 4
- 230000003247 decreasing effect Effects 0.000 description 3
- 230000001419 dependent effect Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 229920001971 elastomer Polymers 0.000 description 3
- 239000011521 glass Substances 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000037361 pathway Effects 0.000 description 3
- 230000003213 activating effect Effects 0.000 description 2
- 229920005549 butyl rubber Polymers 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000000873 masking effect Effects 0.000 description 2
- 230000004888 barrier function Effects 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000006260 foam Substances 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 239000007943 implant Substances 0.000 description 1
- 238000010348 incorporation Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000010399 physical interaction Effects 0.000 description 1
- 239000004033 plastic Substances 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000010183 spectrum analysis Methods 0.000 description 1
- 239000003826 tablet Substances 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/10—Earpieces; Attachments therefor ; Earphones; Monophonic headphones
- H04R1/1041—Mechanical or electronic switches, or control elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03K—PULSE TECHNIQUE
- H03K17/00—Electronic switching or gating, i.e. not by contact-making and –breaking
- H03K17/94—Electronic switching or gating, i.e. not by contact-making and –breaking characterised by the way in which the control signals are generated
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03K—PULSE TECHNIQUE
- H03K2217/00—Indexing scheme related to electronic switching or gating, i.e. not by contact-making or -breaking covered by H03K17/00
- H03K2217/94—Indexing scheme related to electronic switching or gating, i.e. not by contact-making or -breaking covered by H03K17/00 characterised by the way in which the control signal is generated
- H03K2217/94005—Indexing scheme related to electronic switching or gating, i.e. not by contact-making or -breaking covered by H03K17/00 characterised by the way in which the control signal is generated activated by voice or sound
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
- H04R1/22—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired frequency characteristic only
- H04R1/28—Transducer mountings or enclosures modified by provision of mechanical or acoustic impedances, e.g. resonator, damping means
- H04R1/2807—Enclosures comprising vibrating or resonating arrangements
- H04R1/2853—Enclosures comprising vibrating or resonating arrangements using an acoustic labyrinth or a transmission line
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2430/00—Signal processing covered by H04R, not provided for in its groups
- H04R2430/01—Aspects of volume control, not necessarily automatic, in sound systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2430/00—Signal processing covered by H04R, not provided for in its groups
- H04R2430/03—Synergistic effects of band splitting and sub-band processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
- H04R2499/10—General applications
- H04R2499/11—Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/43—Electronic input selection or mixing based on input signal analysis, e.g. mixing or selection between microphone and telecoil or between microphones with different directivity characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
Definitions
- the present invention relates to user interactive electronic devices, and more particularly, though not exclusively, to acoustic detection of a physical input for operating a microphone enabled electronic device.
- Most media based electronic devices are operated by way of a user interface. As devices become smaller there is only limited space for the user interaction and the user is generally required to physically interact with the device, for example, by way of a touch screen. This size limitation for user interaction is more evident with smaller devices, such as earpieces and smart wristwatches.
- the microphones and speakers on such media devices are primarily used for capturing voice and producing sound output.
- Silicon analog and digital microphones are increasingly affordable and common in a variety of mobile electronic devices. These microphones are generally configured as speech sensors; for detecting speech for purposes of voice control of a device or for voice communication or recording with the device. Multiple microphones on a device offer advantages for improving the quality of detected speech using active noise reduction systems.
- U.S. Patent Application 2011/0142269 A1 describes a hearing aid switch that utilizes pressure/sound clues from a filtered input signal to enable actuation initiated by a user by a signature hand movement relative to a wearer's ear.
- the preferred signature hand movement involves patting on the ear meatus at least one time to generate a compression wave commonly thought of as a soft “clap” or “pop”.
- a digital signal processor analyzes the signal looking for a negative pulse, a positive pulse, and dissipation of the hand generated signal.
- 8,358,797 describes a method for changing at least two parameter settings of a device and includes detecting an abnormal change in an external feedback path and an input signal generated by an abnormal pressure wave, and activating a pressure wave detection switch and an abnormal feedback path detection switch for changing the at least one parameter setting in the device.
- a method for acoustical switching suitable for use with a microphone enabled electronic device can include the steps of capturing a first microphone signal from a first microphone on a device, by way of a processor on the device communicatively coupled to the first microphone: analyzing the first microphone signal for a contact event versus a non-contact event; and directing the electronic device to switch a processing state responsive to a detection of either the contact event or non-contact event.
- the processing state responsive to detecting the contact event can comprise at least one of performing a user interface action, a command response, an automatic interaction or a recording.
- the processing state responsive to detecting the non-contact event can comprise at least one of a voice communication, a data communication, an event detection, a speech recognition or a key word detection.
- the method for contact sensing can further include capturing a second microphone signal from a second microphone on the device, and by way of the processor on the device communicatively coupled also to the second microphone: perform a coherence function on the first microphone signal and the second microphone signal, analyze the coherence function to determine if a physical contact due to touch occurred on the device, and provide a change to at least one parameter setting on the electronic device responsive to determining the physical contact occurred.
- the method includes discriminating between the physical contact with a high inter-microphone coherence and an airborne event with a low inter-microphone coherence.
- the method can further include generating a smoothed coherence function from the coherence function, resolving a peak in the smoothed coherence function; comparing the peak in the smoothed coherence function to a threshold; and deciding the physical contact has occurred if the peak is greater than the threshold.
- the method can include resolving one or more peaks in the coherence function; evaluating a time window between the one or more peaks, and setting a contact detection status to a negative value for de-bouncing if the time window is less than a previous time window, otherwise setting the contact detection status to a positive value. This can include counting a number of the contact detection status events for positive values, and differentiating between a single tap and a double tap from analysis of the contact detection status if the number is within a time period.
- the method can further include tuning a cavitational acoustic resonance by way of resonant air channels, and reducing sensitivity of the coherence function to an airborne event from the tuned cavitational acoustic resonance of the first and second microphone signals.
- a spectral notch specific to the airborne sound event can be designed by shaping the resonant air channel to decrease the coherence function for the airborne sound in a frequency band of interest.
- FIG. 1A illustrates a wearable system for detecting physical contact on a headset device in accordance with an exemplary embodiment
- FIG. 1B illustrates another wearable system for detecting physical contact on an eyeglass device in accordance with an exemplary embodiment
- FIG. 1C illustrates a mobile device for coupling with the wearable system in accordance with an exemplary embodiment
- FIG. 1D illustrates another mobile device for coupling with the wearable system in accordance with an exemplary embodiment
- FIG. 1E illustrates an acoustic switch for directing a processing state in accordance with an exemplary embodiment
- FIG. 2 is method for coherence based contact sensing suitable for use with the wearable system in accordance with an exemplary embodiment
- FIG. 3 is flowchart for media setting adjustment and mixing audio signals suitable for use with the wearable system in accordance with an exemplary embodiment
- FIG. 4 is method for detecting a physical tap using coherence analysis suitable for use with the wearable system in accordance with an exemplary embodiment
- FIG. 5 depicts magnitude coherence functions in accordance with the exemplary embodiments for detecting a contact
- FIG. 6 depicts spectral waveforms used in conjunction with coherence functions in accordance with the exemplary embodiments for detecting a contact
- FIG. 7A depicts a block diagram configuration of coherence based contact system for activating audio recordings in accordance with an exemplary embodiment
- FIG. 7B depicts a block diagram configuration of coherence based contact system using multiple microphones in accordance with an exemplary embodiment
- FIG. 7C depicts another block diagram configuration of coherence based contact system using multiple microphones in accordance with an exemplary embodiment
- FIG. 8A illustrates a device body configured with acoustic ports for microphone based coherence analysis in accordance with an exemplary embodiment
- FIG. 8B illustrates a device body configured with a cavitation for microphone based coherence analysis in accordance with an exemplary embodiment
- FIG. 8C illustrates a frequency response for the device body for FIG. 8A and FIG. 8B in accordance with an exemplary embodiment
- FIG. 9A is an exemplary earpiece for use with the coherence based contact system of FIG. 1A in accordance with an exemplary embodiment.
- FIG. 9B is an exemplary mobile device for use with the coherence based contact system of FIG. 1A in accordance with an exemplary embodiment.
- the system analyzes a coherence between the microphone signals generated by the physical contact to discriminate if physical contact occurred. It can differentiate between a purposely initiated contact for such control or whether it was a non-initiated airborne sound.
- the user can simply perform a tap or tapping on the device to control a media setting, for example an adjustment function to control a volume. Other functions are herein contemplated.
- the system 100 includes a first microphone 101 for capturing a first microphone signal, a second microphone 102 for capturing a second microphone signal, and a processor 140 / 160 communicatively coupled to the first microphone 101 and the second microphone 102 to perform a coherence analysis to determine if a physical contact occurred on the device.
- the processor 140 / 160 may reside on a communicatively coupled mobile device or other wearable computing device for sensing a physical contact on the headset device, for example, a finger tap or touch of one of the earpieces.
- the present system 100 By analysis of the inter-microphone coherence and detection of a high peak in the coherence, the present system 100 generates commands to control the device, for example, in this embodiment, to change at least one parameter setting of the device, such as a media control of the headset (e.g., volume, play list, balance, etc.).
- a media control of the headset e.g., volume, play list, balance, etc.
- the system 100 can be configured to be part of any suitable media or computing device.
- the system may be housed in the computing device or may be coupled to the computing device.
- the computing device may include, without being limited to wearable and/or body-borne (also referred to herein as bearable) computing devices.
- wearable/body-borne computing devices include head-mounted displays, earpieces, smartwatches, smartphones, cochlear implants and artificial eyes.
- wearable computing devices relate to devices that may be worn on the body.
- Bearable computing devices relate to devices that may be worn on the body or in the body, such as implantable devices.
- Bearable computing devices may be configured to be temporarily or permanently installed in the body.
- Wearable devices may be worn, for example, on or in clothing, watches, glasses, shoes, as well as any other suitable accessory.
- the devices e.g, headphones, eyeglasses, etc.
- touch and feel is an intuitive means for interacting with the wearable computing device, and so the tapping need occur only somewhere on the body (outer plastic casing, shell, etc.) of the device within mechanical coupling vicinity of the first 101 and second 102 microphones. That is, the user is not required to identify and tap an individual microphone, but rather, tap within proximity of the microphones on the device in a region that the microphones are mechanically coupled for propagation of acoustic signals there through, as will be explained ahead.
- the system 100 can resolve whether the tapping is a physical tapping initiated by a user and/or differentiate between airborne sounds which are not initiated by the user, for example, abrupt noises or loud sounds.
- the system 100 can also be configured for individual earpieces (left or right) or include an additional pair of microphones on a second earpiece in addition to the first earpiece.
- the system 100 can be configured to be optimized for different microphone spacing's and different microphone housing materials as will be described ahead.
- eyeglasses 120 operate as the wearable computing device, for collective processing of acoustic signals (e.g., ambient, environmental, voice, etc.) and media (e.g., accessory earpiece connected to eyeglasses for listening) when communicatively coupled to a media device (e.g., mobile device, cell phone, etc.).
- acoustic signals e.g., ambient, environmental, voice, etc.
- media e.g., accessory earpiece connected to eyeglasses for listening
- a media device e.g., mobile device, cell phone, etc.
- the user may rely on the eyeglasses for voice communication and external sound capture instead of requiring the user to hold the media device in a typical hand-held phone orientation (i.e., cell phone microphone to mouth area, and speaker output to the ears). That is, the eyeglasses sense and pick up the user's voice (and other external sounds) for permitting voice processing.
- An earpiece may also be attached to the eyeglasses 120 for providing audio and voice.
- the first 121 and second 122 microphones are mechanically mounted to one side of eyeglasses.
- the embodiment 120 can be configured for individual sides (left or right) or include an additional pair of microphones on a second side in addition to the first side.
- a processor 140 / 160 communicatively coupled to the first microphone 121 and the second microphone 122 for sensing a physical contact on a device, such as, a finger tap or touch, may be present.
- FIG. 1C depicts a first media device 140 as a mobile device (i.e., smartphone) which can be communicatively coupled to either or both of the wearable computing devices ( 100 / 120 ).
- FIG. 1D depicts a second media device 140 as a wristwatch device which also can be communicatively coupled to the one or more wearable computing devices ( 100 / 120 ).
- the processor performing the coherence analysis for the detection of a physical touch is included thereon, for example, within a digital signal processor or other software programmable device within, or coupled to, the media device 140 or 160 .
- components of the media device for implementing coherence detection processing functionality will be explained in further detail.
- the system 100 may represent a single device or a family of devices configured, for example, in a master-slave or master-master arrangement.
- components of the system 100 may be distributed among one or more devices, such as, but not limited to, the media device illustrated in FIG. 1C and the wristwatch in FIG. 1D . That is, the components of the system 100 may be distributed among several devices (such as a smartphone, a smartwatch, an optical head-mounted display, an earpiece, etc.).
- the devices (for example, those illustrated in FIG. 1A and FIG. 1B ) may be coupled together via any suitable connection, for example, to the media device in FIG. 1C and/or the wristwatch in FIG. 1D , such as, without being limited to, a wired connection, a wireless connection or an optical connection.
- the computing devices shown in FIGS. 1C and 1D can include any device having some processing capability for performing a desired function, for instance, as shown in FIG. 9B .
- Computing devices may provide specific functions, such as heart rate monitoring or pedometer capability, to name a few.
- More advanced computing devices may provide multiple and/or more advanced functions, for instance, to continuously convey heart signals or other continuous biometric data.
- advanced “smart” functions and features similar to those provided on smartphones, smartwatches, optical head-mounted displays or helmet-mounted displays can be included therein.
- Example functions of computing devices may include, without being limited to, capturing images and/or video, displaying images and/or video, presenting audio signals, presenting text messages and/or emails, identifying voice commands from a user, browsing the web, etc.
- a system 180 for acoustical switching suitable for use with a microphone enabled electronic device comprises a first microphone 181 on the device for capturing a first microphone signal, and an acoustic switch 182 communicatively coupled to the first microphone for analyzing the first microphone signal for a contact event versus a non-contact event, and directing the electronic device to switch a processing state responsive to a detection of either the contact event or non-contact event.
- the microphone signal can arise from a sound source such as voice, ambient sounds, environmental sounds, acoustics, abrupt onsets, acoustic events, noise or any combination thereof.
- the acoustic switch can be a processor as described herein, and/or a combination of software and hardware as described herein.
- the acoustic switch can be partially enabled with integrated circuitry for analog processing front-end events, and enabled with digital logic and software programmable devices for back-end processing.
- the acoustic switch by way of a processor on, or operatively coupled to the device, can perform the acoustic switching and/or the processing thereto associated described herein.
- the microphone 181 and the acoustic processor 182 reside on the same device, and may be integrated components or joined.
- the microphone 181 and the acoustic processor 182 reside on different platforms, for example, a microphone with its own circuitry and communicatively coupled to a mobile device, such as a cell phone.
- the system 180 can be implemented in whole or in part by the devices shown in FIGS. 9A and 9B described herein, and with respect to the foregoing methods, though are not limited to such components or configurations and may include more or less than the number of components shown.
- the acoustic switch directs the processing to a respective state.
- the processing state 184 responsive to detecting the non-contact event comprises at least one of a voice communication, a data communication, an event detection, a speech recognition or a key word detection.
- the processing state 185 responsive to detecting the contact event comprises at least one of performing a user interface action, a command response, an automatic interaction or a recording.
- FIG. 2 a general method 200 for contact sensing using coherence analysis is shown.
- the method 200 may be practiced with more or less than the number of steps shown.
- the method 200 can be practiced by the components presented in the figures herein though is not limited to the components shown.
- the reader is also directed to the description of FIG. 9A for a detailed view and description of the components of the earpiece 900 (which may be coupled to the media device 950 of FIG. 9B ); components which may be referred to for describing method 200 .
- the method 200 for detecting physical contact is directed to controlling the functionality of a sound isolating earphone using at least two microphones mounted on the body the earphone.
- a Sound isolating (SI) earphone that is suitable for use with the contact based coherence sensing system 100 .
- SI earphones and headsets are becoming increasingly popular for music listening and voice communication.
- SI earphones enable the user to hear and experience an incoming audio content signal (be it speech from a phone call or music audio from a music player) clearly in loud ambient noise environments, by attenuating the level of ambient sound in the user ear-canal.
- SI earphones/headsets The disadvantage of such SI earphones/headsets is that the user is acoustically detached from their local sound environment, and communication with people in their immediate environment is therefore impaired: i.e. the earphone has a reduced situational awareness due to the acoustic masking properties of the earphone.
- a non Sound Isolating (SI) earphone can reduce the ability of an earphone wearer to hear local sound events as the earphone wearer can be distracted by incoming voice message or reproduced music on the earphones.
- the ambient sound microphone (ASM) located on an SI or non-SI earphone can be used to increase situation awareness of the earphone wearer by passing the ASM signal to the loudspeaker in the earphone.
- ASM ambient sound microphone
- Such a “sound pass through” utility can be activated manually using a simple and intuitive mechanism: by detecting a physical contact on the earphone, i.e. an earphone “tap”, “thump” or “bang”.
- the directional sensitivity of the earphone unit to sound in the wearer's environment can be affected if more than one ambient microphones are used, e.g. using “beam forming” algorithms that require at least two microphones. It is intuitive for the user to use the ambient sound microphones on an earphone to detect a physical user contact (e.g. a finger tap) on the earphone, and to activate a sound pass-through in response to this tap.
- a physical user contact e.g. a finger tap
- An analysis of the electronic coherence between the two microphone signals provides a robust means to detect physical contact, as described herein.
- the earpiece 900 is connected to a voice communication device (e.g. mobile telephone, radio, computer device) and/or audio content delivery device (e.g. portable media player, computer device).
- the communication earphone/headset system comprises a sound isolating component for blocking the users ear meatus (e.g. using foam or an expandable balloon); an Ear Canal Receiver (ECR, i.e.
- a signal processing system receives an Audio Content (AC) signal (e.g. music or speech audio signal) from the said communication device (e.g. mobile phone etc) or the audio content delivery device (e.g. music player); and further receives the at least one ASM signal and the optional ECM signal.
- AC Audio Content
- the signal processing system mixes the at least one ASM and AC signal and transmits the resulting mixed signal to the ECR in the loudspeaker.
- the method 200 can start in a state in which the earpiece 900 is in the user's ear and is actively monitoring for a physical contact, such as a tapping sound.
- the first microphone and the second microphone capture a first signal and second signal respectively at step 202 and 204 .
- the order of the capture for which signal arrives first is a function of the sound source location; it not the microphone number; either the first or second microphone may capture the first microphone signal.
- the coherence based contact detection system analyzes a coherence between the two microphone signals to determine if a physical tap has occurred. The specifics of this method step are discussed in greater detail ahead in the description of FIG. 4 .
- a change of at least one parameter is provided.
- the earpiece 900 adjusts the sound microphone signal gain in step 210 responsive to the coherence.
- the earpiece 900 , or associated device 950 e.g. mobile device, wristwatch, etc.
- the mixing of the at least one ASM and AC signal is controlled by ASM and AC signal gains as illustrated.
- the ASM and AC signal gains are determined by logic incorporating an analysis of the coherence between two ASM signals on the earphone device to detect contact.
- the method 200 is not limited to practice only by the earpiece device 900 .
- Examples of electronic devices that incorporate multiple microphones for voice communications and audio recording or analysis, are listed, as well as an example of a parameter setting that can be adjusted in response to a detected contact:
- FIG. 3 illustrates an exemplary flowchart 300 for mixing the Ambient Sound Microphone (ASM) and Audio Content (AC) signal gain responsive to detected physical contact on the earpiece (earphone) device 900 as practiced by method 200 of FIG. 2 .
- the steps of the flowchart 300 may be practiced by the components of the earpiece device shown in FIG. 9A and/or in conjunction with the components of the devices shown in FIGS. 1C , 1 D and 9 B.
- the flowchart 300 can start in a state in which the earpiece 900 is in the user's ear and is actively monitoring for a physical contact, such as a tapping sound.
- the first microphone and the second microphone capture a first signal and second signal respectively at step 302 and 304 .
- the processor directs the first and second microphone signal buffer to a digital system and analyses the band-limited smoothed magnitude-squared coherence between the two signals.
- the coherence function is then performed at step 306 on the first and second microphone signals.
- One or more peaks of the band-limited smoothed magnitude-squared coherence is then determined from the coherence. For now it is sufficient to know that when a peak in the smoothed coherence is detected, a user finger tap is determined to have occurred.
- the specifics of the peak detection method will be discussed in greater detail ahead in FIG. 4
- the output of the coherence based contact detection system of step 306 is a deciding factor for how the processing proceeds. It will be a “positive” or “negative” state based on the comparison at step 308 .
- the audio content gain is maintained or selectively increased at step 310 for the corresponding AC parameter control 314 .
- the ambient sound microphone gain is maintained or decreased at step 312 for the corresponding ASM parameter control 318 . That is, if the status is “negative”, then the ambient sound microphone gain is decreased AND the audio content signal gain is selectively determined.
- the ordering of the applied parameter change to the AC and ASM is a function of the CDS state to accommodate the user's listening experience. The method 400 continues to monitor the user's environment and adjust the gains as accordingly described in flowchart 400 starting with steps 302 and 304 again.
- FIG. 4 depicts a more detailed method 400 to the flowchart 300 shown in FIG. 3 . It expands upon the calculation specifics of coherence function of step 308 , and more specifically, the fundamental analysis and resulting state of the coherence function for controlling parameters of the device, including for instance the timing and settings for controlling the AC and ASM gains expressed in the flowchart 300 of FIG. 3 .
- the method 400 may repeat some of the steps previously disclosed for completeness. Similarly, the steps of the method 400 may also be practiced by the components of the earpiece device shown in FIG. 9A and/or in conjunction with the components of the devices shown in FIGS. 1C , 1 D and 9 B.
- the method 400 can start in a state 402 in which the earpiece 900 is in the user's ear and is actively monitoring for a physical contact, such as a tapping sound.
- a first microphone signal is received from a first microphone on a device.
- a second microphone signal is received from a second microphone on the device.
- the coherence function is performed on the first microphone signal and the second microphone signal at step 408 . It is at this juncture that the system will analyze the coherence function, perform peak detection, and inter-peak timing relations to determine if a physical contact due to touch occurred on the device, and if so, providing a change to at least one parameter setting on the device responsive to determining the physical contact occurred.
- the magnitude squared coherence estimate, Cxy as determined in step 408 is a function of the power spectral densities, Pxx(f) and Pyy(f), of x and y, and the cross power spectral density, Pxy(f), of x and y,
- the window length for the power spectral densities and cross power spectral density in the preferred embodiment are approximately 3 ms ( ⁇ 2 to 5 ms).
- the time-smoothing for updating the power spectral densities and cross power spectral density in the preferred embodiment is approximately 0.5 seconds (e.g. for the power spectral density level to increase from ⁇ 60 dB to 0 dB) but may be lower to 0.2 ms.
- the magnitude squared coherence estimate is a function of frequency with values between 0 and 1 that indicates how well x corresponds to y at each frequency.
- the signals x and y correspond to the signals from a first and second microphone.
- the reader is referred to the description of FIG. 5 for a detailed description of the squared coherence between two microphones at different frequencies and different microphone spacings.
- the data in the figures of FIG. 5 are used to determine the frequency at which the coherence is analyzed to detect a physical contact (e.g. “tap”) on the body housing the microphones dependent on the microphone spacings.
- a smoothed coherence function is generated from the coherence function, and a peak is calculated from the coherence function in step 412 .
- It may be specifically limited to a “high” frequency band; that is, the smoothed magnitude squared coherence from the frequency band may be between approximately 18 kHz and 20 kHz for analysis.
- FIG. 6 shows a series of coherence functions as will be explained ahead in greater detail.
- one such peak 611 for an exemplary sound event 622 is shown, though multiple peaks spread out over time are herein contemplated.
- the sound event may be produced by an intentional physical touch by the user or an unintentional airborne sound event, for example, a transient or passing abrupt sound.
- One purpose of method 400 as explained herein is to differentiate between the sound events.
- the peak is compared at step 414 to a threshold for deciding if the physical contact has occurred. If the peak is not greater than the threshold, a check on whether a timer was made in reference to the sound event is made at step 418 . If the timer is not started, the CDS status is set to “negative” at step 422 and the method returns to the start state for step 402 . If the timer was previously started, the timer is incremented at step 420 before the CDS status is set to “negative” at step 422 . The method similarly returns to the start state for step 402 . Notably, one or more peaks may be resolved, which includes evaluating a time window between the one or more peaks.
- step 414 if the peak is greater than the threshold, then a check is made to determine if the timer was previously started at step 424 . If the timer was not started, it is reset and started at step 426 , and the method proceeds to set the CDS status to “negative” at step 422 and proceed back to start at step 402 .
- the method steps 428 to 440 are specific for determining the CDS state.
- the contact detection status (CDS) is either set to a negative value for de-bouncing if the time window is less than a previous time window, otherwise the contact detection status is set to a positive value peaks (timer value). Essentially, if the peak value is less than the threshold value, then a “negative” status for the contact detection is assigned, otherwise a candidate “positive” status is assigned. If the event time of this latest candidate “positive” status time is less than a threshold time of a previous “positive” status time (e.g. 0.01 seconds) then the contact detection status is set to “negative” due to “switch bouncing”, otherwise the contact detection status is set to “positive”.
- a threshold time of a previous “positive” status time e.g. 0.01 seconds
- the CDS determination starts at step 428 , wherein, if the timer was previously started, the processor determines the inter-onset interval ( 101 ) between peaks. If the debounce inter-onset time (IOT) is less than a predetermined threshold IOT (storage 432 ) at step 430 then the peak is ignored and the timer is incremented at step 434 . If the IOT is not less than the predetermined 10 T, then at step 436 , a comparison is made to determine if the IOT is greater than a predetermined low IOT threshold but greater than a predetermined higher IOT threshold. These IOT thresholds are retrieved from memory storage. 438 . If the outcome of step 436 is NO, then the timer is stopped and reset at step 440 .
- IOT debounce inter-onset time
- step 436 If however the outcome of step 436 is YES then the CDS status is set to “positive” at step 442 .
- the timer is thereafter stopped and reset at step 444 and the method 400 returns to the start state at step 402 , to continually scan for new peaks as they are determined in real-time.
- the contact detection status is determined by the number of user taps, for example: a single tap if there is a single coherence peak with no other peak within a determined time period (e.g. 5 seconds); a double, triple etc tap is there are two, three etc positive peaks within a determined time period (e.g. 5 seconds).
- the processor counts the number of the contact detection status events for positive values, and differentiates between a single tap and a double tap from analysis of the contact detection status if the number is within a time period.
- FIG. 5 shows an exemplary squared coherence between two microphones at different frequencies and different microphone spacings (i.e. the distance between microphone diaphragms) in a diffuse sound field when the medium is air (top) or butyl rubber (lower), estimated according to the equation below:
- ⁇ p ⁇ ⁇ p 2 ⁇ ( ⁇ , r ) ( sin ⁇ ( ⁇ ⁇ ⁇ r / c ) ⁇ ⁇ ⁇ r / c ) 2 ,
- FIG. 6 The figures in FIG. 6 are used to determine the frequency at which the coherence is analyzed to detect a physical contact (e.g. “tap”) on the body housing the microphones dependent on the microphone spacing.
- a physical contact e.g. “tap”
- analyses of the coherence at above 16 kHz therefore provides a good mean to distinguish between airborne excitation and direct excitation (i.e. a physical tap on the earphone body).
- the material type used to house the microphones will affect the speed of sound in the material (c in the previous equation), thereby affecting the suitable frequency of analysis or threshold value.
- Smoothing of the magnitude squared coherence in the preferred embodiment is obtained by convolving the raw magnitude squared coherence with a hanning window of length 4 ms. Smoothing the coherence with such a method will reduce the peaks in the squared coherence, so the threshold value predicted by analysis of FIG. 5 described above will have to be reduced and may need to be determined empirically.
- the smoothed magnitude squared coherence from the frequency band between approximately 18 kHz and 20 kHz is analyzed.
- An analysis of coherence has advantages over analysis of the compression wave: Existing systems use a microphone signal level analysis to determine contact on a device. Such “compression wave analysis” systems are prone to false positives created by loud ambient sound sources. Furthermore, such compression wave analysis systems often necessarily requires a loud local sound source to determine contact, e.g. a clap or hard contact pressure against the device surface, which may be non-discrete, uncomfortable or impractical to use.
- subplot 610 As shown in subplot 610 , one peak 612 for an exemplary sound event 622 is identified, though multiple peaks spread out over time are illustrated.
- This subplot 610 shows a 17 second recording of an ambient sound microphone signal from one microphone mounted on the body of the earphone 900 . The following sound events are shown:
- Subplot 620 shows a spectrogram of the waveform from the top subplot 610 .
- Subplot 630 shows the smoothed coherence function at approximately 20 kHz. Note that the level of the clap event A shows a much lower peak 631 than the peak 632 for tap event B: i.e. it would be easier to discern the tap events than the clap events, even for the “gentle” tap event E.
- the table clap event C does not show at all in the coherence analysis.
- a coherence threshold value of approximately 0.2 can be used to determine if a physical “tap” has occurred, i.e. if the smoothed squared coherence is greater than 0.2, we determine that a physical tap has occurred.
- the level analysis of the microphone signal shown in FIG. 6 shows large peaks for the clap events and table tap events, but smaller peak value for the tap events. Therefore, a level analysis may lead to false positives for detecting direct physical contact with the earphone body. Such false positives could be annoying or even dangerous for the earphone sound pass-through embodiment: e.g. considering earphone wearer passing a loud jack-hammer, using a simple level analysis of one microphone signal the system may trigger a false positive and pass through this loud ambient sound to the earphone loudspeaker, startling the user or possibly causing hearing damage from the sudden loud sound exposure.
- FIG. 7A depicts another flowchart 700 for coherence based contact detection in accordance with another embodiment.
- the sound recording or analysis system can comprise an audio codec (e.g. mp3 codec).
- the recording media system can be local or remote, where the audio to the remote system can be transmitted via radio (e.g. Bluetooth 2.0, Wifi, GSM phone).
- the location of the system can also be transmitted using a GPS sensor.
- the flowchart 700 can start in a state in which the earpiece 900 is in the user's ear and is actively monitoring for a physical contact (e.g, a tapping sound).
- a first microphone signal is received from a first microphone on a device.
- a second microphone signal is received from a second microphone on the device.
- the coherence function is performed on the first microphone signal and the second microphone signal at step 706 to determine the Contact Detection State (CDS). This is where the system analyzes the coherence function, perform peak detection, and inter-peak timing relations as previously described to determine if a physical contact due to touch occurred on the device, and if so, providing a change to at least one parameter setting on the device responsive to determining the physical contact occurred.
- CDS Contact Detection State
- the system will proceed to activate a sound recording at step 710 , and direct the microphone signal to a recording media.
- the device will buffer in the samples, and store to memory, in a compressed or non-compressed format (e.g., PCM, WAV, AIFF, MP3, etc.).
- This may also include a remote audio recording media (e.g., computer readable FLASH memory) as shown in step 712 , or a local audio recording medial (e.g., computer readable FLASH memory) as shown in step 714 .
- FIG. 7B depicts another flowchart 740 for coherence based contact detection in accordance with another embodiment.
- the system is configured for use with three (3) microphones for coherence contact sensing.
- the coherence functions and analyses described above with respect to flowchart 300 (and method 400 ) are applied collectively to paired microphones.
- a logic unit 748 of the processor combines the contact status of each of the 3 pair-wise systems ( 743 , 744 , 746 ) to determine a single contact status (i.e. positive or negative) at step 750 .
- the logic is a simple “AND” logic, i.e. where each of the three pair-wise microphone systems must be positive to give a net positive contact status.
- a second logic configurations can involve determining a positive contact status if at least 2 out of the 3 pair-wise systems have a positive status.
- a third configuration is a logic OR, where a positive contact status is determine if at least 1 out of the 3 pair-wise systems have a positive status.
- the single coherence value Cxyz can then be used to determine a contact status at processing block 764 using the peak threshold method previously described in detail in the method 400 of FIG. 4 . It should be noted that any number of microphones can be used to determine the single coherence value by multiplying the pairwise coherence values of each microphone as illustrated in the above descriptions.
- FIG. 8A depicts a body of a device enabled for coherence based contact sensing in accordance with one embodiment.
- the subplots 810 , 820 and 830 illustratively summarize the sound path to two microphones from a “non-contact sound event” originating in the air (or non solid) medium versus a sound event originating from contact with the solid medium housing the microphones.
- the resulting inter-microphone coherence of air borne sound events versus contact sound events will generally be lower due to sound reflections in the air pathway, as previously discussed.
- Subplot 810 illustrates the mechanical coupling arrangement of microphones on the device body.
- the device is configured to house at least two microphones 814 within a solid structure 816 of the device body and including two acoustic ports 812 for the respective microphones.
- the acoustic ports 812 channel the sound waves though the solid structure 816 to the microphones 814 .
- the signal path from the acoustic signal travels through the air as illustrated in subplot 820 while the mechanical signal from a finger tap travels through the solid structure and excites the microphone through vibration as illustrated in subplot 830 .
- Subplot 820 illustrates the propagation of sound waves through the air, for example, from an external sound source 823 . From the illustration, it can be seen that sound waves do not significantly transmit through the solid structure 816 , but rather over the air, which are then channeled to the microphones 814 through the acoustic ports 812 .
- subplot 830 illustrates the propagation of sound waves from a physical contact 834 , for example, a finger tapping on the body surface. The finger tab travels through the solid structure as a vibration rather than an acoustic signal traveling through the air. From the illustration, it can be seen that sound waves do propagate within the solid structure 816 more so than over the air, at least, with respect to intensity.
- the characteristics of the wave forms through the solid structure 816 are a function of the material (e.g., porosity, density, etc.) and the spacing of the microphones, and also the acoustic port dimensions.
- FIG. 8B depicts the incorporation of “tuned” acoustic channels within a body of a device enabled for coherence based contact sensing in accordance with one embodiment.
- effect of reduced airborne event coherence versus contact event coherence is especially pronounced at high frequencies.
- the addition of resonant air channels next to the microphones is herein provided to further reduce coherence for airborne events increasing robustness to false positives from non contact (i.e. airborne) sound events.
- the coherence of acoustic signals in the 18-20 kHz band due to the airborne sounds can be intentionally degraded (reduced) by placing a structure in the microphone port that significantly reduces the acoustic signal.
- subplots 840 and 850 Two such designs are shown in subplots 840 and 850 .
- the first step is to add a “quarter wavelength” channel 844 off of the main microphone port 842 .
- a channel 844 with a radius of 2 mm and a length of 4.4 mm creates a strong acoustic notch filter around 19 kHz.
- This additional arrangement provides a “tuned” acoustic channel or cavity next to the microphone inlet and reduces the microphone response to airborne sound at the tuned frequency.
- the acoustic ports (see 812 of FIG. 8A ) have been bored and tunneled to create “tuned” acoustic channels; namely, a main microphone port 842 and the channel 844 .
- the addition of the channel (tunnel) 844 near the microphone 846 reduces coherence for airborne sounds and therefore increases system robustness to false positives.
- a volume can be added to the channel.
- Subplot 850 shows the addition of a volume (cavity) 854 backed to the short channel 853 from the main microphone port 852 to intentionally create a strong acoustic notch filter.
- the tuning of this acoustic port with channel 853 and volume 854 is such that it resonates to a quarter wavelength of the frequency at which the coherence is measured, which is typically the frequency with a half wavelength approximately equal or greater to the spacing between the two microphones.
- the frequency at which the coherence is analyzed is approximately 19 kHz for the design having channel 853 length 2 mm and width 1 mm and volume 854 with a 16 mm 3 volume. (That is, the channel 853 is 1 mm long and 2 mm in diameter, the volume (cavity) is 16 mm3 size to create an acoustic filter notch around 19 kHz.)
- FIG. 8C is illustrates frequency responses for the acoustic porting designs shown in FIG. 8B .
- Subplot 870 of FIG. 8C shows the frequency response of the acoustic model having a short channel as shown in subplot 840 of FIG. 8B as measured at the proximal microphone in response to an external pressure source. Note that the strong notch at 19 kHz will again reduce the acoustic signature by over 20 dB, which will further decrease the acoustic coherence signal in the frequency band of interest and significantly decrease the chance of an acoustic signal causing a false positive detection threshold event.
- FIG. 8C shows the frequency response of the acoustic model having a short channel backed by volume as shown in subplot 850 of FIG. 8B as measured at the proximal microphone in response to an external pressure source. Note that the strong notch at 19 kHz will again reduce the acoustic signature by over 20 dB, which will further decrease the acoustic coherence signal in the frequency band of interest and significantly decrease the chance of an acoustic signal causing a false coherence detection threshold event.
- FIG. 9A is an illustration of an earpiece device 900 that can be connected to the system 100 of FIG. 1A for performing the inventive aspects herein disclosed.
- the earpiece 900 contains numerous electronic components, many audio related, each with separate data lines conveying audio data.
- the headset 100 can include a separate earpiece 900 for both the left and right ear. In such arrangement, there may be anywhere from 8 to 12 data lines, each containing audio, and other control information (e.g., power, ground, signaling, etc.)
- the earpiece 900 comprises an electronic housing unit 901 and a sealing unit 908 .
- the earpiece depicts an electro-acoustical assembly for an in-the-ear acoustic assembly, as it would typically be placed in an ear canal 924 of a user.
- the earpiece can be an in the ear earpiece, behind the ear earpiece, receiver in the ear, partial-fit device, or any other suitable earpiece type.
- the earpiece can partially or fully occlude ear canal 924 , and is suitable for use with users having healthy or abnormal auditory functioning.
- the earpiece includes an Ambient Sound Microphone (ASM) 920 to capture ambient sound, an Ear Canal Receiver (ECR) 914 to deliver audio to an ear canal 924 , and an Ear Canal Microphone (ECM) 906 to capture and assess a sound exposure level within the ear canal 924 .
- the earpiece can partially or fully occlude the ear canal 924 to provide various degrees of acoustic isolation.
- assembly is designed to be inserted into the user's ear canal 924 , and to form an acoustic seal with the walls of the ear canal 924 at a location between the entrance to the ear canal 924 and the tympanic membrane (or ear drum). In general, such a seal is typically achieved by means of a soft and compliant housing of sealing unit 908 .
- Sealing unit 908 is an acoustic barrier having a first side corresponding to ear canal 924 and a second side corresponding to the ambient environment.
- sealing unit 908 includes an ear canal microphone tube 910 and an ear canal receiver tube 914 .
- Sealing unit 908 creates a closed cavity of approximately 5 cc between the first side of sealing unit 908 and the tympanic membrane in ear canal 924 .
- the ECR (speaker) 914 is able to generate a full range bass response when reproducing sounds for the user.
- This seal also serves to significantly reduce the sound pressure level at the user's eardrum resulting from the sound field at the entrance to the ear canal 924 .
- This seal is also a basis for a sound isolating performance of the electro-acoustic assembly.
- the second side of sealing unit 908 corresponds to the earpiece, electronic housing unit 900 , and ambient sound microphone 920 that is exposed to the ambient environment.
- Ambient sound microphone 920 receives ambient sound from the ambient environment around the user.
- Electronic housing unit 900 houses system components such as a microprocessor 916 , memory 904 , battery 902 , ECM 906 , ASM 920 , ECR, 914 , and user interface 922 .
- Microprocessor 916 (or processor 916 ) can be a logic circuit, a digital signal processor, controller, or the like for performing calculations and operations for the earpiece.
- Microprocessor 916 is operatively coupled to memory 904 , ECM 906 , ASM 920 , ECR 914 , and user interface 920 .
- a wire 918 provides an external connection to the earpiece.
- Battery 902 powers the circuits and transducers of the earpiece.
- Battery 902 can be a rechargeable or replaceable battery.
- electronic housing unit 900 is adjacent to sealing unit 908 . Openings in electronic housing unit 900 receive ECM tube 910 and ECR tube 912 to respectively couple to ECM 906 and ECR 914 .
- ECR tube 912 and ECM tube 910 acoustically couple signals to and from ear canal 924 .
- ECR outputs an acoustic signal through ECR tube 912 and into ear canal 924 where it is received by the tympanic membrane of the user of the earpiece.
- ECM 914 receives an acoustic signal present in ear canal 924 though ECM tube 910 . All transducers shown can receive or transmit audio signals to a processor 916 that undertakes audio signal processing and provides a transceiver for audio via the wired (wire 918 ) or a wireless communication path.
- FIG. 9B depicts various components of a multimedia device 950 suitable for use for use with, and/or practicing the aspects of the inventive elements disclosed herein, though is not limited to only those components shown.
- the device 950 comprises a wired and/or wireless transceiver 952 , a user interface (UI) display 954 , a memory 956 , a location unit 958 , and a processor 960 for managing operations thereof.
- the media device 950 can be any intelligent processing platform with Digital signal processing capabilities, application processor, data storage, display, input modality like touch-screen or keypad, microphones, speaker, Bluetooth, and connection to the internet via WAN, Wi-Fi, Ethernet or USB.
- This embodies custom hardware devices, Smartphone, cell phone, mobile device, iPad and iPod like devices, a laptop, a notebook, a tablet, or any other type of portable and mobile communication device.
- a power supply 962 provides energy for electronic components.
- the transceiver 952 can utilize common wire-line access technology to support POTS or VoIP services.
- the transceiver 952 can utilize common technologies to support singly or in combination any number of wireless access technologies including without limitation BluetoothTM Wireless Fidelity (WiFi), Worldwide Interoperability for Microwave Access (WiMAX), Ultra Wide Band (UWB), software defined radio (SDR), and cellular access technologies such as CDMA-1 ⁇ , W-CDMA/HSDPA, GSM/GPRS, EDGE, TDMA/EDGE, and EVDO.
- SDR can be utilized for accessing a public or private communication spectrum according to any number of communication protocols that can be dynamically downloaded over-the-air to the communication device. It should be noted also that next generation wireless access technologies can be applied to the present disclosure.
- the power supply 962 can utilize common power management technologies such as power from USB, replaceable batteries, supply regulation technologies, and charging system technologies for supplying energy to the components of the communication device and to facilitate portable applications. In stationary applications, the power supply 962 can be modified so as to extract energy from a common wall outlet and thereby supply DC power to the components of the communication device 950 .
- the location unit 958 can utilize common technology such as a GPS (Global Positioning System) receiver that can intercept satellite signals and there from determine a location fix of the portable device 950 .
- GPS Global Positioning System
- the controller processor 960 can utilize computing technologies such as a microprocessor and/or digital signal processor (DSP) with associated storage memory such a Flash, ROM, RAM, SRAM, DRAM or other like technologies for controlling operations of the aforementioned components of the communication device.
- DSP digital signal processor
Abstract
Description
-
- a. Smart watches. The smart watch can switch to a “display time” mode when contact is detected, and visually display the time for example using a back-lit LED. As described and illustrated in
FIG. 1E , the smart watch can implement theacoustic switch 182 for acoustic pickup and directing a processing state for contact versus non-contact events. Furthermore, the acoustic pickup can also be utilized to acquire the speech, conversation SPL level, or other nearby stimuli. - b. Smart “eye wear” glasses. The glasses can be configured to take a photograph using a built in camera when contact is detected. Similarly, as described and illustrated in
FIG. 1E , the eyeglasses can implement theacoustic switch 182 for acoustic pickup and directing a processing state for contact versus non-contact events. Furthermore, the acoustic pickup can also be utilized to acquire the speech, conversation SPL level, or other nearby stimuli. - c. Remote control units for home entertainment systems. The remote control device can be configured to change the channel in response to the number of detected contact hits within a defined period of time, for example, “1 hit” in a 2 second window increments the channel, and “2 hits” in a 2 second window decrements the channel playback number. Furthermore, the acoustic pickup can also be utilized to acquire the speech, conversation SPL level, or other nearby stimuli; as such the microphones can be used for voice control of the remote.
- d. Mobile Phones. The mobile phone can be configured to enter into a “voice analysis mode” in response to, for example, 2 physical hits, where at least one of the ambient microphones is directed to a speech analysis system to, for example, initiated a phone-call in response to the voice command “call John”.
- e. Hearing Aids.
- f. Steering wheel to enable a switch or for servicing as a hands-free pickup for a mobile device.
- g. Elevator Switch that can also use the acoustic pickup for communication with fire, emergency, maintenance or other
- h. In a shoe: the contact detection system can be configured to detect a step, i.e. to act as a pedometer.
- i. In the ground, e.g. embedded in earth or concrete.
- j. Mounted on a freestanding structure designed to restrict or prevent movement across a boundary, e.g. fence or wall. The acoustic pickup can be used to detect voices or other stimuli.
- a. Smart watches. The smart watch can switch to a “display time” mode when contact is detected, and visually display the time for example using a back-lit LED. As described and illustrated in
-
- a. For a fixed frequency, the coherence reduces as microphone spacing increases.
- b. For a fixed microphone spacing, the coherence reduces as the sound excitation frequency increases.
- c. For a fixed microphone spacing and fixed excitation frequency, the coherence is greater when the medium through which the sound propagates is a solid medium (e.g. rubber) than when the pathway is air.
-
- 620 Event A: a double clap made by the earphone wearer, approximately 10 cm from the microphone.
- 621 Event B: a double tap event made by the user tapping on the earphone body.
- 622 Event C: A double tap made on a table located approximately 30 cm from the earphone.
- 623 Event D: a second double clap event made by the earphone user, approximately 30 cm from the microphone.
- 624 Event E: a second double tap event made by the user tapping lightly on the earphone body.
Cxyz=CxyCxzCyz
Claims (24)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/079,506 US9271064B2 (en) | 2013-11-13 | 2013-11-13 | Method and system for contact sensing using coherence analysis |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/079,506 US9271064B2 (en) | 2013-11-13 | 2013-11-13 | Method and system for contact sensing using coherence analysis |
Publications (2)
Publication Number | Publication Date |
---|---|
US20150131814A1 US20150131814A1 (en) | 2015-05-14 |
US9271064B2 true US9271064B2 (en) | 2016-02-23 |
Family
ID=53043823
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/079,506 Active 2034-05-14 US9271064B2 (en) | 2013-11-13 | 2013-11-13 | Method and system for contact sensing using coherence analysis |
Country Status (1)
Country | Link |
---|---|
US (1) | US9271064B2 (en) |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9270244B2 (en) * | 2013-03-13 | 2016-02-23 | Personics Holdings, Llc | System and method to detect close voice sources and automatically enhance situation awareness |
GB2533781A (en) * | 2014-12-29 | 2016-07-06 | Nokia Technologies Oy | Method and apparatus for controlling an application |
US20170111723A1 (en) * | 2015-10-20 | 2017-04-20 | Bragi GmbH | Personal Area Network Devices System and Method |
US10635385B2 (en) | 2015-11-13 | 2020-04-28 | Bragi GmbH | Method and apparatus for interfacing with wireless earpieces |
KR200480349Y1 (en) | 2015-12-11 | 2016-05-13 | 박영호 | Bluetooth speaker for public address |
US10110987B2 (en) * | 2015-12-18 | 2018-10-23 | Bose Corporation | Method of controlling an acoustic noise reduction audio system by user taps |
US10091573B2 (en) | 2015-12-18 | 2018-10-02 | Bose Corporation | Method of controlling an acoustic noise reduction audio system by user taps |
US9812149B2 (en) * | 2016-01-28 | 2017-11-07 | Knowles Electronics, Llc | Methods and systems for providing consistency in noise reduction during speech and non-speech periods |
KR102468148B1 (en) * | 2016-02-19 | 2022-11-21 | 삼성전자주식회사 | Electronic device and method for classifying voice and noise thereof |
US10535364B1 (en) * | 2016-09-08 | 2020-01-14 | Amazon Technologies, Inc. | Voice activity detection using air conduction and bone conduction microphones |
WO2018167901A1 (en) * | 2017-03-16 | 2018-09-20 | ヤマハ株式会社 | Headphones |
US10354641B1 (en) | 2018-02-13 | 2019-07-16 | Bose Corporation | Acoustic noise reduction audio system having tap control |
US10425745B1 (en) * | 2018-05-17 | 2019-09-24 | Starkey Laboratories, Inc. | Adaptive binaural beamforming with preservation of spatial cues in hearing assistance devices |
CN110430519A (en) * | 2019-08-07 | 2019-11-08 | 厦门市思芯微科技有限公司 | A kind of acoustics of intelligent sound box is tested automatically and analysis system and method |
US20230144825A1 (en) * | 2021-11-05 | 2023-05-11 | Google Llc | Detecting input gestures using onboard microphones |
Citations (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6628989B1 (en) | 2000-10-16 | 2003-09-30 | Remon Medical Technologies, Ltd. | Acoustic switch and apparatus and methods for using acoustic switches within a body |
US6885260B2 (en) | 2001-05-11 | 2005-04-26 | Ube Industries, Ltd. | Filter using film bulk acoustic resonator and transmission/reception switch |
US7106310B2 (en) | 2001-01-18 | 2006-09-12 | Texzec, Inc. | Acoustic wave touch actuated switch |
US20070079206A1 (en) * | 2005-09-09 | 2007-04-05 | Manish Arora | Method and apparatus to control operation of multimedia device |
US7307627B2 (en) | 2003-05-12 | 2007-12-11 | Illinois Tool Works, Inc. | Individual acoustic wave switch |
US7463249B2 (en) | 2001-01-18 | 2008-12-09 | Illinois Tool Works Inc. | Acoustic wave touch actuated switch with feedback |
US20090046868A1 (en) * | 2004-09-23 | 2009-02-19 | Thomson Licensing | Method and apparatus for controlling a headphone |
US20110096036A1 (en) | 2009-10-23 | 2011-04-28 | Mcintosh Jason | Method and device for an acoustic sensor switch |
US20110142269A1 (en) | 2008-08-12 | 2011-06-16 | Intricon Corporation | Ear Contact Pressure Wave Hearing Aid Switch |
US20110293102A1 (en) * | 2010-06-01 | 2011-12-01 | Sony Corporation | Sound signal processing apparatus, microphone apparatus, sound signal processing method, and program |
US20120070018A1 (en) * | 2010-09-20 | 2012-03-22 | Plantronics, Inc. | Reduced microphone handling noise |
US8358797B2 (en) | 2008-08-12 | 2013-01-22 | Intricon Corporation | Switch for a hearing aid |
US20130028439A1 (en) * | 2011-07-26 | 2013-01-31 | Yuhki Mitsufuji | Input device, signal processing method, program, and recording medium |
US20140119551A1 (en) * | 2011-07-01 | 2014-05-01 | Dolby Laboratories Licensing Corporation | Audio Playback System Monitoring |
-
2013
- 2013-11-13 US US14/079,506 patent/US9271064B2/en active Active
Patent Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6628989B1 (en) | 2000-10-16 | 2003-09-30 | Remon Medical Technologies, Ltd. | Acoustic switch and apparatus and methods for using acoustic switches within a body |
US7106310B2 (en) | 2001-01-18 | 2006-09-12 | Texzec, Inc. | Acoustic wave touch actuated switch |
US7463249B2 (en) | 2001-01-18 | 2008-12-09 | Illinois Tool Works Inc. | Acoustic wave touch actuated switch with feedback |
US6885260B2 (en) | 2001-05-11 | 2005-04-26 | Ube Industries, Ltd. | Filter using film bulk acoustic resonator and transmission/reception switch |
US7307627B2 (en) | 2003-05-12 | 2007-12-11 | Illinois Tool Works, Inc. | Individual acoustic wave switch |
US20090046868A1 (en) * | 2004-09-23 | 2009-02-19 | Thomson Licensing | Method and apparatus for controlling a headphone |
US20070079206A1 (en) * | 2005-09-09 | 2007-04-05 | Manish Arora | Method and apparatus to control operation of multimedia device |
US20110142269A1 (en) | 2008-08-12 | 2011-06-16 | Intricon Corporation | Ear Contact Pressure Wave Hearing Aid Switch |
US8358797B2 (en) | 2008-08-12 | 2013-01-22 | Intricon Corporation | Switch for a hearing aid |
US20110096036A1 (en) | 2009-10-23 | 2011-04-28 | Mcintosh Jason | Method and device for an acoustic sensor switch |
US20110293102A1 (en) * | 2010-06-01 | 2011-12-01 | Sony Corporation | Sound signal processing apparatus, microphone apparatus, sound signal processing method, and program |
US20120070018A1 (en) * | 2010-09-20 | 2012-03-22 | Plantronics, Inc. | Reduced microphone handling noise |
US8787599B2 (en) * | 2010-09-20 | 2014-07-22 | Plantronics, Inc. | Reduced microphone handling noise |
US20140119551A1 (en) * | 2011-07-01 | 2014-05-01 | Dolby Laboratories Licensing Corporation | Audio Playback System Monitoring |
US20130028439A1 (en) * | 2011-07-26 | 2013-01-31 | Yuhki Mitsufuji | Input device, signal processing method, program, and recording medium |
Also Published As
Publication number | Publication date |
---|---|
US20150131814A1 (en) | 2015-05-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9271064B2 (en) | Method and system for contact sensing using coherence analysis | |
US9270244B2 (en) | System and method to detect close voice sources and automatically enhance situation awareness | |
JP6666471B2 (en) | On / off head detection for personal audio equipment | |
US10382853B2 (en) | Method and device for voice operated control | |
US9491542B2 (en) | Automatic sound pass-through method and system for earphones | |
US20180359564A1 (en) | Method And Device For Voice Operated Control | |
US11605395B2 (en) | Method and device for spectral expansion of an audio signal | |
US9271077B2 (en) | Method and system for directional enhancement of sound using small microphone arrays | |
US8577062B2 (en) | Device and method for controlling operation of an earpiece based on voice activity in the presence of audio content | |
US20090067661A1 (en) | Device and method for remote acoustic porting and magnetic acoustic connection | |
US11741985B2 (en) | Method and device for spectral expansion for an audio signal | |
US20220122605A1 (en) | Method and device for voice operated control | |
WO2008128173A1 (en) | Method and device for voice operated control | |
US20220150623A1 (en) | Method and device for voice operated control | |
WO2023093412A1 (en) | Active noise cancellation method and electronic device | |
WO2022254834A1 (en) | Signal processing device, signal processing method, and program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: PERSONICS HOLDINGS, LLC, FLORIDA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:PERSONICS HOLDINGS, INC.;REEL/FRAME:032189/0304 Effective date: 20131231 |
|
AS | Assignment |
Owner name: DM STATON FAMILY LIMITED PARTNERSHIP (AS ASSIGNEE OF MARIA B. STATON), FLORIDA Free format text: SECURITY INTEREST;ASSIGNOR:PERSONICS HOLDINGS, LLC;REEL/FRAME:034170/0933 Effective date: 20141017 Owner name: DM STATON FAMILY LIMITED PARTNERSHIP (AS ASSIGNEE OF MARIA B. STATON), FLORIDA Free format text: SECURITY INTEREST;ASSIGNOR:PERSONICS HOLDINGS, LLC;REEL/FRAME:034170/0771 Effective date: 20131231 Owner name: DM STATON FAMILY LIMITED PARTNERSHIP (AS ASSIGNEE Free format text: SECURITY INTEREST;ASSIGNOR:PERSONICS HOLDINGS, LLC;REEL/FRAME:034170/0771 Effective date: 20131231 Owner name: DM STATON FAMILY LIMITED PARTNERSHIP (AS ASSIGNEE Free format text: SECURITY INTEREST;ASSIGNOR:PERSONICS HOLDINGS, LLC;REEL/FRAME:034170/0933 Effective date: 20141017 |
|
AS | Assignment |
Owner name: PERSONICS HOLDINGS, LLC, FLORIDA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GOLDSTEIN, STEVEN;USHER, JOHN;MCINTOSH, JASON;REEL/FRAME:037416/0644 Effective date: 20151221 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: DM STATION FAMILY LIMITED PARTNERSHIP, ASSIGNEE OF STATON FAMILY INVESTMENTS, LTD., FLORIDA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PERSONICS HOLDINGS, INC.;PERSONICS HOLDINGS, LLC;REEL/FRAME:042992/0493 Effective date: 20170620 Owner name: STATON TECHIYA, LLC, FLORIDA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DM STATION FAMILY LIMITED PARTNERSHIP, ASSIGNEE OF STATON FAMILY INVESTMENTS, LTD.;REEL/FRAME:042992/0524 Effective date: 20170621 Owner name: DM STATION FAMILY LIMITED PARTNERSHIP, ASSIGNEE OF Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:PERSONICS HOLDINGS, INC.;PERSONICS HOLDINGS, LLC;REEL/FRAME:042992/0493 Effective date: 20170620 |
|
AS | Assignment |
Owner name: DM STATON FAMILY LIMITED PARTNERSHIP, ASSIGNEE OF STATON FAMILY INVESTMENTS, LTD., FLORIDA Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE'S NAME PREVIOUSLY RECORDED AT REEL: 042992 FRAME: 0493. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNORS:PERSONICS HOLDINGS, INC.;PERSONICS HOLDINGS, LLC;REEL/FRAME:043392/0961 Effective date: 20170620 Owner name: DM STATON FAMILY LIMITED PARTNERSHIP, ASSIGNEE OF Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNEE'S NAME PREVIOUSLY RECORDED AT REEL: 042992 FRAME: 0493. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT;ASSIGNORS:PERSONICS HOLDINGS, INC.;PERSONICS HOLDINGS, LLC;REEL/FRAME:043392/0961 Effective date: 20170620 Owner name: STATON TECHIYA, LLC, FLORIDA Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE ASSIGNOR'S NAME PREVIOUSLY RECORDED ON REEL 042992 FRAME 0524. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT OF THE ENTIRE INTEREST AND GOOD WILL;ASSIGNOR:DM STATON FAMILY LIMITED PARTNERSHIP, ASSIGNEE OF STATON FAMILY INVESTMENTS, LTD.;REEL/FRAME:043393/0001 Effective date: 20170621 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2552); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY Year of fee payment: 8 |