Sound data output and manipulation using haptic feedback
(19)AUSTRALIAN PATENT OFFICE (54) Title Sound data output and manipulation usinghaptic feedback (51)6 International Patent Classification(s) A63H 005/00 G04B 013/00 (21) Application No: 2002347829 (22) Application Date: 2002.10.08 (87) WIPO No: WO03/032538 (30) Priority Data (31) Number (32) Date (33) Country 09/975,320 2001 .10.10 US (43) Publication Date : 2003 .04.22 (43) Publication Journal Date : 2003 .07.03 (71) Applicant(s) IMMERSION CORPORATION (72) Inventor(s) Chu, Lonny L. (-1-1) Application NoAU2002347829 A8(19)AUSTRALIAN PATENT OFFICE (54) Title Sound data output and manipulation usinghaptic feedback (51)6 International Patent Classification(s) A63H 005/00 G04B 013/00 (21) Application No: 2002347829 (22) Application Date: 2002.10.08 (87) WIPO No: WO03/032538 (30) Priority Data (31) Number (32) Date (33) Country 09/975,320 2001 .10.10 US (43) Publication Date : 2003 .04.22 (43) Publication Journal Date : 2003 .07.03 (71) Applicant(s) IMMERSION CORPORATION (72) Inventor(s) Chu, Lonny L.-1- In an embodiment, a device which comprises means for generating an audio signal based on sound data, the audio signal configured to produce sound from an audio producing device; means for generating a haptic command based on the sound data, the haptic command configured to cause a haptic feedback device to output a haptic sensation, the haptic sensation being associated with at least one characteristic of the sound data; and means for receiving a navigation command from a user experiencing the haptic sensation via the haptic feedback device, the navigation command associated with the sound data and based, at least in part, on the haptic sensation. What is claimed is : CLAIMS 1. A method for associating haptic sensations with sound data to assist in navigating through and editing said sound data, the method comprising: loading at least a portion of said sound data into a memory of a computer; playing said sound data such that an audio signal is generated and used for outputting sound from an audio device, wherein said playing of said sound is controlled by user input received by said computer from a user for navigation through said sound data;STDC0433 and generating haptic commands based on said sound data, said haptic commands used to output haptic sensations to said user by a haptic feedback device manipulated by said user and in communication with said computer, said haptic sensations corresponding to one or more characteristics of said sound data to assist said user in discerning features of said sound data during said navigation through and editing of said sound data.
2. A method as recited in claim 1 wherein said user can control a speed of said playing of said sound data.
3. A method as recited in claim 1 wherein said user can control a direction of said playing of said sound data, said direction including forward and reverse.
4. A method as recited in claim 1 wherein said haptic sensations are continuously output during said playing of said sound data, and wherein an output haptic sensation has a magnitude based on an amplitude of said sound data currently being played.
5. A method as recited in claim 4 wherein said haptic sensations have a magnitude directly proportional to said amplitude of said played sound data. <Desc/Clms Page number 28> 6. A method as recited in claim 4 wherein said haptic sensations have a magnitude inversely proportional to said amplitude of said played sound data.
7. A method as recited in claim 1 wherein said haptic sensations are output only when features of said sound data having predetermined characteristics are played.
8. A method as recited in claim 7 wherein said predetermined characteristics include a rise in amplitude of said sound data over a predetermined threshold.
9. A method as recited in claim 8 wherein said predetermined characteristics include a drop in amplitude of a predetermined amount following said rise in amplitude.
10. A method as recited in claim 7 wherein said predetermined characteristics include a minimum time interval that must occur between two peaks in amplitude to allow a haptic sensation to be output when the second of said peaks is played.
11. A method as recited in claim 1 further comprising filtering said sound data such that undesired frequencies of sound generated by said sound data are removed and a desired range of frequencies remain.
12. A method as recited in claim 11 wherein said desired range of frequencies are associated with a particular type of haptic sensation.
13. A method as recited in claim 1 wherein one of said haptic commands is generated and a haptic sensation output if a marker that was previously stored for a particular location in said sound data is reached during said playing of said sound.
14. A method as recited in claim 1 further comprising storing a portion of said sound data in a secondary buffer and processing said portion of said sound data to find said one or more characteristics in said sound data real-time during said playing of said sound data. <Desc/Clms Page number 29> 15. A method as recited in claim 1 further comprising displaying a visual representation of said sound data and a moving cursor to indicate a current part of said sound data that is being played.
16. A method for pre-processing sound data to allow haptic sensations associated with said sound data to be output when said sound data is played to a user, the method comprising: loading at least a portion of said sound data into a memory of a computer, said sound data describing a sound waveform; processing said sound data to find a sound feature having one or more predetermined characteristics; and when said sound feature is found, storing a marker in a list of markers, wherein said marker indicates a location of said associated sound feature in said sound data, said location to be associated with at least one haptic sensation such that when said sound data is played, said associated at least one haptic sensation is output to a user when said marker is reached during said playing of said sound data.
17. A method as recited in claim 16 wherein said one or more predetermined characteristics include a rise in amplitude of said sound waveform described by said sound data, said rise in amplitude being over a predetermined threshold amplitude.
18. A method as recited in claim 16 wherein said one or more predetermined characteristics include a drop in amplitude of a predetermined amount following said rise in amplitude.
19. A method as recited in claim 16 wherein said one or more predetermined characteristics include a minimum time interval that must occur between two peaks in amplitude of said sound waveform to allow a haptic sensation to be output when the second of said peaks is played. <Desc/Clms Page number 30> 20. A method as recited in claim 16 wherein said processing of said sound data includes filtering said sound data such that undesired frequencies of said sound data are removed.
21. A method as recited in claim 16 further comprising storing an indication of one or more particular haptic sensations that are to be associated with a particular one of said markers.
22. A method as recited in claim 21 wherein a type of said one more particular haptic sensations is based on said one or more predetermined characteristics of said sound feature to which said particular marker refers.
23. A computer readable medium including program instructions for performing steps that associate haptic sensations with sound data to assist in navigating through and editing said sound data, the steps comprising: loading at least a portion of said sound data into a memory of a computer; playing said sound data such that an audio signal is generated and used for outputting audio sound from an audio device, wherein said playing of said sound is controlled by user input received by said computer from a user for navigation of said sound data;STDC0433 and generating haptic commands based on said sound data, said haptic commands used to output haptic sensations to said user by a haptic feedback device manipulated by said user and in communication with said computer, said haptic sensations corresponding to one or more characteristics of said sound data to assist said user in discerning features of said sound data during said navigation through and editing of said sound data.
24. A method as recited in claim 23 wherein said user can control a speed of said playing of said sound data. <Desc/Clms Page number 31> 25. A method as recited in claim 23 wherein said user can control a direction of said playing of said sound data, said direction including forward and reverse.
26. A method as recited in claim 23 wherein said haptic sensations are continuously output during said playing of said sound data, and wherein an output haptic sensation has a magnitude based on an amplitude of said sound data currently being played.
27. A method as recited in claim 23 wherein said haptic sensations are output only when features of said sound data having predetermined characteristics are played.
28. A method as recited in claim 23 wherein one of said haptic commands is generated and a haptic sensation output if a marker that was previously stored for a particular location in said sound data is reached during said playing of said sound.
29. A method as recited in claim 23 further comprising storing a portion of said sound data in a secondary buffer and processing said portion of said sound data to find said one or more characteristics in said sound data real-time during said playing of said sound data.