I lllll llllllll Ill lllll lllll lllll lllll lllll 111111111111111111111111111111111
`US009336689B2
`
`c12) United States Patent
`Romriell et al.
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 9,336,689 B2
`May 10, 2016
`
`(54) METHODS AND APPARATUSES RELATED
`TO TEXT CAPTION ERROR CORRECTION
`
`(71) Applicant: CaptionCall, LLC, Salt Lake City, UT
`(US)
`
`(72)
`
`Inventors: Joseph N. Romriell, Alpine, UT (US);
`Scot Brooksby, Salt Lake City, UT
`(US); Shane Arthur Roylance,
`Farmington, UT (US); Brian Chevrier,
`Salt Lake City, UT (US); Robert
`Cochran Puzey, North Ogden, UT (US)
`
`(73) Assignee: CaptionCall, LLC, Salt Lake City, UT
`(US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 14/530,407
`
`(22) Filed:
`
`Oct. 31, 2014
`
`(65)
`
`Prior Publication Data
`
`US 2015/0051908 Al
`
`Feb. 19, 2015
`
`(63)
`
`(51)
`
`(52)
`
`Related U.S. Application Data
`
`Continuation of application No. 13/768,918, filed on
`Feb. 15, 2013, now abandoned, which is a continuation
`of application No. 12/624,973, filed on Nov. 24, 2009,
`now Pat. No. 8,379,801.
`
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`
`Int. Cl.
`G09B 21100
`G06F 17124
`G06F 17127
`GlOL 15126
`H04L 12166
`U.S. Cl.
`CPC .............. G09B 211009 (2013.01); G06F 17124
`(2013.01); G06F 171273 (2013.01); G06F
`1712765 (2013.01); GlOL 15126 (2013.01);
`GlOL 151265 (2013.01); H04L 12166 (2013.01)
`
`(58) Field of Classification Search
`CPC ... H04L 12/66; G06F 17/24; G06F 17/27565;
`G06F 17/273; GlOL 15/265; GlOL 15/26;
`G09B 21/009
`USPC .............. 379/52, 88.01; 434/179; 704/3, 235,
`704/270, 277, 260; 348/468; 715/224
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,572,423 A
`5,576,955 A
`
`1111996 Church et al.
`1111996 Newbold et al.
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`
`1091303
`
`4/2001
`
`OTHER PUBLICATIONS
`
`Ultratec, Inc. and Captel, Inc., Plaintiffs v. Sorenson Communica(cid:173)
`tions, Inc. and Caption Call, LLC., Defendants, Opinion and Order of
`the United States District Court for the Western District ofWisconsin,
`dated Aug. 28, 2014.
`
`(Continued)
`
`Primary Examiner - Gerald Gauthier
`(74) Attorney, Agent, or Firm - TraskBritt
`
`ABSTRACT
`(57)
`Systems and methods related to providing error correction in
`a text caption are disclosed. A method may comprise display(cid:173)
`ing a text caption including one or more blocks of text on each
`of a first device and a second device remote from the first
`device. The method may also include generating another
`block of text and replacing a block of text of the text caption
`with the another block of text. Furthermore, the method may
`include displaying the text caption on the second device hav(cid:173)
`ing the block of text of the first text caption replaced by the
`another block of text.
`
`20 Claims, 4 Drawing Sheets
`
`Displaying a text caption including one or more blocks of text on
`each of a first device and a second device
`
`Identifying one or more errors within a block of text within the
`text caption
`
`612
`
`'"
`
`Generating a new block of text having a corrected word for each
`identifiedam:ir
`
`604
`
`Replacing the block of text having the one or more errors with the
`new block of text In the text caption
`
`606
`
`Displaying the teJtt caption on the second device wherein the
`block of text having the one or more errors is replaced by the
`new block of text
`
`Tagging each corrected word displayed within the t&xt
`caption
`
`008
`
`610
`
`Ultratec Exhibit 1003
`Ultratec v Sorenson IP Holdings Page 1 of 12
`
`

`

`US 9,336,689 B2
`Page 2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`8,150,689 B2
`8,213,578 B2
`8,296,811 Bl
`8,355,914 B2
`8,379,801 B2 *
`
`8,515,748 B2
`8,775,175 Bl*
`
`200110005825 Al
`200110047258 Al
`200110047270 Al
`2002/0055974 Al
`2002/0091832 Al
`2002/0120647 Al
`2002/0194278 Al
`2003/0023689 Al
`2003/0028448 Al
`2003/0046350 Al
`2003/0125950 Al
`2003/0177009 Al
`2003/0187659 Al
`200410019638 Al
`2004/0064317 Al
`2004/0204941 Al
`2005/0069107 Al
`2005/0096910 Al*
`
`2005/0102140 Al
`2005/0131840 Al
`2005/0226394 Al
`2005/0283726 Al
`2005/0289130 Al
`2006/0047767 Al
`2006/0064631 Al
`2006/0092291 Al
`2006/0095550 Al
`2006/0161631 Al
`2007/0118373 Al
`2007/0118374 Al
`2007/0124387 Al
`2007/0271510 Al
`2008/0155411 Al
`2008/0177623 Al
`2008/0187108 Al
`2008/0260032 Al
`2008/0288250 Al
`2009/0037171 Al
`200910112852 Al *
`2009/0192782 Al*
`
`2009/0263098 Al
`2009/0326938 Al
`2010/0063815 Al
`2010/0135486 Al
`2010/0138221 Al
`2010/0169073 Al*
`
`2010/0287486 Al
`2010/0332217 Al
`201110018812 Al
`2011100407 54 Al
`201110123003 Al*
`
`2012/0130706 Al
`2012/0250836 Al
`2012/0250837 Al
`2013/0158995 Al*
`
`2015/0051908 Al*
`
`4/2012 Beach et al.
`7/2012 Engleke et al.
`10/2012 Begeja et al.
`112013 Joh et al.
`212013 Romriell ............... G06F 17 /273
`345/173
`
`8/2013 Gangemi et al.
`7/2014 Nagel ..................... GlOL 15/22
`704/235
`
`6/2001 Engelke et al.
`1112001 Rodrigo
`1112001 Gusicket al.
`512002 Hawkes et al.
`712002 Low et al.
`8/2002 Amano
`12/2002 Golan
`112003 Brown eta!.
`212003 Joseph et al.
`3/2003 Chintalapati et al.
`7/2003 Avila et al.
`9/2003 Odinak et al.
`10/2003 Cho et al.
`112004 Makagon et al.
`412004 Othmer et al.
`10/2004 Israch et al.
`3/2005 Tanaka
`512005 Watson ................... GlOL 15/26
`704/260
`
`512005 Davne et al.
`6/2005 Pintsov et al.
`10/2005 Engelke et al.
`12/2005 Lunati
`12/2005 Cohen et al.
`3/2006 Dodrill et al.
`3/2006 Parker
`512006 Bodie
`512006 Nemrnaier et al.
`712006 Lira
`5/2007 Wise et al.
`5/2007 Wise et al.
`5/2007 Galloway
`11/2007 Grigoriu et al.
`6/2008 Christensen
`712008 Fritsch et al.
`8/2008 Engelke et al.
`10/2008 Hu et al.
`11/2008 Rennillo et al.
`212009 McFarland et al.
`412009 Kim .................... G06F 3/04895
`712009 Drewes .. ............. G06F 17/2818
`704/3
`
`10/2009 Hyun eta!.
`12/2009 Marila et al.
`3/2010 Cloran et al.
`6/2010 Schneider
`6/2010 Boys
`712010 Almagro ................. GlOL 15/26
`704/3
`
`1112010 Coddington
`12/2010 Wintner et al.
`112011 Baird
`212011 Peto et al.
`5/2011 Romriell ............... G06F 17 /273
`379/52
`
`5/2012 Qiu et al.
`10/2012 Engleke et al.
`10/2012 Engleke et al.
`612013 Romriell ............... G06F 17 /273
`704/235
`212015 Romriell ............... G06F 17 /273
`704/235
`
`OTHER PUBLICATIONS
`
`United States Patent and Trademark Office, Before the Patent Trial
`and Appeal Board pleading, Ultratec, Inc. v. CaptionCall LLC,
`Patent Owner's Response to Decision to Institute Inter Partes Review
`dated Jan. 30, 2014, 60 pages.
`
`6/1997 Bonneau et al.
`2/1998 Luther et al.
`5/1998 Gipson
`9/1998 Ryan
`12/1998 Waibel eta!.
`6/1999 Engelke
`10/1999 Engelke et al.
`1111999 Colwell et al.
`612000 Engelke et al.
`612000 Engelke et al.
`8/2000 Bartosik
`112001 Olmedo
`112001 VanAlstine
`5/2001 Engelke
`6/2001 Begeja et al.
`10/2001 Engelke et al.
`1112001 Lewis et al.
`3/2002 Hu et al.
`10/2002 Gibbon
`12/2002 Engelke et al.
`112003 Engelke et al.
`112003 Engelke .
`
`5/2003 Edgar
`
`H04M 112475
`379/100.09
`
`112003 Angell et al.
`4/2003 Engelke et al.
`5/2003 Engelke .................. GlOL 15/26
`379/100.09
`G09B 17/04
`434/179
`
`5,636,340 A
`5,715,370 A
`5,754,737 A
`5,809,112 A
`5,855,000 A
`5,909,482 A
`5,974,116 A
`5,978,654 A
`6,075,841 A
`6,075,842 A
`6,101,467 A
`6,174,170 Bl
`6,175,819 Bl
`6,233,314 Bl
`6,243,445 Bl
`6,307,921 Bl
`6,314,397 Bl
`6,360,010 Bl
`6,473,778 Bl
`6,493,426 B2
`6,504,910 Bl
`6,510,206 B2 *
`
`6,513,003 Bl
`6,549,611 B2
`6,567 ,503 B2 *
`
`6,568,939 Bl *
`
`6,594,346 B2
`6,603,835 B2
`6,748,053 B2
`6,816,834 B2
`6,820,055 B2
`6,882,707 B2
`6,885,731 B2
`6,934,366 B2
`6,940,617 B2
`6,944,593 B2
`6,990,335 Bl
`6,999,932 Bl
`7,003,082 B2
`7,006,604 B2
`7,035,804 B2
`7,130,401 B2
`7,130,790 Bl
`7,164,753 B2 *
`
`7,197,459 Bl
`7,269,561 B2 *
`
`7/2003 Engelke
`8/2003 Engelke et al.
`6/2004 Engelke et al.
`1112004 Jaroker
`1112004 Saindon et al.
`412005 Engelke et al.
`412005 Engelke et al.
`8/2005 Engelke et al.
`912005 Ma eta!.
`912005 Kuzunuki et al.
`112006 Sharnoon et al.
`212006 Zhou
`212006 Engelke et al.
`212006 Engelke
`412006 Saindon et al.
`10/2006 Rampey et al.
`10/2006 Flanagan et al.
`112007 Engelke .................. GlOL 15/26
`379/52
`
`3/2007 Harinarayan et al.
`9/2007 Mukhtar ................. GlOL 15/26
`704/235
`
`12/2007 Tannenbaum
`7,305,627 B2
`112008 McClelland
`7,315,612 B2
`112008 Engelke et al.
`7,319,740 B2
`5/2008 Rennillo et al.
`7,376,561 B2
`9/2008 Cervantes et al.
`7,428,702 Bl
`7,444,285 B2 * 10/2008 Forbes .................... GlOL 15/26
`379/88.01
`
`7,483,833 B2
`7,511,761 B2 *
`
`7,516,404 Bl
`7,539,086 B2
`7,543,033 B2
`7,555,104 B2
`7,660,398 B2
`7,698,140 B2
`7,769,586 B2
`7,774,694 B2 *
`
`112009 Peters
`3/2009 Lynch ...................... G09G 5124
`348/468
`
`412009 Colby
`512009 Jaroker
`612009 Vincent
`612009 Engelke
`2/2010 Engelke et al.
`4/2010 Bhardwaj et al.
`8/2010 Bennett et al.
`8/2010 Watson ................... GlOL 15/26
`715/224
`
`7,792,675 B2
`7,826,635 B2
`7,881,441 B2
`7,908,145 B2
`8,140,337 B2
`
`9/2010 Ramaswamy et al.
`1112010 Barbara
`212011 Engelke et al.
`3/2011 Bennett et al.
`3/2012 N akazawa et al.
`
`Ultratec Exhibit 1003
`Ultratec v Sorenson IP Holdings Page 2 of 12
`
`

`

`US 9,336,689 B2
`Page 3
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`United States Patent and Trademark Office, Before the Patent Trial
`and Appeal Board pleading, Ultratec, Inc. v. Sorenson Communica(cid:173)
`tions, Inc., Caption Call LLC, and Wilmington Trust, National Asso(cid:173)
`ciation, Petitioner's Reply Under 37 C.F.R. § 42.23 dated Apr. 16,
`2014, 19 pages.
`United States Patent and Trademark Office, Before the Patent Trial
`and Appeal Board pleading, Ultratec, Inc. v. CaptionCall LLC,
`Notice of Disclaimer of Claims 1, 2, 7 and 9 of U.S. Pat. No.
`8,379,801 dated Jul. 1, 2014, 5 pages.
`United States Patent and Trademark Office, Before the Patent Trial
`and Appeal Board pleading, Ultratec, Inc. v. Sorenson Communica(cid:173)
`tions, Inc., Caption Call LLC, and Wilmington Trust, National Asso(cid:173)
`ciation, Decision re Petitioner's Request for Rehearing, dated Jan.
`10, 2014, 6 pages.
`United States Patent and Trademark Office, Before the Patent Trial
`and Appeal Board pleading, Ultratec, Inc. v. , Sorenson Communi(cid:173)
`cations, Inc., CaptionCall LLC, and Wilmington Trust, National
`Association, Request for Rehearing Under 37 C.F.R. § 42.7l(D)
`dated Nov. 27, 2013. 9 pages.
`
`United States Patent and Trademark Office, Before the Patent Trial
`and Appeal Board pleading, Ultratec, Inc. v. CaptionCall LLC, Deci(cid:173)
`sion re Institution of Inter Partes Review, dated Nov. 13, 2013, 25
`pages.
`Disclaimer in Patent Under 37 CFR l.32l(a) for U.S. Pat. No.
`8,379,801, dated Jun. 30, 2014.
`Petition for Inter Partes Review ofU.S. Pat. No. 8,379,801 (including
`exhibits) dated May 17, 2013.
`Advantage Software, About Us webshot; www.eclipsecat.com/
`contentfabout-us; May 15, 2013, 1 page.
`e-Tips Newsletter, Norton 360: Outstanding Protection; Nov. 2007;
`Issue 71; 6 pages.
`CapTel® News, News & Helpful Tips for People Who Use CapTel,
`Summer 2007, newsletter.
`User Guide to Sprint CapTel Services, as early as Mar. 2009.
`All CapTel 800 phones are being updated, Press Release, <http://
`www.kcdhh.org/bulletin/CapTel800Update.pdf>, as early as Oct.
`2011.
`CapTel News from Ultratec-Jul. 2005.
`Ultratec, Inc., v. Captioncall, LLC., IPR2013-00288, Final Written
`Decision dated Oct. 30, 2014.
`* cited by examiner
`
`Ultratec Exhibit 1003
`Ultratec v Sorenson IP Holdings Page 3 of 12
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 1of4
`
`US 9,336,689 B2
`
`(
`c c -
`
`Ultratec Exhibit 1003
`Ultratec v Sorenson IP Holdings Page 4 of 12
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 2 of 4
`
`US 9,336,689 B2
`
`~100
`
`190
`(
`
`~
`
`180
`
`120
`
`(
`
`~134
`
`/152
`
`c154
`
`I
`1156
`,--118
`
`I
`
`~170
`
`l--'132
`
`~
`110
`
`/102
`
`,,,.--104
`
`I
`1106
`c.108
`
`I
`
`FIG. 2
`
`I
`
`I
`
`I
`
`I
`
`Ultratec Exhibit 1003
`Ultratec v Sorenson IP Holdings Page 5 of 12
`
`

`

`\C = N
`0--, °" 00
`\C w w
`rJl
`d
`
`0 .....
`
`.i;...
`
`(.H
`
`.....
`1J1 =(cid:173)
`
`('D
`('D
`
`0 ....
`~ ....
`~
`
`N
`~o
`
`O'I
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`FIG. 5
`
`to the party
`Do you know what time you are going
`~~;1~':n~< !~~ groce~::y
`I am lea~w~rk right now and I am
`
`416
`
`414
`
`Hi John
`
`{510
`
`FIG. 3
`
`FIG. 4
`
`my way horn~ 414
`going to stop at the grocery store on
`I am leaving work right now and I am
`
`416}
`
`1414
`
`to the party
`Do you know what time you are going
`
`way home
`going to top at the grocery door on my
`I am leaving work right now and I am
`
`{316
`
`r314
`
`Hi John
`
`(312
`
`310\
`
`Ultratec Exhibit 1003
`Ultratec v Sorenson IP Holdings Page 6 of 12
`
`

`

`U.S. Patent
`
`May 10, 2016
`
`Sheet 4 of 4
`
`US 9,336,689 B2
`
`600\
`
`Displaying a text caption including one or more blocks of text on
`each of a first device and a second device
`
`v 612
`
`I
`Identifying one or more errors within a block of text within the LF 602
`text caption
`
`I
`Generating a new block of text having a corrected word for each v- 604
`identified error
`
`I
`Replacing the block of text having the one or more errors with the v- 606
`new block of text in the text caption
`
`I
`
`Displaying the text caption on the second device wherein the
`block of text having the one or more errors is replaced by the
`new block of text
`
`608
`LF
`
`I
`
`Tagging each corrected word displayed within the text
`caption
`
`v- 610
`
`FIG. 6
`
`Ultratec Exhibit 1003
`Ultratec v Sorenson IP Holdings Page 7 of 12
`
`

`

`US 9,336,689 B2
`
`1
`METHODS AND APPARATUSES RELATED
`TO TEXT CAPTION ERROR CORRECTION
`
`CROSS-REFERENCE TO RELATED
`APPLICATION
`
`This application is a continuation of U.S. patent applica(cid:173)
`tion Ser. No. 13/768,918, filed Feb. 15, 2013, pending, which
`application is a continuation of U.S. patent application Ser.
`No. 12/624,973, filed Nov. 24, 2009, now U.S. Pat. No. 8,379,
`801, issued Feb. 19, 2013, the disclosure of each of which is
`hereby incorporated herein by this reference in its entirety.
`
`TECHNICAL FIELD
`
`Embodiments of the disclosure relate generally to text
`captioning and more specifically to correction of errors
`within a text caption.
`
`BACKGROUND
`
`2
`caption not only interrupts the continuity of a conversation
`but also fails to provide context of the correction to the hear(cid:173)
`ing-impaired user. Therefore, a hearing-impaired user must
`determine where the corrected text should be inserted into the
`previously provided text caption.
`A need exists to improve text correction of a text captioning
`system. Specifically, there is a need for methods of providing
`text caption correction while providing a user with context of
`a correction and without distracting the user or interrupting
`the continuity of a conversation between a hearing-impaired
`10 user and a hearing user.
`
`BRIEF SUMMARY
`
`In one embodiment of the disclosure, a method of provid-
`15 ing error correction in a text caption is disclosed. The method
`may comprise displaying a text caption including one or more
`blocks of text on each of a first device and a second device
`remote from the first device. The method may also include
`generating another block of text and replacing a block of text
`20 of the text caption with the another block of text. Further(cid:173)
`more, the method may include displaying the text caption on
`the second device having the block of text of the first text
`caption replaced by the another block of text.
`In another embodiment of the disclosure, a communication
`25 system is provided. The communication system may include
`a plurality of devices, wherein each device of the plurality
`includes a processor and a computer-readable medium
`coupled to the processor. The communication system may
`further include a plurality of application programs, wherein
`30 each application program is stored in an associated computer(cid:173)
`readable medium. When executed by the processor, one or
`more application programs are configured to display a text
`caption including one or more blocks of text on a display
`device of each of a first device of the plurality of devices and
`35 a second device of the plurality of devices. One or more
`application programs may be configured to generate another
`block of text and replace one block of text of the text caption
`with the another block of text. Furthermore, one or more
`application programs may be configured to display the cor-
`40 rected text caption on the display device of the second device,
`wherein the at least one block of text of the text caption is
`replaced by the another block of text.
`Another embodiment of the disclosure may include a com(cid:173)
`puter-readable media storage medium storing instructions
`45 that when executed by a processor cause the processor to
`perform instructions in accordance with one or more embodi(cid:173)
`ments of the disclosure.
`
`Modem telecommunication services provide features to
`assist those who are deaf or hearing-impaired. One such
`feature is a telecommunication device for the deaf (TDD).
`Hearing-capable users communicate with hearing-impaired
`users who are users of TDD devices through so-called
`"relays." A relay is a telecommunication intermediary ser(cid:173)
`vice, which is intended to permit a deaf or a hearing-impaired
`person to utilize a normal telephone network. A relay service
`may include an operator, referred to as a "call assistant," who
`serves as a human intermediary between a hearing user and a
`hearing-impaired user. The call assistant communicates with
`the hearing-impaired user using a TDD and communicates
`with the hearing user by voice over a standard telephone line.
`A text captioned telephone system employs a relay service
`in a mode where the relay transmits both the voice of the
`hearing user and a text stream of the words spoken by the
`hearing user. A hearing-impaired user using a text captioned
`telephone, or telephone enabled to do text enhanced tele(cid:173)
`phony, may carry on a normal telephone conversation with a
`hearing user while a text transcription of the words spoken by
`the hearing user is displayed on the text captioned telephone.
`The text transcription may allow the hearing-impaired user to
`confirm his or her understanding of the words spoken by the
`hearing user.
`More specifically, during a communication session, a call
`assistant may listen to the voice signal of a hearing user and
`"revoice" the words to a speech recognition computer pro(cid:173)
`gram tuned to that call assistant's voice. A text transcription
`output from the speech recognition computer is then trans- 50
`mitted to the text captioned telephone being used by the
`hearing-impaired user. Even with revoicing provided by a
`trained call assistant, the text transcription received by the
`hearing-impaired user may include errors. Therefore, correc(cid:173)
`tion of the errors within the text transcription may be 55
`required.
`According to various conventional methods, errors within
`a text caption are corrected by either backspacing an error in
`a text caption and displaying corrected text or providing a
`corrected portion (e.g., a word or a sentence) at the end of a 60
`previously provided text caption. Although, backspacing an
`error in a text caption and displaying corrected text may
`provide a hearing-impaired user with a context for the cor(cid:173)
`rection, this method is distracting to a hearing-impaired user
`and interrupts the continuity of a conversation between a 65
`hearing-impaired user and a hearing user. Further, providing
`a corrected portion at the end of a previously provided text
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 illustrates a hearing-impaired communication sys(cid:173)
`tem, in accordance with an embodiment of the disclosure;
`FIG. 2 is a block diagram of the communication system of
`FIG. 1, according to an embodiment of the disclosure;
`FIG. 3 illustrates a text caption including errors;
`FIG. 4 illustrates a corrected block of text;
`FIG. 5 illustrates a corrected text caption including cor(cid:173)
`rected text and further including identified corrected words,
`in accordance with an embodiment of the disclosure; and
`FIG. 6 is a flow chart illustrating a method of providing
`error correction within a text caption, according to an
`embodiment of the disclosure.
`
`DETAILED DESCRIPTION
`
`In the following detailed description, reference is made to
`the accompanying drawings which form a part hereof and, in
`
`Ultratec Exhibit 1003
`Ultratec v Sorenson IP Holdings Page 8 of 12
`
`

`

`US 9,336,689 B2
`
`3
`which is shown by way of illustration, specific embodiments
`in which the invention may be practiced. These embodiments
`are described in sufficient detail to enable those of ordinary
`skill in the art to practice the invention, and it is to be under(cid:173)
`stood that other embodiments may be utilized, and that struc(cid:173)
`tural, logical, and electrical changes may be made within the
`scope of the disclosure.
`In this description, functions may be shown in block dia(cid:173)
`gram form in order not to obscure the disclosure in unneces(cid:173)
`sary detail. Furthermore, specific implementations shown 10
`and described are only examples and should not be construed
`as the only way to implement the disclosure unless specified
`otherwise herein. Block definitions and partitioning of logic
`between various blocks represent a specific implementation.
`It will be readily apparent to one ofordinary skill in the art that 15
`the various embodiments of the disclosure may be practiced
`by numerous other partitioning solutions. For the most part,
`details concerning timing considerations, and the like, have
`been omitted where such details are not necessary to obtain a
`complete understanding of the disclosure in its various 20
`embodiments and are within the abilities of persons of ordi(cid:173)
`nary skill in the relevant art.
`Referring in general to the following description and
`accompanying drawings, various aspects of the disclosure are
`illustrated to show its structure and method of operation. 25
`Common elements of the illustrated embodiments are desig(cid:173)
`nated with like numerals. It should be understood the figures
`presented are not meant to be illustrative of actual views of
`any particular portion of the actual structure or method, but
`are merely idealized representations which are employed to 30
`more clearly and fully depict the disclosure.
`When executed as firmware or software, the instructions
`for performing the methods and processes described herein
`may be stored on a computer readable medium. A computer
`readable medium includes, but is not limited to, magnetic and 35
`optical storage devices such as disk drives, magnetic tape,
`CDs (compact disks), DVDs (digital versatile discs or digital
`video discs), and semiconductor devices such as RAM,
`DRAM, ROM, EPROM, and Flash memory.
`FIG. 1 illustrates a communication system 100 configured 40
`to facilitate a communication session between a hearing(cid:173)
`impaired user and a hearing-capable user, in accordance with
`one or more embodiments of the disclosure. Communication
`system 100 may include communication device 120, commu(cid:173)
`nication device 190, and a relay service 110. Communication 45
`device 190 may be coupled to communication device 120 via
`a network 180 and communication device 120 may be
`coupled to relay service 110 via a network 170. For example
`only, network 170 and network 180 may each be imple(cid:173)
`mented according to the standards and bandwidth require- 50
`ments of a telephone network such as the Public Switch
`Telephone Network (PSTN). The use of the term "telephone
`network" as used herein also contemplates networks that are
`compatible and configured to provide communications using
`digital standards, an example of which includes Voice Over 55
`Internet Protocol (VOIP). By way of example and not limi(cid:173)
`tation, enhanced bandwidth requirements of various commu(cid:173)
`nication devices may be better accommodated by providing
`high bandwidth networks and connections thereto through
`various enhanced-bandwidth networking technology includ- 60
`ing DSL, Cable, Ethernet, etc.
`As described more fully below, relay service 110 may be
`configured to provide interpretive services to hearing-im(cid:173)
`paired user 140. More specifically, a human "call assistant"
`within relay service 110 may be employed to facilitate a 65
`communication session between a hearing-impaired user 140
`and a hearing-capable user 160. By way of example only,
`
`4
`communication device 190 may comprise a conventional
`voice phone. As such, hearing-capable user 160 may interact
`in a conventional marmer with communication device 120
`through the use of a voice-based dialogue conveyed over
`communication device 190. The voice of hearing-capable
`user 160 may be conveyed over communication device 190
`and may be transmitted over network 180 to communication
`device 120. Furthermore, the voice conveyed over communi(cid:173)
`cation device 190 may be transmitted through communica(cid:173)
`tion device 120, over network 170, and to relay service 110.
`By way of example, communication device 120 may
`include a captioned telephone, a telephone enabled for text
`enhanced telephony, or any other suitable communication
`device configured to receive and display text. Hearing-im(cid:173)
`paired user 140 may interact in a conventional manner with
`communication device 190 through the use of a voice-based
`dialogue conveyed over communication device 120. Fur(cid:173)
`therer more, as described more fully below, communication
`device 120 may be configured to receive and display a text
`transcription of a voice signal sent from relay service 110 via
`network 170.
`In various embodiments of the disclosure, instructions
`implementing an "application program" may be tangibly
`embodied in a computer-readable medium which may
`include one or more fixed or removable data storage devices,
`such as a zip drive, floppy disc drive, hard drive, CD-ROM
`drive, tape drive, flash memory device, etc. Further, an appli(cid:173)
`cation program may include instructions that, when read and
`executed by a processor, may cause the processor to perform
`the steps necessary to implement and/or use embodiments of
`the disclosure. An application program and/or operating
`instructions may also be tangibly embodied in a memory
`and/or data communications devices, thereby making a com(cid:173)
`puter program product or article of manufacture according to
`an embodiment the disclosure. As such, the term "application
`program" as used herein is intended to encompass a computer
`program accessible from any computer readable device or
`media. Furthermore, portions of an application program may
`be distributed such that some of the application program may
`be included on a computer readable media within a processor(cid:173)
`based device (e.g., device 102 or device 152) and some of the
`application program may be included in a removable data
`storage device.
`FIG. 2 illustrates a block diagram of communication sys(cid:173)
`tem 100. As illustrated, relay service 110 may include a
`display device 132 coupled to a processor-based device 102
`comprising a processor 104 and a memory 106, such as ran(cid:173)
`dom access memory (RAM). Device 102 may also implement
`a compiler (not shown) that allows an application program
`108 written in a programming language to be translated into
`processor 104 readable code. Application program 108 may
`be configured to access and manipulate data stored in
`memory 106 of device 102 using relationships and logic that
`are generated using the compiler.
`Communication device 120 may include a display device
`134 and a processor-based device 152 comprising a processor
`154 and a memory 156, such as random access memory
`(RAM). Device 152 may also implement a compiler (not
`shown) that allows an application program 118 to be trans(cid:173)
`lated into processor 104 readable code. Application program
`118 may be configured to access and manipulate data stored
`in memory 156 of device 152 using relationships and logic
`that are generated using the compiler.
`During a contemplated operation of communication sys(cid:173)
`tem 100, hearing-capable user 160 (see FIG. 1) may convey
`speech into communication device 190, which may then
`transmit an associated voice signal to communication device
`
`Ultratec Exhibit 1003
`Ultratec v Sorenson IP Holdings Page 9 of 12
`
`

`

`US 9,336,689 B2
`
`5
`120 over network 180. Connnunication device 120 may then
`transmit the voice signal to relay service 110. Upon receipt of
`a voice signal, a human call assistant (not shown) positioned
`within relay service 110, may listen to the voice signal trans(cid:173)
`mitted from connnunication device 120 and "revoice" the
`words to a speech recognition computer pro gram (not shown)
`within relay service 110. As will be understood by one having
`ordinary skill in the art, revoicing is an operation in which the
`call assistant repeats the words she or he hears upon receipt of
`the voice signal. The speech recognition program may then
`output a text transcription of the call assistant's spoken words
`and, thereafter, the text transcription may be transmitted to
`communication device 120 via network 170. Hereinafter, a
`text transcription output from a speed recognition program
`may also be connnonly referred to as a "block" of text. For 15
`example, a block of text may comprise one or more words,
`one or more sentences, one or more lines of text, or any
`combination thereof.
`Hearing-impaired user 140 (see FIG. 1) positioned proxi(cid:173)
`mate connnunication device 120 may receive, via network
`180, the voice signal transmitted from communication device
`190. Furthermore, a block of a text output from the speech
`recognition program and associated with the transmitted
`voice signal may be displayed within a text caption on display
`device 134 as it is sent from relay service 110 and received by
`communication device 120. As a result, hearing-impaired
`user 140 may listen to the voice signal transmitted from
`communication device 190 and, innnediately thereafter,
`receive a block of text associated with the voice signal. In
`order to enhance the effectiveness of connnunication system
`100, it is desirable for hearing-impaired user 140 to either
`receive the block of text as close to real-time as possible, or as
`close as possible to simultaneous receipt of the voice signal.
`For example only, hearing-impaired user 140 may listen to the
`voice signal transmitted from connnunication device 190
`and, within three second or less, receive a block of text asso(cid:173)
`ciated with the voice signal.
`Furthermore, it should be noted that a block of text, as
`output from the speech recognition program and as transmit(cid:173)
`ted to and displayed within a text caption on display device
`134, may also be displayed within a text caption displayed on
`display device 132. As a result, the call assistant may view the
`text caption, including one or more blocks of text, as dis(cid:173)
`played on display device 134 and as viewed by hearing(cid:173)
`impaired user 140. Therefore, any errors that may exist in the 45
`text caption displayed on display device 134 may also be
`visible to the connnunication assistant on display device 132.
`In accordance with various embodiments of the disclosure,
`application program 108 may be configured to provide in-line
`correction of any errors that may be detected within a text 50
`caption displayed on display device 134. Stated another way,
`application program 108 may be configured to replace any
`detected errors within a displayed text caption with corrected
`text. More specifically, in the event the call assistant notices
`one or more errors within a block of text of the text caption as 55
`displayed on each of display device 134 and display device
`132, the call assistant may edit the block of text including the
`one or more errors through input into device 102 to correct the
`one or more errors. For example only, the call assistant may
`edit a block of text through input into a keyboard (e.g., select- 60
`ing and replacing text, inserting text, and/or deleting text). As
`a result, a corrected block of text including one or more
`corrected words may be generated. Thereafter, the corrected
`block of text including the one or more corrected words may
`be sent to connnunication device 120. Upon receipt of the 65
`corrected block of text at communication device 120, appli(cid:173)
`cation program 118 may be configured to replace the block