Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 1 of 19 PageID# 70
`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 1 of 19 PagelD# 70
`
`
`
`EXHIBIT 2
`EXHIBIT 2
`
`

`

`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 2 of 19 PageID# 71
`Case 3:24-cv-00540-MHL Document “TMKAtiti
` it
`
`US011069337B2
`
`a2) United States Patent
`Naganuma
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 11,069,337 B2
`Jul. 20, 2021
`
`(54)
`
`VOICE-CONTENT CONTROL DEVICE,
`VOICE-CONTENT CONTROL METHOD,
`AND NON-TRANSITORY STORAGE
`MEDIUM
`
`(71) Applicant: IVC KENWOOD Corporation,
`Yokohama (JP)
`
`(72)
`
`Inventor: Tatsumi Naganuma, Yokohama (JP)
`
`(73) Assignee:
`
`IVC KENWOOD Corporation,
`Yokohama (JP)
`
`(*) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 108 days.
`
`(21) Appl. No.: 16/290,983
`
`(22)
`
`Filed:
`
`Mar. 4, 2019
`
`(65)
`
`Prior Publication Data
`
`US 2019/0279611 Al
`
`Sep. 12, 2019
`
`(30)
`
`Foreign Application Priority Data
`
`Mar. 6. 2018
`
`(JP) vcccsseieeeeeces JP2018-0397 54
`
`(51)
`
`Int. Cl.
`GIOL 13/08
`GIOL 17/26
`
`(2013.01)
`(2013.01)
`(Continued)
`
`(52) U.S. CL
`CPC wos GIOL 13/08 (2013.01), GO6F 3/167
`(2013.01); GIOL 15/08 (2013.01); GIOL 15/16
`(2013.01); GIOL I5/T8& (2013.01); GIAL
`15/1807 (2013.01); GIOL 15/22 (2013.01):
`GIOL 17/26 (2013.01), GOL 13/183
`(2013.01):
`
`(Continued)
`(58) Field of Classification Search
`CPC . GO6GN 3/08: GOGN 20/00; GOON 3/02; G1OL
`
`15/22; G10OL 15/16; G10L 15/1815; GIOL,
`2015/223; G1IOL 2015/227; GIOL 15/24:
`GO6F 16/90332: GO6F 40/205; GO6F
`40/30; GO6F 16/285, GO6F 3/167; GO6F
`16/3344: GOGP 40/268; GO6F 40/284;
`
`(Continued)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,852,804 A
`2O13/0325471 AL*
`
`121998 Sako
`12/2013 Rachevsky wu... GO6N 20/00
`704/244
`
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`
`04-204 700
`
`7/1992
`
`Primary Examiner — Linda Wong
`(74) Attorney, Agent, or Firm — Amin, Turocy & Watson,
`LLP
`
`(57)
`
`ABSTRACT
`
`A voice-content control device includes a voice classifying
`unit configured to analyze a voice spoken by a user and
`acquired by a voice acquirmg unit to classify the voice as
`either one ofa first voice or a second voice, a process
`executing unit configured to analyze the acquired voice to
`execute processing required by the user, and a voice-content
`generating unil configured to generate, based on content of
`the executed processing, output sentencethat is text data for
`a voice to be output to the user, wherein the voice-content
`generating unit is further configured to generateafirst outpul
`sentence as the output sentence whenthe analyzed voice has
`been classified as the first voice, and generale a second
`output sentence in which information is omitted as com-
`pared to thefirst output sentence as the output sentence when
`the analyzed voice has been classified as the second voice,
`
`5 Claims, 6 Drawing Sheets
`
`SD
`INTENTION ANALYZYING
`UAT
`Sa
`ACQUEITION CONTENT
`EQOAMATION AOURUNS|
`PROCESS EXECUTING UNIT
`
`
`
`GENERATING UNIT
`ia
`SECOND CMTPUT BENTENK
`GENERATING UNIT
`VOVCeCONTENT
`GENERATING UNIT
`
`
`
`MOCE-CONTENT CONTROLLER:
`
`
`

`

`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 3 of 19 PageID# 72
`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 3 of 19 PagelD# 72
`
`US 11,069,337 B2
` Page 2
`
`(51)
`
`(2006.01)
`(2006.01)
`(2013.01)
`(2013.01)
`(2006.01)
`(2006.01)
`(2013.01)
`
`Int. Cl
`GIOL 15/22
`GU6F 3/16
`GIOL 25/51
`GIOL 15/18
`GIOL 15/16
`GIOL 15/08
`GIOL [5/183
`(52) U.S. Ch
`CPC Luu GIOL 25/51 (2013.01); GLOL 2015/227
`(2013.01); GIOE 2015/2278 (2013.01)
`(58) Field of Classification Search
`CPC .. GO6I 40/289; GOGE 16/3329; GOGF 16/337;
`GO6F 40/00; GO6P 40/36
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`2016/0379638 AL 12/2016 Basye wo GLOL 15/18
`704/235
`3/2017 Shim wo. GLOL 13/033
`2017/0083281 AL®
`4/2017 Gelfenbeyn ........... GO6F 3/167
`2017/O1L0129 AI*
`6/2017 Divakaran ,......., GO6K 9/0022L
`2017/OL60813 Al“
`JOLD/OLSO900 AL*—S/)2019 Tsai cece GIOL 15/22
`20190139541 AL*
`5/2019 Andersen .....
`coo GOL 15/16
`
`.. GO6F 16/3329
`2019/0164554 Al*
`5/2019 Huang .....
`
`-» GLOL 15/30
`2019/0L80740 Al*
`6/2019 Nandy.
`.....
`wee GOIB 5/00
`8/2019 Chandrasekaran
`2019/0266999 AL*
`
`
`* cited by examiner
`
`

`

`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 4 of 19 PageID# 73
`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 4 of 19 PagelD# 73
`
`U.S. Patent
`
`Jul. 20, 2021
`
`Sheet 1 of 6
`
`US 11,069,337 B2
`
`
`
`

`

`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 5 of 19 PageID# 74
`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 5 of 19 PagelD# 74
`
`U.S. Patent
`
`Jul. 20, 2021
`
`Sheet 2 of 6
`
`US 11,069,337 B2
`
`10
`
`|
`
`VOICE
`DETECTING
`eee
`
`|
`
`|
`
`|
`
`[ :
`VOICE GLASSFYING |
`
`[SECOND OUTPUT SENTENCE
`8
`_ |_GENERATING UNIT
`
`COMMUNICA-
`VOICE-CONTENT
`
`20
`
`40
`
`OUTPUT CONTROLLER
`
`CONTROLLER
`
`FIG.2
`
`30
`
`VOICE ACQUIRING UNIT |
`
`}
`
`50
`
`INTENTION ANALYZYING
`UNIT
`
`52
`| ACQUISITION CONTENT _
`INFORMATION ACQUIRING
`UNE
`PROCESS EXECUTING UNIT
`
`36
`
`60
`
`FIRST OUTPUT SENTENCE
`GENERATING UNIT
`
`62
`
`4
`2
`VOICE OUTPUT
`UNIT
`
`14
`
`LIGHTING UNIT
`
`VOICE-CONTENT CONTROLLER
`
`

`

`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 6 of 19 PageID# 75
`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 6 of 19 PagelD# 75
`
`U.S. Patent
`
`Jul. 20, 2021
`
`Sheet 3 of 6
`
`US 11,069,337 B2
`
`FIG.3
`
`INTENTION
`
`ATTRIBUTE
`PARAMETER EO/
`
`ATTRIBUTE
`PARAMETERE0/
`
`INFORMATION ATTRIBUTECONTENT| ATTRIBUTECONTENT DATE
`
`WEATHER
`
`{
`
`DAY Z OF MONTH Y OF
`YEAR X
`
`LOCATION
`
`TOKYO
`
`FIG.4
`
`INTENTION
`INFORMATION
`I
`
`ACQUISITION
`PARAMETER ADb/
`ACQUISITION
`CONTENT
`INFORMATION At
`
`ACQUISITION
`PARAMETER AQ/
`ACQUISITION
`CONTENT
`INFORMATION A4
`
`ACQUISITION
`PARAMETER A0/
`ACQUISITION
`CONTENT
`INFORMATION A‘
`
`WEATHER HIGHESTAIR
`
`
`
`WEATHER AIR TEMPERATURE|CHANCE OF RAINFALL
`
`PARTLY CLOUDY
`
`WNDEGREES
`LOWESTAIR
`TEMPERATURE: 15
`DEGREES
`
`

`

`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 7 of 19 PageID# 76
`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 7 of 19 PagelD# 76
`
`U.S. Patent
`
`Jul. 20, 2021
`
`Sheet 4 of 6
`
`US 11,069,337 B2
`
`
`
`EXTRACT INTENTION INFORMATION|
`rnoM TEXT DATA
`
`
`
`
`EXECUTE PROCESSING FOR
`INTENTION INFORMATION, OR
`
`
`ACQUIRE ACQUISITION
`
`INFORMATION
`
`$20
`
`
`~
`EIRST VOICE?
`=
`
`:
`GENERATE FIRST OUTPUT
`GENERATE SECOND
`SENTENCE
`OUTPUT SENTENCE=|
`
`524
`
` a
`
`OUTPUT OUTPUT SENTENCE
`
`

`

`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 8 of 19 PageID# 77
`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 8 of 19 PagelD# 77
`
`U.S. Patent
`
`Jul. 20, 2021
`
`Sheet 5 of 6
`
`US 11,069,337 B2
`
`S12
`
`ANALYZE INPUTVOICEAND
`GENERATE TEXT DATA
`
`|
`'
`
`
`
`S14
`
`EXTRACT INTENTION INFORMATION
`FROM TEXT DATA
`
`816
`|
`EXECUTE PROCESSING FOR
`INTENTION INFORMATION, OR
`ACQUIRE ACQUISITION
`
`
`
`GENERATE FIRST OUTPUT
`SENTENCE
`
`:
`
`: $
`
`26
`
`OUTPUT OUTPUT SENTENCE
`
`

`

`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 9 of 19 PageID# 78
`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 9 of 19 PagelD# 78
`
`U.S. Patent
`
`Jul. 20, 2021
`
`Sheet 6 of 6
`
`US 11,069,337 B2
`
`FIG.7
`
`
` VOICE ACQUIRING
`NIT‘
`
`
`10
`VOICE DETECT-
`ING UNIT
`
`
`
`COMMU-
`NICATION}
`UNIT
`
`|
`
`3
`
`VOICE OUTPUT
`UNIT
`
`15
`| COMMUNICA-
`TION UNIT
`“TA
`
`LIGHTING UNIT
`
`| RESPONCE DEVICE
`
`ACQUISITION
`CONTENT
`INFORMATION
`ACQUIRING UNIT
`Sh
`INFORMATION |
`ACQUIING UNIT
`PROCESS ewNS
`
`NIT
`
`|
`
`|
`|
`
`FIRST OUTPUT
`||SENTENCE
`GENERATING UNIT
`
`SENTENCE
`| GENERATINGUNIT|
`VOICE-CONTENT
`GENERATING UNIT
`
`40
`
`
`VOICE-CONTENT CONTROLLER
`
`CONTROLLER
`CONTROLLER
`
`

`

`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 10 of 19 PageID# 79
`Case 3:24-cv-00540-MHL Document1-2 Filed 07/25/24 Page 10 of 19 PagelD# 79
`
`US 11,069,337 B2
`
`1
`VOICE-CONTENT CONTROL DEVICE,
`VOICE-CONTENT CONTROL METHOD,
`AND NON-TRANSITORY STORAGE
`MEDIUM
`
`CROSS-REPFERENCE TO RELATED
`APPLICATION
`
`This application claims priority from Japanese Applica-
`tion No. 2018-039754. filed on Mar. 6, 2018, the contents of
`which are incorporated by reference herein in its entirety.
`
`FIELD
`
`The present application relates to a voice-content control
`device, a voice-content contro] method, and a non-transitory
`storage medium.
`
`BACKGROUND
`
`As disclosed in Japanese Examined Patent Publication
`No. H07-109560, for example, a voice contro! device that
`analyzes detected voice of a user and performs processing
`according to the user’s intention has been proposed. Fur-
`thermore, a voice contro! device, which outputs, via voice,
`that processing intended by a user has been performed, or
`oulputs, via voice-content of a user’s inquiry. has also been
`proposed,
`However, when a voice processing device that outputs
`voice is used, the output voice may be heard by a person who
`1s not the user of the voice processing device and is around
`the voice processing device. For example, ifa person around
`the voice processing device is asleep, the output voice may
`be an annoyanceto the person. [n that case, the output voice
`itself may be decreased in sound volume, butif the output
`voice is decreased in sound volume too much, the output
`voice may be hard to be heard by the user himself and the
`user may be unable to understand the content of the voice.
`Therefore, on outputting the voice to the user, influence of
`the output voice to people other than the user is desired to
`be suppressed, and contentof the outpul voice tothe useris
`desired to be made adequately understandable,
`
`SUMMARY
`
`A voice-content control device, a voice-content control
`method, and a non-transilory storage medium are disclosed.
`According to one aspect, there is provided a voice-content
`control device, comprising: a voice classifying unit config-
`ured to analyze a voice spoken by a user and acquired by a
`voice acquiring unit to classify the voice as either one of a
`first voice or a second voice: a process execuling unit
`configured to analyze the voice acquired by the voice
`acquiring unit to execute processing required by the user:
`and a voice-content generating unit configured to generate,
`based on content of the processing executed by the process
`executing unit, output sentence that is text data for a voice
`to be output to the user, wherein the voice-content generat-
`ing unit
`is
`further configured to generate a first output
`sentence as the output sentence when the acquired voice has
`been classified as the first voice, and generate a second
`output sentence in which information is omitted as com-
`pared to the first output sentence as the output sentence when
`the acquired voice has been classified as the second voice.
`According to one aspect, there is provided a voice-content
`control method, comprising: acquiring a yoice spoken by a
`user; analyzing the acquired voice to classify the acquired
`
`wn
`
`0
`
`20
`
`ry
`
`40
`
`45
`
`ay
`
`55
`
`bu
`
`2
`voice as either one ofa first voice and a second voice;
`analyzing the acquired voice to execute processing intended
`by the user; and generating, based on content ofthe executed
`processing. outpul sentence that is text data for a voice to be
`oulpul to the user, wherein al the generating, a first output
`sentence is generated as the oulput sentence when the
`acquired voice has been classified as the first voice, and a
`second output sentence is generated as the output sentence
`in which a part of information included in the first output
`sentence is omitted when the acquired voice has been
`classified as the second voice.
`
`According to one aspect, there is provided a non-transi-
`tory storage medium thal stores a voice-content control
`program that causes a computer to execute: acquiring a
`voice spoken by a user: analyzing the acquired voice to
`classify the acquired voice as either one ofafirst voice and
`a second voice; analyzing the acquired voice to execute
`processing intended by the user; and generating, based on
`content of the executed processing, output sentence thatis
`text data for a voice to be output to the user, wherein at the
`generating. a first output sentence is generated as the output
`sentence when the acquired voice has been classified as the
`first voice, and a second output sentence is generated as the
`output sentence in which a part of information included in
`the first output sentence is omitted whenthe acquired voice
`has been classified as the second voice,
`The above and other objects, features, advantages and
`technical and industrial significance ofthis application will
`be better understood by reading the following detailed
`description ofpresently preferred embodiments of the appli-
`cation, when considered in connection with the accompa-
`nying drawings,
`
`BRIEF DESCRIPTION OFTHE DRAWINGS
`
`FIG, 1 is a schematic diagram ofa voice-content control
`device according to a first embodiment;
`FIG, 2 is a schematic block diagramof the voice-content
`control device according to the first embodiment;
`FIG, 3 is a table illustrating an example ofattribute
`information:
`FIG. 4 is a table illustrating acquisition information:
`FIG. § is a flow chart
`illustrating a flow of output
`processing for output sentence, according to the first
`embodiment;
`FIG. 6 is a Nowchart illustrating another example ofthe
`flow ofthe output processing for the output sentence;
`and
`FIG. 7 is a schematic block diagramof a voice processing
`system according to a second embodiment.
`
`DETAILED DESCRIPTION OF THI
`PREFERRED EMNODIMENTS
`
`Embodiments of the present application are explained in
`detail below with reference to the drawings. The embodi-
`ments explained below are not intended to limit the present
`application.
`
`First Embodiment
`
`is a sche-
`1
`First, a first embodiment is explained. FIG.
`matic diagramofa vaice-content control device according to
`the first embodiment, As shown in FIG, 1, a voice-content
`control device 1 according to the first embodiment detects a
`voice V1 spoken by a user H by a voice detecting unit 10,
`analyzes the detected voice V1 to perform a predetermined
`
`

`

`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 11 of 19 PageID# 80
`Case 3:24-cv-00540-MHL Document1-2 Filed 07/25/24 Page 11 of 19 PagelD# 80
`
`US 11,069,337 B2
`
`3
`processing, and outputs a voice V2 by a voice output unit 12.
`Although the voice V2 is output toward the user [H, when
`other people are present around the voice-content contro!
`device 1. the voice V2 can be heard by the people. Lf for
`example, a person around the voice-content contro] device 1
`is asleep, the voice V2 may be an annoyance to the person.
`The voice-content control device 1 according to this
`embodiment analyzes the voice V1, and adjusts text to be
`output as the voice V2, thereby suppressing influence ofthe
`voice V2 on people other than the user EH and allowing the
`user H to adequately understand content of the voice V2.
`FIG. 2 is a schematic block diagram ofthe voice-content
`control device accordingto the first embodiment. As shown
`in FIG, 2, the output-content control device 1 includes the
`voice detecting unit 10, the voice output unit 12, a lighting
`unit 14, a controller 16, 9 communication unit 18, and a
`storage 20. The voice-content control device 1 is a so-called
`smart speaker(artificial intelligence (Al) speaker), but is not
`limited thereto as long as the device has functions described
`later. The voeice-content control device 1 can be,
`for
`example. a smart phone, a tablet, and the like.
`The voice detecting unit 10 is a microphone and detects
`the voice V1 spoken by the user H. The user H speaks the
`voice V1 toward the voice detecting unit 10 so as to include
`information lor a processing wished to be performed bythe
`voice-content contro] device 1. The voice detecting unit 10
`can be regarded as an input unit that accepts information
`input externally. The input unit may be provided in addition
`to the voice detecting unit 10, and, for example, a switchto
`adjust volume of the yoice V2 by operation performed by the
`user H, and the like may be provided. The voice output unit
`12 is a speaker. and outputs sentences (output sentences
`described later) generated by the controller 16 as the voice
`V2. The lighting unit 14 isa light source, such as a light
`emitting diade (LED), and is turned on by a control ofthe
`controller 16. The communication unit 18 is a mechanism to
`communicate with external servers, such as a Wi-Fi (regis-
`tered trademark) module and an antenna, and communicates
`information with an external server not shownundercontro!
`ofthe controller 16. ‘The communication unit 18 performs
`communication of information with the external servers by
`wireless communication such as Wi-Fi, but the communi-
`cation of information with the external servers may be
`performed also by wired communication by cables con-
`nected. The storage 20 is a memory thal stores information
`onarithmetic calculation of the controller 16 or programs,
`and includes, for example, at least one of a random access
`memory (RAM), a read-only memory (ROM). and an exter-
`nal storage device, such as a Hash memary.
`The controller 16 is an arithmetic unit, namely, a central
`processor (CPU). The controller 16 includes a voice acquir-
`ing unit 30. a voice analyzing unit 32. a process executing
`unit 34, a voice-conlent generating unit 36, a voice classi-
`fying unit 38, and an output controller 40. The voice
`acquiring unit 30, the voice analyzing unit 32, the process
`executing unit 34, the voice-content generating unit 36, the
`voice classifying unit 38, and the output controller 40
`perform processes described later by reading software/pro-
`gram stored in the storage 20.
`The voice acquiring unit 30 acquires the voice V1 that is
`detected by the voice detecting unit 10. The voice analyzing
`unit 32 performs voice analysis of the voice V1 acquired by
`the voice acquiring unit 30, to convert the voice V1 into text
`data. The text data is character data/text data that includes a
`
`sentence spoken as the voice V1. The voice analyzing unit
`32 detects. for example. voice waveform comprising ampli-
`tude and wave length per time from the voice V1. The voice
`
`wn
`
`2
`
`a
`
`40
`
`45
`
`ia
`
`60)
`
`65
`
`4
`analyzing unit 32 then replaces the voice waveformper time
`with characters based on a table in which a relationship
`between the voice waveforms and the characters is stored,
`thereby converting the voice V1 into the text data. Note thal
`the converting method can be arbitrarily chosen as long as
`it enables to convert the voice V1 into the text data.
`Based on the text data generated by the voice analyzing
`unil 32, the process executing unit 34 detects information on
`content of processing that is included in the voice V1 and
`desired to be executed by the voice-content control device1,
`and executes the processing. The process executing unit 34
`has an intention analyzing unit 50, and an acquisition
`content information acquiring unit 52.
`The intention analyzing unit 50 acquires the text data that
`is generated by the voice analyzing unit 32, extracts inten-
`tion information | based on the text data, and extracts the
`attribute information E based onthe intention informationI.
`The attribute information Eis informationthat is associated
`with the intention information I, and is information that
`indicates a condition necessary for acquiring information
`that
`the user H wishes to acquire. Namely, the attribute
`information E is an entity.
`Firstly, processing for extracting the intent information |
`will be described. The intention information 1, that is, an
`intent, is information that indicates what kind ofprocessing
`is intended by the user H to be performed on the voice-
`content control device 1.
`In other words,
`the intention
`information I
`is information that indicates what kind of
`processing is required by the user H to be performed on the
`voice-content control device 1. The intention analyzing unit
`50 extracts the intention information | from the text data by
`using, for example, a natural language processing, In the
`present embodiment, the intention analyzing unit 50 extracts
`the intention information 1
`from the text data based on
`multiple pieces oftraining data stored in the storage 20. The
`training data herein is data in which the intention informa-
`tion | has been assigned to the text data in advance. Thatis,
`the intention analyzing unit 50 extracts the training data that
`is similar to the text data generated by the voice analyzing
`unit 32, and regards the intention information | of the
`extracted training data as the intention informationI of the
`text data generated by the voice analyzing unit 32. Note that
`the training data is not necessarily required to be stored in
`the storage 20, and the intention analyzing unit 50 can search
`for the training data in the external server by controlling the
`communication unit 18. As long as the intention analyzing
`unit 50 extracts the intention information | from text data,
`the extracting method ofthe intention information | can be
`arbitrarily chosen. For example, the intention analyzing unit
`50 can read a relationship table of keywords and the inten-
`tion information | stored in the storage 20, and can extract
`the intention information | that
`is associated with the key-
`word when the keyword in the relationship table is included
`in the text data,
`For example,if the text data corresponds to text “How’s
`the weather today?”, by performing the above described
`analysis, the intention analyzing unit 50 recognizes that
`processing of notifving the user FI of weather information is
`information on the processing required by the user 11. thatis,
`the intention informationI, Furthermore, if, for example, the
`text data corresponds to text “Turn on the light.”, by
`performing the above described analysis, the intention ana-
`lyzing unit 50 recognizes that processing of turning power of
`the light on is information on the processing required by the
`user EL,
`that
`is. the intention information I. As described
`above, the intention information I 1s classified into informa-
`
`

`

`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 12 of 19 PageID# 81
`Case 3:24-cv-00540-MHL Document1-2 Filed 07/25/24 Page 12 of 19 PagelD# 81
`
`US 11,069,337 B2
`
`5
`
`6
`storage 20 in advance. Accordingly, even if a keyword
`indicating the location 1s not included in the text data, the
`intention analyzing unit 50 is able to set the attribute content
`E1 ofthe location to Tokyo. Furthermore.
`the intention
`analyzing untt 50 may set
`the attribute content 1 by
`communicating with the external server through the com-
`munication unit 18, In this case, for example, the intention
`analyzing unit $0 acquires the current location by commu-
`nication with a global positioning system (GPS). and sets the
`location as the attribute content EL.
`
`The intention analyzing unit 50 extracts the intention
`information | and the attribute information FE as described
`above, but without being limited thereto. Any extraction
`methods for the intention information | and attribute intor-
`mation E may be used. FIG. 3 illustrates a case where the
`weather information is the intention information [, but the
`intention information | and attribute information FE are able
`
`to be extracted similarly in other cases. For example, if
`information indicating that the power ofthe light is to be
`turned on is the intention information I, the attribute mlor-
`mation E includes information on the location of the light,
`and information on the date and time when the poweris to
`be turned on.
`information acquiring unit 52
`The acquisition content
`illustrated in PIG, 2 executes, based on content ofthe
`intention information 1, processing required by the user. If
`the intention information I indicates that a device is to be
`controlled,
`the acquisition content
`information acquiring
`unit 52 executes processing of content of the intention
`information [. For example, the acquisition content infor-
`mation acquiring unit 52 turns the power of the light at the
`location indicated by the attribute information Eon.
`FIG, 4 is a table illustrating acquisition information. I the
`intention information |
`indicates notification of required
`information, the acquisition content information acquiring
`unit 52 acquires the required information, that is, acquisition
`information A. The acquisition information A is information
`that the user His to be notified of, and is, in other words,
`information determined by the process executing unit 34 to
`be information that the user 1 requires to be notified of.
`Based on the intention information| extracted by the inten-
`tion analyzing unit 50, the acquisition content information
`acquiring unit 52 acquires the acquisition information A.
`More specifically,
`the acquisition content
`information
`acquiring unit 52 selects and extracts acquisition parameter
`AQ from the extracted intention information 1. The acquisi-
`tion content
`information acquiring unit 52 reads oul a
`relation table between the intention information I and the
`acquisition parameters AQ stored in the storage 20, and
`detects, from the relation table, the intention information |
`matched with the extracted intention information I, The
`acquisition content
`information acquiring unit 52 then
`extracts the attribute parameter AQ associated with the
`matched intention information I, However, the acquisition
`content
`information acquiring unit 52 may communicate
`with the external server via the communication unit 18, and
`acquire the relation table from the external server.
`After having extracted the acquisition parameters AO, the
`acquisition content information acquiring unit 52 acquires,
`based on the attribute information LE, acquisition content
`information Al for each ofthe acquisition parameters AO,
`Specifically, for each of the acquisition parameters AO, the
`acquisition content information acquiring unit 52 acquires
`the acquisition content information Al corresponding to the
`attribute content E1 set for that attribute parameter E0. By
`communicating with the external server/external device via
`ihe communication unt 18, the acquisition content infor-
`
`5
`
`20)
`
`at
`
`tion for notification of the required information, and infor-
`mation lor contral of devices as being required.
`The extracting method of the intention information | using
`text data can be arbitrarily chosen. not limited thereto. For
`example, the output-content control device 1 can be conlig-
`ured ta store a relationship table of keywords and the
`intention information J in the storage 20, and to detect the
`intention information | associated with the keyword when
`the keyword is included intext data of the voice V1 spoken
`by the user H. As an example of this case, a keyword
`“konnichiwa™ may be associated with weather information
`and news, In this case, when the user H speaks the voice V1
`“konnichiwa”, the intention analyzing unit 50 detects the
`weather information and the news as the intention informa-
`tionI.
`Described next is the attribute information E. FIG. 3 is a
`table illustrating an example of the attribute information.
`The attribute information E, that is, the entity, is a condition
`needed upon executionof the processing, whichis required
`by the user H and is extracted as the intention information
`I, that is, a parameter. For example, if the intention infor-
`mation [ is weather information, the attribute information E
`includes information on a location indicating where the
`weather mformation is on, and information on a date indi-
`cating when the weather information is for. Furthermore, as
`illustrated in FIG. 3, the attribute information FE includes an
`attribute parameter [0 and attribute content FL. Theattribute
`parameter EO is information indicating the type of param-
`eter, that is, the kind of condition. and the attribute content
`E1 indicates content of the attribute parameter 0. Thatis,
`if the attribute information E is information on a location,
`the attribute parameter E0 is informationindicating that the
`condition is the location, and the attribute content EJ is
`information indicating that the location is Tokyo. Moreover,
`if the attribute information E is information on a date. the
`attribute parameter EO is information indicating that the
`condition is the date. and the attribute content El is infor-
`mation indicating that the date is the day Z of the month Y
`ol the year X.
`the intention analyzing
`According to this embodiment,
`unit 50 extracts the attribute information FE, based on the
`extracted intention information I. More specifically,
`the
`intention analyzing unit 50 selects and extracts the attribute
`parameter E0 tromthe extracted intention information I, The
`intention analyzing unit $0 reads out arelation table between
`the intention information | and the attribute parameters E0
`stored in the storage 20, and detects, fromthe relation table,
`the intention information J matched with the extracted
`intention information I. The intention analyzing unit 50 then
`7
`extracts the attribute parameter EO associated with the 50
`matched intention information I. However,
`the intention
`analyzing unit 50 may communicate with an external server
`via the communication unit 18, and acquirethe relation table
`from the external server.
`Afier the intention analyzing unit 50 has extracted attri-
`bute parameters EQ),the intention analyzing unit 50 sets the
`attribute content E1 for each ofthe attribute parameters EO.
`The intention analyzing unit 50 extracts the attribute content
`F1 from, for example, the text data generated by the voice
`analyzing unit 32. That is. if a keyword “today” is included
`in the text data, the intention analyzing unit 50 sets the
`attribute content £1 ofthe attribute parameter EO, the date,
`to today. Furthermore, the intention analyzing unit 50 may
`set the attribute content EI for the attribute parameter E0 in
`advance. For example. if the intention information |
`is
`weather information. set data indicating that the attribute
`content E1 of the location ts ‘Tokyo may be stored in the
`
`4
`
`45
`
`60)
`
`65
`
`

`

`Case 3:24-cv-00540-MHL Document 1-2 Filed 07/25/24 Page 13 of 19 PageID# 82
`Case 3:24-cv-00540-MHL Document1-2 Filed 07/25/24 Page 13 of 19 PagelD# 82
`
`US 11,069,337 B2
`
`wn
`
`10
`
`20)
`
`at
`
`7
`mation acquiring unit 52 acquires the acquisition content
`information AI
`from the external server for each of the
`acquisition parameters AQ. However, if the acquisition con-
`tent information AI has been stored in the storage 20, the
`acquisition content
`information acquiring unit 52 may
`acquire the acquisition content
`information Al
`from the
`storage 20. That is, the acquisition content information A1
`can be said to be data that the acquisition content informa-
`lion acquiring unit 52 acquires from a database of the
`external server, the storage 20, or the like.
`As described above, the acquisition content information
`AJ is information that the acquisition content information
`acquiring unit 52 has acquired by the communication with
`the external server, or read-out from the storage 20. In the
`example of FIG. 4, the intention information I is weather.
`and the acquisition parameters AQ are weather, air tempera-
`ture, and chance of rainfall.
`In this case, the wequisition
`content information acquiring unit 52 acquires the acquisi-
`tion content information Al for the respective acquisition
`parameters AO, that is, weather, air temperature, and chance
`ofrainfall, in Tokyo, onthe day Z ofthe month Y ofthe year
`X. In the example of FIG, 4, the acquisition content infor-
`mation Al
`for weather is “partly cloudy”, the acquisition
`content information Al for air temperature is “highest air
`lemperature:
`25 degrees:
`lowest air
`temperature:
`15
`degrees”, and the acquisition content information Al
`for
`chance of rainfall is “20%”.
`As described above, the acquisition content information
`acquiring unit 52 acquires, based on the attribute informa-
`tion E, the acquisition content information Al for each of the
`acquisition parameter AO. According to this embodiment,
`multiple acquisition parameters AQ are associated with the
`intention information I. However, one acquisition parameter
`AO may be associated with the intention information I. In
`this case, the intention information | itself can be said to be :
`the acquisition parameter AO.
`Referring back to FIG, 2, the voice-content generating
`unit 36 generates the output sentence based on the acquisi-
`tion content
`information Al acquired by the acquisition
`content information acquiring unit 52. The output sentence
`is data of the sentence for the voice V2 to be output by the
`voice output unit 12, that is, text data. It can be said that the
`output sentence is dialog data. The voice-content generating
`unit 36 includes a first output sentence generating unit 60
`that generates a first output sentence as the output sentence,
`and a second output sentence generating unit 62 that gen-
`erates a second output sentence as the output sentence, The
`voice classifying unit 38 is explained before explaining of
`the first output sentence generating unit 60 and the second
`output sentence generating unit 62.
`The voice classifying unit 38 analyzes the voice V1