US007233589B2
`
`(12) United States Patent
`Tanigawa et a].
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 7,233,589 B2
`Jun. 19, 2007
`
`(54) COMMUNICATION SYSTEM AND
`COMMUNICATION METHOD
`
`(75) Inventors: Keiko TanigaWa, Kawasaki (JP);
`Kazuma Yumoto, Hachiouji (JP);
`Kenta Shiga, Yokohama (JP); Toru
`Hoshi, Yokohama (JP)
`
`(73)
`
`Assignee: Hitachi, Ltd., Tokyo (JP)
`
`(*)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1170 days.
`
`(21)
`
`(22)
`
`(65)
`
`Appl. No.: 10/231,262
`
`Filed:
`
`Aug. 30, 2002
`
`Prior Publication Data
`
`OTHER PUBLICATIONS
`
`Nikkei Communication, Nov. 5, 2001, pp. 106-113.
`R. Perera, “Instant Messaging Gets Down to Business”, PC
`WORLD.COM, (http://www.pcworld.com) , Nov. 15, 2001.
`Mahy et al, “A Multi-Party Application Framework for SIP”,
`SIPPING Working Group Internet Draft, Feb. 2002.
`“Instant Messaging”, PC WORLD.COM, (http://wwwpcworld.
`com), May 22, 2000.
`M. Goldberg, “Generation IM”, Yahoo ! Internet Life, Apr. 2002,
`pp. l-6.
`D. Willmott, “Which IM Program Is for You?”, Yahoo ! Internet
`Life, Apr. 2002.
`J. Caplan, “amazing IM Tricks ! ”, Yahoo ! Internet Life, Apr. 2002.
`J. Caplan, “Amazing IM Tricks ! ”, Yahoo ! Internet Life, Apr. 2002.
`* cited by examiner
`
`Primary ExamineriPhirin Sam
`(74) Attorney, Agent, or F irmiMattingly, Stanger, Malur &
`Brundidge, PC.
`
`US 2004/0001480 A1
`
`Jan. 1, 2004
`
`(57)
`
`ABSTRACT
`
`Foreign Application Priority Data
`(30)
`Jun. 4, 2002
`
`(JP)
`
`........................... .. 2002-162940
`
`(51) Int. Cl.
`(2006.01)
`H04L 12/66
`(52) US. Cl. ..................................... .. 370/352; 370/493
`(58) Field of Classi?cation Search ................... .. None
`See application ?le for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6/2002
`2002/0073142 A1 *
`l/2003
`2003/0021290 A1 *
`2003/0126213 Al* 7/2003
`2003/0149774 Al* 8/2003 McConnell et al. ...... .. 709/227
`2003/0210770 Al* ll/2003 Krejcarek .............. .. 379/8817
`
`Moran ...................... .. 709/203
`
`The usage of communication systems is improved. An IM
`server 4 manages presence information regarding each of IM
`clients and usable media and user information such that each
`of the IM clients can obtain these kinds of information. In
`order to perform text chatting, the IM server 4 manages a
`connection between each of the IM clients participating the
`chat and the IM server 4, merges text from each of the
`participating IM clients and distributes the result to each of
`the participating IM clients. In order to perform voice
`chatting, an AP server 5 manages a connection between each
`of the IM clients participating the chat and an MD server 6,
`mixes voice from each of the participating IM clients except
`for a focused IM client and distributes the result to the
`focused participating IM clients. This processing is per
`formed for each of the participating IM clients.
`
`12 Claims, 19 Drawing Sheets
`
`APSERVER
`
`5\I\.=
`
`IMSERVER
`
`3
`
`1
`
`I
`
`PUBLICLY
`SWITCHED
`TELEPHONE
`NETWORK
`
`1 1
`
`1:1
`Um FIXED
`TELEPHONE
`
`2
`
`DIO
`[:3 COMMUNI
`T N
`
`1 O
`\I\
`
`9
`
`VR SERVER
`
`IP NETWORK
`
`RADIO
`TERMINAL
`
`7-1
`[P Q TERMlNAL
`
`7-2
`f TERM'NM-
`
`TERMINAL
`
`/
`
`GOOGLE EXHIBIT 1014
`
`Page 1 of 35
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 1 of 19
`
`US 7,233,589 B2
`
`
`
`WNINYALTWININUALaldl
`
`UYSAYASBA
`
`SYOMLINdi
`
`ANOHdSaTSL
`
`daxld
`eulnEteLIL)
`
`LL
`
`Cc
`
`Q3HOLMS
`
`ATONENd
`
`JNOHd3S74L
`
`©HANNAdiv
`
`Sls
`
`SOMMAAUASWI
`
`Page 2 of 35
`
`Page 2 of 35
`
`
`
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 19, 2007
`Jun. 19, 2007
`
`Sheet 2 0f 19
`Sheet 2 of 19
`
`US 7,233,589 B2
`233,589 B2
`US 7
`2
`
`bMYOMLAN
`
`dl
`
`di MYOMLAN di
`
`
`
`sna
`
`dv
`
`yMAAUSS
`
`AndNt
`“WNY3XS
`
`JDVYOLS
`
`
`
`SetsNddAYOWNAN
`
`LaHOVd
`
`ONMEINASSVSa0
`
`Layovd
`
`ONIZAWNYV
`
`NGONY
`
`ONVNWOS
`
`NOILYOd
`ONTLNGIMLSIC
`
`Sv
`
`NOILLYO>
`
`Sv
`
`INdino
`
`SOAs
`
`SOAS
`
`VL¥G
`
`ONISSSIOUdiNdLno
`
`NOWWYOd
`
`ov
`
`yLvd
`
`ONLLYAYO
`ONTAWASSV
`
`NOLLYOd
`
`
`
`
`
`NOLLYOdLNANSOVNVINNOLLOANNOO
`
`6Old
`
`Page 3 of 35
`
`Page 3 of 35
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 3 of 19
`
`US 7,233,589 B2
`
`Ovry
`
`SSWVNOOIN
`Sassayddv
`
`JONAMSANOD
`
`&ols
`
`1x3}
`
`
`
`a
`
`SAD
`
`
`
`“oadO'a'v
` “oyJA'COY
`ISNAGGAE|soNayaINOD
`6erBED
`se
`
`
`
`$0TISVLLNSWEOVNVWNOILVWHOSNISONASIYd
`LevSEYSEVpEYEeyv
`
`a1evsnrasa4NSHINGIN3IT9
`
`SHIREFOBOtoh“Ue:
`
`
`soicick(trieielod910
`SASSauddy
`ceEVLev
`LINAND|LNNODOV
`
`
`
`Page 4 of 35
`
`Page 4 of 35
`
`
`
`
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 19, 2007
`Jun. 19, 2007
`
`Sheet 4 0f 19
`Sheet 4 of 19
`
`US 7,233,589 B2
`US 7,233,589 B2
`
`LXYCMIAN
`
`di
`
`| MYOMLAN dl
`
`8S
`
`NOUMOd
`
`4ov
`
`vols
`—>|onnevtassvaa[>|°NBSoNONILVANS|p}ONIIEMESSY
`
`indinoNdodAYOWSIN
`
`
`Laio%dONIZAWNYindinoLAW
`
`NOILYOdNOILYOdNOULWOd
`
`
`
`JOU
`
`AOIAIG
`
`LY
`
`GMANSdV
`
`srg
`
`
`
`
`
`NOLLUOdONISVNVWNOLLOSNNOD
`
`Page 5 of 35
`
`Page 5 of 35
`
`
`
`

`

`U.S. Patent
`
`Jun. 19
`
`, 2007
`
`Sheet 5 of 19
`
`US 7,233,589 B2
`
`0v
`
`S
`
`9&S
`
`FINAYAANOD
`
`SSNVNXOIN
`
`
`
`QINIGOQONA
`
`sesveseescesLes
`tekickyolkcb*peed.Aal
`
`
`SaSS3yd0VSaAYN
`
`ANSITDLINNODOV
`
`
`
`
`
`98STEV1LNSNSOVNYWNOLLOSNNOD
`
`Page 6 of 35
`
`Page 6 of 35
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 6 of 19
`
`US 7,233,589 B2
`
`9Dis
`
`BL
`
`9
`
`NOILYOd
`ONTEWASsvaa
`LaWdVd
`
`Sv
`
`LNdINO
`
`S030
`
`391AgdSDIAS0
`
`ONIZATWNYNOILYOd
`{AdN!|)|AWNWADG
`ONITENSASSY
`
`NOLLYOdaNvunios
`ZTWNIWYALdl
`S5VYOQLSNdd
`
`LavOVd3DIOA
`ONMEWSSSy3d
`ONISSSOOUd
`ONISSSOONd
`
`LANIVdJOIGA
`ONNANSSSY
`
`|YYOMLAN
`
`dl
`
`J] AYOMLAN di
`
`Lv
`
`vy NOILYOd
`
`JIIOACNV
`
`Jay0Vd
`
`NOWLYOd
`
`ONITVSYO
`
`NOILYOd
`
`CNVWWOD
`
`NOLLYOd
`
`vivd
`
`ONILVAYO
`
`NOILYOd
`
`ONIZAWWNV
`
`NOILYOd
`
`LN3A4
`
`NOILYOd
`
`any
`
`ONLLYSANODWd
`
`|NOWYOd
`
`sng
`
`Page 7 of 35
`
`Page 7 of 35
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 7 0f 19
`
`US 7,233,589 B2
`
`FIGT
`
`631
`
`632
`
`(provision name=“Taro " >
`633
`(user account-1' " client B "
`password= " z9vBx7 '' 5 634
`addr= " m,***_*** " />
`(imsrv addr'; " =l==l==l=?**.*** " />
`<voipsrv addr= “ =I=**.***.*=I==I=* " />
`(/provision)
`
`PROFILE DATA 886
`
`Page 8 of 35
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 8 0f 19
`
`US 7,233,589 B2
`
`QUE
`
`O
`3:1 NOIlHOd ONI'IBWiSSV lEMOVd
`
`V gm
`
`Page 9 of 35
`
`

`

`crvP
`
`~0}MAMASYA"*~|39zLY<=zsnao==5oOLMHOMLANa=d|227h=a
`
`or7_NdLNoNdiWNYSLXSov
`
`
`
`3OA30JDIAB0IOVYOLSndoANOINAWS\N,
`B6
`
`NOLNOd.ONTENSSSvaG[PyONTANESSvaa wadooagd|
`
`LaxOVdAIIOA
`
`
`ONIGNASSYWSdO0N4
`
`FOIA
`
`NOLLVOINNINIOD
`
`éNHOMLAN
`
`Oldva
`
`SNOHdaTSL
`
`€NHOMLAN
`
`CAHOLIMS
`
`ATONGNd
`
`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 9 of 19
`
`US 7,233,589 B2
`
`NOLLNOd
`
`NOLLYOd
`
`Page 10 of 35
`
`Page 10 of 35
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 10 of 19
`
`US 7,233,589 B2
`
`Velols
`
`
`
`NOISSINKYLSANOZY
`
`6WNIWYSLCLdio,b-ZIWNIWALdigwerOLHSAYSS
`a2101sTid
`NOISSINGYLS3ND3e—____|
`
`
`HSofODIs
` SONSYS3NOO135|iyO1OLS
`
`6O00RSSNOLLIGNCO
`daZIMOHLNYNFOOT—£00is
`
`
`GYSANSS
`
`dv
`
`CLWNIWMALclPuayuasyayo
`
`aie}
`
`Page 11 of 35
`
`cools
`
`NIS01
`
`df3ON3SS38d
`
`
`
`NOLVOISILON8001S21S43NOAY
`
`SNILLSS
`
`Page 11 of 35
`
`
`
`
`
`

`

`Boies“WNLal3uereWNLdoyBANS9MAAYESsuanuas—yUAAMASeesdi
`
`
`
`©wey=wayjaonoue}
`{wasn>Bp—
`
`Sin2005)§1<“JOOud‘‘XINLZ0LS9z701S
`=One|—pLOIS—.,Ooowe.Oli(J)fav
`oyeueywr1}DI
`a0,-=
`
`
`
`ismAganaS1OLS
`sloisismAggng
`SDIOA|8Nosy
`LVHD
`
`
`
`|SZOLS
`
`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 11 of 19
`
`US 7,233,589 B2
`
`QNOdS3u
`
`voZcoLS
`
`Page 12 of 35
`
`Page 12 of 35
`
`
`
`

`

`epnpuly
`
`
`
`{Buwoosiauqjagynogepay}z}aneyamSUNOSOU)MO}
`
`>WWaD»DWaID>WWeaD
`
`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 12 of 19
`
`US 7,233,589 B2
`
`
`
`(——)eul]9-gqwel}y
`
`(NEDweND
`
`<uroIpys>
`
`otOlas
`
`Page 13 of 35
`
`
`
`
`
`
`
`“NOAyp230)p3au|BulaWOSaneU|‘HSH
`
`YELPSiu)UlOgIJe}OYDEYSI
`
`
`
`EGuyjJeyoS910A04YOWSaMpINcd
`
`
`
`
`
`BEL‘apooSunosoy]pussaseeiy
`
`Ze
`
`10
`
`“2UE
`
`>HI
`
`Page 13 of 35
`
`

`

`Jun. 19, 2007
`
`Sheet 13 of 19
`
`US 7,233,589 B2
`
`
`
`INLLLVHSDSOIOA
`
`Ss
`=rONES
`
`ONIXINISSN
`LOANNOOSIG|SHOAaSZOLS
`
`Ls
`
`
`
`
`
`L1OtSTVHDSDIOA
`
`ISanO3y
`
`U.S. Patent
`
`yusyowep
`
`oeuey
`
`elSls
`
`etetLPANOTONva,EITWNINYSLdi([3i"ZZWNIWHALdi
`
`odveTWNIWYSLdl3quayoqwaysobeasharsMANwSSpeasVvway
`
`
`
`ou}
`
`Page 14 of 35
`
`Page 14 of 35
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 14 of 19
`
`US 7,233,589 B2
`
`ca
`
`WDOud
`
`INISS
`
`NIILaos
`
`ual.ae
`
`“esviSis
`osialssans
`
`XINZOLS9701S
`Loisisnsane
`
`AAULON——eamSSae
`
`iemse||T¥HOJOIOA
`
`oMeUeY
`
`
`
`
`-TAL:OTOITWNINESLa,aquaybeWBNSOUARES SUaMNSSfFHaANSSVywore
`
`pwswe,——>~-——awe}
`—arlOis—
`
`Page 15 of 35
`
`Page 15 of 35
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 15 of 19
`
`US 7,233,589 B2
`
`oyeury
`
`SiSld
`
`ZFussqqwaryo
`
`LO0éS
`
`
`Sanday
`
`SNILLSSONIXIN
`
`
`
`CWIeattthegameotrefissONESsRINESYUokTWNol
`QNILIESaALAIGNCDai]—faces=a
`eyes=weywa
`1002S5Cl8welClLL(LJLf=TU
`
`CAZNOHLOW
`NESOIe00¢s
`
`OWILLVHOADIOA
`[antSONSSSA|
`TaayNaSaud
`1San03y
`
`MILON
`
`ishAgand9002S
`
`
`
`
`
`soozsAganaissnoay
`
`Page 16 of 35
`
`Page 16 of 35
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 16 of 19
`
`US 7,233,589 B2
`
`igs49}'DSis
`foyeuery
`
`VEZ0eS
`
`6WWNINYSLG2dion=b-LTYNINLddeiTWNINMALdi
`LSSNO3YEol8eeoOLL(LJlL(Le_IV 9102é8
`
`
`pwsua)“——~-——7ony
`
`
`
`O1VH«WNW!=,qquero qquaya,OFMEDSSuesSUAANSSPENSywarp
`
` 8réc0cs
`
`NOILOSNNOOSIGNt
`SOMKOLO
`
`AGQN8
`
`iSi]
`
`9202S
`
`Page 17 of 35
`
`Page 17 of 35
`
`
`
`

`

`a.hoadloytansSuaRES5UAANAS=VUSMUASBh.aVNINAGAL NIMDOna>zo0es|NDOT|901Clae,OO1.(LJLL.-lV6TWNIWUAL
`
`
`
`
`
`5)wayaofqualyoOle}
`aNods=4|ig"sorspss
`
`Z}Dis
`(HONDO:ADIN
`LOOES_ishLawToare
`aL»|4SMa
`
`9NILLAS(HONG
`ONIXIN1831p
`—|aSONAS=Ud|SeOSZMOHKLNY
`
`ICNOdS34]
`
`Vé00es
`
`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 17 of 19
`
`US 7,233,589 B2
`
`oyeuey
`
`S
`
`ONILLYHO
`
`SNO)
`
`FQIOA
`
`Page 18 of 35
`
`Page 18 of 35
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 18 of 19
`
`US 7,233,589 B2
`
`
`
`oyeuey
`
`aumSf}ons
`|~~XI
`31STdN09
`
`ONILLSSONIXIN
`
`
`
`2N
`
`ILiSS
`
`0zoesS
`
`%B
`
`TIVO
`
`
`
`WROILIGGYLS4n038
`
`éLaes
`
`
`
`AQGndLs3and3y
`
`‘10SONAS3Ud
`
`STNELgio,-LTWNINUALdl2:WNINGALdlOLuanuasO8F3S
`ooyd,Niue)SeeaeUAandVHlVary
`fo2.0.022DBEV
`
`
`
`syERASPYaAUaS
`
`Page 19 of 35
`
`Page 19 of 35
`
`
`
`
`
`
`

`

`2§oeILTWNINUALdlju(J|.ifZL[AVWHALdl6TYNIRELdenhs“LTWNIdOlavu
`Hwa4waysoe}
`
`
`
` ‘IWNINYALdiJqoljoqweyoObUaNUASSuaNuaS$YEIAYESyCEAUAS\yquarya
`
`—_—I
`
`
`QNISSSION—‘mmr“ua
`
`f00rSLitaOL
`
`
`
`QNISSIOON|,
`
`issn0sy
`
`Z00rS
`
`y00rvS
`
`LitaOL|
`
`Loors
`
`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 19 of 19
`
`US 7,233,589 B2
`
`1ysoA
`
`cyeuey
`
`6bSls
`
`Page 20 of 35
`
`Page 20 of 35
`
`
`
`
`
`

`

`US 7,233,589 B2
`
`1
`COMMUNICATION SYSTEM AND
`COMMUNICATION METHOD
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates to a communication tech-
`nology such as Instant Messaging (IM).
`A communication technology called IM is gathering
`attentions. Sometimes, a matter to talk with the other party
`may not need a rapid response through a telephone. How-
`ever, if an e-mail containing the matter is sent through an
`E-mail, it is not clear when the other party reads (that is,
`whenhis/her reply is obtained). IM hasa time characteristic
`that is intermediate between the telephone andthe e-mail. In
`IM,a current status (which may be called presence infor-
`mation), such as online or offline or the ability to respond,
`of each party is published. Thus, in response to presence
`information of friends using the same IM system over the
`Internet, a tool to be used for communication with friends,
`such as a chat and an e-mail, can be selected.
`Presently, IM systems are released from many venders,
`and IM clients, each of which is application software to use
`a respective IM system,are distributed. However,there is no
`compatibility among IM systemsofthe respective venders.
`As a result, communication cannot be performed between
`the IM clients of the different venders. Thus, the standard-
`ization is started from a need for the interconnectivity. In
`Instant Messaging & Presence Protocol (IMPP) working
`group of Internet Engineering Task Force (IETF), the archi-
`tecture, the message integration and the security (such as
`authentication and encryption) relating to IM are being
`discussed. A model is defined for Require for Comment
`(RFC) 2778 while a protocol requirements are defined for
`RFC 2779.
`
`Conventionally, the multi-party connection (group chat)
`in IM is mainly used just for fun(hobby use), such as
`gossiping. However,recently, study for using the group chat
`for business has been started. Here,
`it is noted that the
`text-based group chat may be not enough for the business
`uses in many cases. For example, inputting Japanese lan-
`guage takes time. In addition, in order to express a compli-
`cated matter or a shade of meaning, which is hard to be
`understood from a document, the voice support is needed. In
`the one-to-one (Peer-to-Peer) technology, the expansion into
`communication using multimedia has been started. Thefirst
`reference (Nikkei Communication, Nov. 5, 2001, pgs. 106 to
`113) proposes such usage of multimedia that a meeting is
`held by using a voice chat or by exchangingfiles.
`
`SUMMARY OF THE INVENTION
`
`When a communication technology is applied to the
`business purpose, it is desirable that not only text but also
`voice can be handled easily in addition to participation in a
`conference without leaving his/her desk and/or participation
`in conference even from the outside ofhis office.
`
`2
`cases, users must call to a predetermined virtual conference
`room. Thatis, the conventional television conference system
`lacks flexibility.
`The present invention was made in view of these circum-
`stances. It is an object of the present invention to improve
`handling (dealing) of a communication system. More spe-
`cifically, it is an object of the present invention to achieve
`group chat using multimedia. For example, the switching
`between a group chat through electronic documents and a
`group chat through voice and/or the switching between a
`one-to-one chat through voice and a group chat through
`voice can be handled flexibly.
`the present invention
`In order to achieve the objects,
`implements a chat between/among IM clients by using an
`IM presence management server which manages presence
`information indicating a state of each of the IM clients; a
`VoIP communication connection management server which
`manages a connection for voice chatting using VoIP (Voice
`over Internet Protocol); and a media server which imple-
`ments multi-party voice communication by mixing voice
`data.
`
`For example, the IM presence management server man-
`ages, as to each of the IM clients, presence information of
`the IM client, information regarding media (including text
`and voice) which can be used for a chat by the IM client, and
`user information of the IM client, and in accordance with an
`instruction from an IM client, provides the IM client in
`question with the presence information, usable media infor-
`mation and user information of each of the IM clients set as
`buddies of the IM client in question.
`In accordance with information of each of IM clients
`
`(called as text participating clients) participating in a text
`chat, which is notified from the IM client, the IM presence
`management server manages a connection between/among
`the text participating clients and the IM presence manage-
`ment server, merges text data sent from each of the text
`participating clients and distributes the result to each of the
`text participating clients.
`The VoIP communication connection managementserver
`manages a connection between/among each of IM clients
`(called as voice participating clients) and the media serverin
`accordance with information of the voice participating cli-
`ents participating in a voice chat, which is notified from the
`IM client or the IM presence managementserver.
`The media server performs processing for mixing voice
`data sent from each of the voice participating clients except
`for one of the voice participating clients set as a focused
`client and for distributing the result data thus mixed to the
`focused client. These processing is performed as focused
`clients on all of the voice participating clients.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`FIG. 1 is a schematic diagram of an IM-VoIP (Voice over
`Internet Protocol) interconnecting system, which is a com-
`However, the conventional IM systemsare not ready for
`munication system to which one embodimentofthe present
`a group chat using multimedia. By using the system dis-
`invention is applied;
`closed in the reference 1, switching between text and voice
`FIG. 2 is a schematic configuration diagram of an IM
`can be performed in a one-to-one chat. However, no con-
`server 4 shown in FIG. 1;
`sideration is given to the switching between text and voice
`FIG. 3 is a diagram showing an example of a presence
`in a group chat.
`information management table 488 used by a connection
`There is a television conference system as a technology
`managing portion 487 shown in FIG, 2 for managing pres-
`for implementing a group chat other than the IM system.
`ence information;
`However,
`the conventional
`television conference system
`
`requires the buddies to gather in a room suchasatelevision FIG. 4 is a schematic configuration diagram of an AP
`conference room at a scheduled conference time. In some
`server 5 shown in FIG. 1;
`
`60
`
`65
`
`Page 21 of 35
`
`Page 21 of 35
`
`

`

`3
`FIG. 5 is a diagram showing an example of a VoIP
`connection management table 586 used by a connection
`managing portion 585 shownin FIG. 4 for managing voice
`chat;
`FIG. 6 is a schematic configuration diagram of an IP
`terminal 7 shownin FIG.1;
`FIG. 7 is a diagram showing an example ofprofile data
`686 required when an IM client participates in an IM;
`FIG. 8 is a schematic configuration diagram of an MD
`server 6 shown in FIG. 1;
`FIG. 9 is a schematic configuration diagram of a VR
`server 10 shown in FIG. 1;
`FIG. 10 is a diagram for explaining an operation for
`switching from a text chat to a voice chat, whichis a first
`operational example of an IM-VoIP interconnecting system
`according to one embodiment of the present invention:
`FIG. 11 is a diagram for explaining an operation for
`switching from a text chat to a voice chat, whichis a first
`operational example of an IM-VoIP interconnecting system
`according to one embodiment of the present invention;
`FIG. 12 is a diagram showing an example of a user
`interface of an IM client, which is displayed in an IP
`terminal 7;
`FIG. 13 is a diagram for explaining a variation example
`of the first operational example shownin FIG. 11;
`FIG. 14 is a diagram for explaining a variation example
`of the first operational example shownin FIG. 11;
`FIG. 15 is a diagram for explaining an operation for using
`a text chat during a voice chat, which is a second operational
`example of an IM-VoIP interconnecting system according to
`one embodiment of the present invention;
`FIG.16 is a diagram for explaining an operation for using
`a text chat during a voice chat, which is a second operational
`example of an IM-VoIP interconnecting system according to
`one embodiment of the present invention.
`FIG. 17 is a diagram for explaining an operation for
`switching from a one-to-one voice chat to a multi-party
`voice chat, which is a third operational example of an
`IM-VoIP interconnecting system according to one embodi-
`ment of the present invention;
`FIG. 18 is a diagram for explaining an operation for
`switching from a one-to-one voice chat to a multi-party
`voice chat, which is a third operational example of an
`IM-VoIP interconnecting system according to one embodi-
`ment of the present invention; and
`FIG. 19 is a diagram for explaining an operation for
`closing a conference room, which is a fourth operational
`example of an IM-VoIP interconnecting system according to
`one embodiment of the present invention.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`Embodiments of the present invention will be described
`below.
`
`55
`
`FIG. 1 is a schematic diagram of an IM-Voice over
`Internet Protocol (VoIP) interconnecting system, which is a
`communication system to which one embodiment of the
`present invention is applied.
`As shown in FIG. 1, an IP network 1 is connected to an
`IM presence managing Server (called as IM server herein-
`after) 4 which manages presence information of an IM
`client, a VoIP communication connection managing server
`(called as AP server hereinafter) 5 which manages connec-
`tion for a voice chat using VoIP, a media server (called as
`MDserver hereinafter) 6 which implements multi-party
`voice speech by mixing voice data, a plurality of IP termi-
`Page 22 of 35
`
`60
`
`65
`
`US 7,233,589 B2
`
`4
`nals 7-1 to 7-3 in each of which an IM clientis installed, and
`a voice relay server (called as VR server hereinafter) 10
`which performs voice relay among a radio communication
`network 2, a publicly switched telephone network (PSTN)3
`and the IP network 1. Here, a radio terminal 9, such as a
`mobile telephone,
`in which an IM client
`is installed,
`is
`connected to the radio communication network 2. A fixed
`
`5
`
`telephone 11 in which an IM clientis installed is connected
`to the publicly switched telephone network 3.
`In the above-described configuration, the IP terminals 7-1
`to 7-3 can perform a multi-media chat using the IM server
`4, the AP server 5 and the MD server 6 by independently or
`combination with the VoIP telephone8, the radio terminal 9
`or the fixed telephone 11. For example, in a group or in
`one-to-one, a chat through electronic documents such as text
`(called as text chat hereinafter) and a chat through voice
`(called as voice chat hereinafter) can be performed.
`Next, each of devices included in the IM-VoIP intercon-
`necting system will be described.
`First of all, the IM server 4 will be described.
`As described above, the IM server 4 manages presence
`information of the IM clients. A schematic configuration of
`the IM server 4 is shownin FIG.2.
`
`As shown in FIG. 2, the IM server 4 is achieved by
`configuring functional blocks 481 to 487 shown within a
`balloon 48 in FIG. 2 when a CPU 41 executes a predeter-
`mined program loaded onto a memory 42 in a general-
`purpose computer system including the CPU 41,
`the
`memory 42, an external storage device 42, such as an HDD,
`an input device 44, such as a keyboard, a mouse and a pen,
`an output device 45, such as a speaker and a display, an IP
`network interface (IF) 46 which performs communication
`over the IP network 1, and a bus 47 for connecting these
`devices 41 to 46. The program for configuring the functional
`blocks 481 to 487 maybe directly loaded from a storage
`medium (not shown) such as a CD-ROM,through a reading
`device (not shown), or from the IP network 1 to the memory
`42 through the IP network IF 46. Alternatively, the program
`may be stored in the external memory device 43 and then
`may be loaded to the memory 42.
`A packet deassembling portion 481 receives an IP packet
`addressed to itself from the IP network 1. Then, the packet
`deassembling portion 481 processes the IP header and the
`TCP/UDPheader of the packet and extracts a payload.
`An analyzing and data/commanddistributing portion 482
`analyzes the content of the payload processed by the packet
`deassembling portion 481 and distributes the content to a
`data processing portion 483 as data or a commandprocess-
`ing portion 484 as a command.
`The data processing portion 483 performs processing
`required for IM on data distributed by the analyzing and
`data/command distributing portion 482. For example, for
`each conference, a predetermined amountof latest text data
`items received from each of IM clients participating in the
`conference is held. Then,
`the held text data items are
`merged, and then data for text chat is created therefrom.
`The command processing portion 484 performs process-
`ing required for IM in accordance with the content of the
`command distributed by the analyzing and data/command
`distributed portion 482. For example, in accordance with a
`conference setting request command from an IM client, the
`command processing portion 484 performs processing
`required for setting a new conference. Further in accordance
`with a conference participating request command from an
`IM client, the command processing portion 484 performs
`processing required for calling the other IM clients to
`participate in the conference.
`
`Page 22 of 35
`
`

`

`US 7,233,589 B2
`
`5
`An output data creating portion 485 creates output data in
`accordance with results of the processing by the data pro-
`cessing portion 483 and/or the commandprocessing portion
`484.
`
`A packet assembling portion 486 assembles an IP packet
`by adding a TCP/UDP headerandan IP headerto the output
`data created by the output data creating portion 485. The
`packet assembling portion 486 sends the IP packet to the IP
`network 1.
`
`A connection managing portion 487 manages presence
`information of IM clients. In accordance with the presence
`information, the connection managing portion 487 controls
`each of the functional blocks 481 to 486 and relays between
`connections with the IM clients. Thus, the chat between the
`IM clients can be achieved.
`Incidentally, the packet deassembling portion 481 and the
`packet assembling portion 406 may be configured for
`example, in hardware with a use of such as integrated logic
`IC within the IP network IF 46.
`
`FIG. 3 shows an example of a presence information
`management table 488, which is used by the connection
`managing portion 487 for managing presence information.
`The presence information managing table 488 is stored in
`the external storage device 43, for example.
`As shown in FIG. 3, one record 440 includes a field 431
`for registering an account nameofa user of an IM client, a
`field 432 for registering an address of the IM client, a field
`433 for registering a nicknameofthe user of the IM client,
`a field 434 for registering an authentication key for using the
`IM-VoIP interconnecting system, a field 435 for registering
`presence information of the IM client, a field 436 for
`registering a medium (text chat and voice chat), which can
`be used by IM client for a chat, a field 437 for registering an
`address (IP address or DNS and a port number) of a
`conference room in which the IM client participates, a field
`438 for registering a nicknameof the conference room, and
`a field 439 for registering account names of the other IM
`clients with whom the IM client can chat.
`Here, when a terminal in which an IM client is installed
`is an IP terminal, an IP address or a DNSofthe terminal is
`registered, as an address of the IM client, in the field 432.
`When the terminal in which the IM client is installed is a
`
`6
`mined program loaded onto a memory 42 in a computer
`system having the same hardware configuration as that of
`the IM server 4. The program for configuring the functional
`blocks 581 to 585 may bedirectly loaded from a storage
`medium (not shown) such as a CD-ROM,through a reading
`device (not shown), or from the IP network 1 to the memory
`42 through the IP network IF 46. Alternatively, the program
`may be stored in an external memory device 43 and then
`may be loaded to the memory 42.
`A packet deassembling portion 581 receives an IP packet
`addressed to itself from the IP network 1. Then, the packet
`deassembling portion 581 processes an IP header and a
`TCP/UDPheader of the packet and extracts a payload.
`An analyzing and processing portion 582 analyzes the
`contentof the payload processed by the packet deassembling
`portion 581 and performs processing required for a voice
`chat in accordance with the content of the commandstored
`
`in accordance with a voice chat
`therein. For example,
`requesting commandfrom an IM client, processing required
`for calling the other parties of the conference in which the
`IM client participates to participate in a voice-chat is per-
`formed.
`An output data creating portion 583 creates output data in
`accordance with a result of the processing by the analyzing
`and processing portion 582.
`A packet assembling portion 584 assembles an IP packet
`by adding a TCP/UDP headerandan IP headerto the output
`data created by the output data creating portion 583. The
`packet creating portion 584 sends the created IP packet to
`the IP network 1.
`
`A connection managing portion 585 performs connection
`managementof a voice chat. In other words, the connection
`managing portion 585 controls each of the functional blocks
`581 to 584 to manage a connection between an IM client and
`the MD server 6. Thus, voice chatting among IM clients
`through the MD server 6 can be achieved.
`Incidentally, the packet deassembling portion 581 and the
`packet assembling portion 586 may be configured for
`example, in hardware with a use of such as integrated logic
`IC within the IP network IF 46.
`
`20
`
`25
`
`30
`
`40
`
`FIG. 5 shows an example of a VoIP connection manage-
`ment table 586, which is used by the connection managing
`portion 585 for managing a voice chat. The VoIP connection
`mobile telephone and/or a fixed telephone, a telephone
`numberand an IP address or a DNS of a VRserver 10 which
`managing table 586 is stored in the external storage device
`43, for example.
`voice relays between the mobile telephone and/orthe fixed
`As shown in FIG. 5, one record 540 includes a field 531
`telephone and the IP network 1 are registered therein.
`for registering an account nameofa user of an IM client, a
`Any one of information (OFF) indicating that the IM
`field 532 for registering an address of the IM client, a field
`client is offline, information (idle) indicating that the IM
`533 for registering a nickname ofthe user of the IM client,
`client is online but idle, information (text) indicating that the
`a field 534 for registering information regarding a method of
`IM client is text-chatting, information (voice) indicating that
`encoding voice signals carried in a VoIP packet, a field 535
`the IM client is voice-chatting, and information (text/voice)
`
`indicating that the IM client is both text chatting and voice for registering an address (IP address or DNS andaport
`chatting is registered in the field 435 as presence informa-
`numbercalled as “address for voice chat” hereinafter) of the
`tion.
`MDserver6, which is used for a voice chat in which the IM
`client participate, and a field 536 for registering a nickname
`of the conference room.
`Next, the IP terminal 7 will be described.
`Asdescribed above, an IM client to be used for chatting
`is installed in the IM terminal 7. A schematic configuration
`of the IP terminal 7 is shown in FIG.6.
`As shown in FIG. 6, the IP terminal 7 is achieved by
`configuring functional blocks 781 to 794 shown within a
`balloon 78 in FIG. 6, when, a CPU 41 executes a predeter-
`mined program (IM client program) loaded onto a memory
`42 in a computer system having the same hardware con-
`figuration as that of the IM server 4. The program for
`configuring the functional blocks 781 to 794 maybedirectly
`
`45
`
`50
`
`55
`
`60
`
`65
`
`An IP address assigned to the conference room is regis-
`tered in the field 437 as an address of the conference room.
`
`Whena user uses a plurality of IM clients, for example,
`whenthe client terminal 7-1 is used for text-chatting and the
`VoIP telephone 8 is used for voice-chatting, a record 440 is
`registered for each of them.
`The AP server 5 will be described next.
`
`Asdescribed above, the AP server 5 manages a connec-
`tion for voice chatting using VoIP. A schematic configuration
`of the AP server 5 is shown in FIG.4.
`
`As shown in FIG. 4, the AP server 5 is achieved by
`configuring functional blocks 581 to 585 shown within a
`balloon 58 in FIG. 4, when, a CPU 41 executes a predeter-
`Page 23 of 35
`
`Page 23 of 35
`
`

`

`US 7,233,589 B2
`
`7
`loaded from a storage medium (not shown) such as a
`CD-ROM,through a reading device (not shown), or from
`the IP network 1 to the memory 42 through the IP network
`IF 46. Alternatively,
`the program may be stored in the
`external memory device 43 and then may be loaded to the
`memory 42.
`A packet deassembling portion 781 receives an IP packet
`addressed to itself from the IP network 1. Then, the packet
`deassembling portion 781 processes a IP header and a
`TCP/UDPheader of the packet and extracts a payload.
`An analyzing and voice packet/data/commanddistribut-
`ing portion 782 analyzes the content of the payload pro-
`cessed by the packet deassembling portion 781 and distrib-
`utes the content to a voice packet deassembling portion 785
`as a voice packet (VoIP packet), a data processing portion
`783 as data, or a command processing portion 784 as a
`command.
`
`The data processing portion 783 analyzes a type of the
`data distributed by the analyzing and voice packet/data/
`command distributing portion 782 and performs processing
`in accordance with the type. Then,
`the data processing
`portion 783 notifies the result to an output data creating
`portion 786.
`The command processing portion 784 analyzes the con-
`tent of the commanddistributed by the analyzing and voice
`packet/data/commanddistributing portion 782 and performs
`processing in accordance with the content. Then, the com-
`mandprocessing portion 784 notifies the result to the output
`data creating portion 786.
`The output data creating portion 485 creates output data
`in accordance with results of the processing by the data
`processing portion 483 and the commandprocessing portion
`484. The created data is output from a display and