(19) United States
`(2) Patent Application Publication (10) Pub. No.: US 2003/0233417 A1
`Beyda et al.
`(43) Pub. Date:
`Dec. 18, 2003
`
`US 20030233417A1
`
`(54) SYSTEM AND METHOD FOR SIGNALING
`USING INSTANT MESSAGING IN
`MULTIMEDIATELEPHONY-OVER-LAN
`CONFERENCES
`(75) Inventors: William J. Beyda, Cupertino, CA (US);
`Rami Caspi, Sunnyvale, CA (US);
`Richard Crouch, Gustine, CA (US)
`Correspondence Address:
`Siemens Corporation
`Attn: Elsa Keller, Legal Administrator
`Intellectual Property Department
`186 Wood Avenue South
`Iselin, NJ 08830 (US)
`(73) Assignee: Siemens Information and Communica
`tion Networks, In
`(21) Appl. No.:
`10/174,261
`
`(22) Filed:
`
`Jun. 17, 2002
`Publication Classification
`
`(51) Int. Cl." … G06F 15/16
`(52) U.S. Cl. ............................................ 709/206; 709/231
`
`(57)
`
`ABSTRACT
`
`A telecommunications system according to an embodiment
`of the present invention includes an instant messaging (IM)
`server (106) and a VoIP server (110); and a plurality of
`system clients having both Volp and IM sub-clients. The
`system clients can log on to their respective servers in
`parallel. The IM server (106) and the VoIP server (110) can
`communicate lists of common participants and allow for IM
`conferences among at least subsets of the system clients
`while an audio or video teleconference is ongoing.
`
`100
`>
`
`
`
`
`
`PSTN/INTERNET
`104 |
`106
`108
`
`
`
`CELLPHONE
`118
`
`110
`
`F – — -> - - - - - - - - - - - - T
`|
`|
`i
`|
`|
`|
`|
`|
`|
`|M|
`|| CALENDAR || Wo? P ||
`| |SEEVER SERVER ||SERVER
`
`Page 1 of 17
`
`GOOGLE EXHIBIT 1028
`
`

`

`Patent Application Publication Dec. 18, 2003 Sheet 1 0f 10
`
`US 2003/0233417 A1
`
`__.(E-E"mzokado
`_no:8:k.u52%;}?m\nIIII
`IN.II.IIIIIIIIIIL/S_.02NSE
`
`9:xma"
`J"15235
`
`mm>mmm
`
`Page 2 0f 17
`
`Page 2 of 17
`
`
`
`

`

`Patent Application Publication Dec. 18, 2003 Sheet 2 of 10
`
`US 2003/0233417 A1
`
`FIG 2
`
`TELEPHONY
`APPLICATION
`
`IM
`APPLICATION
`
`CALENDAR
`P
`N
`APPLICATIO
`
`
`
`120
`
`Tontor
`
`-3015
`
`º'º - Lº
`
`VIDEO
`
`3000
`
`300+
`PCI
`
`301.4 L-3012
`
`* – DISK
`Vol I/O || 70 || 70
`5019 3016
`3018 3017
`
`S006
`
`J010
`
`INTERFACE
`
`LAN
`
`Page 3 of 17
`
`

`

`Patent Application Publication Dec. 18, 2003 Sheet 3 of 10
`
`US 2003/0233417 A1
`
`HH 4126b 4128b 4150b
`
`TO NETWORK
`
`FIC. 3B
`
`
`
`CALENDAR
`
`| 4202
`
`4203
`
`Page 4 of 17
`
`

`

`Patent Application Publication Dec. 18, 2003 Sheet 4 of 10
`
`US 2003/0233417 A1
`
`
`
`FIG 5
`
`Page 5 of 17
`
`

`

`Patent Application Publication Dec. 18, 2003 Sheet 5 of 10
`
`US 2003/0233417 A1
`
`FIC, 6
`
`IM APPLICATION LOGS
`ON TO IMserver
`
`CALENDAR APPLICATION
`SCHEDULES CONFERENCE
`
`SERVER DETERMINES
`IF IM CAPABLE
`
`
`
`602
`
`604
`
`
`
`606
`
`608
`
`CALENDAR INFORMS IM
`AND TELEPHONE SERVER
`
`
`
`
`
`DETERMINE PRESENCE
`OF IM CLIENTS
`
`TELEPHONE SERVER
`BEGINS CONFERENCE
`
`OPEN CHAT
`
`FIC 7
`
`CALLED PARTY
`RECEIVES CALL
`
`CALLED PARTY'S
`TELEPHONE
`APPLICATION ACTIVATES
`
`CALLED PARTY'S
`IM ACTIVATES
`
`CALLED PARTY'S
`IM RECEIVES LIST
`OF CONFEREES
`
`CALLED PARTY
`SELECTS FOR CHAT
`
`702
`
`704
`
`706
`
`708
`
`710
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 6 of 17
`
`

`

`P
`
`p
`
`US 2003/0233417 A1
`
`
`
`0__1_an_
`
`_
`
`___
`
`
`
`
`nl.Ilmm9E
`
`F/ofl0___.uE>mmmmm>~mmmm>$m_____F_mMZOTEEEESHHIEEIEHEIHIE"P_IIIIIIIIIIIILFIIIIIIIIIIIL_IIIIIIIIIIIILmo:m2m?mmyN/ofi
`
`
` m_Em“__Enema____w____BEES____h_s_mom__MES”;____3__n______m_2_"xoCE”;“unnm_u.fl__.__,_c______mmom____________m______i_wu"flHEBaz<m2:Em"nnmnm___.___ow._______n_“||L||l|_IIIIIll"IIIIIIIIIrIJ
`
`
`
`
`
`
`
`
`Page 7 0f 17
`
`Page 7 of 17
`
`
`

`

`Patent Application Publication Dec. 18, 2003 Sheet 7 of 10
`
`US 2003/0233417 A1
`
`FIC, 9
`
`800'
`
`804d
`
`805
`
`806
`
`CHAT
`D0CUMENT FC WINDOW
`
`802
`
`CHAT
`WINDOW
`
`808
`
`DOCUMENT
`
`804b
`
`900
``
`
`FIC, 1 O
`
`902
`
`904
`
`906
`
`908
`
`1100
`
`1102
`
`1 104
`
`1106
`
`FIC, 1 1
`SETUP IM AND
`TELEPHONE CONFERENCE
`
`CREATE WHITEBOARD
`DOCUMENT
`
`DRAG DOCUMENT
`TO IM WINDOW
`
`ATTACH DOCUMENT
`AND PACKETIZE
`
`SEND VIA IM SERVER
`
`READ INC PACKET
`
`
`
`
`
`
`
`EXPORT TO WHITEBOARD
`APPLICATION
`
`DISPLAY IN W.A.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 8 of 17
`
`

`

`Patent Application Publication Dec. 18, 2003 Sheet 8 0f 10
`
`US 2003/0233417 A1
`
`.___WK0E“Nzoc<2§<320$deE"mmzoEEEu{853%JZOEBEEruuuuuuuuuuuuuuuuI:rIIIIIIII:LuuuuuuuI-“HA,A,
`
`_________Lr.........IL
`
`Duom_¢o_ouom_
`
`
`
`
`
` __"mom___jlllLIIIIIIILIIIIITIJjlllLIIIIITIJjlllLIIIIIIIIIIIITIJL.
`
`
`
`
`
`Page 9 0f 17
`
`m;GEN
`
`©o_¢
`
`
`
`
`_._wom— Till
`
`_______
`
`Page 9 of 17
`
`
`

`

`Patent Application Publication Dec. 18, 2003 Sheet 9 of 10
`
`US 2003/0233417 A1
`
`1200
`----------
`IM2
`
`1402
`
`4106
`F
`
`FIG.
`104
`--------
`IMS
`1– — — — — — — — — — — H — —
`|
`|
`|
`|
`
`14A
`120 b
`-----4----
`IM2
`1– — — — — — — H – —
`
`|
`
`|
`
`º \
`[i]
`\
`|
`1410
`
`|
`
`|
`|
`
`|
`|
`
`FIC, 14B
`
`104
`
`1206
`
`Page 10 of 17
`
`

`

`Patent Application Publication Dec. 18, 2003 Sheet 10 of 10
`
`US 2003/0233417 A1
`
`FIC, 15
`
`H — — — — — — — — — — — — — —I F — — — — — — — — — - H - – - — — — — — — — — — — -1
`|
`|
`|
`|
`|
`|
`
`|*|| || || ||Nº||5|#|Nº|| || ||
`|
`|
`|
`|
`|
`|
`— — — — — — — — — — — — I---, -----T-----, -------- ––––F––
`1502
`1504
`S
`
`|
`|
`
`LOG IN
`
`BUDDY LIST
`
`|
`|
`
`|
`|
`|
`
`1506
`
`1510
`S
`
`CHAT
`
`|
`
`|
`
`| BUDDY ISI
`
`1508
`
`|
`|
`|
`|
`
`|
`
`|
`|
`
`|
`
`| CAL BUDDY -
`
`|
`|
`
`|
`
`|
`|
`|
`|
`|
`|
`|
`|
`CALL
`|
`1512
`|
`-———e--
`|
`|
`1514
`| MFDA CHANNEL
`2
`1516
`
`|
`
`|
`
`Page 11 of 17
`
`

`

`US 2003/0233417 A1
`
`Dec. 18, 2003
`
`SYSTEM AND METHOD FOR SIGNALING USING
`INSTANT MESSAGING IN MULTIMEDIA
`TELEPHONY-OVER-LAN CONFERENCES
`
`CROSS REFERENCE TO RELATED
`APPLICATIONS
`[0001] This application is related to concurrently-filed and
`commonly-owned patent application Ser. No.
`2
`(Attorney Docket: 2002P02748US) titled “System and
`Method for Collaborating Using Instant Messaging in Mul
`timedia Telephony-over-LAN Conferences.”
`
`BACKGROUND OF THE INVENTION
`[0002] 1. Field of the Invention
`[0003] The present invention relates to telecommunica
`tions devices and, in particular, to an improved system and
`method for multimedia telephony-over-LAN conferences.
`[0004] 2. Description of the Related Art
`[0005] An effective and user-friendly way to conduct
`multimedia audio and/or video teleconferences has long
`been sought. The promulgation of the H.323 Recommenda
`tion and the Session Initiation Protocol (SIP), as well as
`other multimedia packet protocols, have raised hopes that
`multimedia audio and video conferencing may yet reach
`their full potentials. Typically, to teleconference using such
`systems, users must log in to a teleconferencing server,
`identify themselves as authorized parties, and be scheduled
`for the conference.
`[0006] Instant Messaging (IM) allows users to log in to an
`Instant Messaging server and conduct a text-based “chat”
`session with groups associated with a user’s “buddy” lists.
`Typically, a user logs in to the server, identifies themselves
`as being present, uploads a “buddy” list, and then is invited
`to chat with those on the buddy list.
`[0007] Telephony systems and instant messaging systems
`both have advantages and circumstances of use in which one
`or the other might be more desirable. Typically, telephony
`systems and instant messaging systems are discrete systems
`requiring users to learn different sets of procedures for
`effective usage of each. Further, such systems do not typi
`cally allow for cross-system interaction. For example, while
`a user could establish an instant messaging session with a
`party while in the middle of a teleconference with that party,
`the instant messaging session would be wholly independent
`of the teleconference.
`[0008] As such, there is a need for a telecommunications
`system having a greater integration of instant messaging and
`teleconferencing. There is a still further need for such a
`system to have a relatively high level of ease of use.
`
`SUMMARY OF THE INVENTION
`[0009] These and other problems in the prior art are
`overcome in large part by a system and method according to
`the present invention.
`[0010] A telecommunications system according to an
`embodiment of the present invention includes a packet
`network; a plurality of network clients operably coupled to
`the packet network, the network clients including a plurality
`of telephony clients and an associated plurality of Instant
`
`Messaging clients. Signaling for the telephony clients is
`adapted to be provided by the Instant Messaging clients
`using Instant Messaging channels.
`[0011] In certain embodiments, the Instant Messaging
`channels carry the telephony call set up and signaling. In
`other embodiments, the telephony call setup and signaling is
`separate, but applications such as whiteboarding use the
`Instant Messaging channels.
`[0012] A telecommunications method according to an
`embodiment of the present invention comprises establishing
`an instant messaging session among a plurality of users; and
`using an instant messaging channel of the instant messaging
`session to establish a multimedia over IP session among the
`users. In certain embodiments, the multimedia over IP
`session comprises a whiteboarding session during a pre
`established multimedia session. Sending the whiteboarding
`session comprises sending a whiteboarding document in a
`chat window of the instant messaging session and identify
`ing the whiteboarding document as a whiteboarding docu
`ment and including size and type information. On receiving,
`the receiving client reads the size and type information and
`provides the whiteboarding document to a whiteboarding
`application before it can be opened in a receive chat window.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`[0013] A better understanding of the invention is obtained
`when the following detailed description is considered in
`conjunction with the following drawings in which:
`[0014] FIG. 1 is a block diagram of a telecommunications
`system according to an embodiment of the present inven
`tion;
`[0015] FIG. 2 is a block diagram of an exemplary tele
`communications client according to an embodiment of the
`present invention;
`[0016] FIG. 3A and FIG. 3B are more detailed block
`diagrams of telecommunications clients according to
`embodiments of the present invention;
`[0017] FIG. 4 is a block diagram of an exemplary tele
`communications server according to an embodiment of the
`present invention;
`[0018] FIG. 5 is a diagram illustrating an exemplary
`graphical user interface according to an embodiment of the
`present invention;
`[0019] FIG. 6 is a flow diagram illustrating operation of
`an embodiment of the present invention;
`[0020 FIG. 7 is a flow diagram illustrating operation of
`an embodiment of the present invention;
`[0021] FIG. 8 is a diagram illustrating signaling according
`to an embodiment of the present invention;
`[0022] FIG. 9 is a diagram illustrating an exemplary
`graphical user interface according to an embodiment of the
`present invention;
`[0023] FIG. 10 is a diagram illustrating an exemplary data
`element according to an embodiment of the present inven
`tion;
`[0024] FIG. 11 is a flow diagram illustrating operation of
`an embodiment of the present invention;
`
`Page 12 of 17
`
`

`

`US 2003/0233417 A1
`
`Dec. 18, 2003
`
`[0025] FIG. 12 is a diagram illustrating signaling accord
`ing to an embodiment of the present invention
`[0026] FIG. 13 is a block diagram of an MCU according
`to an embodiment of the present invention;
`[0027] FIG. 14A and FIG. 14B are diagrams illustrating
`signaling according to embodiments of the present inven
`tion; and
`[0028] FIG. 15 is a diagram illustrating signaling accord
`ing to an embodiment of the present invention.
`DETAILED DESCRIPTION OF THE
`INVENTION
`[0029] Turning now to the drawings and, with particular
`attention to FIG. 1, a diagram of a telecommunications
`system 100 according to an embodiment of the present
`invention is shown. The telecommunications system 100
`includes a packet network such as a LAN 102 to which are
`coupled a plurality of network devices. In particular, shown
`are a server 104, which includes an Instant Messaging server
`106, a calendar server 108, and a telephony server 110. In the
`embodiment illustrated, the server 104 couples to the Public
`Switched Telephone Network (PSTN) or Internet. The
`server 104 may also include gateway and gatekeeper, as well
`as multipoint control unit functionality. It is noted that, while
`a single server 104 is shown, the server functionality may
`also be implemented using a plurality of individual servers
`coupled to the LAN 102. Thus, the figures are exemplary
`only.
`[0030] The telecommunications system also includes a
`plurality of network clients, which may be implemented in
`a variety of configurations. For example, shown are a
`wireless telephone 112, including a cordless base station 116
`and a handset 114; a cellular telephone 118, which couples
`in through a cellular network (not shown) and the PSTN; a
`LAN telephone 124; a computer 120 implementing a tele
`phony application program; and a network client 122 imple
`menting a CTI (computer-telephony integration) type sys
`tem, including a computer 121 and a coupled telephone 123.
`[0031] The network clients 120, 122, 124,118, 112 imple
`ment telephony sub-clients 126a-126e, respectively, and
`Instant Messaging sub-clients 128a-128e, respectively, as
`will be explained in greater detail below. More particularly,
`as shown in FIG. 2, each illustrated network client includes
`a telephony sub-client 126, an Instant Messaging sub-client
`128, and a calendaring application 130. A client 120 may
`also implement a graphical user interface 131. In operation,
`the telephony sub-client or application 126 provides tele
`phone functionality according to the appropriate standard for
`each device, e.g., an IP telephony standard for transmittal on
`the LAN 102. For example, in some embodiments, the
`telephony sub-clients 126 implement the H.323 Recommen
`dation or the Session Initiation Protocol (SIP), although
`other packet multimedia protocols may be employed. The
`Instant Messaging sub-clients 128 implement an appropriate
`Instant Messaging standard. As noted above, suitable Instant
`Messaging standards include the Netscape AOL Instant
`Messenger, AT&T Instant Messenger, Yahoo! Pager, or
`Microsoft Instant Messenger. The calendar application 130
`functions with the calendaring server 108, such as Microsoft
`Exchange server, to schedule teleconferences and chat ses
`sions, as will be explained in greater detail below. Suitable
`calendar software includes Microsoft Outlook.
`
`[0032] Also coupled to the LAN 101 may be a telephony
`feature access server or device 103 which couples the LAN
`101 to a PBX 105, such as a Hicom PBX. A variety of PBX
`telephony devices 140a, 140b may also couple to the PBX
`105. Devices on the network 102 may be configured to
`obtain their telephony services through the PBX 105 via the
`TFA 103. An exemplary telephony feature access server is
`the Hicom Feature Access server, available from Siemens
`Corp., and employing the Cornet protocol. In certain
`embodiments, the TFA 103 may be equipped with Instant
`Messaging, calendaring, and VoIP capabilities in accordance
`with the present invention, either as a server or a network
`device. In what follows, for sake of simplicity, the descrip
`tion will focus primarily on the server 104, it being under
`stood that the teachings of the present invention are equally
`applicable to the TFA 103.
`[0033] FIGS. 3A and FIG. 3B are block diagrams of
`exemplary network clients according to embodiments of the
`present invention. In particular, the client 120 of FIG. 3A is
`exemplary of a personal computer coupled to the network
`via a network interface card (or wireless network interface
`card). As shown, the network client 120 includes a bus 3000,
`such as a PCI bus. Coupled to the bus 3000 are one or more
`processors 3002; memory 3004, such as RAM or SRAM; a
`disk drive 3006; one or more Universal Serial Bus interfaces
`3008; a video interface 3012; and a LAN interface 3010.
`[0034] The processor 3002 may be implemented as any of
`a variety of processors or controllers, including one or more
`x86 or Pentium-type processors and/or one or more digital
`signal processors. The processor 3002 implements an IM
`client 3126, a telephony client 3128, and a calendar client
`3130, according to the present invention. The software for
`implementing the clients may be stored on the disk drive
`3006 and memory 3004 in a known manner. The video
`interface 3012 may couple to a monitor 3013 and display a
`graphical user interface 3014 executable by the processor
`3002 for use with the software clients. The USB interface(s)
`308 interface to I/O devices, such as microphone 3016,
`speaker(s) 3018 and a keyboard 3017 and/or cursor input
`device 3019. The processor may also implement a graphical
`user interface 3131, as will be described in greater detail
`below.
`[0035] Finally, as noted above, a network interface 3010
`interfaces the network client to a network, such as a local
`area network (LAN). The network interface 3010 may be
`implemented as a wireless or wired interface. It is noted that
`the network client 122 (FIG. 1) is generally similar, though
`the telephone functionality is provided by the coupled
`telephone 123 rather than the software client.
`[0036] FIG. 3B illustrates another embodiment of the
`network client and, particularly, a wireless client 112. The
`wireless client 112 may be embodied, for example, as any
`wireless telephone system, including a DECT system or
`other system functioning, for example, in the 2.4 GHz range.
`The network client 112 includes a base station 116 and one
`or more handsets or portable units 114. For convenience,
`only one such handset 114 is shown.
`[0037] The base station 116 includes a master processor
`4000a, a transceiver 4002a, and a network interface 4004a.
`In certain embodiments, both the base station 114 and the
`handset 116 may be used to make telephone calls. Thus, the
`master processor may implement an IM client 4126a, a
`
`Page 13 of 17
`
`

`

`US 2003/0233417 A1
`
`Dec. 18, 2003
`
`telephony client 4128a, and a calendar client 4130a accord
`ing to embodiments of the present invention. Typically, the
`clients are implemented as programs stored in memory (not
`shown). In such an embodiment, the base station 114 will
`also include a display (not shown), such as an LCD display,
`and a keypad (not shown). In other embodiments, the base
`station 114 is not used directly for user interaction and thus
`does not include a display or full IM client, telephony client
`and calendar client functionality.
`[0038] The transceiver 4002a functions to implement cod
`ing and decoding and generally to communicate with the
`handsets 114. The network interface 4004a interfaces the
`network client 112 to the network. Thus, the network inter
`face may be a local area network interface.
`[0039] The handset 114 includes a master processor
`4000b, a transceiver 4002b, a keypad 4006, and a display
`4008. It is noted that in certain embodiments, the keypad is
`implemented as a “virtual” touch screen on the display 4008.
`Thus the figure is exemplary only. The display is used to
`implement a graphical user interface for IM, telephony and
`calendar functionality, as will be explained in greater detail
`below.
`[0040] The master processor 4000b includes a IM client
`412.6b, a telephony client 4128b, and a calendar client
`4130b, as will be explained in greater detail below. Typi
`cally, the clients are implemented as programs stored in
`memory (not shown).
`[0041] As will be explained in greater detail below, in
`operation, the user can use the keypad and screen 4006, 4008
`to set a date and parties for a conference. The network server
`104 then acts to set up and coordinate the conference
`[0042] It is noted that a cellular telephone or PDA 118
`according to embodiments of the present invention is con
`figured generally similarly. Typical cellular telephone stan
`dards include GSM and IS-136 TDMA.
`[0043] FIG. 4 is a block diagram of an exemplary network
`server according to an embodiment of the present invention.
`As shown, the server includes an IM server 106, a calendar
`server 108, and a telephony server 110. The server further
`includes on or more network interfaces 6000 for interfacing
`the servers to the local area network.
`[0044] The calendar server 108 implements a central cal
`endar application 4202 which is used to schedule telecon
`ferences from network clients. The calendar server 108 thus
`further includes a memory 4203 for storing configuration
`information, such as lists of clients, and times of confer
`ences, including the appropriate parties. In operation, the
`calendar application 4202 receives a request for a confer
`ence and stores the date and parties in the memory. The
`conference is then communicated to the IM server 106 and
`the telephony server 110.
`[0045] The IM server 106 likewise includes a memory
`4204 for storing configuration information and for receiving
`“buddy” lists from IM clients. In operation, the IM client
`activates chat sessions among at least a subset of the parties
`to the teleconference, in some embodiments, receiving mes
`sages and retransmitting them to other chat parties.
`[0046] The telephony server 110 also includes a memory
`4206 for storing configuration information and calling and
`called party information. In operation, the telephony server
`
`receives the conference appointment time from the calendar
`server 108 and, at the appointed time, either arranges the
`system for the conference or transmits a request to the host
`user to begin the conference.
`[0047] In addition, the server may include gateway 4102
`and gatekeeper functionality 4104 (in an embodiment
`employing H.323), as well as multipoint control unit (MCU)
`4106 functionality. Finally, while illustrated as a single
`server, each of the IM server, calendar server, and the
`telephony server, and the various other components, could
`be implemented as discrete units. Thus, the figure is exem
`plary only.
`[0048] Operation of an embodiment of the present inven
`tion is explained with reference to FIG. 5 and FIG. 6. In the
`discussion that follows, network client 120 shall be referred
`to, it being understood that the teachings of the present
`invention are equally applicable to the other network clients
`and their functionality generally similar.
`[0049] As shown in FIG. 5, the telephony client 126, the
`instant messaging client 128, and the calendar application
`130 may be implemented including a graphical user inter
`face. In operation, the user may activate one or more instant
`messaging windows 128-1, 128-2 and one or more calendar
`windows 130. In a typical embodiment, the calendar appli
`cation 130 and the instant messaging windows 128-1, 128-2
`are opened from within the telephony application 126-1.
`Thus, these applications may form an integrated part of the
`telephony application, or may be implemented using plug
`ins. In other embodiments, these applications may be imple
`mented using separate applications that are nonetheless
`adapted to communicate with one another. In still other
`embodiments, the calendar application or the instant mes
`saging applications may be opened first, and the other
`applications from them. In the embodiment illustrated, the
`application further includes an address book or buddy list
`402.
`[0050] FIG. 6 is a flow diagram illustrating operation of
`an embodiment of the present invention. As shown, in 602,
`the IM application 128 is used to log on and provide a buddy
`list to the IM server 106. In 604, the calendar application 130
`is used to schedule a teleconference using the calendar
`server 108. For example, the user may set a date and identify
`the parties to the teleconference, and the parties may be
`informed of the time of the conference. At 606, the server
`determines if the parties to the teleconference have IM
`capability. It is noted that in certain embodiments, the party
`information may be handed off to the IM server 106 or the
`telephony server 110 for them to make this determination. In
`608, the calendar server 108 informs the IM server and the
`telephony server of the conference. This can occur at or
`before the scheduled time of the conference; and may occur
`before step 606. At 610, the IM server determines a presence
`of IM clients, at the time of the conference. At 612, the
`telephony server 110 begins the conference by calling all
`parties, i.e., by communicating with the telephone clients. At
`614, the chat channels are opened.
`[0051] It is noted that, in certain embodiments, at the time
`of the conference, the party who called the conference may
`be contacted by the calendar server (e.g., via a pop up
`message box) and given the option of starting the conference
`or delaying it (i.e., if not all participants are present.).
`[0052] FIG. 7 is a flowchart illustrating handling when a
`user or network client receives a conference call in embodi
`
`Page 14 of 17
`
`

`

`US 2003/0233417 A1
`
`Dec. 18, 2003
`
`ments of the present invention. In a step 702, the called party
`receives the call, i.e., a call request and call setup signaling.
`In step 704, the called party’s telephony application acti
`vates and proceeds with appropriate handshake signaling to
`establish the call. In step 706, the called party’s IM appli
`cation activates. For example, this may occur automatically,
`or the called party may manually click on an associated IM
`icon. In 708, the called party’s IM application receives the
`list of participants to the conference. Finally, in step 708, the
`called party can select from the list to establish a chat session
`with various of the participants.
`[0053] Turning now to FIG. 8, a diagram illustrating
`operation of an embodiment of the invention in greater detail
`is shown. Shown are a plurality of network clients 120-1,
`120-2, 120-3, an IM server 106, a calendar server 108, and
`a telephony server 110. The network client 120-1 includes an
`IM application IM1, calendar application C1, and telephony
`application T1; the network client 120-2 includes an IM
`application IM2, calendar application C2, and telephony
`application T2; and the network client 120-3 includes an IM
`application IM3, calendar application C3, and telephony
`application T3.
`[0054] In the example illustrated, it is assumed that the
`conference is set up at the behest of the user of network
`client 120-1. Initially, at 501, the user at network client
`120-1 activates his calendar application C1 to set a time and
`date for the teleconference. As noted above, any calendar
`application may be used. One such application is Microsoft
`Outlook. At 502, the time and date are then uploaded to the
`calendar server 108. A suitable calendar server is the
`Microsoft Exchange server.
`[0055] In certain embodiments, the identity of the other
`parties to the conference is also uploaded to the server. Thus,
`in 504, the calendar server 108 saves the time, date and the
`party information. At 506, the IM server 106 and the
`telephony server 110 may verify the parties identified.
`[0056] Next, in 508, the calendar server 108 sends an
`acknowledgement to the calendar application C1 that set the
`conference. At 510, the calendar server 108 sends a message
`to the network client at the conference time, indicating that
`the conference is to begin. In response, at 512, the telephony
`client T1 sends a conference request message to the tele
`phony server 110. At 514, the telephony server responds to
`set up the call to the originator and the other telephony
`clients T2, T3. The teleconference would typically be imple
`mented in a multicast mode. The instant messaging client
`IM1 then logs on to the instant messaging server 106, at 514.
`Then, at 516, the other instant messaging applications of
`parties to the conference log in. The users are then able to
`maintain a chat session with other parties while the telecon
`ference is ongoing.
`[0057] It is noted that, in certain embodiments, the calen
`dar application C1 informs the other applications IM1 and
`T1 of the time and scheduling of the conference. The
`telephony application T1 and the IM client IM1 then begin
`the conference at the appointed time.
`[0058] In certain embodiments of the present invention,
`signaling for a teleconference may be provided over the
`instant messaging channel. For example, in one embodi
`ment, the instant messaging channel and system may be used
`to convey “whiteboarding” documents. “Whiteboarding
`
`documents” is a generic term for word processing, spread
`sheet, presentation, or similar documents that may be used
`in a conference and “marked up” by various of the parties to
`the conference. In such an embodiment, the user can “drag”
`the whiteboard document to the Instant Messaging window,
`have the document converted into an Instant Messaging
`transmission format, sent along the Instant Messaging chan
`nel, and received and displayed at the other end. Typically,
`the document will be “intercepted” before the Instant Mes
`saging window tries to open it, such that it can be displayed
`in the appropriate application’s window.
`[0059] For example, FIG. 9 illustrates a GUI 800 having
`a whiteboarding document 804a and a chat or instant
`messaging window 806. The user drags the document into
`the chat window, as represented by the arrow 805. In GUI
`802, representative of a receiving client, the chat window
`808 is shown, with the newly received document 804b in its
`own window.
`[0060] In operation, the whiteboarding document is pack
`aged into an instant messaging format for transmission on
`the instant messaging channel. A protocol element identifies
`the packets as belonging to a whiteboarding application.
`One such protocol element is shown in FIG. 10. As shown,
`the element 900 includes a type field 902, a sender field 904,
`a destination field 906, and a message or payload field 908.
`The type field 902 identifies the package as belonging to a
`whiteboarding application and allows the receiving Instant
`Messaging application to port the received document to the
`appropriate application program.
`[0061] FIG. 11 is a flow diagram illustrating operation of
`an embodiment of the present invention. At 1100, a telecon
`ference including Instant Messaging is set up, for example,
`in the manner described above. In step 1102, a user creates
`a whiteboarding document, for example, through use of a
`word processing or spreadsheet application. At step 1104,
`the user drags the document to the chat window of the
`Instant Messaging application. At step 1106, the Instant
`Messaging application identifies the type of document and
`packages it into the Instant Messaging transmission format.
`At step 1108, the document is sent to the destination via the
`Instant Messaging server. At step 1110, the document is
`received and the packet type is read by the Instant Messag
`ing application. Because the Instant Messaging application
`identifies the document as a whiteboarding document, it is
`exported to the appropriate application at 1112. Finally, at
`1114, the document is displayed in the appropriate applica
`tion window.
`[0062] Turning now to FIG. 12, a signaling diagram
`illustrating operation of an embodiment of the present
`invention is shown. Shown are a client 120a, a client 120b,
`and a server 104. The client 120a includes an Instant
`Messaging application IM1, a telephony application T1, and
`a whiteboarding application A1. Similarly, the client 120b
`includes an Instant Messaging application IM2, a telephony
`application T2, and a whiteboarding application A2. The
`server 104 includes an Instant Messaging server IMS and a
`telephony server Ts.
`[0063] At 1202, the application document is loaded from
`the application A1 to the Instant Messaging program IM1. At
`1204, the Instant Messaging program IM1 sends the docu
`ment to the Instant Messaging server IMs. At 1206, the
`Instant Messaging server sends the document to the desti
`
`Page 15 of 17
`
`

`

`US 2003/0233417 A1
`
`Dec. 18, 2003
`
`nation Instant Messaging application IM2. The Instant Mes
`saging application IM2 then reads the header information
`and transfer the document to the application A2, at 1208.
`The document can then be marked up by the client at 120b
`and sent back.
`[0064] In certain embodiments of the present invention,
`the multipoint control unit (MCU) 4106 (FIG. 4) may itself
`be a party to the IM session. As noted above, the MCU 4106
`may be implemented as a unit of the server or as a standalone
`unit. Such an exemplary MCU 4106 is shown in FIG. 13.
`The MCU 4106 includes a multipoint controller 1302, a
`multipoint cha