(12) United States Patent
`Seligmann
`
`USOO633.0022B1
`US 6,330,022 B1
`(10) Patent No.:
`Dec. 11, 2001
`(45) Date of Patent:
`
`(54) DIGITAL PROCESSINGAPPARATUS AND
`METHOD TO SUPPORT VIDEO
`CONFERENCING IN VARIABLE CONTEXTS
`
`(75) Inventor: Doree Seligmann, New York, NY (US)
`(73) Assignee: Lucent Technologies Inc., Murray Hill,
`NJ (US)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(*) Notice:
`
`(21) Appl. No.: 09/186,181
`(22) Filed:
`Nov. 5, 1998
`(51) Int. Cl." ................................................... H04N 7/14
`(52) U.S. Cl. ..................................... 348/14.08; 348/14.09;
`348/14.12
`(58) Field of Search ........................ 348/14, 15; 345/330,
`345/331, 327, 302; 709/204; 370/260, 261;
`H04N 7/14, 7/15
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`8/1983 Fields.
`4,400,724
`3/1991 Ashida et al..
`5,003.532
`2/1993 Zimmermann ....................... 358/209
`5,185,667
`10/1993 Watanabe.
`5,257,306
`6/1994 Nakatsuka.
`5,323.445
`8/1994. Addeo et al..
`5,335,011
`9/1994 Nitta.
`5,347,306
`11/1994 Shibata et al..
`5,365,265
`1/1995 Martin et al. .
`5,384,588
`9/1995 Chen et al..
`5,453,780
`5/1997 Altom et al. ......................... 345/330
`5,627,978
`10/1997 Schoolman ............................. 348/14
`5,677,728
`6/1999 Yamagishi et al.
`... 34.5/507
`5,914,728
`5,953,050 * 9/1999 Kamata et al. ...
`... 379/93.21
`5.999,208
`12/1999 McNerney et al. .................... 348/15
`
`FOREIGN PATENT DOCUMENTS
`407038865A * 2/1995 (JP).
`... HO4N/7/15
`410056626A * 2/1998 (JP).
`... HO4N/7/15
`02000032446A * 11/2000 (JP) ................................. HO4N/7/15
`* cited by examiner
`Primary Examiner-Curtis Kuntz
`ASSistant Examiner Melur Ramakrishnaiah
`(57)
`ABSTRACT
`Video processing apparatus for Supporting a video confer
`ence includes a processor/server, a conference bridge and a
`network of terminals, each terminal equipped with at least a
`Video camera, display and a user input device Such as a
`mouse. A user may Select a conference context including a
`conference location, type (Setting) and purpose that may be
`different from the selection of another conferee. The user
`may select a conference location and type, View a confer
`ence according to a default Selection or design their own
`location and type in advance of the conference. The user also
`may Select personal requirements and Signal the need for
`various resources prior to and in real time during a confer
`ence. At the conference bridge, each conferee that joins a
`Video conference is assigned a position at a conference
`Setting of their Selected context, for example, a round table
`or U shaped table generated as a Video graphic. At each
`terminal, a signal is either received or generated for display
`Such that each conference participant is able to view all other
`participant conferees according to the assigned position
`arrangement within the context Selected by the conferee. An
`algorithm is described for generating Such a signal for
`display. An audio signal can also be generated according to
`the assigned positions of the conferees that can be heard on
`3D or Stereo speakers in Such a way as to relate to the
`assigned positions around the Virtual conference Setting.
`Moreover, as conferees join or leave the conference, the
`Virtual Setting may increase in size to accommodate the new
`conferees.
`
`32 Claims, 6 Drawing Sheets
`
`110
`
`
`
`111b
`
`11a
`
`11
`
`SO
`
`
`
`121
`
`
`
`PSTN
`
`GO
`
`CONFERENCE
`
`AUX PROC SERVER
`
`170
`
`RPX Exhibit 1007
`
`

`

`U.S. Patent
`
`Dec. 11, 2001
`
`Sheet 1 of 6
`
`US 6,330,022 B1
`
`110
`
`FIG. 1
`11b)
`
`111a
`
`120
`
`1OO
`
`
`
`
`
`PSTN
`
`160
`
`CONFERENCE
`
`170
`
`AUX PROC SERVER
`
`OTHER NETWORK
`
`
`
`RPX Exhibit 1007
`
`

`

`U.S. Patent
`
`Dec. 11, 2001
`
`Sheet 2 of 6
`
`US 6,330,022 B1
`
`
`
`FIG. 5
`
`
`
`50
`
`530
`
`WIDEO
`PRESENTATION
`
`RPX Exhibit 1007
`
`

`

`U.S. Patent
`
`Dec. 11, 2001
`
`Sheet 3 of 6
`
`US 6,330,022 B1
`
`FIG. 4
`
`160
`
`
`
`
`
`
`
`
`
`CONTROL
`UNIT
`
`441
`
`
`
`
`
`NETWORK
`INTERFACE
`
`NETWORK
`INTERFACE
`
`NETWORK
`INTERFACE
`
`NETWORK
`
`
`
`
`
`MULTIMEDIA
`ENDPOINT
`
`MULTIMEDIA
`ENDPOINT
`
`MULTIMEDIA
`ENDPOINT
`
`RPX Exhibit 1007
`
`

`

`U.S. Patent
`
`Dec. 11, 2001
`
`Sheet 4 of 6
`
`US 6,330,022 B1
`
`FIG. 6A
`CONTEXT TABLE
`
`LOCATION
`
`WHITE HOUSE OVAL OFFICE
`ROOM 1135C
`AS IS
`
`TYPE
`
`ROUND
`SQUARE
`U-SHAPE
`
`PURPOSE
`
`FIG. SB
`CONTENSE TABLE
`
`
`
`
`
`NAME |
`DON
`
`ADDRESS/
`7
`908-483-2160
`
`MEDIA
`| VIEW HEAR REEEENT FEATURES
`YESIYES
`POWER POINT |
`VBR, NET
`PHOTO
`MEETING
`HDTV
`
`
`
`REPRESENTATION:
`
`RPX Exhibit 1007
`
`

`

`U.S. Patent
`
`Dec. 11, 2001
`
`Sheet S of 6
`
`US 6,330,022 B1
`
`FIG. 7A
`
`Events from "switch/bridge" to server:
`New Call Context (like MMCX)
`- Create new Context-assign a location and type-create Context protocol
`-- Signal browsers
`New-Person
`-- Compute context object for person
`-- signal browsers
`ReMove-Person; Enter-PersOn: Leave-PersOn
`-- signal browsers
`
`ses
`
`FIG 7B
`
`Event from server to "switch/bridge"
`-- notify Of location langle Of SOUnd SOUrces
`-- notify of video requirements
`
`FIG. 70
`
`Events froA Server to browser:
`New-Context/Call (of type t at location l)
`-- Create visual of virtual place with Context object(s) and protocol
`New-Person (at Context object)
`-- Create visual for context object if needed
`- Create visual of person and assign it to Context object
`-- Context object adjusts to accommodate person object
`-- person object places itself at Context object
`-- Create visual of person and assign it to Context object
`Remove-Person (at Context object)
`-- remove visual of person
`-- Context object automatically reconfigures
`-- Leave-Person (at Context object) signal server of person object's "head" position and angle
`-- perSOn Object adopts "absent visual
`-- Context object keeps placeholder for personobject
`Enter-PerSOn
`-- person object adopts present" visual
`-- COntext Object adjust to acCOMMOdate person object
`
`RPX Exhibit 1007
`
`

`

`U.S. Patent
`
`Dec. 11, 2001
`
`Sheet 6 of 6
`
`US 6,330,022 B1
`
`FIG. 7D
`
`Events In BrOWser Signal Srever:
`Person Object Placed:
`-- Signal Server of person object's "head" position and angle
`- Compute resolution (based On transformation matrix and number of pixels required)
`-- Signal server of new video resolution requirements--similar to Level-Of-Detail
`View parameters change:
`-- signal server of "head" position and angle
`-- for every person object with video SOUrce:
`-- Signal server of person object's "head" position and angle
`-- COMpute resolution (based On transformation matrix and number of pixels required);
`-- Signal server of new video resolution requirements--similar to Level-Of-Detail
`
`Freeze Video (person)
`-- Store last frame
`-- Signal server to stop video transmission
`
`RPX Exhibit 1007
`
`

`

`US 6,330,022 B1
`
`1
`DIGITAL PROCESSINGAPPARATUS AND
`METHOD TO SUPPORT VIDEO
`CONFERENCING IN VARIABLE CONTEXTS
`
`I. BACKGROUND OF THE INVENTION
`
`5
`
`15
`
`35
`
`40
`
`25
`
`1. Technical Field
`The present invention relates to the field of video confer
`encing and, more particularly, to apparatus and a method for
`providing a simulated Video conferencing environment
`including live Video conferees in variable conference con
`texts and controlling the conference during events Such as 1)
`conferee removal, 2) temporary conferee departures or
`additions, and 3) changes in conferee requirements.
`2. Description of the Related Arts
`There has developed a need in the telecommunications art
`to provide Simulated/live and real time Video conferencing
`So that conferees may a) select a virtual conference context,
`b) communicate among themselves within the Selected
`context, and c) share multimedia resources and tasks
`dynamically during a "virtual meeting. By “conference
`context,” it is meant a user's Selection of a default or
`personal conference location, type (Setting), purpose and
`other conference environmental factors of a virtual confer
`ence setting. As represented by Shibata et al., U.S. Pat. Nos.
`5,365,265 and 5,604,738, in the video conferencing arts, a
`“chairperSon' is typically designated to control the Virtual
`meeting room and participation of conferees. Thus the
`context is typically Selected and controlled by the chairper
`son. There is no flexibility in the choice of context by the
`conferees Since the chairperSon Selects the context.
`Also, as certain events occur, for example, conferees join,
`temporarily withdraw or permanently leave a conference,
`there is needed a way of Signaling the characteristics of a
`particular conferee and what their individual needs and
`capabilities are. Otherwise, the conferee would have no
`ability to signal their personal conference preferences, and
`the resources needed by the conferee to partake in the
`conference may not be made available. The conference
`would not be as efficient if all conferee requirements pass
`through a chairperson for approval.
`One proposal for providing a multimedia Video confer
`ence is a Services creation and execution environment called
`MR, for Meeting Room. MR is supported by “Archways” a
`45
`distributed computer program that manages conferee, com
`puter and network-based resources to Support multimedia
`communication Sessions among users, especially, within the
`context of long-term collaborations. Consequently, MR is
`especially useful within local area networks or private data
`networks but has not yet been extended to Internet, intranet
`and public Switched telecommunications network applica
`tions.
`Three dimensional graphics are used to provide conferees
`with Simulated conference table Settings. Three dimensional
`Sound Systems enable conferees to hear, for example, con
`ferees virtually Seated to their right through the right Stereo
`channel Signal and Virtually Seated to their left through the
`left Stereo channel Signal. Full body photographs of each
`conferee are image processed to provide graphically
`enhanced taking heads and bodies. By “graphically
`enhanced, it is meant the graphically drawn or painted and
`animated image portions Such as the moving mouth.
`United States Patents and articles describing MR, Arch
`ways and languages and interfaces for use in these and
`similar systems include: U.S. Pat. Nos. 5,471,318 and 5,689,
`553; Ahuja et al., “Archways: Making Remote Multimedia
`
`50
`
`55
`
`60
`
`65
`
`2
`Conversations Persistent and Natural,” Proceedings of Tech
`nology Summit Telecom '95, Oct. 2-7, 1995; Seligman,
`“Interactive Intent-Based Illustrations: A Visual Language
`for 3D Worlds.” Ph.D. Thesis, Dept. of Computer Science,
`Columbia University, 1993 and Seligmann et al., “Providing
`ASSurances in a Multimedia Interactive Environment,” Pro
`ceedings of ACM SIGCHI '95 Human Factors in Computing
`Systems, Denver, Colo., May 7–11, 1995. The status of the
`use of 3D graphics enhancement of multimedia communi
`cation is also described by “Automatically Generated 3D
`Virtual Environments for Multimedia Communication.”
`Fifth International Conference in Central Europe on Com
`puter Graphical and visualization 97, Plzen, Czech
`Republic, Feb. 10–14, 1997.
`In their 553 patent, Ahuja et al. describe calling party
`control of a conference including the naming of a room and
`conferee control of the Saving of predetermined portions of
`a meeting. Meetings may be terminated and restarted, and
`user files may be Stored. Also, a network blackboard feature
`may be provided which is a virtual space for conferees to
`make certain data, Video or other entries that can be
`observed by the other conferees. The network blackboard is
`Selectable from a menu of options provided a conferee on
`conference initiation. But there appears to exist no indi
`vidual party control of the conference room, nor may an
`individual party signal their needs for additional resources or
`for changes to the conference context during a conference.
`Also, there is little disclosure of how the Virtual meeting
`room functions in response to certain events Such as the
`addition or removal of a conferee.
`Other recently issued patents describing video conferenc
`ing apparatus and methods include U.S. Pat. Nos. 5,003,532;
`5,323.445; 5,335,011; 5,347,306; 5.453,780; 5,491,743;
`5,638,114 and 5,675,374.
`Use of Simulated talking heads derived from photographs
`is not as desirable as providing real-time, live images of the
`conferee. These can be provided by a camera associated with
`a personal computer and overlaid on a computer graphics
`generated background, Such as a conference room Setting.
`Also, it is known from U.S. Pat. Nos. 5,185,667; 5,384,588
`and their progeny to capture a 180 degree image with a fish
`eye lens. The captured image may be a view of a conference
`room Such as a jury room, conference room, office Setting or
`other view. A user may pan, tilt, rotate and Zoom within a
`180 degree image of a conference including Such a 180
`degree view in real time from their input device.
`SUMMARY OF THE INVENTION
`Consequently, I have recognized that there is a need in the
`art to provide a conference bridge and methodology for
`providing a real-time live image of a conferee in a simulated
`3-D graphically or photographically derived conference
`arrangement.
`Moreover, I have also recognized that there is a need in
`the art to permit control of Video conferencing by partici
`pants in the conference or from a central control perspective
`upon the occurrence of certain events. The conferees should
`not be limited to a Selected, controlled Virtual conference
`context but should be free to change conference parameters
`and presentations to them. For example, conference partici
`pants may choose to not be seen or heard, to join, tempo
`rarily leave or permanently depart from the conference, to
`have other participants to listen to an audio presentation or
`to View a visual or audiovisual presentation or other event
`that may be anticipated by a participant to happen as the
`conference proceeds. Conference participants should be able
`
`RPX Exhibit 1007
`
`

`

`US 6,330,022 B1
`
`3
`to Selectively choose certain portions or all of a conference
`for participation or recording. During a conference, band
`width requirements for delivery may expand and contract
`depending on the Status of the conference at a given point in
`time and the virtual conference should adapt accordingly.
`According to the present invention, Video conferencing
`provides a real-time live image of conferees of a Video
`conference within a conferee Selected conference context. In
`one embodiment of the invention, Video conferencing appa
`ratus includes a video processor for 1) receiving a video
`Signal representing an image of each conferee, 2) receiving
`conference context Selection input representing the Selection
`of a preferred location and conference type, and 3) assigning
`to each conferee a position at a virtual conference including
`images of the conferees within the Selected conference
`location Such that a conferee may See all other conferees, but
`not their own image, within their personally Selected con
`text. The apparatus outputs a Video signal Such that when the
`Signal is displayed, a viewer will See images for all conferees
`according to the assigned positions within their Selected
`conference context. In addition and in accordance with prior
`art processes, 3D Sound may be provided for the Simulated
`conference of live conferee images, resources may be shared
`and displayed and conference tools controlled via pop-up
`interactive windows.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 provides an overall block diagram of apparatus of
`the present invention including a central conference bridge
`and individual conferee terminal apparatus.
`FIG.2 provides a representation of a band of Video signals
`comprising signals for N conferees in accordance with the
`present invention which is output by the conference bridge
`160 of FIG. 1.
`FIG. 3 provides a specific example of such a band for a
`five conferee (where N equals 5) video conference.
`FIG. 4 is a Schematic block diagram of Video conference
`bridge apparatus of the present invention.
`FIG. 5 is a typical video display seen by a conferee
`participant of a conference accomplished according to the
`principles of the present invention including, for example, a
`Virtual conference portion, a Visual presentation portion and
`an icon portion for actuating pop-up interactive windows.
`FIG. 6A is an example of a context table, for example,
`stored in memory of processor/server 170 or memory 441 of
`controller 440.
`FIG. 6B is an example of a conferee table, for example,
`stored in memory of processor/server 170 or memory 441 of
`controller 440.
`FIGS. 7A through 7D define events occurring at set-up
`and during the conference from the perspective of the
`switch/bridge to server communications (FIG. 7A), from the
`server to the Switch/bridge FIG. 7B), from the server to a
`browser (FIG. 7C), and events in the signal browser (FIG.
`7D).
`
`DETAILED DESCRIPTION
`Referring to FIG. 1, there is shown a schematic block
`diagram of a Video conferencing arrangement according to
`the present invention. To the left is found a first conference
`participant or conferee 100 of a video conference. Each
`participant in a conference according to the present inven
`tion is connected via a local area network, wide area
`network, public Switched telecommunications network, pri
`vate data network, Internet, intranet or other network 150 to
`
`4
`a conference bridge 160 and processor/server 170 of the
`present invention. Conference bridge 160 is called into play
`by processor/server 170. Conference bridge 160 is further
`described, by way of example, in FIG. 4 and its attendant
`description. Auxiliary processor/server 170 is a processor
`known in the art.
`Auxiliary processor/server 170 may be a 3B processor
`asSociated with a Switching center of a long distance carrier
`or a Server/router known in the Internet arts, for example,
`one manufactured by Cisco Systems. In particular, it is
`known in the public Switched telecommunications network
`to provide adjunct processors to toll or local offices which
`include Switching functionality for the provision of Special
`features Such as conferencing, international long distance
`dialing and call forwarding features. Also, the functionality
`of a processor 170 as described herein may be incorporated
`within a toll or local Switch Such as a #4ESS or #5ESS
`electronic Switch manufactured by Lucent Technologies,
`Inc. or other Switch of another manufacturer.
`There may be up to N conferees of a virtual conference
`where N is a number larger than two, Such as up to eight
`conferees, that may visually and aurally communicate with
`one another. Only four such conferees are shown in FIG. 1.
`For the four conferees of FIG. 1, conferee 100 might see an
`image portion 510 such as is shown in FIG. 5 where three
`other participants are Seated around an actual or Simulated
`round conference table. The round conference table is a
`“type' of conference context selected by the conferee
`including a conference location and type. FIG. 5 shows a
`round conference Setting with conferees Seated around a
`conference table which may be a simulated round confer
`ence table. Other conference locations and types may be
`input and selected for display by a conferee. The conference
`participant 100 at their terminal 101 may or may not see
`their own image among the other conferee images of a
`conference image portion 510 depending on his choice.
`According to the present invention, a conferee participant
`may select a context for a virtual conference, the context
`including location, type, and purpose, as will further be
`described herein. The conferee may do So by appropriate
`known data input which may include keyboard, mouse
`input, Voice activated actuation and other known data entry
`devices. In a preferred embodiment and referring to FIG. 5,
`a conferee may click with a mouse button on conference
`context icon 521 of icon screen portion 520 actuating a
`pop-up Screen. By clicking within the pop-up interactive
`Screen (not shown), the conferee may initiate a conference,
`update their preferences or input changes in real time during
`the conference as desired.
`In a Similar manner, the conferee may click on personal
`data icon 522 and pull up another pop-up interactive win
`dow. By clicking within this window, the conferee may
`change personal indicia about himself in real-time, and
`Signal, for example, the existence of a visual media presen
`tation for showing to other conferees. Also, the conferee
`may click on icon 523 for Selecting Screen layout, viewing
`choices and recording their choices. Thus, any conferee may
`establish conference Set-up parameters regarding themselves
`and the context of the conference in real time as will be
`described further in connection with a discussion of FIGS.
`4, 5, 6 and 7.
`Referring to FIG. 5, and within conference image 510, a
`3D like Simulated conference arrangement may be shown
`where the table is graphically or photographically derived
`and each conferee may be a graphical or photographic image
`of a talking head derived from a Single photograph of the
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`RPX Exhibit 1007
`
`

`

`S
`conferee. Each participant may see the same image of all
`participants at the conference or a view from that partici
`pant's perspective, excluding that participant's image.
`According to one aspect of the present invention, each
`conferee receives a display in which each displayed conferee
`is represented by a live Video image as if the conferee were
`present in a life-like conference Setting. The live conferee
`images are overlaid in real time with a photographic image
`of a conference table and the table, in turn, overlaid on a 180
`degree photographic image of a conference room Such as the
`White House Oval Office, whose image has been previously
`Stored in an archive for retrieval. The conferee will appear
`in their Selected context, their image being Superimposed in
`Selected placement within their Selected conference type and
`at their Selected location. The conferee may choose to See or
`not to See themselves. If he chooses not to See himself, he
`will only see the other conferees as he would see if he was
`at a live conference. The conferee may select a conference
`context including 1) the conferee's personal choice of con
`ference room location from a plurality of predetermined
`locations within an archive of locations, 2) a default location
`Selected by a chairperSon as is known from the prior art or
`3) design and Select their own conference room location.
`The Selected context further includes a Selected type of
`conference from choices including a round table conference,
`a Square table, a rectangular table, a poker table, a black-jack
`table, a jury table, a deposition like Setting, a U shape table
`arrangement or any other shape or Seating arrangement and
`table context type the user Selects or designs. The context
`type may be 1) photographically archived as predetermined
`context location data may be archived, 2) may be a dealt type
`Selected by a chairperSon or 3) may be graphically generated
`or designed by the user and arranged within the Selected
`location.
`Also, the conferee may select a purpose for the meeting.
`The purpose for the meeting may be a deposition and So the
`purpose may define certain tools that must be available Such
`as audio and/or video recording tools. The purpose may be
`training and So define different tools that must be present
`Such as a blackboard.
`In accordance with the invention, a conference context
`comprising location, type and purpose Selected by one
`conference may differ from that selected by another
`conferee, and each conferee will receive a display of their
`Selected context and Seating arrangement. To the extent
`necessary, a conferee may negotiate, or arranges with
`another conferee or all other conferees to achieve a mutually
`compatible conference context but Such negotiation is not
`required.
`A conferee participant may be provided with an intelligent
`communication device 101. Such an intelligent communi
`cation device may be a telephone, a personal computer or
`other terminal 101 comprising, for example, a camera,
`microphone and display arrangement 101a, an audio speaker
`arrangement 101b which may be stereo and a network
`interface, for example, a modulator/demodulator or other
`network interface 101c. In one embodiment, the camera of
`the camera, microphone and display arrangement 101 a may
`be conveniently located at the top of the display and the
`microphone to one side of the display. Other arrangements
`may be used to equivalent advantage depending on the
`conference application. The conferee camera may be
`equipped with a fish-eye lens. The camera may receive and,
`with the assistance of a data processor of the intelligent
`communications device, transmit a compressed real-time
`Video image of a conference participant and their environ
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,330,022 B1
`
`6
`ment. The terminal 101 may be equipped with a computer
`mouse, keyboard, the data processor and memory as is well
`known in the art (not shown). The conferee may use
`computer input devices Such as the computer mouse to input
`Selections to the processor including Selections of Video
`conference context that may be different from a context
`Selected at another intelligent communication device. The
`conferee may click on various choices of context location
`and type, other choices of pop-up windows and click in a
`known manner to pan, tilt, rotate and Zoom within the
`conference context and Setting or Select among presentations
`by participants. Processor/server 170 monitors and scans the
`mouse or other conferee inputs for controlling the Virtual
`conference context prior to or during the conference.
`Communication link 102, which connects terminal 101
`with network 150, may be any conventional linking facility
`known in the art Such as a twisted pair, fiber optic facility,
`a hybrid fiber optic/coaxial facility, a Satellite or radio
`frequency Such as cellular wireleSS link or other link to
`network 150. Typically, communications link 102 may be an
`integrated Services digital network (ISDN) link operating at
`data Speeds less than or equal to 64 kilobits per Second or an
`asynchronous digital Subscriberline (ADSL) link at approxi
`mately one megabit per Second or a high Speed cable modem
`link. However, as technology improves, the bandwidth and
`data Speeds may increase even to the point of meeting or
`exceeding one gigabit per Second. Also, terminal 101 may
`incorporate Video and/or audio signal compression and
`decompression circuitry and algorithms, not shown for use
`in transmitting audio and video over link 102. One known
`compression Standard is proposed through the motion pic
`tures electroncis group (MPEG); other Standard compres
`Sion algorithms may be Suitably used to advantage as well.
`Terminal 101 may include decompression circuitry and
`algorithms (not shown) which are used upon receipt of a
`compressed Video/audio Signal to generate an uncompressed
`Signal for display. Alternatively, compression and decom
`pression algorithms and any other required algorithms may
`be downloaded to the terminal 101 from processor/server
`170 prior to or during the conference.
`Camera C of apparatuS 101 a may capture a live, real-time
`video image of conferee 100. Once a conferee is connected
`to bridge 160 via server 170 of the present invention, a video
`Signal, preferably, a compressed digital Video bit Stream
`signal, is supplied over link 102 via network 150 to bridge
`160. The signal is processed at bridge 160 according to the
`present invention. Unless conferee 100 chooses not to be
`Seen, the other conference participants may elect whether or
`not to see conferee 100.
`Central bridge 160 may be a dial-up or otherwise addres
`Sable bridge that is preferably a shared resource of the
`network 150. Central bridge 160 is actuated by auxiliary
`processor/server 170 at conference time after the special
`Service request for a video conference is received at Server
`170 from a conferee 100. In one embodiment of the
`invention, the conference bridge 160 may be a shared
`resource of a conference Service that conferees may access
`whenever they wish. As a first conferee connects to the
`bridge, conferee 100 connects to bridge 160 first and then
`conferee 110, a Second conferee connects to the bridge, the
`conferee accepts an assigned Seat or assigns themselves a
`Seat at a conference table, for example, a round table, upon
`negotiations with others present at the table. The conference
`location and type including the table arrangement may be
`Simulated and displayed as a virtual conference table in a
`virtual meeting room (FIG. 5). Conferee 3 may be conferee
`120 and conferee 4 may be conferee 130. The next person to
`
`RPX Exhibit 1007
`
`

`

`7
`join the conference may assign themselves the next Seat
`around the conference table, and the conference table grow
`in size accordingly. Also, conferees may change Seats as they
`may mutually agree to do So.
`Conference Seating Arrangement
`Now the Virtual conference Seating arrangement and
`automatic composition of a Video Stream according to one
`aspect of the present invention will be described. The
`conferees define a conference Seating arrangement by
`assigning themselves to positions at the conference.
`Furthermore, the conferees define the virtual conference
`they see by Selecting a conference context including a
`conference location, type and purpose among other vari
`ables.
`It is assumed for the purpose of this discussion that the
`conference Seating arrangement has been agreed upon by the
`participants. Continuing with an example of four conferees
`assigned to positions at a conference, the conference bridge
`may receive four Separate Video Signals, one Video signal for
`each conferee over respective links 102, 112, 122 and 132.
`In the more generic example, the Nth conferee is positioned
`at the foremost right of conferee N-1. Referring to FIG. 2,
`a single video signal is developed from N Video signals at
`bridge 160 by video processing circuitry. The video pro
`cessing circuitry operates to concatenate a Single video
`Signal comprising all N Video conferee's Signals in assigned
`Sequence if a particular conferee So Selects or all N-1
`(excluding the particular conferee) if the conferee So Selects.
`At each terminal 101, 111, 121, 131, either signal may be
`centrally processed to create a conferee view (FIG. 4) seen
`by a particular conferee. A view generated from the Signal is
`displayed as conference image 520 (FIG. 5).
`Video signals, no matter how formed, are composed of a
`Series of frames representing the State of a captured image at
`a point in time. FIG. 2 intends to represent Such a single
`frame signal that is the combined representation of N or N-1
`conferee images, where N is 5. FIG. 2 represents a circular
`or round placement of conferees but the present invention is
`not So limited. A conference type may be Selected from
`many choices including round, Square, U-shaped,
`deposition-style, or any design the conferee Selects. When a
`display for a particular conferee is generated at their
`terminal, the display may be generated from a Signal rep
`resenting all other Segments of the Signal but the conferee's
`Signal. In the Selected option where all N signals are formed
`in a band for transmission to a conferee, any number of
`known processes may be used to derive a conferee video
`Signal showing N-1 conferees. One Such process, for
`example, is to compare the conferee's Signal with the
`received band Signal comprising all conferee signals and to
`generate the remaining Signal for display as the Signals
`clockwise and counterclockwise (referring to FIG. 2) of the
`band frame. Conferee 2 will then have a viewpoint generated
`from position 201 of the frame. That is, conferee 1 and N
`will appear to their right and conferees 3 and 4 to their left.
`The image the conferee may see displayed on his terminal
`may show N-1 participants excluding himself.
`An algorithm for formulating a band Signal including
`image Signal portions representing N-1 conferees at confer
`ence bridge 160 involves arranging the assigned conference
`Seating arrangement as a concatenated band of N Video
`signals: /1/2/3/4/5/.../N/ and so on. Let a conferee C be a
`conferee where 1 is less than or equal to C and C is less than
`or equal to N. Then the conference bridge 160 creates two