(12) United States Patent
`Rosenberg
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7.636,805 B2
`Dec. 22, 2009
`
`USOO763 6805B2
`
`(54) METHOD AND APPARATUS FOR
`COMMUNICATING DATA BETWEEN TWO
`HOSTS
`
`75
`(75) Inventor: Aron Rosenberg, Berkeley, CA (US)
`
`(73) Assignee: Logitech Europe S.A. (CH)
`
`(*) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 277 days.
`
`(21) Appl. No.: 10/833,363
`
`(22) Filed:
`
`Apr. 28, 2004
`
`(65)
`
`Prior Publication Data
`US 2005/0086358 A1
`Apr. 21, 2005
`O
`O
`Related U.S. Application Data
`(60) Eyal application No. 60/512,667, filed on Oct.
`
`1 1/2003 Borsato et al.
`6,654,891 B1
`2/2004 Harris
`6,693,663 B1
`6,766,519 B1* 7/2004 Burriss et al. ............... T19,321
`6,904,040 B2
`6/2005 Salapura et al.
`6,973,484 B1
`12/2005 Singhal et al.
`2002fOO67405 A1
`6, 2002 McDiarmid
`2004/0015979 A1
`1/2004 Shen et al. .................. 718/107
`2004/0022222 A1
`2/2004 Clisham
`2004/0022,237 A1
`2/2004 Elliott et al.
`2004/01 17653 A1
`6/2004 Shapira et al.
`2004/01 19814 A1* 6/2004 Clisham et al. .......... 348, 14.08
`2004/O190489 A1* 9, 2004 Palaez et al. ................ 370,351
`2005, OO86289 A1
`4/2005 Rosenberg et al.
`OTHER PUBLICATIONS
`Minturn, et al.: Addressing TCP/IP Processing Challenges. Using the
`IA and IXP Processors; IntelTechnology Journal, vol.7. Issue 4. Nov.
`14, 2003; pp. 39-50.
`Scalable Networking: Network Protocol Offload—Introducing TCP
`Chimney; WinHEC 204 Version—Apr. 9, 2004; 31 pages.
`Transmission Control Protocol: Darpa Internet Program; Protocol
`Specification; RFC: 793; Sep. 1981; 84 pages.
`* cited by examiner
`Primary Examiner Eron J Sorrell
`y
`(74) Attorney, Agent, or Firm—Lando & Anastasi, LLP
`
`(57)
`
`ABSTRACT
`
`(51) Int. Cl.
`(2006.01)
`G06F 3/12
`(52) U.S. Cl. ............................. 710/72; 710/33; 710/65;
`7 1871: 718/107: 370/260
`(58) Field of Classification Search s
`s
`None A method for communicating video data between a first host
`See application file for complete search history
`and a second host. The method may comprise encoding a
`Video signal producing an encoded video signal at the first
`References Cited
`host. The method may also comprise transmitting, by the first
`host, the encoded video signal to the second host. The encod
`U.S. PATENT DOCUMENTS
`ing and transmitting may both be performed by a first single
`multimedia thread of execution associated with an operating
`system at the first host. Other embodiments are also dis
`closed.
`
`(56)
`
`5,790,180 A * 8/1998 Wild ....................... 348.14.11
`5.990,852 A 1 1/1999 Szamre
`6,209,021 B1
`3/2001 Ahimovic et al.
`6,219,706 B1
`4/2001 Fan et al.
`6,222,529 B1
`4/2001 Ouatu-Lascar et al. ...... 71.5/745
`
`12 Claims, 7 Drawing Sheets
`
`
`
`
`
`Thredd One:
`Data Receiving
`Operations and
`Data Processing
`Operations
`
`
`
`
`
`Thread wo:
`Data Packaging
`Operations and
`Dota Transmission
`Operations
`
`Page 1 of 16
`
`GOOGLE EXHIBIT 1014
`
`

`

`U.S. Patent
`
`Dec. 22, 2009
`
`Sheet 1 of 7
`
`US 7.636,805 B2
`
`f03
`
`
`
`102
`
`
`
`Interconnection
`Mechanism
`
`f04
`
`f06
`
`FIG. 1
`
`Page 2 of 16
`
`

`

`U.S. Patent
`
`Dec. 22, 2009
`
`Sheet 2 of 7
`
`US 7.636,805 B2
`
`20t
`
`
`
`2O2
`
`Storage Memory
`System
`
`110101
`
`To
`Processor
`
`FIG. 2
`
`Page 3 of 16
`
`

`

`U.S. Patent
`
`Dec. 22, 2009
`
`Sheet 3 of 7
`
`US 7,636,805 B2
`
`301
`
`50
`
`First Host
`Computer
`
`Second Host
`Computer
`
`504
`
`305
`
`3O2
`
`3O7
`
`306
`
`
`
`Directory
`Server
`
`FIG. 3
`
`Page 4 of 16
`
`

`

`U.S. Patent
`
`Dec. 22, 2009
`
`Sheet 4 of 7
`
`US 7.636,805 B2
`
`-4001
`
`Register
`
`Host Message to
`Directory Server
`
`Host Sends Keep
`Alive Message
`
`Directory Server
`Echos Message
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`40O3.
`
`Request Connection
`
`Establish Connection
`
`F/G, 4
`
`Page 5 of 16
`
`

`

`U.S. Patent
`
`Dec. 22, 2009
`
`Sheet 5 of 7
`
`US 7,636,805 B2
`
`
`
`Await Login;
`Then Login
`
`502
`
`
`
`Send contact Page
`and Trigger
`Parameters
`
`
`
`
`
`50.3
`
`Commence Keep
`Alive Process
`
`FIG. 5
`
`Page 6 of 16
`
`

`

`U.S. Patent
`
`Dec. 22, 2009
`
`Sheet 6 of 7
`
`US 7.636,805 B2
`
`Select Contact
`
`Preliminary Verifications
`
`Directory Server Computes Conference D
`
`603
`
`Pocket to First Host
`
`Send Discovery Start
`Pocket to Second Host
`
`605
`
`Second Host Responds
`
`
`
`6O 7
`
`
`
`Directory Server Computes Firewall States
`
`604
`
`606
`
`603
`
`609
`
`Send iP Address, Port
`And Conference D List
`to First Host
`
`Send IP Address, Port
`And Conference D List
`to Second Host
`
`600
`
`
`
`
`
`
`
`
`
`613
`
`First Host Sends
`Packets to Second
`Host
`
`
`
`
`
`Second Host Sends
`Pockets to First
`Host
`
`connection is established by
`Exchange of Packets
`
`
`
`
`
`FIG. 6
`
`Page 7 of 16
`
`

`

`U.S. Patent
`
`Dec. 22, 2009
`
`Sheet 7 of 7
`
`US 7,636,805 B2
`
`
`
`TO
`Comm
`Channel
`
`
`
`
`
`
`
`
`
`
`
`Thredd One:
`Data Receiving
`Operations and
`Data Processing
`Operations
`
`Thredd Two:
`Data Packaging
`Operations and
`Dato Transmission
`Operations
`
`FIG 7
`
`Page 8 of 16
`
`

`

`1.
`METHOD AND APPARATUS FOR
`COMMUNICATING DATA BETWEEN TWO
`HOSTS
`
`RELATED APPLICATIONS
`
`This application claims priority under 35 U.S.C. S 119(e)
`to U.S. Provisional Application Ser. No. 60/512,667 entitled
`“Method and Apparatus for Communicating Video Informa
`tion.” filed on Oct. 20, 2003, which is herein incorporated by
`reference in its entirety.
`
`BACKGROUND
`
`The invention relates generally to the transmission of real
`time audio data, especially and/or video information, and
`more particularly to systems and methods for video confer
`encing.
`There are many systems and techniques for transmitting
`video information especially real-time video information
`combined with corresponding audio or other information
`Such as document displays. Some effective conventional
`techniques involve special transmission and reception sys
`tems and require dedicated communication links to encode,
`transmit, receive and decode video or other information. The
`encoding, transmission, and decoding operations are gener
`ally resource intensive in terms of the processing (e.g.,
`memory, CPU speed) and transmission requirements (e.g.,
`communication link bandwidth) necessary to provide an
`adequate video presentation. However, Such special systems
`are generally expensive to own and operate and therefore are
`not available to an average consumer.
`Many commercial products including hardware and/or
`Software components have become available to the average
`consumer for transmitting video information over public net
`works, such as the Internet. These systems may be, for
`example, coupled with a personal computer for use over the
`Internet or other communication network. For example, a
`Video conferencing or video distribution system may be con
`figured to transmit video information over the Internet among
`a group of PCs. However, due to the substantial resource
`requirements necessary for transmitting such information,
`and the limited and/or unreliable resources available on pub
`lic networks, performance of Such systems generally fall
`short of expectations, and Such systems are rendered less
`usable than more expensive specialized systems.
`The quality of a combined audio-video communication as
`perceived by a user is highly correlated to both the overall
`latency of the communication and to the difference between
`the latency of the audio communication and the latency of the
`Video communication between the sending and receiving sys
`temS.
`For example, latency is commonly experienced during a
`typical cell phone call, in that during the conversation, there
`are delay periods due to latency that are periodically per
`ceived by the user. In such a scenario, the user feels like the
`call is not in real time, and this latency affects the actual and
`perceived quality of the cell phone connection. With respect
`to video transmission, users experience latency as time lag or
`jumpiness' between consecutive video frames. Such perfor
`mance is present, for example, in conventional Software video
`conferencing Solutions.
`Video quality may particularly suffer when transmitting
`data over communication networks such as the Internet. Due
`to bandwidth availability and latency, conventional video
`conferencing Solutions generally provide frame rates
`between 2 and 10 frames per second. For instance, the system
`
`2
`CUseeMe (available from First Virtual Communications,
`Redwood City, Calif.) allows video transmission between
`hosts using a video conference server provides a 2-3 frame
`per second video signal with a latency of 450 milliseconds
`between capture at one host and presentation at another host.
`Another Internet teleconferencing system, NetMeeting
`(available from the Microsoft Corporation, Redmond, Wash.)
`delivers video information at approximately 6-10 frames per
`second with a latency of approximately 230 milliseconds.
`Further, most of these conventional systems are unable to
`deliver an acceptable image size at a good quality. Conven
`tional systems that operate over public networks such as the
`Internet are capable of delivering, at most, 320x240 pixel
`video data at 6-10 fps. By contrast, video delivered at
`approximately 24-30 frames per second (theatrical motion
`pictures are shown at 24 fps, while television displays at 30
`fps) provides a perception to the viewer of full-motion video.
`Therefore, it would be beneficial to have a system that deliv
`ers a quality video signal without latency or jumpiness
`between Successive frames.
`Contributions to latency may include and are not limited to
`intermediate systems, network latency and latency due to
`processing. Some conventional systems use intermediate sys
`tems to handle video data transmitted between hosts. More
`specifically, video data from one host is transmitted to another
`host through an intermediate system. Intermediate systems
`may include various security and network components as
`well, including but not limited to firewalls, routers and others.
`The extra handling performed by these systems, which
`includes receiving, processing, buffering, and other steps at
`the intermediate system, adds latency to the transmission.
`In some conventional systems, there is significant network
`latency due to one or more components of the network con
`nection. This latency is due, for example, to additional net
`work transmissions through intermediate systems as dis
`cussed above, and latency in creating and establishing
`network connections. Further, there are other problems, in
`addition to latency, with establishing connections through
`firewalls and other secure networking systems, as discussed
`below.
`There are other contributing factors to latency at either or
`both of the receiving and transmitting hosts. For instance,
`latency is added at either the sending or receiving host due to
`over-handling of the video data. Excess buffering, thread-to
`thread copying of video data, and other factors contribute to
`this type of latency.
`Modern computer networks enable communication
`between computers in part by assigning each connected com
`puter an address (for example, an Internet Protocol (IP)
`address), and one or more ports through which communica
`tion may proceed between the assigned IP addresses. Once a
`logical connection between computers is established, various
`techniques assure the authenticity of the ongoing information
`transfers, for example assigning sequence numbers to packets
`forming part of an ongoing connection. Some network inter
`connectivity features and security systems such as firewalls,
`network address translation (NAT) features, and others con
`ventionally used, interfere with efficient, latency-free, real
`time transmission of high bandwidth information, such as
`video information. What is needed, therefore, is an improved
`method for communicating video information.
`When communication between a first host computer and a
`second host computer is desired, and the computers are con
`nected to each other through a communications network
`without an intervening firewall or an intervening device per
`forming NAT, either host may initiate the connection by sim
`ply sending a suitable message addressed to a Suitable port for
`
`US 7,636,805 B2
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Page 9 of 16
`
`

`

`US 7,636,805 B2
`
`25
`
`40
`
`55
`
`60
`
`3
`the message type at the address of the other host computer.
`Communication using a client/server architecture, or a peer
`to-peer architecture, or any other Suitable architecture can be
`initiated in this way, absent an intervening firewall or NAT
`device.
`Communication without an intervening firewall or NAT
`device Sometimes occurs when host computers are connected
`to a common local area network (LAN). Such as a corporate
`network or home network. More commonly, when the host 10
`computers are interconnected through a wide area network
`(WAN), such as the Internet, and sometimes in LAN configu
`ration, one or both host computers may be connected to the
`WAN through a firewall or NAT system. A firewall or NAT
`15
`system has the effect of partially or completely masking, from
`computer systems reachable through the WAN, the host com
`puters behind such firewall or NAT systems. This masking is
`performed by rejecting unexpected messages, i.e., messages
`sent to closed or incorrect addresses or ports or messages 20
`purporting to be part of an ongoing exchange, but having
`incorrect sequence numbers.
`To make a conventional feature such as worldwide web
`browsing work, several operations occur. Connections from a
`local host computer to a remote server are initiated by the
`local computer. If the local computer is behind a firewall, the
`local computerinitiates communication with a desired server,
`through the firewall. By initiating the communication through
`the firewall, the local computer instructs the firewall to allow 30
`certain types of communications back from the server for a
`certain period of time. When the server replies using a correct
`address, port and sequence number, the firewall recognizes
`the response as expected and passes the response on to the
`local computer. While the server may also be located behind 35
`a firewall, that firewall conventionally has a known port open
`to inbound traffic, so that certain types of contact with the
`server are permitted by the firewall.
`To allow a direct peer-to-peer connection, a port is conven
`tionally opened in the firewall of each participant that is
`connected to the network through a firewall, so that contact
`with any participant may be initiated by any other participant,
`however, this leaves each participant with at least one port of
`their firewall open and Vulnerable to security breaches. In 4s
`Such applications as video and/or teleconferencing through
`the Internet, where direct, peer-to-peer connections are desir
`able for the purpose of minimizing latency, it is highly desir
`able to minimize the security risk to the computers of partici
`pants, while also allowing any participant to initiate a 50
`connection with minimal effort.
`In some conventional systems, an event loop is used to
`process video information. This is generally in the form of a
`single thread (e.g., a thread of execution executed by a pro
`cessor) that executes in an infinite loop. The thread waits until
`an event happens, and when an event occurs, the thread acts
`upon the event. Generally, only a single event can be pro
`cessed at a time. Other threads can add events to the threads
`workload, but the other threads cannot actually handle these
`events. This event loading causes the thread to become over
`loaded, and therefore, a particular event (e.g., a video trans
`mission, encoding or decoding event) that needs to be pro
`cessed is delayed. A simple example describing this issue is a
`worker Scenario where a particular worker has multiple 65
`bosses, each of which generates work for the worker to per
`form. In performing a particular work task for one of the
`
`4
`bosses, tasks requested of other bosses become delayed and
`must wait for the task currently being performed to be com
`pleted.
`
`SUMMARY OF INVENTION
`
`According to one aspect of the invention, methods and
`apparatuses are provided that establish a low-latency connec
`tion between two hosts. In one embodiment of the present
`invention, Such a connection may be made over a public
`network such as the Internet. However, it should be realized
`that various aspects of the present invention may be used with
`any network type (e.g., within an enterprise network, virtual
`private network (VPN), etc.) or any combination of networks
`and network communication types (e.g., Ethernet), and the
`invention is not limited to any particular network or combi
`nation of networks. Moreover, although the invention will be
`explained in the context of video conferencing, aspects of the
`invention have wider data communication applicability.
`The human threshold for perceiving latency is approxi
`mately 100 m.sec. Therefore, by communicating a video sig
`nal at or below this threshold between two hosts increases the
`perceived quality of the transmission to a user viewing the
`Video signal.
`Firewalls pose a particular latency problem for transmit
`ting data by requiring reconnection of connections that have
`timed out. As previously explained, firewalls allow and main
`tain connections that are initiated from a secure side of the
`firewall. As also previously explained, reverse traffic from
`other hosts are allowed in if they can be attributed to an
`allowed connection initiated from the secure side. For secu
`rity reasons, firewalls delete allowed connections after a
`period of nonuse.
`In one aspect of the invention, a host operating on a secure
`side of a firewall is adapted to periodically transmit a packet
`to a directory server to maintain that port open to the firewall
`for receipt of packets from one or more other hosts.
`According to one aspect of the invention, it is realized that
`the lowest timeout value on commercially-available firewalls
`is 30 seconds. Therefore, periodic keep-alive messages are
`transmitted by the host to the directory server with a period
`shorter than the timeout to maintain the connection estab
`lished through the firewall.
`Port connections to a particular host may be registered with
`the directory server, and port connection information for the
`particular host can be communicated to another host for the
`purpose of establishing a connection between the two hosts.
`That is, video traffic is sent by the other host to the first host,
`located to the registered port at the firewall. Information
`stored in the directory server may include an IP address, a port
`number, and other security information expected by the fire
`wall. In this manner, the first host on the secure side of the
`firewall maintains a connection through which another host
`can pass video traffic. It is conversely true that the second host
`may also be located beyond a firewall, and may also register
`its port information with the directory server in order to
`facilitate the transfer of video information from the first host
`to the second host through the firewall associated with the
`second host. In this manner, a true host to host (or peer to peer)
`connection may be established between hosts. Because the
`connection does not involve an intermediate server in the
`ordinary transmission of data packets (e.g., video data, audio
`data, etc.) latency is reduced.
`A method for communicating data between at least a first
`host and a second host comprises: identifying, at a server,
`address information of the first host, to which the second host
`may communicate data through a network security system
`
`Page 10 of 16
`
`

`

`US 7,636,805 B2
`
`5
`
`10
`
`15
`
`5
`coupled between the first host and the server; and communi
`cating, from the second host to the first host, data using the
`address information of the first host. The method may further
`comprise: identifying, at the server, address information of
`the second host, to which the first host may communicate
`data, through a network security system coupled between the
`second host and the server; and communicating, from the first
`host to the second host, data using the address information of
`the second host. The second host may be adapted to perform
`the act of communicating without use of an intermediate
`server. The method may also further comprise communicat
`ing periodically, from the first host to the server, through the
`network security system, so as to maintain an open commu
`nication channel through the network security system to the
`first host at the address of the first host. The address informa
`tion of the first host may include an address and a port infor
`mation.
`Preferably, both video and audio information is transmitted
`in real or near-real time between two hosts such that a user of
`at least one of the hosts perceives a real-time communication,
`that is, latency should be reduced below 100 mS for each.
`According to various aspects of the present invention, one or
`more of the above methods may be performed to reduce
`latency between hosts, and the invention is not limited to any
`particular combination of methods. For instance, separate
`receive and transmit threads may be used independently of
`any network considerations and still provide a performance
`improvement in transmitting and processing a video signal.
`According to another aspect of the present invention, a
`system is provided that delivers full-motion video with a low
`latency to provide a near real-time perception by a user. In one
`aspect of the present invention, video is provided at near
`telephone (POTS) latency, and therefore allows the transmis
`sion of video information over a public network such as the
`Internet and transmission of audio information over a stan
`dard telephone connection. It is realized that latency of a
`telephone signal is approximately 60 m.sec. Therefore, by
`communicating video information between hosts near this
`latency, there is no perceived delay between the video and
`audio information, and therefore, the transmission of video
`may be easily added to an existing telephone connection.
`Flexibility is increased, as the telephone is more frequently
`used to perform communications (versus the time and effort
`to establish a video conference) and therefore, augmenting a
`standard telephone call with video is more likely to be per
`formed.
`
`6
`FIG. 7 is a threading diagram showing process flow and
`data flow according to aspects of an embodiment of the inven
`tion.
`
`DETAILED DESCRIPTION
`
`This invention is not limited in its application to the details
`of construction and the arrangement of components set forth
`in the following description or illustrated in the drawings. The
`invention is capable of other embodiments and of being prac
`ticed or of being carried out in various ways. Also, the phrase
`ology and terminology used herein is for the purpose of
`description and should not be regarded as limiting. The use of
`“including.” “comprising.” or “having.” “containing.
`“involving, and variations thereofherein, is meant to encom
`pass the items listed thereafter and equivalents thereofas well
`as additional items.
`Embodiments of aspects of the present invention form and
`maintain an open channel for communication between two or
`more systems (e.g., computer systems) coupled by a commu
`nication network. In one embodiment, one or more interme
`diate security devices (e.g., firewalls, NAT devices) are inter
`posed between the two or more systems. The open channel
`may be formed first by communicating with a directory Ser
`vice, and then establishing direct communication in which the
`directory service remains outside of the communication path
`of the channel.
`In conventional media transmission systems such as video
`conferencing and other systems, information is transmitted
`between computers by way of an intermediate server. This
`intermediate server receives information from one computer,
`buffers the information, and transmits the information to one
`or more other computers. According to one aspect of the
`present invention, a communication method is provided that
`allows peer-to-peer communication without an intermediate
`server system. A direct peer-to-peer communication reduces
`the amount of latency in communicating information (e.g.,
`Video, audio) information between a sending and receiving
`system, as the need for buffering information at another sys
`tem is eliminated, and multiple receive and transmits of the
`information (e.g., to/from the intermediate server) is no
`longer necessary. In one aspect of the present invention, it is
`realized that latency in communicating information between
`systems affects the perceived quality of the communication
`between systems (e.g., in a videoconferencing system), and it
`may be beneficial to minimize the amount of latency in com
`municating the information between systems. In one embodi
`ment, the directory service system maintains information
`relating to the two or more systems that allows any of the
`individual systems to establish one or more direct communi
`cation channels with another system that is registered with the
`directory service system.
`According to another aspect of the invention, messages are
`periodically sent to maintain the open connection through an
`intermediate security system (e.g., NAT device, firewall) such
`that a system located beyond the intermediate security system
`may receive peer-to-peer connection requests directly. By
`maintaining the connection through the security system,
`latency associated with creating a connection is reduced.
`Other aspects of the present invention relate to reducing
`latency of the communication between systems by modifying
`ways by which the sending and receiving systems process
`data. According to one aspect, it is realized that existing
`systems use inefficient programming methods for sending,
`receiving and processing data.
`According to aspects of embodiments of the invention
`shown in FIG. 7, implementing dual, single threads, one of
`
`25
`
`30
`
`35
`
`40
`
`45
`
`BRIEF DESCRIPTION OF DRAWINGS
`
`The accompanying drawings, are not intended to be drawn
`to Scale. In the drawings, each identical or nearly identical
`component that is illustrated in various figures is represented
`by a like numeral. For purposes of clarity, not every compo
`nent may be labeled in every drawing. In the drawings:
`FIG. 1 is a schematic block diagram of a computer system
`in which aspects of the invention may be embodied;
`FIG. 2 is a schematic block diagram of the storage system
`of the computer system of FIG. 1;
`FIG.3 is a schematic block diagram of a network topology
`in which aspects of embodiments of the invention may be
`realized:
`FIG. 4 is a flow diagram illustrating a method embodying
`aspects of embodiments of the invention;
`FIG. 5 is a flow diagram detailing the registration process
`of the method of FIG. 4;
`FIG. 6 is a flow diagram detailing the conference initiation
`process of the method of FIG. 4; and
`
`50
`
`55
`
`60
`
`65
`
`Page 11 of 16
`
`

`

`US 7,636,805 B2
`
`10
`
`15
`
`25
`
`30
`
`7
`which, block 701, handles receipt of data and one of which,
`block 702, handles transmission of data allows more efficient
`handling of video data. Receiving and processing of a packet,
`block 701, is made independent of sending video data, block
`702, and receiving and sending can occur concurrently
`because the threads, blocks 701 and 702, execute indepen
`dently. In some conventional systems, an event loop is used to
`process video information. The event loop takes the form of a
`single thread that executes in an infinite loop. The thread
`waits until an event occurs, and then the thread acts upon the
`event. In Such a design, only a single event can be processed
`at a time. Other threads can add events to the event loop
`threads workload, but the other threads cannot actually
`handle these events. This event loading causes the single
`event loop thread to become overloaded, and therefore, a
`particular event, e.g., a video transmission, encoding or
`decoding event that needs to be processed is delayed.
`In one embodiment, the system is dual-threaded, having a
`separate receiving thread, block 701, and transmission thread,
`block 702, however the receiving thread is single-threaded as
`to all receiving functionality and the transmission thread is
`single-threaded as to all transmission functionality. The send
`ing or transmission thread, of one embodiment encodes video
`data and the audio data and places them on the transmitter. In
`another embodiment, the sending thread may be a video
`multimedia thread that performs encoding and transmission.
`This thread may be, for example, a near real-time thread
`included in the OS (e.g., Windows NT or other operating
`system) that calls a function periodically corresponding to the
`encoding function. For example, in Windows, an encoding
`function may be provided to the OS by a callback at 30 fps.
`The function is then called every /30 second, and this single
`thread is used to perform the encoding and transmission of
`data. Because there are no thread-to-thread copies of data,
`latency is reduced. Also, because the thread is at a near-real
`time priority, the thread experiences less delay than a thread
`having lesser priority.
`Similarly, in one embodiment the receiving thread, which
`is separate from the transmitting thread, receives the video
`data and the audio data and decodes them for display and/or
`playback. Decoding may be performed by an operating sys
`tem callback, as well.
`According to embodiments of other aspects of the inven
`tion, a dual, single-thread Software design reduces processing
`latency, so that conventional, low-latency audio signal paths
`can be used to transmit and receive an audio signal while a
`corresponding video signal is simultaneously transmitted and
`received through a computer communication network having
`sufficiently low latency that the video signal and the audio
`signal are perceived as being synchronized.
`Embodiments of aspects of the present invention may be
`practiced on special purpose or general purpose computers, as
`now described.
`Various embodiments according to the invention may be
`55
`implemented on one or more computer systems. These com
`puter systems may be, for example, general-purpose comput
`ers such as those based on Intel PENTIUM-type processor,
`Motorola PowerPC, Sun UltraSPARC, Hewlett-Packard PA
`RISC processors, or any other type of processor. It should be
`60
`appreciated that one or more of any type computer system
`may be used to collect information and communicate over a
`network, according to various embodiments of the invention.
`Further, the exemplary video conferencing system may be
`located on a single computer or may be distributed among a
`plurality of computers attached by a communications net
`work.
`
`45
`
`8
`A general-purpose computer system according to one
`embodiment of the invention is configured to performany of
`the described video conferencing functions including but not
`limited to collecting video or audio information, transmitting
`the video or audio information, receiving the video or audio
`information or performing directory services. It should be
`appreciated that the system may perform other functions,
`including network communication, and the invention is not
`limited to having any particular function or set of functions.
`For example, various aspects of the invention may be
`implemented as specialized software executing in a general
`purpose computer system 100 such as that shown in FIG. 1.
`The computer system 100 may include a processor 103 con
`nected to one or more memory devices 104. Such as a disk
`drive, memory, or other device for storing data. Memory 104
`is typically used for storing programs and data during opera
`tion of the computer system 100. Components of computer
`system 100 may be coupled by an interconnection mecha
`nism 105, which may include one or more busses (e.g.,
`between components that are integrated within a same
`machine) and/or a network (e.g., between components that
`reside on separate discrete machines). The interconnection
`mechanism 105 enables communications (e.g., data, instruc
`tions) to be exchanged between system components of sys
`tem 100.
`Computer system 100 also includes one or more input
`devices 102, for example, a keyboard, mouse, trackball,
`microphone, touch screen, and one or more output devices
`101, for example, a printing device, display screen, speaker.
`In addition, computer system 100 may