(12) United States Patent
`Bordonaro et al.
`
`USOO6868094B1
`(10) Patent No.:
`US 6,868,094 B1
`(45) Date of Patent:
`Mar. 15, 2005
`
`(54) METHOD AND APPARATUS FOR
`MEASURING NETWORK DATA PACKET
`DELAY, JITTER AND LOSS
`
`(75) Inventors: Frank G. Bordonaro, Los Gatos, CA
`(US); Kui Zhang, Cupertino, CA (US);
`Satyanarayana Rao Raparla, San
`Jose, CA (US)
`(73) Assignee: Cisco Technology, Inc., San Jose, CA
`(US)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(*) Notice:
`
`(21) Appl. No.: 09/434,845
`(22) Filed:
`Nov. 4, 1999
`e T9
`Related U.S. Application Data
`(63) Continuation-in-part of application No. 09/346,080, filed on
`Jul. 1, 1999.
`(51) Int. Cl.................................................... H04J 3/06
`(52) U.S. Cl. ....................................................... 370/516
`(58) Field of Search ................................. 370/516, 359,
`370/471-477, 352-358, 330, 498, 509,
`452, 276, 238, 435, 252, 253; 35.9/137,
`158, 165; 455/13.4; 342/45; 714/712; 709/29
`References Cited
`
`(56)
`
`U.S. PATENT DOCUMENTS
`5,101,208 A
`3/1992 Parker et al. ................. 342/45
`
`5,450,394 A * 9/1995 Gruber et al. .............. 370/253
`6.259,677 B1 * 7/2001 Jain
`... 370/252
`6,345,288 B1
`2/2002 Reed et al. ................. 709/201
`6,363,056 B1 * 3/2002 Beigi et al. ................. 370/252
`6,397,359 B1 * 5/2002 Chandra et al. ............ 714/712
`6,430,394 B1 * 8/2002 Boyden ..................... 455/13.4
`
`* cited by examiner
`
`Primary Examiner-Chi Pham
`ASSistant Examiner-Prenell Jones
`(74) Attorney, Agent, or Firm Marger Johnson &
`McCollom, PC
`(57)
`
`ABSTRACT
`
`Internet protocol (IP) performance monitoring method and
`apparatus generate a timing probe data to be sent over the
`network, the timing probe data packet containing at least a
`send time of day (STOD) stamp for a sender of the timing
`probe data packet. The timing probe data packet is sent over
`the network from the Sender to a receiver. The timing probe
`data packet contents including at least the STOD Stamp are
`analyzed as a performance measure of the network. After
`Sending and before analyzing, data including at least a
`receive time of day (RTOD) stamp is written into the probe
`data packet at the receiver, and probe data packet is echoed
`by the receiver. In this way, the probe packet Sender per
`forms the analysis based upon the STOD stamp and the
`RTOD stamp.
`
`28 Claims, 4 Drawing Sheets
`
`
`
`Sender
`transmits probe
`packet
`containing
`Seridsen,
`Send time
`stoo);
`reserving
`Recuseqno.
`Recime
`Riton)
`
`Resorder
`receives probe
`packet;
`incretents
`receive cune
`echoes notified
`probe packet
`her
`containing
`recwtime
`(RToo),
`Recysen
`from receive
`counter,
`Deltairneir
`reserved fields
`
`
`
`
`
`records jitter
`betwee:
`consecutive
`probe packets
`
`Hewlett Packard Enterprise Co. Ex. 1008, Page 1 of 16
`Hewlett Packard Enterprise Co. v. Intellectual Ventures II LLC
`IPR2021-01378
`
`

`

`U.S. Patent
`
`Mar. 15, 2005
`
`Sheet 1 of 4
`
`US 6,868,094 B1
`
`
`
`FG. 1A
`
`Responder
`
`Hewlett Packard Enterprise Co. Ex. 1008, Page 2 of 16
`Hewlett Packard Enterprise Co. v. Intellectual Ventures II LLC
`IPR2021-01378
`
`

`

`U.S. Patent
`
`Mar. 15, 2005
`
`Sheet 2 of 4
`
`US 6,868,094 B1
`
`(a)
`
`Pa
`
`1 Oms
`Pb
`
`1 Oms (16a)
`
`9ms (16b)
`
`9ms
`
`1 Oms (16a)
`Pa
`O-m-b-
`
`(b)
`
`1 Ons
`12ms (16b)
`Pb
`O-Hamm-mm-o-
`
`12ms
`
`1 Oms (16a)
`Pa
`O-o-
`
`(C)
`
`1 Oms
`22ms (16b)
`Pb
`O-H-H-O-
`
`22ms
`
`FIG. 2
`
`Hewlett Packard Enterprise Co. Ex. 1008, Page 3 of 16
`Hewlett Packard Enterprise Co. v. Intellectual Ventures II LLC
`IPR2021-01378
`
`

`

`U.S. Patent
`
`Mar. 15, 2005
`
`Sheet 3 of 4
`
`US 6,868,094 B1
`
`Round-trip
`Latency
`
`One-way
`Latency
`
`Inter-packet
`Jitter
`
`Packet
`loss (SD)
`
`Packet
`loss (DS)
`
`18a
`
`YV
`
`58
`
`56
`
`AN
`
`
`
`TOD cloc
`
`
`
`
`
`36/N
`Subtractor
`
`Subtractor
`
`Subtractor
`
`Subtractor
`
`ENA, Vil
`INNAN
`
`
`
`
`
`Probe
`type
`
`Delta
`time :
`
`Send time
`(STOD)
`
`Recwtime
`(RTOD)
`
`s
`
`-
`C-
`no
`
`-
`C
`no
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FG. 3
`
`Hewlett Packard Enterprise Co. Ex. 1008, Page 4 of 16
`Hewlett Packard Enterprise Co. v. Intellectual Ventures II LLC
`IPR2021-01378
`
`

`

`U.S. Patent
`
`Mar. 15, 2005
`
`Sheet 4 of 4
`
`US 6,868,094 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Sender
`transmits probe
`packet
`containing
`Send Segno,
`Send time
`(STOD);
`reserving
`Recy seqLino,
`Recw time
`(RTOD)
`
`Responder
`receives probe
`packet;
`increments
`receive COUnter,
`echoes modified
`probe packet
`further
`containing
`Reev time
`(RTOD),
`Recy Sedno
`(from receive
`counter),
`Delta time in
`reserved fields
`
`Sender receives
`and stores
`echoed probe
`packet;
`calculates and
`optionally
`records one-way
`and/or round-trip
`talencies:
`determines loss
`
`Ele
`p
`
`
`
`
`
`106
`
`Sender
`calculates and
`optionally
`records jitter
`between
`consecutive
`probe packets
`
`FG. 4
`
`Hewlett Packard Enterprise Co. Ex. 1008, Page 5 of 16
`Hewlett Packard Enterprise Co. v. Intellectual Ventures II LLC
`IPR2021-01378
`
`

`

`US 6,868,094 B1
`
`1
`METHOD AND APPARATUS FOR
`MEASURING NETWORK DATA PACKET
`DELAY, JITTER AND LOSS
`
`RELATED APPLICATIONS
`This patent application is related as a continuation-in-part
`of co-pending U.S. patent application Ser. No. 09/346,080
`entitled A PROTOCOL TO COORDINATE NETWORK
`END POINTS TO MEASURE NETWORK LATENCY,
`filed Jul. 1, 1999, the disclosure of which is incorporated
`herein by reference.
`BACKGROUND OF THE INVENTION
`This invention relates to Internet protocol (IP) network
`Systems in which Voice or other time-Sensitive data are sent
`in packets from a Server to a client or Vice versa, and more
`Specifically to method and apparatus for measuring data
`packet delay, jitter and loSS in Such Systems.
`Network applications Such as virtual private network
`(VPN), voice over IP (VoIP) or voice over frame relay
`(VOFR) network may require an IP service provider (ISP) to
`monitor data packet loSS in a network and/or inter-packet
`jitter (inter-packet in latency in arrival time). Such may be
`required as a part of a Service agreement between an ISP and
`a user/client. The Service provider needs a way to measure
`data packet jitter and loSS and the users/clients need a way
`to monitor data packet jitter and loSS to ensure they are
`getting the level of service the ISP agreed to provide.
`The above-referenced A PROTOCOL TO COORDI
`NATE NETWORK END POINTS TO MEASURE NET
`WORKLATENCY patent application, which is commonly
`owned by the assignee Cisco Technology, Inc., describes a
`network endpoints coordination protocol (NECP) that
`claims utility in measuring network latency between net
`work endpoints.
`
`15
`
`25
`
`35
`
`SUMMARY OF THE INVENTION
`A method of monitoring performance of an Internet
`protocol (IP) network is described. The method includes
`generating a timing probe data packet to be sent over the
`network, the timing probe data packet containing at least a
`send time of day (STOD) stamp for a sender of the timing
`probe data packet. The method further includes Sending the
`timing probe data packet over the network from the Sender
`to a receiver. Finally, the method further includes analyzing
`the timing probe data packet contents including at least the
`STOD stamp as a performance measure of the network.
`Between the Sending and the analyzing, the method includes
`Writing, by the receiver, into the timing probe data packet,
`data including at least a receive time of day (RTOD) stamp,
`and echoing the timing probe data packet by the receiver
`thereof. Thus, the Sender of the timing probe data packet
`performs the analysis based upon the STOD stamp and the
`RTOD stamp.
`Preferably, the generating is performed in Such manner
`that the timing probe data packet further contains a Send
`Sequence Stamp, wherein the writing further includes a
`receive Sequence Stamp, and wherein the analysis is based
`further upon the Send Sequence Stamp and the receive
`Sequence Stamp. The analysis may include first calculating
`the difference between the STOD stamp and the RTOD
`Stamp as a latency performance measure of the network.
`Most preferably, the generating, Sending and analyzing are
`repeated for at least two Successive ones of Such timing
`probe data packets.
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`The analysis then includes three further calculations.
`First, the difference between the STOD stamp and the RTOD
`Stamp for a first one of the Successive ones of Such timing
`probe data packets is calculated. Second, the difference
`between the STOD stamp and the RTOD stamp for a second
`one of the Successive ones of Such timing probe data packets
`is calculated. Finally, the difference between the first and
`Second calculated differences is calculated as an inter-packet
`jitter performance measure of the network.
`The foregoing and other objects, features and advantages
`of the invention will become more readily apparent from the
`following detailed description of a preferred embodiment
`which proceeds with reference to the drawings.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIGS. 1A and 1B represent a system block diagram of an
`Internet protocol (IP) network featuring the data packet jitter
`and loSS measurement apparatus according to the invention
`at slightly different moments of time.
`FIG. 2 is a Schematic diagram illustrating three different
`hypothetical packet-timing Scenarios.
`FIG. 3 is a detailed block diagram of the measurement
`apparatus in accordance with the invention.
`FIG. 4 is a flow chart illustrating the performance mea
`Surement method in accordance with the invention.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENT
`In a network Such as a VoIP or VoFR network, voice is
`digitized and packetized for transmission over the network
`in accordance with what will be referred to herein as a
`datagram-based protocol. Under Such protocols, there is a
`potential for having timing-Sensitive digitized voice data
`packets routed variously between the Source and the desti
`nation. Packetization and differential routing of data packets
`in a accordance with a datagram-based protocol is beneficial
`in terms of optimizing use of bandwidth, but creates a risk
`that Voice data packets may arrive at the destination out of
`Sequence due to different routing path delays or latencies.
`Such out-of-Sequence arrival of Voice and other time
`Sensitive data packets represents a risk of data loSS.
`FIGS. 1A and 1B illustrate a network 10 including a
`multiple Voice Sources, e.g. telephones, 12a and multiple
`Voice destinations, e.g. telephones, 12b connected within the
`network. Either of telephones 12a or 12b is capable of being
`a Source or destination of Voice in a two-way conversation.
`A normal conversation is half-duplex, with one or the other
`of telephones 12a, 12b being a voice Source and with the
`complementary one of telephones 12a, 12b being a voice
`destination. Network 10 typically includes thousands or tens
`of thousands of lines with telephones Such as telephones
`12a, 12b connected in Such a conversation. Telephones 12a,
`12b typically are connected to network 10 via so-called
`Voice gatewayS 14a, 14b, which perform the digitization,
`packetization and optional compression of Voice signals that
`renders them network-compatible.
`Those of skill in the art know that network 10 typically
`includes hundreds of Such gateways 14a, 14b, with each
`gateway typically Serving hundreds or thousands of Such
`telephones 12a, 12b. Network 10 also typically includes a
`web of plural routes or paths 16 therethrough that represent
`alternative channels through which Voice or other time
`Sensitive data packets Such as multimedia information hav
`ing an audio component may be routed, as in the burgeoning
`VPN, VoIP or VoFR networks.
`
`Hewlett Packard Enterprise Co. Ex. 1008, Page 6 of 16
`Hewlett Packard Enterprise Co. v. Intellectual Ventures II LLC
`IPR2021-01378
`
`

`

`US 6,868,094 B1
`
`15
`
`35
`
`40
`
`25
`
`3
`The routes through network 10 will be understood to
`impose different latencies, or delays, upon the transmission
`timing of data packets traveling therethrough. Because dif
`ferent but related data packets, e.g. Successive data packets
`from the same Source, may be differently routed through the
`network, they typically may arrive at the infended destina
`tion at different times. Such latency will be understood to be
`caused largely by the amount of time data packets may
`reside temporarily in network nodes along the way as part of
`the normal routing from their Source and destination within
`network 10.
`Those of skill in the art will appreciate that routing within
`the network is performed by routing Software that keeps
`track of traffic on various data channels. The routing Soft
`ware then assigns bandwidth within Such channels. In this
`manner, the Software determines the routing of various data
`packets through the network to maximize use of network
`bandwidth while Serving as many customers, e.g. telephone
`callers, Internet users/client and Service providers, as poS
`sible. It is this desirable flexibility in routing that results
`inevitably in latencies through the network of variously
`routed data packets.
`Because the latencies among various data packets are not
`constant, Successive data packets from the same Source can
`arrive at the destination out of Sequence. Out-of-Sequence
`arrivals of data packets may be treated at the destination as
`representing data loSS, i.e. the later arrival of an earlier data
`packet may be deemed loSS of that late-arriving data packet.
`By Sending a sequence number within at least two Succes
`Sive dedicated probe data packets, the destination of the
`probe data packets can detect out-of-Sequence arrivals and
`thus can monitor the performance of the network by mea
`Suring data packet loSS.
`By time Stamping a dedicated probe data packet at the
`Source, latency through the network may be measured at the
`destination. By time Stamping a probe data packet at the
`Source and also at the destination, and then by echoing the
`probe data packet back to the Source, two-way latency
`through the network may be measured at the Source. By time
`Stamping Successive probe data packets, variance in network
`latencies as between the Successive probe data packets may
`be measured. Such variance will be referred to herein as data
`packet jitter.
`Three classes of performance metrics are possible. One
`45
`way metrics include measures of absolute latency for a data
`packet through the network, and require only time Stamping
`at the Source and time receipting at the destination. One-way
`metrics also may include measures of relative latency as
`among two or more data packets, with the same modest
`requirement. Two-way metrics include measures of absolute
`latency for a data packet through the network, and require
`time Stamping at the Source, time Stamping at the
`destination, echoing at the destination and a simple calcu
`lation at the Source. Two-way metricS also may include
`measures of relative latency as among two or more data
`packets, with the same modest requirement. Thus, data
`packet jitter may be understood to be detectable without
`echo, while data packet loSS may be understood to require
`echo for detection.
`In either case, the Overhead required to measure network
`performance is minimal. This is true even with the So-called
`active sampling technique that characterizes the invention,
`whereby dedicated test probe data packets are Sent and
`received over the network.
`There are many protocols and Services under which Voice
`data or other timing-Sensitive data may be conveyed within
`
`50
`
`55
`
`60
`
`65
`
`4
`a network architecture. The most common protocol is the
`user datagram protocol (UDP), which will be used herein for
`illustration of the preferred embodiment of the invention.
`UDP is believed to be most illustrative of the invention
`because it is the most prevalent protocol providing for the
`real-time conveyance in a network of multimedia, Voice and
`other time-Sensitive data Subject to data loSS and jitter. Those
`of skill in the art will appreciate that use of other protocols
`and Services in conjunction with the methods and apparatus
`described herein are contemplated as being within the Scope
`and Spirit of the invention.
`Referring still to FIG. 1A, the invention preferably takes
`the form of a Service assurance agent (SAA) 18, which may
`be seen to include components SAA sender 18a and SAA
`responder 18b. SAA 18 may be understood to be any agent
`coupled to or within a network 10 which performs the
`invented data latency and inter-packet jitter performance
`metrics. It will be appreciated that SAA 18 may be inte
`grated into the resident network operating System (OS) or
`may reside in a dedicated or Shared Server node of network
`10 such as voice gateways 14a, 14b. SAA 18 may be
`invoked upon command by the network OS or on demand by
`quality assurance perSons or customers. Preferably, it is
`invoked by the network OS periodically as a pro-active
`management and reporting tool.
`SAA sender 18a instructs SAA responder 18b to comply
`with the network metrics protocols described in connection
`with the NEPC application referenced above and the per
`direction and round-trip data packet delay, inter-packet jitter
`and loss protocol described and illustrated herein. SAA
`Sender 18a is so called because it is the initiator of network
`performance measurements in accordance with the inven
`tion. It will also be understood that voice gateways 14a, 14b
`and other servers involved in the invented network metrics
`may themselves contain dedicated SAA responder 18b Soft
`ware that responds to delay, jitter and loSS probes Sent by
`SAA Sender 18a Software.
`FIGS. 1A and 1B illustrate how jitter probe packets Pa
`and Pb may be sent out over network 10 and may be routed
`differently therethrough between sender 18a and responder
`18b. For example, jitter probes 20a and 20b (representing
`probe data packets Pa and Pb, respectively) are routed
`respectively along paths 16a and 16b through network 10, as
`shown in FIG. 1A, from sender 18a to responder 18b. Jitter
`probes 20a' and 20b' (representing echo data packets Pa'and
`Pb', respectively) are routed respectively along different
`return paths 16a' and 16b' through network 10, as shown in
`FIG. 1B, from responder 18b to sender 18a. In FIG. 1B,
`responder 18b has responded to sender 18a's probe by
`echoing nearly identical probes as were received by
`responder 18b back to sender 18a, as will be further
`explained by reference to FIGS. 3 and 4. Thus FIG. 1B may
`be seen to represent a moment of time slightly later, e.g. a
`fraction of a second, than that represented by FIG. 1A.
`Those of skill in the art will appreciate that a so-called
`jitter probe is defined in accordance with the invention.
`The UDP jitter probe generally designated 20 in FIGS. 1A
`and 1B will be described in detail herein. Other probes may,
`within the Spirit and Scope of the invention, be similarly
`defined to achieve at least one-way or round-trip data packet
`latency metrics as well as inter-packet jitter and loSS metrics.
`It will be appreciated that in Some cases, the accuracy of the
`metricS is affected by the load on the central processor units
`(CPUs) of the source router and destination web server.
`SAA 18 preferably resides on a server node and executes
`as Software, firmware or hardware, all within the Spirit and
`
`Hewlett Packard Enterprise Co. Ex. 1008, Page 7 of 16
`Hewlett Packard Enterprise Co. v. Intellectual Ventures II LLC
`IPR2021-01378
`
`

`

`US 6,868,094 B1
`
`15
`
`25
`
`35
`
`40
`
`S
`scope of the invention. Preferably, SAA 18 performs active
`performance assessment and assurance of the network to
`which it is connected, thereby to ensure customer Satisfac
`tion. In a way, SAA 18 acts as a router dedicated not to
`normal network routing of client requests to ISPs or of voice
`or other time-Sensitive data between telephone conversants,
`but dedicated instead to network performance assurance. Of
`course, those of skill in the art will appreciate that SAA18
`uses existing network channels and the above-described
`NECP to measure a) network data packet latency per
`direction, b) round-trip network data packet latency, c)
`inter-packet network jitter and d) data packet loss per
`direction. Those of skill also will appreciate that alternative
`protocols, within the Spirit and Scope of the invention, may
`be used.
`It may also be appreciated that there may be within
`network 10 what will be referred to herein as data jitter. Data
`jitter refers to inter-packet delay variance, i.e. Variations in
`transit time between a Source and a destination. This is
`because routers within network 10 typically route packetized
`data in accordance with traffic demand and channel capacity
`in an attempt to maximize bandwidth and to minimize
`response time. As a result, related data packets and even
`Successive data packets may be routed differently through
`network 10. This is illustrated in FIG. 1A by a web of routes
`16 within network 10, two typical outgoing paths (from
`sender to responder) 16a, 16b being highlighted by bold
`lines making intermediate Stops at different Switching nodes
`along the way. Paths 16a, 16b thus represent differential
`route timing within network 10, Since they pass through
`different numbers of Switches each typically imposing delay.
`It is further illustrated in FIG. 1B by a web of routes 16
`within network 10, two typical incoming or return paths
`(from responder to sender) 16a", 16b' also being highlighted
`by bold lines representing differential echo data packet
`timing.
`Also illustrated in FIGS. 1A and 1B is the fact that data
`packets that are related in Some way—e.g. data probes Pa
`and Pa" or data probes Pb and Pb' related generally as query
`and echo data-nevertheless may be routed differently within
`network 10. Thus, a number of timing variables are intro
`duced by the otherwise-beneficial discretionary routing of
`data within network 10.
`AS between Successive data packets, a first data packet
`may transit network 10 from source to destination (sender
`18a to responder 18b) in a first amount of elapsed time
`(represented in FIG. 1A by boldface highlighted route 16a).
`Such routing timing typically is measured in milliseconds. A
`Second data packet in the Succession of data packets may
`transit from Source to destination in a Second amount of
`elapsed time that is greater or Smaller than the first, but still
`typically measured in milliseconds. Such is illustrated in
`FIG. 1A by boldface highlighted route 16b having fewer
`interposed node Switches than route 16a and thus represent
`ing a Smaller elapsed time. If the route timing of the Second
`packet in Succession is slightly larger, then there is no
`out-of-Sequence receipt of the Second packet but there may
`be out-of-Sequence receipt of a third packet that arrives
`ahead of the delayed Second packet. Some delay of course
`is expected and may well represent acceptable network
`performance. From the example immediately above, delay
`variance may result in data packet loSS because the Sequen
`tial arrival at the destination as among Successive packets is
`different from the sequence in which they left the source.
`If the route timing of the Second packet in Succession is
`Smaller (as illustrated in FIG. 1A, where path 16b has fewer
`interposed Switch nodes than path 16.a.) then there risk of
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`data loSS Since the later-Sent packet arrives at the destination
`before the earlier-Sent packet. Again, loSS results from
`out-of-Sequence arrival of packets at the destination. Data
`loSS, as opposed to data delay, typically is defined by
`protocol. For example, respondent routing Software at the
`Voice destination times out Successive data packets that are
`Separated by more than a given duration, e.g. two Seconds,
`and treats the variance in transit time as a data loSS.
`Referring briefly to FIG. 2, three different packet-timing
`Scenarios, or cases, will be described. Suppose two probe
`data packets Pa, Pb are Sent 10 ms apart, as indicated to the
`left of three horizontal line pairs labeled (a), (b), (c) indi
`cating the three different cases. The two packets Pa, Pb
`experience different transit times in reaching the destination,
`as indicated above each line. The different transit times
`cause the two packets Pa, Pb to arrive at the destination with
`different delay variances, as indicated to the right of the three
`horizontal line pairs. In case (a), the packets arrive in
`Sequence. Network 10 response is good. The delay variance
`is 9 mS-10 ms=~1 ms, implying that variance is good. In
`case (b), the 12 mS-10 ms=+2 ms delay variance implies
`Some congestion or delay in network 10. Still, packets Pa, Pb
`arrive in sequence. In case (c), the 22 mS-10 ms =+12 ms
`delay variance is large and may cause packets to arrive out
`of Sequence. (For example, if a third packet PC were sent 10
`ms after Pb and arrived at the destination 10 ms later, then
`the destination might receive third packet PC before it
`receives Second packet Pb.)
`Because the transit, or route, timing varies among routes
`and even over the same route with time, there is Some
`variance in data packet arrival at the destination, or
`responder, address. Such variance in arrival time, which as
`illustrated above by reference to FIG. 2 may or may not
`represent data loSS, is called data packet jitter.
`The same transit timing variance as between Successive
`data packets through network 10 applies also to return paths
`16a' and 16b', wherein it may be seen from FIG.1B that path
`16a' has more interposed switch nodes than path 16b'. This
`differential packet routing Similarly will be understood to
`represent differential transit time for the two echo probe
`packets returned by responder 18b to sender 18a, Subject to
`the same latency, jitter and loSS metrics. Thus, the invention
`may be seen to be equally applicable to measuring network
`data packet delay, jitter and loSS in either direction.
`In accordance with the invention, data jitter and loSS are
`monitored in network 10 as a performance measure of the
`network. Data jitter and loSS are monitored in accordance
`with the invention by what will be referred to herein as
`active-Sampling method and apparatus. Under this invented
`monitoring protocol, one or more test, or So-called probe,
`data packets containing packet timing and Sequence infor
`mation are transmitted to a remote network address and a
`response of the network to Such transmission is analyzed and
`recorded. Such measurement of the network's performance
`may be used for internal benchmarking or external reporting,
`e.g. to network customerS Such as telephone Service
`providers, to improve Service or to document compliance
`with performance criteria.
`SAA 18 preferably takes the form of Software executing
`on one or more server nodes within network 10. SAA18 will
`be understood to have two components, in accordance with
`the invention. An SAA Sender 18a sends a dedicated net
`work probe packet for the purpose of network performance
`assurance. An SAA receiver, or responder, 18b receives the
`dedicated network probe packet sent by SAA sender 18a and
`responds in a predetermined manner. It will be seen that
`
`Hewlett Packard Enterprise Co. Ex. 1008, Page 8 of 16
`Hewlett Packard Enterprise Co. v. Intellectual Ventures II LLC
`IPR2021-01378
`
`

`

`7
`SAA responder 18b may perform one-way metrics, e.g.
`measuring one-way latency or jitter. It will also be seen that
`SAA responder 18b may simply echo the probe data packet
`back to SAA sender 18a whereby sender 18a may perform
`two-way, or round-trip, metrics.
`Referring now to FIG.3, an UDP jitter probe 20 is defined
`for use by SAA18 in various performance measurements in
`network 10. Jitter probe 20 allows the user to accurately
`measure jitter, packet loSS and the round trip delay. This
`allows customers to base Service level agreements on the
`results. The amount of time a probe packet spends in the
`Source and destination SAA routerS is not counted in the
`measurements. Accordingly, the measurement and reporting
`based thereon accurately reflect the performance of the
`network.
`To achieve the required accuracy and to provide the
`desired packet-loSS and inter-packet jitter metrics, the fol
`lowing fields are included in the structure of UDP jitter
`probe 20.
`1. Probe Type (2 bytes)
`Tells responder 18b what kind of probe this is. This may
`Simply be a numeric code representing the probe type, to
`distinguish among various types of probes constructed in
`accordance with the invention that may be concurrently in
`transit within a given network 10. Responder 18b reads the
`probe type field first, and interprets the remaining fields in
`accordance with an established protocol for the given probe
`type.
`2. Responder Processing Time delta time (2 bytes)
`Responder 18b places in this field the amount of time in
`milliseconds spent in the responder. This number will be
`Subtracted from the difference between the send time and
`receive time for the same packet. Thus, the internal proceSS
`ing time of the Software within responder 18b will not figure
`into a measure of network latency.
`3. Sender Timestamp PX send time (sender) (4 bytes)
`Sender 18a places a time stamp in this field whenever it
`Sends the packet. Also referred to herein as Send time of day
`(STOD).
`4. Receiver Timestamp PX recv time (responder) (4
`bytes)
`Responder 18b places a timestamp in this field upon
`receipt of the packet. Also referred to herein as receive time
`of day (RTOD).
`5. Send Sequence Number PX send seq no (2 bytes)
`This field is set by sender 18a and represents the number
`of packets the Sender has sent out thus far during this
`instance of jitter probe 20.
`6. Receive Sequence Number Px recV seq no (2 bytes)
`This field is set by responder 18b and represents the
`number of packets the responder has received during this
`instance of jitter probe 20.
`Accordingly, it will be appreciated that at least Sixteen
`bytes are needed in the UDP payload to carry the above
`fields. This is possible in conformity with typical data packet
`payloads traversing existing networkS 10. For example, a
`VoIP packet typically contains about thirty-two bytes in the
`UDP payload.
`Referring still to FIG. 3, SAA sender 118a and responder
`118b are illustrated respectively above and below probe 20.
`Sender 18a is capable of writing and reading the Probe type
`field, the use of this particular field forming no part of the
`present invention. Sender 18a reads the Delta time field
`which contains the elapsed process time, or processor
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,868,094 B1
`
`8
`overhead, of responder 18b in echoing the probe data packet
`20. This Delta time field is written into probe data packet
`20 by responder 118b via Process timer 22 before the probe
`packet is echoed back to sender 18a. Sender 18a also is
`capable of writing and reading the Send time field (STOD),
`although only the reading thereof is illustrated for the Sake
`of Simplicity.
`Sender 18a calculates a first difference between the
`Send time field (STOD) and the Delta time field in any
`Suitable manner. This first difference calculation is illus
`trated in FIG.3 as a subtractor 24, although those of skill in
`the art will appreciate that any Suitable means of determin
`ing the first difference is contemplated. Sender 18a further
`calculates a second difference between the first difference
`(representing the adjusted receipt time of the probe packet)
`and the present time of day (TOD) via a TOD clock 26. This
`Second difference calculation is illustrated in FIG. 3 as a
`subtractor 28, though again those of skill in the art will
`appreciate than any Suitable means for determining the
`second difference is contemplated. Those of skill in the art
`will appreciate that this Second difference represents the
`unburdened (i.e. adjusted So as not to include responder
`processing overhead) two-way or round-trip latency, or
`delay, of probe data packet 20 as it is routed through network
`10 from SAA sender 18a to SAA responder 18b and back to
`SAA sender 18a Thus, a first useful network performance
`measure is realized in accordance with the invention.
`Sender 18a also reads the Recv time field (RTOD) which
`contains the time of day probe packet 20 was received. This
`RecV t