Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 1 of 23 Page ID #:116
`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 1 of 23 Page ID #:116
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`EXHIBIT B
`
`EXHIBIT B
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 2 of 23 Page ID #:117
`Case8=20-CV-00529 ”we“ 1'2 Fil11111111llllflllllll’lllllllllIlfllI11|||||’lll|ifilllfilll‘llllllll
`
`US008259121B2
`
`(12) United States Patent
`US 8,259,121 B2
`(10) Patent No.:
`Law et al.
`
`(45) Date of Patent: Sep. 4, 2012
`
`SYSTEM AND METHOD FOR PROCESSING
`DATA USING A NETWORK
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6/1997 Law .............................. 711/157
`5,638,533 A *
`7/1997 Cismas ...........
`.. 348/441
`5,646,693 A *
`
`......
`.. 711/168
`5,822,779 A * 10/1998 Intrater et al.
`5,828,903 A * 10/1998 Sethuram et al.
`710/53
`
`.. 345/418
`5,841,439 A * 11/1998 Pose et al.
`...........
`
`.. 717/109
`5,987,246 A * 11/1999 Thomsen et al.
`.................. 711/219
`6,101,591 A *
`8/2000 Foster et al.
`6,412,061 B1
`6/2002 Dye
`6,429,902 B1 *
`8/2002 Har-Chen et al.
`6,489,953 B1 * 12/2002 Chen ...........
`6,525,738 B1 *
`2/2003 Devins et al.
`6,628,243 B1*
`9/2003 Lyons et al.
`6,636,214 B1 * 10/2003 Leather et al.
`
`............. 348/518
`.. 345/213
`345/553
`345/1.1
`................ 345/422
`
`
`
`(Continued)
`
`EP
`
`FOREIGN PATENT DOCUMENTS
`1091318 A2
`4/2001
`
`(Continued)
`OTHER PUBLICATIONS
`
`European Patent Office, Communication with European Search
`Report, in Application No. 030239685, dated Oct. 1, 2010, pp. 1-3.
`
`Primary Examiner 7 Hoang-Vu A Nguyen-Ba
`(74) Attorney, Agent,
`or Firm 7 Thomas,
`Horstemeyer & Risley LLP.
`
`Kayden,
`
`ABSTRACT
`(57)
`Systems and methods are disclosed for video processing
`modules. More specifically a network is disclosed for pro-
`cessing data. The network comprises a register DMA control-
`ler adapted to support register access and at least one node
`adapted to the data. At least one link communicates with the
`node, and is adapted to transmit data and at least one network
`module communicates with at least the link, and is adapted to
`route data to at least the link.
`
`(54)
`
`(75)
`
`Inventors: Patrick Law, Milpitas, CA (US);
`Darren Neuman, San Jose, CA (US);
`David Baer, San Jose, CA (US)
`
`(73)
`
`Assignee: Broadcom Corporation, Irvine, CA
`0J3)
`
`(*)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1509 days.
`
`(21)
`
`Appl. No.: 10/314,525
`
`(22)
`
`Filed:
`
`Dec. 9, 2002
`
`(65)
`
`Prior Publication Data
`
`US 2004/0078418 A1
`
`Apr. 22, 2004
`
`Related US. Application Data
`
`(60)
`
`Provisional application No. 60/420,151, filed on Oct.
`22, 2002.
`
`Int. Cl.
`
`(51)
`
`(2006.01)
`G06T1/20
`(2006.01)
`G06F 12/02
`(2006.01)
`G06F 13/28
`(2006.01)
`G06F 3/00
`(2006.01)
`G06K 9/54
`US. Cl.
`......... 345/506; 345/567; 382/303; 382/304
`Field of Classification Search .................. 345/505,
`345/501, 502, 506, 567; 382/303, 304; 710/22,
`7 1 0/29
`
`(52)
`(58)
`
`See application file for complete search history.
`
`36 Claims, 11 Drawing Sheets
`
`(.0[\JN
`
`/918A
`
`\ an19131631"\
`anKJOUJBUJ
`
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 3 of 23 Page ID #:118
`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 3 of 23 Page ID #:118
`
`US 8,259,121 B2
`
`Page 2
`
`US. PATENT DOCUMENTS
`
`3/2004 MacInnisetal.
`6,700,588 131*
`............. 345/629
`6,801,591 131* 10/2004 Frencken .........
`375/373
`
`6,919,896 B2*
`7/2005 Sasakiet al.
`.....
`345/505
`
`7,034,828 131*
`4/2006 Drebin etal.
`345/426
`7,054,867 B2*
`5/2006 Bosleyetal.
`........ 707/10
`
`
`......
`7,218,676 B2*
`5/2007 Kono etal.
`. 375/240.25
`7,230,651 B2*
`6/2007 Schoneretal.
`............... 348/500
`2002/0066007 A1 *
`5/2002 Wise et al.
`.................... 712/300
`
`2003/0080963 A1*
`5/2003 Van Hook etal.
`............ 345/501
`*
`.
`
`gggg/gfi‘jfigg 2% $388; £311}: a"; """
`' """"""""""
`g
`FOREIGN PATENT DOCUMENTS
`
`315/23
`
`3/1999
`W099/13637 A2
`W0
`W099/52277 A1 * 10/1999
`W0
`WO'01/22736
`”001
`W0
`* Cited by examiner
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 4 of 23 Page ID #:119
`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 4 of 23 Page ID #:119
`
`US. Patent
`
`Sep. 4, 2012
`
`Sheet 1 of 11
`
`US 8,259,121 132
`
`/A\-110
`
`.¢/\120
`/
`30.000fps
`
`
` 11
`
`
`
`Recovery
`
`
`Data
`
`Processing
`
`
`
`30.001fps
`
`299991115
`
`Fig. 1
`
`~\21o
`
`
`
`218 fl, v/
`
`
`"Data oncessnng"
`
`All decodlng, capture.
`
`
`
`playback, graphics, etc 4“\
`
`runs from a fixed clock
`
`
`Data Is supplied to the
`
`output at the rate it IS
`STR Recovery
`requested
`
`
`7
`
`214
`
`(212
`
`data request’
`
`
`
`4—~\\
`
`\
`
`222
`
`data request'
`
`
`
`data IS output at
`a fixed clock rate
`
`R
`.. A data rate is fed back
`
`
`to rate manager
`L215
`
`
`1
`
`—216
`
`Fig. 2
`
`\ 220
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 5 of 23 Page ID #:120
`2
`,m
`
`8P
`
`
`
`m4,m.9;wW\8m
`
`
`
`
`
`
`
`
`
`
`
`
`0.m72aL
`_tIwnm5
`e.NNm%Sl\CU.com
`
`
`#.1mH
`
`5Vivamud\._m__o:coo
`mmA<55555%
`
`ew,lw2mgP8.,43m3milfl|l
`
`O2
`
`mBv.3
`
`n0e2
`
`UHF9.?
`
`/23t
`
`mmKeB0mONV
`wm3082>A|gT:.82>MSPF\we.
`
`>
`
`
`
`
`
`
`
`
`
`
`
`
`9¢IIII.II2053m05m5082>xhoEmz29:22hwuoomo
`
`a+
`
`
`
`11!!E0®E>pmom.
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 6 of 23 Page ID #:121
`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 6 of 23 Page ID #:121
`
`US. Patent
`
`Sep. 4, 2012
`
`Sheet 3 of 11
`
`US 8,259,121 132
`
`
`VideoOut
`
`
`51BB
`
`NetworkNodeM51GB
`
` [500
`
`Fig.5
`
`
`
`
`
`MemoryBus Q2
`VideoBus 514
`
`
`516A Node
`
` RegisterDMA Controller m
`
`
`
`RegisterBus 512
`
`
`
`

`

`cU.
`
`
`
` O.tp.“m0t2a8Pe.
`
`mS8o\\
`
`edhllllllllMS_ama.00$:quF_“mo:
`9850me
`wa85_a
`
`
`GUOZ0OH.n_<mmm2onmore55%mmaagate;mm..w:uO|.mm|u_|>lu
`
`
`IIIIIIIIJC6.1Germvmo_m&282«82Emmmflfi
`
`
`x5282muozfoimzwuozboEwEm4,mare
`
`_H_.=OI8U_>
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`V
`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 7 of 23 Page ID #:122
`25O
`
`22mB”.2”.1mn,
`
`B0/4
`
`5:95:00mm<505&3”.wmm
`
`P8S
`
`e9.wx,mE
`
`Pa
`,m«8/17Lmgmgmam$660M
`BUN5
`
`
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 8 of 23 Page ID #:123
`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 8 of 23 Page ID #:123
`
`US. Patent
`
`Sep. 4, 2012
`
`Sheet 5 of 11
`
`US 8,259,121 132
`
`7121
`
`RBUS
`
`MBUS
`
`(:22
`
`730
`728
`
`Register DMA
`
`Controller
`
`7—1O‘DMA Triggers
`
`
`732
`
`714>
`
`VBUS
`
`734) (735
`
`Fig. 7
`
`818A
`
`8188
`
`,\
`818
`818"\
`links
`
`
`RBUS (optional) <- —- — —
`
`aux (optional) <— - - - —
`
`\
`links
`
`870/4
`
`818
`
`f\‘_
`IInks
`
`MBUS (optional) 4— - — — —
`
`aux (optional) <— — — - —
`
`
`\
`Entry Node
`
`84%
`
`
`818
`
`links
`
`818’\
`
`
`Exit Node
`RBUS (optional)<—————
`816B
`
`812d
`
`aux (optional) <— — — — —
`870J
`
`dedicated output
`
`Fig. 8
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 9 of 23 Page ID #:124
`2
`D.
`,m
`9
`
`O1mnmmmaemam
`
`9memwm
`
`
`
`e9wn,P8SaU
`
`mn,
`
`42mB
`
`#.1m.3
`
`CU
`
`8.P/kw.//1%z:aS.mmx:
`
`HE.9“.
`
`:Zu9:820{0ng
`my6EmuEmu
`
`
`0%m65:0030c_o=:ooSo:
`
`1..2Qmm920:3mamam4.,Nr/.I\VwW65:830:Etcoo26cDsSS3%
`
`
`

`

`aD.
`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 10 of 23 Page ID #:125
`H2
`9
`O
`5
`g
`
`1nmm,m4U.
`
`1mn30fl7tmwmow
`
`sSa.cU
`
`8.P
`
`om2‘¢‘WeHmmuumoBEBQ$8305.53:mmcmzuxw
`
`
`
`e.33\52kE:\EEK
`
`Emazmmm“H!“HEMHIMin
`
`S:
`
`$5202.3:m&\.EtwuNtTw
`C0..
`
`
`a5.,MmP8mmSQfl
`
`2mB#1mnW9,
`
`3U
`
`1__eoo
`
`,mMM
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 11 of 23 Page ID #:126
`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 11 of 23 Page ID #:126
`
`U.S. Patent
`
`Sep.4,2012
`
`Sheet80f11
`
`US 8,259,121 B2
`
` 0— Nn
`
`vm my»
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 12 of 23 Page ID #:127
`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 12 of 23 Page ID #:127
`
`U.S. Patent
`
`Sep. 4, 2012
`
`Sheet 9 of 11
`
`US 8,259,121 B2
`
`53:0owE>
`
`mrm?
`
`
`
`NwNw
`
`
`
`orNF
`
`
`
`Sumo...82>
`
`932mumuoowo
`
`Ne.9;
`
`agamwwwo
`
`<55Ewammmmutmmam89>8&2
`
`
`
`
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 13 of 23 Page ID #:128
`Case 8:20-cv-00529 Document 1—2 Filed 03/13/20 Page 13 of 23 Page ID #:128
`
`US. Patent
`
`Sep. 4, 2012
`
`Sheet 10 of 11
`
`US 8,259,121 132
`
`
`Generate DMA
`Trigger at End of
`Picture
`
`
`RUL
`
` Fetch Predefined
`
`1310
`
`1312
`
`
`Notify Decoder
`About End of Picture
`
`
` 1314
`
`1316
`
` Re-Enable Feeder
`
`1318
`
` Configure Video Feeder
`for Next Picture
`
` Fetch Specrfled Picture
`From Frame Buffer
` 1320
` 1322
`Display Pipeline
`
`Feed Speficied Picture TO
`
`Fig. 13
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 14 of 23 Page ID #:129
`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 14 of 23 Page ID #:129
`
`US. Patent
`
`Sep. 4, 2012
`
`Sheet 11 of 11
`
`US 8,259,121 132
`
`Method
`
`Register
`
`Register Bus
`
`Register DMA M
`
`BVB Control
`Packet
`
`Single Buffering
`
`Double Bufferlng
`
`Single Buffering
`
`Double Buffering
`
`Single Buffering
`
`Double Buffering
`lnband Control
`
`Fig. 14
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 15 of 23 Page ID #:130
`Case 8:20-cv-00529 Document 1—2 Filed 03/13/20 Page 15 of 23 Page ID #:130
`
`US 8,259,121 B2
`
`1
`SYSTEM AND METHOD FOR PROCESSING
`DATA USING A NETWORK
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is related to, and claims benefit of and
`priority from, Provisional Application No. 60/420,151 dated
`Oct. 22, 2002, titled “Network Environment for Video Pro-
`cessing Modules”, the complete subject matter of which is
`incorporated herein by reference in its entirety. This applica-
`tion is also related to the following applications, each of
`which is incorporated herein by reference in its entirety for all
`purposes: US. patent application Ser. No. 10/300,371, filed
`Nov. 20, 2002,
`titled “A/V Decoder Having A Clocking
`Scheme That Is Independent Of Input Data Streams”; US.
`Provisional Application No. 60/420,347, filed Oct. 22, 2002,
`titled “Video Bus For aVideo Decoding System”; US. patent
`application Ser. No. 10/300,370, filed Nov. 20, 2002, titled
`“Hardware Assisted Format Change Mechanism in a Display
`Controller”; US. patent application Ser. No. 10/114,798,
`filedApr. 1, 2002, titled “Video Decoding System Supporting
`Multiple Standards”; and US. Provisional Application No.
`60/420,308, filed Oct. 22, 2002, titled “Multi-Pass System
`and Method Supporting Multiple Streams of Video”.
`
`FEDERALLY SPONSORED RESEARCH OR
`DEVELOPMENT
`
`[Not Applicable]
`
`SEQUENCE LISTING
`
`[Not Applicable]
`
`MICROFICHE/COPYRIGHT REFERENCE
`
`[Not Applicable]
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates to a network adapted to pro-
`cess data. More specifically, the present invention relates to a
`network environment in an A/V system using “A/V decod-
`ers”, where the A/V decoders are adapted to process, decode
`or decompress one or more input data streams (alternatively
`referred to as “input data”, “input data streams” or “data
`streams”).
`There is currently no known methodological way to con-
`nect video processing modules in A/V systems. Most video
`processing modules are connected together in an ad-hoc man-
`ner. As a result, such ad-hoc designs may become difficult to
`verify, maintain and reuse. Furthermore, as more features are
`added to the A/V systems (i.e., incorporating more video
`processing modules for example) it becomes more difficult to
`design and integrate such features properly. This may result in
`long development cycles, poor design reuse and an unreliable
`product.
`Further limitations and disadvantages of conventional and
`traditional approaches will become apparent to one of skill in
`the art, through comparison of such systems with the present
`invention as set forth in the remainder of the present applica-
`tion with reference to the drawings.
`
`BRIEF SUMMARY OF THE INVENTION
`
`There is a need for an architecture or network that provides
`a general model illustrating how various video processing
`
`2
`modules behaves in a network environment. Further, an
`exemplary embodiment of such network should reduce the
`number of clock domains, ease design reuse and perform
`format changes in a robust manner.
`Features of the present invention may be found in a net-
`work environment in an A/V system and method supporting a
`pull data flow scheme for an A/V decoder. The network is
`adapted to video process modules using a pull data flow (an
`output rate driven by data flow for example).
`One embodiment of the present invention relates to a net-
`work for processing data to form at least one display pipeline
`therein by selecting and concatenating at least two nodes from
`a plurality ofnodes in the network together. It is contemplated
`that this selection and concatenation happens on the fly (i.e.,
`in real time). In this embodiment, the network is further
`adapted to form a plurality of the same or different display
`pipelines using at least the two nodes. It is contemplated that
`the network may change the functionality ofthe display pipe-
`line by concatenating more than two nodes together. In one
`embodiment, the network is adapted to form at least two
`display pipelines having different and/or independent data
`rates (using a flow control valve or module for example). It is
`further contemplated that such network is adapted to form at
`least two of the display pipelines using a handshaking or
`ready/accept protocol.
`In another embodiment, the network comprises at least a
`register DMA controller adapted to support register access.
`The register DMA controller is further adapted to obtain at
`least one instruction from a register update list and provide
`that instruction to the display pipeline. It is further contem-
`plated that the register DMA controller may obtain the
`instruction in response to a trigger event.
`Yet another embodiment of the present invention relates to
`a network for processing data. In this embodiment, the net-
`work comprises a register DMA controller adapted to support
`register access and a plurality of nodes adapted to process the
`data. The network further comprises at least one link commu-
`nicating with the nodes and adapted to transmit the data
`between the nodes, and at least one network module commu-
`nicating with at least the link and adapted to route the data
`thereto, wherein the network is adapted to form at least one
`display pipeline therein by selecting and concatenating at
`least two nodes from the plurality of nodes.
`Another embodiment of the present invention relates to a
`method of processing data using a network. In this embodi-
`ment, the network comprises forming a first display pipeline
`using at least one node in the network and processing the data
`using the first display pipeline. The method further comprises
`forming a second display pipeline using at least one node in
`the network and processing the data using the second display
`pipeline, where the first and second display pipelines are
`different.
`
`Still another embodiment ofthe present invention relates to
`a method of processing data using a network. In this embodi-
`ment, the network comprises forming a display pipeline by
`selecting and concatenating at least two nodes from a plural-
`ity of nodes in the network on the fly (i.e., in real time) and
`processing the data using the display pipeline.
`Another embodiment of the present invention relates to a
`method of programming an A/V system using a network. In
`this embodiment, the network comprises generating at least
`one trigger at an end ofa first picture and obtaining at least one
`register update list from a main memory. The network notifies
`a decoder about the end of the first picture and configures at
`least one node in the network for a second picture. The net-
`work enables the at least one node, obtains the second picture
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 16 of 23 Page ID #:131
`Case 8:20-cv-00529 Document 1—2 Filed 03/13/20 Page 16 of 23 Page ID #:131
`
`US 8,259,121 B2
`
`3
`from a frame buffer, and provides the second picture to a
`display pipeline in the network.
`These and other advantages and novel features of the
`present invention, as well as details of an illustrated embodi-
`ment thereof, will be more fully understood from the follow-
`ing description and drawings.
`
`BRIEF DESCRIPTION OF SEVERAL VIEWS OF
`THE DRAWINGS
`
`FIG. 1 illustrates one embodiment of a block diagram of an
`A/V decoder in accordance with the present invention;
`FIG. 2 illustrates another embodiment of a block diagram
`of an A/V decoder in accordance with the present invention;
`FIG. 3 illustrates one embodiment of a block diagram of an
`A/V system having a network in accordance with the present
`invention;
`FIG. 4 illustrates another embodiment of a block diagram
`of an A/V system having a network in accordance with the
`present invention;
`FIG. 5 illustrates one embodiment of a block diagram of a
`network environment for videoprocessing modules;
`FIG. 6 illustrates another embodiment of a block diagram
`of a network environment in accordance with the present
`invention;
`FIG. 7 illustrates one embodiment of a register DMA con-
`troller in accordance with one embodiment of the present
`invention;
`FIG. 8 illustrates embodiments of block diagrams ofnodes
`in accordance with the present invention;
`FIG. 9 illustrates one embodiment of an entry node in
`accordance with one embodiment of the present invention;
`FIG. 1 0 illustrates one embodiment ofa network module in
`
`10
`
`15
`
`20
`
`25
`
`30
`
`accordance with one embodiment of the present invention;
`FIGS. 11A, 11B, 11C, 11D, 11E, 11F and 11G illustrate
`embodiments of switched used in a network module in accor-
`
`35
`
`dance with one embodiment of the present invention;
`FIG. 12 illustrates one embodiment of a programming
`model in accordance with one embodiment of the present
`invention;
`FIG. 13 illustrates one embodiment of a high level flow
`chart of a programmable method using at least one node in
`accordance with one embodiment of the present invention;
`and
`
`FIG. 14 illustrates three methods used to write or imple-
`ment control registers in accordance with one embodiment of
`the present invention.
`
`DETAILED DESCRIPTION OF THE INVENTION
`
`The following description is made with reference to the
`appended figures.
`One embodiment of the present invention relates to a net-
`work environment. More specifically, one embodiment
`relates to a network environment in an A/V decoder device
`
`that decodes one or more input data streams with multiple
`output rates using a single clock reference. This embodiment
`enables video processing modules having multiple time bases
`to be implemented using a single clock reference (alterna-
`tively referred to as a “system clock”). FIGS. 1 and 2 illustrate
`block diagrams of embodiments of anA/V decoders in accor-
`dance with the present invention.
`FIG. 1 illustrates one embodiment of a high level block
`diagram of an embodiment of an A/V decoder, generally
`designated 110. More detail about the A/V decoder is pro-
`vided in US. patent application Ser. No. 10/300,371 filed
`Nov. 20, 2002,
`titled “A/V Decoder Having A Clocking
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`Scheme That Is Independent Of Input Data Streams”, the
`complete subject matter of which is incorporated herein by
`reference in its entirety. In the illustrated embodiment, the
`decoder 110 comprises a system time reference recovery
`device 112 (alternatively referred to as an “STR recovery
`device”) having one or more input data streams 118.
`The STR recovery device 112 is illustrated communicating
`with anA/V data processing device 114. In one embodiment
`of the invention, STR refers to a reference time value. It is
`anticipated that different or more complex systems are also
`possible and within the scope of the present invention. For
`example if the A/V decoder 110 has more than one data
`source, the decoder may include more than one STR recovery
`device, where the number of STR recovery devices may or
`may not correspond to the number of data sources.
`As an alternative to the MPEG scheme, an A/V system
`incorporating an A/V decoder may accept analog television
`signals as inputs. In this embodiment, the analog video input
`goes through, and is processed or decoded by, the A/V data
`processing device 114, which may comprise a video decoder
`or VDEC. Likewise, analog audio goes through, and is pro-
`cessed or decoded by, the A/V data processing device 114
`which may further comprise a BTSC audio decoder (altema-
`tively referred to as a “ADEC” or “BTSC”).
`One embodiment of the present invention uses a system
`clock (a fixed system clock for example) to control the data
`processing. More specifically, the system clock may be used
`to control the data process in a network in accordance with the
`present invention. It is contemplated that the STR recovery
`device 112 may be locked to the analog video line rate. The
`analog hysncs are converted into a pseudo-STR using a
`simple counter in one embodiment. The STR recovery device
`112 locks to this pseudo-STR and broadcasts the recovered
`STR to the rest ofthe decoder 110. The broadcast STR is used
`
`to control the output rates as provided previously.
`FIG. 1 further illustrates a rate managed output device 116,
`which is illustrated as communicating with the data process-
`ing device 114. In the illustrated embodiment, the rate man-
`aged output device 116 has one or more A/V outputs 120,
`which are output at the same or different rates. In FIG. 1, three
`A/V outputs, generally designated 120, are illustrated. For
`example, one A/V output is output at 29.999 frames per
`second (alternatively referred to as “fps”), one is output at
`30.001 fps and one is output at 30.000 fps.
`In one embodiment, the A/V data processing device 114
`includes a network environment for video processing mod-
`ules. The data processing device 114 bases audio and video
`processing on multiples of a single, fixed clock, a 27 MHZ
`crystal clock for example. It is contemplated that, as a single
`fixed clock is used, the processing is not constrained by clock
`boundaries. Video and audio may be muxed between mod-
`ules. It is further contemplated that such architecture may be
`made orthogonal, and easy to control.
`In accordance with one embodiment, all data, including all
`audio and video data, is processed by a network environment
`and transferred using a “pull” model or mode, even though
`typical A/V streams (e.g., MPEG) are adapted to operate
`according to a push model or mode. The outputs request data
`as needed. Each module in the A/V decoder 110 may supply
`data to its outputs at the rate it is requested. Because a pull
`model or mode is used, the data processing clock (i.e., the
`system clock) is not tied to the input data rate. For example,
`the audio decoder may be clocked at 243 MHZ, 133 MHZ, or
`any other reasonable rate. The audio decoder clock does not
`need to “track” the input data rate.
`Conventional A/V decoders use aVCXO or VCXO-PLL to
`
`lock the chip clock to the input data rate. However, one
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 17 of 23 Page ID #:132
`Case 8:20-cv-00529 Document 1—2 Filed 03/13/20 Page 17 of 23 Page ID #:132
`
`US 8,259,121 B2
`
`5
`embodiment of the present invention uses rate managed out-
`put devices 116 and the associated SRC devices to change or
`adjust the Video and audio output rates.
`It is contemplated that, in one embodiment of the present
`invention, the output data rate tracks the STR. If the A/V
`decoder decodes multiple video streams, there may be mul-
`tiple STRs. Each output data rate tracks an associated STR.
`The process ofcontrolling the output rates may be called “rate
`management.” In one embodiment, the rate managed output
`device 116 (alternatively referred to as a “output rate man-
`ager” or “output rate manager PLL”), comprising for example
`a digital PLL, is used to compare the output rate with the STR,
`and adjust the output rate accordingly, such that the output
`data rate matches the STR and the input data rate. In one
`embodiment, the A/V decoder may include several output
`rate managers, one for each output of the A/V decoder. More
`detail about rate managers is provided in U.S. Provisional
`Application No. 60/420,344 filed Oct. 22, 2002, titled “Data
`Rate Management System and Method for A/V Decoder”.
`FIG. 2 illustrates another embodiment of a block diagram
`of an A/V decoder, generally designated 210, in accordance
`with one embodiment of the present invention. In the illus-
`trated embodiment, the decoder 210 comprises an STR recov-
`ery device 212 having one or more input data streams 218 and
`a STR broadcast output.
`In the illustrated embodiment, the input data streams (alter-
`natively referred to as “system clock sources” or “system
`reference sources”) 218 comprise an MPEG (PCR/SCR)
`stream, a 656 (hysnc) stream and a VDEC (hysnc) stream.
`While three input streams are illustrated, more complex sys-
`tems, having more or different input data streams are contem-
`plated. In the illustrated embodiment, the input time refer-
`ences are MPEG PCR/SCR values. However, for analog
`video or ITU656 video inputs, the hsync timing may be used
`as the time reference or a fixed timing reference may be used
`for PVR playback.
`The STR recovery device 212 is illustrated as communi-
`cating (indirectly in this embodiment) with a data processing
`device 214. In one embodiment, the SRT recovery device 212
`controls the output data rates (in conjunction with a rate
`managed output and SRC devices). The data processing
`device 214 is adapted to decode, capture, play back and
`produce graphics, etc. from the data inputs (i.e., the input data
`streams 218) using a fixed clock or timing reference. That is
`the data processing devices may decode, capture, play back
`and produce graphics, etc. using a fixed clock (i.e., the system
`clock for example). In one embodiment, the data is supplied
`to an output device or buffer 222 as requested (i.e., the output
`device requests data from the data processing device or the
`data is “pulled”). It is contemplated that, in one embodiment,
`the data processing device 214 comprises or includes a net-
`work environment for video processing modules in accor-
`dance with the present invention.
`A rate managed output device 216 is illustrated as commu-
`nicating (indirectly in this embodiment) with at least the data
`processing device 214. More specifically, the rate managed
`output device 216 communicates with the STR recovery
`device 212 and the output device 222. In the illustrated
`embodiment, the rate managed output device 216 comprises
`at least local STR and compare devices 215 and 217 respec-
`tively, while the output device 222 comprises at least an SRC
`device 223.
`
`In one embodiment, the output device 222 outputs data 220
`at a fixed clock rate (i.e., the system clock rate) as it is
`requested. The output device 222 submits data requests to the
`data processing device 214, and thus pulls the data. The data
`request is also submitted or mirrored to the rate managed
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`output device 216, where it is compared with the STR broad-
`cast in the compare module 217. A rate control signal is
`communicated to the output device 222 (specifically the SRC
`device 223), ensuring that the data 220 is output at the fixed
`clock rate, and the output data rate matches the input data rate.
`The digital sample rate converter converts data from an input
`sample rate to an output sample rate. In one embodiment, the
`output sample rate may differ from the input sample rate. By
`adjusting the SRC parameters,
`the rate managed output
`device 216B changes the rate ofthe sample rate at the input of
`the SRC device 223B. This change to the sample rate changes
`the rate the data is requested from the data processing device
`214B.
`
`FIG. 3 illustrates one embodiment of a block diagram of an
`A/V system, generally designated 300, having a network in
`accordance with the present invention. It is contemplated that
`the illustrated A/V system may be similar to those A/V sys-
`tems provided previously. It is also contemplated that the
`network may be used in different systems. In this embodi-
`ment, system 300 includes a decoder 310 (an MPEG decoder
`for example) adapted to receive video inputs or data 308. In
`this embodiment, the decoder 310 includes one or more STR
`recovery devices 312, used, with the system clock (a fixed
`system clock for example) to control the data processing
`similar to that provided previously. However, other decoders,
`with or without STR recovery devices are contemplated.
`A memory or frame buffer 314 is illustrated coupled to the
`decoder 310 and receives data therefrom. The memory 314 is
`shown coupled to network 316 as illustrated, which is adapted
`to transport and process video or data, outputting video out or
`data 320. In one embodiment, the network 316 is adapted to
`support a pull data flow. The network 316 includes one or
`more counters 318 (coupled to the STR recovery device via
`feedback loop 322) that, along with the rate managed output
`device (not shown) control the data rate of the output.
`FIG. 4 illustrates one embodiment of a block diagram of a
`network, similar to the network 316 of FIG. 3 in accordance
`with the present invention. In this embodiment, the network
`416 is adapted to receive video-in 408 (from a memory for
`example) and output video out 420.
`FIG. 4 further illustrates at least one display pipeline 440
`inside the network 416. In one embodiment of the present
`invention, the display pipeline 440 is changeably formed by
`chaining, coupling or concatenating one or more network
`nodes together, depending on the network requirements, on
`the fly (i .e., in real time). It is contemplated that the nodes may
`be re-configured, so that a plurality of display pipelines 440
`may be formed, each pipeline having different functionality
`depending on the nodes that are concatenated together. More-
`over, in one embodiment, it is contemplated that the network
`440 may change the display pipeline 440 every l/so‘h of a
`second for example.
`In this embodiment, a register DMA controller 442 (alter-
`natively referred to as an “RDC”) is illustrated coupled to the
`network 416 and one or more register update lists 446 (alter-
`natively referred to as an “RUL”). The RDC 442 is adapted to
`support multiple, configurable pipelines 440 by accessing
`and fetching (i.e., obtaining) one or more instructions from
`the RUL 446 and providing such instructions to the display
`pipeline 440. In one embodiment, the RDC 442 accesses the
`RUL 446 (fetching the instructions) in response to the one or
`more trigger signals 444 (real time DMA trigger signals or
`events generated by the last node in the pipeline 440 for
`example). It is contemplated that, if the network 416 did not
`have an RDC 442 associated therewith, the network 416
`would have to reconfigure the pipeline one register at a time.
`
`

`

`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 18 of 23 Page ID #:133
`Case 8:20-cv-00529 Document 1-2 Filed 03/13/20 Page 18 of 23 Page ID #:133
`
`US 8,259,121 B2
`
`7
`FIG. 5 illustrates one embodiment of a block diagram of a
`network environment (alternatively referred to as a “display
`engine”) for Video processing modules in accordance with the
`present invention. The network, generally designated 500, is
`adapted to support a pull data scheme and comprises at least
`a register DMA controller, one or more nodes, one or more
`links, and one or more network modules. In this embodiment,
`the register DMA controller 510 (or register DMA controller)
`is responsible for register access within the system 500. The
`register DMA controller 510 connects the register bus 512
`(alternatively referred to as “RBUS”) with the video register
`bus 514 (alternatively referred to as “VBUS”).
`The system 500, in one embodiment, further comprises one
`or more nodes 516 (two nodes 516A & 516B are illustrated).
`Nodes 51 6 are modules that process video information (nodes
`516A & 516B are illustrated having video-in signals 514 and
`video-out signals 526 respectively). Some examples ofnodes
`comprise video scalers, 2D graphics compositors, video
`encoders, etc.
`FIG. 5 further illustrates one or more links 518 (links 518A
`& 518B are illustrated). In this embodiment, the links 518
`comprise a set of signals or buses that tie or connect at least
`two nodes together (link 518A is illustrated coupling node
`516A to network module 520 while link 518B is illustrated
`
`coupling network module 520 to node 516B). The links 518
`are adapted to transfer information using a predefined proto-
`col. More detail about the links is provided in US. Provi-
`sional Application No. 60/420,347 filed Oct. 22, 2002, titled
`“Video Bus For a Video Decoding System”, the complete
`subject matter ofwhich is incorporated herein by reference in
`its entirety.
`Additionally, system 500 comprises one or more network
`modules 520 that, in this embodiment, are specialized nodes
`that don’t perform video processing functions. Rather, the
`network module 520 connects at least two or more links 518
`
`together, routing information between them. In general, the
`system 500 may include a number of pipelines (i.e., display
`pipelines) formed