Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 1 of 34 Page ID #:264
`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 1 of 34 Page ID #:264
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`EXHIBIT I
`
`EXHIBIT I
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 2 of 34 Page ID #:265
`I IIIII IIIIIIII Ill lllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111
`US009332283B2
`
`c12) United States Patent
`Chen et al.
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 9,332,283 B2
`May 3, 2016
`
`(54) SIGNALING OF PREDICTION SIZE UNIT IN
`ACCORDANCE WITH VIDEO CODING
`
`(75)
`
`Inventors: Peisong Chen, San Diego, CA (US);
`Brian Heng, Irvine, CA (US); Wade K.
`Wan, Orange, CA (US)
`
`(73) Assignee: BROADCOM CORPORATION,
`Irvine, CA (US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 860 days.
`
`(21) Appl. No.: 13/523,822
`
`(22) Filed:
`
`Jun.14,2012
`
`(65)
`
`Prior Publication Data
`
`Mar. 28, 2013
`US 2013/0077684 Al
`Related U.S. Application Data
`
`(60)
`
`Provisional application No. 61/539,948, filed on Sep.
`27, 2011.
`
`(51)
`
`(52)
`
`(58)
`
`Int. Cl.
`H04N7/12
`H04N 11/02
`H04N 11/04
`H04N 19/96
`H04N 19/503
`
`(2006.01)
`(2006.01)
`(2006.01)
`(2014.01)
`(2014.01)
`(Continued)
`
`U.S. Cl.
`CPC ............. H04N 19/96 (2014.11); H04N 19/503
`(2014.11); H04N 19/70 (2014.11); H04N
`19/119 (2014.11); H04N 19/174 (2014.11)
`Field of Classification Search
`CPC . H04N 19/119; H04N 19/115; H04N 19/169;
`H04N 19/174; H04N 19/177; H04N 19/96
`USPC ........................................ 375/240.01-240.29
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`2004/0213469 Al* 10/2004 Apostolopoulos et al. ... 382/239
`2005/0038837 Al
`2/2005 Marpe et al.
`(Continued)
`
`OTHER PUBLICATIONS
`
`Bross, et al.; WD4: Working Draft 4 of High-Efficiency Video Cod(cid:173)
`ing; Joint Colaborative Team on Video Coding (JCT-VC) ofITU-T
`SG16 WP3 and ISO/IEC JTC1/SC29/WG11; 6th Meeting: Torino,
`IT; Jul. 14-22, 2011; 216 pgs.
`(Continued)
`
`Primary Examiner - Andy Rao
`Assistant Examiner -
`Jared Walker
`(74) Attorney, Agent, or Firm - Garlick & Markison;
`Shayne X. Short
`
`(57)
`
`ABSTRACT
`
`Signaling of prediction size unit in accordance with video
`coding. In accordance with video coding, various binariza(cid:173)
`tion may be performed. In accordance with coding related to
`different types of slices (e.g., I, P, B slices), one or more
`binary trees may be employed for performing various respec(cid:173)
`tive operations ( e.g., coding unit (CU) prediction and predic(cid:173)
`tion unit (PU) partition mode operations). In one implemen(cid:173)
`tation, a common or singular binary tree is employed to
`encode jointly CU prediction and PU partition mode in a
`single syntax element for both P slices and B slices. That is to
`say, in such an implementation, instead of employing differ(cid:173)
`ent respective binary trees for at least these different respec(cid:173)
`tive processes/operations, a common or single binary tree
`may be employed for them both. Appropriate coordination
`between and encoder/transmitter device and a decoder/re(cid:173)
`ceiver device may be performed to ensure appropriate han(cid:173)
`dling of different respective phases of video coding.
`
`20 Claims, 17 Drawing Sheets
`
`( commu+nication
`
`channel ma '1
`
`-r sa~
`
`satellite dish 132
`
`satellite dish 134
`
`I
`
`i
`
`I
`
`130
`
`tower 142
`
`tower 144
`
`local antenna 152
`local antenna 154
`
`wired 150
`
`0
`0
`0
`
`decoder
`lli
`
`receiver 116.
`
`device 11Q
`
`0
`0
`0
`
`E/0
`interface
`~
`
`encoder
`W.!
`
`transmitter 126.
`
`device 1.29.
`
`0
`0
`0
`
`0/E
`interface
`1M
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 3 of 34 Page ID #:266
`
`US 9,332,283 B2
`Page 2
`
`(51)
`
`Int. Cl.
`H04N 19/70
`H04N 19/174
`H04N 19/119
`
`(2014.01)
`(2014.01)
`(2014.01)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`2010/0098155 Al
`2011/0206123 Al*
`2011/0228858 Al
`
`4/2010 Demircin et al.
`8/2011 Panchal et al. ........... 375/240.15
`9/2011 Budagavi et al.
`
`OTHER PUBLICATIONS
`
`European Patent Office; European Search Report; EP App No.
`12005568.6; Dec. 17, 2012; 5 pgs.
`
`2010/0086032 Al*
`
`4/2010 Chen et al. ............... 375/240.12
`
`* cited by examiner
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 4 of 34 Page ID #:267
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`....
`0 ....
`....
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`-....J
`
`N
`~
`
`~
`
`0 ....
`
`O'I
`
`~
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`device 120
`
`transmitter 126
`
`128
`
`encoder
`
`receiver 122
`
`124
`
`decoder
`
`164
`
`interface
`
`0/E
`
`0
`0
`0
`
`FIG.1
`
`'---------------------~
`,
`'
`
`0
`0
`0
`
`wired 150
`
`local antenna 154
`
`local antenna 152
`
`'C=' wireless 140
`
`:s;;;:
`
`tower 144
`
`tower 142
`
`162
`
`interface
`
`E/0
`
`0
`0
`0
`
`satellite dish 134
`
`100
`
`130
`
`J="+"=L
`
`satellite
`
`device 110
`
`receiver 116
`
`118
`
`decoder
`
`transmitter 112
`
`114
`
`encoder
`
`satellite dish 132
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 5 of 34 Page ID #:268
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`....
`0 ....
`N
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`-....J
`
`N
`~
`
`~
`
`0 ....
`
`O'I
`
`~
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`receiver 298
`
`transmitter 297
`
`channel299
`
`communication
`
`transmit signal 205
`
`I; filtered, continuous-time
`
`transmit signal 204
`
`continuous-time
`
`---
`
`filter 234
`transmit
`
`Lu
`
`232
`DAG
`
`transmit driver 230
`
`sequence of discrete-valued modulation symbols 203
`
`------------------------------
`
`---generator
`
`metric
`
`270
`
`280
`
`decoder
`
`and/or bit, e.g., LLRs)
`r-metrics 209 (symbol
`-----------------------------
`
`processing module 280b
`
`symbols and information bits encoded therein 210
`
`best estimates of discrete-valued modulation
`
`·-------------------------------
`
`bits 202
`
`encoded information
`
`mapper
`symbol
`
`encoder
`
`222 L 224
`
`encoder and symbol mapper 220
`
`r------------------------------,
`
`processing module 280a
`
`information bits 201
`
`200
`
`FIG. 2
`
`receive signal 207
`
`filtered, continuous-time
`
`receive signal 206
`
`I'-continuous-time
`
`--
`
`r""(! filter 262
`
`receive
`
`264
`ADC
`
`AFE 260
`
`discrete time receive signals 208
`
`I
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 6 of 34 Page ID #:269
`
`~
`
`('D
`
`....
`0 ....
`.....
`rJJ =- ('D
`
`-....J
`
`N
`~
`
`~
`
`0 ....
`
`O'I
`
`~
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`FIG. 3H
`
`(e.g., TV)
`
`'-capable device
`I
`I
`
`to display
`
`processing device
`
`FIG. JG
`
`FIG. 3F
`
`I
`
`FIG. 3E
`
`SD, HD, etc.)
`
`DVD player (e.g., Blu-Ray,
`
`(e.g., TV)
`
`capable device
`
`to display
`
`digital image
`
`.. --'
`
`source device
`
`provider and/or other
`from media content
`
`308
`
`device (e.g., TV)
`to display capable
`
`307
`
`I
`
`FIG. JD
`
`I
`
`FIG. JC
`
`(e.g., 4:3)
`
`SDTV
`
`HDTV (e.g., 16:9)
`
`306
`
`I
`
`HD)
`
`(SD and/or
`set top box
`
`content '
`
`provider
`
`from media
`
`D DODD
`
`DODD
`
`I
`
`305
`
`FIG. 38
`
`I
`
`FIG. 3A
`
`A w
`D
`
`V
`
`-
`
`3 E1-1
`D
`
`- E
`
`source device
`
`provider and/or other
`from media content
`
`304
`
`303
`
`source device
`
`provider and/or other
`from media content
`
`302
`
`301
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 7 of 34 Page ID #:270
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`architecture
`
`• video encoder
`
`FIG.4
`
`....
`0 ....
`.....
`rJJ =(cid:173)
`
`-....J
`
`.i;...
`
`('D
`('D
`
`N
`~
`
`~
`
`0 ....
`
`O'I
`
`~
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`stream
`output bit
`
`400
`
`quantization
`
`inverse
`
`(e.g., IDCT)
`transform
`inverse
`
`actual measured value thereof)
`
`data rate or PHY rate (or change, or
`predicted/anticipated value of video
`
`CAVLC)
`
`(e.g., CABAG,
`entropy encoder
`
`quantization
`
`(e.g., OCT)
`transform
`
`mode 2 (inter)
`mode 1 (intra)
`mode O (neither)
`mode selection:
`
`estimation
`
`motion
`
`vector
`motion
`
`compensation
`
`motion
`
`signal
`
`input video
`
`\
`
`pixels, (e.g., N=16)
`
`j
`
`macro blocks (MBs), N x N
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 8 of 34 Page ID #:271
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`....
`0 ....
`Ul
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`-....J
`
`N
`~
`~
`~
`~
`
`~
`
`0 ....
`
`O'I
`
`~ = ~
`
`~
`~
`~
`•
`00
`
`e •
`
`architecture
`
`• video encoder
`
`FIG. 5
`
`filter
`
`de-blocking
`
`selection
`
`angular mode
`
`I
`
`I intra prediction IE------.
`
`estimation
`
`motion
`
`L---il
`
`vector
`motion
`
`mode 2 (inter)
`mode 1 (intra)
`mode O (neither)
`mode selection:
`
`I
`
`I
`
`compensation I
`
`motion
`
`~
`
`+
`
`decision
`
`intra/inter mode
`
`entropy encoder if
`
`CAVLC)
`
`(e.g., CABAG,
`
`---,
`
`stream
`output bit
`
`(e.g., IDCT)
`
`inverse transform
`quantization and
`
`inverse
`
`-------------
`
`500
`
`macro blocks (MBs), N x N
`
`pixels, (e.g., N=16)
`
`signal
`
`input video
`
`and quantization
`
`transform (e.g., OCT)
`
`\
`I I I I I r +~
`j
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 9 of 34 Page ID #:272
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`types)
`
`as ALF, SAO, and/or other filter
`(with other in-loop filters (such
`• video encoder architecture
`
`filter
`
`de-blocking
`
`FIG. 6
`
`filter(s)
`
`loop
`
`other in
`
`selection
`
`angular mode
`
`7
`
`....
`0 ....
`O'I
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`-....J
`
`I intra prediction IE--
`
`motion ~
`
`estimation
`
`L---il
`
`picture buffer
`
`vector
`motion
`
`mode 2 (inter)
`mode 1 (intra)
`mode O (neither)
`mode selection:
`
`I
`
`I
`
`compensation I
`
`motion
`
`~
`
`N
`~
`
`~
`
`0 ....
`
`O'I
`
`~
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`T
`
`'" 7
`
`stream
`output bit
`
`inverse transform
`quantization and
`
`(e.g., IDCT)
`
`+ 0
`
`intrarinter mode I
`
`dec1s1on
`
`inverse
`
`CAVLC)
`
`(e.g., CABAC,
`entropy encoder
`
`and quantization -------------
`
`transform (e.g., OCT)
`
`+o---,
`
`signal
`
`input video
`
`\
`
`r
`
`11111
`
`600
`
`blocks (MBs), N x N pixels,
`coding units (CUs) or macro
`
`j (e.g., N=16)
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 10 of 34 Page ID #:273
`
`U.S. Patent
`
`May 3, 2016
`
`Sheet 7 of 17
`
`US 9,332,283 B2
`
`(].)
`
`m >,.!!2
`
`-
`~ ·§_
`0
`·- "O
`> (].)
`~ "O
`C. 0
`(.)
`
`Q_ (.)
`
`co
`..-..
`::::i
`"O "O
`·- (].)
`Cl) "O
`~ 0
`X C
`I
`(].)
`u Cl)
`X Q)
`119
`"O
`X
`
`u
`X
`Q)
`X
`·o..
`
`C
`
`(].) ,_ ,_
`
`::::i
`(.)
`
`I
`I
`I
`
`I
`
`I
`
`I
`
`C
`
`0 t5 C
`·- 0
`"O ·-
`~ t5
`C. ~
`I · -
`(0 "O
`b
`C
`
`\
`\
`\
`
`\
`
`0
`0
`0;
`~
`'.;:;;
`8
`%'.;'.
`~
`0
`0
`0;
`V"
`~
`'.;:;;
`~
`8
`w / /
`% 0; ~
`
`~ ~
`
`,,__
`C
`0
`t5
`'5
`~ X
`C.....::
`'-"'(].)
`
`Q_
`
`(].) ,_
`
`(].) -~ g C.
`'* ,_
`
`...........
`
`~
`1,.V
`
`V
`/ "
`
`v'/.'
`V//
`
`0
`
`0 0
`
`0 ~ i i :i ~
`
`..::0::
`
`..-.. uz
`.Q X
`:: z c'-"'
`(].) ,_ ,_
`
`::::i
`(.)
`
`s1ax!d papm
`A1snO!AaJd
`
`......
`(!)
`LL
`
`C
`0
`~
`"O
`(].) ,_
`C.
`I ro ,_
`c
`•
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 11 of 34 Page ID #:274
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`....
`0 ....
`.....
`rJJ =(cid:173)
`
`QO
`
`('D
`('D
`
`-....J
`
`N
`~
`
`~
`
`0 ....
`
`O'I
`
`~
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`800
`
`prediction direction)
`motion vector (inter(cid:173)
`
`frame (or pictu~
`
`sequence V
`
`---· p1xe, Xp
`reference (prediction)
`
`I
`
`•
`
`(gets encoded)
`
`xd = Xe -Xp residual
`
`picture)
`
`current frame (or
`
`FIG. 8
`
`• inter-prediction
`
`current pixel, Xe
`
`' ' ' ' ' ' ' ' ' ' ' ' ' ' '
`
`-----------------
`
`picture(s)
`
`previous frame(s) or
`
`........... -~---
`
`I-'
`I ~I
`r
`
`I
`
`I
`
`I
`I
`
`I-'
`
`....................
`
`I"
`
`1
`I
`:
`,,' I
`,,1
`
`........
`
`..
`
`..
`
`........
`
`,,'
`
`......
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 12 of 34 Page ID #:275
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`architecture
`
`• video decoder
`
`FIG. 9
`
`....
`0 ....
`.....
`rJJ =(cid:173)
`
`l,O
`
`('D
`('D
`
`-....J
`
`compensation
`
`motion
`
`intra prediction
`
`1
`
`1
`
`intra/inter mode
`
`mode 2 (inter)
`mode 1 (intra)
`mode O (neither)
`mode selection:
`
`decision
`
`N
`~
`
`~
`
`0 ....
`
`O'I
`
`~
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`~ de-blocking
`
`filter
`
`T
`
`)
`
`(e.g., IDCT)
`
`inverse transform
`quantization and
`
`inverse
`
`CAVLC)
`
`(e.g., CABAG,
`entropy decoder
`
`stream
`input bit
`
`signal
`
`output video
`
`picture(s)
`frame(s)/
`
`900
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 13 of 34 Page ID #:276
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`....
`0 ....
`0
`....
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`-....J
`
`N
`~
`
`~
`
`0 ....
`
`O'I
`
`~
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`filter types)
`
`loop filters (such as ALF, SAO, and/or other
`• video decoder architecture (with other in(cid:173)
`
`FIG.10
`
`compensation
`
`motion
`
`...._ __ intra prediction
`
`mode 2 (inter)
`mode 1 (intra)
`mode O (neither)
`mode selection:
`
`1----'
`
`decision
`
`intra/inter mode
`
`filter( s)
`
`loop
`
`other in
`
`de-blocking
`
`filter
`
`,--,--..JJ
`
`(e.g., IDCT)
`
`inverse transform
`quantization and
`
`inverse
`
`CAVLC)
`
`(e.g., CABAG,
`entropy decoder
`
`stream
`input bit
`
`signal
`
`output video
`
`picture(s)
`frame(s)/
`
`1000
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 14 of 34 Page ID #:277
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`FIG. 11
`
`depth = 5)
`
`max. hierarchical
`
`structure (LCU = 128,
`
`• recursive CU
`
`....
`0 ....
`....
`....
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`-....J
`
`N
`~~
`~
`~
`
`0 ....
`
`O'I
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`1100
`
`Z _____ _
`
`0
`
`;i-;
`
`2
`
`3
`
`I
`I
`I
`I
`
`= 1
`
`split flag
`
`2 I 3
`
`o I 1
`= 1
`
`split flag
`
`2N
`
`=O 2NG
`
`split flag
`
`2N
`
`=O 2NG
`
`split flag
`
`-------------------------·
`
`2N
`
`Last depth: no split flag
`------------
`
`2NG
`
`depth = 4, N = 4
`
`0
`0
`0
`
`depth = 1, N = 32
`
`depth = 0, N = 64
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 15 of 34 Page ID #:278
`
`U.S. Patent
`
`May 3, 2016
`
`Sheet 12 of 17
`
`US 9,332,283 B2
`
`rn
`Cl)
`"'O
`0
`E
`=> a..
`•
`
`(!)
`LL
`
`~I
`
`z
`i------+------ix
`z
`
`X
`
`z
`z
`N
`
`z
`N
`X
`z
`
`z
`N
`X
`z
`N
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 16 of 34 Page ID #:279
`
`U.S. Patent
`
`May 3, 2016
`
`Sheet 13 of 17
`
`US 9,332,283 B2
`
`~I
`
`0
`
`.....
`
`z
`C'\I
`X
`z
`C'\I
`
`z
`C'\I
`X
`z
`
`0
`
`.....
`
`X
`
`z
`z
`C'\I
`
`0
`
`.....
`
`z
`C'\I
`X
`z
`C'\I
`
`..... 0
`..... 0
`0
`z z z
`z z
`C'\I C'\I z
`
`X
`
`C'\I
`X
`
`C'\I
`X
`
`C'-·
`·x
`
`-(/)
`Q) -
`
`0
`C:
`z
`
`X
`
`z
`
`z
`C'\I
`X
`z
`
`0
`
`.....
`
`X
`
`z
`z
`C'\I
`
`Q)
`Q)
`
`-
`
`.... (/)z
`>, C: X
`.... 0 co·- z
`C:::::
`·- t::::: 0)
`.0 co C:
`"O 0.. '6
`-~ ::J .:!
`~ 0... ~
`o- Q)
`E o •
`•
`
`z
`z
`
`X
`
`M
`""""
`C)
`LL
`
`.....
`..... 0
`..... 0
`0
`
`0
`0
`0
`
`z
`C'\I z z z
`C'\I
`z z
`X
`z
`z
`C'\I C'\I
`
`X
`
`X
`
`X
`
`Cl
`C:
`
`- "O
`::J
`0...
`0
`(/) 0
`- ·-
`C: CJ
`Q)
`Q) 0 C:
`5....:;:::::; Q)
`>, i::: Q)
`.... co CJ
`co 0.. =
`C:
`i5
`•
`•
`
`(/)
`0...
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 17 of 34 Page ID #:280
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`....
`0 ....
`....
`.....
`rJJ =(cid:173)
`
`.i;...
`
`('D
`('D
`
`-....J
`
`N
`~
`
`~
`
`0 ....
`
`O'I
`
`~
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`• alternatively, employ same binary tree for both
`
`P and B slices
`
`FIG.14
`
`• B slice encoding
`
`mode and PU partition mode for B slices
`• binary tree of jointly coded CU prediction
`
`~
`
`0
`
`1
`
`~ 1
`
`0
`
`smallest CU
`
`1400
`
`2N x N
`
`inter
`
`2N x 2N
`
`inter
`
`00000
`00001
`0001
`001
`01
`1
`
`intra 2N x 2N
`intra N x N
`inter N x N
`inter N x 2N
`inter 2N x N
`inter 2N x 2N
`
`2N x 2N
`
`intra
`
`~ 1
`
`0
`
`NxN
`intra
`
`~ 1
`
`0
`
`~ 1
`
`0
`
`NxN
`inter
`
`N x 2N
`inter
`
`000
`001
`01
`1
`
`intra 2N x 2N
`inter N x 2N
`inter 2N x N
`inter 2N x 2N
`
`2N x 2N
`
`intra
`
`0
`
`~
`
`1
`
`N x 2N
`inter
`
`~
`
`0
`
`1
`
`~ 1
`
`0
`
`non-smallest CU
`
`2N x N
`
`inter
`
`2N x 2N
`
`inter
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 18 of 34 Page ID #:281
`
`U.S. Patent
`
`May 3, 2016
`
`Sheet 15 of 17
`
`US 9,332,283 B2
`
`(].)
`(..)
`
`..c: --~
`
`rJ>
`C
`0
`:;:::;
`t
`co
`0..
`=> z .Q -.::I"
`a_ X 0 ~
`C " '
`'+-
`0 Z
`::::l ~
`(].) Ol ·c LL
`~ .!: 8 'o
`-uCO l
`2:'~:::::..c
`~ ~ oi'o
`:0 (].) .!: 8
`"O e "O C
`(].)
`8 (].)
`!f:
`"O
`0
`E
`•
`
`C
`
`,-...
`
`C
`(].)
`(].)
`u
`ci5
`a..
`•
`
`.....
`..... 0
`z z
`z z
`N N
`
`N
`X
`
`X
`
`0
`0
`
`z
`N
`X
`z
`
`Lt') -(!)
`
`LL
`
`~I
`
`z
`N
`X
`z
`
`0
`
`.....
`
`X
`
`z
`z
`N
`
`0
`
`.....
`
`z
`N
`X
`z
`N
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 19 of 34 Page ID #:282
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`....
`0 ....
`O'I
`....
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`-....J
`
`N
`~
`
`~
`
`0 ....
`
`O'I
`
`~
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`FIG. 168
`
`FIG. 16A
`
`)
`
`I
`
`C
`
`bitstream 1621
`
`one B slice in accordance with generating the output
`
`element for both the at least one P slice and the at least
`
`prediction unit (PU) partition mode in a single syntax
`
`to encode jointly coding unit (CU) prediction and
`
`employing a single binary tree within the video encoder
`
`I
`
`signal to generate an output bitstream 1611
`
`operating a video encoder to encode an input video
`
`)
`
`I
`
`C
`
`"'
`
`I
`
`,
`
`accordance with generating the output bitstream 1620
`to process at least one P slice and at least one B slice in
`employing a single binary tree within the video encoder
`
`I
`
`signal to generate an output bitstream 161 O
`
`operating a video encoder to encode an input video
`
`)
`
`I
`
`'-
`
`1601
`
`1600
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 20 of 34 Page ID #:283
`
`00 w = N
`'N
`N
`\C w w
`d r.,;_
`
`....
`0 ....
`-....J
`....
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`-....J
`
`N
`~
`
`~
`
`0 ....
`
`O'I
`
`~
`~
`
`~ = ~
`
`~
`~
`~
`•
`00
`~
`
`FIG. 178
`
`FIG. 17A
`
`)
`
`I
`
`(
`
`version of the input video signal) 1731
`
`generate an output video signal (e.g., re-generated
`
`having traversed the communication channel) to
`
`from or corresponding to the output bitstream, after
`
`bitstream (e.g., output bitstream, or the signal generated
`
`operating the video decoder to decode an input
`
`I
`
`decoder via a communication channel 1721
`
`from or corresponding to the output bitstream, to a video
`transmitting the output bitstream, or a signal generated
`
`I
`
`signal to generate an output bitstream 1711
`
`operating a video encoder to encode an input video
`
`)
`
`I
`
`C
`
`)
`
`I
`
`\.
`r
`
`generating the output bitstream 1730
`
`and the at least one B slice in accordance with
`
`a second syntax element for both the at least one P slice
`encoder to encode prediction unit (PU) partition mode in
`
`employing the single binary tree within the video
`
`I
`
`bitstream 1720
`
`one B slice in accordance with generating the output
`
`element for both the at least one P slice and the at least
`
`to encode coding unit (CU) prediction in a first syntax
`employing a single binary tree within the video encoder
`
`I
`
`signal to generate an output bitstream 1710
`
`operating a video encoder to encode an input video
`
`)
`
`I
`
`C
`
`1701
`
`1700
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 21 of 34 Page ID #:284
`
`1
`SIGNALING OF PREDICTION SIZE UNIT IN
`ACCORDANCE WITH VIDEO CODING
`
`2
`BRIEF DESCRIPTION OF THE SEVERAL
`VIEWS OF THE DRAWINGS
`
`US 9,332,283 B2
`
`CROSS REFERENCE TO RELATED
`PATENTS/PATENT APPLICATIONS
`
`Provisional Priority Claims
`
`The present U.S. Utility Patent Application claims priority
`pursuant to 35 U.S.C. §119(e) to the following U.S. Provi- 10
`sional Patent Application which is hereby incorporated herein
`by reference in its entirety and made part of the present U.S.
`Utility Patent Application for all purposes:
`1. U.S. Provisional Patent Application Ser. No. 61/539, 15
`948, entitled "Signaling of prediction size unit in accordance
`with video coding,", filed Sep. 27, 2011.
`
`Incorporation by Reference
`
`FIG. 1 and FIG. 2 illustrate various embodiments of com-
`5 munication systems.
`FIG. 3A illustrates an embodiment of a computer.
`FIG. 38 illustrates an embodiment of a laptop computer.
`FIG. 3C illustrates an embodiment of a high definition
`(HD) television.
`FIG. 3D illustrates an embodiment of a standard definition
`(SD) television.
`FIG. 3E illustrates an embodiment of a handheld media
`unit.
`FIG. 3F illustrates an embodiment of a set top box (STB).
`FIG. 3G illustrates an embodiment of a digital video disc
`(DVD) player.
`FIG. 3H illustrates an embodiment of a generic digital
`image and/or video processing device.
`FIG. 4, FIG. 5, and FIG. 6 are diagrams illustrating various
`20 embodiments of video encoding architectures.
`FIG. 7 is a diagram illustrating an embodiment of intra(cid:173)
`prediction processing.
`FIG. 8 is a diagram illustrating an embodiment of inter(cid:173)
`prediction processing.
`FIG. 9 and FIG. 10 are diagrams illustrating various
`embodiments of video decoding architectures.
`FIG. 11 illustrates an embodiment of recursive coding unit
`(CU) structure.
`FIG. 12 illustrates an embodiment of prediction unit (PU)
`30 modes.
`FIG. 13 illustrates an embodiment of a binary tree, includ(cid:173)
`ing a modification thereof, employed for P slice encoding in
`one implementation and for both P and B slice encoding in
`another implementation.
`FIG. 14 illustrates an embodiment of a binary tree as may
`be employed for B slice encoding in one implementation and
`for both P and B slice encoding in another implementation.
`FIG. 15 illustrates an embodiment of a binary tree as may
`be employed for P slice encoding in conjunction with B slice
`40 encoding as performed in accordance with FIG. 12.
`FIG. 16A, FIG. l6B, FIG. 17A, and FIG. l7B, illustrate
`various embodiments of methods performed in accordance
`with video coding ( e.g., within one or more communication
`devices).
`
`The following standards/draft standards are hereby incor(cid:173)
`porated herein by reference in their entirety and are made part
`of the present U.S. Utility Patent Application for all purposes:
`1. "High efficiency video coding (HEVC) text specifica(cid:173)
`tion draft 6," Joint Collaborative Team on Video Coding (JCT- 25
`VC) ofITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11,
`7th Meeting: Geneva, CH, 21-30 Nov. 2011, Document:
`JCTVC-H1003, 259 pages.
`2. International Telecommunication Union, ITU-T, TELE(cid:173)
`COMMUNICATION STANDARDIZATION SECTOR OF
`ITU, H.264 (March 2010), SERIES H: AUDIOVISUAL
`AND MULTIMEDIA SYSTEMS, Infrastructure of audiovi(cid:173)
`sual services--Coding of moving video, Advanced video
`coding for generic audiovisual services, Recommendation
`ITU-T H.264, also alternatively referred to as International
`Telecomm ISO/IEC 14496-1 O-MPEG-4 Part 10, AVC (Ad(cid:173)
`vanced Video Coding), H.264/MPEG-4 Part 10 or AVC (Ad(cid:173)
`vanced Video Coding), ITU H.264/MPEG4-AVC, or equiva(cid:173)
`lent.
`
`35
`
`BACKGROUND OF THE INVENTION
`
`1. Technical Field of the Invention
`The invention relates generally to digital video processing; 45
`and, more particularly, it relates to signaling in accordance
`with such digital video processing.
`2. Description of Related Art
`Communication systems that operate to communicate
`digital media ( e.g., images, video, data, etc.) have been under 50
`continual development for many years. With respect to such
`communication systems employing some form of video data,
`a number of digital images are output or displayed at some
`frame rate ( e.g., frames per second) to effectuate a video
`signal suitable for output and consumption. Within many 55
`such communication systems operating using video data,
`there can be a trade-off between throughput ( e.g., number of
`image frames that may be transmitted from a first location to
`a second location) and video and/or image quality of the
`signal eventually to be output or displayed. The present art 60
`does not adequately or acceptably provide a means by which
`video data may be transmitted from a first location to a second
`location in accordance with providing an adequate or accept(cid:173)
`able video and/or image quality, ensuring a relatively low
`amount of overhead associated with the communications, 65
`relatively low complexity of the communication devices at
`respective ends of communication links, etc.
`
`DETAILED DESCRIPTION OF THE INVENTION
`
`Within many devices that use digital media such as digital
`video, respective images thereof, being digital in nature, are
`represented using pixels. Within certain communication sys(cid:173)
`tems, digital media can be transmitted from a first location to
`a second location at which such media can be output or
`displayed. The goal of digital communications systems,
`including those that operate to communicate digital video, is
`to transmit digital data from one location, or subsystem, to
`another either error free or with an acceptably low error rate.
`As shown in FIG. 1, data may be transmitted over a variety of
`communications channels in a wide variety of communica(cid:173)
`tion systems: magnetic media, wired, wireless, fiber, copper,
`and/or other types of media as well.
`FIG. 1 and FIG. 2 are diagrams illustrate various embodi(cid:173)
`ments of communication systems, 100 and 200, respectively.
`Referring to FIG. 1, this embodiment of a communication
`system 100 is a communication channel 199 that communi(cid:173)
`catively couples a communication device 110 (including a
`transmitter 112 having an encoder 114 and including a
`receiver 116 having a decoder 118) situated at one end of the
`
`

`

`Case 8:20-cv-00529 Document 1-9 Filed 03/13/20 Page 22 of 34 Page ID #:285
`
`US 9,332,283 B2
`
`3
`communication channel 199 to another communication
`device 120 (including a transmitter 126 having an encoder
`128 and including a receiver 122 having a decoder 124) at the
`other end of the communication channel 199. In some
`embodiments, either of the communication devices 110 and
`120 may only include a transmitter or a receiver. There are
`several different types of media by which the communication
`channel 199 may be implemented (e.g., a satellite communi(cid:173)
`cation channel 130 using satellite dishes 132 and 134, a
`wireless communication channel 140 using towers 142 and
`144 and/or local antennae 152 and 154, a wired communica(cid:173)
`tion channel 150, and/or a fiber-optic communication channel
`160 using electrical to optical (E/0) interface 162 and optical
`to electrical (0/E) interface 164)). In addition, more than one
`type of media may be implemented and interfaced together
`thereby forming the communication channel 199.
`It is noted that such communication devices 110 and/or 120
`may be stationary or mobile without departing from the scope
`and spirit of the invention. For example, either one or both of
`the communication devices 110 and 120 may be implemented 20
`in a fixed location or may be a mobile communication device
`with capability to associate with and/or communicate with
`more than one network access point ( e.g., different respective
`access points (APs) in the context ofa mobile communication
`system including one or more wireless local area networks 25
`(WLANs), different respective satellites in the context of a
`mobile communication system including one or more satel(cid:173)
`lite, or generally, different respective network access points in
`the context of a mobile communication system including one
`or more network access points by which communications 30
`may be effectuated with communication devices 110 and/or
`120.
`To reduce transmission errors that may undesirably be
`incurred within a communication system, error correction
`and channel coding schemes are often employed. Generally,
`these error correction and channel coding schemes involve
`the use of an encoder at the transmitter end of the communi(cid:173)
`cation channel 199 and a decoder at the receiver end of the
`communication channel 199.
`Any of various types of ECC codes described can be
`employed within any such desired communication system
`(e.g., including those variations described with respect to
`FIG. 1 ), any information storage device (e.g., hard disk drives
`(HDDs), network information storage devices and/or servers,
`etc.) or any application in which information encoding and/or
`decoding is desired.
`Generally speaking, when considering a communication
`system in which video data is communicated from one loca(cid:173)
`tion, or subsystem, to another, video data encoding may gen(cid:173)
`erally be viewed as being performed at a transmitting end of
`the communication channel 199, and video data decoding
`may generally be viewed as being performed at a receiving
`end of the communication channel 199.
`Also, while the embodiment of this diagram shows bi(cid:173)
`directional communication being capable between the com(cid:173)
`munication devices 110 and 120, it is of course noted that, in
`some embodiments, the communication device 110 may
`include only video data encoding capability, and the commu(cid:173)
`nication device 120 may include only video data decoding
`capability, or vice versa (e.g., in a uni-directional communi(cid:173)
`cation embodiment such as in accordance with a video broad(cid:173)
`cast embodiment).
`Referring to the communication system 200 of FIG. 2, at a
`transmitting end ofa communication channel 299, informa(cid:173)
`tion bits 201 ( e.g., corresponding particularly to video data in 65
`one embodiment) are provided to a transmitter 297 that is
`operable to perform encoding of these information bits 201
`
`4
`using an encoder and symbol mapper 220 (which may be
`viewed as being distinct functional blocks 222 and 224,
`respectively) thereby generating a sequence of discrete-val(cid:173)
`ued modulation symbols 203 that is provided to a transmit
`5 driver 230 that uses a DAC (Digital to Analog Converter) 232
`to generate a continuous-time transmit signal 204 and a trans(cid:173)
`mit filter 234 to generate a filtered, continuous-time transmit
`signal 205 that substantially comports with the communica(cid:173)
`tion channel 299. At a receiving end of the communication
`10 channel 299, continuous-time receive signal 206 is provided
`to an AFE (Analog Front End) 260 that includes a receive
`filter 262 (that generates a filtered, continuous-time receive
`signal 207) and an ADC (Analog to Digital Converter) 264
`(that generates discrete-time receive signals 208). A metric
`15 generator 270 calculates metrics 209 ( e.g., on either a symbol
`and/or bit basis) that are employed by a decoder 280 to make
`best estimates of the discrete-valued modulation symbols and
`information bits encoded therein 210.
`Within each of the transmitter 297 and the receiver 298, any
`desired integration of various components, blocks, functional
`blocks, circuitries, etc. Therein may be implemented. For
`example, this diagram shows a processing module 280a as
`including the encoder and symbol mapper 220 and all asso(cid:173)
`ciated, corresponding components therein, and a processing
`module 280 is shown as including the metric generator 270
`and the decoder 280 and all associated, corresponding com-
`ponents therein. Such processing modules 280a and 280b
`may be respective integrated circuits. Of course, other bound(cid:173)
`aries and groupings may alternatively be performed without
`departing from the scope and spirit of the invention. For
`example, all components within the transmitter 297 may be
`included within a first processing module or integrated cir(cid:173)
`cuit, and all components within the receiver 298 may be
`included within a second processing module or integrated
`35 circuit. Alternatively, any other combination of components
`within each of the transmitter 297 and the receiver 298 may be
`made in other embodiments.
`As with the previous embodiment, such a communication
`system 200 may be employed for the communication of video
`40 data is communicated from one location, or subsystem, to
`another ( e.g., from transmitter 297 to the receiver 298 via the
`communication channel 299).
`Digital image and/or video processing of digital images
`and/or media (including the respective images within a digital
`45 video signal) may be performed by any of the various devices
`depicted below in FIG. 3A-3H to allow a user to view such
`digital images and/or video. These various devices do not
`include an exhaustive list of devices in which the image
`and/or video processing described herein may be effectuated,
`50 and it is noted that any generic digital image and/or video
`processing device may be implemented to perform the pro(cid:173)
`cessing described herein without departing from the scope
`and spirit of the invention.
`FIG. 3A illustrates an embodiment ofa computer 301. The
`55 computer 301 can be a desktop computer, or an enterprise
`storage devices such a server, of a host computer that is
`attached to a storage array such as a redundant array of inde(cid:173)
`pendent disks (RAID) array, storage router, edge router, stor(cid:173)
`age switch and/or storage director. A user is able to view still
`60 digital images and/or video (e.g., a sequence of digital
`images) using the computer 301. Oftentimes, v