US006456663B1
`
`US 6,456,663 BI
`(10) Patent No:
`az United States Patent
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Sep. 24, 2002
`(45) Date of Patent:
`Kim
`
`
`
`
`
`
`
`
`
`(54) DCT DOMAIN DOWN CONVERSION
`SYSTEM THAT COMPENSATESFOR IDCT
`
`
`
`
`
`MISMATCH
`
`
`
`
`
`
`
`
`
`(75)
`Inventor: Hee-Yong Kim, Plainsboro, NJ (US)
`
`
`
`
`
`
`(73) Assignee: Matsushita Electric Industrial Co.,
`
`
`Ltd., Osaka (JP)
`
`
`
`Subject to anydisclaimer, the term ofthis
`
`
`
`
`
`
`
`
`
`
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`
`
`
`(*) Notice:
`
`
`
`
`
`
`(58) Field of Search
`
`
`
`
`
`
`
`(21) Appl. No.: 09/537,346
`
`og,
`Filed:
`Mar. 29, 2000
`(22)
`
`
`
`
`int. Cd oe esesceeeeeteneeeeetens HO4N 7/30
`SL)
`
`
`
`
`
`
`7
`(51)
`fi
`
`
`
`
`
`
`UWS. Ch occ 375/240.25; 375/240.24;
`(52)
`
`375/240.21
`
`
`375/240.25. 240.24
`—375/240 31: HOAN 7/30
`
`
`
`
`
`~~
`
`
`(56)
`
`
`
`
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`
`
`
`
`
`
`.. 348/390.1
`6/1992 Raychaudhuriet al.
`12/1996 Suzuki et al. vw... 714/800
`
`
`
`
`
`
`/1997 Haskell etal. .............. 341/200
`
`
`
`
`
`10/1998 Malladi et al.
`........ 375/240.03
`
`
`
`
`
`
`
`5,122,875 A
`5,590,139 A
`
`5,604,502 A
`
`5,818,532 A
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`............. 375/240.16
`1/2001 Kim etal.
`*
`6,175,592 Bl
`6,310,919 B1 * 10/2001 Florencio ............... 375/240. 16
`
`
`
`
`
`
`
`
`
`
`
`
`6,373,905 Bl
`*
`4/2002 Yasuda et al.
`oo . 375/340
`6,384,864 Bl
`*
`5/2002 Kim .......ee 348/441
`
`
`
`
`
`
`
`
`* cited by examiner
`
`
`
`Primary Examiner—Howard Britton
`
`
`
`
`(74) Attorney, Agent, or Firm—RatnerPrestia
`
`
`(67)
`ABSTRACT
`
`
`
`
`
`
`
`
`
`
`
`To reducealiasing during the down conversion and decoding
`of video signals that have been encoded according to the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`moving picture experts group (MPEG)standard, a discrcte
`
`
`
`
`
`
`
`
`
`cosine transform (DCT) domain filter is applied to the
`
`
`
`
`
`
`
`unquantized DCT coefficient values. Also, partly because
`the inverse discrete transform (IDCT) operation of the
`
`
`
`
`
`
`
`
`standard
`may
`implemented,
`mismatc
`be
`
`
`
`
`
`
`IDCT
`MPEG
`dard
`may
`impl
`mi
`be
`d,
`h
`
`
`
`
`
`control processing is implemented. Concurrent implemen-
`
`
`
`
`
`
`
`
`
`
`tation of the IDCT mismatch control process and the DCT
`
`
`
`
`
`
`
`
`domain filter does not consistently produce the highest
`quality picture. Thus, the current invention is related to a
`
`
`
`
`
`
`
`robust DCT domain filter designed to maintain the higher
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`quality in downconverted images. The DCT domain filter
`
`
`
`
`
`
`
`
`sets the filter coefficient corresponding to the highest fre-
`
`
`
`
`
`
`
`quency band to unity to prevent modification of any coef-
`ficient value that has been modified by the IDCT mismatch
`
`
`
`
`
`
`
`
`operation.
`
`
`
`
`
`13 Claims, 4 Drawing Sheets
`
`
`
`
`200
`
`—_
`
`
`
`HD
`
`BIT STREAM
`
`
`
`
`2~(®)
`
`
`6
`
`28
`
`
`
`
`
`32
`
`
`
`
`
`
`
`20
`
` R/L
`
`DECODER
`
`Mv
`
`
`
`POLY
`PHASE
`
`LLP SAMPLE
`
`
`
`
`
`
`
`
`
`
`
`
`
`HALF PIXEL
`GENERATOR
`
`DCT COEFFICIENT
`
`
`/~ PROCESSOR A.
`36
`33
`
`
`3
`(
`
`4
`=]
`
`
`IDCT
`
`
`i DOMAIN i IDCT i
`MISMATCH
`
`
`
`
`FILTER
`
`
`CONTROL
`
`
`
`
`
`()-—
`
`62
`
`
`
`DOWN
`SAMPLE
`
`
`
`CONTROLLER
`
`
`
`
`
`
`
`
`
`
`
`emane||
`
`ramen
`
`1
`
`MOTION
`
`
`40 REFERENCE|]Bigcx
`DISPLAY CONVERSION
`
`
`
`
`FRAME
`
`HPF
`Ll Ver
`|
`2
`
`b—>1 FILTER
`FLreRy
`
`!
`DISPLAY
`
`Loe
`RE-ORDER
`
`
`
`
`
`
`
`
`.
`
`O
`
`62
`
`|
`
`Page 1 of 12
`
`GOOGLEEXHIBIT 1013
`
`Page 1 of 12
`
`GOOGLE EXHIBIT 1013
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Sep. 24, 2002
`
`
`
`
`
`Sheet 1 of 4
`
`
`
`US 6,456,663 B1
`
`0c
`
`
`
`NOILVWILS3NOILOW
`
`GNYNOILOIGIud
`
`NOLLYSNIdW00
`
`(LUWHOld)|ls
`
`
`
`WVAYLS
`
`
`
`lla©
`
`bl
`
`LNdLAG
`
`44444
`
`Lod
`
`HOLVSIN
`
`TOULNOD
`
`ASYAANI
`
`YAZILNVYAD
`
`Y3ZILNVND
`
`OJGIA
`
`LAN
`
`TWNOIS
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 2 of 12
`
`Page 2 of 12
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Sep. 24, 2002
`
`Sheet 2 of 4
`
`US 6,456,663 B1
`
`tNa1914$3001900900¢
`
`med:Wes.
`
`
`NOISUIANOOAVIdSIQ49078(A)On
`|YatqdWAITl4«—#,||anddA—
`!4304034
`
`+HYITIOWLNOD
`
`
`
`TXldJWH
`
`YOLVYANI
`
`a-W)
`
`°ASVHd
`
`AW
`
`NOLOaE
`
`¢
`
`AW1dSI0
`
`TdWys49YASHrawIdoaidcSWYIHLSLid
`NMOG1GVWdH
`
`TOUINOD
`
`¥voges90ud
`
`8¢
`
`Page 3 of 12
`
`JansenleA10dTuo)«=~
`
`Page 3 of 12
`
`
`
`
`

`

`U.S. Patent
`
`
`
`
`Sep. 24, 2002
`
`
`
`
`
`Sheet 3 of 4
`
`
`US 6,456,663 B1
`
`
`
`
`
`
`
`
`PERFORM TWO DIMENSIONAL SUMMATION
`
`
`
`OF DCT COEFFICIENTS:
`
`
`ON
`M
`
`
`& F(mn)
`SUM=¥
`
`m=0 n=0
`
`
`
`
`
`42
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`DCT COEFFICIENTS
`
`
`
`
`
` REPLACE F(M.N)
`
`
`
`
`| WITH F(M.N) 1
`
`
`
`
`
`
`
`
`
`
`WITH F(M,N) + 1
`
`REPLACE F(M,N)
`
`
`
`
`
`
`
`PROVIDE RESULTANT
`
`
`DCT COEFFICIENTS
`
`
`TO DCT COEFFICIENT
`PROCESSOR
`
`
`
`
`
`
`
`
`
`
`02
`
`
`
`
`
`
`FIG. 3
`
`Page 4 of 12
`
`Page 4 of 12
`
`

`

`U.S. Patent
`
`
`
`
`Sep. 24, 2002
`
`
`
`
`
`Sheet 4 of 4
`
`
`US 6,456,663 B1
`
`
`
`I!
`
`
`
`
`FIRSTOUTPUT PIXEL
`X7 Xg X5 K4 Xg X9 Xy X! Xq Ky Xo X3 X4 X5 X6 X7 ! X7 Xp X5 X4 X3 Xo Xy Xg
`
`
`
`
`
`
`
`
`ALG Bo MN CATiteee
`
`
`
`
`
`
`
`
`
`Meg yg hia hyp hg hg hg | hy hg hs hg hg hp hy hg
`
`!
`SECOND OUTPUTPXEL
`
`
`
`
`
`
`X7 Xe X5 Xq X3 Xo Xq Xqt Xq Xq Xo Xg Xq Xp AG Xz | Xz Xs Xs Xq Xq Xp Ky Xp
`MENG DO MES OS AT NNT ACIS NDT eryBe
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`mi
`I
`
`
`
`|I
`
`|l
`
`I
`
`x2
`
`III
`
`|
`
`
`
`
`
`
`
`
`
`
`
`
`
`DOWN
`
`
`
`ee ee ee
`
`
`
`
`
`
`
`x0
`
`
`FIG. 4
`
`
`(PRIOR ART)
` PERFORM
`CONVERSION?
`
`
`
`(BLOCK.
`
`
`
`
`
`/ MODIFEDBY
`
`SI. MISMATCH 77°77
`
`
`
`
`
`CONTROL? 7
`
`
`
`SET H(N) 101
`2 ee ee ee
`
`
`
`
`
`PERFORM DCT DOMAIN
`
`
`LOW PASS FILTERING
`
`
`
`ON THE DCT COEFFICIENTS
`
`
`
`
`
`
`
`xi
`
`Page 5 of 12
`
`Page 5 of 12
`
`

`

`
`1
`DCT DOMAIN DOWN CONVERSION
`
`
`
`
`SYSTEM THAT COMPENSATES FOR IDCT
`
`
`
`
`MISMATCH
`
`FIELD OF THE INVENTION
`
`
`
`This invention relates to a decoder which converts and
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`formats an encoded high resolution video signal, e.g.,
`
`
`
`
`
`
`
`
`MPEG-2 cncoded video signal, and more specifically to a
`
`
`
`
`
`
`
`method and apparatus for adaptively compensating for
`encoder/decoder mismatch.
`
`
`BACKGROUNDOF THE INVENTION
`
`
`
`
`
`
`
`
`
`
`
`
`In the United States a standard has been proposed for
`
`
`
`
`
`
`
`digitally encoded high definition television signals (HDTV).
`
`
`
`
`
`
`
`
`
`A portion of this standard is essentially the same as the
`
`
`
`
`
`
`MPEG-2 standard, proposed by the Moving Picture Experts
`
`
`
`
`
`
`
`Group (MPEG)ofthe International Organization for Stan-
`
`
`
`
`
`
`dardization (SO). The standard is described in an Interna-
`
`
`
`
`
`
`tional Standard (IS) publication entitled, “Information
`
`
`
`
`
`
`Technology—Generic Coding of Moving Pictures and Asso-
`ciated Audio, Recommendation H.626”, ISO/IEC 13818-2,
`
`
`
`
`
`
`IS, 11/94 which is available from the ISO and which is
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`hereby incorporated by reference for its teaching on the
`
`
`
`
`
`MPEG-2 digital video coding standard.
`
`
`
`
`
`
`
`The MPEG-2 standard is actually several different stan-
`
`
`
`
`
`
`
`
`dards. In MPEG-2, several different profiles are defined,
`
`
`
`
`
`
`each corresponding to a different level of complexity of the
`
`
`
`
`
`
`
`
`
`encoded image. For each profile, different levels are defined,
`
`
`
`
`
`
`each level corresponding to a different image resolution.
`Oneof the MPEG-2 standards, known as Main Profile, Main
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Level is intended for coding video signals conforming to
`
`
`
`
`
`
`
`existing television standards (i.e., NTSC and PAL). Another
`
`
`
`
`
`
`
`
`standard, known as Main Profile, High Level, is intended for
`
`
`
`
`
`
`coding high-definition television images. Images encoded
`
`
`
`
`
`
`
`
`according to the Main Profile, High Level standard may
`
`
`
`
`
`
`
`
`
`have as many as 1,152 lines per image frame and 1,920
`
`
`
`pixels per line.
`The Main Profile, Main Level standard, on the other hand,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`defines a maximumpicture size of 720 pixels per line and
`
`
`
`
`
`
`
`
`
`576 lines per frame. At a frame rate of 30 frames per second,
`
`
`
`
`
`
`
`
`signals encoded according to this standard have a data rate
`
`
`
`
`
`
`
`of 720x576x30 or 12,441,600 pixels per second. By
`
`
`
`
`
`
`
`contrast, images encoded according to the Main Profile,
`
`
`
`
`
`
`
`
`
`High Level standard have a maximumdata rate of 1,152x
`
`
`
`
`
`
`
`
`1,920x30 or 66,355,200 pixels per second. This data rate is
`
`
`
`
`
`
`
`
`
`
`
`more than five times the data rate of image data encoded
`
`
`
`
`
`
`
`
`according to the Main Profile, Main Level standard. The
`
`
`
`
`
`
`
`
`standard proposed for HDTV encoding in the United States
`
`
`
`
`
`
`
`
`
`is a subsetof this standard, having as many as 1,080 lines per 5
`
`
`
`
`
`
`
`
`
`
`frame, 1,920 pixels per line and a maximum framerate, for
`
`
`
`
`
`
`
`
`
`this frame size, of 30 frames per second. The maximum data
`
`
`
`
`
`
`
`
`
`
`rate for this proposed standardis still far greater than the
`maximum data rate for the Main Profile, Main Level stan-
`
`
`
`
`
`
`
`
`
`
`dard.
`
`
`
`
`
`
`
`
`The MPEG-2 standard defines a complex syntax which
`contains a mixture of data and control information. Some of
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`this control information is used to enable signals having
`
`
`
`
`
`
`
`
`several different formats to be covered by the standard.
`
`
`
`
`
`
`
`These formats define images having differing numbers of
`
`
`
`
`
`
`
`
`picture elements (pixels) per line, differing numbersoflines
`
`
`
`
`
`
`
`
`per frameor field, and differing numbers of framesorfields
`
`
`
`
`
`
`
`
`
`per second. In addition, the basic syntax of the MPEG-2
`
`
`
`
`
`
`
`Main Profile defines the compressed MPEG-2 bit stream
`
`
`
`
`
`
`
`representing a sequence of images in five layers,
`the
`
`
`
`
`
`
`
`
`
`sequencelayer, the group ofpictures layer, the picture layer,
`
`
`
`
`
`
`
`
`
`the slice layer and the macroblock layer. Each of these layers
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`2
`
`
`
`
`
`
`
`is introduced with control information. Finally, other control
`
`
`
`
`
`
`information, also known as side information, (e.g. frame
`
`
`
`
`
`
`type, macroblock pattern, image motion vectors, coefficient
`
`
`
`
`
`zig-zag patterns and dequantization information) is inter-
`
`
`
`
`
`spersed throughout the encoded bit stream.
`
`
`
`
`
`Implementation of this standard in television studios and
`
`
`
`
`
`
`in viewer’s homes is expected to be incremental. At least
`
`
`
`
`
`
`
`until the television studios provide a large amount of pro-
`
`
`
`
`
`
`
`gramming in HDTV format, viewersare likely to retain their
`
`
`
`
`
`
`standard definition television (SDTV) receivers but may
`
`
`
`
`
`
`want to view HDTV programming in SDTV format. Thus,
`
`
`
`
`
`
`
`the operation of decoding the encoded bitstream may
`
`
`
`
`
`
`include the process of down conversion. Down conversion
`
`
`
`
`
`
`
`converts a high definition input picture into a lower resolu-
`
`
`
`
`
`
`
`tion picture tor display on a lowerresolution monitor. Down
`
`
`
`
`
`
`
`conversion of high resolution Main Profile, High Level
`
`
`
`
`
`
`
`pictures to Main Profile, Main Levelpictures, or other lower
`
`
`
`
`
`
`resolution picture formats, has gained increased importance
`
`
`
`
`
`
`for reducing implementation costs of HDTV. Down conver-
`
`
`
`
`
`
`sion allows replacement of expensive high definition moni-
`
`
`
`
`
`
`
`
`tors used with Main Profile, High Level encoded pictures
`
`
`
`
`
`
`
`with inexpensive existing monitors that have a lower picture
`
`
`
`
`
`
`
`resolution to support, for example, Main Profile, Main Level
`
`
`
`
`
`encoded pictures, such as NTSC or PAL.
`
`
`
`
`
`
`
`
`Processing of video signals in the MPEG-2 standard
`
`
`
`
`
`
`
`
`includes converting the video signals between the spatial
`
`
`
`
`
`
`
`
`domain and the frequency domain using discrete cosine
`
`
`
`
`
`
`
`transforms (DCTs) and inverse discrete cosine transforms
`
`
`
`
`
`
`
`
`(IDC'ls) during the respective encoding and decoding stages
`
`
`
`
`
`
`
`
`
`of the process. When the DCT used by an encoder and the
`
`
`
`
`
`
`IDCT used by a decoder have different implementations, a
`
`
`
`
`
`
`
`difference may occurin the reconstructed pixels between the
`
`
`
`
`
`
`
`
`encoder and the decoder. This difference may accumulate
`
`
`
`
`
`
`
`
`and becomevisible in the decoded picture. This distortionis
`ey} wn
`5 called IDCT mismatch distortion because the visible distor-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`tion in the decodedpicture is caused by different DCT/ADCT
`
`
`
`
`
`
`
`implementations in the encoder and decoder. IDCT mis-
`
`
`
`
`
`
`
`
`match is a serious problem for high quality coding schemes
`
`
`
`
`
`
`
`such as those conforming to the MPEG-1 and MPEG-2
`
`
`
`
`
`
`
`standards. Thus, in order to achieve high coding quality,
`IDCT mismatch must be controlled.
`
`
`
`
`
`
`
`
`
`
`
`IDCT mismatch occurs whenthe result of an IDCT is very
`
`
`
`
`
`
`
`
`close to a half integer. A slight difference between the
`encoder and decoder can result in two different rounded
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`integer valucs. This difference is most
`likely to cause
`
`
`
`
`
`
`
`
`problems whenthe values of the IDCT results are close to
`
`
`
`
`
`
`
`
`
`
`a half integer(e.g., 1.5). When the IDCTresults are rounded
`
`
`
`
`
`
`
`
`
`to the nearest integer, one implementation may round up,
`
`
`
`
`
`
`
`
`because its resultant value is onlyslightly greater than the
`
`
`
`
`
`
`
`
`
`valuc of the half integer, while the other implementation
`
`
`
`
`
`
`
`
`rounds down, because its resultant value is only slightly less
`
`
`
`
`
`
`
`
`than the value of the half integer. Accordingly, if a decoder
`
`
`
`
`
`
`
`
`
`that rounds up processes a signal from an encoder that
`
`
`
`
`
`
`
`
`rounds down or vice versa, IDCT mismatch errors may
`
`
`
`
`
`
`
`occur. When a decoded frame containing errors is used to
`
`
`
`
`
`
`
`
`decode a sequence of predicted frames,
`the error may
`
`
`
`
`
`
`
`
`
`become morevisible with each predicted frame that is based
`
`
`
`
`
`
`
`
`on the erroneous frame. One approach to control IDCT
`
`
`
`
`
`
`mismatch includes oddification methods. The processing of
`
`
`
`
`
`
`oddification typically involves setting specific coefficients to
`
`
`
`
`
`
`
`
`an odd value.In this approach, the reconstructed or dequan-
`tized DCT data is oddified at the decoder before the IDCT
`
`
`
`
`
`
`
`
`
`step.
`
`
`
`>
`
`
`
`
`
`
`
`
`
`SUMMARYOF THE INVENTION
`
`
`
`
`
`
`
`
`
`
`The present invention provides an apparatus for use in a
`
`
`
`
`
`
`
`
`
`video decoder which decodes digital video signals that have
`
`60
`
`
`
`US 6,456,663 B1
`
`
`
`
`
`
`
`
`
`
`
`
`10
`
`i}wn
`
`40
`
`
`
`Page 6 of 12
`
`Page 6 of 12
`
`

`

`
`
`US 6,456,663 B1
`
`
`4
`
`
`
`
`
`
`
`The summation circuit 2, determines the pixel-by-pixel
`
`
`
`
`
`
`
`
`difference between the current video input signal picture
`
`
`
`
`
`
`block and ils corresponding motion compensated prediction
`
`
`
`
`
`
`
`block 4. The resulting blocksof differences 6, are coupled to
`
`
`
`
`
`
`
`
`the discrete cosine transform (DCT) processor 8. The DCT
`
`
`
`
`
`
`processor 8, applies orthogonal transform processing to the
`
`
`
`
`
`
`
`difference blocks 6. The resulting blocks of frequency
`
`
`
`
`
`
`
`domain transformcoefficients are provided to the quantizer
`
`
`
`
`
`
`
`
`
`10. The quantizer 10 quantizes the blocks of transform
`
`
`
`
`
`
`
`coefficients to reduce the number ofbits used to represent
`
`
`
`
`
`
`
`the transform coefficients. The variable-length coder 12
`
`
`
`
`
`
`
`subjects the blocks of quantized transform coefficients from
`
`
`
`
`
`
`
`the quantizer 10 to variable-length coding, such as Huff-
`
`
`
`
`
`
`
`
`mann coding and run-length coding. The resulting blocks of
`
`
`
`
`
`
`
`
`coded transform coefficients, along with motion vectors, are
`
`
`
`
`
`
`
`
`
`
`then fed as a bit stream, via the output buffer 14, to a digital
`transmission medium.
`
`
`
`
`
`
`
`
`
`
`
`Acontrol signal indicating the numberofbits stored in the
`
`
`
`
`
`
`
`
`
`output buffer 14 is fed back to the quantizer 10. The
`
`
`
`
`
`
`
`quantizer 10 adjusts the quantizing step size in responsc to
`
`
`
`
`
`
`
`
`
`
`the control signal to prevent
`the output buffer 14 from
`
`
`
`
`
`
`overflowing or underflowing and also to maintain a required
`
`
`
`
`
`
`
`
`bit rate. Increasing or decreasing the quantizing step size
`
`
`
`
`
`
`
`respectively decreases or increases the numberofbits fed
`
`
`
`
`
`into the output buffer 14.
`
`
`
`
`
`
`The block of quantized transform coefficients provided by
`
`
`
`
`
`
`
`
`
`quantizer 10, is also coupled to the inverse quantizer 16. The
`
`
`
`
`
`inverse quantizer 16 performs proccssing complementary to
`
`
`
`
`
`
`
`he quantizing processing performed by the quantizer 10.
`
`
`
`
`
`
`
`The inverse quantized data is subjected to mismatch control
`
`
`
`
`
`
`
`17, and the resulting block of transform coefficients is fed to
`
`
`
`
`
`
`
`
`he inverse discrete cosine transform (IDCT) processor 18,
`
`
`
`
`
`whereit is inverscly orthogonally transformed by processing
`
`
`
`
`
`
`complementary to the orthogonal transform processing per-
`
`
`
`
`
`
`ormedby the discrete cosine transform processor8.
`
`
`
`
`
`
`
`The resulting restored spatial domain difference block is
`
`
`
`
`
`
`
`
`coupled to the summationcircuit 20. The summation circuit
`
`
`
`
`
`
`
`20 is also coupled to receive the motion compensated
`
`
`
`
`
`
`
`
`prediction block 4 for the current video input signal picture
`
`
`
`
`
`
`
`
`block from the motion estimation, prediction, and compen-
`
`
`
`
`
`
`
`
`sation circuit 22. The summation circuit 20 performs pixcl-
`
`
`
`
`
`
`
`by-pixel addition betweenthe restored difference block from
`he inverse discrete cosine transform circuit 18 and the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`matching motion compensated prediction block 4 from the
`
`
`
`
`
`
`motion estimation, prediction, and compensation circuit 22
`
`
`
`
`
`
`
`o provide a reconstructed picture block to the motion
`
`
`
`
`
`
`estimation, prediction, and compensation circuit 22.
`
`
`
`
`
`
`FIG. 2 is a block diagram illustrating an exemplary
`
`
`
`
`
`
`configuration of a MPEG decoding and decompression
`
`
`
`
`
`
`system incorporating down conversion. This embodiment of
`
`
`
`
`
`
`
`
`a decoding and decompression system 200 includes a vari-
`
`
`
`
`
`
`
`able length decoder (VLD) 28, a run-length (R/L) decoder
`
`
`
`
`
`
`
`
`30, an inverse quantizer 32, IDCT mismatch control 33, a
`
`
`
`
`
`
`
`
`controller 40, and a DCTcocfficicnt processor 34. As shown
`
`
`
`
`
`
`
`in FIG. 2, the DCT coefficient processor 34 comprises a
`DCT domain filter 36, and an inverse discrete cosine trans-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`form (IDCT) processor 38. In an alternate embodiment of a
`
`
`
`
`
`
`decoding and decompression system without down
`
`
`
`
`
`
`
`conversion, the DCT coefficient processor comprises only
`
`
`
`
`
`
`
`
`the IDCT processor. Note that, for completeness, FIG. 2
`
`
`
`
`
`
`
`depicts the primary components of a MPEG decoding sys-
`
`
`
`
`
`
`tem incorporating down conversion. A more detailed
`
`
`
`
`
`
`
`description of this decoding processor may be found in
`
`
`
`
`
`
`pending U.S. patent application No. 09/169,790.
`
`
`
`
`
`
`
`The digital television system may receive either high-
`
`
`
`
`
`
`definition television (HDTV)signals, that needto befiltered
`
`
`
`
`
`
`
`ey}wn
`
`
`
`40
`
`
`
`
`
`
`
`
`
`60
`
`
`
`
`
`
`
`
`
`3
`
`
`
`
`
`
`
`
`been encodedinto frequency domain coefficient values. The
`
`
`
`
`
`
`apparatus comprises a mismatch control processor, a fre-
`
`
`
`
`
`
`
`quency domain filter having filter coefficients corresponding
`
`
`
`
`
`
`
`
`to frequency bands, and an inverse frequency domaintrans-
`
`
`
`
`
`
`
`form processor. The mismatch control processor is coupled
`
`
`
`
`
`
`
`
`to receive the frequency domain coefficient values and to
`
`
`
`
`
`
`
`process the frequency domain cocfficicnt valucs according
`
`
`
`
`
`
`
`to a mismatch control algorithm to produce processed fre-
`
`
`
`
`
`
`
`quency domain coefficient values. The frequency domain
`
`
`
`
`
`
`
`
`filter is coupled to receive the processed frequency domain
`
`
`
`
`
`
`
`coefficient values and to provide lowpass filtered frequency
`
`
`
`
`
`
`domain cocfficicnt valucs. If down conversion is performed,
`
`
`
`
`
`
`
`the frequency domainfilter coefficient corresponding to the
`
`
`
`
`
`
`
`
`highest frequency bandis set to 1 at least for image blocks
`
`
`
`
`
`
`
`
`tha have been modified by the mismatch control processor.
`
`
`
`
`
`
`
`The inverse frequency domain transform processor
`is
`
`
`
`
`
`
`
`
`coupled to the frequency domainfilter for transforming the
`
`
`
`
`
`
`
`output coefficient values provided by the frequency domain
`
`
`
`
`
`
`filter into spatial domain picture elements.
`
`
`
`
`
`
`
`
`
`
`It is to be understood that both the foregoing gencral
`
`
`
`
`
`
`
`description and the following detailed description are
`
`
`
`
`
`
`
`
`exemplary, but are not restrictive, of the invention.
`BRIEF DESCRIPTION OF THE DRAWING
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The invention is best understood from the following
`
`
`
`
`
`
`
`
`detailed description when read in connection with the
`
`
`
`
`
`
`accompanying drawing. It is emphasized that, according to
`
`
`
`
`
`
`
`
`commonpractice, the various features of the drawing are not
`
`
`
`
`
`
`
`
`
`to scale. On the contrary,
`the dimensions of the various
`
`
`
`
`
`
`
`features are arbitrarily expanded or reduced for clarity.
`
`
`
`
`
`
`
`Included in the drawing are the following figures:
`
`
`
`
`
`
`
`
`FIG. 1 (prior art)
`is a block diagram illustrating an
`
`
`
`
`
`
`exemplary configuration of a Moving Picture Experts Group
`
`
`
`
`
`(MPEG) encoding and compression system;
`
`
`
`
`
`FIG. 2 is a block diagram illustrating an exemplary
`
`
`
`
`
`configuration of a MPEG decoding and decompression
`
`
`
`
`system incorporating down conversion;
`
`
`
`
`
`FIG. 3 is a flow diagram illustrating an exemplary IDCT
`
`
`
`
`
`mismatch control process in MPEG;and
`
`
`
`
`
`
`
`
`
`FIG. 4 (priorart) illustrates the multiplication pairs for the
`
`
`
`
`
`
`
`
`
`first and second output pixel values of a block mirrorfilter.
`
`
`
`
`
`FIG. 5 is a flow diagram of an exemplary embodiment of
`
`
`
`
`
`
`
`
`
`a DCT lowpassfilter in accordance with the invention;
`DETAILED DESCRIPTION OF THE
`
`
`INVENTION
`
`
`
`
`
`
`
`
`
`
`
`Although illustrated and described above with reference
`
`
`
`
`
`
`
`to certain specific embodiments,
`the present invention is
`neverthcless not intended to be limited to the details shown.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Rather, various modifications may be made in the details
`
`
`
`
`
`
`
`
`
`within the scope and range of equivalents of the claims and
`
`
`
`
`
`without departing from the invention.
`
`
`
`
`
`
`
`FIG. 1 is a block diagram illustrating an exemplary
`
`
`
`
`
`
`configuration of a Moving Picture Experts Group (MPEG)
`
`
`
`
`encoding and compression system.
`
`
`
`
`
`
`
`The system shown in FIG. 1 compresses each picture of
`
`
`
`
`
`
`
`
`
`the video input signal, block-by-block, until all the blocks
`
`
`
`
`
`
`
`
`constituting the picture have been processed. A block may
`
`
`
`
`
`
`
`comprise a group of 8x8 pixels and a macroblock may
`
`
`
`
`
`
`
`comprise a group of 16x16 luminescence pixels and two to
`
`
`
`
`
`
`
`four 8x8 blocks of chrominance pixcls. A current macrob-
`
`
`
`
`
`
`
`lock is fed into motion estimation block 22 to generate a
`
`
`
`
`
`
`motion estimation based on a previous reference picture.
`
`
`
`
`
`
`
`
`The summation circuit 2 is coupled to receive both the video
`
`
`
`
`
`
`
`
`input signal and the motion compensated predictionsignal 4.
`
`Page 7 of 12
`
`Page 7 of 12
`
`

`

`
`6
`IDCT mismatch control. Thus it is desirable to reduce the
`
`
`
`
`
`
`
`
`
`
`
`
`
`possible occurrence of half integer resultant values.
`
`
`
`
`
`
`FIG.3 is a flow diagram illustrating an exemplary IDCT
`
`
`
`
`
`
`
`mismatch control process in MPEG. DCTcoefficients as
`
`
`
`
`
`
`
`
`provided by the inverse quantizer 32 are subjected to a
`
`
`
`
`
`
`
`summation process in step 42. This summation proccss is
`
`
`
`
`
`
`
`typically performed on a block of 8x8 DCT coefficients. The
`
`
`
`
`
`
`
`
`summation process in step 42 is in accordance with the
`
`
`following formula.
`MON
`Sum= 91>) Fm)
`m=0 n=0
`
`
`5
`
`
`
`
`
`
`
`
`
`and downsampled before they can be displayed on the
`
`
`
`
`
`
`viewer’s standard definition television (SDTV) monitor, or
`
`
`
`
`
`
`
`
`SDTYsignals that may be displayed on the SDTV monitor.
`Controller 40 determines whether the DCT coefficients are
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`to be downsampled and generates a control signal 62.
`
`
`
`
`
`
`
`Control signal 62 is provided to switches 41 and 45, and to
`
`
`
`
`
`
`
`
`the DCT coefficient processor 34. For example, when an
`
`
`
`
`
`
`
`HDTVsignal is received, controller 40 provides control
`
`
`
`
`
`
`
`
`signal 62 such that switch 41 is open and switch 45 provides
`
`
`
`
`
`
`
`
`upsampled data to the half pixel generator (i.e., switch 45 is
`
`
`
`
`
`
`
`
`
`
`
`in the up position in FIG. 2). Control signal 62 is also
`
`
`
`
`
`
`
`
`provided to the DCT coefficient processor 34 such that the
`
`
`
`
`
`
`
`
`
`
`DCTcoefficients of each block are lowpassfiltered in the
`
`
`
`
`
`
`
`DCT domain during HDTVreception, before conversion to
`
`
`
`the spatial domain.
`
`
`
`
`
`
`
`When SDTVsignals are received, no down conversion or
`
`
`
`
`
`
`
`
`filtering is needed as these signals may be decoded and
`
`
`
`
`
`
`
`F(m,n) represents a two dimensional matrix of DCT
`
`
`
`
`
`
`
`
`displayed on the SDTV monitor. In this instance, the con-
`
`
`
`
`
`coefficients located by indices m and n. M is the highest
`
`
`
`
`
`
`
`troller 40 provides control signal 62 such that switch 41 is
`
`
`
`
`
`
`
`
`value of the index m, andNis the highest value of the index
`
`
`
`
`
`
`
`
`
`closed and switch 45 provides motionblock data to the half,
`
`
`
`
`
`
`
`
`
`n. At step 44, it is determined if the value produced by the
`
`
`
`
`
`
`
`
`pixel generator (i.e., switch 45 is in the lower position in
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`summation process of step 42 is even or odd. If the sum-
`FIG. 2), thus bypassing the downsampling and upsampling
`
`
`
`
`
`
`
`
`
`mation valueis odd, the DCTcoefficients are provided to the
`
`
`
`
`
`
`
`
`operations. The controller 40 also controls the DCT coeffi-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`cient processor 34 to bypass the DCT domain filter when
`DCT coefficient processor 34 as provided by the inverse
`
`
`
`
`
`
`
`
`
`
`
`
`
`decoding the SDTV signals.
`quantizer 32. If, however, the value of the summation is
`
`
`
`
`
`
`
`
`Processor 34 may also monitor the IDCT mismatch
`
`
`
`
`
`
`
`
`
`even, then at step 46 it
`is determined if the value of a
`
`
`
`
`
`
`
`control processor 33 to determine which blocks of DCT
`
`
`
`
`
`
`particular coefficient, F(M,N) is even or odd. If F(M,N)is
`
`
`
`
`
`
`
`
`coefficients are modified by the processor 33 and which
`
`
`
`
`
`
`
`
`
`the value of F(M,N) is replaced with the value
`even,
`
`
`
`
`
`
`
`
`
`blocks are not modified. The processor 34 then uses this
`
`
`
`
`
`
`
`
`
`{F(M,N)+1} at step 48. Then the DCT coefficients with the
`
`
`
`
`
`
`
`
`information to control the value of the highest frequency 3
`
`
`
`
`
`
`
`
`replacement value are provided, at step 52,
`to the DCT
`filter coefficient of the DCT domain filter 36, as described
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`coefficient processor 34. If the value of F(M,N)is odd, the
`
`
`
`
`
`
`
`below. According to this alternate embodiment of the
`
`
`
`
`
`
`
`value of F(M,N)is replaced with the value {F(M,N)-1} at
`
`
`
`
`
`
`
`
`invention, the highest trequencyfilter coefficient of the DCT
`
`
`
`
`
`
`
`
`
`step 50. This is equivalent to toggling theleast significantbit
`
`
`
`
`
`
`
`
`
`
`
`domain filter 36 is set to unity only when the filter 36 is
`
`
`
`
`
`
`
`(LSB)of the coefficient F(M,N). Then the DCTcoefficients
`
`
`
`
`
`
`
`
`processing a block that was modified by the mismatch
`
`
`
`
`
`
`
`
`
`
`~ with the replacement value are provided to the DCTcoet-
`
`
`
`
`
`
`
`
`control processor 33. As a further refinement of this
`
`
`
`ficient processor 34.
`
`
`
`
`
`
`
`embodiment, the highest frequency filter coefficient of the
`
`
`
`
`
`
`
`
`DCYdomainfilter may be set to unity only when processing
`
`
`
`
`
`
`the exemplary
`When down conversion is performed,
`the row ofcoefficients in the modified block that includesthe
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`embodimentof the DCT coefficient processor 34 as depicted
`
`
`
`
`modified coefficient value ['(M,N).
`
`
`
`
`
`
`
`
`in FIG. 2 comprises a DCT domainfilter 36 and an IDCT
`
`
`
`
`
`
`
`
`In operation,
`the encoded bit-stream is received and
`
`
`
`
`
`
`
`
`processor 38. The derivation and advantages of using the
`
`
`
`
`
`
`decoded by VLD 28. In addition to header information used
`
`
`
`
`
`
`
`
`DCT domainfilter are described in an application for patent,
`
`
`
`
`
`
`
`
`
`by digital television system, the VLD 28 provides run length
`DOWN CONVERSION SYSTEM USING A PRE-
`
`
`
`
`
`encoded DCT coefficients for each block and macroblock,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`DECIMATIONFILTER, Ser. No. 09/169,790. Briefly, the
`and motion vector information. The DCT coefficients are run
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`~ DCT domainfilter 36, which processes the DCT coefficients
`
`
`
`
`
`
`
`
`length decoded in the R/L decader 30 and inverse quantized
`
`
`
`
`
`
`
`in the frequency domain, is an alternative to implementing
`
`
`
`
`by the inverse quantizer 32.
`
`
`
`
`
`
`
`
`
`a lowpassfilter in the spatial domain. For example, lowpass
`
`
`
`
`
`
`
`The inverse quantizer 32 provides the DCT coefficients to
`
`
`
`
`
`
`
`the IDCT mismatch controller 33. The IDCT mismatch
`filtering in the spatial domain is accomplished in the fre-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`quency domain by multiplying the DCT coefficients by
`
`
`
`
`
`
`
`controller 33 provides DCTcoefficients to the DCTfilter 36
`
`
`
`
`
`
`
`weighting coefficients prior to performing the IDCTprocess.
`
`
`
`
`
`
`
`
`which may perform a lowpass filtering in the frequency
`
`
`
`
`
`
`
`In a mathematical illustration, spatial values, x(n), can be
`
`
`
`
`
`
`
`domain by weighting the DCT coefficients with predeter-
`
`
`
`
`
`
`
`obtained by the IDCT process described by the following
`
`
`
`
`
`
`
`
`mined filter coefficient values before providing them to the
`
`cquation:
`
`
`
`
`
`
`
`
`IDCT processor 38. The IDCT processor 38 converts the
`
`
`
`
`
`
`
`
`filtered DCT coefficients into spatial pixel values by per-
`
`
`
`
`
`
`
`forming an inverse discrete cosine transform operation.
`
`
`
`
`
`
`
`
`
`MPEGdoes not specify the detail of the IDCT implemen-
`
`
`
`
`
`
`
`tation. Therefore, forms of implementation can differ. This
`
`
`
`
`
`
`
`difference is mostlikely to become manifest when the values
`
`
`
`
`
`
`
`
`
`
`of the IDCTresults are close to a half integer (e.g., 1.5).
`
`
`
`
`
`
`
`
`Whenthe IDCT results are rounded to the nearest integer, it
`
`
`
`
`
`
`
`
`
`is possible that one implementation will round up, because
`
`
`
`
`
`
`
`
`
`its resultant valuc is only slightly greater than the valuc of
`
`
`
`
`
`
`
`
`
`
`the half integer, while the other will round down,becauseits
`
`
`
`
`
`
`
`
`
`resultant value is only slightly less than the value of the half
`
`
`
`
`
`
`
`
`
`integer. This mismatch becomes bigger when there are more
`
`
`
`
`
`
`
`prediction frames. To reduce the mismatch, MPEG employs
`
`
`
`ia
`—oak(n+ 1/2)
`XA) = wo atk) C(k) «cos—————__,,
`
`
`
`
`
`
`
`
`
`
`
`where a(k)=% for k=0 and 1 otherwise.
`
`
`
`
`
`
`Here a one dimensional DCT is illustrated for simplicity.
`
`
`
`
`
`The weighting coefficients, used to accomplish lowpass
`
`
`
`
`
`
`
`filtering, are obtained by transforming the lowpassfilter
`
`
`
`
`
`
`impulse response in the spatial domain to weighting coef-
`
`
`
`
`
`
`
`ficients in the frequency domain. These weighting coeffi-
`
`
`
`
`
`
`
`cients are represented by H'(k) in the following equation:
`
`
`
`
`
`
`
`
`
`US 6,456,663 B1
`
`
`
`
`
`40
`
`
`
`50
`
`
`
`
`
`60
`
`
`
`
`
`Page 8 of 12
`
`Page 8 of 12
`
`

`

`
`
`US 6,456,663 B1
`
`
`8
`
`
`
`
`
`
`plary implementations of the 8 point DCT