(12) United States Patent
`Nakano et al.
`
`USOO6765616B1
`(10) Patent No.:
`US 6,765,616 B1
`(45) Date of Patent:
`Jul. 20, 2004
`
`(54) ELECTRIC CAMERA
`
`(75) Inventors: Takahiro Nakano, Hitachinaka (JP);
`Ryuji Nishimura, Yokohama (JP);
`Toshiro Kinugasa, Hiratsuka (JP)
`
`(73) Assignee: Hitachi, Ltd., Tokyo (JP)
`
`(*) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 09/520,836
`(22) Filed:
`Mar. 8, 2000
`(30)
`Foreign Application Priority Data
`Jan. 11, 2000 (JP) ....................................... 2000-006064
`2
`
`6,529.236 B1 * 3/2003 Watanabe ................ 348/230.1
`6,580,457 B1 * 6/2003 Armstrong et al. ......... 348/317
`2001/0043276 A1 * 11/2001 Ueno ... .
`. . . . . . . 348/322
`2002/0118291 A1 * 8/2002 Ishigami et al. ............ 348/311
`FOREIGN PATENT DOCUMENTS
`
`OOO84.0503 A2 * 6/1998
`11-187306
`* 7/1999
`
`EP
`JP
`* cited b
`cited by examiner
`Primary Examiner Aung Moe
`(74) Attorney, Agent, or Firm-Antonelli, Terry, Stout &
`Kraus, LLP
`ABSTRACT
`(57)
`A photography related to Video cameras, camcorders, digital
`still cameras and others using a Solid-State image Sensing
`device, and particularly an electric camera using a Solid-State
`
`(51) Int. C.7 - - - - - - - - - - - - - - - - - - - - - - - - - HO)4N 5/335; HO4N 9/083
`
`image Sensing device with large number of pixels. The USC
`
`(56)
`
`(52) U.S. Cl. ........................ 348/322; 348/312; 348/273
`(58) Field of Search ................................. 348/322, 312,
`348/317, 311, 294, 273, 262, 263, 319,
`320, 222.1
`
`of an image Sensing device with a Sufficient number of pixels
`for Still image photographing ensures good performance for
`the moving image photographing and for the monitoring of
`a Static image photographing. The image Sensing device
`used has an arbitrary number of Vertically arranged pixels
`equal to or more than three times the number of effective
`References Cited
`scanning lines of the television System. During the moving
`U.S. PATENT DOCUMENTS
`image photographing and during the monitoring of a Static
`a
`image photographing, the pixels are cyclically mixed
`4,434,435 A * 2/1984 Fujimoto .................... 348/277
`together or culled and desired television Signals are gener
`4,910,599 A * 3/1990 Hashimoto ............... 348/240.2
`ated by performing the Signal reading processing during the
`5,264,939 A * 11/1993 Chang ........................ 348/322
`Vertical blanking periods and the interpolation processing.
`5,450,129 A : 9/1995 Matoba et al. .............. 348/294
`E: A : s ENE, - - - - - - - - - - - - - - 3.36 During the Static image recording, Signals of all pixels are
`5986,698 A 11/1999 Nobuoka. E, read out independently and recorded.
`all SK C al. .............
`6,122,007 A
`9/2000 Ishibashi ................. 348/231.6
`6,181,375 B1 * 1/2001 Mitsui et al. .......... 348/240.99
`20 Claims, 8 Drawing Sheets
`
`V1 V2 V3
`
`
`
`OUTPUT-3
`34
`
`IPR2022-01287
`Maxell Ex. No. 2002
`Page 1 of 18
`
`

`

`U.S. Patent
`
`Jul. 20, 2004
`
`Sheet 1 of 8
`
`US 6,765,616 B1
`
`8 |
`
`
`
`
`
`
`
`
`
`68Z9Ge ``\
`
`
`
`BHD LHEdW Zs?
`
`IPR2022-01287
`Maxell Ex. No. 2002
`Page 2 of 18
`
`

`

`U.S. Patent
`
`Jul. 20, 2004
`
`Sheet 2 of 8
`
`US 6,765,616 B1
`
`FG.2
`
`Y
`YH
`
`Y
`Y
`||
`
`output-31
`
`34
`
`t T t
`YE Gl I H
`(Gi- CEO
`"
`" '
`"
`
`31 / V1 V2 V3
`YE GE EE
`Y
`G CEP
`E GE CE
`
`Y
`
`33
`
`FIG.4
`
`- A FIELD (n+1)TH LINE
`B FIELD TH LINE
`
`A FIELD nTH LINE
`
`IPR2022-01287
`Maxell Ex. No. 2002
`Page 3 of 18
`
`

`

`U.S. Patent
`
`Jul. 20, 2004
`
`Sheet 3 of 8
`
`US 6,765,616 B1
`
`I,II,II,II,~||||||||(~~~~[[T]TT|| || | || || || ||- - - - - - - - - - - - - - - - - - - -|||||||Tl)| || || || ||
`
`
`
`
`
`- - - - - - - - - - - - - - - -|III
`
`THAET MOT
`]'a
`
`
`
`TBAET H9||H
`
`
`
`TBAET MOT
`
`
`
`TBAET BTGQIW | ZA
`
`
`TEAET ETIOJOJ|W
`TBAET MOT
`Jeae
`
`IPR2022-01287
`Maxell Ex. No. 2002
`Page 4 of 18
`
`

`

`U.S. Patent
`
`Jul. 20, 2004
`
`Sheet 4 of 8
`
`US 6,765,616 B1
`
`FIG.5
`
`AREAB
`
`AREA A
`
`C
`O
`
`d
`K
`
`SS 8
`S. C S
`s X
`XX SS
`
`××××××××××× Ø
`
`?????????
`
`1600
`
`FIG.6
`
`NAREA A
`N
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`A FIELD GRAVITY CENTER OF (n+1)TH LINE
`
`- - - - - - - - B FELD GRAVITY CENTER OF TH LINE
`
`A FIELD GRAVITY CENTER OF nTH LINE
`
`IPR2022-01287
`Maxell Ex. No. 2002
`Page 5 of 18
`
`

`

`U.S. Patent
`
`Jul. 20, 2004
`
`Sheet 5 of 8
`
`US 6,765,616 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`||f|OHIO LITTO HIOBOIAEO|
`
`IPR2022-01287
`Maxell Ex. No. 2002
`Page 6 of 18
`
`

`

`U.S. Patent
`
`Jul. 20, 2004
`
`Sheet 6 of 8
`
`US 6,765,616 B1
`
`FIG.8
`
`- A FIELD GRAVITY CENTER OF (n+1)TH LINE
`
`-
`
`a BFIELD GRAVITY CENTER OF nTH LINE
`or
`a
`- - A FIELD GRAVITY CENTER OF nTH LINE
`
`FIG.9
`
`AREA A
`AREA B
`
`AREA A : 96OHIGH x 128OWDE
`
`AREA B : 72OHIGH x 96OWIDE
`
`AREA C : 48OHIGH x 64OWDE
`
`AREA C
`
`\
`
`12OO
`
`|- co
`
`IPR2022-01287
`Maxell Ex. No. 2002
`Page 7 of 18
`
`

`

`U.S. Patent
`
`Jul. 20, 2004
`
`Sheet 7 of 8
`
`US 6,765,616 B1
`
`FIG 10
`
`y
`Y
`Y
`
`32
`v1 V2 V3 V4 V5 V6
`
`3O
`31
`GE ?
`y
`is
`a
`YH G
`1c
`YH Gl CH
`III
`
`C
`G
`l Y
`G. C.
`Y
`Y
`G. C.
`Y
`Y
`AC
`OUT F
`34
`
`33
`
`FIG. 12
`
`nTH LINE - n'TH LINE
`- (n'+1)TH LINE
`(n+1)TH LINE
`ID - : NE
`-- (n'--
`(n+2)TH LINE - (n'+4)TH LINE
`
`(BEFORE INTERPOLATION)
`
`(AFTER INTERPOLATION)
`
`
`
`IPR2022-01287
`Maxell Ex. No. 2002
`Page 8 of 18
`
`

`

`U.S. Patent
`
`Jul. 20, 2004
`
`Sheet 8 of 8
`
`US 6,765,616 B1
`
`GOldadONIYNV1EWOLLYSA
`
`
`
`
`
`QOIHSdONINNVOS
`
`
`
`
`
`
`
`
`
`
`
`
`
`IWOLLYSA| QOldadONINNVOSSAILO3454TWOLLYSAdOldAdONIMNV1dIWOlLeaA
`
`WLNOZIHOH
`
`
`
`GOIW3adONINNVOS
`
`
`
`GOldadONMNV1d
`
`WINOZIHOH
`
` LhSlsZL
`
`
`
`
`
`
`TSAS1S1CCIN
`
`AAASTdGIW
`
`Ad1M01
`
`
`
`TaAa1S1GGIN
`
`TaAq1S100lW
`
`TaAd1M01
`
`WATM01
`
`74aAS1HOIH
`
`
`
`TaAS1FICCIN
`
`T3A51S100IW
`
`TAAd1MO
`
`7aA51MO1
`
`74aA31M01
`
`VaA57HOI
`
`V3AS1HOIH
`
`IPR2022-01287
`
`Maxell Ex. No. 2002
`
`Page 9 of 18
`
`IPR2022-01287
`Maxell Ex. No. 2002
`Page 9 of 18
`
`
`

`

`US 6,765,616 B1
`
`1
`ELECTRIC CAMERA
`
`2
`publication, signals from all the effective pixels are read out
`taking two or more times the field period of the television
`system, stored in a memory means such as a field memory,
`and then subjected to interpolation processing for transfor
`mation into signals conforming to the field cycle and hori
`Zontal scan cycle of television.
`This conventional camera, however, requires a large pro
`cessing circuit, such as field memory, for signal conversion.
`Another drawback is that the image Sensing device readout
`cycle is a plurality of times the field cycle, degrading the
`dynamic resolution. Even with the use of this circuit, the
`number of pixels obtained as the static image signals is
`limited to the number of effective pixels used for moving
`videos plus the area of image stabilization pixels.
`In a digital still camera designed for taking still images,
`there has been a trend in recent years toward an increasing
`number of pixels used on the moving video image Sensing
`device in order to obtain higher resolution Static image
`signals. When taking a moving image or monitoring the
`video, it is necessary to generate signals that conform to the
`television system. The number of pixels on Such an image
`sensing device, however, does not necessarily match the
`number of scanning lines of the television System and
`therefore some form of conversion means is required.
`The conversion means may involve, as in the Video
`camera with the area of image Stabilization pixels, reading
`out signals from the image Sensing device taking a longer
`time than the field cycle and interpolating them to generate
`television signals. This method has, in addition to the
`problem described above, a drawback that the readout cycle
`increases as the number of pixels increases, further degrad
`ing the dynamic resolution.
`To mitigate this problem, JP-A-9-270959 discloses an
`apparatus which mixes together or culls the pixel Signals
`inside the image sensing device to reduce the number of
`signals to be read and therefore the read cycle. Although this
`apparatus alleviates the problem of the degraded dynamic
`resolution, it requires a large processing circuit Such as field
`memory to perform time-axis transformation to generate
`signals conforming to the television System and the image
`sensing device itself needs to have a special structure for
`performing desired mixing and culling.
`SUMMARY OF THE INVENTION
`The present invention relates to a photography of Video
`cameras, camcorders, digital still cameras and others using
`a solid-state image sensing device, and more particularly to
`an electric camera using a Solid-state image Sensing device
`with a large number of pixels.
`The conventional electric cameras, as described above,
`have drawbacks that when taking a still picture by using a
`video camera, the number of pixels is not Sufficient and that
`when taking a moving image with a still camera, the
`associated circuit inevitably increases and the dynamic
`image quality deteriorates. Taking both moving and Static
`images of satisfactory quality with a single camera is
`difficult to achieve. In addition to solving the above
`problems, to obtain good dynamic picture quality by using
`an image sensing device having a large number of pixels
`intended for still images requires extracting a pixel area that
`is used to realize an image Stabilizing function. The con
`ventional art and cameras do not offer a means to accomplish
`this function.
`An object of the present invention is to provide an electric
`camera that solves these problems and which uses an image
`sensing device with a sufficient number of pixels for still
`
`BACKGROUND OF THE INVENTION
`The present invention relates to a photography related to
`video cameras, camcorders, digital still cameras and others
`using a Solid-state image Sensing device, and more particu
`larly to an electric camera using a Solid-state image Sensing
`device with a large number of pixels.
`Electric cameras using Solid-state image Sensors Such as
`CCDs (charge-coupled devices) include a so-called Video
`camera or camcorder for taking moving images and a
`so-called digital still camera for taking still images. In recent
`years, video cameras with a still image taking function and
`digital still cameras with a moving image taking function
`have become available.
`In a video camera to photograph moving images, it is
`generally assumed that the video is viewed on a display Such
`as television monitor and thus the camera is designed to
`produce output signals conforming to a television System
`Such as NTSC and PAL. Therefore, the effective number of
`vertically arranged pixels or picture elements on the image
`sensing device used in Such a camera is determined to enable
`television signals to be generated. The NTSC system, for
`example, performs interlaced scanning on two fields, each of
`which has an effective scanning line number of about 240
`lines (the number of scanning lines actually displayed on the
`monitor which is equal to the number of Scanning lines in the
`vertical blanking period subtracted from the total number of
`scanning lines in each field). To realize this, the image
`sensing device has about 480 pixel rows as the standard
`effective number of vertically arranged pixels. That is, the
`signals of two vertically adjoining pixels in each field are
`mixed together inside or outside the image sensing device to
`generate about 240 scanning lines, and the combinations of
`pixels to be cyclically mixed together are changed from one
`field to another to achieve the interlaced Scanning.
`Some image sensing devices to take moving images
`according to the NTSC system have an area of pixels for
`image stabilization added to the area of effective pixel area,
`thus bringing the effective number of Vertically arranged
`pixels to about 480 or more. In this case, an area beyond
`480th pixels is read out at high speed during the Vertical
`blanking period and therefore the signals thus read out are
`not used as effective signals. Therefore, the video signals can
`only be generated from those signals coming from the area
`of about 480 vertically arranged pixels. When Such a camera
`is used to photograph a still image, it is relatively easy to
`generate a static image signal conforming to, for example,
`JPEG (Joint Photographic Expert Group) from the signals
`coming from the same pixel area that is used to take a
`moving image. A problem remains, however, that the num
`ber of vertically arranged pixels obtained is limited to
`around 480, making it impossible to produce more detailed
`Static image signals.
`In a camera having an image sensing device with the area
`of pixels for image stabilization mentioned above, a method
`of alleviating this problem may involve using the entire area
`of effective pixels including the area of image Stabilization
`pixels in photographing a still image. Even when photo
`graphing a still image, however, the photographed image
`needs to be monitored for check and, for that purpose, it is
`necessary to generate signals conforming to the television
`system from signals read out from all effective pixels.
`An example of such a conventional camera has been
`proposed in JP-A-11-187306. In the camera disclosed in this
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`IPR2022-01287
`Maxell Ex. No. 2002
`Page 10 of 18
`
`

`

`3
`images and enables taking of highly detailed Still images and
`a moving video taking with reduced image quality degra
`dation without increasing circuitry Such as field memory. It
`is also an object of the present invention to provide an
`electric camera that can also realize the image Stabilizing
`function.
`According to one aspect of this invention, the electric
`camera to realize the above objectives has: an image Sensing
`device with a light receiving Surface having N vertically
`arranged pixels and an arbitrary number of pixels arranged
`horizontally, N being equal to or more than three times the
`number of effective Scanning lines M of a display Screen of
`a television System; a driver to drive the image Sensing
`device to vertically mix or cull signal charges accumulated
`in individual pixels of every K pixels to produce a number
`of lines of output Signals which corresponds to the number
`of effective Scanning lines M, Kbeing at least one of integers
`equal to or less than an integral part of a quotient of N
`divided by M, and a Signal processing unit to generate image
`Signals by using the output signals of the image Sensing
`device.
`AS explained above, Since this invention eliminates the
`limit on the number of Vertically arranged pixels, an electric
`camera can be provided which enables taking of highly
`detailed Still images and a Satisfactory moving video taking
`by using an image Sensing device with a large enough pixel
`number even for Still images.
`
`15
`
`25
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a block diagram Showing the configuration of a
`first embodiment of an electric camera according to the
`present invention.
`FIG. 2 is a Schematic diagram showing the Structure of an
`image Sensing device in the first embodiment of the electric
`camera of the invention.
`FIG. 3 is a drive pulse timing diagram in the first
`embodiment of the electric camera of the invention.
`FIG. 4 is a Schematic diagram showing a mixing operation
`in the first embodiment of the electric camera of the inven
`tion.
`FIG. 5 is a Schematic diagram showing a readout area in
`the first embodiment of the electric camera of the invention.
`FIG. 6 is a Schematic diagram showing a mixing operation
`in the first embodiment of the electric camera of the inven
`tion.
`FIG. 7 is a block diagram Showing the configuration of a
`Second embodiment of an electric camera according to the
`present invention.
`FIG. 8 is a Schematic diagram showing a mixing operation
`in the Second embodiment of the electric camera of the
`invention.
`FIG. 9 is a Schematic diagram showing a readout area in
`the Second embodiment of the electric camera of the inven
`tion.
`FIG. 10 is a schematic diagram showing the structure of
`an image Sensing device in a third embodiment of the
`electric camera according to the present invention.
`FIG. 11 is a drive pulse timing diagram in the third
`embodiment of the electric camera of the invention.
`FIG. 12 is a Schematic diagram showing an interpolation
`operation in the third embodiment of the electric camera of
`the invention.
`FIGS. 13A and 13B are schematic diagrams showing the
`arrangement of color filters in the image Sensing device in a
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,765,616 B1
`
`4
`fourth embodiment of the electric camera according to the
`present invention.
`
`DESCRIPTION OF THE EMBODIMENTS
`Now embodiments of the present invention will be
`described by referring to the accompanying drawings. FIG.
`1 is a block diagram showing the configuration of one
`embodiment of an electric camera according to the inven
`tion.
`In FIG. 1, reference number 1 represents a lens, 2 an
`aperture, 3 an image Sensing device, 4 a drive circuit, 5 a
`gain adjust circuit, 6 an analog-digital (A/D) conversion
`circuit, 7 a signal processing circuit, 8 a vertical interpola
`tion circuit to perform interpolation in a vertical direction, 9
`a horizontal interpolation circuit to perform interpolation in
`a horizontal direction, 10 a recording unit including record
`ing media Such as magnetic tape, Semiconductor memory
`and optical disk to record a video Signal, 11 a control circuit
`to control these constitutional elements according to the
`operating State, 12 an encoder circuit to modulate the Video
`Signal into a Standard television signal, 13 a digital-analog
`(D/A) conversion circuit, 14 a mode selector Switch to
`change over the operation mode between the moving video
`taking and the Still image taking, 15 a record button to Start
`or Stop the recording, 16a and 16b gyro Sensors to detect
`Vertical image-unstability and lateral image-unstability,
`respectively, and 17 an image-unstability decision circuit to
`determine the image-instability from Signals output from the
`gyro SenSOrS.
`In the above configuration, light coming from the lens 1
`through the aperture 2 is focused on a light receiving Surface
`of the image Sensing device 3 where it is converted into an
`electric Signal. In this embodiment the image Sensing device
`3 is of a CCD type. FIG. 2 shows the structure of this image
`sensing device 3. In FIG. 2, denoted 30 are pixels each
`formed of a photodiode, which are arranged horizontally and
`Vertically in a grid pattern. On these grid-arrayed pixels
`three types of color filters that pass yellow (Ye), green (G)
`and cyan (Cy), respectively, are arranged in Such a way that
`the combination of these three colorS is repeated horizon
`tally every three pixels and that the filters of the same colors
`are lined vertically in So-called vertical Stripes. Although an
`arbitrary number of pixels may be used, this embodiment
`has an array of 1200 pixels vertically and 1600 pixels
`horizontally. A vertical transfer unit 32 is a CCD which is
`driven by three phase pulses V1,V2, V3. This CCD has a
`three-gate Structure in which each pixel corresponds to three
`phase pulses and thus can vertically transfer a signal charge
`of each pixel independently. Transfer gates 31 for transfer
`ring the charge of each pixel to the vertical transfer unit 32
`are commonly connected to a gate of the vertical transfer
`unit 32 that corresponds to the V2 pulse. An operation to
`transfer the charge from each pixel to the vertical transfer
`unit 32 in response to a peak value of the pulse applied to the
`commonly connected gate and an operation to transfer the
`charge Vertically are performed Separately. A horizontal
`transfer unit 33 horizontally transferS the charges Supplied
`from the Vertical transfer units 32 and outputs them Succes
`sively through an output amplifier 34 from the output
`terminal.
`Referring back to FIG. 1, the operation performed when
`the moving Video mode is Selected by the mode Selector
`Switch 14 will be explained. The number of vertically
`arranged pixels on the image Sensing device in this embodi
`ment is 1200, so if the number of effective scanning lines in
`the field of the NTSC system is assumed to be 240 lines, then
`
`IPR2022-01287
`Maxell Ex. No. 2002
`Page 11 of 18
`
`

`

`US 6,765,616 B1
`
`S
`vertically mixing five pixels (=1200 pixel rows/240 scan
`ning lines) can match the number of lines of output signals
`from the image Sensing device to the number of effective
`Scanning lines.
`However, in this embodiment, to realize the image Stabi
`lizing function described later, four vertically arranged pix
`els are mixed together during motion image taking mode.
`When four vertically arranged pixels are to be cyclically
`mixed together, the signals from the area of 960 pixels (=240
`Scanning linesx4 pixels) out of the 1200 vertically arranged
`pixels are used as effective signals and the remaining 240
`pixels (=1200 (all pixels)-960 (effective pixels)) are not
`used for image forming. FIG. 3 shows the timing of a
`Vertical drive pulse for the image Sensing device in this
`operation mode, with V1, V2 and V3 representing three
`phase drive pulses applied to each gate of the CCD or
`vertical transfer unit 32.
`In FIG. 3, in a period T1 included in the vertical blanking
`period, the drive pulse V2 is held high to transfer the Signal
`charge accumulated in each pixel to under the V2 gate of the
`vertical CCD. Next, in a period T2, while the V2 pulse is still
`at middle level, the V3 pulse is raised from low level to
`middle level; next, while the V3 pulse is at middle level, the
`V2 pulse is changed from middle level to low level, after
`which the V1 pulse is changed from low level to middle
`level; next, while the V1 pulse is at middle level, the V3
`pulse is changed from middle level to low level, after which
`the V2 pulse is changed to middle level and finally the V1
`pulse is changed from middle level to low level. With this
`Sequence of pulse operations, the Signal charges under the
`V2 gate for one pixel row are transferred and held again
`under the V2 gate.
`By repeating this Series of operations, the Signal charges
`for a desired number of pixel rows can be transferred. In
`FIG. 3, during a period T3 included in the vertical blanking
`period before the vertical effective Scanning period (the
`Vertical Scanning period minus the vertical blanking period
`which corresponds to the actually displayed image) and
`during a period T4 included in the vertical blanking period
`after the Vertical effective Scanning period, the above trans
`fer operation for one pixel row is repeated a total of 240
`times to transfer the Signal charges of the 240 pixel rows not
`used for image generation to the horizontal transfer unit 33
`during the vertical blanking period. For example, if this
`transfer operation is performed 120 times during the period
`T3 and 120 times during the period T4, the Signal charges
`from upper 120 pixel rows and lower 120 pixel rows on the
`light receiving Surface are transferred to the horizontal
`transfer unit 33 during the period T3 and period T4 within
`the Vertical blanking period. During each of the Subsequent
`periods T5 and T6 in the vertical blanking period, the
`horizontal transfer unit 33 is driven for a predetermined
`period to output the charges transferred to the horizontal
`transfer unit 33 from the output terminal. These charges are
`not used as valid Signals as they are output during the
`Vertical blanking period.
`Next, in the vertical effective scanning period of FIG. 3,
`the above one-pixel-row transfer operation is performed four
`times during each horizontal blanking period to transfer the
`Signal charges of four pixel rows to the horizontal transfer
`unit 33 where they are mixed together. Then, during a
`horizontal effective Scanning period (the horizontal Scanning
`period minus the horizontal blanking period which corre
`sponds to the actually displayed image), the horizontal
`transfer unit 33 is driven to read out the Signal charges from
`the horizontal transfer unit to produce an output Signal
`conforming to the television System. If the above operation
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`is performed on the A field and if, on the B field, the number
`of pixel rows transferred during the period T3 is set to 122
`rows and that during the period T4 is set to 118 rows, then
`the combination of four pixel rows to be cyclically mixed
`together shifts by two rows between the two fields, thus
`allowing the interlaced Scanning to be performed as shown
`in FIG. 4. (FIG. 4 shows the light receiving surface of the
`image Sensing device and its relation to the displayed Screen
`is vertically inverted.)
`Let us return to FIG. 1. The output Signal from the image
`Sensing device 3 is adjusted in gain by the gain adjust circuit
`5 and then converted by the A/D conversion circuit 6 into a
`digital signal. The digital Signal is then processed by the
`Signal processing circuit 7 that performs color Signal pro
`cessing and luminance signal processing, Such as generation
`of color Signals, gamma correction, white balance proceSS
`ing and outline enhancement. The image Sensing device in
`this embodiment has an array of Vertical Stripes of yellow
`(Ye), green (G) and cyan (Cy) color filters, So the color
`Signals for Ye, G and Cy are obtained as a Series of color
`points from one line of output Signals at all times no matter
`how many pixels are vertically combined. From these color
`Signals three primary color Signals R, G, B can be obtained
`from the following calculations.
`
`The R, G and B Signals undergoes the white balance
`processing and gamma correction processing in the signal
`processing circuit 7 and are then converted into color
`difference signals such as R-Y, B-Y or U and V. The
`luminance Signals and the color difference signals are then
`entered through the vertical interpolation circuit 8 into the
`horizontal interpolation circuit 9. In this operation State the
`Signals just pass through the vertical interpolation circuit 8
`without being processed. The horizontal interpolation circuit
`9 performs interpolation on the Signals in the horizontal
`direction.
`FIG. 5 shows the light receiving surface of the image
`Sensing device. AS described above, in the operating State of
`this embodiment, the Signals read out during the vertical
`effective Scanning period correspond to an area having 960
`of the 1200 vertically arranged pixels and a horizontal width
`of 1600 pixels, as shown shaded at A in FIG. 5. If the entire
`light receiving Surface of the image Sensing device has a
`4-to-3 (width to height) aspect ratio, the Shaded area A is
`more laterally elongate than this aspect ratio. Hence, if the
`Signals of all horizontal pixels of the light receiving Surface
`are displayed, for example, on an NTSC standard television
`monitor with the 4-to-3 aspect ratio, the image displayed is
`compressed horizontally and lookS vertically elongate, com
`pared with the original image. It is therefore necessary to
`output during the horizontal effective Scanning period only
`those signals coming from a pixel area with the horizontal
`width conforming to the aspect ratio of the television
`system, as shown by a shaded area B. When the television
`System has an 4-to-3 aspect ratio, the number of pixels in the
`horizontal width of the shaded area B is 1280 (=960 (vertical
`effective pixels)x4/3).
`Returning back to FIG. 1, the horizontal interpolation
`circuit 9 performs interpolation processing on the Signals
`from the horizontal 1280 pixels to expand the Signals So that
`they can be output over the entire horizontal effective
`Scanning period. It also performs Switching among different
`
`IPR2022-01287
`Maxell Ex. No. 2002
`Page 12 of 18
`
`

`

`7
`clocks as required. With the above operation, an area having
`960 pixels in height and 1280 pixels in width is demarcated
`from the light receiving Surface as Signals conforming to the
`television System. Then, the luminance Signal and the color
`difference signal are encoded by the encoder circuit 12 into
`television signals, which are then converted by the D/A
`conversion circuit 13 into analog Signals for output. When
`the recording is Specified by the record button 15, the Signals
`are recorded by the recording unit 10. At this time, the
`signals may be compressed in the MPEG (Moving Picture
`Expert Group) format.
`Next, the image Stabilizing operation will be explained.
`Image-unstability information obtained by the gyro Sensors
`16a, 16b that detect vertical and horizontal image
`unstabilities is entered into the image-unstability decision
`circuit 17, which checks the received information for the
`amount and direction of the image-unstability and converts
`them into the number of pixels in vertical and horizontal
`directions on the light receiving Surface of the image Sensing
`device. Based on the converted pixel numbers, the position
`of an extracted area (effective pixel area) on the light
`receiving Surface is shifted in a direction that cancels the
`image-unstability. This can correct the image-unstability.
`The positional shifting of the extracted area is performed as
`follows. The shifting in the vertical direction can be made by
`changing the number of pixel rows transferred during the
`periods T3 and T4 in FIG.3 and the shifting in the horizontal
`direction made by changing the interpolation Start position
`in the horizontal interpolation circuit 9.
`The operation during the moving Video mode has been
`described above. Next, the operation performed when the
`static image mode is selected by the mode selector switch 14
`will be explained.
`In the Static image mode, too, until the recording is
`requested by the record button 15, the camera outputs
`Signals compatible with the television System to monitor the
`angle of View. Unlike the moving Video photographing, all
`of the effective pixels on the image Sensing device are used
`in this embodiment during the Still image photographing to
`produce Signals with as high a resolution as possible. Hence,
`during the monitoring the television Signals need to be
`generated from the Signals coming from the entire pixel area.
`The image sensing device of this embodiment has 1200
`Vertically arranged pixels, and the number of lines of output
`Signals from the image Sensing device can be made to match
`the number of effective scanning lines of NTSC system,
`which is assumed to have 240 Scanning lines, by Vertically
`mixing five pixels (=1200/240). To make the image Sensing
`device operate in this manner, the one-pixel-row transfer
`operation is performed five times during each horizontal
`blanking period in the vertical effective Scanning period
`shown in the pulse timing diagram of FIG. 3. With this
`operation, the Signal charges of five pixel rows can be mixed
`by the horizontal transfer unit 33. As for the transfer
`operations during the periods T3 and T4 in the vertical
`blanking period, because the interlaced Scanning is carried
`out, only two pixel rows are transferred during the period T3
`on the B field, with no transfer operations performed in other
`vertical blanking periods (In this embodiment, 1200/240=5
`with no remainder produced, So no further transfer is nec
`essary; if, however, a remainder occurs, the remaining pixels
`need only be transferred during the periods T3 and T4).
`The charges mixed by the horizontal transfer unit 33 are
`read out by driving the horizontal transfer unit 33 during the
`horizontal effective scanning period. With the above
`operations, the Signal charges of all pixels on the image
`Sensing device can be read out in a manner conforming to
`
`8
`the television System. The output Signal from the image
`Sensing device 3 is, as during the moving image
`photographing, adjusted in gain by the gain adjust circuit 5
`and converted by the A/D conversion circuit 6 into a digital
`Signal, which is then Subjected to the color Signal processing
`and the luminance Signal processing in the Signal processing
`circuit 7 before being entered into the vertical interpolation
`circuit 8. During the Static image monitoring, the vertical
`interpolation circuit 8 performs a vertical gravity center
`correction on the received signals.
`FIG. 6 shows combinations of pixels to be cyclically
`mixed on the A field and the B field and also the vertical
`position of the gravity center of the mixed signals. In the
`interlaced Scanning, Scanning lines of the A field and the B
`field are located at the centers of adjoining Scanning lines on
`other field. Hence, the Signal Samplings in the camera
`system for the two fields must be 180 degrees out of phase
`in the Vertical direction. In the operati