United States Patent [19]
`Maurinus et al.
`
`111111111111111111111111111111111111111111111111111111111111111111111111111
`USOO5606365A
`[11] Patent Number:
`[45] Date of Patent:
`
`5,606,365
`Feb. 25, 1997
`
`[54]
`
`[75]
`
`INTERACTIVE CAMERA FOR NETWORK
`PROCESSING OF CAPTURED IMAGES
`
`Inventors: Martin A. Maurinus, Rochester;
`Wilbert F. Janson, Jr., Shortsville,
`both of N.Y.
`
`[73] Assignee: Eastman Kodak Company, Rochester,
`N.Y.
`
`[21]
`
`Appl. No.: 412,775
`
`[22]
`
`Filed:
`
`Mar. 28, 1995
`
`[51]
`[52]
`[58]
`
`Int. Cl.6
`....................................................... H04N 3/00
`U.S. Cl. ........................... 3481222; 348/552; 3481247
`Field of Search ..................................... 348/222, 220,
`348/223, 232, 242, 230, 247, 246, 251,
`254, 13, 12, 187, 466, 143, 148, 157, 159,
`17,6,468,552; H04N 3/00, 3/310, 51232
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`3,984,625
`4,524,381
`4,574,319
`4,962,521
`5,042,061
`5,045,948
`5,063,587
`5,101,499
`5,164,836
`5,191,410
`5,202,957
`5,218,627
`5,220,420
`5,257,306
`5,264,935
`5,461,440
`
`1011976 Camras .................................... 178/5.6
`611985 Konishi ..................................... 358/29
`311986 Konishi ................................... 358/335
`1011990 Komatsu et aI. ......................... 379/53
`8/1991 Kaneko et aI. .. ......................... 379/53
`9/1991 Streck et al .......................... 3581194.1
`1111991 Semasa et aI. ............................ 379/53
`311992 Streck et al ................................. 455/4
`11/1992 Jackson.
`3/1993 McCalley.
`4/1993 Serrao ....................................... 379/53
`611993 Corey et aI. .............................. 379/53
`611993 Roarty ...................................... 358/86
`1011993 Watanabe .................................. 379/53
`1111993 Nakajima ................................ 3581181
`1011995 Toyoda.
`
`OTHER PUBLICATIONS
`
`Junko Yoshido, "Set-top box could be budget buster",
`Electronic Engineering TIMES, Nov. 29, 1993.
`Junko Yoshido, "TV set-top box choices add to cable
`confusion", Electronic Engineering TIMES, Dec. 6, 1993.
`
`Primary Examiner-John K. Peng
`Assistant Examiner-Nathan J. Flynn
`Attorney, Agent, or Firm-David M. Woods
`
`[57]
`
`ABSTRACT
`
`A digital still camera and interactive system and method for
`video display or reproduction of captured images employing
`an image processing system on an interactive network for
`receiving and converting a captured raw, digitized image
`information set into a display image format for display as a
`video image, storage, and/or reproduction as a print. A
`simplified digital camera without signal compensating and
`processing circuits is employed by a user to capture an
`image and to transmit the raw image information sets along
`with a camera identification code through an interactive
`television network to an interactive control node for pro(cid:173)
`cessing with color correction and pixel defect correction
`using the cameras correction and defect maps and other
`image enhancement, and to convert the processed image
`data to the specified display image format. The processed
`video display signal is transmitted back to the user's or other
`designated receiver's interactive television home interface
`controller. The camera specific, correction maps and the
`color and defect correction algorithms may be provided by
`the camera manufacturer to the network on registration of
`the camera and accessed or downloaded by the user or the
`user's designee. Processing of the raw image information
`sets may be at a control node within the network or in the
`user's home interface controller.
`
`30 Claims, 5 Drawing Sheets
`
`50
`
`HEAD END
`COMPUTER
`AND DATA
`BANK
`
`52
`
`58
`
`48
`
`CAMERA
`MANUFACTURING
`
`10
`
`UNIFIED PATENTS INC.
`EX1006
`Page 1 of 14
`
`

`
`42
`
`FLASH}
`
`24
`
`~)
`
`40
`)
`
`PHOTOMETER
`
`USER - - - .....
`INPUTS
`
`22
`/
`
`:1 CAMERA CONTROL
`---..... ~, --------------~
`v 44
`
`I
`CONTROL DISPLAY
`
`30
`
`AID
`CONVERTER
`
`34
`/
`BUFFER
`MEMORY
`
`I MICRO
`I
`TIMING
`DRIVER i----1GENERATOR 1--1 PROCESSOR
`I
`I
`32
`28
`
`\14
`
`10
`
`46
`/
`
`DATA
`COMMUNICA TION 1 ___
`CONTROLLER
`
`16
`
`-
`
`NON VOLATILE
`MEMORY
`
`12
`L
`ROM
`L--~I CAMERA
`10
`
`FIG. 1
`
`Cj
`•
`rJ).
`•
`~
`
`~ a
`
`~ ?-
`
`N
`~tTI
`
`~
`~
`~
`-...l
`
`00 =-
`m.
`s,
`
`~
`
`tTl
`
`Ut
`
`"" =" = ="
`"" ~
`=" Ut
`
`UNIFIED PATENTS INC.
`EX1006
`Page 2 of 14
`
`

`
`u.s. Patent
`
`Feb. 25, 1997
`
`Sheet 2 of 5
`
`5,606,365
`
`(!)
`Z
`<~
`ett-
`Wc,)
`~<
`<u..
`0:::>
`Z «
`
`~
`
`ex)
`to
`
`W
`0
`0
`Z
`
`C\I
`10
`
`oet<
`ZWt-
`Wt-<~.
`:::>oZ
`00.. <
`<~Oal
`WOZ
`:J:O<
`
`~I
`
`C\I
`
`a
`
`-
`
`C)
`U.
`
`<C
`c:::
`W
`~ «
`
`C,)
`
`" -:J:
`
`0
`T""
`
`m~
`
`0
`CO
`
`UNIFIED PATENTS INC.
`EX1006
`Page 3 of 14
`
`

`
`u.s. Patent
`
`Feb. 25, 1997
`
`Sheet 3 of 5
`
`5,606,365
`
`o
`<0
`
`(W')
`
`.
`CJ
`U.
`
`W
`..J
`
`m «
`
`(.)
`
`0
`'r."
`
`«
`a:
`w
`~ «
`
`(,)
`
`RI
`
`C\I
`r-..
`
`w c
`0
`z
`
`~I
`
`C\I
`I.(')
`
`ca:<
`zw ....
`w .... <~
`=>cz
`cQ. «
`«~or:n
`woz
`::I:o«
`
`~ r-..
`
`UNIFIED PATENTS INC.
`EX1006
`Page 4 of 14
`
`

`
`88
`FEEDER
`IINSERTER
`
`'t
`
`92
`
`F~·4~~,~.4bl
`FIG,4
`
`96 CABLE
`\1>ROP
`
`DIGITAL
`COMPRESSED
`IMAGE
`
`VIDEO/DISPLA Y
`IMAGE FORMAT
`
`48
`
`83
`
`52
`CABLE TV HEAD END
`DATA BANK CAMERA
`SERIALIZED COLOR
`CORRECTION AND PIXEL
`DEFECT MAP IS STORED
`
`HEAD END COMPUTER
`
`CAMERA
`MANUFACTURER
`DOWNLOADS
`COLOR
`CORRECTION
`AND PIXEL
`DEFECT MAP
`
`10
`
`91
`
`108
`
`CD PLAYERI
`WRITER
`
`54
`
`HIC
`HOME
`INTERFACE
`CONTROLLERt-1 ---I
`
`110
`
`60
`
`I USER'S I
`
`TV
`
`FIG. 4a
`
`•
`
`d • r:.n
`~ s. ('D = ~
`
`~ ?-
`~Ul ,...
`
`N
`
`\C
`\C
`'I
`
`r.n
`I:r'
`~
`~
`~ sa,
`
`Ul
`
`til
`-..
`
`=" = ="
`-.. w
`="
`til
`
`UNIFIED PATENTS INC.
`EX1006
`Page 5 of 14
`
`

`
`100
`
`FEEDER
`INSERTE
`
`102
`
`NODE
`SERIALIZED
`CAMERA COLOR
`CORRECTION AND
`PIXEL DEFECT MAP
`
`.1-104
`
`PRINT(cid:173)
`MAKER
`
`76
`
`106-
`
`101
`
`CAMERA
`
`1011
`
`CAMERA
`
`PCS I'SO
`
`FIG. 4b
`
`PERSONAL
`COMMUNICATIONS
`SYSTEM
`
`d •
`00
`•
`~
`
`~ = ~
`
`~
`
`N
`Y.
`'"""
`\C
`\C
`....:J
`
`00 =(cid:173)~
`
`~
`!JI
`~
`!JI
`
`til
`-..
`
`Q\ = Q\
`
`-..
`W
`Q\
`til
`
`UNIFIED PATENTS INC.
`EX1006
`Page 6 of 14
`
`

`
`5,606,365
`
`1
`INTERACTIVE CAMERA FOR NETWORK
`PROCESSING OF CAPTURED IMAGES
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`Reference is hereby made to commonly assigned, co(cid:173)
`pending U.S. patent application Ser. Nos.: 085,516, filed
`Jun. 30, 1993, in the names of K. Parulski et al. and titled
`SINGLE SENSOR COLOR CAMERA WITH USER
`SELECTABLE IMAGE RECORD SIZE; and 167,756, filed
`Dec. 15, 1993, in the names of K. Parulski et al. for
`PORTABLE ELECTRONIC CAMERA WITH COM(cid:173)
`PUTER DOCKING ADAPTOR PROVIDING COM(cid:173)
`PUTER CAPTURE CONTROL.
`
`FIELD OF THE INVENTION
`
`This invention relates to the field of electronic imaging
`employing an interactive image capture system with distrib(cid:173)
`uted processing of image information, and more particularly
`to a simplified digital camera and centralized image pro(cid:173)
`cessing system.
`
`BACKGROUND OF THE INVENTION
`
`Electronic still cameras are known in the prior art for
`capturing images of scenes and providing the captured
`images to a computer for display on a computer monitor. In
`the above-referenced '516 and '756 applications, electronic
`still cameras are described for capturing images and dis(cid:173)
`playing the captured images on the monitor employing a
`computer docking station and interface to download image
`data to computer memory for display and more permanent
`storage of the video image data. In those cameras, the 35
`camera on-board memory may then be employed to capture
`further images until it is again filled.
`In the '756 patent, the camera is either operable by itself
`in a stand alone mode to store images in on-board memory
`or may be tethered to the computer in a docked mode to
`directly transfer the video image data to the computer
`memory. In the '516 patent, an electronic still camera is
`described having a resolution mode switch for selecting the
`pixel resolution.
`Electronic still cameras of the type described in the ' 516
`and '756 applications employ two-dimensional, charge(cid:173)
`coupled-device (CCD) arrays of CCD photosites. The CCD
`photo sites are overlaid with a three color filter in a check(cid:173)
`erboard pattern. The red, green and blue analog, raw image 50
`pixel signals of the CCD photosites are clocked out in a
`video frame format. The raw image analog pixel signals are
`digitized, and luminance and chrominance pixel information
`is developed from the red, green and blue raw image pixel
`signals. Typically, the luminance signal is derived from the 55
`green filtered photosite pixel signals, and chrorninance is
`developed from the red and blue filtered photosite pixel
`amplitude difference signals, with a green signal interpola(cid:173)
`tion. The digitized video frame signal sets are processed
`within the cameras to correct for defective photosites within 60
`the array and for color balance.
`The corrected digital pixel signals are compressed and
`formatted into a video frame signal for display on a monitor
`or television at the same or a later time. Typically, the video
`frame signals for each image captured are stored in flash
`EEPROM or other memory for later read-out to a personal
`computer (PC). A central processing unit governs the opera-
`
`5
`
`2
`tions of the circuitry and hardware under the control of
`programmed operating routines.
`Currently, a PC electronic still camera is being marketed
`by Apple Computer, Inc. which operates in high and low
`resolution modes to take a limited number of pictures for
`display through a PC interface. Such electronic still cameras
`are relatively complex and expensive when constructed to
`have high resolution and image storage capabilities.
`In U.S. Pat. No. 5,220,420, an interactive multimedia
`10 system with distributed processing and storage of video
`frames and associated data and sound in nodes disposed
`is
`throughout a cable
`television distribution system
`described. An interface to allow users to input video images
`from a camcorder or VCR into a video to RF modulator in
`15 a Home Interface Control (HIC) is also described. The
`ability to display photographic quality images in both still
`frame and short full-motion segments on a home television
`receiver is attributed to the system, although the available
`video camcorders or VCRs do not support photographic
`20 quality images.
`An RF to video demodulator and a video digitizer are
`provided in the node coupled to the HIC to demodulate and
`digitize the video data input from a camcorder or VCR to
`25 forward it to a regional processing center. The regional
`processing center converts and normalizes incoming digi(cid:173)
`tized video frames into a system standardized format and
`stores them in an object database in each of the system nodes
`for selective retrieval by a viewer. No specific camcorder or
`30 video camera type is described in the '420 patent, but it is
`clear that a conventional analog video frame output utilizing
`the traditional NTSC format is supplied to the HIC to be
`converted to a digital format by the video digitizer 164.
`
`PROBLEMS TO BE SOLVED BY THE
`INVENTION
`
`The expense and complexity of the conventional high
`resolution electronic still image camera inhibits its accep(cid:173)
`tance in the market for conventional, non-network image
`40 capture and storage. The cost of participating in an interac(cid:173)
`tive system is magnified by the expense of acquiring suitable
`high resolution video image capture cameras.
`
`SUMMARY OF THE INVENTION
`
`45
`
`It is therefore a primary object of the invention to simplify
`and make less expensive a high resolution electronic still
`image camera system for use in conjunction with an inter(cid:173)
`active home information system.
`It is a further object of the present invention to provide an
`interactive system for use with a simplified electronic still
`camera for capturing high resolution digitized images and
`storing the captured images for viewing on a subscriber's
`television or computer display monitor.
`The present invention preferably employs an electronic
`still camera for use in association with a system for provid(cid:173)
`ing signal processing of raw, digitized image information
`representing the color and intensity value of each pixel of a
`captured image for a specified display or reproduction
`format, the camera comprising: a camera controller respon(cid:173)
`sive to camera control signals for controlling operations of
`the camera; means for storing a camera identification code;
`a solid state, image capture array of color filtered, pixel
`photosites; an imaging lens system for focusing the image to
`65 be captured on the image capture array; image capture
`means responsive to an exposure control signal for exposing
`the image capture array to the focused image through the
`
`UNIFIED PATENTS INC.
`EX1006
`Page 7 of 14
`
`

`
`5,606,365
`
`4
`scribed to by a purchaser to enable electronic image capture
`and video display of the captured images. The expected
`capabilities of the emerging interactive networks and ser(cid:173)
`vices to be provided on such networks are taken advantage
`5 of in economizing and adding flexibility to the camera.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`These and other objects, advantages and features of the
`invention will become apparent from the detailed descrip(cid:173)
`tion given hereinafter in relation to the accompanying draw(cid:173)
`ings, in which:
`FIG. 1 is a schematic illustration of the components of an
`electronic still camera in which the present invention may be
`practiced;
`FIG. 2 is a schematic illustration of the interrelationship
`between the camera of FIG. 1 and an interactive network for
`communication through an HIC;
`FIG. 3 is a schematic illustration of the interrelationship
`between the camera of FIG. 1 and an interactive network
`providing signal communication through a PCS or cellular
`phone link; and
`FIG. 4 is a schematic illustration of the interconnection of
`25 an interactive network leg with the simplified electronic still
`camera through the HIC or the alternative links.
`
`3
`imaging lens system for a predetermined exposure interval
`in order to develop a raw, analog image information set for
`each color from which image luminance and chrominance
`signals may be derived for each captured image; driver
`means responsive to the camera controller for clocking out
`the raw, analog image information set from the image
`capture array upon completion of the exposure; means for
`digitizing the raw, analog image information set; data stor(cid:173)
`age means responsive to the camera controller for tempo(cid:173)
`rarily storing at least one of the raw, digitized image 10
`information sets, each representing a captured image; and
`data communication means for retrieving and transmitting
`each stored raw, digitized image information set to further
`processing equipment.high resolution electronic still camera
`for use in association with an interactive system for provid- 15
`ing centralized signal processing of raw, digitized image
`information representing the color and intensity value of
`each pixel of a captured image for a specified display or
`reproduction format.
`The camera does not include on-board signal processing 20
`for processing the raw, digitized image information set to
`account for dark signal current variances or defects in
`individual CCD photosites of the array that are unavoidable
`with current CCD technology at a reasonable price or for
`other filtered photosite or camera related defects.
`In a further aspect of the invention, such a camera is
`employed in a system for processing the raw, digitized
`image information sets to correct for such variances or
`defects remotely from the camera at the user's HIC or
`upstream in the interactive network accessed from the user' s 30
`HIC for providing a corrected image data set to the user or
`others designated by the user. Other designees include a
`print or CD maker or other user's Set-Top Boxes for making
`print or CD reproductions or for distributing images to
`others.
`The camera having an identification code keyed to a
`camera specific image defect correction map derived at
`manufacture from test of the camera to correct for image
`capture defects and a data communication controller for 40
`transmitting the camera identification code and the raw,
`digitized image information sets to a receiver of an interac(cid:173)
`tive network is preferably used in a system and method of
`processing and correcting raw, digitized image information
`sets and converting each processed and corrected image 45
`information set into a display image format comprising the
`means for and steps of: transferring the image defect cor(cid:173)
`rection map to a repository of an interactive network acces(cid:173)
`sible through a service; transferring color correction and
`processing software algorithms to a repository of the inter- 50
`active network accessible through a service; upon receipt of
`a request for a processing raw, digitized image information
`set, the request including the camera identification code and
`entered in the network, transferring the camera specific
`defect correction map and the color correction and process- 55
`ing software from the repository to an image information
`processing system; in the image information processing
`system, processing and correcting each raw, digitized image
`information set into a designated display image format; and
`transferring the display image format sets to a user defined 60
`recipient on the network.
`
`35
`
`ADVANTAGES OF THE INVENTION
`
`In accordance with the present invention, and due to the 65
`various features thereof, an inexpensive, high resolution
`camera may be employed with an interactive network sub-
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`As stated above, the present invention preferably employs
`a high resolution, but simplified, electronic still image
`camera for capturing an image as a set of raw, digitized
`image information corresponding to the pixel intensities of
`an image capture CCD element array. FIG. 1 is a schematic
`block diagram of one such camera 10 for providing the raw,
`digitized image information along with a camera serial
`number or identification (ID) code to one of the systems
`described in detail below for processing the image informa(cid:173)
`tion for color correction and correction for CCD element
`pixel defects.
`The camera 10 may be a simplified version of the cameras
`disclosed in the above-referenced co-pending '516 and '756
`applications, incorporated herein by reference. The camera
`10 may be operated in a stand-alone mode or a "docked"
`mode as described in the '756 application. The simplified
`camera 10 comprises the camera micro controller 14 respon(cid:173)
`sive to camera control signals, e.g. depression of the image
`capture button by the user, from control block 22 for
`controlling operations of the camera 10. The microcontroller
`14 also controls the exposure parameters and illumination by
`processing output signals from photometer 40 and operating
`flash unit 42 when measured ambient light conditions
`require artificial illumination. A control liquid crystal dis(cid:173)
`play 44 is also provided in which camera operating and
`status messages may be provided to the user by the micro(cid:173)
`controller 14 in a manner well known in the art.
`A trichromatic, solid state image capture array 20, e.g. a
`CCD array of the type described above, is located behind an
`imaging lens system 24 and an optional blur filter for
`focusing the image to be captured on the image capture array
`20. A shutter and aperture mechanism 26 is responsive to an
`exposure control signal from microcontroller 14 for expos(cid:173)
`ing the image capture array 20 to the image focused thereon
`by the lens system 24 in a conventional fashion. The image
`capture array 20 is responsive to a driver 28 in order to the
`develop trichromatic, raw, analog image information which
`
`UNIFIED PATENTS INC.
`EX1006
`Page 8 of 14
`
`

`
`5,606,365
`
`5
`is amplified and supplied to an AiD converter 30. The array
`driver 28, operated by a timing generator 32 in response to
`the microcontroller 14, clocks out the raw, analog image
`information to the AiD converter after the exposure of the
`array 20 is complete. The AiD converter 30 digitizes the raw,
`analog image information as an information set and supplies
`it to a buffer memory 34. The information set is then
`retrieved by the microcontroller 14 and optionally subjected
`to compression before being stored in non-volatile memory
`16 along with the camera ID code. The camera serial number
`or ID code is stored in ROM 12 at manufacture.
`The memory 16 is preferably a non-removable flash,
`electrically-programmable read only memory (FLASH
`EPROM), although a hard disk drive, a floppy disk drive, a
`removable memory IC card or a similar device could be
`used. Different types of compression could be employed,
`e.g. a conventional adaptive differential pulse code modu(cid:173)
`lation (ADPCM) technique.
`The contents of the memory 16 are read out through a data
`communications controller 46 enabled by the microcontrol(cid:173)
`ler 14 in response to a user initiated camera control signal.
`The stored sets of the raw, digitized image information each
`representing a captured image are thereby transmitted to an
`image processing system and converted into a display or
`reproduction image format for display or for use in making 25
`a print or creating a Photo CD of the captured image.
`The ID code for the camera 10, stored in memory 12 at
`manufacture, is read out by a camera microcontroller 14 and
`appended to the raw, digitized image information set for 30
`each image frame that is transferred to non-volatile memory
`16. The camera 10 is tested during manufacture to determine
`which of the CCD array photo sites of the CCD array 20 are
`defective. A pixel defect map associated with the ID number
`is created that identifies the defective photo sites by row and 35
`column number. In addition, a color correction map is
`created that corrects for non-uniformity in the checkerboard
`pattern, color filter array over the photo sites as well as color
`sensitivity variations in the photosites. The color correction
`map compensates for gamma and individual photosite or 40
`overall white balance errors in a well known manner. Optical
`aberrations in the lens system 24 of the individual camera 10
`could also be measured and stored with the color correction
`map. In addition, imbalances in the flash illumination could
`also be measured at manufacture of the camera and stored as 45
`an illumination correction map keyed to the photosite loca(cid:173)
`tions. In accordance with the present invention, the ID code
`retrieved from memory 12 and stored with the raw, digitized
`image information sets is used to in tum retrieve the cor(cid:173)
`rection maps of that same camera for use in processing the 50
`information sets.
`In this regard, the raw, digitized image information set for
`each captured image is transmitted remotely of the camera
`10 with the camera ID code to complete color processing
`and to construct a corrected image information set in the 55
`display image format that is returned to the user for display
`on a video monitor. The corrected image information may be
`formatted to an NTSC standard for display on a conven(cid:173)
`tional television or formatted in a reproduction format for a
`particular color printer for printing as a color print or as a 60
`CD.
`More particularly, at the remote processing and correction
`location, the raw, digitized image information is processed
`in a conventional fashion to derive luminance and chromi(cid:173)
`nance. A green layer interpolation is effected, and color
`difference generation of Red-Green and Blue-Green is
`effected.
`
`6
`The image capture defect or correction maps stored for
`the particular camera are retrieved from a source using the
`camera ID code. In the examples described below, the
`correction maps are stored in a central or somewhat distrib-
`5 uted repository (data bank) of any suitable storage medium
`and format and accessed in a variety of ways. When
`accessed, white balance normalization is effected employing
`the color correction map. Color correction is accomplished
`by shifting or scaling any or all pixel values that require
`10 normalization.
`The pixel defect map is employed to identify false data
`bits and to initiate interpolation of scene content from
`surrounding valid pixel values. A processing algorithm may
`be employed that takes into account color information of
`adjacent pixels to generate suitable replacement color infor-
`15 mation for the defective pixel location in a manner well
`known in the art. Similarly, optical aberrations and artificial
`illumination hot spots could also be corrected for, if the
`camera specific correction maps are derived at manufacture
`and stored in the repository. In all cases, correction of the
`20 pixel signal levels can be effected by either a scaling
`function, an offset, or a combination of both, in a manner
`well known in the art.
`Thus, the camera 10 of FIG. 1 is simplified so as to reduce
`the expense of the camera while retaining a high resolution
`and image quality of the resulting displays or reproductions
`of the captured images. In particular, the CCD array pref-
`erably provides a resolution which would be able to produce
`an image equivalent to what the printing industry refers to as
`a screen of 140 when appearing in the intended display
`image or reproduction image medium.
`Turning now to FIGS. 2-4, the camera 10 of FIG. 1 is
`preferably employed in a first system embodiment of the
`invention employing an interactive network of the type
`disclosed in the above-referenced' 420 patent, incorporated
`herein by reference, modified to provide the signal process-
`ing and correction of the raw, digitized image information
`into a video image signal. The structure and organization of
`the interactive network, popularly referred to as the "Infor(cid:173)
`mation Highway", are not yet known, as the specifications
`and standards are presently subject to negotiations between
`government and a wide cross-section of telecommunication
`and imaging industry representatives. Presumably, house(cid:173)
`to-house wire or optical fiber cable will be a principal
`communication mode, although satellite transceiver com(cid:173)
`munication may also be included in the network.
`The HIC or Set-Top Box for interactive TV applications
`will likely be based on a microcomputer controlling signal
`input and output channels and responsive to several user
`inputs, but the specific configuration is not yet known. Two
`possible HIC configurations are described in articles by
`Junko Yoshido appearing in Electronic Engineering TIMES
`in the Nov. 29, 1993, and Dec. 6, 1993, editions at pages 1
`and 8 in each issue. In each case, the system includes
`upstream and downstream data and signal transmission for
`providing the interactive capability of the home user to
`communicate with other users, access network service pro(cid:173)
`viders and select programming. It is not yet known if signal
`communication between peripheral equipment, including a
`personal computer, television, printer, CD player, writer,
`modem or image capture means, including the electronic
`still camera of the present invention, will be by wire cable,
`infra-red light beams, optical fiber or RF transmission. The
`HIC may also be structured as shown in the above-refer-
`65 enced '420 patent to include a remote control, infrared
`transceiver for communication with a remote control unit
`and keyboard, a video cable input from a video camera or
`
`UNIFIED PATENTS INC.
`EX1006
`Page 9 of 14
`
`

`
`5,606,365
`
`7
`VCR, a hard copy printer cable output and a telephone
`company subscriber line interface.
`Despite the present configuration uncertainties, it is pos(cid:173)
`sible to set forth the characteristics of a system in accordance
`with the present invention that may be implemented with the 5
`interactive network and the general characteristics of the
`components under consideration. In accordance with a first
`variation of the first embodiment of the system the present
`invention, a user who purchases a camera 10 shown in FIG.
`1 would sign up for image capture service on the user's 10
`interactive account. The ill code of the user's camera 10 is
`then linked to the user's account, and the correction and
`processing code, including the specific camera correction
`maps, is delivered electronically from the camera manufac(cid:173)
`turing facility and made resident in a network repository, e.g. 15
`a headend computer or a node. Then, in use as described
`below, the user would be able to initiate upstream transmis(cid:173)
`sion of the raw, digitized image information to the network
`for processing and correction and return of the corrected
`image information to the user's HIC for display, storage or 20
`printing.
`Alternatively, since the HIC will likely be based on a
`powerful micro-processor based computer and will have
`high level video signal processing for other reasons, it would
`be possible to do the processing and correction at the user' s 25
`HIe. In this embodiment, the user sign up for service where
`the conversion software along with the camera's correction
`code would be downloaded to the HIC for use by the user.
`This could be effected on a one time basis on registration of
`the camera.
`Turning to FIG. 2, it schematically illustrates a first
`simplified interactive network 50 for accomplishing this
`operation using the Set-Top Box or HIC 54 as the commu(cid:173)
`nication link for both transmitting the raw, digitized image 35
`information from the camera 10 upstream on the network 50
`and receiving the processed and corrected digitized image
`information and/or video image frame for display on the
`user's television 60. The correction and conversion software
`as well as the specific correction code associated with the 40
`camera ID code is provided from the camera manufacturer
`48 to a headend computer 52 coupled to the network 50. In
`a first variation of this embodiment of the system, at the time
`of service enrollment, the specific correction code and
`software for the specific camera serial number would be 45
`transferred to the interactive control node 58 accessed by the
`subscriber/user of the camera 10.
`The Set-Top Box or HIC 54 is coupled to the monitor or
`television 60 in the manner described in the above-refer(cid:173)
`enced '420 patent, for example, to receive and transmit
`video images and user generated signals. The raw, digitized
`image information sets and associated camera ill code are
`received by the HIC 54 through a communications link 62
`(of any of the types described above) from the data com(cid:173)
`munications controller 46 of camera 10. The HIC 54 trans(cid:173)
`mits the raw, digitized image information sets and associated
`camera ill code through house cable 56 and a tap (shown in
`FIG. 4) to a line cable 64 and from there to the interactive
`control node 58.
`At the control node 58, the processing and correction is
`conducted as described above to derive the corrected, digital
`image information set. The control node 58 may take the
`form of that shown in FIGS. 6A and 6B and described in the
`, 420 patent, incorporated herein by reference. The corrected,
`digital image information set in the display image format or
`the video frame is then transmitted back to the HIC 54 on
`house cable 56.
`
`8
`techniques may optionally be
`Data compression
`employed to facilitate the transrilission of the digitiZed
`image information sets from the HIC 54 to the interactive
`control node 58. At the interactive control node 58, each
`compressed raw, digitized image information set is restored
`prior to color processing and correction. Then, the corrected
`image information set is transformed by imaging software to
`an optionally compressed video image frame signal. The
`compressed or un-compressed video image frame signal is
`transmitted back to the HIC 54 for decompression and
`display by the monitor or television 60 or for storage in
`memory of an associated PC or print-out by a color printer
`attached to the HIC 54.
`Alternatively, in a variation, the HIC 54 may include, in
`resident memory, the image correction software for color
`balance and pixel defect correction and for conversion of the
`corrected signals into video signals. In one variation of this
`alternative, the HIC 54 would automatically request and
`retrieve the stored correction code from the interact