(12) United States Patent
`Steinberg et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,574,016 B2
`Aug. 11, 2009
`
`US007574016B2
`
`4,975,969 A 12/1990 Tal
`5,008,946 A
`4, 1991 Ando
`tsu .................. 358/227
`RE33,682 E
`9, 1991 Hi
`5,051.770 A
`9, 1991 CON,
`W - J.
`5,111,231 A
`5/1992 Tokunaga
`
`(Continued)
`FOREIGN PATENT DOCUMENTS
`1 12831.6 A1
`8/2001
`
`EP
`
`(Continued)
`OTHER PUBLICATIONS
`
`P. Vuylsteke, et al., Range Image Acquisition with a Single Binary
`Encoded Light Pattern, IEEE Transactions on Pattern Analysis and
`Machine Intelligence, abstract printed from http://cs.dl.computer.org/
`comp/trans/tp/1990/02/i0148abs.htm, 1990, 1 page.
`(Continued)
`
`Primary Examiner Daniel G Mariam
`Assistant Examiner Manav Seth
`(74) Attorney, Agent, or Firm—Andrew V. Smith
`(57)
`ABSTRACT
`
`A method of processing a digital image using face detection
`within the image achieves one or more desired image pro
`cessing parameters. A group of pixels is identified that corre
`spond to an image of a face within the digital image. Default
`values are determined of one or more parameters of at least
`Some portion of the digital image. Values are adjusted of the
`one or more parameters within the digitally-detected image
`based upon an analysis of the digital image including the
`image of the face and the default values.
`
`24 Claims, 18 Drawing Sheets
`
`210
`
`212
`
`214
`
`(54) DIGITAL IMAGE PROCESSING USING FACE
`DETECTION INFORMATION
`
`(75) Inventors: Eran Steinberg, San Francisco, CA
`(US); Yury Prilutsky, San Mateo, CA
`(US); Peter Corcoran, Claregalway
`(IE); Petronel Bigioi, Galway (IE)
`(73) Assignee: FotoNation Vision Limited, Galway
`(IE)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1142 days.
`(21) Appl. No.: 10/608,810
`(22) Filed:
`Jun. 26, 2003
`
`(*) Notice:
`
`(65)
`
`Prior Publication Data
`
`Sep. 14, 2006
`
`US 2006/0204055A1
`(51) Int. Cl
`(2006.01)
`Gook 9/00
`(2006.01)
`H04N 7/8
`(52) U.S. Cl. ....................... 382/103: 382/118; 382/254;
`348/152: 348/169; 348/207.99
`(58) Field of Classification Search ................... 348/61,
`348/68, 77 78,143, 152, 155, 158, 169,
`348/207.99, 220.1-229.1: 382/103, 115-118,
`382/254, 274
`See application file for complete search history.
`References Cited
`U.S. PATENT DOCUMENTS
`
`(56)
`
`9, 1977 Mashimo et al. .......... 354.23 D
`4,047,187 A
`3/1982 Stauffer ...................... 250,201
`4,317,991 A
`1, 1983 Stauffer ..........
`... 354,195
`4,367,027 A
`9, 1983 Mashimo et al. .............. 354/23
`RE31,370 E
`5, 1984 Murakoshi
`4,448,510 A
`1/1987 Hiramatsu .................. 358/227
`4,638,364 A
`1, 1989 Izumi et al.
`4,796,043 A
`4,970,663 A 11, 1990 Bedell et al.
`4,970,683 A 11, 1990 Harshaw et al.
`
`
`
`220
`
`222
`
`224
`
`Google Exhibit 1005 - Google v. CSI
`IPR2025-00877 - Page 001
`
`

`

`US 7,574,016 B2
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`9, 1992 Ueno et al.
`5,150,432 A
`5,161,204 A 1 1/1992 Hutcheson et al.
`5,164,992 A 1 1/1992 Turk et al. ..................... 382.2
`5,291,234 A
`3, 1994 Shindo et al. ............... 354,402
`5,305,048 A
`4, 1994 Suzuki et al.
`5,331.544 A
`7, 1994 Lu et al.
`5,353,058 A 10/1994 Takei
`5,384,615 A
`1/1995 Hsieh et al.
`5.432,863. A
`7, 1995 Benati et al.
`5,450,504 A
`9, 1995 Calia
`5,465,308 A 1 1/1995 Hutcheson et al.
`5.488.429 A
`1/1996 Kojima et al. ............... 348,653
`5,493.409 A
`2f1996 Maeda et al.
`5,638,136 A
`6/1997 Kojima et al. ............... 348,653
`5,710,833. A
`1/1998 Moghaddam et al. ....... 382,228
`5,715,325 A
`2/1998 Bang et al.
`5,724.456 A
`3/1998 Boyack et al. .............. 382.274
`5,745,668 A
`4, 1998 Poggio et al. ............... 395/175
`5,764,803 A
`6/1998 Jacquin et al.
`5,771,307 A
`6, 1998 Lu et al.
`5,774,129 A
`6/1998 Poggio et al. ............... 345,441
`5,781,650 A
`7, 1998 Lobo et al. ........
`... 382f1 18
`5,812, 193 A
`9, 1998 Tomitaka et al. .....
`... 348,369
`5,818,975 A 10, 1998 Goodwin et al. ..
`... 382.274
`5,835,616 A 11, 1998 Lobo et al. ........
`... 382f1 18
`5,844,573 A 12/1998 Poggio et al. ............... 345,441
`5,850,470 A 12/1998 Kung et al.
`5,852,669 A 12/1998 Elftheriadis et al.
`... TO7.6
`5,852,823. A 12/1998 De Bonet ....
`... 382f1 18
`RE36,041 E
`1/1999 Turk et al. ..
`5,870,138 A
`2, 1999 Smith et al. ................. 348/143
`5,905,807 A
`5/1999 Kado et al.
`5,911,139 A
`6/1999 Jain et al. ...................... 707/3
`5,966,549 A 10, 1999 Hara et al.
`... 382.282
`5,978,519 A 11/1999 Bollman et al. ...
`5.991,456 A 11/1999 Rahman et al. ............. 382,254
`6,028,960 A
`2/2000 Graf et al.
`6,035,074 A
`3/2000 Fujimoto et al.
`6,061,055 A
`5, 2000 Marks
`6,072,094. A
`6/2000 Karady et al. ............... 585,538
`6,097.470 A
`8/2000 Buhr et al. .................... 355/38
`6,101,271 A
`8/2000 Yamashita et al. .......... 382, 167
`6,108,437 A
`8, 2000 Lin
`6,128,397 A 10/2000 Baluja et al. ................ 382,118
`6,128,398 A 10/2000 Kuperstein et al.
`6,134,339 A 10/2000 Luo
`6,148,092 A 1 1/2000 Qian .......................... 382,118
`6,151,073. A 11/2000 Steinberg et al. ............ 348,371
`6,173,068 B1
`1/2001 Prokoski
`6,188,777 B1
`2/2001 Darrell et al. ............... 382,103
`6, 192,149 B1
`2/2001 Eschbach et al. ............ 382,168
`6,246,779 B1
`6/2001 Fukui et al.
`6,246,790 B1
`6/2001 Huang et al.
`6,249,315 B1
`6/2001 Holm ......................... 348,251
`6,252,976 B1
`6/2001 Schildkraut et al.
`6,263,113 B1
`7/2001 Abdel-Mottaleb et al. ... 38.2/237
`6,268,939 B1
`7/2001 Klassen et al. .............. 358,518
`6,278.491 B1
`8/2001 Wang et al.
`6,282,317 B1
`8/2001 Luo et al. ................... 382,203
`6,301,370 B1 10/2001 Steffens et al. .............. 382,103
`6,301,440 B1
`10/2001 Bolle et al.
`6,332,033 B1
`12/2001 Qian .......................... 382,124
`6,349,373 B2
`2/2002 Sitka et al. ..
`... 711/161
`6,351,556 B1
`2/2002 Louiet al. .................. 382, 164
`6,393,148 B1
`5/2002 Bhaskar ...................... 382/169
`6,404,900 B1
`6/2002 Qian et al.
`382,103
`6,407,777 B1
`6/2002 DeLuca ...................... 348,576
`6.421,468 B1
`7/2002 Ratnakar et al. ............ 382,254
`6.426,779 B1
`7/2002 Noguchi et al.
`6,438,264 B1
`8/2002 Gallagher et al. ........... 382, 167
`6,456,732 B1
`9/2002 Kimbell et al. ............. 382/112
`
`10/2002 Kumada et al. ............. 345,590
`6,459,436 B1
`10/2002 Kresch
`6,463,163 B1
`10/2002 Gilman et al. ............... 358/19
`6,473,199 B1
`12/2002 Gotsman et al. ............ 382,224
`6,501,857 B1
`12/2002 Nishida ......
`707/104.1
`6,502,107 B1
`1/2003 Hong et al. ................. 382,103
`6,504,942 B1
`1/2003 Luo et al. ................... 382,165
`6,504,951 B1
`2/2003 Parulski et al.
`396.287
`6,516,154 B1
`2/2003 Yan ............................ 382,118
`6,526,161 B1
`3/2003 Kinjo
`6,529,630 B1
`4/2003 Ishikawa et al.
`6,549,641 B2
`5/2003 Brogliatti et al.......... TO7 104.1
`6,564.225 B1
`5/2003 Shiimori ..................... 725/105
`6,567,983 B1
`7/2003 Anderson et al.
`6,587,119 B1
`6,661,907 B2 12/2003 Ho et al.
`6,700.999 B1* 3/2004 Yang .......................... 382,118
`6,747,690 B2
`6/2004 Molgaard
`6,754,368 B1
`6/2004 Cohen
`6,760,485 B1
`7/2004 Gilman et al.
`6,778,216 B1
`8, 2004 Lin
`6,792,135 B1
`9/2004 Toyama
`6,816,611 B1
`1 1/2004 Hagiwara et al.
`6.829,009 B2 12/2004 Sugimoto
`6,900,840 B1
`5/2005 Schinner et al.
`6,937,773 B1
`8, 2005 Nozawa et al.
`6,940,545 B1
`9/2005 Ray et al.
`6.959,109 B2 10/2005 Moustafa
`6,965,684 B2 11/2005 Chen et al.
`6,977,687 B1
`12/2005 Suh
`7,027,621 B1
`4/2006 Prokoski
`7,034,848 B2
`4, 2006 Sobol
`7,035,462 B2
`4/2006 White et al.
`7,039,222 B2
`5, 2006 Simon et al.
`7,042,511 B2
`5, 2006 Lin
`7,043,465 B2
`5, 2006 Pirim
`7.057,653 B1
`6/2006 Kubo
`7,106,374 B1
`9, 2006 Bandera et al.
`7,106,887 B2
`9/2006 Kinjo
`7,110,569 B2
`9/2006 Brodsky et al.
`7,194,114 B2
`3/2007 Schneiderman
`7,227.976 B1
`6/2007 Jung et al.
`7,269,292 B2
`9/2007 Steinberg
`7,295.233 B2 11/2007 Steinberg et al.
`7,315,630 B2
`1/2008 Steinberg et al.
`7,315,631 B1
`1/2008 Corcoran et al.
`7,317,815 B2
`1/2008 Steinberg et al.
`7.336,821 B2
`2/2008 Ciuc et al.
`7.362,368 B2
`4/2008 Steinberg et al.
`7,403,643 B2
`7/2008 Ianculescu et al.
`7.440,593 B1
`10/2008 Steinberg et al.
`2001/0005222 A1
`6/2001 Yamaguchi
`2001/0028731 A1 10, 2001 Covell et al. ................ 382,118
`2001/0031142 A1 10, 2001 Whiteside .................... 396.61
`2001/0038714 A1 11/2001 Masumoto et al.
`2002/0105662 A1
`8/2002 Patton et al. ................. 358/19
`2002/01061 14 A1
`8/2002 Yan et al. .................... 382,118
`2002/01 14535 A1
`8/2002 Luo ........................... 382.282
`2002/0118287 A1
`8/2002 Grosvenor et al.
`2002/0136433 Al
`9, 2002 Lin ............................ 382,118
`2002/0168108 Al 1 1/2002 Louiet al. .................. 382,190
`2002/0172419 A1 1 1/2002 Lin et al. .................... 382, 167
`2002/0176609 A1 11, 2002 Hsieh et al.
`2002fO181801 All
`12/2002 Needham et al.
`2002/019 1861 Al 12/2002 Cheatle ...................... 382.282
`2003/0023974 A1
`1/2003 Dagtas et al.
`2003/0025812 A1
`2/2003 Slatter ..................... 348,240.2
`2003.0035573 A1
`2/2003 Dutta et al. ...
`... 38.2/128
`2003/004.8950 A1
`3/2003 Savakis et al. .............. 382,225
`2003/0052991 A1
`3/2003 Stavely et al. ............... 348/370
`2003/0059107 A1
`3, 2003 Sun et al. .....
`382,165
`2003/0059121 A1
`3/2003 Savakis et al. .............. 382,239
`2003/0071908 A1
`4/2003 Sannoh ....................... 348/345
`2003/0084.065 A1
`5/2003 Lin et al. ................. TO7 104.1
`
`
`
`Google Exhibit 1005 - Google v. CSI
`IPR2025-00877 - Page 002
`
`

`

`US 7,574,016 B2
`Page 3
`
`6/2003 Flickner et al. ............. 348.150
`2003/0107649 A1
`6/2003 Goldberg .................... 382,115
`2003.0118216 A1
`7/2003 Geng
`2003/O123713 A1
`7/2003 Krishnamurthy et al.
`2003.01.23751 A1
`7/2003 Yamazaki et al.
`2003. O142209 A1
`8, 2003 Lin
`2003. O151674 A1
`9, 2003 Edwards et al.
`2003/O169907 A1
`2003/0202715 A1 10/2003 Kinjo ......................... 382.282
`2004/0022435 A1
`2, 2004 Ishida
`2004/OO95359 A1
`5, 2004 Simon et al.
`2004/O120399 A1
`6, 2004 Kato
`2004/0170397 A1
`9, 2004 Ono
`2004/017.5021 A1
`9, 2004 Porter et al.
`2004/0179719 A1
`9, 2004 Chen et al.
`2004/0218832 A1 11/2004 Luo et al.
`2004/0228505 A1 1 1/2004 Sugimoto
`2004/0264744 Al 12/2004 Zhang et al.
`2005/0041121 A1
`2/2005 Steinberg et al.
`2005.0068446 A1
`3/2005 Steinberg et al.
`2005.0068452 A1
`3/2005 Steinberg et al.
`2005.0089218 A1
`4, 2005 Chiba
`2005, 0104848 A1
`5/2005 Yamaguchi et al.
`2005/0105780 A1
`5, 2005 Ioffe
`2005, 0140801 A1
`6/2005 Prilutsky et al.
`2005.0185054 A1
`8, 2005 Edwards et al.
`2005/0275721 A1 12, 2005 Ishii
`2006, OOO6077 A1
`1/2006 Mosher et al.
`2006,0008152 A1
`1/2006 Kumar et al.
`2006,0008173 A1
`1/2006 Matsugu et al.
`2006, OO18517 A1
`1/2006 Chen et al.
`2006, OO29265 A1
`2/2006 Kim et al.
`2006, OO39690 A1
`2/2006 Steinberg et al.
`2006/0050933 A1
`3, 2006 Adam et al.
`2006/0098875 A1
`5/2006 Sugimoto
`2006/0098890 A1
`5/2006 Steinberg et al.
`2006/O120599 A1
`6/2006 Steinberg et al.
`2006, O140455 A1
`6/2006 Costache et al.
`2006, O1471.92 A1
`7/2006 Zhang et al.
`2006/0203107 A1
`9/2006 Steinberg et al.
`2006/0203108 A1
`9/2006 Steinberg et al.
`2006/0204034 A1
`9/2006 Steinberg et al.
`2006/0204054 A1
`9/2006 Steinberg et al.
`2006/0204055 A1
`9/2006 Steinberg et al.
`2006/0204056 A1
`9/2006 Steinberg et al.
`2006/0204057 A1
`9/2006 Steinberg
`2006/020411.0 A1
`9/2006 Steinberg et al.
`2006/0210264 A1
`9/2006 Saga
`2006/0215924 A1
`9/2006 Steinberg et al.
`2006/0257047 A1 1 1/2006 Kameyama et al.
`2006/0268150 Al 1 1/2006 Kameyama et al.
`2006/026927O A1 11, 2006 Yoda et al.
`2006/028O380 A1 12, 2006 Li et al.
`2006/0285754 A1 12/2006 Steinberg et al.
`2006,029.1739 A1 12, 2006 Li et al.
`2007/0070440 A1
`3/2007 Li et al.
`2007/0071347 A1
`3/2007 Li et al.
`2007/011 O305 A1
`5, 2007 Corcoran et al.
`2007/0116379 A1
`5, 2007 Corcoran et al.
`2007, 0116380 A1
`5, 2007 Ciuc et al.
`2007/O12205.6 A1
`5/2007 Steinberg et al.
`2007/O160307 A1
`7/2007 Steinberg et al.
`2007. O189606 A1
`8, 2007 Ciuc et al.
`2007/O189748 A1
`8, 2007 Drimbarean et al.
`2007/O189757 A1
`8/2007 Steinberg et al.
`2007/02O1724 A1
`8/2007 Steinberg et al.
`2007/0296833 A1 12/2007 Corcoran et al.
`2008, OO13798 A1
`1/2008 Ionita et al.
`2008.0031498 A1
`2/2008 Corcoran et al.
`2008.OO37827 A1
`2/2008 Corcoran et al.
`2008, 0037838 A1
`2/2008 Ianculescu et al.
`2008.OO37839 A1
`2/2008 Corcoran et al.
`2008, 003784.0 A1
`2/2008 Steinberg et al.
`2008.0043.122 A1
`2/2008 Steinberg et al.
`
`2/2008 Ciuc et al.
`2008.004997O A1
`3/2008 Steinberg et al.
`2008/0055433 A1
`3/2008 David et al.
`2008.0075385 A1
`7/2008 Petrescu et al.
`2008. O175481 A1
`8, 2008 Ionita et al.
`2008/0205712 A1
`2008, 0240555 A1 10, 2008 Nanu et al.
`
`FOREIGN PATENT DOCUMENTS
`1626.569
`2, 2006
`EP
`2370 438 A
`6, 2002
`GB
`5-260360
`10, 1993
`JP
`05-164475
`6, 2005
`JP
`06-005662
`1, 2006
`JP
`06-254358
`9, 2006
`JP
`WO O2/O52835
`T 2002
`WO
`WO WO 2007-095477
`8, 2007
`WO WO 2007-095483
`8, 2007
`WO WO 2007-095553
`8, 2007
`WO WO 2007-142621
`12/2007
`WO WO 2008-O15586
`2, 2008
`WO WO 2008-018887
`2, 2008
`WO WO 2008-02328O
`2, 2008
`WO WO 2008-104549
`9, 2008
`
`OTHER PUBLICATIONS
`
`D. Shock, et al., Comparison of Rural Remote Site Production of
`Digital Images Employing a Film Digitizer or a Computed Radiog
`raphy (CR) System, 4th International Conference on Image Manage
`ment and Communication (IMAC '95), Aug. 21-24, 1995, Abstract
`printed from http://cs.dl.computer.org/comp/proceedings/imac/
`1995,7560/00/756.00071 abs.htm.
`J.W. Sublett, et al. Design and Implementation of a Digital
`Teleultrasound System for Real-Time Remote Diagnosis, Eight
`Annual IEEE Symposium on Computer-Based Medical Systems
`(CBMS '95)m Jun. 9-10, 1995, Lubbock, Texas, Abstract printed
`from http://cs.dl.computer.org/comp/proceedings/cbms/1995/7117/
`007 1170292.abs.htm.
`Sing Bing Kang, et al., A Multibaseline Stereo System with Active
`Illumination and Real-Time Image Acquisition, Fifth International
`Conference on Computer Vision, Jun. 20-23, 1995, Massachusetts
`Institute of Technology, Cambridge, Massachusetts, Abstract printed
`from http://cs.dl.computer.org/comp/proceedings/iccv/1995/7042/
`0070420088abs.htm.
`M. Shand, Flexible Image Acquisition Using Reconfigurable Hard
`ware, IEEE Symposium of FPGA's for Custom Computing
`Machines (FCCM 95), Apr. 19-21, 1995, Napa Valley, California,
`Abstract printed from http://cs.dl.computer.org/comp/proceedings/
`fecm/1995,7086700,7086O125abs.htm.
`Daechul Park, et al., Lenticular Stereoscopic Imaging and Displaying
`Techniques with no Special Glasses, 1995 International Conference
`on Image Processing (vol. 3)-vol. 3, Oct. 23-26, 1995, Washington,
`D.C., Abstract printed from http://cs.dl.computer.org/compf proceed
`ings/icip/1995/7310/03/73103137abs.htm.
`H.H. Garnaoui, et al., Visual Masking and the Design of Magnetic
`Resonance Image Acquisition, 1995 International Conference on
`Image Processing (vol. 1)-vol. 1, Oct. 23-26, 1995, Washington,
`D.C., Abstract printed from http://cs.dl.computer.org/compf proceed
`ings/icip/1995/7310/01/73 100625abs.htm.
`John Dalton, Digital Cameras and Electronic Color Image Acquisi
`tion, Compcon Spring 96 41st IEEE International Conference,
`Feb. 25-28, 1996, San Jose, California, Abstract printed from http://
`csdl.computer.org/comp/proceedings/compcon, 1996/7414/00/
`74.140431 abs.htm.
`Arun Krishman, Panoramic Image Acquisition, 1996 Conference on
`Computer Vision and Pattern Recognition (CVPR 96), Jun. 18-20,
`1996, San Francisco, CA, Abstract printed from http://cs.dl.computer.
`org/comp/proceedings/cvpril 1996/7258/00/72580379abs.htm.
`Tony S. Jebara, 3D Pose Estimation and Normalization for Face
`Recognition, Department of Electrical Engineering, McGill Univer
`sity, A Thesis submitted to the Faculty of Graduate Studies and
`Research in Partial fulfilment of the requirements of the degree of
`Bachelor of Engineering, May 1996, pp. 1-121.
`
`Google Exhibit 1005 - Google v. CSI
`IPR2025-00877 - Page 003
`
`

`

`US 7,574,016 B2
`Page 4
`
`David Beymer, Pose-Invariant Face Recognition Using Real and
`VirtualViews, Massachusetts Institute of Technology Artificial Intel
`ligence Laboratory, A.I.Technical Report No. 1574, Mar. 1996, pp.
`1-176.
`Wen-Chen Hu, et al., A Line String Image Representation for Image
`Storage and Retrieval, 1997 International Conference on Multimedia
`Computing and Systems (ICMCS '97), Jun. 3-6, 1997, Ottawa,
`Ontario, Canada, Abstract printed from http://cs.dl.computer.org/
`comp/proceedings/icmcs/1997/7819/00/78190434abs.htm.
`Yuan Y. Tang, et al., Information Acquisition and Storage of Forms in
`Document Processing, 4th International Conference Document
`Analysis and Recognition (ICDAR'97) vol. 1 and vol.II, Aug. 18-20,
`1997, Ulm, Germany, Abstract printed from http://cs.dl.computer.org/
`comp/proceedings/icdar/1997/7898/00/78980170abs.htm.
`J.-A. Beraldin, et al., Object Model Creation from Multiple Range
`Images: Acquisition, Calibration, Model Building and Verification,
`International Conference on Recent Advances in 3-D Digital Imaging
`and Modeling, May 12-15, 1997, Ottawa, Ontario, Canada, Abstract
`printed from http://cs.dl.computer.org/compf proceedings nrc? 1997/
`7943,00/7943,0326abs.htm.
`Jun Zhang, et al., Face Recognition: Eigenface, Elastic, Matching,
`and Neutral Nets, Proceedings of the IEEE, Sep. 1997, pp. 1423
`1435, vol. 85, No. 9.
`Hiroyuki Aoki, et al. An Image Storage System Using Complex
`Valued Associative Memories, International Conference on Pattern
`Recognition (ICPR 00)-vol. 2, Sep. 3-8, 2000, Barcelona, Spain,
`Abstract printed from http://cs.dl.computer.org/comp/proceedings/
`icpr/2000/0750/02/07502626abs.htm.
`Daniel Skocaj, Range Image Acquisition of Objects with Non-Uni
`form Albedo Using Structured Light Range Sensor, International
`Conference on Pattern Recognition (ICPR 00)-vol. 1, Sep. 3-8.
`2000, Barcelona, Spain, Abstract printed from http://cs.dl.computer.
`org/comp/proceedings?icpril 2000/0750/01/0750 1778abs.htm.
`Reinhold Huber, et al., Adaptive Aperture Control for Image Acqui
`sition, Sixth IEEE Workshop on Applications of Computer Vision,
`Dec. 3-4, 2002, Orlando Florida, Abstract printed from http://cs.dl.
`computer.org/comp/proceedings/wacv/2002/1858/00/18580320abs.
`htm.
`Michal Kozubek, et al., Automated Multi-view 3D Image Acquisition
`in Human Genome Research, 1st International Symposium on 3D
`Data Processing Visualization and Transmission (3DPVT '02), Jun.
`19-21, 2002, Padova, Italy, Abstract printed from http://cs.dl.com
`puter.org/comp/proceedings/3dpvt/2002/1521/00/1521.009 labs.
`htm.
`Nobuyuki Kita, et al., Archiving Technology for Plant Inspection
`Images Captured by Mobile Active Cameras 4D Visible
`Memory—1st International Symposium on 3D Data Processing
`Visualization and Transmission (3DPVT '02), Jun. 19-21, 2002,
`Padova, Italy, Abstract printed from http://cs.dl.computer.org/comp/
`proceedings/3dpvt/2002/1521/00/15210208abs.htm.
`Anna T. Fernandez, et al., Synthetic Elevation Beamforming and
`Image Acquisition Capabilities Using an 8x128 1.75D Array, The
`Technical Institute of Electrical and Electronics Engineers, Abstract
`printed from http://www.ieee-uffc.org/archive/uffc/transtoc/abs/03/
`tO310040.htm.
`Henry A. Rowley, et al., Neutral Network-Based Face Detection,
`IEEE Transactions on Pattern Analysis and Machine Intelligence,
`Jan. 1998, pp. 23-38, vol. 20, No. 1.
`Mathew Gaubatz, et al., Automatic Red-Eye Detection and Correc
`tion, IEEE ICIP 2002, pp. I-804-I-807.
`Ming-HsuanYang, et al., Detecting Faces in Images: A Survey, IEEE
`Transactions on Pattern Analysis and Machine Intelligence, Jan.
`2002, pp. 34-58, vol. 24, No. 1.
`Kresimir Matkovic, et al., The 3D Wunderkammer an Indexing by
`Placing Approach to the Image Storage and Retrieval. Theory and
`Practice of Computer Graphics 2003, Jun. 3-5, 2003, University of
`Birmingham, UK, Abstract printed from http://cs.dl.computer.org/
`comp proceedings/tpcg/2003/1942/00/19420034abs.htm.
`Twins Crack Face Recognition Puzzle, Internet article http://www.
`cnn.com/2003/TECH/ptech/03/10/israel.twins.reut/index.html,
`printed Mar. 10, 2003, 3 pages.
`
`U.S. Appl. No. 10/608,772, entitled “Method of improving orienta
`tion and color balance of digital images using face detection infor
`mation'.
`PCT Application No. PCT/US2006/021393, filed Jun. 2, 2006,
`entitled “Modification of Post-Viewing Parameters for Digital
`Images Using Image Region or Feature Information'.
`PCT Application No. PCT/US2006/060392, filed Oct. 31, 2006,
`entitled “Digital Image Processing Using Face Detection and Skin
`Tone Information'.
`Batur, A. et al., "Adaptive Active Appearance Models.” IEEE, vol. 14.
`No. 11, Nov. 2005, pp. 1707-1721.
`Buenaposada, J., “Efficiently Estimating 1-3, 16 Facial Expressions
`and Illumination in Appearance-Based Tracking.” Proc. British
`Machine Vision Conf. Sep. 4, 2006, XP002494036, 10 pages.
`Cootes, T. F. et al., “A Comparative Evaluation of Active Appearance
`Model Algorithms.” Proc. 9" British Machine vision Conf. Brit.
`Machine Vision Assoc., 1998, pp. 680-689.
`Cootes, T. F. et al., “Modeling Facial Shape and Appearance.” Hand
`book of FaceRecognition, Chapter 3, Springer, 2005, XP002494037.
`26 pgs.
`Cootes, T. F. et al., “On Representing Edge Structure for Model
`Matching.” CVPR vol. 1, 2001, pp. 1114-1119.
`Crowley, J. et al., “Multi-Modal Tracking of Faces for Video Com
`munication.” In Computer Vision and Patent Recognition, 1997.
`http://citeseerist.psu.edu/crowley97multimodal.html.
`Donner, Rene et al., “Fast Active Appearance Model Search using
`Canonical Correlation Analysis.” IEEE, vol. 28, No. 10, Oct. 2006,
`pp. 1690-1694.
`Edwards, G. J. et al., “Advances in Active Appearance Models.”
`Proceedings of the 7th IEEE International Conference on Computer
`Vision (ICCV 99), vol. 1, pp. 137-142, Kerkyra, Greece, Sep. 1999.
`Edwards, G. J. et al., "Learning to Identify and Track Faces in Image
`Sequences.” Automatic Face and Gesture Recognition, Proc. IEEE,
`Apr. 14, 1998, XPO10277593, pp. 260-265.
`Feraud, R. et al., “A Fast and Accurate Face Detector Based on Neural
`Networks.” IEEE, vol. 23, No. 1, Jan. 2001, pp. 42-53.
`Froba, B. et al., “Face Detection With The Modified Census Trans
`form.” Dept. of Applied Elect. Proceedings of 6' IEEE, 2004, pp.
`1-6, XP-002455697.
`Froba, B. et al., “RealTime Face Detection.” Dept. of Applied Elec
`tronics, 2002, XP-2455696A, pp. 1-6.
`Gerbrands, J. On the Relationships Between SVD, KLT, and PCA.
`1981, Pattern Recognition, vol. 14, Nos. 1-6, pp. 375-381.
`Goodall, Charles, “Procrustes Methods in the Statistical Analysis of
`Shape.” Journal of the Royal Statistical Society, Series B., vol. 53,
`No. 2, 1991, pp. 285-339.
`Hou, Xinwen et al., “Direct Appearance Models.” IEEE, 2001, pp.
`I828-833.
`Huang et al., “Detection of human faces using decision trees' 2nd
`International Conference on Automatic Face and Gesture Recogni
`tion (FG 96). p. 248 IEEE Xplore http://doi.ieeecomputersociety.
`org/10. 1109/AFGR/1996.html, 2 pgs.
`International Search Report and Written Opinion, PCT/US2006/
`021393, filed Jun. 2, 2006, entitled, Modification of Post-Viewing
`Parameters for Digital Images Using Image Region or Feature Infor
`mation, ISR/WO dated Mar. 29, 2007, 12 pgs.
`International Search Report and Written Opinion, PCT/US2006/
`060392, filed Oct. 31, 2006, entitled, “Digital Image Processing
`using Face Detection and Skin Tone Information.” ISR/WO dated
`Sep. 19, 2008, 9 pgs.
`International Search Report and Written Opinion, PCT/IB2007/
`003724. Aug. 28, 2008, 9 pgs.
`International Search Report and Written Opinion, PCT/EP2007/
`006540, Nov. 8, 2007, 11 pgs.
`International Search Report and Written Opinion, PCT/EP2007/
`006540, Nov. 7, 2008, 6 pgs.
`International Search Report and Written Opinion, PCT/EP2008/
`001510, May 29, 2008, 13 pgs.
`International Search Report and Written Opinion, PCT/EP2008/
`052329, Sep. 15, 2008, 12 pgs.
`Jones, M. et al., “Fast Multi-View Face Detection. Mitsubishi Elec
`tric Research Laboratories, 2003.http://www.merl.com/papers/docs/
`TR2003-96.pdf, 10 pgs.
`
`Google Exhibit 1005 - Google v. CSI
`IPR2025-00877 - Page 004
`
`

`

`US 7,574,016 B2
`Page 5
`
`Kouzani, A. Z., “Illumination-Effects Compensation in Facial
`Images.” IEEE, 1999, p. VI-840-VI-844.
`Lai. J. et al., “Face recognition using holistic Fourier in variant
`features.” Pattern Recognition 34 (2001), pp. 95-109. http://
`digitalimaging.inf.brad.ac.uk/publication/prj4-1.pdf.
`Lei, Z. et al., “A CBIR Method Based on Color-Spatial Feature.”
`IEEE Region 10th Ann. Int. Conf. 1999 (TENCON'99, Cheju, Korea,
`1999).
`Lienhard, R. et al., “A Detector Tree of Boosted Classifiers for Real
`TimeObject Detection and Tracking.” Proc. of the 2003 Intl. Conf. on
`Multimedia and Expo Vo. 1, pp. 277-280 (2003), ISBN: 0-7803
`7965-9, Publisher IEEE Computer Society, Washington, DC, USA.
`Matthews, I. et al., “Active appearance models revisited.” Interna
`tional Journal of ComputerVision, vol. 60, No. 2, pp. 135-164. 2004.
`MekuZ, N. et al., "Adaptive Step Size Window Matching for Detec
`tion.” Proc. Of the 18" Intl. Conf. on Pattern Recog., vol. 2, Aug.
`2006, pp. 259-262.
`Mitra, S. et al., “Gaussian Mixture Models Based on the Frequency
`Spectra for Human Identification and Illumination Classification.”
`Automatic Identification Advanced Tech., 4" IEEE Workshop on
`Buffalo, NY, IEEE, 2005, pp. 245-250.
`Nordstrom, M. M. et al., “The IMM Face Database: An Annotated
`Dataset of 240 Face Images.” Informatics and Mathematical Model
`ling; Tech. Univ. of Denmark, Denmark: 2004.
`Ohta, Y-I. et al., “Color Information for Region Segmentation.”
`Comp. Graphics and Image Processing, Academic Press, NY. vol. 13,
`No. 3, Jul. 1, 1980, XPO08026458, pp. 222-241.
`Romdhani, S. et al., “Face Identification by Fitting a 3D Morphable
`Model using Linear Shape and Texture Error Functions.” European
`Conf. on Computer Vision, Berlin, DE, Jan. 1, 2002, XP003018283,
`15 pgs.
`Ryu et al., "Coarse-to-Fine Classification for Image-Based Face
`Detection.” Carnegie Melon Univ., Pittsburgh, PA, May 1999, ch. 6.
`p.92. Subsection 8.3.
`Sharma, G. et al., “Digital Color Imaging, IEEE Transactions. On
`Image Processing.” vol. 6, No. 7, 1997, pp. 901-932.
`
`Sim, T. et al., “The CMU Pose, Illumination, and Expression (PIE).”
`Database of Human Faces Robotics Institute, Carnegie Mellon Uni
`versity, Pittsburgh, PA. Tech. Report, CMU-RI-TR-01-02, Jan. 2001,
`9 pages.
`Smeraldi, F. et al., “Facial Feature Detection by Saccatic Exploration
`of the Gabor Decomposition.” Image Processing, 1998 Intl. Conf.
`Oct. 4, 1998, pp. 163-167.
`Soriano, M. et al., “Making Saturated Facial Images Useful Again.”
`Proc. Of the Spie, Spie, Bellingham, VA, vol. 3826, Jun. 17, 1999, pp.
`113-121.
`Stegman, M. B. et al., “FAME-A Flexible Appearance Modelling
`Environment.” IEEE Transactions on Medical Imaging, vol. 22, No.
`10, pp. 1319-1331.
`Stegman, M. B. et al., “Multi-Band Modelling of Appearance.”
`ImageVision Comp., vol. 21. No. 1, Jan. 10, 2003, XP009104697, pp.
`61-67.
`Stricker et al., “Similarity of Color Images.” SPIE Proc., 1995, pp.
`1-12.
`Tianhorng, C. et al., “Texture Analysis and Classification with Tree
`Structured Wavelet Transform.” IEEE Trans. Of Image Processing,
`vol. 2, No. 4, Oct. 1993, pp. 429-441.
`Tjahyadi et al., “Application of the DCT Energy Histogram for Face
`Recognition.” Proc. Of the 2" Intl. Conf. on Information Technology
`for Applicaiotn, ICITA, 2004, pp. 305-310.
`Tkalcic, M. et al., "Colour Spaces Perceptual. Historical and
`Applicational Background.” IEEE, Eurocon, 2003, Sep. 22-24, 2003,
`vol. 1, pp. 304-308.
`Turk, M. et al., “Eigenfaces for Recognition.” Journal of Cognitive
`Neuroscience, Mass. Institute of Technology, vol. 3, No. 1, 1991, pp.
`71-86.
`Wan, S. J. et al., “Variance Based Color Image Quantization for
`Frame Buffer Display.” Color Res. Applicat., vol. 15, No. 1, 1990, pp.
`52-58.
`* cited by examiner
`
`Google Exhibit 1005 - Google v. CSI
`IPR2025-00877 - Page 005
`
`

`

`U.S. Patent
`
`Aug.
`11, 2009
`
`Sheet 1 of 18
`
`US 7,574,016 B2
`
`O2
`Open image
`
`103
`Initiate Picture Rendering
`
`104
`initiate Picture Taking Mode
`
`
`
`YMMMMMS
`
`06
`Were faces found in the image?
`
`
`
`
`
`
`
`116
`Are there any faces in
`the image?
`
`12
`Are there any more
`faces in the image?
`
`
`
`
`
`
`
`18
`Manually
`Mark the
`faces in the
`image
`
`:
`----------------------. YES----
`
`
`
`O
`Exit
`
`108
`Automatically Mark
`faces
`
`
`
`
`
`
`
`
`
`
`
`Are any of the regions
`detected not faces?
`
`YES
`
`4
`Manually
`remove false
`detected faces
`in the image
`
`Y
`120
`Proposed Zooming
`and Panning to
`and fron the
`faces
`
`y
`30
`Detect Orientation
`of faces
`
`Y
`
`colored
`tased on skin
`tone of faces
`
`based
`Crop in
`t of
`of the
`the faces
`
`mm
`
`y
`160
`Digital Fill flash of
`faces
`
`y
`
`|
`170
`Exposure Fill flash
`based on faces
`
`18O
`Automatic focus
`on faces
`
`Figure 1 a
`
`Google Exhibit 1005 - Google v. CSI
`IPR2025-00877 - Page 006
`
`

`

`U.S. Patent
`
`Aug. 11, 2009
`
`Sheet 2 of 18
`
`US 7,574,016 B2
`
`
`
`
`
`
`
`
`
`
`
`1000
`initiate Pre
`Acquisition
`Stage
`
`
`
`
`
`
`
`
`
`10O2
`Determinefload
`Default values
`for image
`attributes
`
`
`
`
`
`
`
`
`
`
`
`
`
`1004
`Camera
`determines
`acquisition
`parameters
`
`1010
`Camera captures
`image
`
`1020
`in camera seeks
`faces
`
`1032
`Exit
`
`
`
`NO
`y
`
`Manually
`add
`face
`regions
`
`1030
`Were faces
`detected?
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Manually
`remove false
`detected
`regions
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1040
`identify, Mark
`egions with
`face(s)
`
`1050
`Compared values of
`attributes in captured
`regions to default inage
`attributes values
`
`
`
`
`
`1060
`Adjust image until
`detected image attributes
`match default image
`attributes
`
`1046
`Manually grade
`selected
`fegions based
`on importance
`
`O70
`Adjust camera acquisition
`parameters to correspond
`with adjustment of image
`attribute adjustments
`
`
`
`1090
`Photographer takes
`picture
`\
`\-
`
`Figure 1b
`
`Google Exhibit 1005 - Google v. CSI
`IPR2025-00877 - Page 007
`
`

`

`U.S. Patent
`
`Aug. 11, 2009
`
`Sheet 3 of 18
`
`US 7,574,016 B2
`
`
`
`
`
`
`
`1104
`Device
`determines
`render ing
`parameters
`
`1110
`Digital image is
`downloaded to the
`device
`
`1120
`Device seeks
`faces
`
`
`
`
`
`
`
`
`
`
`
`1132
`Exit - Continue the
`rendering process
`without change
`
`—NO
`
`
`
`
`
`
`
`1102
`Determinefload
`Default waites
`for in age
`attributes
`
`
`
`
`
`
`
`1100
`Initiate Pre
`rendering Stage
`
`
`
`
`
`1130
`Were faces
`detected?
`
`
`
`
`
`
`
`1140
`identify, Mark
`regions with
`face(s)
`
`150
`Compared values of
`attributes in downloaded
`regions to defaultinage
`attributes values
`
`
`
`
`
`160
`Adjust image until
`detected image attributes
`match default image
`attributes
`
`
`
`
`
`
`
`
`
`114.6
`Manually grade
`selected
`regions based
`on importance
`
`
`
`1170
`Adjust output device
`parameters to correspond
`with adjustinent of image
`attribute adjustments
`
`
`
`
`
`
`
`1134
`Manually
`add
`face
`regions
`
`
`
`
`
`
`
`1144
`Manually
`remove false
`detected
`regions
`
`
`
`1180
`Repeat Process
`for multiple images
`sent to device
`
`190
`Device performs
`rendering
`
`Figure 1C
`
`Google Exhibit 1005 - Google v. CSI
`IPR2025-00877 - Page 008
`
`

`

`U.S. Patent
`
`Aug. 11, 2009
`
`US 7,574,016 B2
`
`
`
`Figure 2a
`
`
`
`222
`
`224
`
`Figure 2b
`
`Google Exhibit 1005 - Google v. CSI
`IPR2025-00877 - Page 009
`
`

`

`U.S. Patent
`
`Aug. 11, 2009
`
`Sheet 5 of 18
`
`US 7,574,016 B2
`
`
`
`s
`
`Google Exhibit 1005 - Google v. CSI
`IPR2025-00877 - Page 0010
`
`

`

`U.S. Patent
`
`Aug. 11, 2009
`
`Sheet 6 of 18
`
`US 7,574,016 B2
`
`
`
`Google Exhibit 1005 - Google v. CSI
`IPR2025-00877 - Page 0011
`
`

`

`U.S. Patent
`
`Aug. 11, 2009
`
`Sheet 7 of 18
`
`US 7,574,016 B2
`
`
`
`Figure 3b
`
`
`
`340
`
`iCure 3C
`
`322
`
`324
`
`SSS
`
`.
`
`Google Exhibit 1005 - Google v. CSI
`IPR2025-00877 - Page 0012
`
`

`

`U.S. Patent
`
`Aug. 11, 2009
`
`Sheet 8 of 18
`
`US 7,574,016 B2
`
`Figure 3e
`
`52
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Figure 3f
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`six
`
`O C C C
`C
`s C
`
`c) X
`
`C S.
`
`C C
`
`s
`
`S
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`370
`
`
`
`
`
`S
`
`{XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
`CXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
`CXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
`
`XXX
`&S
`SSESS&SSSSSSSSSS SEX & XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
`XXXXXXXXXXXXXXXXXXXXXXXXXX