IN THE UNITED STATES PATENT AND TRADEMARK OFFICE
`Examiner: KE, PENG
`
`Atty Docket No. GRND-L5
`
`Examiner: KE,PENG
`
`Atty Docket No. GRND-L6
`
`App'n No.: 90/012,689
`Patent No.: 7,542,035
`Confirmation No.: 2147
`
`App'n No.: 90/012,590
`Patent No~: 6,243,099
`Confirmation No.: 2143
`
`DECLARATION OF FORD OXAAL
`
`l, Ford Oxaal, declare as follows:
`
`I reside at 48 Western A venue, · Cohoes, NY 1204 7.
`
`I am the inventor of US Patent Numbers 6,243,099 and 7,542,035 {respectively the
`'099 and '035 patents), the patents being reexamined here, and of US Patent
`Number 5,684,937 (the '937 patent), which is incorporated by reference in the
`patents being reexamined. I make this declaration in defense of the '099 and '035
`patents being reexamined.
`
`As stated in the '93 7 patent for the system that I disclosed therein . in 1992, "The
`princip[al} purpose of the present invention is to provide an image processing
`system which uses the two methods [disclosed in the patent] to form visible stimuli
`which depict visual shape." ("937 patent, col. 1, lines 31-34 (emphasis added). As
`c.ommon in image processing technology, the implementation involved analytical
`la ~nd 2a-4b), and a mathematical mapping or
`ray-tracing (e.g., Figs.
`
`~R=ul=e ~13=2'--"'D=ec=l=ar....,a=ti~.on~o=f~F-"'ord~O=x=a=ai'-,----"A~pi!-=or=iJ-=15=,-""-2=01=3-------Page 1
`
`

`

`transformation (e.g., cols. 5-6), implemented by an algorithm applying geometric
`formulas on a pixel-by-pixel basis (e.g., cols. 9-1 0). This technology was useful
`for "de-warping" a "full-surround image" such as captured by a fish-eye lens and
`camera in the sense of presenting a human viewer of the displayed irnage with
`shapes that appear comfortably "normal", whether in linear perspective or in
`circular perspective.
`
`Years later I learned of the "de-warping" image processing technology of IPIX,
`previously known as Omniview, which was disclosed in US Patent Number Re
`36,207 (the Zimmerman patent, originally filed 1991). That image processing
`technology also involved analytical ray tracing, applying geometric fonnulas to
`map an image (sensor) pixel to a displayed pixel (e.g., cols. 5-7), wherein a
`conceptual hemisphere (Fig. 4) was shown as part of the mathematical or
`geometric analysis to determine the geometric transformations to be applied to the
`image pixels.
`
`In the 1991-1998 timeframe, I was aware ofsuch image processing technologies as
`that used by IPIX. However evert from my own experience in implementing the
`'937 patent, the software had to be custom-written and thus was too inflexible for
`my purposes.
`
`Prior to flling the provisional patent application on January 12, 1998 that underlay
`the patents being reexamined, I saw the possibility of applying a different
`technology, that of computer graphics, to the presentation to the human viewer of
`~·full-surround image data." Computer graphics technology- used to create virtual
`3D worlds -had advanced to both software and hardware systems such as OpenGL
`to allow flexible creation of 3D objects that "existed" in the virtual world of
`
`=R=nl=e~l3=2~D=e=c=la=ra=t=io=n~o~fF~. =~=d~O=x=aa=I_-=A~p=r•~·t=l5=·=2=01=3 _______________ Page2
`
`

`

`computer memory and rendering of those object.~ using "texture mapping" to in
`essence paste textures onto the virtual objects for viewing from different angles
`and fields of view. This was first a technology of virtual world synthesis.
`Although real world images were used as textures, it was to create a synthetic
`virtual world, often for fantasy games. Part of that virtual world might be the use
`of pictures of real-world environments to reflect from a virtual object intended to
`I did not and do not know of anyone then applying the
`be shiny (reflective).
`computer graphics technology to create a virtual object on the inside of which (like
`a hemisphere in a planetarium) would be projected the same view presented to a
`fish-eye camera and then using computer graphics technology to view different
`parts of that image-mapped inside of the virtual object from the same point from
`which it was projected. This was what I believe I first did as shown in the code in
`Figures 9 and 10 of the '099 patent and in the code deposit for the '03 5 patent.
`
`This approach was counterc.intuitive relative to my '937 patent (and !PIX)
`analytical image processing approach because it added a step of creating an
`intermediate virtual object resident in computer memory. However~ it involved far
`less custom programming and thus was much more flexible and scalable. Unlike
`the pixel-to-pixel mapping of the analytical image processing approach that was
`efficient for a predetermined frustum of view (only those pixels need to be
`mapped), among the inventions claimed in the reexamined patents are~ (1) a
`polyhedral CG object (thus "p-surface;') is created using CG functions, the object
`approximating (see '099 patent, col. 7, lines 22-23, 31-34; col. 7, line 64 to col. 8,
`'035 patent, col.8 lines 54-55, 62-65, col. 9, lines 27-34) a smooth
`line 5;
`hemisphere with greater fidelity with the increase of the number of primitives,
`typically triangles (50X50 in the code example of the reexamined patents), (2) onto
`the inside of the p-surface the. full-surround image is texture-mapped by "textel"
`
`· =•a=ra=t=io=n~o~fF~o=ro~O=x=aru=--~A=p=ri=l=15=,=20=1=3~--~---------Page3
`· ~=·
`=R=ul=e~l3=2~D=·
`
`

`

`flat triangles using the CG system - not pixel by pixel - as if projected from a
`particular CG point of projection (see '099 patent, col. 5 lines 30-49 [''point VP"]),
`(3) the texture-mapped p-surface "exists" as a virtual object in memory, (4)
`different portions of the inside of the texture-mapped p-sutface may be viewed
`simultaneously using the CG syste1n (as intended for the independent viewing
`system, '099 patent coL 6, lines 34-40; '035 patent, col. 7, line 65 to col. 8, line 5),
`and (5) by making the CG viewpoint coincident with the point from which the full
`surround image was projected, thus enabling a viewer to interactively navigate the
`view, looking up down and all around, in normal perspective, with the CG
`processor/system doing all the work. (see '099 patent, col. 5 lines 30-49 ["point
`Thus, the additional step of creating a texture-mapped p-surface in
`VP"]).
`memory allowed varying approximations according to camera resolution and CG
`processor capability {particularly for video application) as well as simultaneous
`independent viewing. Systems which map the full-surround image all the way,
`pixel-by-pixel, to the view screen, may employ a pixel accurate or better
`mathematical model, but do not gain the flexibility benefits of the CG
`methodology. For example, the claimed invention enables a user to interactively
`change the direction of view and thereby view any selected portion of the full
`surround image in normal perspective, without requiring custom software to
`recompute the view each time the direction of view is changed.
`
`This aspect of the inventions of the reexamined patents has enjoyed considerable
`the 360-degree video
`in
`implemented since 2006
`commercial success as
`the current owner of the patents. See
`surveillance systems distributed by
`In those systems, a sequence of full(cid:173)
`http://www.oncamgrandeye.com/.
`surround images are captured by a fish-eye video camera and are texture-mapped
`using OpenGL onto the inside of a p-surface as if projected from the pqint of view
`
`"'"'R~u=le'--"'1=32""·-=D=ec~la'""'r...,a""'"tio=n.<....:o=f'-"F'""o=rd,._=O~xaa=,_._l_-~A,..,p:.:..cri,..l =15=··-=2=01=3c..._._ _ _ _ _ .. _ _ _ Page 4
`
`

`

`of the camera, and multiple views of a scene can be displayed simultaneously
`using OpenGL operations. Even with _increased processor speed, such multiple
`displays strain the prior art pixel-by'"pixel mapping systems.
`
`The commercial success of this approach is also shown by its use in Google Street
`View.
`
`I further declare that all statements made on my own knowledge are true and all
`statements made on infonnation and belief are believed to be true; and further that
`these statements were made with the knowledge that willful false statement and the
`like punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of
`the United States Code, and may jeopardize the validity of the present application or
`any patent issued thereon.
`
`Dated: April15, 2013
`
`0~('\.~~a-,
`Ford Oxaal
`
`--
`
`Rule. 132Declaration of Ford Oxaal- April IS, 2013
`
`Page 5
`
`