Filed: April 10, 2023
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`
`______________________
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`
`______________________
`APPLE INC.
`
`Petitioner,
`
`v.
`
`SPACETIME3D, INC.
`Patent Owner
`
`______________________
`Case IPR2023-00343
`U.S. Patent No. 9,304,654
`
`______________________
`DECLARATION OF EDDIE BAKHASH
`
`
`
`
`
`
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 1
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`
`I, E. Eddie Bakhash, declare and state:
`
`the President and founder of SpaceTime3D, Inc.
`I am
`1.
`(“SpaceTime3D”), and I am the sole inventor of U.S. Patent No. 9,304,654
`(“the ‘654 Patent”), entitled “System and Method for Displaying a Timeline
`Associated with a Plurality of Applications.” I make this Declaration in
`support of Patent Owner’s Preliminary Response to Petition for Inter Partes
`Review of the ‘654 Patent (“Petition”).
`2.
`In addition to being the inventor of the ‘654 Patent, I am also the
`inventor of several other pending and issued United States patents, including
`U.S. Patent Nos. 7,735,018 (“the ‘018 Patent”), 8,881,048 (“the ‘048 Patent”),
`9,696,868, and 11,112,931, and U.S. Patent Application Nos. 17/388,948,
`17/388,970, 17/388,937, and 17/888,453, all disclosing and claiming new and
`novel two-dimensional (“2D”) and three-dimensional (“3D”) graphical user
`interfaces (“GUIs”).
`3.
`I have a bachelor of science degree from New York University
`Stern School of Business. I also have over 20 years of work experience in the
`computer industry, during which time I designed and developed several 2D and
`3D GUIs, including GUIs for SpaceTime3D and AmericanPearl.com, Inc. As
`such, I consider myself a person of ordinary skill in the art (“POSITA”) with
`respect to the subject matter of the ’654 Patent.
`4.
`SpaceTime3D is an application software company incorporated in
`New York, NY. It is also the assignee of the ‘654 Patent, a true and correct
`copy of which is attached as Exhibit A. The ’654 Patent is a continuation of
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 2
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`the application resulting in the “‘048 Patent,” which is a continuation of the
`application resulting in the “’018 Patent,” which claims priority to a
`provisional patent application that was filed on September 13, 2005, Serial No.
`60/717,019.
`three-dimensional
`improved
`The ‘654 Patent provides “an
`5.
`graphical user interface.” According to the specification, “[b]ecause the 3D
`GUI creates the illusion of infinite space in 3D, it can create a visual history of
`the user’s computing session, whereby the user can visit past visual computing
`events (or a snapshot in time) by simply navigating to previously recorded
`states or viewpoints. Accordingly, the 3D GUI can function as a visual
`chronological history of the user’s computing session.” Exhibit A at col. 5, ll.
`6-21.
`
`The ‘654 Patent specification describes several ways of presenting
`6.
`and interacting with information in 3D space. The claimed invention focuses
`on one way, where images of applications are presented in 3D space, and
`individual applications are interacted with, one at a time, in 2D space. In
`particular, Claim 1 of the ‘654 Patent provides for a 3D GUI where (1) a user
`provides a plurality of inputs, (2) corresponding applications are open, and (3)
`images of the applications are generated and presented in 3D space, arranged
`chronologically, with the first one being in the foreground, the second one in
`the background, etc. Id. at col. 37, l. 44 – col. 38, l. 35.
`7. When the user interacts with the first image (in the foreground), a
`corresponding active application is presented to the user in 2D space (the GUI
`switches from 3D to 2D space), where the application replaces the plurality of
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 3
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`images. Id. The user can then interact with the application in 2D space to
`modify the application-specific data associated therewith. Id.
`8.
`In other words, the claimed invention allows the user to browse
`through open applications (or images thereof) in 3D space and to modify a
`particular application by switching from 3D space, where the plurality of
`images are presented, to 2D space, where an active application (corresponding
`to the selected image) is presented, thereby allowing the user to interact with
`the active application to modify data associated therewith. The claimed
`invention further provides that when switching from 3D space to 2D space, the
`selected webpage replaces the plurality of images. Id. It is this switching
`back-and-forth between 2D and 3D space, with images in 3D space being
`updated to where the user last left off, that creates “a visual chronological
`history of the user’s computing session.” See, e.g., id. at col. 5, ll. 6-21.
`9.
`The claims of the ‘654 Patent are particularly advantageous in
`ultra-mobile personal computer (“UMPC”) devices (e.g., smartphones) having
`limited screen sizes, reduced processing power, and limited electrical (e.g.,
`battery) power, by allowing a user to interact with a first, active application in a
`2D space, and switching to a second, active application by (i) replacing the first
`application in 2D space with images of a plurality of applications in 3D space,
`(ii) allowing the user to select an image of the second application, and (iii)
`replacing the plurality of images in 3D space with the second, active
`application in a 2D space.
`10. By interacting with individual, active applications in 2D space and
`scrolling through images of a plurality of webpages in 3D space, which is a
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 4
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`non-conventional technique that exploits the fact that images of applications
`can be more easily manipulated in 3D space than the active applications
`themselves, the claimed invention improves the functionality of the computing
`(e.g., UMPC) devices by requiring less power, minimal (or reduced)
`processing, and smaller screen sizes.
`the
`identifies problems with
`11. The ‘654 Patent specifically
`conventional system of inputting information into a computer to achieve a
`given output, which involves a series of tedious steps—repetitive mouse clicks
`and keyboard inputs—to run applications and documents or navigate to certain
`information. See, e.g., ‘654 Patent at col. 1, ll. 38-55. To switch to a different
`application or document on a computer, the user often has to close his/her
`current applications and documents, hide them or overlap them on a finite
`desktop display by drawing them on top of each other, and then mine through
`folder within folder to find them again at a later date. Because the user’s
`desktop screen real estate is finite, he/she must redo these same tasks over and
`over again. Id.
`12. This conventional system wastes the end user’s time by (1)
`requiring many mouse clicks to open and close applications and/or documents,
`(2) requiring the user to remember the combinations of programs and
`documents he/she may need for a given purpose, and (3) requiring the user to
`create elaborate hierarchical folder systems to aid in the process of storing and
`recalling applications and/or documents. Id.
`13.
`In 2007, SpaceTime3D released a 3D web browser, generally
`referred to as “SpaceTime.” The web browser incorporated features of the
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 5
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`claimed invention in that it allowed a user to navigate through a plurality of
`webpages by presenting images of the webpages in 3D space. When the user
`double-clicked on a particular image, the corresponding webpage was
`presented in 2D space, allowing the user to interact with the webpage to
`acquire additional information (e.g., another webpage via a link embedded
`therein, etc.). As in the claimed invention, the webpage presented in 2D space
`replaces the plurality of images presented in 3D space, and vice versa.
`14. The SpaceTime web browser received praise by technical journals
`and newspapers, including but not limited to the San Jose Mercury News, The
`Washington Post, The Wall Street Journal, The Economist, Popular Science,
`PC World, Tech Digest, TechNewsWorld, and InternetNews.com. A true and
`correct copy of an article from the San Jose Mercury News, entitled “3-D
`Software Shows Promise,” is attached hereto as Exhibit B; a true and correct
`copy of an article from TechCrunch, entitled “SpaceTime: 3D Browser Eye
`Candy,” is attached hereto as Exhibit C; a true and correct copy of an article
`from TechNewsWorld, discussing the “Product of the Week,” is attached hereto
`as Exhibit D; and another article from TechNewsWorld entitled “SpaceTime
`Browser Adds New Dimension to Search, is attached as Exhibit E.
`15. For instance, reporter Dean Takahashi of the San Jose Mercury
`News exclaimed that SpaceTime3D’s product “is the most advanced 3-D
`navigation system I’ve seen.” See Exhibit B. Greg Kumparak of TechCrunch
`stated that “[i]t’s difficult to describe the user experience … [i]t’s pure eye
`candy” (see Exhibit C), and Rob Enderle of TechNewsWorld stated that “I can
`already image SpaceTime’s tag line: We don’t do searches – we make search
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 6
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`results amazing” (see Exhibit D).
`16.
`It was not just the 3D browsing that was found to be “eye candy,”
`but the incorporation of the claimed invention, where users can switch back
`and forth between 3D and 2D space, where images of webpages are presented
`in 3D space and individual, active webpages are presented in 2D space. For
`example, reporter Jack Germain of TechNewsWorld wrote that “[w]ith
`SpaceTime, I have an unlimited 3-D space. This lets me map out my browsing
`progress in a visual time line, treating each Web site as an object that I can
`manipulate and rearrange within the 3-D environment. SpaceTime also lets me
`alternate between 3-D and 2-D perspectives by double clicking on a 3-D
`display and then clicking the Return button. This process eliminates the hassle
`of reading and closing pop-up windows and clicking on the Back button.” See
`Exhibit E.
`17. Samsung Electronics, the manufacturer of the Galaxy smartphone,
`was also impressed by SpaceTime3D and its technology. For example, on
`February 1, 2008, a Samsung USA employee in charge of purchasing new
`technologies emailed SpaceTime3D: “Hello, my name is Young Choi with
`Samsung Electronics America. It seems like your 3d browser can be at its best
`when it is used in UMPC [mobile devices]. Samsung has a number of UMPC
`products, and I would like to find out if SpaceTime is planning to introduce a
`browser targeted penetrate in UMPC.” See Exhibit F. Attached hereto as
`Exhibits F-J are true and correct copies of emails between SpaceTime3D and
`Samsung.
`18.
`
`I responded to Samsung’s inquiry, explaining that by placing
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 7
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`SpaceTime3D on UMPCs, SpaceTime3D offers an advanced, cutting-edge 3D
`graphical user interface that eliminates the restrictions of the 2D Vista
`graphical user interface on small devices (difficult to input information and
`find things) by allowing end users to (1) easily shuffle through information in
`small spaces and (2) interact with their information in 3D by changing
`perspectives when necessary.
`I was then invited to attend Petitioner’s “2nd Samsung Sourcing
`19.
`Conference and Event” in order to “discuss future collaboration with Samsung
`Digital Media Groups.” See Exhibit G. According to Samsung, “[t]his event
`will provide an opportunity for us to get an understanding of Spacetime
`technology and solutions to discuss its applicability to our Digital Media
`Products. Samsung is looking forward to discussing potential collaboration to
`implement Spacetime’s technology to our Digital Media Groups.” See id.
`20.
`I attended the event on or around May 19-21, 2008, and provided
`Samsung with a PowerPoint presentation to facilitate the parties’ discussion of
`a potential licensing arrangement. That presentation made clear that
`SpaceTime3D’s technology was (at that time) patent-pending.
`21. Following the Conference, Samsung USA notified me that
`Samsung Electronics, in Korea, would also like to review SpaceTime3D’s
`technology, and on June 4, 2008, Samsung USA invited me “to travel to Korea
`to meet with [their] UI engineers in Suwon for further technical discussions
`and demos.” See Exhibit H.
`22. While I was unable to travel to South Korea, I did meet with
`Samsung in San Jose, California, and continued to provide Samsung with
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 8
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`information regarding SpaceTime3D’s technology.
`23. Negotiations between the parties continued into 2010. For
`example, on April 16, 2010, Samsung noted “excit[ement] to hear that
`[SpaceTime3D] just released a very powerful new GUI for tablets and
`netbooks,” inquired about the “basic platform,” expressed a “love to see the
`new development of
`[SpaceTime3D’s]
`technology,” and asked
`that
`SpaceTime3D “[p]lease send [them] the changes and improvements you’ve
`make [sic] until now. Samsung’s new tablet runs Android so I’m wondering if
`your browser is compatible.” See Exhibit I. At this time, Samsung offered to
`“start talking about technical collaboration and potential co-development
`agreement,” first “by getting our engineers interested by showing them what
`SpaceTime is offering and how it makes the user experience better.” See
`Exhibit J.
`24. While discussions with Samsung broke down, years later, after
`litigation involving the ‘654 Patent, Samsung (via a confidential “Settlement
`and License Agreement”) acquired licenses to several SpaceTime3D patents,
`including the ‘654 Patent. LG Electronics Inc. has also inquired about a
`license to the ‘654 Patent. See SpaceTime3D, Inc. v. LG Electronics Inc. et al.,
`No. 2:22-cv-00049 (E.D. Tex.) (Order dated March 14, 2023, staying the
`litigation to “finalize discussions regarding resolution of the case”). The terms
`of that license are currently being negotiated and should be finalized prior to
`the deadline for the Board’s institution decision on the Petition.
`25. The back and forth between SpaceTime3D and Samsung, the
`articles in trade journals and newspapers touting SpaceTime3D’s web browser,
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 9
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`and the Samsung (and soon LG) licenses, are evidence of commercial success,
`recognition, and praise by others in the industry.
`26.
`I have reviewed the Petition and the Exhibits filed therewith
`(APPLE1001-1026),
`including
`the Declaration of Dr. Henry Fuchs
`(APPLE1003). I respectfully disagree with Petitioner’s and Dr. Fuchs’
`conclusions that the claims of the ‘654 Patent are invalid over U.S. Patent
`Publication No. 2005/0091596 (“Anthony”) and U.S. Patent Publication No.
`2005/0088447 (“Hanggie”) (Ground 1a and 2a), or Anthony, Hanggie, and
`U.S. Patent Publication No. 2006/0107229 (“Matthews”) (Ground 1b and 2b).
`27. First, the Petition relies on prior art that is cumulative to prior art
`that was considered by the Examiner during prosecution of the ‘654 Patent.
`During prosecution, SpaceTime3D distinguished its invention from several
`prior art references, including U.S. Patent Publication No. 2002/0033848
`(“Sciammarella”), U.S. Patent No. 6,725,427 (“Freeman”), U.S. Patent No.
`6,768,999 (“Prager”), U.S. Patent Publication No. 2001/0050687 (“Iida”), and
`U.S. Patent Publication No. 2003/0179231 (“Kamiwada”). These prior art
`references are virtually identical (in all material respects) to the prior art cited
`in the Petition.
`28. Petitioner relies on Anthony for its disclosure of displaying items
`in a 3D space based on an “ordering attribute.” In one embodiment the
`selected “ordering attribute” is chronological, “e.g., by using a date of creation
`or date of edit attribute as the ordering attribute.” APPLE1007 at [0042].
`However, as Petitioner points out, it is “data objects” (e.g., digital photograph
`images, etc.) that are arranged based on the ordering attribute, not applications,
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 10
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`as claimed. Pet. at 23-24.
`29. Petitioner argues that “a POSITA would have understood and
`found it obvious that a plurality of application windows could be organized and
`represented within Anthony’s timeline.” Pet. at 24. I disagree. Anthony
`provides that in an alternate embodiment, “file type” can be an ordering
`attribute. See, e.g., APPLE1007 at [0042] and [0049] (“[I]f the ordering
`attribute were file type, the focal group header 602a might contain a label that
`simply says, ‘Excel Spreadsheets’ or some other file type as is known in the
`art.”). Thus, a POSITA would understand that “data objects” (i.e., files) can be
`organized by file type (e.g., alphabetically, etc.)—not a “file type” that is
`organized by “date of creation or date of edit,” as Petitioner suggestions.
`30. Anthony also does not disclose or suggest a 2D space, or
`switching back-and-forth between 2D and 3D space. Petitioner argues that this
`feature is disclosed in Hanggie. Again, I disagree. According to Petitioner,
`Hanggie teaches “control buttons [e.g., a maximize button] for enabling
`selection between 2D and 3D viewing modes.” Pet. at 25. In particular,
`Petitioner argues that clicking on the “maximize” button “would result in
`Anthony’s user interface switching from a 3D timeline view … to a 2D view of
`the maximized window.” I emphatically disagree as there are serious flaws
`with this argument.
`31. Hanggie describes a “3D compositing desktop window manager”
`for presenting content in a 3D environment. APPLE1006 at [0001]. “In order
`to draw the window in 3D, the [3D compositing desktop window manager]
`may define the window anatomy using various components, including a base
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 11
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`content object and one or more child content objects.” Id. at [0057]. As shown
`in Figure 3, “[t]he frame or base content 303 of the window 301 may host child
`content including buttons 305 (e.g., used to restore, maximize, minimize, close
`the window, etc.), and indicative icon 307, scrollbars 309, menu bar 311, and
`window caption text 313.” Id. at [0058].
`32. Hanggie goes on to provide that not only can the window be
`interacted with in 3D space (e.g., by minimizing, maximizing, scrolling, etc.),
`but so too can the contents of the window (e.g., “text by a word processor,
`numeric grid by a spreadsheet, or images by a photo editing application”). Id.
`at [0060]. This is accomplished by “map[ping] mouse hit-test points relative to
`application-defined, content-local, 3D coordinates, and deliver[ing] mouse
`event notifications to the application.” Id.
`33. While Hanggie teaches that the window in Figure 3 can be
`“maximized” or enlarged, that is not the same as switching to a 2D space as
`Petitioner contends. In Hanggie, not only is everything presented in 3D space,
`but the windows themselves are 3D. This can be seen in Figure 3, where the
`window has depth in addition to height and width. Thus, maximizing the
`window would merely enlarge the 3D window—not switch from a 3D space to
`a 2D space. While it is true that both 2D and 3D objects can be presented in
`3D space, a 3D object can only be presented in 3D space as the 3D object (by
`definition) has depth, which requires a space having depth.
`34. However, even if maximizing a window in 3D is essentially
`switching to a 2D space (it is not), the combination argued by Petitioner would
`not have resulted in the claimed invention. In the ‘654 Patent, Claim 1 requires
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 12
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`two things when switching from 3D to 2D space: (1) switching from a
`plurality of inactive images to an individual, active application; and (2)
`replacing the plurality of images (including the image that is being interacted
`with) with an individual, active application. Neither of these are disclosed or
`suggested in the prior art cited by Petitioner.
`In Hanggie, every item presented in 3D space is active (both the
`35.
`window and its contents). Thus, maximizing a window would not result in
`going from an inactive image of an application to an active version of the
`application, as claimed. Instead, it would merely result in enlarging the active
`window that was previously presented in 3D space.
` Second, and
`importantantly, there is no “replacing said plurality of images within said
`three-dimensional space [including the image being interacted with] with one
`of said first, second, and third objects corresponding to said one of said
`plurality of application within a two-dimensional space in response to the first
`interaction,” as claimed.
`36. This is true regardless of whether the claimed combination is
`Anthony in view of Hanggie (Ground 1a) or Hanggie in view of Anthony
`(Ground 2a) (collectively “HAC”), as Petitioner is relying exclusively on
`Hanggie’s “maximizing” button for switching from 3D to 2D. See Petition at
`27 (“[T]he POSITA would have found it obvious to implement the maximize
`control button such that manual user selection of that button would result in
`Anthony’s user interface switching from a 3D timeline view … to a 2D view of
`the maximized window.”) and 33 (“[T]he POSITA would have found it
`obvious to implement Hanggie’s maximize control button such that manual
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 13
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`user selection of that button would result in Hanggie’s user interface switching
`from a 3D timeline view, to a 2D view of the maximize window.”).
`37. The “maximize” button merely enlarges the window to occupy all,
`or almost all, parts of a display. It does not change the functionality of the
`window, and certainly doesn’t replace an inactive image of an application with
`an active version of the application, as claimed. Not only
`is
`the feature
`missing from the cited references, but Hanggie actually teaches away from the
`claimed invention. In Hanggie, everything is presented and active in 3D space.
`There is no switching from 3D to 2D. While Hanggie acknowledges a need to
`reduce power in portable devices, it does so by disabling 3D. APPLE1006 at
`[0092]. In contradistinction, the claim invention enables 3D for the
`presentation of images, which are easier to manipulate, skew, etc., in 3D space.
`As such, the cited prior art can be said to teach away from the claimed
`invention, which is evidence of non-obviousness.
`38.
`In arguing obviousness, Petitioner makes several inaccurate
`characterizations of the prior art. For example, Hanggie provides that “in
`systems with low video memory, or with no 3D acceleration hardware” the 3D
`compositing desktop window manager (“CDWM”) can be disabled (e.g.,
`enabling legacy drawing mode). Petitioner then goes on to conflate this with
`the “maximize” button, arguing that “by selecting the maximize button on a
`window in Anthony’s timeline, the window would be reconfigured in the
`legacy mode and display in 2D to occupy all or most part of the screen in a
`‘maximize’ capacity.” Pet. at 53.
`39. But the maximize button and legacy mode are completely
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 14
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`different features in Hanggie and have nothing to do with one another. As
`shown in Figure 3, the “maximize” button, similar to the “close” button, the
`“file” pulldown, the “edit” pulldown, etc., are merely window properties.
`Clearly, disabling the CDWM “in systems with low video memory, or with non
`3D acceleration hardware” has nothing to do with a “maximize” window
`property.
`In Grounds 1b and 2b, Petition argues that Claims 6, 9, 14, and 18
`40.
`are obvious over Anthony and Hanggie (collectively HAC) in view of U.S.
`Patent Publication No. 2006/0107229 (“Matthews”). As such, Matthews is
`only cited by Petitioner for its disclosure of two features: (1) “displaying a
`fourth image on said display device, said fourth image being an image of a
`desktop having at least one other application” and (2) “where each one of said
`plurality of applications comprises a web browser application.” While I
`disagree with Petitioner’s characterization of Matthews, it doesn’t make up for
`the deficiencies of Anthony and Hanggie, as discussed above. As such, if the
`Board denies Grounds 1a and 2a, at least with respect to Claims 1, 10, and 19,
`then it should also deny Grounds 1b and 2b.
`41.
`In sum, the prior art cited by Petitioner is drastically different
`from the ‘654 Patent. None of the references teach, disclose, or suggest a GUI
`that toggles back-and-forth between 2D and 3D space, where individual, active
`applications are displayed in 2D space and images of applications are presented
`in 3D space. In Anthony and Hanggie, everything takes place in 3D space, i.e.,
`there is no switching back-and-forth. While this works well on personal
`computers (PCs) having unlimited power and a relatively powerful processor,
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 15
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`it does not work well on ultra mobile computers having small displays and
`limited power. On such devices, the claimed invention is advantageous as it is
`easier to manipulate (e.g., scroll through, rotate, etc.) images in 3D space than
`live, active webpages.
`42.
`In addition, the claimed invention does not merely toggle between
`2D and 3D space, but does so in a way to ensure that the images in 3D space
`are presented exactly as the user last left them. In this way, the inactive images
`serve as a visual display of the state or history of each application—displaying
`them in 3D space the way they looked when last edited in 2D space. This
`allows one to visually and intuitively navigate between different applications.
`To this end, the ‘654 Patent provides:
`
`Because the 3D GUI creates the illusion of infinite space in 3D, it can
`create a visual history of the user’s computing session, whereby the user
`can visit past visual computing events (or a snapshot in time) by simply
`navigating to previously recorded states or viewpoints. Accordingly, the
`3D GUI can function as a visual chronological history of the user’s
`computing session … the 3D GUI will navigate the user through a 3D
`space that is the visual history of where the user last left off.
`
`APPLE1001 at 5:6-21.
`43. To this end, Claim 1 provides “generating a plurality of images …
`wherein a first image is an image of at least a portion of a first object generated
`by said first application and having first application-specific data, said second
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 16
`
`

`

`Declaration of E. Eddie Bakhash
`
`IPR2023-00343 (U.S. Patent No. 9,304,654)
`
`
`image is an image of at least a portion of a second object generated by said
`second application and having second application-specific data, and said third
`image is an image of at least a portion of a third object generated by said third
`application and having third application-specific data; and displaying said
`plurality of images in a three-dimensional space on said display device in
`[chronological] order.” If the user clicks on an image in 3D space, an active
`version of the corresponding application is presented in 2D space, “replacing
`said plurality of images within said three-dimensional space,” where the
`application-specific data can be modified. Not only does the prior art cited by
`Petitioner not switch back-and-forth between 2D and 3D, but it does not do so
`in a way that creates “a visual history of the user’s computing session.”
`
` I
`
` declare under penalty of perjury under the laws of the United States
`that the foregoing is true and correct. Executed this 10th date of April 2023, at
`Palo Alto, California.
`
`
`
`
`
`
`
`
`
`
`
`____________________________
`E. Eddie Bakhash
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 17
`
`

`

`
`
`
`
`
`
`
`EXHIBIT A
`EXHIBIT A
`
`
`
`
`
`
`
`
`
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 18
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 18
`
`

`

`US008881048B2
`
`US 8,881,048 B2
`(10) Patent No.:
`a2) United States Patent
`Bakhash
`(45) Date of Patent:
`*Nov. 4, 2014
`
`
`(54) SYSTEM AND METHOD FOR PROVIDING
`THREE-DIMENSIONAL GRAPHICAL USER
`INTERFACE
`
`USPC. ow. 715/781, 848, 764, 766, 767, 782, 788;
`707/706
`See application file for complete search history.
`
`(76)
`
`Inventor: E. Eddie Bakhash, New York, NY (US)
`
`(56)
`
`References Cited
`
`(*) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C, 1540) by 472 days.
`This patent is subject to a terminal dis-
`claimer
`,
`(21) Appl. No.: 12/751,879
`
`U.S. PATENT DOCUMENTS
`5319387 A
`6/1994 Yoshika
`oshikawa
`319,
`5.428.735 A
`6/1995. Kahl etal.
`6,121,969 A
`9/2000 Jain et al.
`6,499,029 Bl
`12/2002 Kurapatietal.
`6,577,330 Bl
`6/2003 Tsuda etal.
`6,725,427 B2
`4/2004 Freeman et al.
`(Continued)
`
`(65)
`
`Prior Publication Data
`US 2011/0029907 Al
`Feb. 3, 2011
`
`“Visualization Using Timelines”, Gerald M. Karam; TRIO Telecom-
`munications Software MethodsProject; PublishedAug. 17, 1994; pp.
`125-137; XP000476876.
`
`Related U.S. Application Data
`(63) Continuation of application No. 11/531,676, filed on
`Sep. 13, 2006, now Pat. No. 7,735,018.
`(60) Provisional application No. 60/717,019,filed on Sep.
`13, 2005.
`
`(Continued)
`
`Primary Examiner — Ting Lee
`(74) Attorney, Agent, or Firm — FitzsimmonsIP Law
`
`ABSTRACT
`(57)
`Methods and systemsare provided for providing an improved
`three-dimensional graphical user interface. In one embodi-
`ment, the method generally comprises: receiving an input
`from an end user, and capturing computing output from at
`least one computer source in response to the received end-
`user input. The computing output can be presented as two or
`more objects within a three-dimensional virtual space dis-
`played to the end user. In one embodiment, the methodfurther
`comprises generating a timelinethat includes an icon for each
`object presented within the virtual space. In another embodi-
`ment, the method further comprises providing a database for
`storing and categorizing data regarding each object presented
`within the virtual space.
`
`18 Claims, 30 Drawing Sheets
`
`(51)
`
`(2011.01)
`(2006.01)
`(2006.01)
`(2013.01)
`
`Int. Cl.
`GO6T 15/20
`GIB 27/34
`GO6F 17/30
`GO6F 3/0481
`(52) U.S. CL
`CPC veeseeeee G06T 15/20 (2013.01); GO6F 17/30873
`(2013.01); GO6F 3/04815 (2013.01); GO6F
`17/30864 (2013.01); G1IB 2734 (2013.01);
`G06F 17/30572 (2013.01)
`USPC. ow. 715/782; 715/764; 715/766; 715/767,
`715/781; 715/788
`
`(58) Field of Classification Search
`CPC wee GO6F 3/04815; GO6F 3/04817; GO6F
`17/30864
`
`
`
`
`
`
`
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 19
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 19
`
`

`

`US 8,881,048 B2
`Page 2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6,768,999 B2
`2001/0050687 Al
`2002/0033848 Al
`2003/0164827 Al
`2003/0179231 Al
`2004/0268451 Al
`2007/0043700 Al*
`
`7/2004
`12/2001
`3/2002
`9/2003
`9/2003
`12/2004
`2/2007
`
`Prageret al.
`lida et al.
`Sciammarellaet al.
`Gottesman et al.
`Kamiwadaetal.
`Robbinetal.
`Dawsonet al. wees 707/3
`
`OTHER PUBLICATIONS
`
`“Execution Patterns in Object-Oriented Visualization”, Wim De
`Pauw et al.; Watson Research Center; Published Apr. 27, 1998; pp.
`219-234; XP002534202.
`“Dissatisfaction Sows Innovation”, Andrew Coulter Enright; The
`Treehouse + the Cave; Published Dec. 29, 2004; 6 Pages;
`XP-002466348.
`
`* cited by examiner
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 20
`
`SPACETIME3D
`EXHIBIT 2001 - PAGE 20
`
`

`

`U.S. Patent
`
`Nov. 4, 2014
`
`Sheet 1 of 30
`
`US 8,881,048 B2
`
`10~.
`
`32
`
`
`
`Load Custom Script andits Icon to Alter
`
`
`Application Logic of 3D Cartesian