United States Patent [19J
`Wharton et al.
`
`I 1111111111111111 11111 111111111111111 IIIII IIIII IIIII IIIII IIIIII Ill lllll llll
`US005831664A
`[11] Patent Number:
`[45] Date of Patent:
`
`5,831,664
`Nov. 3, 1998
`
`[54]
`
`[75]
`
`METHOD AND SYSTEM FOR
`SYNCHRONIZING DATA BETWEEN AT
`LEAST ONE MOBILE INTERFACE DEVICE
`AND AN INTERACTIVE TERMINAL
`
`Inventors: Cathleen S. Wharton, Louisville; Scott
`P. Robertson, Lafayette; Catherine A.
`Ashworth, Boulder; Marita Franzke,
`Denver, all of Colo.
`
`[73]
`
`Assignees: MediaOne Group, Inc., Englewood; U
`S West, Inc., Denver, both of Colo.
`
`[21]
`
`Appl. No.: 572,790
`
`[22]
`
`Filed:
`
`Dec. 15, 1995
`
`[51]
`[52]
`
`[58]
`
`[56]
`
`Int. Cl.6
`................................ H04N 7/10; H04N 7/14
`U.S. Cl. ................................... 348/13; 348/10; 348/8;
`455/6.2; 455/4.2; 345/1; 345/2; 345/156
`Field of Search .................................. 348/10, 11, 12,
`348/13, 7, 6, 8; 455/6.2, 6.3, 4.2, 5.1; 345/1,
`2, 3, 156, 158, 173, 179; H04N 7/10, 7/14
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,285,278
`5,307,055
`5,519,433
`5,526,023
`
`2/1994 Holman ..................................... 348/12
`4/1994 Baskin et al. ............................... 345/1
`5/1996 Lappington et al.
`..................... 348/10
`6/1996 Sugimoto et al.
`...................... 345/173
`
`5,574,964 11/1996 Hamlin ...................................... 348/13
`5,675,362 10/1997 Clough et al. .............................. 345/2
`
`Primary Examiner-John K. Peng
`Assistant Examiner-Linus H. Lo
`Attorney, Agent, or Firm-Brooks & Kushman, P.C.
`
`[57]
`
`ABSTRACT
`
`A method and system for synchronizing display of data
`relating to a predetermined application between an interac(cid:173)
`tive terminal and at least one mobile interface device having
`a display. The at least one mobile interface device receives
`a user input signal and transmits the user input signal for
`synchronization with the interactive terminal. A set-top
`transceiver device is operatively coupled to the interactive
`terminal and receives the user input signal from the at least
`one mobile interface device. The set-top transceiver device
`also transmits a synchronization signal to the at least one
`mobile interface device and the interactive terminal. A
`central processing unit having a database is operatively
`coupled to the set-top transceiver device for receiving and
`processing the user input signal based on the database to
`obtain the synchronization signal for receipt by the set-top
`transceiver device. The synchronization signal includes a
`first display control signal for controlling the display of the
`at least one mobile interface device and a second display
`control signal for controlling the display of the interactive
`terminal.
`
`45 Claims, 10 Drawing Sheets
`
`-,~ ~-
`'· -
`
`CXX)IOOO
`
`16
`
`18
`
`CABLE
`
`SETTOPBOX
`
`SERVER
`
`14
`
`~ -
`'
`--
`
`'-
`
`"10
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 1
`
`

`

`U.S. Patent
`
`Nov. 3, 1998
`
`Sheet 1 of 10
`
`5,831,664
`
`-,~ ~-
`'· -
`
`ooo;ooo
`...
`
`lli!P 1
`
`16
`
`18
`
`CABLE
`
`SETTOPBOX
`
`SERVER
`
`14
`
`~ -
`'
`--
`'
`
`'--10
`
`16
`
`18
`
`---------
`
`SETTOPBOX
`
`SERVER
`
`14
`
`'--10
`
`,•
`
`I
`I
`
`I
`I
`I
`
`PDA
`
`TELEVISION
`
`lli~2
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 2
`
`

`

`"'-'
`0--,
`0--,
`~ ....
`~
`00
`....
`Ul
`
`'"""' 0
`0 ....,
`N
`~ ....
`'JJ. =(cid:173)~
`
`00
`\0
`'"""'
`\0
`~~
`~
`
`z 0
`
`~ = ......
`~ ......
`~
`•
`r:JJ.
`d •
`
`000 • 000
`
`'
`
`,J
`
`lfilfl 3P
`• '
`
`[MessagePad]
`
`( Search ) ( Houses) ( Print ) ( Save )
`
`Press Here for PDA Home Picture
`
`to sleeping quarters and a ma,ter bath.
`The staircase with bright windows leads
`gourmet kitchen and open breakfast area.
`informal traffic pattern extends to the
`the sunny great room with fireplace. The
`From the entry, enjoy the clear view to
`
`1650 A<X'II St. -$135.000
`
`c Description
`)C
`)(
`(
`( Aoorplan & Videos) (
`
`)
`)
`)
`
`,
`
`Maps
`
`Realtor
`
`Location
`
`Fact Sheet
`Select Type of Information Desired
`
`US WEST Homes
`
`Newton
`
`,
`
`lfitp 311
`
`oooiooo
`
`( Search)
`
`I Message Pad !
`
`CJ
`
`2
`
`Flora St.
`c1
`
`CJ3
`
`Baseline
`
`4
`
`Highway36
`
`28th St.
`
`Broadway
`
`tap on the house number with the pen.
`For infonnation on one of these homes
`
`Your search resulted in 8 houses.
`
`US WEST Homes
`
`Newton
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 3
`
`

`

`"'-'
`0--,
`0--,
`~ ....
`~
`00
`....
`Ul
`
`~
`
`'"""' 0
`0 ....,
`~ ....
`'JJ. =(cid:173)~
`
`00
`\0
`'"""'
`\0
`~~
`~
`
`z 0
`
`~ = ......
`~ ......
`~
`•
`r:JJ.
`d •
`
`tli//-3"
`
`oooiooo
`
`(_.Searc...--h....,) (Houses) ( Print ) ( ___ Sa_ve_)
`
`( Stop )
`
`( Play )
`
`I Message Pad I
`
`llitl-sc
`
`j Message Pad j
`
`ooo;ooo
`
`( Search ) C Houses ) C Print ) ( Save )
`
`Berber Carpet, fireplace, 800sf.
`
`Press Here to See Front View of House
`
`MBR
`
`I I
`
`)
`)
`) I I
`
`1650 flora St. -$135.000
`
`Fact Sheet
`Select Type of Information Desired
`
`US WEST Homes
`
`I'
`
`Maps
`Realtor
`
`) (
`
`Description
`
`(
`
`(
`( Aoorplan & Videos) (
`
`) (
`
`Location
`
`II I
`
`I
`
`Fromth1
`
`1650 flora St. -$135.000
`
`)
`
`)
`
`)
`
`Maps
`Realtor
`
`Fact Sheet
`Select Type of Information Desired
`
`US WEST Homes
`
`) (
`(
`( floorplan & Videos) (
`
`) (
`
`Description
`Locatioo
`
`(
`
`Newton
`
`Newton
`
`I II L"· LR~
`
`I ( Bact )
`( Frmt )
`
`-
`
`•
`
`BRl
`
`E
`
`G
`
`BA28 H
`
`K
`
`-,
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 4
`
`

`

`"'-'
`0--,
`0--,
`~ ....
`~
`00
`....
`Ul
`
`'"""' 0
`0 ....,
`~ ....
`'JJ. =(cid:173)~
`
`.i;;..
`
`00
`\0
`'"""'
`\0
`~~
`~
`
`z 0
`
`~ = ......
`~ ......
`~
`•
`r:JJ.
`d •
`
`000 tooo
`
`)
`
`t,ftp 3/
`'
`
`I Message Paci I
`
`\.
`
`( Print ) ( Save )
`
`( Search ) ( Houses )
`
`I
`
`)
`)
`)
`"
`
`CIVIC
`
`-
`
`Ubrary
`
`--
`Courlhowe a Station
`
`-
`
`I
`
`) -'"-
`-281h St -
`
`Baseline
`
`~36
`
`\lahlclae
`Motor
`
`Police
`
`I Broadway
`
`/
`
`1650 Aora St. -$135.000
`
`Fact Sheet
`Select Type of Information Desired
`
`Maps
`
`Realtor
`
`Description )(
`)(
`
`(
`(
`( Aoorplan & Video§) (
`
`Location
`
`US WEST Homes
`
`Newton
`
`,
`
`t,ftp 3e
`
`oootooo
`
`( Print ) ( Save )
`
`( Search ) ( Houses )
`
`! Message Pad !
`
`)
`
`)
`
`)
`
`1650 Aora St. -$135.000
`
`Maps
`
`Realtor
`
`) (
`
`) (
`
`Description
`
`Location
`
`(
`
`(
`
`Fact Sheet
`Select Type of Infonnation Desired
`
`( Aoorplan & Video§) (
`
`US WEST Homes
`
`Newton
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 5
`
`

`

`U.S. Patent
`
`Nov. 3, 1998
`
`Sheet 5 of 10
`
`5,831,664
`
`I Newton I
`US WEST Homes
`Your search resulted in 8 houses.
`For information on one of these bom~
`tap on the house number with the pen.
`
`12
`
`281h St.
`
`4
`
`Highway36
`
`5
`
`I
`oootooo
`
`c1
`
`Flora St.
`
`Cl
`2
`
`I Message Pad I
`
`PDA
`
`Message:
`Device:PDA
`Application: Real-Estate
`Screen: House :792
`Action: #1
`
`16
`
`SETT~ BOX _,/
`
`Message:
`Device:PDA
`Application: Real-Estate
`Screen: House :792
`Action: #1
`-------------
`Device:TV
`
`14
`_/
`
`SERVER
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 6
`
`

`

`U.S. Patent
`
`Nov. 3, 1998
`
`Sheet 6 of 10
`
`5,831,664
`
`18
`
`SERVER
`
`Mesaage:
`TV: Show House
`Box: Store House Video
`PDA: Download house data
`Show Description Screen
`
`16
`
`TV
`SETTOPBOX
`
`Message:
`PDA: Download house data
`Show Description Sceen
`
`• PDA
`
`14
`
`)
`)
`)
`
`12
`
`/"
`
`I Newton I
`US WEST Homes
`Select Type of Information Desired
`( Floorplan & Videos) (
`Fact Sheet
`)(
`(
`Loaltion
`)(
`( Description
`16SO Flora St. - $135.000
`
`Realtor
`
`Maps
`
`From the entry, enjoy the clear view to
`the sunny great room with fireplace. The
`informal traffic pattern extends to the
`gourmet kitchen and open breakfast area.
`The staircase with bright windows leads
`to sleeping quarters and a master bath.
`
`Press Here for PDA House Picture
`
`( Search ) ( Houses)
`
`( Print ) ( Save )
`
`000 • 000
`• '
`
`j Message Pad I
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 7
`
`

`

`U.S. Patent
`
`Nov. 3, 1998
`
`Sheet 7 of 10
`
`5,831,664
`
`I Newton I
`
`I'
`
`12
`) l-/
`)
`)
`
`Realtor
`Maps
`
`US WEST Homes
`Select Type of Information Desired
`( Floorplan & Videos) (
`Fact Sheet
`)(
`(
`Location
`)(
`(
`Description
`1650 Flora St. - $135.000
`
`- -K
`I
`.
`..
`E I
`BA2 I BAI I H
`-
`• ~
`-
`MBR ~
`I Berber Carpet, fireplace, 800sf.
`I
`( Play )
`( Stop )
`( Search ) ( Houses) ( Print ) ( Save )
`
`DR
`
`LR
`
`G
`
`I
`
`I
`
`BRl
`
`000 • 000
`• t
`I Message Pad I
`
`-- PDA
`
`Message:
`Device:PDA
`Application: Real-Estate
`Screen: House :795
`Action: K
`
`16
`
`s
`ETTiiP BOX _./
`
`Message:
`Device:PDA
`Application: Real-Estate
`Screen: House :795
`Action: K
`
`14
`_/
`
`Device:TV
`
`18
`
`SERVER
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 8
`
`

`

`U.S. Patent
`
`Nov. 3, 1998
`
`Sheet 8 of 10
`
`5,831,664
`
`14
`
`12
`
`! Newton I
`US WEST Homes
`Select Type of Information Desired
`( Floorplan & Videos) (
`Fact Sheet
`) ~
`c Description
`(
`Location
`) _
`
`Realtor
`Maps
`
`1650 Flora St. • $135.000
`
`)
`~
`.
`
`18
`
`SEAVER
`
`Message:
`1V: Show Kitchen
`Box: Store Kitchen Video
`PDA: Show Kitchen text
`
`16
`
`1V
`SETTOPBOX
`
`Message:
`PDA: Show Kitchen Text
`
`t
`PDA
`
`DR
`
`LR
`
`I
`
`-
`-,
`K
`BA2 I BAl I H
`I
`
`0
`
`E .I
`
`BRl
`
`- M
`
`(§I)
`BR ~
`
`Gas Range, OW, Refridgerator
`
`( Stop )
`( Play )
`(,-~---) ( Houses) ( Print ) (--Sa-ve ..... )
`
`oooiooo
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 9
`
`

`

`U.S. Patent
`
`Nov. 3, 1998
`
`Sheet 9 of 10
`
`5,831,664
`
`House
`Picture
`
`28
`
`Floorplan
`(
`34
`
`LR
`
`Located near
`beautiful Lake
`Winabegoin
`the stunning
`community ...
`
`Location
`
`24
`
`26
`
`This house
`has three
`bedrooms
`
`..
`
`House
`
`~t=J
`20 House
`Browsing
`
`22/,--lnspectio_n_\ _____ _
`
`32
`
`Tod Miller
`EXPENSO
`REALITY
`
`1311 9th St.
`Boulder, CO
`
`Realtor
`
`~Maps ' . .~
`
`Schools ~ Civic
`
`Restaurants
`
`Description
`
`* 1600 sf
`*3BR
`*2BA
`*LR
`*DR
`
`Fact Sheet
`
`30
`
`© Realtor
`
`Picture
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 10
`
`

`

`U.S. Patent
`
`Nov. 3, 1998
`
`Sheet 10 of 10
`
`5,831,664
`
`14
`
`\
`
`I
`I
`
`/
`
`"
`~ ~
`~~~
`~~~
`I Newton I
`US WEST Homes
`Y 0111 search resulted in 8 houses.
`For information on one of tbese homes
`tap on the house number with the pen.
`
`/
`
`"' ....
`
`CJ7 /
`
`CJB
`
`/ Broadway
`
`211th St.
`
`Hlghway36
`
`~4
`
`14
`
`\
`
`I
`I
`
`/
`
`"-
`
`I~
`
`..... r
`
`I Newton I
`US WEST Homes
`Select Type of Information Desired
`I F1oorplan &: Videos
`J
`Loc:atiOD
`I
`)~
`Realtor
`L--
`)
`Description
`Maps
`1650 Flora St. - $135.000
`
`"I
`
`3
`b
`LI
`
`From the em,y, enjoy the clear view to
`the sunny great room with fueplace. Tbe
`Informal 1nffic pattern eztend:s to the
`gourmet kitcben and open breakfast affll.
`The 1taln:aae with bright windows lads
`to sleeping quarten and a master bath.
`
`l,.A"
`~V
`
`r-- ---
`oooiooo
`
`~ Press Here for PDA House Pic1ure
`
`(Search)~ ~ O!!D
`
`\.
`
`I Message Pad I
`
`I
`oooiooo
`
`-Ina
`
`CJ 3 V
`
`/
`
`CJ 1
`Flora St
`
`C,
`2
`
`5
`
`a
`3
`\.
`r---
`
`( Searcb)
`
`\.
`
`I Mesaage Pad I
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 11
`
`

`

`5,831,664
`
`1
`METHOD AND SYSTEM FOR
`SYNCHRONIZING DATA BETWEEN AT
`LEAST ONE MOBILE INTERFACE DEVICE
`AND AN INTERACTIVE TERMINAL
`
`TECHNICAL FIELD
`
`This invention relates to a method and system for syn(cid:173)
`chronizing the display of data between at least one mobile
`interface device having a display and an interactive terminal
`having a display.
`
`BACKGROUND ART
`
`Remote control devices which control television receivers
`have been available to consumers for many years. These
`devices typically operate by generating an infra-red remote
`control beam modulated with commands for either raising or
`lowering the volume of a television receiver, changing
`channels, power up, power down, and for operating other
`more common controls on a television receiver. These
`remote control devices typically have fixed buttons that
`influence selection of items identified on the television
`screen and contain no information about the state of the
`television offering. Furthermore, these remote control
`devices only send information in one direction to the tele(cid:173)
`vision receiver.
`The functionality of remote control devices has changed
`with the introduction of interactive terminals, such as tele(cid:173)
`vision receivers, and computer terminals. An interactive
`terminal allows a user to change a control graphic display
`based on an input signal from the user. Remote control
`devices currently available for interactive television (ITV)
`are similar to traditional television remotes in three ways:
`button presses for input, unidirectional communication, and
`limited information display. The entire television presenta(cid:173)
`tion is controlled through the pressing of buttons. In a typical
`ITV service, a menu of options is displayed on the television
`from which one option is selected by pressing the number
`key or arrow button on the remote control device.
`Furthermore, a typical remote control device sends an infra(cid:173)
`red signal to the television or set-top box, but the remote
`control device does not receive input from the same devices.
`Consequently, there is no need for an information display on
`the remote. Remote control devices also display a meager
`amount of information in the form of textually labeled
`buttons.
`Because of the characteristics of existing remote control
`devices, developers of interactive television applications
`must make the applications consistent with existing remote
`control devices. The general usability of interactive televi(cid:173)
`sion offerings is also limited, for example, by requiring
`labeling of screen objects with numbers that will correspond
`to the channel keys on remote control devices.
`The need exists, therefore, for a remote control interface
`that can be dynamically reconfigured to correspond with an
`application. A need also exists for an interactive system that
`allows a wide range of interfaces to be presented to the user.
`
`SUMMARY OF THE INVENTION
`
`It is an object of the present invention to provide a method
`and system for synchronizing display of data relating to a
`pre-determined application between an interactive terminal
`and a mobile interface device having a display.
`It is another object of the present invention to provide a 65
`mobile interface device having a reconfigurable display to
`correspond with data displayed on the interactive terminal.
`
`2
`It is a further object of the present invention to provide a
`mobile interface device which augments the data displayed
`on the interactive terminal.
`Still further, it is an object of the present invention to
`5 provide a mobile interface that accepts both continuous and
`discrete interactions and movements for synchronization
`with an interactive terminal.
`It is yet another object of the present invention to provide
`a method and system that allows multiple mobile interface
`10 devices to interact with an interactive terminal.
`In carrying out the above objects and other objects,
`features and advantages of the present invention, a method
`is provided for synchronizing display of data relating to a
`predetermined application between the interactive terminal
`15 and at least one mobile interface device having a display.
`The method includes the step of providing each of the at
`least one mobile interface device with a user input receiving
`means for receiving a user input signal and for transmitting
`the user input signal for synchronization with the interactive
`20 terminal. The method also includes the step of providing a
`set-top transceiver device operatively coupled to the inter(cid:173)
`active terminal for receiving the user input signal from the
`at least one mobile interface device and for transmitting a
`synchronization signal to the at least one mobile interface
`25 device and the interactive terminal. Still further, the method
`includes the step of providing a central processing unit
`having a data base and operatively coupled to the set-top
`transceiver device for receiving and processing the user
`input signal based on the database to obtain the synchroni-
`30 zation signal for receipt by the set-top transceiver device.
`The synchronization signal includes a first display control
`signal and a second display control signal, the first display
`control signal for controlling the display of the at least one
`mobile interface device and the second display control
`35 signal for controlling the display of the interactive terminal.
`In further carrying out the above objects and other objects,
`features and advantages, of the present invention, a system
`is also provided for carrying out the steps of the above(cid:173)
`described method. The system includes at least one mobile
`40 interface device having a display and a user input receiving
`means for receiving a user input signal and for transmitting
`the user input signal for synchronization with the interactive
`terminal. The system also includes a set-top transceiver
`device operatively coupled to the interactive terminal for
`45 receiving the user input signal from the at least one mobile
`interface device, and for transmitting a synchronization
`signal to the at least one mobile interface device and the
`interactive terminal. The system further includes a central
`processing unit having a database and operatively coupled to
`50 the set-top transceiver device for receiving and processing
`the user input signal based on the database to obtain the
`synchronization signal for receipt by the set-top transceiver
`device.
`The above objects and other objects, features and advan-
`55 tages of the present invention are readily apparent from the
`following detailed description of the best mode for carrying
`out the invention when taken in connection with the accom(cid:173)
`panying drawings.
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a schematic block diagram of the system of the
`present invention;
`FIG. 2 is a block diagram of a second embodiment of the
`system of the present invention;
`FIGS. 3a-3/ are schematic diagrams illustrating the vari(cid:173)
`ous display screens available for the mobile interface device
`in a real estate application;
`
`60
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 12
`
`

`

`5,831,664
`
`3
`FIG. 4 is a schematic diagram illustrating the sequence of
`steps performed by the system of the present invention when
`the mobile interface device receives a user input signal;
`FIG. 5 is a schematic diagram illustrating the sequence
`and steps performed by the system of the present invention
`when the central processing unit receives the data signal
`representing the user input signal;
`FIG. 6 is a schematic diagram illustrating the sequence of
`steps performed by the system of the present invention when
`the mobile interface device receives a second user input 10
`signal;
`FIG. 7 is a schematic diagram illustrating the sequence of
`steps performed by the system of the present invention when
`the central processing unit receives a data signal represent-
`ing the second user input signal;
`FIG. 8 is a schematic diagram illustrating the organization
`of the various screen displays available for the mobile
`interface device in a real estate application; and
`FIG. 9 is a schematic diagram illustrating synchronization 20
`of the mobile interface device display and an interactive
`terminal display.
`
`15
`
`4
`set-top transceiver device 16 then receives signals from the
`server 18 and broadcasts a first display signal to the mobile
`interface device 12 and transmits a second display signal to
`the interactive terminal 14. Thus, the set-top transceiver
`5 device 16 is a two-way communication processor.
`The server 18 receives data from the set-top transceiver
`device 16. The data includes identification of all the devices
`12, 14 that are interacting, what application is being used,
`and what action has taken place. The server 18 sends data to
`the set-top transceiver device 16, including target devices
`and information specific to each target device. The set-top
`transceiver device 16 then sends video information directly
`to the interactive terminal 14 and broadcasts information to
`the mobile interface device 12.
`The system of the present invention can be expanded to
`support multiple devices, as shown in FIG. 2. When multiple
`mobile interface devices 12 are involved, an action on one
`mobile interface device 12 is broadcast to all others. Each
`mobile interface device 12 takes its appropriate action and
`broadcasts the result. Thus, the mobile interface devices 12
`remain coordinated, and each has information on what other
`mobile interface devices 12 are in the application environ(cid:173)
`ment. In the case of a mobile interface device 12, such as a
`PDA, the PDAcan be used by itself as a stand-alone unit. In
`25 this case, a broadcast is not replied to, so the application
`knows that it is being used on a stand-alone basis and can
`operate accordingly.
`
`BEST MODES FOR CARRYING OUT IBE
`INVENTION
`Turning now to FIG. 1, there is shown a schematic block
`diagram of the system of the present invention, denoted
`generally by reference numeral 10. The system 10 includes
`at least one mobile interface device 12, such as an Apple
`Newton™ personal digital assistant (PDA) for transmitting 30
`user input signals. The mobile interface device 12 is a
`hand-held, mobile computing device that has built-in infra(cid:173)
`red and serial communications capabilities and is designed
`to interact with other devices. The system 10 further
`includes an interactive terminal 14, such as a television 35
`receiver. The interactive terminal 14 may also be a computer
`terminal or any other similar device. The mobile interface
`device 12 communicates with the interactive terminal 14
`through a set-top transceiver device 16. Communication
`between the mobile interface device 12 and the set-top
`transceiver device 16 may be either wireless infra-red tech(cid:173)
`nology or wireline communications. The system 10 further
`includes a central processing unit 18, such as a data server,
`for communicating with the set-top transceiver device 16.
`Communication between the set-top transceiver device 16
`and the server 18 may be accomplished through either a
`wireless or wireline bi-directional communication.
`Bi-directional communication enables complex user-system
`interactions. For example, a user can perform an action on
`the mobile interface device 12 which is received by the
`set-top transceiver device 16, which then downloads data
`from the server 18 and which can then be displayed on the
`interactive terminal 14 and/or transmitted back to the mobile
`interface device 12.
`A user interacts with the mobile interface device 12 in one 55
`of many ways. The user may interact with the mobile
`interface device 12 using a key pad, a touch screen, voice
`input, or in any other available manner. In the touch screen
`embodiment, the user interacts with the mobile interface
`device 12 by touching control buttons or icons, or by 60
`continuously tracing a path. Interaction with the touch
`screen can be accomplished using either a stylus or the
`user's fingers.
`The set-top transceiver device 16 includes a processor and
`data storage capability. The set-top transceiver device 16 65
`processes signals received by the mobile interface device 12
`and transmits a corresponding signal to the server 18. The
`
`REAL ESTATE APPLICATION
`
`The system of the present invention will be described in
`conjunction with a real estate application using a PDA and
`a television (TV). It should be understood, however, that the
`example described is for illustrative purposes only. The
`invention has many applications, including, but not limited
`to real estate.
`The domain of a real estate application is information(cid:173)
`rich, requires considerable data base browsing, involves
`comparison of multiple selection criteria, and benefits
`40 greatly from multi-media presentation format. It is a domain
`which traditionally requires the intervention of a human
`agent to mediate search and help in decision making. It also
`requires home buyers to be present in the new location and
`to spend considerable time traveling to homes and neigh-
`45 borhoods for a first-time look. A service that reduces any of
`these requirements is highly valuable.
`Turning now to FIGS. 3a-3/, there is shown the various
`possible screen displays available to the PDA 12 in the real
`estate application. FIG. 3a illustrates the House Selection
`50 Screen which includes the results of an initial search of a set
`of candidate homes. The homes are presented in the form of
`a map on the PDA 12 and a mosaic of pictures are displayed
`on the TV 14. The map includes the major roads for the area
`and the locations of the homes. This combination display is
`useful for the "catalog shopping" phase when a home buyer
`wants to see what the homes look like. Without the TV 14,
`the PDA 12 is still useful as a map which can be used to
`drive to different house locations. From the House Selection
`Screen, the user may return to an earlier search phase by
`pressing the "SEARCH" button at the bottom of the screen.
`If the user wishes to see details about a home, he or she taps
`on the appropriate home number on the map of the PDA 12.
`FIG. 3b illustrates the House Information Screen which
`contains a text description of the selected home from the
`House Selection Screen. If a TV 14 is used, the TV 14
`displays a large view of the home. A low-resolution view of
`the home can also be presented on the House Information
`
`Petitioner Microsoft Corporation - Ex. 1012, p. 13
`
`

`

`5,831,664
`
`10
`
`5
`Screen. The image on the PDA 12, which appears as an
`overlay of the text, is useful if a TV 14 is not available. The
`"HOUSES" button shown in FIG. 3b will return the user to
`the House Selection Screen. The "SEARCH" button remains
`available to initiate a new search. "PRINT" and "SAVE" 5
`buttons are available at the bottom of the screen to create
`more permanent records of the information about the home
`being inspected. The House Information Screen contains six
`home information buttons at the top and an information box
`in the center. The six information buttons provide access to
`various categories of information about the selected home.
`This information appears in the information box and may
`have accompanying information on the screen of the TV 14.
`When a user presses the "LOCATION" button on the
`House Information Screen, plain text describing the location
`is displayed in the information window. A picture of the
`street is shown on the TV 14, if available A "PLAY" button
`on the PDA 12 allows a user to see a "walking tour" of the
`street.
`When the "DESCRIPTION" button is selected, plain text
`describing the home is displayed in the information window.
`This description corresponds to the "ADVERTISING"
`description that typically appears in printed catalogs or
`newspapers. A front view of the home is shown on the TV
`14.
`When a user selects the "FACT SHEET" button, plain text
`with technical details about the home is displayed in the
`information window. This text contains specifics like the
`number of bedrooms, square footage, date of construction,
`type of heating, etc., that would appear on a typical detailed
`spec sheet from a commercial listing service. A picture of the
`home may also be shown on the TV 14.
`When a user selects the "REALTOR" button, plain text
`giving background information about the realtor is displayed
`in the information window. The realtor's picture appears on
`the television. A "PLAY" button on the PDA 12 results in the
`display of a video of the realtor describing him or herself and
`the home. A button on the PDA 12 can be selected to show
`a low resolution picture of the realtor on the PDA 12 in case
`a TV 14 is not available.
`Upon selecting the "FLOOR PLANS AND VIDEOS"
`button, a floor plan of the currently selected home appears
`in the information window, as shown in FIG. 3d. When a
`room is selected, as with a stylus, information about the
`room (e.g., square footage, flooring type, etc.) appears in a
`text window on the PDA 12. In addition, if the user is using
`a TV 14, selecting a room causes an image of the room to
`be displayed on the TV 14. "PLAY" and "STOP" video
`control buttons appear on the screen of the PDA 12 when a 50
`floor plan is being studied. When the "PLAY" button is
`selected, a video showing the entire room is presented on the
`interactive terminal 14. Thus, the user can use the PDA 12
`to control a video walk-through of the home. "BACK" and
`"FRONT" buttons are also shown on the PDA 12 and 55
`pressing one of these yields either a front or back view of the
`home on the TV 14.
`If a TV 14 is not available, the floor plan presentation on
`the PDA 12 is still useful as a stand alone application. Users
`can navigate through a home and learn details about each
`room by taking the PDA 12 with them on a real walk(cid:173)
`through, for example.
`When a user selects the "MAPS" button on the House
`Information Screen, several map icons are displayed in the
`information window of the PDA 12, as shown in FIG. 3e.
`The virtual buttons are labeled with different types of maps,
`for example "RTD" (the local bus system), "SCHOOLS",
`
`6
`"CIVIC", etc. The TV 14 displays a detailed map of the
`region. When a user selects one of the map types on the PDA
`12, the PDA 12 information screen shows a very general
`schematic of the appropriate map with important locations
`identified. The more detailed map on the TV 14 displays a
`highlight of the selected map component. For example,
`when "CIVIC" is selected, a map of main roads is shown on
`the PDA 12, as shown in FIG. 3/, with locations like the
`library and police station indicated. These landmarks are
`also highlighted on the detailed map displayed on the TV 14.
`When an item like a landmark or bus stop is selected on the
`PDA 12, an address appears on the PDA 12 and a narration,
`if available, appears on the TV 14. To see another map, the
`user need only to re-select the "MAPS" button. Without a
`TV 14, the maps can be used to navigate to different
`15 locations or to simply learn about the neighborhood. With a
`TV 14, the user gains a visual feel for the neighborhood and
`can gather considerable information about each of the estab(cid:173)
`lishments.
`Consider the example in which the PDA 12 displays a
`20 plurality of home locations based on a user search criteria,
`as shown in FIG. 4. When the user selects the tome labeled
`"l" on the PDA 12, the PDA 12 sends a signal to the set-top
`transceiver device 16. The signal contains information about
`the device, application, data, and user action. In this
`25 example, the device is a PDA, the application is real estate,
`the data corresponds to the current screen display, which is
`House:792, and the user action is the selection of number
`"l". The set-top transceiver device 16 adds information
`about the other device (i.e., the TV 14) and sends this signal
`30 to the server 18. If the PDA had the data on each home, the
`PDA 12 could run its internal application to update the
`screen, otherwise, the PDA 12 will wait for a download from
`the set-top transceiver device 16.
`Upon receiving the signal from the set-top transceiver
`35 device 16, the server 18 processes the signal to obtain a
`synchronization signal. The synchronization signal includes
`a first display control signal for controlling the display of the
`PDA 12 and a second display control signal for controlling
`the display of the TV 14. As shown in FIG. 5, the server
`40 sends instructions and new data to the set-top transceiver
`device 16 for receipt by the PDA 12 and the TV 14. A
`television, a new picture is displayed on the TV 14. Video
`data about the home may be downloaded into the set-top
`transceiver device 16 if there is enough memory. In the case
`45 of the PDA 12, data about the specific home is downloaded
`and the House Information Screen is displayed. Thus, when
`more information about the home is needed, it is locally
`available on the PDA 12.
`FIGS. 6 and 7 illustrate the information processed when
`a user selects a kitchen, i.e. the "K" button, from the floor
`plan screen displayed on the PDA 12. Upon selecting the
`kitchen area, the interactive terminal 14 is instructed to
`display the kitchen area of the home and the PDA 12 is
`instructed to display text relating to the kitchen area.
`Turning now to FIG. 8, there is shown the organization of
`the screens displayed on the PDA 12. After browsing
`through a plurality of homes, and selecting a specific home
`for further information, the user is transferred to the House
`Information Screen, as indicated at 20,22, respectively. At
`60 the House Selection Screen, the user may select the "LOCA(cid:173)
`TION" button to obtain information regarding the location
`of the home, as shown at 24. Alte