(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2004/0093376 A1
`(43) Pub. Date:
`May 13, 2004
`De BOOr et al.
`
`US 20040093376A1
`
`(54) WIRELESS COMMUNICATION DEVICE
`WITH MARKUP LANGUAGE BASED
`MAN-MACHINE INTERFACE
`(76) Inventors: Adam De Boor, Alameda, CA (US);
`Michael D. Eggers, Oakland, CA (US)
`Correspondence Address:
`Peter J. Yim
`Morrison & Foerster LLP
`425 Market St.
`San Francisco, CA 94105-2482 (US)
`(21) Appl. No.:
`10/688,514
`(22) Filed:
`Oct. 17, 2003
`Related U.S. Application Data
`(63) Continuation of application No. 10/215,760, filed on
`Aug. 9, 2002, now Pat. No. 6,675,204, which is a
`continuation of application No. 09/907,091, filed on
`Jul. 16, 2001, now Pat. No. 6,470,381, which is a
`continuation of application No. 09/604.833, filed on
`Jun. 27, 2000, now Pat. No. 6,317,781, which is a
`continuation of application No. 09/057,394, filed on
`Apr. 8, 1998, now Pat. No. 6,173,316.
`
`Publication Classification
`
`(51) Int. Cl." ............................ G06F 15/16; G06F 17/60
`(52) U.S. Cl. ............................. 709/203; 705/14; 709/217
`
`(57)
`
`ABSTRACT
`
`A wireleSS communications device with a markup language
`based man-machine interface provides a user interface for
`telecommunications functionality, including dialing tele
`phone numbers, answering telephone calls, creating mes
`Sages, Sending messages, receiving messages, establishing
`configuration Settings defined in markup language Such as
`HTML, and accessed through a browser program executed
`by the wireleSS communication device. This feature enables
`direct access to Internet and World Wide Web content, Such
`as Web pages, to be directly integrated with telecommuni
`cation functions of the device, and allows Web content to be
`Seamlessly integrated with other data types, Since all data
`presented to the user via the user interface is presented via
`markup language-based pages. The browser processes an
`extended form of HTML that provides new tags and
`attributes that enhance the navigational, logical, and display
`capabilities of conventional HTML, and particularly adapt
`HTML to be displayed and used on wireless communication
`devices with Small Screen displayS.
`
`ShelGetURL(URL)
`
`Extract rotocontaine
`For UR
`
`5OO
`
`5O2
`
`Protocol Present?
`
`Yes
`
`503
`Priority >= top
`URt priority?
`
`cal GetURL of
`Protocolander to
`Ottain
`ContentStreat
`
`7 508
`
`SO9
`
`0.
`
`Build futi URL from
`New UR and
`Referring URL
`
`mic
`
`inser UR in stack
`above as R.
`whose priority is <=
`URL priority
`
`/ 511
`
`Graunsuccess D
`
`
`
`7 510
`
`Use type Suing
`for Streait to
`locate Content
`adet
`
`51
`Found Conte?
`andle?
`
`No
`
`5
`
`close Steam and
`Return
`to
`
`
`
`- 526
`Reopen Previous
`Urland return to
`
`516
`Reset SoftKeys, ite,
`Content Window
`Size, and Satus
`Message to their
`default State
`
`
`
`
`
`
`
`Pass Streal to
`Contenhandles
`Open Function
`
`Open Cal
`Successful?
`
`
`
`Yes
`
`52
`
`Page 1 of 69
`
`GOOGLE EXHIBIT 1020
`
`

`

`Patent Application Publication May 13, 2004 Sheet 1 of 32
`
`US 2004/0093376A1
`
`102
`MM
`104
`USER INTERFACE DEFINITION FLES
`107
`BROWSER
`
`108
`URLHSTORY
`STACK
`
`
`
`
`
`110
`CALL BACK OUEUE
`
`
`
`
`
`
`
`112
`PROTOCOL HANDLERS
`
`114
`CONTENT HANDLERS
`
`
`
`116
`PORTABLE COMPONENTS
`
`-
`
`120
`TELEPHONE CONTROL MODULE
`
`
`
`
`
`118
`
`
`
`PORTABILITY
`LAYER
`
`122
`REAL TIME OPERATING SYSTEM
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`124
`PROCESSOR
`
`
`
`
`
`126
`MEMORY
`
`100
`WIRELESS COMMUNICATION DEVICE
`128
`
`130
`
`132
`FUNCTION
`
`134
`NUMBER KEYS
`
`
`
`FIG. 1
`
`
`
`
`
`
`
`136
`SCREEN
`DISPLAY
`
`Page 2 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 2 of 32
`
`US 2004/0093376 A1
`
`
`
`snº 0-3
`
`
`
`\u0?oolaspuß _º a Ifiu.e?u)
`
`OZZ
`
`
`
`faouue ||0.10S-,9 || Z.
`
`Page 3 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 3 of 32
`
`US 2004/0093376 A1
`
`
`
`
`
`
`
`
`
`1 O4
`USER INTERFACE DEFINITION FILES
`
`.r
`
`
`
`107
`
`SHELL
`
`108
`.
`URL. HSTORY
`STACK
`
`1 10
`CALL BACK OUEUE
`
`f 12
`PROTOCOL HANDLERS
`
`1 12a
`BUILT IN
`
`f 12d
`FILE
`
`f 12c
`EXTRA
`
`1 12e
`HTP
`
`1 12f
`MESSAGE
`
`112g
`TELEPHONE
`
`114
`CONTENT HANDLERS
`
`1 14a
`ADVERT
`
`114C
`HTMLp
`
`(E
`MANAGER
`
`f 14d
`MAN
`
`
`
`
`
`-
`
`116
`PORTABLE COMPONENTS
`
`224
`GRAPHCS
`
`226
`U GAOGETS
`
`228
`TEXT ENTRY
`
`23O
`EXT WRAP
`
`
`
`
`
`
`
`
`
`232
`DATA STORE
`
`
`
`234
`FE SYSTEM
`
`236
`LOCALZATION
`
`
`
`118
`PORTABUTY LAYER
`142
`SSAS
`
`144
`PLATFORM
`
`
`
`
`
`
`
`112b
`CONFIG
`PROTOCOL
`HANDLER
`
`146
`TMER
`
`148
`OSPLAY
`
`
`
`150
`FLASH DRVER
`
`FIG. 3
`
`Page 4 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 4 of 32
`
`US 2004/0093376 A1
`
`4.
`
`
`
`
`
`
`
`-
`
`
`
`402
`
`URL
`
`108
`
`40
`
`
`
`EXTRADATA
`
`408
`410
`EXT
`PRIVILEGE 414 PRIORITY 416
`
`Priority is
`
`402
`
`
`
`
`
`l
`CONTENTHANDLER 406
`Extra Data
`40s
`
`NEXT
`
`410
`
`STATE
`
`18
`
`412
`
`OPEN
`close
`
`
`
`
`
`
`
`
`
`
`
`FIG. 4
`
`ACTIVATE
`PROCESSKEY
`
`
`
`Page 5 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 5 of 32
`
`US 2004/0093376 A1
`
`SheltGetURL(URL )
`
`Extract Protocol Name
`Front URL
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Protocol Present?
`
`Priority >= top
`URL priority?
`
`Yes
`
`Cat GetURL of
`Protocol handler to
`Obtain
`ContentStream
`
`7 508
`
`509
`
`Build Full URL. From
`New URL and
`Referring URL
`
`insert URL in stack
`above last URL
`whose priority is <=
`URL priority
`
`/ 511
`
`504
`
`Return Success
`
`/ 510
`
`No
`
`have Stream?
`
`506
`
`Yes
`
`Use Type String
`for Stream to
`Locate Content
`Hander
`
`/ 512
`
`
`
`514
`Found Conten
`Handler?
`
`Yes
`
`
`
`
`
`
`
`
`
`N O
`Close Stream and
`Return Error
`
`52O
`
`FIG. 5
`
`7 526
`Reopen Previous
`URL and Return Error
`
`eset Softkeys,
`Content Wind
`Size, and Status
`Message to their
`Default State
`
`Pass Stream to
`Content Handler's
`Open Function
`
`
`
`7 522
`
`No
`
`
`
`Open Calt
`Successful?
`
`Yes
`
`524
`
`Page 6 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 6 of 32
`
`US 2004/0093376 A1
`
`
`
`
`
`HTML paCTIVATEstring
`
`
`
`
`
`EMBEDOED
`OSC
`
`PASS STRINGO
`EMBEDOEO OBEC
`
`604
`
`STRING
`PROCESSEO
`
`no
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`61.2
`
`614
`
`616
`
`626
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`STRING
`preSE FORMID
`
`634
`STRING =
`SUBMIT:FORMiO,
`ABE
`
`CAL SHELLO POP TOP
`URL OF HISORY STACK
`
`CAL SELOOSPAY
`MOST RECEN PAGE
`BEFORE COP AG
`
`CURREN
`OECS
`YPERLINK
`
`GE URL ASSOCATED
`WT HYPERLINK, PASS TO
`SEACTIVAE
`
`CAL SHELLO CLEAR Uf
`HSTORY SACK OSPAY
`MAN PAGE
`
`
`
`RESET FORM=FORMED TO
`ORIGINAL STATE
`
`
`
`SUBMT FORMsFORM D TO
`SE
`
`FIG. 6A
`
`Page 7 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 7 of 32
`
`US 2004/0093376A1
`
`
`
`
`
`
`
`
`
`638
`
`
`
`640
`
`
`
`w
`STRING-"SELECT"
`
`ACTIVATE SELECTED USER
`NTERFACE GAOGET
`
`
`
`STRINGS:
`"NONE"
`
`644
`STRING
`ANY OTHER VALUE
`(I.E. A URL)
`
`
`
`646
`
`PASS STRING (URL) TO
`SHELO FETCH
`
`
`
`RETURN
`
`FIG. 6B
`
`Page 8 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 8 of 32
`
`US 2004/0093376A1
`
`
`
`
`
`•< Tun? º quæ o.ax// : »Tp j =NOILOVI „ BIIe O que o ºd u ="IGGWI
`
`>
`
`Page 9 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 9 of 32
`
`US 2004/0093376 A1
`
`
`
`
`
`
`
`kHTML
`CHEAD
`CTOP
`<TITLE:Phone BookC/TITLE
`CKEY KEY =l Label = New Action = "new">
`CKEY KEYs clear actions deletes
`<HELP>Press any number key to search for
`an entry </HELP>
`CHELP) Press Send to call someone</HELP
`</HEAD>
`CBODY >
`<OBJECT
`CODE="phone : list?editkey=2&showstatus">
`</BODY >
`</HTML>
`
`
`
`
`
`(262)3335214
`Daily horoscope
`D
`
`(262)333524
`Daily Horoscope
`D
`
`fost
`loog Olso
`to search for an entry
`(262)3335214
`Daily Horoscope
`Georse Smithers
`
`sco Oo
`ost
`Press Send to
`(262)333524
`Daily Horoscope
`George Smithers
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 10 of 69
`
`

`

`Patent Application Publication
`
`May 13, 2004 Sheet 10 of 32 US 2004/0093376 A1
`
`
`
`Page 11 of 69
`
`

`

`Patent Application Publication
`
`May 13, 2004 Sheet 11 of 32
`
`US 2004/0093376A1
`
`
`
`Page 12 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 12 of 32
`
`US 2004/0093376 A1
`
`
`
`RET?,
`
`Page 13 of 69
`
`

`

`Patent Application Publication
`
`May 13, 2004 Sheet 13 of 32
`
`US 2004/0093376A1
`
`
`
`Page 14 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 14 of 32 US 2004/0093376 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`file info.html
`CHTML
`<INC SRC=bbody.html>
`<centerd C. font size= i Information
`Services</fonts <br></center
`<FORM ACTION=phone : indira
`CSELECT SIZE=3 NAME=ur SIMPLE
`&OPTION
`VALUE=http://svcs/traffic.html>Traffic
`Watch.</OPTION:
`OPTION
`VALUE=http://svcs/billing.html>Billing
`Inquiries</OPTION>
`OPTION
`VALUE=http://svcs/portfolio.html> Stock
`Portfolio</OPTION>
`<OPTION VALUE=http://svcs/weather.html>Local
`Weather.</OPTION>
`</SELECT>
`(INPUT TYPE=Submit KEYs sends
`</FORMS
`<INC SRC=endbbody.html>
`</HTML>
`file bbody.html:
`< Specifies a body surrounded by a
`border with the operator logo at the
`top. The including file can add a
`subcaption immediately below, but left
`and right margin are set so text will
`not impinge on the border >
`<BODY BACKGROUND=sq-back.gif
`BGPROPERTIESs fixed
`(BLOCKQUOTE
`CBR
`<INC SRC= stdlogo.html>
`file endbbody.html:
`<! Closes the tags that were opened by
`bbody.html>
`</BLOCKQUOTE:
`</BODY).
`file stologo.html:
`<! Provides the operator logo with the text
`flow set to receive a subcaption for the
`logo >
`<IMG SRC=l-airtel.gif>
`&BR
`
`
`
`
`
`oDO... O
`Ai t e
`Information Services
`T offic Watch
`Billing Inquiries
`StockPortfolio
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 15 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 15 of 32
`
`US 2004/0093376 A1
`
`czy
`
`4u333',
`
`
`
`
`
`
`
`Page 16 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 16 of 32 US 2004/0093376A1
`
`HTMLp Process Key (Key)
`
`-
`
`KEY BOUND?
`
`1504
`
`CAL SHELLACTIVATE
`ON BOUND ACTION
`
`UP OR DOWN
`KEY?
`
`CALL
`SHELLDEFAULT
`PROCESS KEY
`
`USER
`INTERFACE
`GAOGETS?
`
`SCROLL NEXT
`LINE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`NEX USER
`NTERFACE GADGE
`VSBLE7
`
`SCROLL NEXT LNE N
`CHOSEN ORECTION
`
`
`
`
`
`
`
`
`
`NEX USER
`INTERFACE GADGET
`VISIBLE
`
`MAKE NEXT USER
`NTERFACE GADGE
`THE CURRENT
`GADGE
`
`RETURN
`
`FIG. 15
`
`Page 17 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 17 of 32
`
`US 2004/0093376 A1
`
`
`
`
`
`
`
`|--~~~~);- ?65 // : ?an? „ = NoI. Low Lsoa =GOHLEW WHO **
`
`
`
`
`
`
`
`< XCIOE >
`<rIWJLH >
`
`Page 18 of 69
`
`

`

`Patent Application Publication
`
`May 13, 2004 Sheet 18 of 32
`
`US 2004/0093376 A1
`
`
`
`Page 19 of 69
`
`

`

`Patent Application Publication
`
`May 13, 2004 Sheet 19 of 32
`
`US 2004/0093376A1
`
`
`
`Page 20 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 20 of 32 US 2004/0093376A1
`
`CLENT
`
`CHTML
`CHEAD
`<TITLE Purchase Skis C/TITLEd
`
`
`
`CBODY >
`Finally, your credit card info:
`FORM METHOD=GET
`ACTION="http://skishop/cgi-bin/confirm">
`CINPUT TYPE=hidden NAME=name
`VALUE=" George Will">
`<INPUT TYPE=hidden NAME = Street VALUE="2
`Anywhere">
`<INPUT TYPE=hidden NAME=city
`VALUE=" Madison">
`INPUT TYPE=hidden NAME = State VALUE="WI"
`<INPUT TYPE=hidden NAME = zip
`VALUE="53705">
`Credit Card: <br>
`<INPUT TYPE=RADIO NAME=cardtype
`VALUE=visa VISA
`<INPUT TYPE=RADIO NAME=card type
`VALUE-IncMasterCard
`<INPUT. TYPE=RADIO NAME=cardtype
`VALUE=annex>American Express <br>
`Number: <INPUT TYPE=TEXT NAME = CInumber >
`<INPUT KEY=send TYPE=submit name=buy
`value="Review" >
`</FORM
`</BODY>
`</HTML>
`GET http://skishop/cgi
`bin/credit?name=George+Will&street=2+Anyw
`here&city=Madison&state=WI&zip=53705&card
`type=visa&cnumber=7732222156661000
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG. 17b. 1
`PRIOR ART
`
`
`
`insert credit card
`parameters into template
`maintained on the Server,
`and upload to client.
`
`Page 21 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 21 of 32 US 2004/0093376 A1
`
`CLIENT
`
`SERVER
`
`CHTML
`CHEAD
`<TITLE: Confirin Order.</TITLE
`
`
`
`
`
`as ODY
`kDL>
`<DT>Name: </DT2 <DD> <b>George Will.</b></DD>
`<DT>Address: C/DT> <DD><b>2 Anywhere,
`Madison, WI, 53705 CAb></DDs
`<DTs Chedit Card: </DTs (DDs Cbs VISA (bre
`773.2222 lis866l OOOC ?hoe C/DDs
`</DL>
`FORM METHOD-POST
`ACTION="http://skishop/cgi-bin/purchase" >
`(INPUT TYPE=hidden NAME=natine
`VALUE=" George Will" >
`CINPUT TYPE=hidden NAME=street VALUE="2
`Anywhere">
`<INPUT TYPE=hidden NAME-city
`VALUE="Madison" >
`CINPUT TYPE=hidden NAME=state VALUE="WI" >
`<INPUT TYPE=hidden NAME-zip
`VALUE-53705
`<INPUT TYPE=hidden NAME = card type
`WALUEs "visa">
`<INPUT TYPE=hidden NAME=CInumber
`WALUE="732222 S S 6 SOOO" >
`<INPUT KEY=1 TYPE=submit name=buy
`value="Buy!">
`</FORM>
`</BODY>
`</HTML
`POST http://skishop/cgi
`bin/purchase?name=George+Will&street=2+An
`ywhere&city=Madison&state=WI&zip=53705&ca
`rd type=visa&cnumber=773222215666l OOO
`
`
`
`
`
`
`
`FIG. 17b.2
`PRIORART
`
`Process transaction with
`
`all parameters.
`
`m
`
`Page 22 of 69
`
`

`

`Patent Application Publication
`
`May 13, 2004 Sheet 22 of 32
`
`US 2004/0093376 A1
`
`
`
`Tulau º urog ? se?ox nd * º Ilºg
`
`Page 23 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 23 of 32
`
`US 2004/0093376 A1
`
`
`
`
`
`< „tun? - n?pazo // : 3 T | ? „=NOILOVLXGN=GOHJL3W WHO dº
`
`
`
`
`
`: ggarppe InoK peau ?A MON
`
`Page 24 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 24 of 32
`
`US 2004/0093376 A1
`
`
`
`§§§§§§-<TIWJLH >
`
`+Tuu?u º q ! p3, o ; e IT J
`
`Page 25 of 69
`
`

`

`Patent Application Publication
`
`May 13, 2004 Sheet 25 of 32 US 2004/0093376A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 26 of 69
`
`

`

`US 2004/0093376 Al
`
`C 8
`
`b
`Ola
`
`JLLHew
`
`oootegg9stez7zeLe/zequnus
`
`esta/aeddAqprzeo
`
`jAng/Ang
`
`
`
`ezeymiuy7/399298
`
`TLTMeb2z09H/sweu
`
`uostpen/Aato
`
`so.es/atz
`
`IM/937eI8
`
`Patent Application Publication May 13, 2004 Sheet 26 of 32
`
`LIDAIIS0}JUISMO]
`yEsSULT?
`
`ageyoind
`
`/utq-t65o//Lsod
`
`Page 27 of 69
`
`Page 27 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 27 of 32
`
`US 2004/0093376 A1
`
`
`
`Page 28 of 69
`
`

`

`Patent Application Publication
`
`May 13, 2004 Sheet 28 of 32 US 2004/0093376A1
`
`
`
`Page 29 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 29 of 32
`
`US 2004/0093376 A1
`
`
`
`
`
`
`
`~^ W JO 9
`
`G69988835CZCZGGG99G]
`
`Page 30 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 30 of 32
`
`US 2004/0093376 A1
`
`
`
`<XIGJENGO/>
`
`Page 31 of 69
`
`

`

`Patent Application Publication May 13, 2004 Sheet 31 of 32 US 2004/0093376A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`builtin:keyl
`
`builtinsmiley
`builtinsignal
`
`le.
`
`builtin:keyS
`
`G)
`
`
`
`
`
`
`
`
`
`
`
`a
`
`&
`a
`
`o
`
`|
`
`|
`
`builtin-type
`
`
`
`
`
`
`
`
`
`
`
`builtin:type6
`builtin:type7
`
`2.
`
`builtinbiglype
`builtin:bigtype2
`builtin:bigtype3
`builtin:big type4
`builtin:big type5
`builtin:big type6
`builtin:big type7
`
`FIG. 24
`
`Page 32 of 69
`
`

`

`Patent Application Publication
`
`May 13, 2004 Sheet 32 of 32 US 2004/0093376A1
`
`
`
`Page 33 of 69
`
`

`

`US 2004/0093376 A1
`
`May 13, 2004
`
`WIRELESS COMMUNICATION DEVICE WITH
`MARKUP LANGUAGE BASED MAN-MACHINE
`INTERFACE
`
`RELATED APPLICATIONS
`0001. This application is a continuation of application
`Ser. No. 09/907,091, filed Jul. 16, 2001, which is a continu
`ation of application Ser. No. 09/604.833, filed Jun. 27, 2000,
`which is a continuation of application Ser. No. 09/057,394,
`filed Apr. 8, 1998, now U.S. Pat. No. 6,173,316, which is
`incorporated herein by reference in its entirety.
`
`BACKGROUND
`0002) 1. Field of Invention
`0003. This invention relates to man-machine interfaces
`for wireleSS communication devices, and more particularly,
`to man-machine interfaces constructed from markup lan
`guageS.
`0004 2. Background of the Invention
`0005 Wireless communication devices are becoming
`increasingly prevalent for personal communication needs.
`These devices include, for example, cellular telephones,
`alphanumeric pagers, "palmtop' computers and personal
`information managers (PIMS), and other small, primarily
`handheld communication and computing devices. WireleSS
`communication devices have matured considerably in their
`features, and now Support not only basic point-to-point
`communication functions like telephone calling, but more
`advanced communications functions, Such as electronic
`mail, facsimile receipt and transmission, Internet access and
`browsing of the World Wide Web, and the like.
`0006 Generally, wireless communication devices have
`Software that manage various handset functions and the
`telecommunications connection to the base Station. The
`Software that manages all the telephony functions is typi
`cally referred to as the telephone Stack, and the Software that
`manages the Screen display and processes user inputs of key
`presses is referred to as the Man-Machine-Interface or
`“MMI.” The MMI is the topmost, and most visible layer of
`the wireleSS communication device's Software.
`0007 Because wireless communication devices have
`generally reached a very desirable and Small form factor, the
`primary determinant of a Successful device will likely be in
`its feature Set and its ease of use. Thus, the ability to quickly
`design, test, and deliver wireleSS communication devices
`that are both easy to use, and have a rich Set of features
`attributes that are often opposed to one another-will be
`essential to Successful product performance.
`0008 However, wireless communication devices present
`a variety of more challenging design and implementation
`issues that do not arise with larger processor-based Systems,
`Such as notebook and desktop computers, which may also
`have similar telecommunication features. These design chal
`lenges include the design of the user interface, the customi
`Zation of the devices for particular Service operators, the
`integration of Internet and World Wide Web access with
`other communication functionality, and the Software devel
`opment process.
`
`0009 User Interface Design Constraints
`0010 Unlike desktop and notebook computers, wireless
`communication devices have a form factor which requires a
`very Small Screen display size. Desktop computers typically
`have displays with at least 14" Screen size, and resolution
`typically between 640x480 and 1024x768 pixels. In con
`trast, wireleSS communication devices typically have a
`Screen size between 25x25 mm and 80x120 mm, and
`resolutions between 90x60 to 120x120 pixels, or about
`3-8% of the size of the desktop or notebook screen. As a
`direct result, the user interface design of the wireleSS com
`munication device must provide access to essentially the
`Same features as desktop computers, Such as electronic mail,
`facsimiles, and Web browsing, yet with only a fraction of the
`Screen area for displaying text, images, icons, and the like.
`This problem of constructing the user interface to provide
`these features is particularly significant when handling Web
`based content, Since conventional Web content, Such as
`forms, assume the larger Screen size of conventional desktop
`computers. Displaying Such forms on the Small Screen of a
`wireleSS communication device results in jumbled and dif
`ficult to use content.
`0011) Another user interface limitation of wireless com
`munication devices is the Severely restricted Set of inputs
`available to the user. Conventional desktop or notebook
`computers have cursor based pointing devices, Such as
`computer mouse, trackballs, joysticks, and the like, and full
`keyboard. This enables navigation of Web content by click
`ing and dragging of Scroll bars, clicking of hypertext links,
`and keyboard tabbing between fields of forms, such as
`HTML forms. Wireless communication devices have a very
`limited number of inputs, typically up and down keys, and
`one to three Softkeys.
`0012. Accordingly, it is desirable to provide a software
`architecture for the MMI of a wireless communication
`device that enables the customization and use of user
`interface with Web content accounting for the limited screen
`resolution and input functionality of the wireleSS commu
`nication device.
`Integration of Internet/Web Functional with Tele
`0013)
`phony
`0014 With the advent of the Internet and the World Wide
`Web, the highest performance wireleSS communication
`devices provide complete Internet access and the ability to
`directly browse the World Wide Web. Current devices
`provide Internet and World Wide Web access through a
`Strictly modal interface, in which the user must Select
`between using the wireleSS communication device in a
`browser mode, or in its native telecommunications mode for
`making telephone calls, accessing a stored telephone book,
`sending facsimiles, and the like. In the “browser mode” the
`user cannot dial a telephone number to make a telephone
`call; likewise in the telephony mode, the user cannot access
`a Web site. Thus, the user is unable to operate the wireless
`communication device in a Seamless fashion that allows
`Web content to be downloaded and manipulated in context
`of the telephone functions, Such as embedding an item of
`Web content that is obtained while browsing into the user's
`telephone book, or into an email message.
`0015 Accordingly, it is desirable to provide an MMI in
`which Internet and World Wide Web access features are
`
`Page 34 of 69
`
`

`

`US 2004/0093376 A1
`
`May 13, 2004
`
`Seamlessly integrated with the telephony and other controls
`of the wireleSS communication device So that user can acceSS
`any feature of the wireleSS communication device at any
`time.
`0016 Software Engineering of the MMI
`0.017. Typically, an MMI is implemented as a module in
`a larger piece of code that manages the telephone control
`functions. The MMI is coded in the same computer language
`as the rest of the telephone control software. This makes the
`MMI difficult to modify without using the same program
`ming skills and tools used to create the entire telephone
`control Software. In other words, changing anything in the
`MMI requires the services of a skilled programmer familiar
`with the underlying telephony programming details and
`computer language. In addition, Since the MMI is an integral
`part of the code for the telephone control Software, imple
`menting new changes in the MMI means compiling a new
`image of all the telephone control Software, and testing the
`result to ensure that the new MMI features are compatible
`with all other code modules. In short, problems introduced
`by modifying the MMI software can potentially cause the
`handset to malfunction, disrupting Service on the network to
`other users. Depending on the extent of the modifications,
`the change of any portion of the telephone control Software
`can result in bugs, and/or the need for new type approval of
`the entire wireleSS communication device. Thus, it is desir
`able to provide a Software architecture which Separates the
`design and implementation of the MMI functionality from
`the implementation of the telephone control Software, allow
`ing the manufacturer to quickly and Safely customize the
`MMI design to suit the needs of a particular customer
`0018 Customizing of the MMI for Service Operators:
`“Branding”
`0019. In the wireless communication device industry, the
`Services operators, Such as cellular Service providers, are
`interested in attracting and retaining their customers by
`aggressively branding their wireleSS communication device
`products, and offering new telephony features and network
`Services to the user. Important among these are Services that
`add value to the user, Such as voice mail, electronic mes
`Saging, Internet access, and the like as mentioned above.
`"Branding” is the embedding of insignia, logos, or other
`indicia into the MMI of the wireless communication device
`and its features that identifies it to the consumer as origi
`nating from the Service operator.
`0020. The manufacturers of the wireless communication
`device, who typically provide only the basic hardware
`components, must therefore provide a way for the Service
`operator to integrate these features and Services into the
`wireleSS communication device by Software programming,
`and provide a mechanism for branding the features. A key
`problem is that these Services are necessarily different in
`their functionality and requirements, and the task of provid
`ing users with a current array of Services and features is a
`difficult one.
`0021 Wireless communication device manufacturers
`have traditionally attacked this problem by making a special
`version of the wireleSS communication device control Soft
`ware for each Service operator Selling that wireleSS commu
`nication device in conjunction with its own communication
`Services. Each Specific version of the wireleSS communica
`
`tion device contains the device manufacturer's branding, the
`operator's branding, and Support for whatever features and
`Services the Service operator Supports. Each of these ver
`Sions becomes a different piece of Software to be tested,
`maintained, and modified as new features or Services are
`provided to the consumer. This significantly increases the
`Software development expense and maintenance issues. Fur
`ther, unless the wireleSS communication device manufac
`turer provides the Service operator with the Source code of
`the MMI and telephone control software, it requires the
`wireleSS communication device manufacturer to be directly
`involved in the branding and MMI design requirements of
`the Service operator. Thus, it is desirable to provide a
`Software architecture for an MMI that allows the wireless
`communication device manufacturer to provide a single
`body of telephone control Software to each Service operator,
`and allows each Service operator to independently, and
`without the assistance of the wireleSS communication device
`manufacturer, design, implement, and brand the MMI for the
`wireleSS communication device.
`SUMMARY OF THE INVENTION
`0022. The present invention overcomes the various limi
`tations of conventional wireleSS communication devices by
`providing a wireless communication device with an MMI
`that is based on a markup language. A markup language is
`a computer programming language that allows the content of
`a page or a Screen display to be defined by the inclusion of
`predefined Symbols in the content itself indicating the logi
`cal components of the content, instructions for the layout of
`the content on the page or Screen, or other data which can be
`interpreted by Some automatic System responsible for dis
`playing, manipulating or modifying the content.
`0023. In one aspect the present invention provides a
`wireleSS communication device including a user interface
`defined in a markup language. To effect this, the present
`invention includes a markup language browser that it uses to
`provide both telephony control of the wireleSS communica
`tion device, in response to user Selection of telephony
`functions in the user interface, and Internet access via the
`HyperText Transport Protocol (HTTP), in response to user
`Selection of data items associated with content located on the
`Internet.
`In one embodiment, the telecommunication control
`0024.
`and other functions of the wireleSS communication device
`are defined in various user interface pages written in a
`markup language. Each control function is associated with,
`or activated by a Uniform Resource Locator (URL). A URL
`is a data item specifying a protocol for obtaining a data item,
`and which data item should be fetched or manipulated. The
`user interface pages are Stored in a local memory of the
`wireleSS communication device, and fetched by the browser,
`which decodes them and displays the appropriate user
`interface elements. The browser can also modelessly fetch
`markup language pages or other content that is Stored
`remotely, by accessing Such pages via a telecommunications
`network Such as the World Wide Web, and likewise decode
`and display these remotely accessed pages. When a user
`interface page is displayed, user Selection of a control
`function passes a URL or command data to the browser. The
`browser effects a telecommunication function in response
`the received URL or command.
`0025 The browser preferably includes a number of pro
`tocol handlers, including a telephony protocol handler, and
`
`Page 35 of 69
`
`

`

`US 2004/0093376 A1
`
`May 13, 2004
`
`a local file protocol handler, and a remote file protocol
`handler, and a number of content handlers, including a
`markup language handler. The telephony protocol handler
`decodes URLs for telephony control features such as tele
`phone dialing and answering, and activates underlying func
`tions of telephony control Software controlling the hardware
`of the wireleSS communication device. Any content of the
`URL that is needed to display the telephony controls is
`provided to the markup language content handler, which
`parses the content and displays it on a Screen display. The
`markup language content handler is generally responsible
`for displaying any fetched markup language pages, includ
`ing all user interface pages, and for receiving user inputs to
`these pages via forms and other input means.
`0026. The markup language handler generally receives
`content from two Sources, the local file protocol handler and
`the remote file protocol handler. The remote file protocol
`handler decodes URLs for accessing content on the World
`Wide Web, and provides the fetched content, such as a Web
`page, form, applet, or the like to the markup language
`content handler for outputting the content to the Screen
`display of the wireleSS communication device. One Suitable
`remote file protocol handler implements HTTP. The local
`file protocol handler decodes URLS for accessing local user
`interface files and provides Such content to the markup
`language content handler. In a preferred embodiment of the
`MMI, the user interface is defined in HyperText Markup
`Language, or “HTML,” and the browser includes a HTML
`content handler that displays both Web content and user
`interface featured defined in HTML.
`0027. The use of a markup language to define the MMI
`of a wireleSS communication device provides numerous
`advantages over conventional MMI software architectures.
`First, the use of a markup language allows for complete and
`seamless integration of Internet and World Wide Web access
`into the telephony and other features of the wireleSS com
`munication device. Since the MMI uses a markup language
`such as HTML to display all the functional screens, the
`World Wide Web content (which is also written in HTML)
`has the same appearance as other features of the wireleSS
`communication device. More particularly, the pages of the
`MMI are accessed using URLs, just as Web content is
`Similarly accessed. When displaying a functional page the
`wireleSS communication device accesses a local URL, when
`displaying Web content, the wireleSS communication device
`automatically initiates a connection with a Web server to
`obtain the Web content. The markup language based MMI
`thus allows for a modeleSS user interface that enables the
`user to access the Internet and the World Wide Web at any
`time, without having to Switch the wireleSS communication
`device between telephony and “browser' modes, as in
`conventional devices.
`0028. As a further benefit of the markup language based
`MMI, Web content such as Web pages, forms, and the like,
`from the World Wide Web can be accessed and incorporated
`directly into telephony, messaging, and other non-Internet
`based features of the wireleSS communication device. For
`example, in a preferred embodiment, a wireleSS communi
`cation device has a telephone book of Stored telephone
`numbers and names. Conventionally, the user would have to
`manually key these entries in using the keypad of the
`wireleSS communication device. In a wireleSS communica
`tion device using an MMI in accordance with the present
`
`invention, the user could add an entry to the telephone book
`Simply by accessing a telephone directory on the World
`Wide Web, which can contain HTML that allows the user to
`easily Store the information directly into the telephone book.
`0029. The use of a markup language also reduces the
`complexity of the Software engineering process for creating
`the MMI for a particular wireless communication device.
`First, since the MMI of the present invention is based on a
`markup language, only a very limited amount of program
`ming skill is needed to design a fully featured user interface,
`unlike a conventional MMI which requires a programmer
`skilled in C or other low level language programming.
`Editing and modifying the user interface requ