as) United States
`a2) Patent Application Publication 10) Pub. No.: US 2008/0094356 Al
`(43) Pub. Date: Apr. 24, 2008
`
`ORDINGet al.
`
`US 20080094356A1
`
`(54) METHODS FOR DETERMINING A CURSOR
`POSITION FROM A FINGER CONTACT
`WITH A TOUCH SCREEN DISPLAY
`
`cation No. 60/879,253, filed on Jan. 7, 2007. Provi-
`sional application No. 60/824,769, filed on Sep. 6,
`2006.
`
`Publication Classification
`
`(51)
`
`Int. CL
`(2006.01)
`GO6F 3/033
`(2006.01)
`GO6F 3/041
`(52) U.S. CMe
`ccsscssscsssscssscsstensssessseestenessnseevesee 345/157
`
`(57)
`
`ABSTRACT
`
`(76)
`
`Inventors: Bas ORDING, San Francisco, CA
`(US); Scott FORSTALL, Mountain
`View, CA (US); Greg CHRISTIE, San
`Jose, CA (US)
`
`Correspondence Address:
`MORGAN LEWIS & BOCKIUS LLP/ APPLE
`COMPUTERINC.
`2 PALO ALTO SQUARE
`3000 EL CAMINO REAL
`PALO ALTO, CA 94306 (US)
`
`(22)
`
`Filed:
`
`Sep. 4, 2007
`
`Related U.S. Application Data
`
`A portable device with a touch screen display detects a
`contact area of a finger with the touch screen display and
`then determinesa first position associated with the contact
`area. The cursor position of the finger contact is determined,
`(21) Appl. No.:—11/850,015
`at least in part, based on: the first position, one or more
`distances between the first position and one or more of the
`user interface objects; and one or more activation suscepti-
`bility numbers, each associated with a respective user inter-
`face object in the plurality of user interface objects. If the
`cursor position falls into the hidden hit region of a virtual
`push button on the touch screen display, the portable device
`is activated to perform operations associated with the virtual
`push button.
`
`(60) Provisional application No. 60/937,993, filed on Jun.
`29, 2007. Provisional application No. 60/946,716,
`filed on Jun. 27, 2007. Provisional application No.
`60/879,469, filed on Jan. 8, 2007. Provisional appli-
`
`External Port 124
`
`
`
`Portable Multifunction Device 100
`
`
`
`
`
`Touch Screen 112
`
`113
`
`Home
`204
`
`Accelerometer(s)
`168
`
`APPLE 1006
`
`APPLE 1006
`
`1
`
`

`

`Patent Application Publication Apr. 24, 2008 Sheet 1 of 10
`
`US 2008/0094356 Al
`
`136
`
`148
`
`149
`
`149-1
`
`149-2
`
`149-3
`
`149-4
`
`149-5
`
`149-6
`
`150
`
`151
`
`Portable Multifunction Device
`100
`126 Applications (continued)
`M0
`Calendar Module
`432
`Widget Modules
`Weather Widget
`Stocks Widget
`134
`135
`Calculator Widget
`136
`Alarm Clock Widget
`437
`Dictionary Widget
`138
`139 PP
`440
`User-Created Widget(s)
`141
`Widget Creator Module
`442
`Search Module
`143
`144
`145
`146
`147
`
`
`
`Power
`System
`External
`Port
`
`162
`
`124
`
`Speaker
`111
`
`RF Circuitry
`—
`—Rudio i
`110 +)
`Microphone
`113
`
`Sensor
`
`16
`
`Accelerometer(s)
`168
`
`106
`
`103
`
`158
`/O Subsystem
`
`Optical
`Other Input
`
`
`
`Display
`sensor(s)
`
`Controller
`Controller(s)
`Controller
`
`156—“|__
`
`103
`
`103
`
`Display System
`
`Touch-Sensitive
`
`112
`
`Optical
`Sensor(s)
`164
`
`Figure 1A
`
`103
`
`
`
`Other Input
`Control
`Devices
`
`160
`
`116
`
`Memory
`102
`
`Operating System
`Communication Module
`
`Contact/Motion Module
`
`Graphics Module
`Text Input Module
`
`GPS Module
`
`
`
`|2|2oO©°avu5Ooo>@°a<===s°2°=2@=°<3=aoOoMD==©°a < @
`Tso]221 Slalo2/3/38
`
`elephone Module
`
`-mail Client Module
`
`nstant Messaging Module
`
`Camera Module
`
`mage Management Module
`
`Music Player Module
`Browsing Module
`
`122
`
`104 |;Controller
`
`
`103
`e
`5 103
`
`:N
`
`
`
`Peripherals
`Interface
`
`
`,——"™
`
`118
`
`103
`
`103
`
`103
`
`
`
`2
`
`

`

`Patent Application Publication Apr. 24,2008 Sheet 2 of 10
`
`US 2008/0094356 Al
`
`Memory 102
`
`126
`
`Portable Multifunction Device
`109
`
`Applications (continued)
`
`Widget Modules
`
`tae
`
`Me
`40
`or
`
`
`
`492 CalculatorWidget_|aoe , 149-4
`
`Contacts Module
`
`14905
`
` Video Conference Module
`
`Dictionary Widget
`ee eye
`User-Created Widget(s)
`150
`Widget Creator Module
`454
`Search Module
`
`Power
`System
`
`162
`
`1 1
`
`52
`153
`
`elephone Module
`
`-mail Client Module
`
`nstant Messaging Module
`Blogging Module
`Camera Module
`
`mage Management Module
`Video and Music Player Module
`
`Notes Module
`
`Map Module
`Browsing Module
`
`104
`
`422
`
`120
`
`106
`
`156
`
`112
`
`yr 147
`Port
`18
`RF Circuitry
`193__|
`493
`Ly 108
`5 103/—nudio i
`
`410 |)
`Microphone
`113
`
`193.
`
`Controller
`
`Processor(s)
`
`Peripherals
`Interface
`
`103
`
`I
`
`
`-——™
`
`
`
`
`OtherInput
`Display
`
`
`Controller
`Controller(s)
`
`
`Proximity
`
`i
`Accelerometer(s)
`
`168
`
`103
`
`/O Subsystem
`Optical
`sensor(s)
`Controller
`
`164
`
`103
`
`103
`
`Touch-Sensitive
`Display System
`
`OtherInput
`Control
`Devices
`
`Optical
`Sensor(s)
`
`Figure 1B
`
`Speaker
`
`111
`
`160
`
`116
`
`3
`
`

`

`Patent Application Publication Apr. 24,2008 Sheet 3 of 10
`
`US 2008/0094356 Al
`
`Portable Multifunction Device 100
`SIM Card Slot 210
`Optical
`ptica
`Sensor 164
`
`Speaker 111
`
`Proximiy
`roximity
`Sensor 166
`
`212
`7).
`prone
`“. phone
`jack
`
`206
`
`200
`
`oo
`<<
`
`ao
`OQ
`N
`
`
`
`202
`
`Touch Screen 11
`
`Microphone
`
`113
`
`a-
`
`NS
`
`ovff
`
`Home )
`
`. 204 ;
`N.
`\
`y,
`ae
`
`Accelerometer(s)
`
`168
`
`External Port 124
`Figure 2
`
`4
`
`4
`
`

`

`Patent Application Publication Apr. 24,2008 Sheet 4 of 10
`
`US 2008/0094356 Al
`
`206
`
`Portable Multifunction Device
`100
`
`
`
`Current Time ~ 308
`310“ Day and Date~ 312
`
`-
`
`Wallpaper image
`314
`
`Slide to unlock
`
`Touch Screen 11
`
`ao
`
`“
`
`ae
`
`Microphone
`
`413
`
`( Home )
`
`\ 204 /
`~S
`
`Accelerometer(s)
`
`168
`
`Figure 3
`
`5
`
`

`

`Patent Application Publication Apr. 24,2008 Sheet 5 of 10
`
`US 2008/0094356 Al
`
`206
`
`Portable Multifunction Device
`100
`
`
`
`
`
` QO9e
`
`
`
`Photos
`144
`
`Camera
`143
`
`
`
`
`
`
`
`Videos
`145
`
`
`Jan
`
`
`
`15
`
`
`
`Figure 4A
`
`Weather
`149-14
`
`Stocks
`149-2
`
`Calendar
`148
`
`a
`C
`ifN aaa fo “User-
`
`+-X+ |
`
`Z
`
`\
`\
`
`|
`
`\
`
`NS
`
`Y
`
`J
`
`NM
`
`\
`
`Calculator
`149-3
`
`Alarm
`149-4
`
`~~
`
`\
`
`ABC
`
`/
`Dictionary
`149-5
`
`| Created
`
`\ Widget /
`Widget
`149-6
`
`
`
`fa
`
`Browser
`147
`
`Music
`146
`
`(Mewrnone|“Home (Acceererneterts)(Mewrnone|204 (Acceererneterts)
`
`Touch Screen 112
`
`6
`
`6
`
`

`

`Patent Application Publication Apr. 24,2008 Sheet 6 of 10
`
`US 2008/0094356 Al
`
`206
`
`Portable Multifunction Device
`100
`
`
`
`
`
`
`
`
`
`
`
`
`
`468
`
`Jan
`15
`OO®@
`
`
`
`
`
`
`
`
`
`
`
`Calendar
`148
`
`Photos
`144
`
`a
`
`~N,
`
`f
`
`\
`
`woe
`
`\
`os
`
`/
`
`a“
`
`Calculator
`149-3
`
`Stocks
`149-2
`
`Weather
`149-1
`
`Ny
`J NS
`
`8
`Settings
`412
`
`vu
`
`
`
`iPod
`Browser
`152
`147
`J
`
`Touch Screen 11
`
`Accelerometer(s)
`
`113)
`
`4
`
`N
`
`( Home
`| 204
`
`Figure 4B
`
`7
`
`7
`
`

`

`Patent Application Publication Apr. 24,2008 Sheet 7 of 10
`
`US 2008/0094356 Al
`
`
`
`Detect a contact area of a finger with the touch
`screen display 501
`
`
`
`
`
`
`
`
`Determinea first position associated with the
`contact area 503
`
`
`
`
`
`Determine a cursor position based onthe first
`position, one or more distances between thefirst
`position and one or more of the user interface
`objects; and one or more activation susceptibility
`numbers, each associated with a respective user
`interface object 505
`
`Figure 5
`
`8
`
`

`

`Patent Application Publication Apr. 24,2008 Sheet 8 of 10
`
`US 2008/0094356 Al
`
`5610
`5. masa
`*
`*:
`°.
`
`le
`
`*.
`
`.
`
`at
`
`Figure 6C
`
`Figure 6D
`
`W,' and W," are two numbers with opposite signs
`
`
`Ad.
`*
`3
`Pp4
`
`Ww
`
`9606
`
`W2
`
`5604
`
`W,”
`.
`5602
`
`Wa
`
`4
`
`9608
`
`W3
`
`.
`
`5606
`
`
`
`W2
`e
`5604
`iP,
`el
`ad*
`p
`
`W
`
`
`5608
`
`Figure 6E
`
`Figure 6F
`
`9
`
`

`

`Patent Application Publication Apr. 24,2008 Sheet 9 of 10
`
`US 2008/0094356 Al
`
`
`FigureZA Figure6H
`
`W,
`
`9602
`
`W2
`
`9604
`
`1
`
` Figure6l
`
`(
`
`
`Ww;|Py Wa
`
`5606
`
`5608
`
`5606
`
`5608
`
`Figure 6K
`
`Figure 6L
`
`10
`
`

`

`Patent Application Publication Apr. 24,2008 Sheet 10 of 10
`
`US 2008/0094356 Al
`
`
`
`
`
`a“ “
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`a
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`6
`ZN
`— 4
`N
`/
`
`N
`2
`
`/
`
`|
`“
`S
`\
`
`7
`-
`
`/
`
`
`
`v
`
`
`
`\
`
`
`
`
`
`SN
`\
`
`
`
`
`
`
`
`.
`/
`\
`A
`/
`A
`\
`4 SU NUNN “Le”
`
`/
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Figure 60
`
`11
`
`11
`
`

`

`US 2008/0094356 Al
`
`Apr. 24, 2008
`
`METHODS FOR DETERMINING A CURSOR
`POSITION FROM A FINGER CONTACT WITH A
`TOUCH SCREEN DISPLAY
`
`RELATED APPLICATIONS
`
`[0001] This application claimspriority to U.S. Provisional
`Patent Application Nos. 60/937,993, “Portable Multifunc-
`tion Device,”filed Jun. 29, 2007; 60/946,716, “Methods for
`Determining a Cursor Position from a Finger Contact with
`a Touch Screen Display,” filed Jun. 27, 2007; 60/879,469,
`“Portable Multifunction Device,” filed Jan. 8, 2007; 60/879,
`253, “Portable Multifunction Device,” filed Jan. 7, 2007;
`and 60/824,769, “Portable Multifunction Device,”filed Sep.
`6, 2006. All of these applications are incorporated by
`referenced herein in their entirety.
`
`[0002] This application is related to the following appli-
`cations:
`(1) U.S. patent application Ser. No. 10/188,182,
`“Touch Pad For Handheld Device,” filed Jul. 1, 2002; (2)
`US. patent application Ser. No. 10/722,948, “Touch Pad For
`Handheld Device,” filed Nov. 25, 2003;
`(3) U.S. patent
`application Ser. No. 10/643,256, “Movable Touch Pad With
`Added Functionality,” filed Aug. 18, 2003; (4) U.S. patent
`application Ser. No. 10/654,108, “Ambidextrous Mouse,”
`filed Sep. 2, 2003;
`(5) U.S. patent application Ser. No.
`10/840,862, “Multipoint Touchscreen,” filed May 6, 2004;
`(6) US. patent application Ser. No. 10/903,964, “Gestures
`For Touch Sensitive Input Devices,” filed Jul. 30, 2004; (7)
`USS. patent application Ser. No. 11/038,590, “Mode-Based
`Graphical User
`Interfaces For Touch Sensitive Input
`Devices”filed Jan. 18, 2005; (8) U.S. patent application Ser.
`No. 11/057,050, “Display Actuator,” filed Feb. 11, 2005; (9)
`USS. Provisional Patent Application No. 60/658,777, “Multi-
`Functional Hand-Held Device,” filed Mar. 4, 2005; (10) U.S.
`patent application Ser. No. 11/367,749, “Multi-Functional
`Hand-Held Device,” filed Mar. 3, 2006; and (11) U.S. patent
`application Ser. No. 11/228,737, “Activating Virtual Keys of
`a Touch-Screen Virtual Keyboard,”filed Jul. 30, 2004. All of
`these applications are incorporated by reference herein.
`
`TECHNICAL FIELD
`
`[0003] The disclosed embodiments relate generally to
`portable electronic devices, and more particularly, to por-
`table devices that adaptively determine a cursor position
`from a finger contact with a touch screen display and then
`perform operations according to the cursor position.
`
`BACKGROUND
`
`[0004] As portable electronic devices become more com-
`pact, and the number of functions performed by a given
`device increase,
`it has become a significant challenge to
`design a user interface that allows users to easily interact
`with a multifunction device. This challenge is particular
`significant for handheld portable devices, which have much
`smaller screens than desktop or laptop computers. This
`situation is unfortunate because the user interface is the
`gateway through which users receive not only content but
`also responses to user actions or behaviors, including user
`attempts to access a device’s features, tools, and functions.
`Some portable communication devices (e.g., mobile tele-
`phones, sometimes called mobile phones, cell phones, cel-
`lular telephones, and the like) have resorted to adding more
`pushbuttons, increasing the density of push buttons, over-
`
`loading the functions of pushbuttons, or using complex
`menu systemsto allow a user to access, store and manipulate
`data. These conventional user interfaces often result
`in
`complicated key sequences and menu hierarchies that must
`be memorized by the user.
`
`[0005] Many conventional user interfaces, such as those
`that include physical pushbuttons, are also inflexible. This
`may prevent a user interface from being configured and/or
`adapted by either an application running on the portable
`device or by users. When coupled with the time consuming
`requirement to memorize multiple key sequences and menu
`hierarchies, and the difficulty in activating a desired push-
`button, such inflexibility is frustrating to most users.
`
`To improve usability, some portable devices use a
`[0006]
`touch screen to render virtual push buttons such as soft
`keyboards and dial pads. From a user’s finger contact with
`the virtual push buttons, a portable device determines the
`service(s) requested by the user and takes actions accord-
`ingly. But because different users often have different shapes
`of fingerprints, it has been a challenge for these portable
`devices to accurately and adaptively identify a user-desired
`virtual push button based on the different shapes of finger-
`prints and different contexts associated with different ser-
`vices supported by a portable device.
`
`[0007] Accordingly, there is a need for portable multifunc-
`tion devices that are configured to adaptively determine a
`cursor position from a finger contact with a touch screen and
`then perform operations according to the cursor position.
`Such configuration increases the effectiveness, efficiency
`and user satisfaction with portable multifunction devices.
`
`SUMMARY
`
`[0008] The above deficiencies and other problems associ-
`ated with user interfaces for portable devices are reduced or
`eliminated by the disclosed portable multifunction device. In
`some embodiments, the device has a touch-sensitive display
`(also known as a “touch screen”) with a graphical user
`interface (GUI), one or more processors, memory and one or
`more modules, programsorsets of instructions stored in the
`memory for performing multiple functions. In some embodi-
`ments, the user interacts with the GUI primarily through
`finger contacts and gestures on the touch-sensitive display.
`In some embodiments, the functions may include telephon-
`ing, video conferencing, e-mailing, instant messaging, blog-
`ging, digital photographing,digital videoing, web browsing,
`digital music playing, and/or digital video playing. Instruc-
`tions for performing these functions may be included in a
`computer readable storage medium or other computer pro-
`gram product configured for execution by one or more
`processors.
`
`[0009] One aspect of the invention involves a computer-
`implemented method performedby a portable multifunction
`device with a touch screen display. The portable device
`detects a contact area of a finger with the touch screen
`display and then determinesa first position associated with
`the contact area. The cursorposition of the finger contactis
`determined, at least in part, based on:the first position, one
`or more distances betweenthefirst position and one or more
`of the user interface objects; and one or more activation
`susceptibility numbers, each associated with a respective
`user interface object in the plurality of user interface objects.
`
`12
`
`12
`
`

`

`US 2008/0094356 Al
`
`Apr. 24, 2008
`
`[0010] Another aspect of the invention involves a graphi-
`cal user interface on a portable multifunction device with a
`touch screen display. The graphical user interface includes a
`cursor and a plurality of user interface objects. The cursor’s
`position is determined, at least in part, based on: a first
`position associated with a contact area of a finger with the
`touch screen display, one or more distances betweenthefirst
`position and one or more ofthe user interface objects, and
`one or more activation susceptibility numbers, each associ-
`ated with a respective user interface objectin the plurality of
`user interface objects.
`
`[0011] Another aspect of the invention involves a portable
`electronic device with a touch screen display with a plurality
`of user interface objects. The device includes one or more
`processors, memory, and a program stored in the memory
`and configured to be executed by the one or more proces-
`sors. The program includes:
`instructions for detecting a
`contact area of a finger with the touch screen display,
`instructions for determining a first position associated with
`the contact area, and instructions for determining a cursor
`position, at least in part, based on:the first position, one or
`more distances betweenthefirst position and one or more of
`the user interface objects, and one or more activation sus-
`ceptibility numbers, each associated with a respective user
`interface object in the plurality of user interface objects.
`
`[0012] Another aspect of the invention involves a com-
`puter-program product that includes a computer readable
`storage medium and a computer program mechanism (e.g.,
`one or more computer programs) embedded therein. The
`computer program mechanism includes instructions that,
`when executed by a portable electronic device, cause the
`device to: detect a contact area of a finger with the touch
`screen display; determinea first position associated with the
`contact area; and determine a cursorposition, at least in part,
`based on:the first position, one or more distances between
`the first position and one or more of the user interface
`objects, and one or more activation susceptibility numbers,
`each associated with a respective user interface object in the
`plurality of user interface objects.
`
`[0013] Another aspect of the invention involves a portable
`electronic device with a touch screen display. The device
`includes: meansfor detecting a contact area of a finger with
`the touch screen display; means for determining a first
`position associated with the contact area; and means for
`determining a cursor position, at least in part, based on: the
`first position, one or more distances betweenthefirst posi-
`tion and one or more of the user interface objects, and one
`or more activation susceptibility numbers, each associated
`with a respective user interface object in the plurality of user
`interface objects.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0014] For a better understanding of the aforementioned
`embodiments of the invention as well as additional embodi-
`ments thereof, reference should be madeto the Description
`of Embodiments below, in conjunction with the following
`drawings in which like reference numerals refer to corre-
`sponding parts throughout the figures.
`
`[0015] FIGS. 1A and 1B are block diagrams illustrating
`portable multifunction devices with touch-sensitive displays
`in accordance with some embodiments.
`
`FIG. 2 illustrates a portable multifunction device
`[0016]
`having a touch screen in accordance with some embodi-
`ments.
`
`FIG. 3 illustrates an exemplary user interface for
`[0017]
`unlocking a portable electronic device in accordance with
`some embodiments.
`
`[0018] FIGS. 4A and 4Billustrate exemplary user inter-
`faces for a menu ofapplications on a portable multifunction
`device in accordance with some embodiments.
`
`FIG. 5 is a flow diagram illustrating a process for
`[0019]
`determining a cursor position from a finger contact with a
`touch screen in accordance with some embodiments.
`
`[0020] FIGS. 6A-6L illustrate exemplary methods for
`determining a cursor position on a touch screen display in
`accordance with some embodiments.
`
`[0021] FIGS. 6M-60illustrate an exemplary method for
`dynamically adjusting susceptibility numbers associated
`with soft keyboard keys as a word is typed with the soft
`keyboard keys in accordance with some embodiments.
`
`DESCRIPTION OF EMBODIMENTS
`
`[0022] Reference will now be made in detail to embodi-
`ments, examples of whichare illustrated in the accompany-
`ing drawings. In the following detailed description, numer-
`ous specific details are set forth in order to provide a
`thorough understanding of the present invention. However,
`it will be apparent to one of ordinary skill in the art that the
`present invention may bepracticed without these specific
`details. In other instances, well-known methods, procedures,
`components, circuits, and networks have not been described
`in detail so as not to unnecessarily obscure aspects of the
`embodiments.
`
`It will also be understood that, although the terms
`[0023]
`first, second, etc. may be used herein to describe various
`elements, these elements should not be limited by these
`terms. These terms are only used to distinguish one element
`from another. For example, a first gesture could be termed a
`second gesture, and, similarly, a second gesture could be
`termed a first gesture, without departing from the scope of
`the present invention.
`
`[0024] The terminology used in the description of the
`invention herein is for the purpose of describing particular
`embodiments only and is not intended to be limiting of the
`invention. As used in the description of the invention and the
`appendedclaims, the singular forms “a”, “an” and “the” are
`intended to include the plural forms as well, unless the
`context clearly indicates otherwise. It will also be under-
`stood that the term “and/or” as used herein refers to and
`
`encompasses any and all possible combinations of one or
`more of the associated listed items.
`It will be further
`
`understoodthat the terms “comprises” and/or “comprising,”
`when used in this specification, specify the presence of
`stated features, integers, steps, operations, elements, and/or
`components, but do not preclude the presence or addition of
`one or more other features,
`integers, steps, operations,
`elements, components, and/or groups thereof.
`
`[0025] Embodiments of a portable multifunction device,
`user interfaces for such devices, and associated processes for
`using such devices are described. In some embodiments, the
`device is a portable communications device such as a mobile
`
`13
`
`13
`
`

`

`US 2008/0094356 Al
`
`Apr. 24, 2008
`
`telephone that also contains other functions, such as PDA
`and/or music player functions.
`
`[0026] The user interface may include a physical click
`wheel in addition to a touch screen or a virtual click wheel
`
`is a user-
`displayed on the touch screen. A click wheel
`interface device that may provide navigation commands
`based on an angular displacement of the wheel or a point of
`contact with the wheel by a user of the device. A click wheel
`mayalso be used to provide a user command corresponding
`to selection of one or more items, for example, when the user
`of the device presses down onat least a portion of the wheel
`or the center of the wheel. Alternatively, breaking contact
`with a click wheel image on a touch screen surface may
`indicate a user command corresponding to selection. For
`simplicity, in the discussion that follows, a portable multi-
`function device that includes a touch screen is used as an
`
`exemplary embodiment. It should be understood, however,
`that some of the user interfaces and associated processes
`maybe applied to other devices, such as personal computers
`and laptop computers, which may include one or more other
`physical user-interface devices, such as a physical click
`wheel, a physical keyboard, a mouse and/or a joystick.
`
`[0027] The device supports a variety of applications, such
`as one or more of the following: a telephone application, a
`video conferencing application, an e-mail application, an
`instant messaging application, a blogging application, a
`photo managementapplication, a digital camera application,
`a digital video camera application, a web browsing appli-
`cation, a digital music player application, and/or a digital
`video player application.
`
`[0028] The various applications that may be executed on
`the device may use at least one common physical user-
`interface device, such as the touch screen. One or more
`functions of the touch screen as well as corresponding
`information displayed on the device may be adjusted and/or
`varied from one application to the next and/or within a
`respective application.
`In this way, a common physical
`architecture (such as the touch screen) of the device may
`support the variety of applications with user interfaces that
`are intuitive and transparent.
`
`[0029] The user interfaces may include one or more soft
`keyboard embodiments. The soft keyboard embodiments
`may include standard (QWERTY)and/or non-standard con-
`figurations of symbols on the displayed icons of the key-
`board, such as those described in U.S. patent application Ser.
`Nos. 11/459,606, “Keyboards For Portable Electronic
`Devices,” filed Jul. 24, 2006, and 11/459,615, “Touch
`Screen Keyboards For Portable Electronic Devices,” filed
`Jul. 24, 2006, the contents of which are hereby incorporated
`by reference. The keyboard embodiments may include a
`reduced number of icons (or soft keys) relative to the
`numberof keys in existing physical keyboards, such as that
`for a typewriter. This may makeit easier for users to select
`one or more icons in the keyboard, and thus, one or more
`corresponding symbols. The keyboard embodiments may be
`adaptive. For example, displayed icons may be modified in
`accordance with user actions, such as selecting one or more
`icons and/or one or more corresponding symbols. One or
`more applications on the portable device may utilize com-
`mon and/or different keyboard embodiments. Thus,
`the
`keyboard embodiment used maybetailored to at least some
`of the applications. In some embodiments, one or more
`
`keyboard embodiments may betailored to a respective user.
`For example, one or more keyboard embodiments may be
`tailored to a respective user based on a word usagehistory
`(lexicography, slang,
`individual usage) of the respective
`user. Some of the keyboard embodiments may be adjusted to
`reduce a probability of a user error when selecting one or
`more icons, and thus one or more symbols, when using the
`soft keyboard embodiments.
`
`[0030] Attention is now directed towards embodiments of
`the device. FIGS. 1A and 1B are block diagramsillustrating
`portable multifunction devices 100 with touch-sensitive
`displays 112 in accordance with some embodiments. The
`touch-sensitive display 112 is sometimes called a “touch
`screen” for convenience, and may also be knownasor called
`a touch-sensitive display system. The device 100 may
`include a memory 102 (which may include one or more
`computer readable storage mediums), a memory controller
`122, one or more processing units (CPU’s) 120, a periph-
`erals interface 118, RF circuitry 108, audio circuitry 110, a
`speaker 111, a microphone 113, an input/output (I/O) sub-
`system 106, other input or control devices 116, and an
`external port 124. The device 100 may include one or more
`optical sensors 164. These components may communicate
`over one or more communication buses or signal lines 103.
`
`It should be appreciated that the device 100 is only
`[0031]
`one example of a portable multifunction device 100, and that
`the device 100 may have more or fewer components than
`shown, may combine two or more components, or a may
`have a different configuration or arrangement of the com-
`ponents. The various components shown in FIGS. 1A and
`1B may be implemented in hardware, software or a combi-
`nation of hardware and software,
`including one or more
`signal processing and/or application specific integrated cir-
`cuits.
`
`[0032] Memory 102 may include high-speed random
`access memory and mayalso include non-volatile memory,
`such as one or more magnetic disk storage devices, flash
`memory devices, or other non-volatile solid-state memory
`devices. Access to memory 102 by other components of the
`device 100, such as the CPU 120 and the peripherals
`interface 118, may be controlled by the memory controller
`122.
`
`[0033] The peripherals interface 118 couples the input and
`output peripherals of the device to the CPU 120 and memory
`102. The one or more processors 120 run or execute various
`software programs and/or sets of instructions stored in
`memory 102 to perform various functions for the device 100
`and to process data.
`
`the peripherals interface
`In some embodiments,
`[0034]
`118, the CPU 120, and the memory controller 122 may be
`implemented on a single chip, such as a chip 104. In some
`other embodiments, they may be implemented on separate
`chips.
`
`[0035] The RF (radio frequency) circuitry 108 receives
`and sends RF signals, also called electromagnetic signals.
`The RF circuitry 108 converts electrical signals to/from
`electromagnetic signals and communicates with communi-
`cations networks and other communications devices via the
`
`electromagnetic signals. The RF circuitry 108 may include
`well-knowncircuitry for performing these functions, includ-
`ing but not limited to an antenna system, an RFtransceiver,
`
`14
`
`14
`
`

`

`US 2008/0094356 Al
`
`Apr. 24, 2008
`
`one or more amplifiers, a tuner, one or more oscillators, a
`digital signal processor, a CODEC chipset, a subscriber
`identity module (SIM) card, memory, and so forth. The RF
`circuitry 108 may communicate with networks, such as the
`Internet, also referred to as the World Wide Web (WWW),
`an intranet and/or a wireless network, such as a cellular
`telephone network, a wireless local area network (LAN)
`and/or a metropolitan area network (MAN), and other
`devices by wireless communication. The wireless commu-
`nication may use any of a plurality of communications
`standards, protocols and technologies,
`including but not
`limited to Global System for Mobile Communications
`(GSM), Enhanced Data GSM Environment (EDGE), high-
`speed downlink packet access (HSDPA), wideband code
`division multiple access (W-CDMA), codedivision multiple
`access (CDMA), time division multiple access (TDMA),
`Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a,
`TEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), voice
`over Internet Protocol
`(VoIP), Wi-MAX, a protocol for
`email (e.g., Internet message access protocol (MAP) and/or
`post office protocol (POP)), instant messaging (e.g., exten-
`sible messaging and presence protocol (XMPP), Session
`Initiation Protocol
`for Instant Messaging and Presence
`Leveraging Extensions (SIMPLE), and/or Instant Messag-
`ing and Presence Service (IMPS)), and/or Short Message
`Service (SMS)), or any other suitable communication pro-
`tocol, including communication protocols not yet developed
`as of the filing date of this document.
`[0036] The audio circuitry 110, the speaker 111, and the
`microphone 113 provide an audio interface between a user
`and the device 100. The audio circuitry 110 receives audio
`data from the peripherals interface 118, converts the audio
`data to an electrical signal, and transmitsthe electrical signal
`to the speaker 111. The speaker 111 converts the electrical
`signal to human-audible sound waves. The audio circuitry
`110 also receives electrical signals converted by the micro-
`phone 113 from sound waves. The audio circuitry 110
`converts the electrical signal to audio data and transmits the
`audio data to the peripherals interface 118 for processing.
`Audio data may be retrieved from and/or transmitted to
`memory 102 and/or the RF circuitry 108 by the peripherals
`interface 118. In some embodiments, the audio circuitry 110
`also includes a headset jack (e.g. 212, FIG. 2). The headset
`jack provides an interface between the audio circuitry 110
`and removable audio input/output peripherals, such as out-
`put-only headphonesor a headset with both output (e.g., a
`headphone for one or both ears) and input (e.g., a micro-
`phone).
`input/output
`[0037] The I/O subsystem 106 couples
`peripherals on the device 100, such as the touch screen 112
`and other input/control devices 116, to the peripherals inter-
`face 118. The I/O subsystem 106 may include a display
`controller 156 and one or more input controllers 160 for
`other input or control devices. The one or more input
`controllers 160 receive/send electrical signals from/to other
`input or control devices 116. The other input/control devices
`116 may include physical buttons(e.g., push buttons, rocker
`buttons, etc.), dials, slider switches, joysticks, click wheels,
`and so forth. In some alternate embodiments, input control-
`ler(s) 160 may be coupled to any (or none) of the following:
`a keyboard, infrared port, USB port, and a pointer device
`such as a mouse. The one or more buttons(e.g., 208, FIG.
`2) may include an up/downbutton for volume controlof the
`speaker 111 and/or the microphone 113. The one or more
`
`buttons may include a push button (e.g., 206, FIG. 2). A
`quick press of the push button may disengage a lock of the
`touch screen 112 or begin a process that uses gestures on the
`touch screen to unlock the device, as described in U.S.
`patent application Ser. No. 11/322,549, “Unlocking a Device
`by Performing Gestures on an Unlock Image,”filed Dec. 23,
`2005, which is hereby incorporated by reference. A longer
`press of the push button (e.g., 206) may turn power to the
`device 100 on or off. The user may be able to customize a
`functionality of one or more of the buttons. The touch screen
`112 is used to implementvirtual or soft buttons and one or
`more soft keyboards.
`
`[0038] The touch-sensitive touch screen 112 provides an
`input interface and an output interface between the device
`and a user. The display controller 156 receives and/or sends
`electrical signals from/to the touch screen 112. The touch
`screen 112 displays visual output to the user. The visual
`output may include graphics, text,
`icons, video, and any
`combination thereof (collectively termed “graphics”). In
`some embodiments, someor all of the visual output may
`correspond to user-interface objects, further details of which
`are described below.
`
`[0039] A touch screen 112 has a touch-sensitive surface,
`sensor or set of sensors that accepts input from the user
`based on haptic and/or tactile contact. The touch screen 112
`and the display controller 156 (along with any associated
`modules and/or sets of instructions in memory 102) detect
`contact (and any movementor breaking of the contact) on
`the touch screen 112 and converts the detected contact into
`
`interaction with user-interface objects