(12) United States Patent
`(10) Patent N0.:
`US 6,677,965 B1
`
`Ullmann et al.
`(45) Date Of Patent:
`Jan. 13, 2004
`
`U5006677965B1
`
`(54) RUBBER BAND GRAPHICAL USER
`INTERFACE CONTROL
`
`(75)
`
`73 As '
`)
`(
`Signee
`( * ) Notice:
`
`:
`
`Inventors: Cristi N. Ullmann, Austin, TX (US);
`.
`1:41.191? Cigygn, Rougd ROG? TXT(;J(S),
`(UISC) ae
`' mwn’
`eorge OWH’
`I t
`l B '
`M h'
`t'
`610:;333233 AEISIELCESN;$1;ng
`L
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 1540,) by 447 days.
`
`(21) Appl. No: 09/615,771
`(22)
`Filed:
`Jul. 13, 2000
`
`7
`
`iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii345/78céoif57égg
`(2:) 31; (El
`'
`”””””””””””””””””
`_l’
`(
`)
`_'
`'
`(58) Field of Search ................................. 343/684, 688,
`345/784—787’ 830’ 973
`References Cited
`US. PATENT DOCUMENTS
`
`(56)
`
`5,495,566
`5,508,717
`5,615,367
`5,634,095
`5,664,128
`5,721,847
`5,790,1 15
`5,838,326
`
`>>>>>>>>
`
`2/1996
`4/1996
`3/1997
`5/1997
`9/1997
`2/1998
`8/1998
`’l ’1/1998
`
`4646464646
`
`*
`
`Kwatinetz ................... 345/785
`Miller
`.........
`345/858
`
`............. 707/102
`Bennett et al.
`Wang et al.
`................ 345/763
`.. 345/708
`Bauer .......
`
`.. 395/333
`Johnson
`
`, 345/327
`Pleyer et al.
`Card et al. .................. 345/775
`
`................... 725/87
`1/1999 Dunn et a].
`5,861,906 A *
`5,963,191 A * 10/1999 Jaaskelainen, Jr.
`.......... 345/856
`6/2000 DeStefano .......
`6,075,531 A *
`345/788
`
`fif‘lacfilmi 6" A ~
`$3373:
`3588(1)
`21:24:32 :1 *
`
`’
`..........
`1rsc
`’
`,
`,5
`*
`........
`.345/857
`6,480,211 B1 * 11/2002 Masri
`
`345/442
`6,483,509 B1 * 11/2002 Rabenhorst
`OTHER PUBLICATIONS
`
`Screen Dumps of Microsoft Publisher (pp. 1—4, 1999).*
`* cited b
`examiner
`.
`y
`.
`.
`.
`.
`.
`Przmary Examzner—Kristme chald
`Assistant Examiner—Le Nguyen
`(74) Attorney, Agent, or Firm—Robert H. Frantz; David A.
`Mlms’ Jr‘
`ABSTRACT
`(57)
`A rubber band variable-rate GUI control for use in conjunc-
`tion with GUI controls, such as sliders, scroll bars, virtual
`joysticks, and oppositional buttons (up/down,
`left/right,
`plus/minus, next/previous, etc.) Auser may place a pointer
`or cursor over a GUI control, select (e.g. click and hold a
`mouse button) and drag away from the control, and a Virtual
`rubber band extending between the pointer and the GUI
`control is displayed. The further the user moves or “pulls”
`the pointer from the GUI control,
`the thinner and more
`elyptical the rubber band is displayed, and the faster the
`selections scrolls or the control operation is repeated. When
`the user moves the pointer closer to GUI control, the rubber
`band is displayed fatter and more circular in shape, and the
`scrolling or control operation repetition slows.
`
`30 Claims, 6 Drawing Sheets
`
`201
`
`
`
`Page 1 of 14
`
`GOOGLE EXHIBIT 1026
`
`Page 1 of 14
`
`GOOGLE EXHIBIT 1026
`
`

`

`US. Patent
`
`Jan. 13, 2004
`
`Sheet 1 0f 6
`
`US 6,677,965 B1
`
`__._._____.___....__._.....________._
`
`._n
`
`_
`
`_________.__....__-..____n_
`
`.I
`
`oEBtGm
`
`wESun
`
`
`529$gmJ._.
`.dOmEEon0H“Susanna:
`
`5355::
`
`m9826.8:DmU
`
`_,
`
`.Xv
`
`m1—1
`____——_“’h_________—____-
`
`N.SRR
`
`BEBE
`
`Ho
`
`#2353
`
`m2‘an
`
`
`
`mot/omvamanoEmEoDno539:0"
`
`@39th
`
`
`
`.5Zflea“?9mm
`
`5M
`
`hill‘iSE
`
`Page 2 of 14
`
`Page 2 of 14
`
`
`
`
`

`

`US. Patent
`
`Jan.13,2004
`
`Sheet2 0f6
`
`US 6,677,965 B1
`
`
`
`
`:58onm
`
`
`
`m23m:
`
`CN
`
`2%Esaw5%Em2i
`
`wXuoEmem
`
`Nil?85
`
`Page 3 of 14
`
`Page 3 of 14
`
`

`

`US. Patent
`
`Jan. 13, 2004
`
`Sheet 3 0f 6
`
`US 6,677,965 B1
`
` vm
`
`EmtDmtbmjmw>.:.Zm_mmDo
`
`
`
`
`
`620:0Emt:
`
`E8E
`
`
`
`Page 4 of 14
`
`Page 4 of 14
`
`
`

`

`US. Patent
`
`Jan. 13, 2004
`
`Sheet 4 0f 6
`
`US 6,677,965 B1
`
`I
`
` QEEzmtombmdmBENEB“86.5
`
`
`
`
`
`32%:
`
`uv23mm
`
`
`
`Page 5 of 14
`
`Page 5 of 14
`
`
`
`
`

`

`US. Patent
`
`Jan. 13, 2004
`
`Sheet 5 0f 6
`
`US 6,677,965 B1
`
`201
`
`
`
`
`
`Page 6 of 14
`
`Page 6 of 14
`
`

`

`U
`
`tn
`
`J
`
`US 6,677,965 B1
`
`wamhfinsm52:3cweSwagn
`
`nE§R
`
`..80IoS.a:a2
`mE?9P38a55
`
`mmn,mm.....................................S
`
`Page 7 of 14
`
`Page 7 of 14
`
`
`

`

`US 6,677,965 B1
`
`1
`RUBBER BAND GRAPHICAL USER
`INTERFACE CONTROL
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is related to US. application Ser. No.
`09/578,748, filed on May 25, 2000, by Cristi Ullmann, et al.,
`and to US. application Ser. No. 09/579,257, filed on May
`25, 2000, by Allen Wynn, et al., and to U.S. application Ser.
`No. 09/578,749, filed on May 25, 2000, by Allen Wynn, et
`al., all of which are commonly assigned.
`
`10
`
`FEDERALLY SPONSORED RESEARCII AND
`DEVELOPMENT STATEMENT
`
`This invention was not developed in conjunction with any
`Federally-sponsored contract.
`
`MICROFICIIE APPENDIX
`
`Not applicable.
`
`INCORPORATION BY REFERENCE
`
`The related applications, all filed on May 25, 2000, now
`Ser. Nos. 09/578,748; 09/578,749; 09/579,257 by Cristi
`Ullmann, et al., Allen Wynn, et al; and Allen Wynn, et al.,
`respectively, are incorporated herein by reference in their
`entirety, including drawings, and hereby are made a part of
`this application.
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`This invention relates to the arts of graphical user inter—
`face controls provided on computer displays, and to related
`displays such as personal digital assistants (“PDA”) and
`wireless communications devices.
`
`2. Description of the Related Art
`Graphical user displays on electronic devices such as
`computers, personal digital assistants, and wireless commu—
`nications devices are common in the art. As these devices
`become more and more integral to activities in the daily lives
`of people such as web browsing, word processing, and even
`making telephone calls, the efficiency and ergonomics of
`these interfaces are extremely important. Auser will quickly
`notice whether one user interface is easier and quicker to use
`than another based upon the number of operations required
`to perform a certain task. Further, the level of “intuitiveness”
`implemented in the user interface can greatly increase the
`usefulness and speed with which the user can operate the
`device.
`
`Many graphical user interfaces follow the same conven-
`tions for navigation of and selection of controls on the user
`interface. Such common graphical user interfaces (“GUI”)
`include windows or frames-type displays in Microsoft’s
`Windows operating system, UNIX operating system, IBM’s
`OS/2 and AIX operating systems, Microsoft’s Windows CE
`operating system, Apple Computer’s operating system, and
`many “proprietary” windows—like GUI’s on various devices
`such as PDA’s, cell phones, electronic organizers, etc.
`FIG. I shows the basic organization of many prior art
`computer systems and computer—based devices. The system
`(10) typically includes a microprocessor or central processor
`unit (“CPU”) including computer memory, and may also
`include interfaces (15) to hard disks and removable disk
`media (14), and possibly interfaces to computer or commu-
`nications networks (11) such as a network interface card
`
`I\)m
`
`LA)LII
`
`40
`
`60
`
`2
`(“NIC”) (17) or wireless communications interface. Com—
`monly used NICs include EtherNet
`local area network
`(“LAN”) cards, dial-up modem cards, and wireless commu-
`nications circuits. Some of these circuits may be provided
`integral to the device (10), such as the case with web-
`enabled cell phones, or may be available as add-in circuits
`such as PCI cards or PCMCIA cards for personal computers.
`Additional typical hardware provided in the system (10)
`includes a group of user input/output (“I/O”) (16) devices
`(13), such as a display, keyboard, and/or pointing device,
`accompanied by common user I/O interface circuits to allow
`the CPU and system software to access and use the user I/O
`devices (13). Common display devices include VGA and
`LCD monitors for personal computers, and LCD panels for
`PDA and cell phones. Common pointing devices include
`“arrow keys”, “tab” keys, mouse and scroll-mouse, joystick,
`track balls, glide point
`touch sensitive pads, and touch
`sensitive displays with stylus pens. Some devices, such as
`certain PDA’s, do not include an actual keyboard, but rather
`rely upon a touch-sensitive LCD display with virtual point-
`ing device and stylus.
`The software functionality of computer-based systems
`(10) can be divided into low-level device drivers and BIOS
`(18), an operating system (103), and applications programs.
`Low—level device drivers provide hardware—specific inter—
`face functions which allow the operating system and appli-
`cation programs to access the system hardware through
`common or generic application program interfaces (“API”).
`Some device drivers include necessary communications
`3rotocols and methods associated with a particular hardware
`device, such as a modem or wireless communications inter-
`ace. Application programs can be further divided into
`aortable applications, or applets (101), a portable application
`arogram interpreter (19), and non-portable, system-
`dependent applications (102). Such division of hardware and
`software functionality is well-understood in the art, and can
`3e applied to large computer systems such as an IBM
`AS/400 equally as well to small computer systems such as
`PDA’s running Windows CE.
`On the system display is typically shown a conventional
`rame or “window” of information related to a specific
`system function or program, such as a word processor,
`spreadsheet, to-do list, address book, e-mail client, or web
`Drowser. FIG. 2 shows a typical arrangement of a frame (20)
`including a title bar (21), tool bar (24), display area (25), and
`a pointer (201). The pointer (201) moves in response to the
`system pointing device and/or keyboard, such as typing on
`he arrow keys, movement of a mouse, track ball, or joystick.
`The title bar may include “buttons” for minimizing or
`restoring the frame (23), closing the program (22), as well as
`a display of the name of the function or program. The tool
`war (24) typically includes a number of text and/or iconic
`options, such as “file” and “edit” drop-down lists.
`The display area may be further navigable using vertical
`scroll (27) and horizontal scroll (26) bars, which are espe-
`cially useful
`for system displays of information which
`cannot be shown completely within one frame on the
`display. “Clicking” on the right button (29) or left button
`(28), or “dragging” the horizontal position indicator (201)
`causes the information in the display area (25) to be panned
`or scrolled left or right. Likewise, using the up button (29),
`down button (28), or vertical position indicator (200) can
`cause panning or scrolling in a vertical motion. These types
`of scroll bars are common on word processors, spreadsheets,
`graphical design packages,and web browsers.
`In typical operation of these types of GUI displays, the
`user moves the pointer (201) to a button or text option, and
`
`
`
`Page 8 of 14
`
`Page 8 of 14
`
`

`

`US 6,677,965 B1
`
`3
`selects that option by clicking on it and/or dragging it. In
`some cases, keyboard-only navigation or selection is pos-
`sible by using special combinations of keys, such as ALT-F
`to select the “file” drop-down menu. FIGS. 35: through 36
`show a typical slider or scroll control (30),
`typical
`list
`control (34) with up/down choice buttons, and a virtualjoy-
`stick (38) which is described in the related application. The
`slider control (30) typically includes a slider control button
`(31), and two oppositional movement buttons (32 and 33).
`Similar vertical slider controls are also well known in the art.
`The typical
`list control
`(34) includes a display of the
`currently selected list item (35), up/down (next/previous,
`etc.) choice buttons (36, 37), and may include a drop-down
`list of possible choices. The virtual joystick (38) includes a
`stick portion (39) which can be moved by selecting it with
`the pointer and moving the pointer control device, such as a
`mouse, left, right, up or down.
`The article “Multistream input: An experimental study of
`document scrolling methods” in the IBM Systems Journal
`Vol. 38, No. 4, 1999, “investigates alternative methods to
`improve user performance for browsing World Wide Web
`and other documents.” This investigation involved a task
`requiring both scrolling and pointing, and compared the
`following input devices:
`(a) standard mouse, (b) wheel
`mouse, (c) joystick mouse (combination Standard Mouse
`with a joystick), and (d) trackpoint (such as found on an IBM
`ThinkPad computer) with supplemental Standard Mouse.
`The results of the experimental study showed that a
`mouse with a finger wheel did not improve a user’s perfor-
`mance or subjective rating over a standard mouse. Indeed,
`the participants commented that it was tedious and tiring to
`repeatedly roll the wheel, although this mode was intuitive.
`The researchers believed that the lack of “self—centering”
`placed the wheel mouse at a fundamental disadvantage to do
`elIective rate control. The results also indicated that
`the
`Joystick Mouse and the Trackpoint with supplemental Stan-
`dard Mouse performed better than the Standard Mouse
`alone, both in terms of performance and user perception.
`“Self-centering” is a very useful attribute of an input device
`when scrolling through documents, maps, etc. In addition,
`there are some instances where the user may wish to scroll
`in a direction other than directly up, down, left or right. For
`example, a user may want to scroll a map at an angle towards
`a destination city.
`In the related patent applications, a virtual joystick, an
`infinite spinlist and a spinlist with preview and postview lists
`were disclosed. All of these new GUI controls enhance the
`efficiency and the intuitiveness of graphical user interfaces.
`However, a intuitive control for rate of change is needed
`beyond that currently found in the art. For example, when a
`user is panning around a large document in framed display,
`only a portion of the whole document may be displayed and
`viewed at a time. Repeated clicking on up/down and left/
`right icons, grabbing a sliding of slider buttons, and grab-
`bing a tilting of virtual joysticks do not provide intuitive
`interaction as one may find in the “real world” through
`similar viewing arrangements. An example of such a real
`world viewing arrangement is viewing part of a total scene
`through the viewfinder of a camera. When manually moved
`to find a particular subject in the scene, a human user will
`move the camera quickly until the general area of the subject
`is found, and will slow movement until
`the subject
`is
`properly centered in the view finder. To draw an analogous
`comparison to present day GUI controls and cameras, one
`can imagine a camera mounted to a motorized tripod, with
`the motor controls only having a single speed of up/down
`control and left/right control. To center a subject in the
`
`10
`
`I\)m
`
`LA)LII
`
`40
`
`60
`
`4
`viewfinder of the camera, the user would repeatedly operate
`the up/down and left/right controls until
`the subject
`is
`centered, using a few long duration moves to find the general
`area of the subject, and many short “bursts” to fine position
`the subject. If the subject is initially far from the current
`view of the camera (or frame), this can take very long to pan
`the subject
`into the frame. To compensate for this,
`the
`constant panning speed may be increased, which makes it
`more difficult
`to achieve the fine positioning within the
`frame. In the real world, this kind of problem, such as for
`remotely controlled security cameras,
`is solved by using
`variable-rate panning controls, such as potentiometers, but
`the problem is prevalent
`in computer frame-based GUI
`environments.
`
`Therefor, there is a need in the art for a virtual control
`method which allows for intuitive variable-rate control of
`common GUI control types. This new control method is
`preferably compatible with a wide range of existing GUI
`controls and the method of rate variation indication is easily
`recognizable in order to enhance the efficiency and intu-
`itiveness of the new method.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The following detailed description when taken in con-
`junction with the figures presented herein present a complete
`description of the present invention.
`FIG. 1 shows the prior art arrangement of computers and
`computer-based devices with graphical user interfaces.
`FIG. 2 presents an example GUI frame.
`FIGS. 3a through 36 illustrates several GUI controls,
`including a slider bar, a 2-dimensional virtual joystick, and
`list up/down buttons.
`FIGS. 4a through 4e show a rubber band control applied
`to several GUI controls,
`including a slider bar,
`a
`2-dimensional virtual joystick, and list up/down buttons.
`FIG. 5 illustrates a new type of GUI control, a “fixed
`slider”, usable with the new rubber band control.
`FIG. 6 shows an sample nonlinear function for use in
`determining the nonlinear response to the position of the
`joystick.
`
`SUMMARY OF THE INVENTION
`
`The rubber band variable-rate GUI control is used in
`
`conjunction with existing GUI controls, such as sliders,
`scroll bars, virtual joysticks, and oppositional buttons (up/
`down, left/right, plus/minus, next/previous, etc.). The rubber
`band method allows for a user to place a pointer or cursor
`over a GUI control, and select (e.g. click and hold a mouse
`button) and drag away from the control, at which time a
`virtual rubber band extending between the pointer and the
`GUI control is displayed. The further the user moves or
`“pulls” the pointer from the GUI control, the thinner and
`more elyptical the rubber band will be displayed, and the
`faster the selections will scroll or the control operation will
`be repeated. When the user moves the pointer closer to GUI
`control, the rubber band will be displayed fatter and more
`circular in shape, and the scrolling or control operation
`repetition will slow. If the pointer is moved past the point of
`origin for the GUI control, the rubber band is displayed from
`the other side of where the pointer started and scrolling will
`reverse. De-selection of the control, such as by releasing a
`mouse button, causes the rubber band display to be removed,
`and operation of the GUI control is ceased. The rubber band
`control operation may be vertical, horizontal, or a combi-
`nation of both, on the display. A new variation of a slider
`
`Page 9 of 14
`
`Page 9 of 14
`
`

`

`US 6,677,965 B1
`
`5
`control, called a “fixed slider”, which is enabled by the use
`and existence of the rubber band control is also disclosed.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`The foregoing and other objects, features and advantages
`of the invention will be apparent from the following more
`particular description of a preferred embodiment of the
`invention, as illustrated in the accompanying drawings
`wherein like reference numbers represent like parts of the
`invention,
`
`The invention is preferably realized as a user graphical
`interface control or dialog object integrated into a function
`or application program on a computer or computer-based
`system. The computer-based system is preferably provided
`with an operating system which provides common GUI
`control functions, such as pointer location and movement
`detection, event handling based upon areas on the display in
`which the pointer is located, and event handling of pointer
`selection such as mouse clicks. As such, the invention can be
`realized on personal computers, palm-top computers and
`personal digital assistants, as well as other devices incorpo—
`rating graphical user displays such as cell phones and
`wireless web browsers.
`
`The following disclosure sets forth the invention as a
`modification or enhancement to a pointer display and control
`in a graphical user interface, which allows it to be used and
`compatible with existing GUI controls. However, it will be
`readily recognized by those skilled in the art that the method
`of the invention can be directly incorporated in to each GUI
`control as well. The rubber band control operation may be
`vertical, horizontal, or a combination of both, on the display,
`and may include 1, 2 and 3 dimensional or variable opera-
`tion.
`The rubber band variable—rate GUI control is used in
`conjunction with existing GUI controls, such as sliders,
`scroll bars, virtual joysticks, and oppositional buttons (up/
`down, left/right, plus/minus, next/previous, etc.), and with
`new types of GUI controls such as a “fixed scroll button” or
`“fixed slider”, which are made possible by the rubber band
`control.
`
`Operation of the Rubber Band Control
`The rubber band control allows a user to place a pointer
`or cursor over a GUI control, and select and drag away from
`the control, at which time a virtual rubber band extending
`between the pointer and the GUI control is displayed, as
`shown in FIGS. 4a through 4f. FIG. 4a shows a typical
`horizontal slider or scroll control (40) with the rubber band
`display (42) between the pointer (201) and the scroll button
`(41). The further the user moves or “pulls” the pointer (201)
`from the GUI control,
`the thinner and more elyptical in
`shape the rubber band will be displayed, and the faster the
`selections will scroll or
`the control operation will be
`repeated, as shown in FIG. 4b. When the user moves the
`pointer closer to GUI control,
`the rubber band will be
`displayed fatter and more round in shape, and the scrolling
`or control operation repetition will slow. If the pointer is
`moved pass the point of origin for the GUI control,
`the
`rubber band is displayed from the other side of where the
`pointer started and scrolling will reverse, as shown in FIG.
`4c. The the control is de-selected, such as by releasing the
`mouse button or lifting a touch—screen stylus, the rubber
`band display is removed and operation of the control is
`ceased.
`FIG. 4d illustrates how the rubber band control can be
`
`applied to a list control with up/down buttons (43), which
`
`6
`pulling the rubber band further from either the up or down
`buttons would cause faster sequencing through the list
`choices. FIG. 4e shows use of the rubber band on a virtual
`
`5
`
`joystick (44, 45), which was described in the related
`application, to allow rate control of the joystick in one or two
`variables.
`
`10
`
`I\)m
`
`LA)LII
`
`40
`
`In FIG. 5, a new type of control not previously possible
`but enabled by the rubber band control is shown. While this
`control (50) has a basic appearance of a scroll or slider
`control, the center button (51) is “pinned” or fixed at the
`center of the control. Clicking on it (or otherwise selecting
`it) and “stretching” the rubber band (42) causes it to operate
`in either direction at varying rates depending on the distance
`drawn between the pointer and the center button (51). The
`user scrolls by “tugging” the scroll button. The remainder of
`the control (up/down arrows at end, and empty portions of
`scroll bar) function exactly like a normal scroll—bar. This can
`of course be applied to a vertical scroll bar or control, as
`well. As disclosed previously, de-selection of the control
`causes the rubber band display to be removed and operation
`of the control to stop.
`
`Rubber Band Software Control
`
`The result of the stretching of the rubber band between the
`pointer and the control can be a linear or nonlinear relation-
`ship to the distance between the pointer and the control. For
`example, for each unit of distance increase between the
`pointer and the control (the length of the rubber band), a unit
`of rate change can be increased, for a linear relationship. For
`a nonlinear relationship, each unit increase in the distance
`between the pointer and the control (the length of the rubber
`band) and cause increasingly more rate change. FIG. 6
`shows an example nonlinear function for possible use in
`implementing a nonlinear rubber band control.
`In the preferred embodiment, the rubber band control is
`implemented as a “control of a control”. In other words,
`instead of the pointer directly operating a GUI control, such
`as a slider control button,
`the rubber band control will
`operate the control. So, the rubber band control software is
`activated when the pointer is moved in the vicinity of the
`GUI control and when the GUI control button has been
`selected. As the user moves the pointer, the rubber band
`control software receives pointer control status, such as
`mouse location, movement and mouse button states. The
`rubber band control software then processes the pointer
`control device data, and outputs data directly to the GUI
`control software to emulate normal inputs from the mouse.
`TABLE 1 shows the pseudocode to realize this control flow,
`which can be implemented by those skilled in the art using
`any appropriate programming language and methodology
`suitable for the computing platform target, such as “C”,
`“C++”, Microsoft Windows CE, or Java.
`
`TABLE 1
`
`Rubber Band Control Pseudocode
`
`60
`
`W'hile pointer has selected control
`get position of pointer “ptripos”
`get position of control “ctlipos”
`calculate distance D between pointer and control
`waititime = function of distance D
`bandithick = function of distance D
`band shape = function of distance D
`draw rubber band with thickness and shape from ptripos to ctlipos
`operate the targeted control one time (Le. emulate a click on the
`control)
`
`Page 10 of 14
`
`Page 10 of 14
`
`

`

`US 6,677,965 B1
`
`7
`
`TABLE 1-continued
`
`Rubber Band Control Pseudocode
`
`wait for waititime
`End While
`
`the control software
`In the pseudocode of TABLE 1,
`retrieves the current pointer position and control button
`positions, either through a function call to the operating
`system, passing in of variables, or accessing global vari-
`ables. Many systems will represent these positions as x-y
`coordinates, often in units of pixels or PELS, either in
`absolute terms (from a fixed point on the display) or frame-
`relative terms (from a fixed point in the frame).
`
`The distance between the pointer and the control button
`can be found simply by applying the Pythagorean theorem
`between the x-y coordinates of the pointer and the x-y
`coordinates of the control button, by calculating the square
`root of the sum of the squared difference in the X positions
`and the squared difference in the y positions, otherwise
`known as the Pythagorean theorem:
`
`D=SQRT {(Xpuiutar—Xcuutrulibutlull)2+(ypuiu[EI'—yCL)lttIDl7bI/ttt0l1)2}
`
`Then, this distance value can be used to determine the
`display thickness and amount of elyptical shape of the
`rubber band, and the time to wait for the next position update
`and control operation. TABLE 2 shows an example to
`determine the rubber band thickness, which can be easily
`implemented as a look up table or code function. It allows
`a number of distance ranges, delimited by C,- constants, to set
`several discrete values of the thickness T,— of the rubber band
`displayed. “Roundness” of the shape of the rubber band can
`be indexed to T, or determined by an alternate function or
`table.
`
`TABLE 2
`
`Rubber Band Thickness
`
`Distance (pel)
`0 < D E C1
`C2 < D 2 03
`C3 < D § 04
`
`Thickness (pel)
`T1
`T2
`T3
`
`Cmaxrl < D E Cum
`
`me
`
`TABLE 3 shows an example function with both linear and
`nonlinear (in this case accelerating) response to the wait
`time based upon distance, which can also be implemented in
`a variety of manners, such as a look up table or software
`function. In the example of TABLE 3, Wm, is a standard wait
`value. For a linear relationship, the time waited between
`each successive loop through the code and operation of the
`control is inversely proportional to the distance between the
`pointer and the control. Thus, the greater the distance, the
`faster the operation of the control. For a nonlinear response,
`any appropriate function or table of data can be used, such
`as the inverse hyperbolic relationship shown in TABLE 3.
`
`8
`
`TABLE 3
`
`Wait Time
`
`Linear
`Wait (msec)
`
`Ws‘d/C1
`Wad/C7
`Wm/C3
`
`Distance (pel)
`
`0 < D 2‘ C1
`C7 < D E C3
`C3 < D g C4
`
`Cmax 1 < D é me
`
`We‘d/Cmax
`
`Nonlinear
`Wait (msec)
`
`Wm/DZ
`ng/D2
`Wad/DZ
`
`Ward/D2
`
`Finally, in the code of TABLE 1, the actual control is
`operated once, such as emulating a click on the up, down,
`left or right button, by the rubber band control code, thus
`causing one step of action on the targeted GUI control. Then,
`the wait is executed, and the loop is repeated if the control
`is still selected.
`
`New Types of Controls Enabled by the Rubber
`Band Control
`
`Whereas prior art GUI controls are defined based upon the
`need to display movement of the control, the advent of the
`rubber band display and control allows definition of new
`controls which do not themselves move. Such as control is
`a “fixed scroll bar”, as shown in FIG. 5. This control
`operates and appears much the same as a normal scroll bar,
`with operable left and right buttons. But, the center button
`(51) does not move and is “pinned” in place. Pulling or
`tugging on the center button with the rubber band control
`would operate the scrolling action as if the button were
`actually moving. This allows the scroll control to be dis-
`played in much less display space, while it
`is intuitively
`obvious to a user how to operate it. Conservation of display
`space is increasing important as the display space available
`on miniaturized computer-based devices such as PDAs and
`cell phones is very limited, and as full-size GUI computer
`displays become more crowded with control and short cut
`icons.
`
`Summ ary
`
`The rubber band GUI control element as disclosed pro—
`vides enhanced control of GUI-based computer applications.
`One primary advantage is that it can be used in conjunction
`with existing GUI controls to give improved, variable rate
`operation of those controls. Another advantage is that
`it
`allows definition of new types of immovable GUI controls
`which conserve display space. It will be understood from the
`foregoing description that various modifications and
`changes may be made in the preferred embodiment of the
`present invention without departing from its true spirit, such
`as the use of alternate programming methodologies or
`languages, alternate computer platforms and software, oper—
`ating systems and user control objects. It is intended that this
`description is for purposes of illustration only and should not
`be construed in a limiting sense. The scope of this invention
`should be limited only by the language of the following
`claims.
`What is claimed is:
`1. A method for interfacing with a user of a computer
`system having a graphical user display, the method com-
`prising the steps of:
`(a) controlling a pointer element 011 the graphical user
`display with a user pointer controller, the pointer con-
`troller having position and selection status responsive
`to operation by a user;
`
`10
`
`I\)m
`
`LA)LII
`
`40
`
`60
`
`Page 11 of 14
`
`Page 11 of 14
`
`

`

`US 6,677,965 B1
`
`9
`(b) displaying on said graphical user display a control
`icon having one or more action areas, said action areas
`comprising at least one forward action area or one
`backward action area;
`(c) monitoring said position and selection status of said
`pointer controller to detect if one of said action areas
`has been selected by said user;
`(d) altering the appearance of said control icon and said
`pointer to visually indicate the selection of said action
`area by displaying a virtual rubber band on said graphi-
`cal user display, said virtual rubber band encircling said
`action area and said pointer element; and
`(e) allowing repetitive operation of said action area while
`said selection is detected by autonomously repeatedly
`periodically generating a signal to said computer sys-
`tem indicative of a selection made by said user of said
`control icon action area.
`2. The method of claim 1, wherein said forward action
`area is presented as a button.
`3. The method of claim 1, wherein said backward action
`area is presented as a button.
`4. The method of claim 1, wherein said action areas
`comprise a slider action area.
`5. The method of claim 4, wherein said backward action
`area is presented as a scroll bar control button.
`6. The method of claim 1, wherein said action areas
`comprise a virtual joystick action area.
`7. The method of claim 1, wherein said periodic genera-
`tion of a signal to the computer system further comprises
`modifying the rate of periodic generation of a signal relative
`to a determined distance between the pointer element and
`the control icon.
`8. The method of claim 1, wherein said periodic genera-
`tion of a signal to the computer system further comprises
`modifying the apparent thickness of the virtual rubber band
`relative to a determined distance between the pointer ele—
`ment and the control icon.
`9. The method of claim 1, wherein said periodic genera-
`tion of a signal to the computer system further comprises
`modifying the apparent oval shape of the virtual rubber band
`relative to a determined distance between the pointer ele—
`ment and the control icon.
`10. The method of claim 1, wherein said step of display-
`ing on said graphical user display a control
`icon further
`comprises displaying a slider bar control icon having two
`oppositional action areas and a virtual slider button, said
`virtual slider button being fixed in place and operable by said
`virtual rubber band.
`11. A computer-readable storage medium having stored
`therein computer-readable program code for interfacing with
`a user of a computer system, the computer system having