Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 1 of 19
`Case 1:21—cv-01319 Document 1-4 Filed 02/15/21 Page 1 of 19
`
`
`Exhibit 4
`
`Exhibit 4
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 2 of 19
`
`(12) United States Patent
`Rodriguez et al.
`
`USOO6704034B1
`(10) Patent No.:
`US 6,704,034 B1
`(45) Date of Patent:
`Mar. 9, 2004
`
`(54) METHOD AND APPARATUS FOR
`PROVIDING ACCESSIBILITY THROUGH A
`CONTEXT SENSITIVE MAGNIFYING GLASS
`(75) Inventors: Herman Rodriguez, Austin, TX (US);
`Newton James Smith, Jr., Austin, TX
`(US); Clifford Jay Spinac, Austin, TX
`(US)
`(73) Assignee: International Business Machines
`Corporation, Armonk, NY (US)
`Subject to any disclaimer, the term of this
`p isis,s listed under 35
`S.C. 154(b) by 482 days.
`(21) Appl. No.: 09/671,070
`(22) Filed:
`Sep. 28, 2000
`7
`(51) Int. Cl.' .................................................. G09G 5/00
`(52) U.S. Cl. ........................................ 345/860; 345/764
`(58) Field of Search ................................. 345/856-866,
`345/157-160, 764-767; 715/501.1,
`
`(*) Notice:
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`4,757,311. A 7/1988 Nakamura et al.
`4.941,195 A 7/1990 Tanaka et al.
`5,565,888 A 10/1996 Selker
`
`
`
`300
`
`aCKSO. C. a.
`
`2
`
`2
`
`5,731,805 A 3/1998 Tognazzini et al.
`5,754,348 A * 5/1998 Soohoo .................... 359/802
`5.835,923 A 11/1998 Shibata et al.
`5.841,420 A * 11/1998 Kaply et al. ................ 345/421
`5,936,625 A 8/1999 Kahl et al.
`E. A
`to East l
`6,100.889 A 8/2000 Sciammarella et al.
`6,230,170 B1 * 5/2001 Zellweger et al. .......... 71.5/512
`6,252,596 B1
`6/2001 Garland ...................... 345/810
`6,331,861 B1 12/2001 Gever et al. ................ 345/629
`6,535,889 B1 * 3/2003 Headricket al. ........ 707/104.1
`OTHER PUBLICATIONS
`IBM Technical Disclosure Bulletin, Electronic Magnifying
`Glasses, vol. 37, No. 03, Mar. 1994, pp. 353–354.
`* cited by examiner
`Primary Examiner Sy D. Luu
`(74) Attorney, Agent, or Firm-Duke W. Yee; Jeffrey S.
`LaBaw; Stephen J. Walder, Jr.
`(57)
`ABSTRACT
`A method and apparatus in a data processing System for
`presenting a set of objects within the data processing System.
`Responsive to detecting movement of a pointer over an
`object within the Set of objects, an object type is identified
`for the object. Presentation of the object is then magnified
`based on the object type.
`
`52 Claims, 9 Drawing Sheets
`
`- - - - - - - - - - - - - - - - - - - - - - -
`
`- - - -
`
`Shanghai's Oriental Pearl Tower is on the
`Pudong Side of the Bund, the Waterfront
`along the Haungpu River.i is considered
`- - - - - - - - - - - - - - - - - - - - - - -
`the largest man made structure in Asia.
`It is also a good example of China's use
`of modern architecture in Pudong.
`
`TT T T
`
`

`

`I1._Ht
`
`29mMENEN.:N
`
`9017,m6,
`
`3mamNNNCNN
`
`afigs:552
`
`1B4
`
`M.4,N95%
`
`9
`mm
`.1esaC
`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 3 of 19
`w
`12:/un
`g
`
`US. Patent
`
`m0;amnew
`
`wmm2m8
`9.,SN
`
` 9r1a.3M”mas:EOE:
`
`
`muemmhxosm.05222:IEEO:I$88081SN
`
`
`
`1..s82352Baaéo205245525mo:58
`
`
`
`4mE52$5,?momwmwz65g:12%:mg
`
`
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 4 of 19
`
`U.S. Patent
`
`Mar. 9, 2004
`
`Sheet 2 of 9
`
`US 6,704,034 B1
`
`Joe,
`
`FIC 3A
`
`
`
`Y
`
`308
`Shanghai's Oriental Pearl Tower is on the
`Pudong Side of the Bund, the waterfront
`along the Haungpu River. It is considered
`the largest man made structure in Asia.
`It is also a good example of China's use
`of modern architecture in Pudong.
`
`302
`Y
`
`Y
`Shanghai's Oriental Pearl Tower is on the
`Pudong Side of the Bund, the waterfront
`olong the Haungpu River.
`It is considered
`the largest mon made structure in Asia.
`It is also a good example of China's use
`of modern architecture in Pudong.
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 5 of 19
`
`U.S. Patent
`Joe,
`
`
`
`Mar. 9, 2004
`Sheet 3 of 9
`FIC 3C
`
`US 6,704,034 B1
`
`7-506
`
`3,
`
`310
`N
`Shanghai's Oriental Pearl Tower is on the
`Pudong Side of the Bund, the waterfront
`along the Houngpu River.
`It is considered
`the largest man made structure in Asia.
`It is also a good example of China's use
`of modern architecture in Pudong.
`
`302
`Y
`
`308
`N
`Shanghai's Oriental Pearl Tower is on the
`Pudong Side of the Bund, the waterfront
`along the Haungpu River.
`It is considered
`the largest man made structure in Asia.
`It is also a good exomple of Chino's use
`of modern architecture in Pudong.
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 6 of 19
`
`U.S. Patent
`
`Mar. 9, 2004
`
`Sheet 4 of 9
`
`US 6,704,034 B1
`
`Jo,
`
`
`
`FIC. 3E
`
`502
`Y
`
`308
`N
`Shanghai's Oriental Pearl Tower is on the
`Pudong Side of the Bund, the waterfront
`along the Haungpu River,
`it is considered
`the largest mon made structure in Asia.
`It is also a good example of China's use
`of modern architecture in Pudong. 306
`
`Shanghai's Oriental Pearl Tower is on the
`Pudong, Side of the Bund, the waterfront
`Olong the Haungpu River, it is considered
`- - - - - - - - - - - - - - - - - - - - - - -
`the largest man made structure in Asia.
`It is also a good example of Chino's use
`of modern architecture in Pudong. w
`302
`
`-
`
`a
`
`area
`
`a
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 7 of 19
`
`U.S. Patent
`
`Mar. 9, 2004
`
`Sheet 5 of 9
`
`US 6,704,034 B1
`
`
`
`Shanghai's Oriental Pedr TOWer is On the
`Pudong Side of the Bund, the waterfront
`along the Haungpu River, i. considered
`it
`It is also a good example of China's use
`of modern architecture in Pudong.
`
`- Shanghai's Oriental Pearl Tower is on the
`Egg Side of the Bund, the Waterfront
`along
`he Haungpu River.
`It is considered
`the largest man made structure in Asia.
`It is also a good example of China's use
`of modern architecture in Pudong.
`
`- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 8 of 19
`
`U.S. Patent
`
`Mar. 9, 2004
`
`Sheet 6 of 9
`
`US 6,704,034 B1
`
`302
`
`Shanghai's Oriental Pearl Tower is on the
`Pudong Side of the Bund, the waterfront
`along the Haungpu River.
`It is considered
`the largest man made structure in Asia.
`It is also a good example of China's use
`of modern architecture in Pudong.
`
`
`
`
`
`402-owORDS
`40 osENTENCE
`406 to PARAGRAPH
`
`
`
`oOVERLAY-22
`oREFLOW -2
`
`INCREASE
`20X v1.2
`DECRElst OTHER AUDIO
`
`408
`
`409
`
`ls
`
`v T.
`
`7
`we
`
`414
`
`DECREASE
`
`D
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 9 of 19
`
`U.S. Patent
`
`Mar. 9, 2004
`
`Sheet 7 of 9
`
`US 6,704,034 B1
`
`FIC 6
`
`
`
`500
`
`
`
`504
`
`MAGNIFY INFORMATION
`USING CONTEXT
`
`FIC 7
`
`
`
`LOCATE START OF
`TEXT OBJECT
`
`OCATE END OF
`TEXT OBJECT
`
`QUERY AVAILABLE FONTS
`FOR MAGNIFICATION
`
`SELECT MAGNIFICATION
`FONT FOR TEXT
`
`MAGNIFY TEXT USNG A
`SELECTED FONT SIZE
`
`CREATE OVERLAY
`MAGNIFICATION WINDOW
`FOR TEXT
`
`DISPLAY MAGNIFIED
`OBJECT IN WINDOW
`
`
`
`
`
`
`
`
`
`
`
`700
`
`702
`
`704
`
`706
`
`708
`
`710
`
`712
`
`
`
`600
`
`
`
`
`
`
`
`FIG. 6
`
`DETECT POINTER
`MOVEMENT OVER
`NEW OBJECT TO
`MAGNIFY
`
`OBJECT
`S TEXT2
`
`
`
`
`
`
`
`
`
`OBJECT
`IS IMAGE
`
`
`
`
`
`MAGNIFY MAGE
`OBJECT
`
`610
`
`OBJECT
`S AUDIO
`t
`
`MAGNIFY AUDIOT
`OBJECT
`
`PROCESS
`OBJECT
`
`614
`
`604
`
`MAGNIFY TEXT
`
`
`
`
`
`
`
`
`
`
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 10 of 19
`
`U.S. Patent
`
`Mar. 9, 2004
`
`Sheet 8 of 9
`
`US 6,704,034 B1
`
`FIC. 8
`
`FIC. 9
`
`800
`
`QUERY AVAILABLE
`IMAGE OBJECTS
`
`
`
`802
`
`
`
`
`
`804
`
`
`
`806
`
`SELECT MAGNIFIED
`OBJECT FROM LIST
`OF IMAGE OBJECTS
`
`CREATE OVERLAY
`MAGNIFICATION
`WINDOW FOR
`IMAGE OBJECT
`
`DISPLAY MAGNIFIED
`MAGE OBJECT IN
`WINDOW
`
`
`
`SEND AUDIO DATA
`STREAM WITH
`ADJUSTED
`VOLUME/TONE
`CONTROL TO
`AUDIO SUBSYSTEM
`
`900
`
`
`
`REDUCE
`VOLUME OF OTHER
`AUDIO?
`
`YES
`
`FIC. 1 O
`
`FIC. 1 1
`
`1000
`
`REQUESTED
`MAGNIFICATION POWER
`MATCHES AVAILABLE FONT
`FROM LIST?
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`TEXT OBJECT TO
`DISPLAY USES
`FONT FROM LIST
`
`STRETCH TEXT
`USING BEST FIT
`FONT FROM
`AVAILABLE LIST
`OF FONTS
`
`REFLOW TEXT
`OBJECT AROUND
`OVERLAY WINDOW
`
`
`
`ADJUST OVERLAY
`WINDOWSIZETO
`FIT MAGNIFIED
`TEXT OBJECT
`
`1 104
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 11 of 19
`
`U.S. Patent
`
`Mar. 9, 2004
`
`Sheet 9 of 9
`
`US 6,704,034 B1
`
`FIC. 12
`
`
`
`
`
`1200
`
`REQUESTED
`MAGNIFICATION POWER
`MATCHES AVAILABLE IMAGE
`OBJECT FROM
`
`
`
`IMAGE OBJECT TO
`DISPLAY USES
`IMAGE FROM LIST
`
`STRETCH IMAGE OBJECT
`USING BEST FIT FROM
`AVAILABLE LIST OF IMAGES
`
`1204
`
`FIC 13
`
`MAGNIFIED
`IMAGE OBJECT
`AVAILABLE2
`
`SIZE OF OVERLAY
`WINDOW ADJUSTED
`FOR IMAGE OBJECT
`
`
`
`
`
`SIZE OF OVERLAY
`WINDOW ADJUSTED FOR
`STRETCH OF IMAGE
`
`1304
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 12 of 19
`
`US 6,704,034 B1
`
`1
`METHOD AND APPARATUS FOR
`PROVIDING ACCESSIBILITY THROUGH A
`CONTEXT SENSITIVE MAGNIFYING GLASS
`
`15
`
`BACKGROUND OF THE INVENTION
`1. Technical Field
`The present invention relates generally to an improved
`data processing System and in particular, to a method and
`apparatus for presenting information in a data processing
`System. Still more particularly, the present invention pro
`vides a method and apparatus for providing a user interface
`to magnify information based on context.
`2. Description of Related Art
`An important characteristic of modern computing Systems
`is the interface between a user and the computer. Early
`interactive interfaces were text based wherein a user com
`municated with the computer by typing a Sequence of
`characters on a keyboard and the computer communicated
`with the user by displaying characters on an output device
`commonly a display Screen. These input characterS Specified
`a command to the computer's operating System or to an
`application program executing on the computer. This com
`mand invokes program logic to perform a given operation.
`25
`Modern computer Systems use a image user interface (GUI)
`to simplify the interaction between a user and a computer. A
`GUI equipped computer communicates with a user by
`displaying graphics, including text and icons, on a display
`Screen and the user communicates with the machine both by
`typing in textual information in response to dialogs and by
`manipulating the displayed icons with a pointing device,
`Such as a mouse.
`Many modern GUIs provide a window environment. In a
`typical window environment, the image display portrayed
`on the display Screen is arranged to resemble the Surface of
`an electronic “desktop' and each application program run
`ning on the computer is represented as one or more elec
`tronic "paper sheets' displayed as rectangular regions on the
`display Screen. These rectangular regions are called “win
`dows'. Each window may include a multitude of panes, with
`each pane being an area for a particular type of information
`(textual, Still image, moving image, etc.).
`Each window displayS information generated by an asso
`ciated application or System program. Further, Several win
`dows may be Simultaneously present on the desktop with
`each containing information generated by a program. A
`program presents information to the user through each
`window by drawing or "painting images, graphics or text
`within the window. The user also can move a window to a
`different location on the display Screen and change its size
`and appearance to arrange the desktop in a convenient
`manner. The user communicates with the program by “point
`ing at objects displayed in the window with a cursor
`controlled by a pointing device and manipulating the objects
`as desired. In Some cases, the program requests additional
`information from the user in response to a manipulation.
`This request is presented as a “dialog that allows the user
`to provide the requested information to the dialog from the
`keyboard.
`With these windows, information is presented in a image
`fashion. Text may be presented in different font types and
`different font sizes between windows and within the same
`window. For example, one window may represent a browser
`program used to View web pages from different Sources on
`the Internet. A web page may contain text in different fonts,
`font sizes, and colors. Comprehension of information
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`through graphic user interfaces may be difficult for users
`who have disabilities, Such as Visual or hearing disabilities.
`For example, for visually impaired users, a need is present
`to enlarge certain portions of a computer Screen, especially
`portions containing text having a relatively Small font size.
`Presently, Some tools are available for magnifying por
`tions of the Screen for a user. These tools, however, typically
`magnify a portion of the Screen without regard to the type of
`content, including the Size of the text or image. In other
`words, equal magnification is provided regardless of the font
`Size of the text. Further, the magnification is often accom
`plished using pixel amplification in which pixels are dupli
`cated to create a larger image. This process is performed
`without changing the resolution, resulting in jagged edges in
`the text. This type of magnification often does not increase
`the readability of the text being magnified or the details of
`the image.
`Therefore, it would be advantageous to have an improved
`method and apparatus for magnifying objects displayed in a
`data processing System.
`SUMMARY OF THE INVENTION
`The present invention provides a method and apparatus in
`a data processing System for presenting a Set of objects
`within the data processing System. Responsive to detecting
`movement of a pointer over an object within the Set of
`objects, an object type is identified for the object. Presen
`tation of the object is then magnified based on the object
`type.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`The novel features believed characteristic of the invention
`are set forth in the appended claims. The invention itself,
`however, as well as a preferred mode of use, further objec
`tives and advantages thereof, will best be understood by
`reference to the following detailed description of an illus
`trative embodiment when read in conjunction with the
`accompanying drawings, wherein:
`FIG. 1 is a pictorial representation of a data processing
`System in which the present invention may be implemented
`in accordance with a preferred embodiment of the present
`invention;
`FIG. 2 is a block diagram of a data processing System in
`which the present invention may be implemented;
`FIGS. 3A-3I are windows illustrating context sensitive
`magnification of objects in accordance with a preferred
`embodiment of the present invention;
`FIG. 4 is a diagram illustrating an options menu for
`Setting preferences used to magnify objects in accordance
`with a preferred embodiment of the present invention;
`FIG. 5 is a flowchart of a process used for magnifying
`objects in accordance with a preferred embodiment of the
`present invention;
`FIG. 6 is a flowchart of a proceSS used for Selecting an
`object for magnification in accordance with a preferred
`embodiment of the present invention;
`FIG. 7 is a flowchart of a process used for magnifying text
`in accordance with a preferred embodiment of the present
`invention;
`FIG. 8 is a flowchart of a process used for magnifying an
`image object in accordance with a preferred embodiment of
`the present invention;
`FIG. 9 is a flowchart of a process used for magnifying an
`audio object in accordance with a preferred embodiment of
`the present invention;
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 13 of 19
`
`US 6,704,034 B1
`
`3
`FIG. 10 is a flowchart of a process used for selecting a
`font type used to magnify a text object in accordance with
`a preferred embodiment of the present invention;
`FIG. 11 is a flowchart of a proceSS used for creating an
`overlay window in accordance with a preferred embodiment
`of the present invention;
`FIG. 12 is a flowchart of a proceSS used for Selecting a
`magnification for an image object in accordance with a
`preferred embodiment of the present invention; and
`FIG. 13 is a flowchart of a process used for creating an
`overlay window for an image object in accordance with a
`preferred embodiment of the present invention.
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENT
`With reference now to the figures and in particular with
`reference to FIG. 1, a pictorial representation of a data
`processing System in which the present invention may be
`implemented is depicted in accordance with a preferred
`embodiment of the present invention. A computer 100 is
`depicted which includes a system unit 110, a video display
`terminal 102, a keyboard 104, storage devices 108, which
`may include floppy drives and other types of permanent and
`removable Storage media, and mouse 106. Input/output
`devices, Such as Speaker 112, headphones 114, and micro
`phone 116 are present in personal computer 100. Additional
`input devices may be included with personal computer 100,
`Such as, for example, a joystick, touchpad, touch Screen,
`trackball, microphone, and the like. Computer 100 can be
`implemented using any Suitable computer, Such as an IBM
`RS/6000TM computer or IntelliStationTM computer, which
`are products of International Business Machines
`Corporation, located in Armonk, N.Y. Although the depicted
`representation shows a computer, other embodiments of the
`present invention may be implemented in other types of data
`processing Systems, Such as a network computer, a handheld
`computer, or a wireless device. Computer 100 also prefer
`ably includes a image user interface that may be imple
`mented by means of Systems Software residing in computer
`readable media in operation within computer 100.
`With reference now to FIG. 2, a block diagram of a data
`processing System is shown in which the present invention
`may be implemented. Data processing System 200 is an
`example of a computer, such as computer 100 in FIG. 1, in
`which code or instructions implementing the processes of
`the present invention may be located. Data processing
`System 200 employs a peripheral component interconnect
`(PCI) local bus architecture. Although the depicted example
`employs a PCI bus, other bus architectures Such as Accel
`erated Graphics Port (AGP) and Industry Standard Archi
`tecture (ISA) may be used. Processor 202 and main memory
`204 are connected to PCI local bus 206 through PCI bridge
`208. PCI bridge 208 also may include an integrated memory
`controller and cache memory for processor 202. Additional
`connections to PCI local bus 206 may be made through
`direct component interconnection or through add-in boards.
`In the depicted example, local area network (LAN) adapter
`210, Small computer system interface SCSI hostbus adapter
`212, and expansion bus interface 214 are connected to PCI
`local buS 206 by direct component connection. In contrast,
`audio adapter 216, graphics adapter 218, and audio/video
`adapter 219 are connected to PCI local bus 206 by add-in
`boards inserted into expansion slots. Expansion bus inter
`face 214 provides a connection for a keyboard and mouse
`adapter 220, modem 222, and additional memory 224. SCSI
`host bus adapter 212 provides a connection for hard disk
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`drive 226, tape drive 228, and CD-ROM drive 230. Typical
`PCI local bus implementations will support three or four PCI
`expansion slots or add-in connectors.
`An operating System runs on processor 202 and is used to
`coordinate and provide control of various components
`within data processing system 200 in FIG. 2. The operating
`System may be a commercially available operating System
`Such as Windows 2000TM, which is available from Microsoft
`Corporation. An object oriented programming System Such
`as Java" may run in conjunction with the operating System
`and provides calls to the operating system from JavaTM
`programs or applications executing on data processing Sys
`tem 200. Instructions for the operating system, the object
`oriented programming System, and applications or programs
`are located on Storage devices, Such as hard disk drive 226,
`and may be loaded into main memory 204 for execution by
`processor 202.
`Those of ordinary skill in the art will appreciate that the
`hardware in FIG. 2 may vary depending on the implemen
`tation. Other internal hardware or peripheral devices, Such as
`flash ROM (or equivalent nonvolatile memory) or optical
`disk drives and the like, may be used in addition to or in
`place of the hardware depicted in FIG. 2. Also, the processes
`of the present invention may be applied to a multiprocessor
`data processing System.
`For example, data processing System 200, if optionally
`configured as a network computer, may not include SCSI
`host bus adapter 212, hard disk drive 226, tape drive 228,
`and CD-ROM 230, as noted by dotted line 232 in FIG. 2
`denoting optional inclusion. In that case, the computer, to be
`properly called a client computer, may include Some type of
`network communication interface, Such as LAN adapter
`210, modem 222, or the like. AS another example, data
`processing System 200 may be a Stand-alone System con
`figured to be bootable without relying on Some type of
`network communication interface, whether or not data pro
`cessing System 200 comprises Some type of network com
`munication interface. As a further example, data processing
`system 200 may be a Personal Digital Assistant (PDA)
`device which is configured with ROM and/or flash ROM in
`order to provide non-volatile memory for Storing operating
`System files and/or user-generated data.
`The depicted example in FIG. 2 and above-described
`examples are not meant to imply architectural limitations.
`For example, data processing System 200 also may be a
`notebook computer or hand held computer in addition to
`taking the form of a PDA. Data processing system 200 also
`may be a kiosk or a Web appliance.
`The processes of the present invention are performed by
`processor 202 using computer implemented instructions,
`which may be located in a memory Such as, for example,
`main memory 204, memory 224, or in one or more periph
`eral devices 226-230.
`The present invention provides a method, apparatus, and
`computer implemented instructions for magnifying objects
`presented in a data processing System based on the context
`of the objects. In the depicted examples, an object has an
`object type, Such as, for example, a text object, an image
`object, and an audio object. Theses different object types are
`examples of different contexts for information being pre
`Sented in the data processing System. The mechanism of the
`present invention identifies an object context when the focus
`on the information being presented changes. The change in
`focus in these examples is detected through the movement of
`a pointer on the Screen. Typically, a pointer has a hotspot,
`which is a point or portion of the pointer used to Select
`objects that may be acted upon through user input.
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 14 of 19
`
`US 6,704,034 B1
`
`S
`Turning next to FIGS. 3A-3I, windows illustrating con
`text Sensitive magnification of objects are depicted in accor
`dance with a preferred embodiment of the present invention.
`In these examples, two mechanisms are illustrated for mag
`nifying objects. One mechanism involves overlays in which
`an overlay window containing the magnified object is dis
`played over the original window. A Second mechanism
`involves reflow in which the text object in the original
`window is possibly reformatted. The mechanisms for mag
`nifying objects in these examples may be implemented
`through the operating System, which will provide applica
`tion programming interfaces (APIs) or directly in an appli
`cation Such as a browser program. Since only the application
`understands the context, it must provide the magnification or
`use the APIS provided by the operating System.
`In FIG.3A, window 300 contains text 302 and image 304.
`In FIG.3B, a pointer 306 is moved over word “Tower” 308.
`The mechanism of the present invention magnifies the word
`“Tower” as illustrated by word 310. Word 310 is displayed
`in overlay window 312, which is not shown not seen by the
`user viewing window 300. The original content is still
`present, but not seen below the overlay window. Word 310
`is presented to the user using a Selected font and font Size.
`In these examples, the font size is larger than the original
`fontsize for text 302. In this example, a preference has been
`set such that an object is defined to be a word. Text 302 has
`an object type of text object in these examples. Of course,
`other Settings may be Selected in which the object is, for
`example, a group of Words, a Sentence, a line, a part of a line,
`or a paragraph. In this example, a group of words may be
`words adjacent to the Selected word or Some number of
`words based on a user preference. With respect to the
`amount of text that is Selected to be an object, this text may
`be defined in a number of ways including using text delin
`eation tags, Such as, for example, hypertext markup lan
`guage (HTML) codes in a browser.
`In FIG. 3C, word 310 in text 302 is presented in a larger
`font Size without using an overlay window Such as that
`illustrated in FIG. 3B by reformatting and reflowing the text.
`Again, the magnification of word 310 occurs when pointer
`306 is moved over word “Tower 308.
`Of course, other types of objects may be Selected depend
`ing on the implementation. Image 304 also may be an object,
`which would have an object type of an image object. For
`example, in FIG. 3D pointer 306 is over image 304, which
`is presented in a magnified or larger form. In this example,
`image 304 overlays text 302 because the text is not refor
`matted in response to the magnification of image 304. In
`FIG. 3E, text 302 is reformatted to wrap around image 304
`in response to image 304 being magnified.
`In FIG. 3F, the object is defined to be an entire sentence.
`Further, in this example, an overlay window 314 is displayed
`over the original content for text 302 and image 304.
`Optionally, overlay window 314 may be movable by the user
`to allow the user to view the magnified content as well as the
`content in its original form. In FIG. 3G, text 302 has the font
`Size for characters in Sentence 316 changed to magnify the
`presentation of Sentence 316 to a user. This magnification
`occurs whenever pointer 306 points to sentence 316. Further,
`in this example, text 302 wraps around image 304 to avoid
`obscuring image 304. Additionally, depending on the par
`ticular implementation, a user input requesting magnifica
`tion may magnify multiple objects based on the object type.
`This magnification of multiple objects may be, for example,
`based on the proximity or distance of the objects from the
`hotspot of the pointer. Each object is magnified based on its
`object type, Such as text or image. An alternate embodiment
`
`6
`may Select multiple objects for magnification through the
`use of a combination of keystrokes and mouse pointer
`location. When multiple objects are magnified
`Simultaneously, each object will have its own magnification
`window.
`In FIG.3H, the object is defined to be an entire paragraph.
`All of text 302 is placed into an overlay window 318 in this
`example. This overlay window is displayed over the original
`text and reformatted. As a result, image 304 is obscured by
`the text in overlay window 318. Next, in FIG. 3I, the text in
`the original window is reformatted and displayed. In this
`case, all of text 302 is magnified for presentation to a user.
`In the depicted examples, this magnification occurs through
`changing the font size, rather than manipulating pixels
`through a pixel amplification process.
`Turning next to FIG. 4, a diagram illustrating an options
`menu for Setting preferences used to magnify objects is
`depicted in accordance with a preferred embodiment of the
`present invention. Options menu 400 provides different
`ways to Set preferences used to magnify objects having
`different types of context. In this example, preferences may
`be set for context having object types of a text object, an
`image object, and an audio object. With respect to text
`objects, the text object may be set to be a word or a group
`of words, a Sentence, or a paragraph through Selections 402,
`404, and 406. Control 407 provides a mechanism to set the
`number of words in a group. This group may be, for
`example, one word, three words, or Seven words. Based on
`these Selections, a parser process may use the preferences to
`parse the text to obtain groupings for text objects.
`Alternatively, a process may be used to add tags to delineate
`the text objects. The magnification of the object in this
`example is set through control 408, which allows a font size
`to be selected for a text object based on a percentage
`increase. Alternatively, the increase in the size of the text
`may be set through the use of a point Size for the text. This
`allows a user to Set how large text is to be increased by
`inputting or Selecting a percent increase in size for the text.
`Based on this percentage, a different font is Selected for use
`in displaying the information in the text object. Optionally,
`a different font type may be selected in addition to the
`percent change in the size of the text using control 409. In
`particular, the user may Select a font type that is more
`readable than the current type being used.
`Next, control 410 allows input to set an image size for
`image imageS as a function of a percentage increase. This
`presentation requires that the requested bitmap be available
`for use. With respect to web pages, a Server may contain and
`Send different Size bitmaps for an image. Alternatively, other
`formats may be used to transfer images, Such as Vector
`graphics, low compression jpeg, medium compression jpeg,
`and high compression jpeg.
`Selections 422 and 424 allow for the magnification to take
`the form of an overlay or reflow with respect to text objects.
`With respect to an audio object, control 412 allows
`increasing the Volume at which an audio object is presented
`to a user. In this example, the increase is Selected as a
`percentage of the original Volume. For example, an audio
`object of a news broadcast may be increased in Volume by
`twenty percent from the normal Setting if a pointer is moved
`over and selects the audio object. Selection 414 also allows
`for other audio objects being presented at the same time to
`be decreased in Volume. The percentage decrease is con
`trolled through control 416 in this example.
`When Selections are to be accepted, then the Selection of
`“Ok” button 418 will save the options selected by the user.
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`

`

`Case 1:21-cv-01319 Document 1-4 Filed 02/15/21 Page 15 of 19
`
`US 6,704,034 B1
`
`7
`If the options are not to be saved, the selection of “Cancel”
`button 420 will prevent any changes in the options.
`Turning next to FIG. 5, a flowchart of a process used for
`magnifying objects is depicted in accordance with a pre
`ferred embodiment of the present invention. The process in
`FIG. 5 for magnifying objects may be initiated and termi
`nated using a number of different mechanisms. For example,
`a mouse button or Selection of an entry or command from a
`drop down menu may be used to initiate or terminate the
`proceSSeS.
`The process begins by waiting for pointer movement (step
`500). Next, a determination is made as to whether a new
`object is pointed to by the pointer (step 502). This determi
`nation may be made using a parsing process that identifies
`an object to magnify. For example, the process may identify
`an object to magnify based on Selecting one or more objects
`closest to the hotspot of the pointer. If a new object is
`pointed to by the pointer, the information in the new object
`is magnified based on the context of the object (step 504)
`with the process then returning to step 500. For example, if
`the context is a text object, the object is magnified by
`changing the fontsize and/or font type of the text within the
`text object. If the object is an audio object, the object may
`be magnified by increasing the Volume at which the audio
`object is presented. The volume at which other audio objects
`are being presented also may be decreased.
`With reference again to step 502, if a new object is not
`pointed to by the pointer, the process returns to step 500.
`Turning next to FIG. 6, a flowchart of a process used for
`Selecting an object for magnification is depicted in accor
`dance with a prefe