(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2001/0053978A1
`(43) Pub. Date:
`Dec. 20, 2001
`LEWIS et al.
`
`US 2001.0053978A1
`
`(54) SYSTEM AND METHOD FOR PROVIDING
`USER-DIRECTED CONSTRAINTS FOR
`HANDWRITING RECOGNITION
`
`(76) Inventors: JAMES R. LEWIS, DEL RAY
`BEACH, FL (US); MICHAEL P.
`PERRONE, YORKTOWN, NY (US);
`JOHN F. PITRELLI, DANBURY, CT
`(US); EUGENE. H. RATZLAFF,
`HOPEWELL JUNCTION, NY (US);
`JAYASHREE SUBRAHMONIA,
`WHITE PLAINS, NY (US)
`Correspondence Address:
`FRANK CHAU
`F CHAU & ASSOCATES
`1900 HEMPSTEAD TURNPIKE
`SUTE 501
`EAST MEADOW, NY 11554
`(*) Notice:
`This is a publication of a continued pros
`ecution application (CPA) filed under 37
`CFR 1.53(d).
`
`(21) Appl. No.:
`(22) Filed:
`
`09/238,288
`Jan. 28, 1999
`Publication Classification
`
`(51) Int. Cl." ..................................................... G10L 21/00
`(52) U.S. Cl. .............................................................. 7041275
`(57)
`ABSTRACT
`A data recognition System and method which allows a user
`to Select between a “default recognition” mode and a “con
`Strained recognition” mode via a user interface. In the
`default recognition mode, a recognition engine utilizes pre
`determined default recognition parameters to decode data
`(e.g., handwriting and speech). In the constrained recogni
`tion mode, the user can Select one or more of a plurality of
`recognition constraints which temporarily modify the
`default recognition parameters to decode uncharacteristic
`and/or special data. The recognition parameters associated
`with the Selected constraint enable the recognition engine to
`utilize Specific information to decode the Special data,
`thereby providing increased recognition accuracy.
`
`"N,
`
`
`
`
`
`-- /6
`
`USER
`INTERFACE
`
`| 8
`
`USER-SELECED
`RECOGNITION
`CONSTRANTS
`
`DEFAULT
`RECOGNITION
`PARAMETERS
`
`
`
`HANDWRITING
`DATA STORE
`
`RECOGNTON
`ENGINE
`
`WORD LISTS/
`GRA ri?m ARS
`
`Writer
`Prototypes
`(Models)
`
`
`
`OUTPUT DEVICE
`
`
`
`
`
`
`
`
`
`Recognition
`Constraints
`b
`2 c. 1
`User interface
`
`INPUT DEVICE
`
`
`
`
`
`
`
`
`
`
`
`
`
`GOOGLE EXHIBIT 1012
`
`Page 1 of 12
`
`

`

`Patent Application Publication Dec. 20, 2001 Sheet 1 of 4
`
`US 2001/0053978 A1
`
`
`
`
`
`USER
`INTERFACE
`
`(s
`
`USER-SELECTED
`RECOGNITION
`CONSTRAINTS
`
`DEFAULT
`RECOGNITION
`PARAMETERS
`
`
`
`
`
`
`
`2 /
`-
`
`WORD LISTS/
`GeAnna RS
`
`
`
`Writer
`Prototypes
`(Models)
`
`
`
`HANDWRITING
`DATA STORE
`
`RECOGNITION
`ENGINE
`
`Recognition
`Constraints
`2c -1
`ty b
`
`User interface
`
`INPUT DEVICE
`
`OUTPUT DEVICE
`
`w
`N- 2S
`
`FIG. 1
`
`
`
`
`
`
`
`
`
`Page 2 of 12
`
`

`

`Patent Application Publication
`
`Dec. 20, 2001 Sheet 2 of 4
`
`US 2001/0053978A1
`
`FIG. 2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`2 C. H
`
`Recognize Selected
`link in ACCordance
`With Default
`Parameters
`
`
`
`input Handwriting
`Data
`
`2 o o
`
`Select Recognition
`User interface
`
`19 c.
`
`
`
`Select link
`
`
`
`Default
`Recognition
`Mode
`Selected?
`
`Constrained
`Recognition
`Mode Selected?
`
`
`
`Select Constraint(s)
`
`More ink to Be
`Labeled?
`
`Recognition
`Deferred?
`
`Attach Constraint
`Label(s) To Selected
`Ink For Processing
`At A Later Time
`
`
`
`Recognize Selected
`ink in Accordance Wtih
`Selected Constraint(s)
`
`Output Recognition
`Results
`
`
`
`More link to Be
`
`NO
`
`2.13
`Ar
`Exit User interface
`
`A
`
`Page 3 of 12
`
`

`

`Patent Application Publication Dec. 20, 2001 Sheet 3 of 4
`
`US 2001/0053978 A1
`
`input Handwritin
`
`Select Recognition
`Usef Interface
`
`3 o
`
`Default - 2 O2
`Recognition
`Mode
`Selected?
`
`2 os
`
`No-> A
`
`No
`
`Constrained
`Recognition
`Mode Selected?
`
`Yes
`
`
`
`
`
`
`
`
`
`
`
`FIG. 3
`
`Yes
`
`- 203
`
`Select ink
`
`-
`
`~ 3 C (2
`
`2 o 2
`
`Yes
`
`No
`
`Apply Previous
`Constraint(s)?
`
`- 37.2
`
`
`
`Yes
`
`Attach Constraint
`Label(s) To Selected
`ink For Processing
`At A Later Time
`
`3 / o
`
`
`
`Recognition
`Deferred?
`
`Recognize Selected
`ink in Accordance
`With Default
`Parameters
`
`Recognize Selected
`Ink in Accordance Wtih
`Selected Constraint(s)
`
`O 7
`3
`
`
`
`
`
`Mape
`
`Yes
`
`Output Recognition
`Results
`
`f 3
`3
`
`No
`w
`A
`
`
`
`
`
`More ink to Be
`Recognized?
`
`
`
`Yes
`3 14-/
`
`3 S -
`Exit User interface
`
`No
`
`A
`
`Page 4 of 12
`
`

`

`Patent Application Publication Dec. 20, 2001 Sheet 4 of 4
`
`US 2001/0053978A1
`
`
`
`Pe.
`Select
`Recognize
`Recognize as>Printed mixed case
`Printed all caps
`Script mixed cast
`Standard type (ala Grafitti)
`
`Page 5 of 12
`
`

`

`US 2001/0053978 A1
`
`Dec. 20, 2001
`
`SYSTEMAND METHOD FOR PROVIDING
`USER-DIRECTED CONSTRAINTS FOR
`HANDWRITING RECOGNITION
`BACKGROUND
`0001) 1. Technical Field
`0002 The present application relates generally to
`machine recognition and, more particularly, to System and
`methods for providing user-directed recognition constraints,
`wherein a user interface allows a writer to Select one or more
`recognition constraints which temporarily modify default
`recognition parameters So as to decode uncharacteristic
`and/or Special data with increased accuracy.
`0003 2. Description of the Related Art
`0004.
`In general, conventional machine recognition sys
`tems recognize input data by decoding the data using a
`plurality of trained models. For instance, Speech recognition
`Systems decode input Speech utterances using trained speech
`models to recognize spoken words. Likewise, handwriting
`recognition Systems recognize input handwriting data using
`trained character models to convert the handwriting data to
`machine printed text.
`0005 One example of a handwriting recognition system
`is the CROSSPADTM personal digital notepad (PDN) device
`by the A. T. Cross Company) which is sold with Interna
`tional Business Machines Corporation's INK MAN
`AGER" handwriting text recognition Software package.
`The CROSSPAD device allows a user to produce handwrit
`ten records on a standard paper notepad (which is placed
`over a digitizing tablet) using an electronic inking stylus
`(i.e., pen). A data stream representing the handwriting
`Strokes is generated by the digitizing tablet (simultaneously
`with the inking) by detecting RF emissions from the Stylus,
`thereby capturing and Storing an electronic carbon copy of
`the handwritten record. The handwriting data that is elec
`tronically recorded by the CROSSPAD device can be sub
`Sequently uploaded to a computer and processed by the INK
`MANAGER system which is stored in the computer.
`0006 Typically, machine recognition of input data (e.g.,
`speech or handwriting) is performed using predetermined
`default recognition parameters which are automatically
`applied by the recognition engine for decoding the input
`data. These default recognition parameters may be those
`default parameters which are originally programmed in the
`System (i.e., "out-of-the-box” default recognition param
`eters). In addition, the default recognition parameters may
`also comprise user-Selected default recognition parameters
`(i.e., user preferences) which modify and/or replace the
`original out-of-the-box default parameters. For instance,
`handwriting recognition systems such as the INK MAN
`AGER system allow the user to initially indicate, for
`example, the user's normal style of writing (e.g., a user
`preference) prior to using the System. In particular, before
`using the INK MANAGER system, the user will be
`prompted to Select (if the user desires) one of the following
`Writing Style preferences: pure discrete, pure upper case, and
`cursive (which includes all other handwriting Such as print
`ing and script mixed). During recognition, the INK MAN
`AGER System will automatically apply any default param
`eters associated with a previously Selected user preference.
`The INK MANAGER system includes other default settings
`to determine which, if any, word lists will be used for
`decoding handwriting.
`
`0007. In most situations, utilizing the default recognition
`parameters will result in optimal data recognition. In certain
`circumstances, however, utilizing the default recognition
`parameters may result in Sub-optimal recognition accuracy.
`For example, with regard to handwriting recognition, Such
`Special circumstances occur when a user writes in an unchar
`acteristic manner (e.g., when a user writes an address using
`all capital letters rather than the user's typical Style of
`writing addresses with printed mixed case). Decoding Such
`uncharacteristic handwriting data using default recognition
`parameters can result in poor recognition accuracy.
`0008. There is a need, therefore, for a system and meth
`ods for use with machine recognition that allow a user to
`temporarily modify default recognition parameters by
`Selecting one or more recognition constraints to decode
`uncharacteristic and/or special data So as to obtain increased
`recognition accuracy.
`
`SUMMARY
`0009. The present application is directed to system and
`methods which can be implemented in a machine recogni
`tion device to provide a user (via a user interface) the option
`of Selecting one of a plurality of recognition modes includ
`ing a “default recognition' mode and a “constrained recog
`nition” mode. In the default recognition mode, the recogni
`tion engine utilizes predetermined default recognition
`parameters (e.g., out-of-the-box and/or user-Selected default
`parameters) to decode Selected data. In the constrained
`recognition mode, the user can Select one or more of a
`plurality of recognition constraints which are temporarily
`applied by a recognition engine for decoding uncharacter
`istic and/or Special data. When the user Selects a particular
`recognition constraint, default recognition parameters are
`temporarily modified to reflect the selected constraint. The
`recognition parameters associated with the Selected con
`Straint enable the recognition engine to temporarily utilize
`Specific information for decoding the Special data, thereby
`providing increased recognition accuracy.
`0010. These and other features and advantages of the
`present System and method will become apparent from the
`following detailed description of preferred embodiments,
`which is to be read in connection with the accompanying
`drawings.
`BRIEF DESCRIPTION OF THE DRAWINGS
`0011 FIG. 1 is a block diagram of a handwriting recog
`nition System which provides user-Selected recognition con
`Straints according to an embodiment of the present inven
`tion;
`0012 FIG. 2 is a flow diagram of a method for selecting
`and applying recognition constraints in accordance with one
`aspect of the present invention;
`0013 FIG. 3 is a flow diagram of a method for selecting
`and applying recognition constraints in accordance with
`another aspect of the present invention;
`0014 FIGS. 4a and 4b are diagrams of exemplary cas
`cading menus for providing handwriting recognition con
`Straints in accordance with one aspect the present invention;
`and
`0015 FIG. 5 is a diagram of an exemplary dialog for
`Selecting recognition constraints in accordance with one
`aspect of the present invention.
`
`Page 6 of 12
`
`

`

`US 2001/0053978 A1
`
`Dec. 20, 2001
`
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS
`0016. It is to be understood that, notwithstanding that the
`illustrative embodiments herein are directed to handwriting
`recognition, the present invention may be implemented in
`other machine recognition applications Such as Speech rec
`ognition.
`0017 Referring now to FIG. 1, a block diagram illus
`trates a System for providing user-Selected recognition con
`Straints for handwriting recognition in accordance with an
`embodiment of the present invention. The system 10
`includes an input device 12 (or “data collection device') for
`collecting data, e.g., handwritten text. The input device 12
`can be any conventional device Such as a digitizing tablet
`(i.e., pen-based computer) for real-time digitization and
`recognition of text which is directly written on the tablet, or
`an OCR (optical character recognition) Scanner for inputting
`handwritten text. A memory 14 is included for Storing
`handwriting data (hereinafter, alternatively referred to as
`“ink”) which is input via the input device 12 for “non-real
`time’ processing. The ink may be Stored in any Suitable
`conventional format Such as a bitmap image (which is
`typically generated with “off-line’ OCR scanning, where
`“off-line” refers to recognition of handwriting that has
`already been written on paper using, e.g., a pencil or pen) or
`as a Sequence of X-Y coordinates which represent the
`location of a pen writing tip on the input device 12 at
`Successive points in time (which is generated with an
`“on-line” pen-based computer, where “on-line” refers to
`recognition of handwriting written with an electronic Stylus
`onto a tablet or pen computer). It is to be understood that
`memory 14 may also be memory (Such as flash memory)
`included within the input device 12 for storing the ink prior
`to being uploaded to, and decoded by, a handwriting recog
`nition engine 22. It is to be appreciated that “real-time”
`recognition may also be performed by decoding handwriting
`data received by input device 12 as it is being collected by
`the input device 12.
`0018. The system 10 further includes a user interface 16
`which may be, for example, a computer display having a
`Suitable GUI (graphic user interface) which allows a user to
`perform handwriting recognition in accordance with the
`present invention. The user interface 16 provides the user
`with the option of Selecting a “default recognition” mode,
`whereby predetermined default recognition parameters 18
`are applied to the recognition engine 22. The user interface
`16 also provides the user the option of Selecting a “con
`Strained recognition” mode, whereby the user can Select one
`or more available recognition constraints 20 that are tem
`porarily applied to the recognition engine 22. Specifically,
`the recognition constraints 20 present the user with task
`oriented choices, the Selection of which determines the
`recognition parameters that are temporarily utilized by the
`recognition engine 22 for decoding uncharacteristic and/or
`Special text.
`0019. A word list/grammar store 24 contains a plurality
`of word lists and associated grammars which may be applied
`to the recognition engine 22 based on the Selected recogni
`tion mode. Each word list may contain, for example, words
`or numbers that are associated with a particular application
`(e.g., for each of the available recognition modes), or even
`for particular fields in a data-entry tool. By way of example,
`
`a word list for telephone numbers may contain all the
`numeric characters, as well as hyphen and parentheses
`characters. In addition, a word list for recognizing dates may
`contain years, months and days both numerically and in
`word form. A grammar is a constraint which may be applied
`in connection with a word list. By way of example, a
`grammar associated with a “date' word list may be
`employed to limit the order in which each data field of a date
`(e.g., month-day-year) can be recognized. In particular,
`when recognizing a handwritten date, the first field “month”
`of the grammar would constrain the universe of available
`words in the word list to those pertaining only to months, the
`second field “day' of the grammar would constrain the
`universe of words in the word list to days, and, likewise, the
`third field “year' of the grammar would constrain the
`universe of allowable words in the word list solely to years.
`0020. A writer prototypes store 26 contains one or more
`writer prototypes (models). Each writer prototype contains
`one or more sets of character models (e.g., upper case
`character model, lower case character model). It is to be
`understood that the system of FIG. 1 presumes a trained
`handwriting recognition System. The trained recognition
`System may be a writer-independent System (i.e., a System
`which is trained to recognize the handwriting of many
`different writers), in which case the writer prototypes store
`may contain a writer-independent prototype (model). Alter
`natively, the recognition System may be a writer-dependent
`recognition System (i.e., a System trained to recognize the
`handwriting of a particular writer), in which case the writer
`prototypes Store 26 may contain one or more writer-depen
`dent prototypes, as well as a writer-independent prototype.
`0021. The recognition engine 22 will decode handwriting
`data using recognition parameters associated with the
`Selected recognition mode. For example, if a user Selects the
`default recognition mode to decode the user's handwriting,
`predetermined default recognition parameters (which can be
`out-of-the-box or user-specific default constraints (user pref
`erences) as discussed above) are temporarily applied to the
`recognition engine. For instance assume the user's default
`constraints (e.g., user preference) indicate that the user
`writes in cursive and that the writer-dependent prototype of
`the user is trained on cursive data. In the default recognition
`mode, the recognition engine 22 will decode the user's
`handwriting data using the writer-dependent prototype 26
`asSociated with the user, as well as the relevant default word
`lists/grammarS 24, regardless of whether the handwriting to
`be decoded includes upper case letters or numbers. This may
`result in decreased recognition accuracy.
`0022. On the other hand, if the user selects the “con
`Strained recognition” mode, the recognition constraint(s)
`selected by the user will temporarily modify the predeter
`mined default recognition parameters utilized by the recog
`nition engine 22 and/or cause one or more relevant word
`lists/grammars to be retrieved from the word list/grammar
`Store 24. For example, one type of constraint may cause the
`recognition engine 22 to temporarily utilize a particular
`writer-prototype 26 or use a particular character Set (e.g.,
`upper case character model) of a given writer prototype
`("model constraint”). For instance, assume in the above
`example that the user (whose user preference indicates
`cursive writing) has written in upper case. The user can
`override his/her user preference by Selecting an “upper case”
`model constraint. In this situation, assuming the user's
`
`Page 7 of 12
`
`

`

`US 2001/0053978 A1
`
`Dec. 20, 2001
`
`writer-dependent prototype does not include an upper case
`character model, the Selected “upper case” constraint may
`cause the recognition engine 22 to “Swap out the writer
`dependent prototype for the writer-independent prototype
`and apply an upper case model included within the writer
`independent prototype to decode the Special text.
`0023. Another type of constraint may cause the recogni
`tion engine 22 to temporarily utilize a specific word list/
`grammar 24 during decoding (“word list constraint”). For
`instance, assume further (in the above example) that the
`handwriting data of the user contains one or more written
`dates. A word list constraint may be Selected for decoding
`the written dates, whereby the recognition engine 22 applies
`the “dates' word list So as to constrain the universe of
`allowed words to the words contained in Such word list. In
`this example, the Selected word list constraint causes the
`writer's user preference to be adjusted (constrained) and not
`overridden.
`0024. Another type of constraint may be temporarily
`applied to limit the types of characters which may be
`recognized (“character Set constraint”). For instance, if
`certain characters for a particular field or word type can only
`be, for example, letters “a”“b” or “c” or numbers “1”, “2” or
`“3,” a character set constraint may be selected to limit the
`characters which may be recognized for the field or word
`type.
`0.025. Another type of recognition constraint may cause
`the recognition engine 22 to temporarily modify or adjust
`the default decoding algorithm and/or a specific parameter
`of the decoding process ("decoding algorithm constraint”).
`For instance, if the handwriting data contains a plurality
`closely written words, a constraint may be selected to
`modify the process by which the recognition engine 22
`detects Spaces, and thereby differentiates, between different
`words. Advantageously, the recognition constraints allow
`the recognition engine 22 to achieve higher decoding accu
`racy when recognizing handwriting data comprising Special
`and/or uncharacteristic text. The decoding results of the
`System are output via the output device 28 (which may be a
`computer monitor or printing device).
`0.026
`Referring now to FIGS. 4a, 4b and 5, diagrams
`illustrate Several examples of user interfaces which may be
`utilized for handwriting recognition in accordance with the
`present invention. In particular, FIG. 4a illustrates one
`embodiment of a GUI comprising a cascading menu for
`Selecting one or more available recognition constraints in the
`handwriting recognition System of the present invention. A
`recognition menu option (e.g., "Ink") may be selected from
`a menu bar (or toolbar) to provide recognition functions.
`Such functions include, for instance, a menu function for
`allowing a user to mark the desired text to be recognized
`(e.g., “Select”), a menu function (e.g., “Recognize’) for
`Selecting the default recognition mode, and a menu function
`(e.g., “Recognize as”) for Selecting the constrained recog
`nition mode, which cascades to a set of recognition con
`Straints (e.g., “printed mixed case,”“printed all caps, etc. as
`shown in FIG. 4a).
`0027. Another embodiment for the GUI interface is illus
`trated in FIG. 4b., which is similar to the GUI of FIG. 4a
`except that rather than displaying the recognition constraint
`menu items as a second pull down (i.e., cascade), the
`recognition options (e.g., “Recognize Dates') are displayed
`
`at the same level in the menu hierarchy as the default
`recognition mode (e.g., “Recognize”). menu item. A further
`embodiment of the GUI in accordance with the present
`invention is illustrated in FIG. 5, wherein dialog box is
`produced (in response to selecting the constrained recogni
`tion mode) which allows the user to make easily understood
`choices. Advantageously, the GUI utilized in accordance
`with the present invention allows even a computer-naive
`user to easily find and Select one or more of the available
`recognition constraints.
`0028. It is to be appreciated that the selection of desired
`recognition constraints may also be implemented via the
`input device 12. For example, with the CROSSPAD/INK
`MANAGER system described above, rather than decoding
`all the uploaded handwriting data at once, the user can mark
`(or Select highlight) specific handwritten text using the
`CROSSPAD device (as well as the INK MANAGER sys
`tem), and then subsequently direct the INK MANAGER
`System to recognize the marked text. Specifically, as shown
`in FIG. 1, the input device 12 (e.g., CROSSPAD) contains
`a user interface 12b which allows a user to mark (or Select)
`certain ink and “tag” (or label) Selected recognition con
`straints 12a to the marked ink. The user interface 12b of the
`input device 12 may comprise, for example, an LCD display
`which displays 2 to 3 lines of text to indicate the state of the
`device or provide menu options for performing various
`functions Such as Selecting ink and labelling the Selected ink
`with desired recognition constraints. Advantageously, the
`user can “tag” (label) the Selected ink with desired recog
`nition constraint, Store the "tagged” ink in memory 14, and
`upload the "tagged” ink to the recognition engine 22 for
`processing at a later time.
`0029. Another example for illustrating user-selected rec
`ognition constraints will now be discussed with reference to
`FIG. 5. Assume the user has written down a set of part
`numbers for an inventory and wants them to be recognized.
`ASSume further that, following the user's typical convention,
`the user has written the alphabetic portions of the part
`numbers using uppercase print. This special case can be
`addressed by adding, for example, a “Part numbers' menu
`item as one of the "text Style” options shown in the dialog
`user interface of FIG. 5. In this case, the selected handwrit
`ing would be decoded using the writer-independent models
`for interpreting text printed as all caps, the System may
`retrieve the word list 24 which corresponds the target text
`being identified as part numbers. Furthermore, additional
`“text style” options (such as “Address” and “Normal text”)
`would allow a user to apply the available “handwriting
`styles” menu items (shown for example in FIG. 5) so as to
`realize a more Specific task-oriented recognition of particu
`lar handwriting.
`0030. It is to be understood that the various recognition
`constraints illustrated in the GUI interfaces of FIGS. 4a, 4b
`and 5 are not meant to exhaustively list all possible hand
`Writing Styles and text types, nor all the possible classes of
`constraints that may be available for Selection by a user. For
`example, textstyles could be extended to include other types
`of text, Such as numbers-only or Symbols. The recognition
`constraints may include other types of constraints Such as
`“left-handed” and “right-handed” constraints, as well as
`"paper ruling constraints to provide information regarding
`the vertical spacing between lines on a piece of paper (which
`is used in conjunction with the CROSSPAD discussed
`
`Page 8 of 12
`
`

`

`US 2001/0053978 A1
`
`Dec. 20, 2001
`
`above). Other type of constraint categories, Such as Field
`Description, could include items. Such as dates, URLS, math,
`e-mail, addresses, Social Security numbers, phone numbers,
`Zip codes, State abbreviations and numerous other possibili
`ties that one of skill in the art may envision. Since the
`universe of potential recognition constraints that may be
`utilized is virtually limitless (except for practical consider
`ations), the term “recognition constraints' as used herein
`refers, collectively, to the universe of all possible con
`Straints. In particular, the term “recognition constraint’ as
`used herein refers to any generic type of handwriting that has
`Some feature which distinguishes it from other types of
`Writing.
`0031. It is to be further understood that the system and
`methods described herein may be implemented in various
`forms of hardware, Software, firmware, or a combination
`thereof. In particular, the present invention is preferably
`implemented in Software as an application program which is
`executed on a general purpose computer having any Suitable
`and preferred microprocessor architecture. It is to be further
`understood that because Some of the elements of the present
`System are preferably implemented as Software modules
`(Such as the recognition engine 22) the actual connections
`shown in FIG. 1 may differ depending upon the manner in
`which the System is programmed. Of course, Special purpose
`microprocessors may be employed to implement the present
`System. Given the teachings herein, one of ordinary skill in
`the related art will be able to contemplate these and similar
`implementations or configurations of the present invention.
`0.032
`Preferably, the system and methods described
`herein are implemented on a computer platform including
`hardware Such as one or more central processing units
`(CPU), a random access memory (RAM), and input/output
`(I/O) interface(s). The computer platform also includes an
`operating System and microinstruction code. The various
`processes and functions described herein may be either part
`of the microinstruction code or application programs which
`are executed via the operating System. In addition, various
`other peripheral devices may be connected to the computer
`platform Such as an additional data Storage device and a
`printing device.
`0033. Various methods of operation of the system shown
`in FIG. 1 will now be discussed in detail with reference to
`the flow diagrams of FIGS. 2 and 3. In particular, the flow
`diagram of FIG. 2, illustrates a “one shot' method for
`Selecting recognition modes, wherein Selected recognition
`constraints are applied to marked ink only once and then
`reset, thereby allowing the user to Select different recogni
`tion constraints for Subsequently marked text. On the other
`hand, the method illustrated in FIG. 3 implements a “sticky
`constraint', wherein a Selected recognition constraint is Set
`and applied to all marked ink until the user Selects another
`recognition constraint.
`0034) Referring now to FIG. 2, initially, the handwriting
`data will be collected by, and uploaded from, the input
`device 12 (step 200). Next, the user will select the “recog
`nition' user interface when the user desires to perform
`machine recognition of the uploaded handwriting data (Step
`201). For example, as illustrated in FIGS. 4a and 4b, the
`user would select the “Ink” menu (or toolbar button) to
`provide the initial recognition options. Next, the user will
`Select (or mark) the ink to be recognized (step 202) via the
`
`“Select” option shown in FIGS. 4a and 4b, for example, or
`any other Suitable menu option which allows a user to Select
`the text to be recognized.
`0035) Next, the user will select either the “default rec
`ognition” mode (step 203) or the “constrained recognition”
`mode (step 205) from the displayed recognition menus. If
`the “default recognition” mode is selected (affirmative result
`in step 203) (via the “Recognize” menu item shown in
`FIGS. 4a and 4b, for example), then the recognition engine
`22 (FIG. 1) will decode the selected ink using the default
`recognition parameters 26 (step 204). After default recog
`nition is finished, the recognition results are output (Step
`211).
`0036. On the other hand, the user can select the “con
`strained recognition” mode (affirmative result in step 205)
`(via the “Recognize as” menu item shown in FIGS. 4a and
`4b, for example). The user will then be presented with a list
`of available “recognition constraints' and the user will Select
`one or more desired constraints (step 206). After the user
`Selects the desired constraint(s), a determination is made as
`to whether recognition of the Selected ink is deferred (Step
`207). In particular, the system provides the user the option
`(e.g., via a menu or tool bar item or default Setting) of
`Selecting between having the Selected ink recognized imme
`diately (step 208) or having recognition deferred to a later
`time (step 209).
`0037) If recognition is not deferred (negative determina
`tion is step 207), the selected ink will be recognized in
`accordance with the selected constraints (step 208) and the
`recognition results will be output (step 211). If the user
`wants additional ink recognized (affirmative result in Step
`212), the user will select the desired ink (return to step 202)
`and the recognition proceSS may be repeated for the newly
`selected ink (steps 203, 204 and 211 or steps 205, 206, 208,
`and 211).
`0038. On the other hand, if recognition is deferred (affir
`mative determination in step 207), the selected ink will be
`tagged with one or more labels associated with the Selected
`constraint(s) that are to be applied during Subsequent rec
`ognition (step 209). If the user wants to label additional ink
`for Subsequent recognition (affirmative result in Step 210),
`the user will select the desired ink (return to step 202) and
`the labelling proceSS will be repeated for the newly Selected
`ink (steps 205, 206, and 209). Accordingly, in the “one shot”
`method of FIG. 2, after the selected ink is either labelled
`(step 209) or recognized (step 208), since the selected
`recognition constraints are applied to marked ink only once
`and then reset, the user must actively Select a desired
`recognition mode for Subsequently marked ink.
`0039) Referring now to FIG. 3, a method for selecting
`recognition modes in accordance with another aspect of the
`present invention is shown. Initially, the handwriting data
`will be collected by, and uploaded from, the input device 12
`(step 300). Next, the user will select the “recognition” user
`interface when the user desires to perform machine recog
`nition of the uploaded ink (step 301). Next, the user will
`select either the “default recognition” mode (step 302) or the
`“constrained recognition” mode (step 305) from the dis
`played recognition menus. If the “default recognition” mode
`is selected (affirmative result in step 302), the user will select
`(or mark) the ink to be recognized (step 303), and the
`recognition engine 22 will decode the Selected ink using the
`
`Page 9 of 12
`
`

`

`US 2001/0053978 A1
`
`Dec. 20, 2001
`
`default recognition parameters 26 (step 304). After default
`recognition is finished, the recognition results are output
`(step