United States Patent [19J
`Boulton et al.
`
`I lllll llllllll Ill lllll lllll lllll 1111111111111111111111111111111 11111111 1111
`US005566291A
`[lll Patent Number:
`[45] Date of Patent:
`
`5,566,291
`Oct. 15, 1996
`
`[54] METHOD AND APPARATUS FOR
`IMPLEMENTING USER FEEDBACK
`
`[75]
`
`Inventors: David A. Boulton, Apros; Wayne
`Vucenic, Cupertino; John P. Stallings,
`Campbell, all of Calif.
`
`[73] Assignee: Diacom Technologies, Inc., Scotts
`Valley, Calif.
`
`[21] Appl. No.: 466,463
`
`[22] Filed:
`
`Jun. 6, 1995
`
`Related U.S. Application Data
`
`[60] Division of Ser. No. 362,801, Dec. 22, 1994, which is a
`continuation-in-part of Ser. No. 173,962, Dec. 23, 1993,
`abandoned.
`........................................................ G06F 3/14
`Int. Cl.6
`[51]
`[52] U.S. Cl . .......................... 395/161; 395/159; 3951155;
`434/118
`[58] Field of Search ..................................... 395/161, 159,
`3951155, 156, 153, 157; 434/118, 322,
`323, 350
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`4,503,499
`4,809,170
`4,845,665
`4,860,203
`4,866,635
`4,939,689
`5,008,853
`5,047,918
`5,047,977
`5,086,393
`5,175,814
`
`3/1985 Mason et al ........................ 395/100 X
`2/1989 Leblang et al. ......................... 395nOO
`7/1989 Heath et al ............................. 3951157
`811989 Corrigan et al. ........................ 395noo
`9/1989 Kahn et al ................................ 395/51
`711990 Davis et al ............................. 395/600
`4/1991 Bly et al ................................. 3951153
`9/1991 Schwartz et al. ....................... 395/600
`911991 Hill et al ............................. 395/600 X
`2/1992 Kerr et al ........................... 434/118 X
`12/1992 Anick et al. ............................ 3951161
`
`5,202,828
`5,220,675
`5,231,578
`5,237,678
`5,261,093
`5,363,507
`5,433,615
`
`4/1993 Vertelney et al. . ................. 3951159 X
`6/1993 Padawer et al ..................... 3951155 X
`7/1993 Levin et al .......................... 364/419.1
`8/1993 Kuechler et al. ....................... 395/600
`11/1993 Asmuth ................................... 395/600
`11/1994 Nakayama et al .................. 395/153 X
`711995 Clark ....................................... 434/322
`
`Primary Examiner-Raymond J. Bayerl
`Attorney, Agent, or Firm-Hickman Beyer & Weaver
`[57]
`ABSTRACT
`
`A method and apparatus for implementing user feedback. A
`user may activate an enter feedback mode command in a
`computer environment to provide feedback in a feedback
`interface. A feedback record is created and the user's context
`within a product, process, service, or issue to which the
`feedback refers is recorded in the feedback record. Attribute
`icons that the user may select to identify the nature of the
`feedback are displayed in the feedback interface. Selected
`attributes, the time at which the feedback is made, the
`physical location and identity of the user, and comments by
`the user are recorded in the feedback record. A feedback
`visualizer for a reviewer for organizing and presenting user
`feedback receives feedback from users, the feedback being
`collected and stored in a database. The visualizer identifies
`a reviewer's visualization preferences, which include indi(cid:173)
`cations of feedback attributes that the reviewer desires to
`review. Collected feedback records are then filtered in
`accordance with the visualization preferences to create a
`combined set of records that include the records having the
`preferred attributes. The visualizer graphically displays an
`indication of the number of records in the combined set. A
`reviewer can change his visualization preferences, and the
`feedback records are refiltered and displayed in accordance
`with the changed preferences. The reviewer can also select
`a specific feedback record to display message content stored
`in the record and respond to the feedback.
`
`40 Claims, 34 Drawing Sheets
`
`,.,-42
`jf
`
`AXR-1006-Page 1 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 1of34
`
`5,566,291
`
`) "
`
`"'""
`C\I
`
`co
`C\I
`
`\. Do
`\ \ 0 G~
`1
`
`a.. I W a: <( ...J
`
`Cil ::> Cl)
`
`0
`Cf')
`
`0
`C\I
`
`Cl a: <(
`0
`Cil >-
`w
`~
`
`0
`
`C\I
`Cf')
`
`=>
`a_
`(.)
`
`O>
`C\I
`
`a.. W a: -
`
`0 ,...
`~
`
`,...
`C\I
`
`~ ,...
`
`co
`,...
`
`~ w ~o a:>-
`
`Cil ::> Cl)
`
`>-
`a:
`0
`~
`w
`~
`
`co
`,...
`
`Cl)
`...J
`>- Cl <(
`a: w a:
`0 a.. w
`~a.. I
`w <(a..
`~ ~ a:
`w
`a..
`
`AXR-1006-Page 2 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 2 of 34
`
`5,566,291
`
`48·"
`r __ J _________ _
`l
`i
`!
`
`42
`
`,-40
`
`44
`
`FEEDBACK
`STORAGE
`BLOCK
`
`!figure 2
`
`I
`l
`l
`l
`l
`
`L _ £t§Y~YiJ_E_R_! _AyJ_t19 £' __ :
`
`I
`1
`
`24---
`
`~52
`
`!figure3a
`
`AXR-1006-Page 3 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 3 of 34
`
`5,566,291
`
`64
`
`HEPRIH
`I NSTRllHT I ON
`lilOSSRH'I
`
`65
`
`1 I
`
`· ········:••··r••.\
`
`.. ·.····•··········•·•:cr·.•··· }'•···.• .. •·.,•·•······.!
`
`[ACCEssrj '
`.:'>:•· .-,,.y..- ,;.
`:Jigure36
`
`65
`
`67
`
`,.......
`(
`52
`
`:Jigure3c
`
`AXR-1006-Page 4 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 4 of 34
`
`5,566,291
`
`52
`
`r:111seout te:St procedures for
`ti 1uu1log testing of electronic
`ii phone systems are completed
`<I bg a main tecbnichut., It is
`j ti the technidnn5 job to
`.>I complete thr-ee main steps in
`I the closeout procedure .
`
`. ·.i
`
`58
`
`72
`
`!figure3J
`
`82
`
`56
`
`!figure 4
`
`AXR-1006-Page 5 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 5 of 34
`
`5,566,291
`
`76
`
`:.Figure 5a
`
`:.Figure 5 b
`
`AXR-1006-Page 6 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 6 of 34
`
`5,566,291
`
`y-42
`
`76
`
`:Figure 5c
`
`,r-42
`
`i2i ·
`
`123 ·
`
`1 TESTING TIME UNCLEJIR
`#2 PRRftMETERS UNCLEAR
`#3 l.UHO IS RESPONSIBLE:·?
`4 I.UHRI DO I DO fffTER TESTI Ntr?
`1 STEPS NOT CLEAR
`2 HELP MENUS NOT USEFUL
`
`120
`
`:Figure 6
`
`AXR-1006-Page 7 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 7 of 34
`
`5,566,291
`
`81
`
`y-42
`
`76
`92
`
`Loc:al prcn:::edures aren't
`clear as to when we
`· need to perform closeout ~
`tests. Should they be done
`I on lnstaHaUon only! repair
`I only, or both?
`
`l
`
`'figure 7
`
`AXR-1006-Page 8 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 8 of 34
`
`5,566,291
`
`132
`
`146
`As ls the case vdh mostof cur knnwledge of ltt!ence many people contributed
`
`to our uncle .. r. RJ .. iif of g:ravfty, '\Vlth gravity fou:r 1moph! 1n partienL:n· st.and
`
`Ne>11tun ard Ehlstdn.
`
`i:mt.Artrt.::ds, ·
`145"-
`Arlstotle, a g:reek phHosephermdentist thought that everything ''hil<W a. place•.
`For el!:a:rni;!le~rocks ci!i:m't naturally fly throtr€h the air, they ilrn found ::;n tlw
`e Hwught that tht' tea.sun he;,.wy cbjeds faH to the grm.md was
`grn;:mcl.
`simply that tlmy ooionged there,
`In tho mythology of science, §.alfam {~lso known ,for tJ:e :siesrnpe} b said y:i
`have stno::t atop the Leaning i ower or Pisa .in Ita:yattd ctroppetl oo;octs ot
`diffe:rent weights whlle formulating h\~ law$' of faJHng ~4\es, These laws
`stated that except for th.e res!stan~ cl the air all bodies would faj) 1P the
`ground at the .same rate.
`
`It wa:s lsaae Ne%•Wn some: SD years after Gaii!oo who cxttinded tho laws of
`botli!!s Into hJgfamousthree l:tw; cl motion and shewed
`ts fall at il ronstJn1t
`He described gr.!!.vity as the
`fo~ that.binds the uni'!erse
`mutual attraction.
`
`Later Albert :Eb~n ;11tltled to our understanding eyf gr,a;,r\ty by s."ying ;;hat
`the reasert things were attrav.ed. to each other ls that every Phj11et warps t,~e
`space amund itcauslng a a:m.e ltke i11cl.et1t.atlon wbich ot.0.er dajocts fa!i into,
`
`:Figure 8a
`
`k is the case with most of OIIr knowledge: of science many people con.trfumed
`to,otrr ~d~. rstand:J.nr ofgr~vity. '!$;fi;11 gr~vity. fo.ur people iP particular stand
`ot.t.: MiS"J:Jde,
`New wn ano. El.nstmn. 150
`fi .. dsun:l.e, a gniek phlloscyher/sdentist. tJ1ot1g.1-it
`For example; rocks dc:n 't naturally fly t:·
`grncrnd. Ifo tJmug;ht thatthe ftl.ason hitavy
`simply that they oolonged there,.··~···~---~--'-------J""-----
`
`~ the mytho!ogy of s:ciern::e, GI ti' tt,, ~l Z
`
`"""'ve
`
`.aning 1
`•
`·mulat.i."lghislaws:nffallihgbodles. Theselaws
`stated thatexceot for the resistanci'l cf the air at bc<lies w01.tld fall tu J.ht
`gn:mnd attht>simerata.
`
`1
`I
`l
`l
`!twas Isaac N ewtor1 some 8(.J yearn aftar Garnoo whci ext1.mrled the laws cf
`fa· l·H·ng ~ ... '$11$ Jn. whw famous t.'1re.e Jawi; cf motion a. nci. showed. that the rearon.1'
`l!uch ci:)Jl!i:'.'.ts fall at a C1'.lnstant ;atg
`·
`, Re descriOOd gravity i.!.S d:m
`i ·
`mutual attraction.
`force th.at.binds the 1.mbarse toge<
`
`Later Alhe:rt Ehtsteln added Wat.tr t.tnclerstar<liing df gravity by .saying \hat
`the r-e>it:son things were attract.ad to ea:::h other h; thateveryobjec:t warps the
`space around !tcausi.•1g a cone like Jndentatlcn which ntherooject~ fo.U J.ntn.
`
`136
`
`140
`
`142
`144
`
`-138
`
`134
`
`138
`
`144
`
`AXR-1006-Page 9 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 9 of 34
`
`5,566,291
`
`132
`
`/
`f
`• & it the c'1!se wit:h mastcfour kncrwledgt! of science n1any µoople enntr-foutetl ~u
`· to our tmderstandirt ·of gravity. With gravity four poople in particular tot.1ntl >
`~i~
`Newton and Einstein,
`
`• lffisttitle, a gnmk philm>ephiit!tderttist thought thatev11rything 'has a µlat.'>il',
`· Fer e:irn:mple., rocks dent :rwturaHy fly farough the ak, they are fm:L'lci on the
`graund. Ee tb:n.rght th.flt tlm !'f!<t!inn hflavy (Jhject:s faU to d1egrm:.md '>Yas
`. simpliy that they ool.cnged t..hr<ro.
`
`• In dw myd11olpgy of science, Gahlno (a!ro kncvm for th!! ml~pe) ts said tc
`have stncd atm:; the Ledjthtg Tttwe1· of Fisa in ltah and drnpP">d cliiect;; d
`·different weights whihi furmttlating hls !aws cf f ;11L"1g hxilru;. Th~@ hi.vn;
`·stated that except. for the rtIBista.ncn cf the air ail !::ociies wm . .dd fol! tc the
`ground at the same rate,
`
`:.Figure 9
`
`• ;;&;~
`
`As fa the case with mcsto.f our kno-wledg0 cf ~mnc1.1many petJpk ccntrlhuted
`
`to our ~"lt:l:e:rs:;acr cf gravity. Vl~th gravity four people iri particular st.>tmJ
`N ewwn and Einsrem.
`cut: t.>nstntle, ·
`145-..J
`ilxistntle, a gr~k phUnsopherisdentirt thought that everything 'has a place'
`For cxampk, rucks dtm'ttmturaily fly thrnugh the air, they a.re found on the
`grrn.md. Ee thcru:ght that the nJ.ason lwavy l<bjm:ts faJJ to the grnund was
`simply that they bebngccl friern.
`
`Jn 1hs m}rfudotiY ef scieit:::e, Ga!Heo {aJml known for tne telMCope) is $.<::titl tD
`have st::md atop the .Leaning Tower cf Pisa in Italy and cl:rcpped abjects of
`diff"!!nmt weights: whil*1: formulating hls law~:of falling botllet. These la\~~s
`eU!tetl that exi:::eptfor· t.lm re:sistance of the a.tr aH bod\e5 wou.ld fall to the
`gr-cu:nd at thu same rate,
`
`I What did Ga!i!eo flnd with a
`telescope?
`
`:.Figure 10
`
`·::"{ II
`II
`II
`+
`
`+ -
`
`,c-130
`
`&a
`fl;
`II
`fl
`
`" +
`+ -
`
`AXR-1006-Page 10 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 10 of 34
`
`132
`
`5,566,291
`
`130
`.~
`
`As is the tase w1tl1 most cf cur knttwledge of science many poop\e con~:rfouted
`to our unde:rst.'lncilng ef gtaYity, V!it.\.i gra"'itf four people in particular stand
`0rr.t:A.:rfa"";otfo1,m1B }fow!:r.:ln and Elnstatn.
`
`Aristntle.> .a greak pbi1C1rophcr/~i~nti.st thought th.~.t- ~s:-te:rythlng j~r1as a plac.e'i,
`Fer example, rocks don't naturally fly through the air, they aJ:"$ found on the
`grtmnd. Ee thought that the reason l:ma11y objects fall to thi; gro;;ncl was
`simply that they Gebngetl there,
`In tlu: :myt.hobgy of %'.ienct\ G,1\iloo (also known for the t8Jes:::cpe) is said w
`have st.cud atop to'le U<!nL'1g Tower of Pisa in It.~ly and dropped obj ens of
`d\f:ferentwe\g,hts whil.e fonnv.l~ting his laws of falllngbod\es. Th~ b.ws
`stated that except for tlw r~IBtance of the air all bod.Jes would fall w the
`ground at the same rate.
`
`The !aws of falling bodies also stated thaL.
`
`:figure 11
`
`•As is the t~ wlt.'1 most cf cur kno•Nletlge cf science tna."lJ people ccntributed
`to ,~~rr tmd~ts- of ~r,avity. "\¥\t.'1 f ,r;ir; four poop le in patticular stanct
`. m,,. An:rto"'"''
`Ne 'I; tJm and Eins\i!!n.
`
`Ar!JttQtle, a !P"ffk ph\Jnsopherkc\imtJ.st thovght ~hat everything "has a p},;ci;",
`· E'or eJ<.<i.mpie, rocks. don't narnrally fly thn:mgh tll~H;,ir, t . .hey ari; found on the
`•ground. He th=ght ths,t thu rm111on heavy objects faH to the grourd ,,.,,,,,.,,
`•simply that they belonged 1.lmre
`
`·In the mythalogy cf sclence, (falHeo (a!ro known fo:r the teies::cpe) ls said tc
`have stood amp the Leanlng Tovter of Pisa Jn Italy and drnpped cbjrr::ts cf
`dlffor;mt weights whi!~ formulating hi~ laws of f:-H~:g bOO:te,s. These ;a·Ns
`5\;!.wt\ that except for t;,'le ~;s~nce Gftol<l'! aw al! oouies vroud faH ~.o tne
`growd at the same raw.
`
`Joe: did you know this about GalHeo before?
`
`e
`II
`+
`+
`
`&!
`~
`
`II
`e
`Ii
`+
`+
`WI
`
`151
`
`AXR-1006-Page 11 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 11 of 34
`
`5,566,291
`
`-171
`The Germ:i.m --.American phy:!ic!st
`A!bert.Eirtst,ein, b. Ulm, Germany,
`Mar. 14, H:l79. d. :Princett:i:rdi.J.,
`Apr. Hl, 1.855, rontril.mtim:i. rrnirn
`HHtn. an;t other sdmtist to tht!
`2tlth«Ct!ntnry visio.n of phyilc11
`reality .. In the wal<.e of V\Tc:r.ld Vl ar
`I, Einstein's thoo:ries-"especially his
`thooryofrelativity--seemed 10
`many
`quaJity
`removed frnm the war and 115
`aftermath. Seldom !'>.a1 a. scientht
`roceived such publ.Jc attention for
`
`for
`
`As i$ t.':!e {;~with m""'t of {}Ur lm.!:lwlli!Cigep:f scle.ncema.ny people eontrnutetl
`..... ·er.$1'.a.~m.n~in·g. Q, ! grav,Jt:y,, ~.ff
`.. · lth r~vtty, fou:rpeop!e in particular stand
`m:ur·unrl··,
`cu:.:Ar:tstntie, bahl!!!t, Ne.wton .l;1nd · ~ 169
`k .• ristntle, a g:n"!'!!,kphU~be:risdenti5t d1\lught thateverythJ:ng "has a ?ace",
`li\:n" ~mp~, :roeu clm:t't l'MttunUy fly th:raugh the air, t,h;i;y «re i'l)uml. m the
`
`:.Figure 13a
`
`170
`,..,,..,,__
`The. ~r.man-A.niITTPJn phy:rldst
`AloortEtns>:~n,b. 1Ylrn,t:~..rrnany,
`Mar 14,1819.d Prim:~ioD«.N'L
`Apr' 1 s, 1955, contdhuti!tl mure
`than any other sclmtist to the
`20th-cent,ury vision
`l
`reality, 1n the wake
`· in's theorie;."espe::iai17 Js
`of telativlN·-St!err'..ed ro
`ple to ~inttc a pmll
`human thought, one far
`thPm ttm wa:r anrl ~ts
`
`tist thought tliate'\·erything "has a plx:e ',
`fly throtigh thii ~Jr, thti!y a;Ftl' fmmd on the
`
`:.Figure 136
`
`AXR-1006-Page 12 of 64
`
`

`
`~
`\,C
`N
`~ ....
`~
`Ol
`....
`Ol
`
`~
`g,
`N
`~ .....
`rJJ. =-~
`
`'= ~
`'=
`.....
`~tit
`.....
`~
`0
`
`~
`•
`00.
`•
`Cj
`
`~ ;-a
`
`,r-130
`
`:.Figure 14
`
`-.__,.
`1·68
`
`1eo--'
`
`1!52
`·-"'··""-w.www,.uwwwNmm,.mmm>~·>j,,,,,_ --·>wmmmm>~mwww,_www>
`
`'158
`
`''!63
`
`aving cultivated the rruitof pure learning.
`and 1bs afterrnath. Seldom has a sctenttst received such pubhc attenif~n
`·le to poin!:, to a pm"equality of lmrn~ri u:ought~.one far rernm•'.ed ft·or;11the
`
`.Einstein's theories--especially his theory of relatJvit,y ~-seemed ton·.
`
`scientist tn the 20th"century vision of physical reality. In the· \Vake of
`187 9, d. Princeton, N .J., Apr, 18, 1 ~)55, contributed more than any other
`The German-A.merican physidstAlbert Einstein, b. Ulm, Germany5 lvfar,
`
`AXR-1006-Page 13 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 13 of 34
`
`5,566,291
`
`180
`.................
`
`182
`............. _,
`
`184 .............. _,
`
`;...-...... ....
`186
`
`188 :...-...... ....
`
`190 1..-....... ....
`
`192
`'...-...... ....
`
`194 ................
`
`200 , ...............
`202 ,.........._.,...
`
`206 ;.-...... ....
`
`SITUATION ATTRIBUTE
`
`RESPONSE ATTRIBUTE
`
`SHARING ATTRIBUTE
`
`SHARING CONTENT
`
`OTHER ATTRIBUTES
`
`DATE
`
`LOCATION, IDENTITY
`
`CONTEXT
`
`OBJECT IDENTIFIER
`
`SUB-OBJECT IDENTIFIER
`
`RESPONSIBLE PERSON
`IDENTIFIER
`
`~174
`
`:Figure15
`
`176
`
`:...-...... ....
`208
`
`RESPONSIBLE PERSON
`ORGANIZATIONAL IDENTIFIER
`
`, ............. .....
`210
`
`SENDER ORGANIZATIONAL
`IDENTIFIER
`
`204
`
`, ........... _,
`READ STATUS
`---------------------
`212{213
`....
`............
`214
`CLEARED/CLOSED STATUS
`
`MESSAGE DATA
`
`194~
`
`197
`
`198
`
`196
`687, 381, 28, 461, 320, 0, 0, 0, Technical EOS, Testing EOS, Analog
`Testing EOS, Procedures EOS, Closeout Test EOS, EOS, EOS, EOS
`
`199
`
`:Figure15a
`
`AXR-1006-Page 14 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 14 of 34
`
`5,566,291
`
`AXR-1006-Page 15 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 15 of 34
`
`5,566,291
`
`I
`I I I
`I I
`t·'-
`1~
`I ~
`I ~.
`;:t
`I
`I~
`
`AXR-1006-Page 16 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 16 of 34
`
`5,566,291
`
`AXR-1006-Page 17 of 64
`
`

`
`~
`
`~
`\C
`~ --N
`--Ul
`Ul
`
`"""' w
`c
`-...l
`~
`~ ......
`Cl) =-~
`
`.i;:..
`
`~
`
`'-= ="'
`'-=
`
`~Ol
`~
`:"""
`(')
`0
`
`= ~
`~
`'"'C
`•
`00.
`d •
`
`'Figure19
`
`.. -270
`·~··
`,. t
`
`---~
`
`< :§L..273
`
`-.w.• • ..-m
`
`. .·.· < : ·· .. ·
`
`• .··
`...
`
`,,,,.,..,...,,,.,,,..,.,,..._, , ,,.-.• ,...,.,..,.,•,•,•,•,.,-,•,•,•,• . ....,..-.-.-.---,-,-,-.-.-.,..,.,•,•,•,•.v-.•
`
`268
`
`222'
`
`,.,.--,.,,.,
`
`.~·
`
`216'
`
`· Cl
`
`E1 ···
`
`272~
`i~~I
`
`1
`
`269 ~··~iJ:Jy
`
`'-.. ~,,..L ...
`
`252
`
`SAVED SETTINGS
`
`CHANGE
`
`2760 ,«,
`r::f J5
`~
`
`D
`0
`D
`
`SAVED SETTING*
`
`· M. f£RRY
`tP. BROWN
`j= ,J., S M.fTH. . .
`BLDG A
`
`SCHOOL 1 (43Z)
`
`51§.f'R!CT A (4516}
`AUTHORL
`
`I3lbt~ A,
`
`OISTR!CT A (4516)
`
`CLASS 1
`,
`'(432)
`
`CLA$$2:
`
`[It~~~
`
`r
`
`~
`
`254
`
`(''263
`
`I P'··J''·~ ... ''I~"'
`
`~. r ... :n~ ... ~<:,:
`
`I
`
`SCIENCE
`
`t ~il~'r.?~~ ....
`L PRINCIPLES
`~H!STOFPt
`
`BIOLOGY
`
`CHEMISTRY
`
`PRINCIPLES
`
`[MAXWEL
`
`[
`
`r-11111111
`H!ITTORY (10)
`
`PHYSICS {18)
`
`E {:236)
`
`,.~
`
`254
`
`Hi::SPO~li,!:ERB*"O*'N~· ~---""'·
`
`I LOWER SIUPERV!SORS
`
`SUPE!iV!SORS
`
`231!
`
`2241~··
`
`\ '-1>,,
`255
`
`Screen Panels Records---.. 253
`
`Feedback. Visualizer
`
`AXR-1006-Page 18 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`282
`
`Sheet 18 of 34
`
`5,566,291
`:Jigure20
`¥"' 280
`
`284
`
`USER NAVIGATING
`COMPUTER
`ENVIRONMENT
`
`NO
`
`288
`
`DISPLAY FEEDBACK
`WINDOW
`
`290
`
`INITIALIZE RECORD
`
`INSERT TECHNICAL
`ATTRIBUTES INTO
`RECORD
`
`291
`
`PROCESS USER
`INPUT AND FILL
`RECORD
`ACCORDINGLY
`
`292
`
`295
`LOOK UP
`ORGAN(cid:173)
`IZATIONAL AND
`PERSONAL
`ATTRIBUTES
`AND INSERT
`INTO FEED-
`BACK RECORD
`
`296
`
`OUTPUT
`RECORD
`
`NO
`
`AXR-1006-Page 19 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 19 of 34
`
`5,566,291
`
`305
`
`CHANGE
`FEEDBACK
`INTERFACE
`ACCORDING TO
`~-"" USER CHANGES
`
`:.Figure21
`
`306
`
`308
`
`UPDATE DISPLAY
`TO SHOW
`SELECTED
`SITUATION ICON
`
`INSERT
`SELECTION
`INTO RECORD
`
`UPDATE DISPLAY
`TO SHOW
`SELECTED
`RESPONSE ICON
`
`320
`
`UPDATE DISPLAY
`TO SHOW
`SELECTED
`SHARING ICON
`
`326
`
`DISPLAY
`PROMPTED
`FEEDBACK
`ENTRIES IN
`POP-UP WINDOW
`
`316
`INSERT
`.._____ SELECTION
`INTO
`RECORD
`
`322
`INSERT
`SELECTION
`INTO RECORD
`
`328
`INSERT
`SELECTED
`ENTRY INTO
`RECORD (IF
`ANY)
`
`332
`INSERT
`MESSAGE INTO .,___ __ __,
`RECORD
`
`336
`
`PROCESS
`SELECTION
`
`AXR-1006-Page 20 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 20 of 34
`
`5,566,291
`
`342
`
`/340
`
`READ DEFAULT
`VIEWER
`PREFERENCES
`
`348
`
`READ VIEWER'S
`ACCESS TO
`FEEDBACK RECORDS
`
`--344
`
`SCAN DATABASE AND
`CREATE VISUALIZER
`RECORDS FOR ALL
`ALLOWED RECORDS
`
`346
`
`CALCULATE
`COMBINED SET
`
`349
`
`DISPLAY INITIAL
`VISUALIZER
`WINDOW
`
`350
`
`PROCESS VIEWER
`INPUT AND UPDATE
`DISPLAY
`ACCORDINGLY
`
`352
`
`354
`
`:Figure22
`
`AXR-1006-Page 21 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 21 of 34
`
`5,566,291
`
`356
`
`.....-346
`
`358
`
`i =0
`
`i = i + 1
`
`IS i < NUMBER OF
`RECORDS IN
`DATABASE?
`
`NO
`
`YES
`
`NO
`
`CREATE VISUALIZER
`RECORD HAVING
`INDEXES FOR
`ATTRIBUTES, DATES,
`CONTEXT, AND
`LOCATION
`
`362
`
`INCREMENT
`VISUALIZER
`RECORD COUNTER
`
`364
`
`:Jigure23
`
`AXR-1006-Page 22 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 22 of 34
`
`5,566,291
`
`~368
`
`:Jigure23a
`
`370
`............. _...
`
`SITUATION ATTRIBUTE
`INDEX
`
`371 ............. _... RESPONSE ATTRIBUTE
`INDEX
`
`372 ............. ~
`
`SHARING ATTRIBUTE
`INDEX
`
`374 , ............. _
`
`376 ............. _
`
`CONTEXT INDEX
`
`LEARNER LOCATION
`INDEX
`
`377 ............. _... RESPONSIBLE PERSON
`LOCATION INDEX
`
`378 , ............. i-
`
`DATE INDEX
`
`380
`
`1 ..................
`
`RECORD NUMBER
`
`, .............
`382
`
`~
`
`ON/OFF FLAG
`
`SCIENCE
`
`PHYSICS
`
`r--375
`
`HISTORY
`
`;
`
`:figure 23 6
`
`PEOPLE t EINSTEIN
`
`NEWTON
`
`MAXWELL
`
`EVENTS
`
`PRINCIPLES
`
`CHEMISTRY
`
`AXR-1006-Page 23 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 23 of 34
`
`5,566,291
`
`388
`
`:Figure 24
`
`CLEAR SEGMENT
`COUNTERS FOR
`EACH FILTER
`SEGMENT
`
`390
`
`392
`
`i = 0
`
`i = i + 1
`
`IS i < NUMBER OF
`VISUALIZER
`RECORDS?
`
`NO
`
`YES
`
`STRIKES = 0
`
`393
`
`NO
`
`j=O
`
`j = j + 1
`
`IS j < NUMBER OF
`FILTERS IN
`RECORD(i)?
`
`YES
`
`394
`
`STRIKES = STRIKES + 1
`FIL TERN UM = j
`
`412
`
`INCREMENT
`SEGMENT
`COUNTER OF
`SEGMENT
`DESIGNATED IN
`FIL TER(FIL TERN UM
`
`408
`
`j=O
`
`j = j + 1
`
`ISj<NUMBEROF NO
`FILTERS IN
`RECORD(i)?
`
`YES
`
`INCREMENT
`SEGMENT
`COUNTER OF
`SEGMENT
`DESIGNATED IN
`FIL TERU)
`
`YES
`
`410
`
`AXR-1006-Page 24 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 24 of 34
`
`5,566,291
`
`416
`
`/352
`
`420
`PROCESS
`CHANGES
`
`426
`
`DISPLAY LIST OF
`REQUESTED
`RECORDS
`
`DISPLAY
`INFO FROM
`SELECTED
`RECORD
`
`430
`
`NO
`
`YES
`
`425
`
`1'igure25
`
`AXR-1006-Page 25 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 25 of 34
`
`5,566,291
`
`434
`
`/420
`
`PROCESS FILTER
`~--~ CHANGES AND UPDATE 1 - - - - - - - ,
`DISPLAY ACCORDINGLY
`
`438
`
`444
`
`UPDATE BAR
`CHART DISPLAY
`BASED ON TIME
`RESOLUTION
`
`448
`
`UPDATE BAR CHART
`DISPLAY BASED ON
`SCROLL OR SCALING
`BUTTON SELECTIONS
`
`452
`RECALCULATE
`COMBINED SET
`BASED ON SAVED
`FILTERS
`
`454
`
`UPDATE
`DISPLAY
`
`DISPLAY
`CHANGE
`SETTINGS
`DIALOG BOX
`
`458
`
`PROCESS AND
`STORE USER
`CHANGES TO
`SAVE
`SETTINGS
`
`460
`
`:.Figure26
`
`AXR-1006-Page 26 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`¥"438
`
`Sheet 26 of 34
`
`5,566,291
`:Jigure27
`
`466
`
`TOGGLE/SET
`FILTER
`SEGMENT
`SETTINGS
`
`CALCULATE
`COMBINED
`SET AND
`UPDATE
`DISPLAY
`
`468
`
`TURN SELECTED
`FILTER SEGMENT ON
`AND OTHER SEGMENT
`SETTINGS OF THE
`FILTER OFF
`
`474
`
`478
`
`DISPLAY
`SEGMENT
`VALUES
`
`NO
`
`482
`
`REMOVE
`SEGMENT
`VALUES
`
`490
`
`COPY ENTIRE SET
`OF FILTERS INTO
`TEMP FILTERS
`
`ADJUST TEMP
`FILTERS AS IN
`STEP 474
`
`488
`
`UPDATE
`DISPLAY
`
`CALCULATE
`COMBINED SET
`---l USING TEMP
`FILTERS
`
`492
`
`NO
`
`498
`
`REDISPLAY
`ORIGINAL
`COMBINED
`SET
`
`AXR-1006-Page 27 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 27 of 34
`
`5,566,291
`
`,...
`CO·
`
`l()
`
`)\
`
`•• j
`i c
`J
`
`I .,
`
`1.:.·
`
`1.:
`J j
`= j
`....
`
`~' ' N
`
`~
`lO
`
`. r
`
`e
`E
`
`Q 1
`z
`I@
`a,!=.•.·
`LU
`en
`i ~
`
`!
`j
`i c
`J
`
`)
`,...
`(0
`lD
`
`1
`!
`i!
`~.1 .... ·'.
`.. = oc
`
`...
`. i:
`j··.··.·.
`
`AXR-1006-Page 28 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 28 of 34
`
`5,566,291
`
`~42'
`
`86
`
`81
`
`!figure29a
`
`92
`
`v-42"
`
`!figure 296
`
`AXR-1006-Page 29 of 64
`
`

`
`~
`\C
`N
`~ ....
`~
`01
`....
`01
`
`~ +:>.
`s,
`\C
`N
`00 =-[
`
`=--
`\C
`""""'
`\C
`
`""""' ~Ol
`~
`0
`
`~ = ;-a
`
`•
`00
`d •
`
`568 -
`
`'figure 30
`
`564_...
`J
`h£/e(.Jr;e.nHhe screen d-0esn1 right up~
`
`562
`
`556 -
`
`11t.N?94, ldt1, N<'VNIFYI, f2vM!/¢t;~,
`Wi13t94. Qu!?'»!.iQ11,. Snen. i'\fic:.nwn
`to/131'.!( Un»~*eN~<:!. Uns
`
`554 -
`
`sso-"1ma1~i
`
`;,-·-'.\
`
`.,?. ~
`
`-/
`
`"""'
`
`'
`46",.
`
`.
`
`AXR-1006-Page 30 of 64
`
`

`
`~
`\C
`"' N
`~
`~
`"' Ul
`Ul
`
`~
`~
`
`Q
`~
`
`00
`
`s,
`[ .....
`
`~
`
`="'
`\C
`\C
`"'"" ~
`0 p.
`
`~ ;-= ~
`
`~
`•
`'fJl
`•
`Cj
`
`"~590
`
`577~"'°'
`
`.1 igure 31 '-588
`
`'•
`
`581"'/ (C1"
`
`578
`(
`
`574
`l
`,.~
`
`.. Unselected
`Responses
`
`BDifficu!ty
`
`·
`
`~sgo
`
`User Help
`Appearance
`
`Q,~6 USER JOB
`
`Manutact
`Sales
`
`.-==l-Managemnt
`~·Engineer
`·
`
`• Sta.ff
`
`Qi!? USER REGl:ON
`
`Westem U)3
`Central U.S,
`Eastem U,S,
`Outs1de U,S.
`Unknown
`
`~579
`Di .. aste-~
`J Complaint
`
`".f."<-· ·.
`...
`
`:
`'
`
`··.·:
`
`Problem
`tdea
`
`I
`
`;
`
`:
`:
`
`1
`,__ ___
`
`Unselected
`
`w.www<•<w••···1 Question
`
`f"'
`
`~579 I 5129 USER AGE
`70+
`51-70
`3 H50
`16<30
`5~ 15
`
`.___--1 Anonymous
`Not My Boss
`Experts
`Everyone
`Unse:l0cted
`
`~~~
`
`I
`
`577--~-
`
`577--·
`
`Tatl-590
`Ln
`Mi
`
`--·
`
`USER TEt
`
`Ta
`
`·-------------~~~~-------------------·~==-"''"'"'""'""'"""''"'"'""'"""""""""'""'"'"'"'-=-«•--~~~~
`
`;1 :ii::::.:::::{:;;;:~,
`
`<=b:t:)~
`
`I Database
`
`--=== .. ~· 1 Ut!Htles
`Drawing
`Spreads ht
`~__,,1 Word Proc ..
`
`I
`
`
`
`4.
`
`J
`
`!
`
`I
`
`Unselected
`
`~~,., J ; fiZil
`
`. 1
`
`(584
`
`Becord&--253
`
`Visualizer Attributes
`
`[[e ·~579 r577 Q:l•IQl-579 I APPUCATiON PROGRAM
`
`~--' I'm Stopped
`
`. I Soon ~579
`
`'\
`
`ASAP
`
`Mo Hurrv
`Never/FY!
`Unselected
`
`r
`
`....
`
`577----·
`
`FUe ccreen Pane•ls
`
`!J:iil
`
`-
`
`-
`
`AXR-1006-Page 31 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 31 of 34
`
`5,566,291
`
`j •
`
`,..,
`
`"".
`
`ti
`
`AXR-1006-Page 32 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`291'-
`
`612
`
`Sheet 32 of 34
`
`5,566,291
`:Figure 33
`
`GET OPERATING CONTEXT
`INFORMATION (IF ANY) FROM
`APPLICATION AND INSERT INTO RECORD
`
`614
`
`GET NETWORK CHARACTERISTICS
`AND INSERT NETWORK TYPE AND
`NETWORK ADDRESS INTO RECORD
`
`616
`
`GET HARDWARE PLATFORM
`CHARACTERISTICS AND
`INSERT INTO RECORD
`
`618
`
`GET OPERATING SYSTEM
`NAME AND VERSION AND
`INSERT INTO RECORD
`
`620
`
`GET CURRENT APPLICATION
`AND VERSION AND INSERT
`INTO RECORD
`
`622
`
`GET DATABASE TYPE, NAME
`AND LOCATION AND INSERT
`INTO RECORD
`
`GET DOCUMENT NAME AND
`LOCATION AND INSERT INTO
`RECORD
`
`624
`
`626
`
`GET LOCATION IN DOCUMENT
`WHERE FEEDBACK WAS INITIATED
`AND INSERT INTO RECORD
`
`GET VERSION OF FEEDBACK
`RECORD FORMAT AND INSERT INTO
`RECORD
`
`628
`
`630
`
`DONE
`
`632
`
`AXR-1006-Page 33 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`Sheet 33 of 34
`
`5,566,291
`
`295~
`
`!Figure 34
`
`640
`
`LOOKUP ORGANIZATIONAL
`LEVELS AND INSERT INTO
`RECORD
`
`642
`
`LOOKUP TASK AND
`INSERT INTO RECORD
`
`644
`
`LOOKUP USER IDENTITY AND
`DEMOGRAPHICS AND INSERT
`INTO RECORD
`
`646
`
`GET TIME AND DATE AND
`INSERT INTO RECORD
`
`648
`
`650
`
`AXR-1006-Page 34 of 64
`
`

`
`U.S. Patent
`
`Oct. 15, 1996
`
`660~
`
`662
`
`Sheet 34 of 34
`
`5,566,291
`!Jigure35
`
`USER CALLS
`FEEDBACK SYSTEM
`
`664
`
`PRESENT AVAILABLE
`SELECTIONS
`
`666
`
`RECEIVE
`SELECTION
`
`668
`
`YES
`
`PROMPT VOICE
`FEEDBACK
`
`672
`
`RECEIVE VOICE
`FEEDBACK
`
`674
`
`PROMPT
`"SEND" OR "CANCEL"
`
`676
`
`OUTPUT
`RECORD
`
`680
`
`682
`
`AXR-1006-Page 35 of 64
`
`

`
`5,566,291
`
`1
`METHOD AND APPARATUS FOR
`IMPLEMENTING USER FEEDBACK
`
`CROSS REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a divisional application of patent
`application Ser. No. 08/362,801, filed Dec. 22, 1994 on
`behalf of Boulton et al., entitled, "Method and Apparatus for
`Implementing User Feedback", which is a continuation-in- 10
`part of patent application Ser. No. 08/173,962, filed Dec. 23,
`1993, now abandoned, on behalf of Boulton et al., entitled
`"Method and Apparatus for Implementing User Feedback",
`which is incorporated herein by reference in its entirety.
`
`BACKGROUND OF THE INVENTION
`
`2
`"learn" from its customers' experiences of those products
`and services through customer feedback. Feedback is also
`necessary for internal departments and processes within a
`business. The more effectively that employees can learn a
`5 new procedure or suggest improvements to, for example,
`manufacturing processes, the better will the business per(cid:173)
`form competitively.
`Feedback is just as necessary in education. A student may
`often reach a point in course material where he or she does
`not understand the concepts being taught. If the student can
`indicate to the teacher where the difficult material is, the
`teacher can be more aware of difficult concepts and can
`improve the course. Or, the student might eventually
`struggle to understand a concept and have his or her own
`15 suggestions of how to better present the material. If these
`learning needs of the learner are not addressed using feed(cid:173)
`back, the learner may become used to skipping over difficult
`concepts, ignoring their own questions about the concepts
`since those questions are not answered within the system. If
`the learner can provide feedback about a course, the course
`can be improved and the outcome is better learners.
`Feedback is also very necessary in other environments
`besides learning environments. For example, a testing envi(cid:173)
`ronment in which a product or service, such as software, is
`25 being tested for operability, reliability, ease of use, etc., can
`implement user feedback. Product users and testers can send
`feedback about products or services which can greatly aid
`developers in testing and developing those products and
`services. Similarly, any product, process, service, or issue
`30 experienced by users can be improved through the collection
`and analysis of feedback.
`In the past, the need to collect feedback from learners and
`other types of users has been recognized by organizations.
`Typically, however, feedback-gathering systems have been
`inefficient at best. One of the most prevalent problems in
`prior feedback-gathering systems is motivating users to
`supply accurate and worthwhile feedback. There are several
`reasons why users typically resist feedback. One reason is
`that users complain that giving feedback takes too much
`time and is too distracting. Conventional feedback-gathering
`methods typically take the form of supplying the user a
`questionnaire or similar response opportunity after at least a
`large portion of the material being taught has been presented
`or after a service has been experienced. For feedback to be
`given in any amount of detail, relatively lengthy question-
`naires are required which can take a great a deal of time to
`complete. In addition, since such questionnaires are typi(cid:173)
`cally filled out long after any problems were experienced, it
`may be difficult for the user to remember the details of a
`problem or suggestion. For example, it may be difficult for
`a learner to remember all the details of course materials and
`in which areas he or she had trouble and/or suggestions.
`Another reason users typically are not motivated to supply
`feedback is that they know what is wrong or what needs to
`be changed, but they don't know enough about the entire
`situation or concept to explain the problem. That is, a user
`often does not know or cannot explain the context of the
`problem or suggestion that is necessary to "place" the
`feedback in the mind of the recipient. The step of describing
`the context of a user's problem can therefore take a lot of
`time and potentially cause misunderstandings between the
`reviewer or provider and the user.
`Other reasons why users are not motivated to supply
`feedback are that users often do not know whom to give the
`feedback to and are reluctant to provide critical and useful
`feedback for fear of possible reprisals. The user may want
`
`20
`
`35
`
`45
`
`1. Field of the Invention
`This invention relates generally to a system that facilitates
`the collection, organization, and visualization of user feed(cid:173)
`back. In one described implementation a system for permit(cid:173)
`ting users of computer software to make feedback as to the
`content of the software while the software is in use is
`described.
`2. Background of the Related Art
`Organizational learning systems are incorporated in such
`organizations as schools, which teach students a curriculum,
`and businesses, which teach employees skills. An organiza(cid:173)
`tional learning system typically includes at least two basic
`roles: a learner and a teacher. The learner wishes to learn
`new concepts, procedures, or other information. The teacher
`ideally wishes to present the information to the learner in a
`way that best meets a learner's individual learning needs.
`However, it often occurs that the learner "stutters" in the
`learning process or "flow": the learner comes upon a con(cid:173)
`cept, instructions, or information that he or she does not
`understand, either partially or completely. It is in the teach(cid:173)
`er's interest to know which information the learner had
`trouble understanding, if the learner had trouble due to the 40
`presentation method or style, and how the presented infor(cid:173)
`mation can be improved to facilitate learning. Once the
`teacher has obtained and understood these aspects of the
`taught information, the teacher can change and improve the
`presentation of the information to more fully meet the
`learners' learning needs and facilitate the learning process.
`A critical resource for the teacher to discover how to
`improve taught information and the learning organization is
`learner "feedback", which is the information, comments,
`suggestions, and questions sent from the learners to the 50
`teacher about the taught information.
`The most essential feature of any self-regulating, self(cid:173)
`correcting, or self-optimizing system is feedback. Without
`feedback, the system is stagnant: learners or users will
`continue to have problems learning or interfacing within the 55
`system, and have no way to voice those problems to effect
`improvements. Learners know which parts of taught infor(cid:173)
`mation gave them the most trouble or could best be
`improved, and it is