USOO885.6030B2
`
`(12) United States Patent
`Russek
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,856,030 B2
`Oct. 7, 2014
`
`(54) METHOD, SYSTEMAND SOFTWARE FOR
`ASSOCATING ATTRIBUTES WITHIN
`DIGITAL MEDIA PRESENTATIONS
`
`(75) Inventor: David J. Russek, Maple Glen, PA (US)
`
`(73) Assignee: Sevenecho, LLC, Norristown, PA (US)
`(*) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1841 da
`.S.C. 154(b) by
`yS.
`
`(21) Appl. No.: 10/819,514
`(22) Filed:
`Apr. 7, 2004
`
`5,692.212 A 1 1/1997 Roach
`5,725,472 A
`3, 1998 Weathers
`5,734,795 A
`3/1998 Rogers
`5,751,953 A
`5, 1998 Shiels
`
`.
`
`5,754,787 A * 5/1998 Dedrick .
`5,774,664 A
`6/1998 Hidary
`5,805,784. A
`9, 1998 Crawford
`5,848,934 A 12, 1998 Shiels
`!!
`38. San
`w 4
`S
`5,913,310 A
`6, 1999 Brown
`5,918,603 A
`7/1999 Brown
`5,935,003 A
`8/1999 Stephens
`5,945,988 A
`8/1999 Williams
`(Continued)
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. TO9,228
`
`US 2004/O199923 A1
`
`Oct. 7, 2004
`
`WO
`
`O177939 A1 10, 2001
`
`Related U.S. Application Data
`(60) Provisional application No. 60/460,998, filed on Apr.
`7, 2003.
`
`(2012.01)
`(2012.01)
`(2006.01)
`
`51) Int. Cl.
`(51)
`G06O 30/02
`G06O 30/00
`G06F 7/30
`(52) U.S. Cl.
`CPC .......... G06F 17/30867 (2013.01); G06Q30/00
`(2013.01); G06F 17/30265 (2013.01)
`USPC ........................................ 705/14.66; 705/319
`(58) Field of Classification Search
`CPC ........................... GO6O 5O/O1 GO6O 3OfO269
`USPC
`Q 705 f14 , 66,319
`See application file for complete search history. s
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6, 1992 Beitel et al.
`5,119,474 A
`5,586,967 A 12/1996 Davis
`5,604.855 A
`2, 1997 Crawford
`5,625,570 A
`4/1997 Vizireanu
`
`OTHER PUBLICATIONS
`
`Quintana, Y.: “Design of an Object-Oriented Multimedia Database
`for Personalize Multimedia News'. Proc. 1996 Canadian Conf. on
`Electrical and Computer Engg., May 26, 1996-May 29, 1996; pp.
`282-285, XP002460458, Calgary, Canada.
`
`Primary Examiner — Donald L. Champagne
`(74) Attorney, Agent, or Firm — Pepper Hamilton LLP
`
`ABSTRACT
`(57)
`Disclosed are a system, method and Software to associate
`attributes with digital media assets. Digital media contains
`specific assets, such as images, that can be replaced with other
`assets. The system, method and Software permit the associa
`tion of attributes with specific assets. The association of
`attributes and assets enables the provision of content that is
`enhanced and more impacting for a user.
`
`2 Claims, 25 Drawing Sheets
`
`
`
`
`
`
`
`
`
`RID a2 Sis
`s
`COMMUNITYUSER sa
`ONLINE
`"N"Y ACTIVE v. NACTIVE!
`52
`
`s:
`
`
`
`STOREDUSER
`PROFE
`
`9
`se oé
`s
`SS’s se
`l
`c
`
`s
`
`co"ENT
`
`531
`
`S
`f2O2S
`SDIGITALASSET
`REPOSITORY
`
`MODIFIECONTENT - 551
`STORAGE
`
`

`

`US 8,856.030 B2
`Page 2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,977,968 A 1 1/1999 Le Blanc
`5,987,415. A 1 1/1999 Breese
`5.999,172 A 12/1999 Roach
`6,029, 195 A
`2, 2000 Herz
`6,078.936 A
`6, 2000 Martin
`6,08877 A
`7, 2000 Reed
`6,20846 A
`9, 2000 Hin
`6,135,944. A 10/2000 Bowman
`6,175,772 B1
`1/2001 Kami
`6.190.314 B1
`2/2001 Ark
`6,205473 B1
`3/2001 Gilmour
`6.232,925 B1
`4/2001 Shiels
`6249.780 B
`62001 Miokawa
`6.293904 B
`9,200 Blazey
`6.306.077 B1
`10, 2001 Prab
`6,3194 B
`02001 Sheth
`6,317,127 B1
`1 1/2001 Daily
`6,330,595 B1 12/2001 Ullman
`6,345,288 B1
`2/2002 Reed
`6,357,042 B2
`3/2002 Srin
`6.421,669 B1
`7/2002 Gilm
`6.425,825 B1
`7/2002 Sitrick
`6,466.232 B1
`10/2002 Newell
`6,499,741 B2 12/2002 Yamazaki
`
`1/2003 Uwakubo
`6,513,011 B1
`1/2003. Abato
`6,513,069 B1
`2/2003 Surve
`6,520,905 B1
`2/2003 Morris
`6,526,395 B1
`3/2003. Manico
`6,527,700 B1
`3/2003 Chase
`6,529,864 B1
`4/2003 Flannery
`6,545.209 B1
`6/2003 Pak
`6,581,037 B1
`7, 2003 Obrador
`6,585,521 B1
`8/2003 Sadakuni
`6,604,091 B2
`2/2006 Balthaser
`7,000, 180 B2
`1/2007 Mascarenhas
`7,162,432 B2
`1/2009 Herz ..................................... 1.1
`7.483,871 B2 *
`1/2014 Ward et al. ...................... 725/40
`8,635,649 B2 *
`2001/0049588 A1 12/2001 Bausch et al.
`2002/0013943 A1
`1/2002 Haberman
`2002/0092019 A1
`7, 2002 Marcus
`2002/O122430 A1
`9, 2002 Haberman
`2002/015964.0 A1 10/2002 Vaithilinga
`2003/0001880 A1
`1/2003 Holtz et al.
`2003/OOO9078 A1
`1/2003 Fedorovskaya
`2003.0036899 A1
`2/2003 Leslie et al.
`2003/0045957 A1
`3, 2003 Haberman
`2003/0131351 A1
`7/2003 Shapira
`2003/0142689 A1
`7/2003 Haberman
`2004/026781.6 A1 12/2004 Russek
`* cited by examiner
`
`

`

`U.S. Patent
`
`US 8,856,030 B2
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 1 0f 25
`
`US 8,856,030 132
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SF__
`
`m0wzon_m
`
`._.m_._.~_<
`
`kzmhzoo
`
`Dz<>._m_>_m_wm<
`
`202.39me5
`
`m>:.<mmo
`
`mohowmfi
`
`So:
`
`mOHDmEHmE
`
`
`
`8:502Ezawwzoo
`
`EEzoox:
`
`229%
`
`5928
`
`
`
`5.Qt
`
`TTI-1001, Page 3
`
`
`
`
`
`

`

`U
`
`7,
`
`w
`
`06,5008,
`
`m
`
`mNON
`
`ozaomommOm
`
`mos):
`
`S._.IIIIIII_
`
`t.1IIIIIII_mvow__mmmozmanmm80%:u"8N
`89>P.ozaomoxoé“m\
`
`m365
`
`3SN
`
`mmoiméo8Nmozazém
`
`%6&2:8N
`
`.EEBV59
`
`.0230mebah
`
`mNGE
`
`382365
`
`m8%?NE
`
`TTI-1001, Page 4
`
`

`

`U.S. Patent
`
`6,5008,
`
`2
`
`fGIIIIIInmmunMWJIV<AI__Asjm,,,,,,vONme.m%mumgmfive
`
`8llllll5«mm
`
`4anmmm:m:._................,.--------..--------------_.u...u............................mHH-........_u-H.H.n---.um32%;;.n_u_u_u_uwmzocosm
`
`
`
`
`
`
`
`7,8N3<zommmmmn_n_n_u_msvom92awnu_u_u_um832Omom/\Jn(lllll__fIIIII_WIIIIII_”Illlll_mJ<meufiz_
`
`um262ENFS"IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII.n
`8N3<zom~mmmmm>52muas):
`
`E833:"m32%;:WmsSmwzwnmzm
`
`BImmat
`
`moi:a05:5.mom
`
`TTI-1001, Page 5
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 4 0f 25
`
`US 8,856,030 B2
`
`5.2me
`
`mozmimmvfi
`
`mmOQE.IV
`
`omN_._<zowmmn_
`
`moszmExm.
`
`925.;m>_._.o<mm._.z_
`
`
`®®08
`
`vow
`
`mmEZDEOnEO
`
`vGI
`
`TTI-1001, Page 6
`
`

`

`U.S. Patent
`
`US 8,856,030 B2
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 5 0f 25
`
`US 8,856,030 B2
`
`
`
`
`
`
`
`
`
`| 99
`
`5m
`
`Hzmhzoommw:
`
` Eocmonmmmmwmjéoa‘Es
`
`
`.;sn:mmm:
`
`mzjzo
`
`>:z:s=200
`
`Em
`
`
`
`5mHzmhzooomEEO—z
`
`mo<mokm
`
`
`
`mmm:ammohm
`
`mjioma
`
`«6.Qt
`
`TTI-1001, Page 7
`
`
`
`

`

`U.S. Patent
`
`US 8,856,030 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`819, SOIWWNAG
`
`Z09
`
`NJOLINOW
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 7 Of 25
`
`US 8,856,030 B2
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 7 0f 25
`
`US 8,856,030 B2
`
`cy
`O
`CO
`0")
`O
`
`O(
`
`U
`2
`LIJ
`H
`a
`:Z
`co
`O
`(3
`O
`
`Z'
`
`OU
`
`‘—
`
`O(
`
`O
`
`v
`O
`CO
`
`VOC
`
`O
`
`r
`C
`CO
`
`_|
`
`<Z 9p
`
`.OEL
`
`IJ
`
`C
`t
`O
`H
`O
`s
`L
`
`FIG.6
`
`NO(
`
`.0
`cn
`O
`gCd
`
`.J
`SO
`)
`-
`3.
`CD
`O
`D
`
`OU
`
`TTI-1001, Page 9
`
`

`

`U.S. Patent
`
`U.S. Patent
`
`027,aO
`
`3,m00w
`
`0w
`
`2B
`
`%zOF<_>_mOu_z_
`
`>>mzu_O
`
`zoEm500<MEE
`
`ooNFNmzoEzoOomm
`m,o:
`
`\zoEmSOMEELzo_._.o:n_ommm_m
`
`mw<zoz<4\wzaz<kmmmozo
`
`
`
`szE>>Iommmm
`
`mNat
`
`max
`
`w:
`
`
`
`moovow
`
`._<mmm_>
`
`Oz_wmmoomn_
`
`S03NE
`
`90970/
`
`
`
`
`
`
`
`
`
`
`
`TTI-1001, Page 10
`
`
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 9 Of 25
`
`US 8,856,030 B2
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 9 0f 25
`
`US 8,856,030 B2
`
`
`
`._<zmmhz_m._n=._..5_>_
`
`m>_.r<mm<z
`
`zOEmmomwm
`
`zo_._.<o_u__._.zmn__
`
`\l/
`
`Amomm5:3/xmo>>m2<mm
`
`OZ9
`
`mgimmm::._.._.<
`
`mowmooz
`
`zo_._.o<mm.rz_
`
`A/
`no._m_>m_._
`
`Hzm§m>40>z_
`
`A
`
`250m
`
`20:15me
`
`zo_»<o_u_:zmn__
`
`xmogmzém
`
`
`
`Amomm§_<
`
`908
`
`._<_oOm\mnSOmw\N8
`
`mE\v/
`
`m—w9:20830>
`
`mazmzofifimm262828832.:
`
`228me#88
`
`Esznz05
`v503E.
`
`amm_._._z_u_u_<
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`mGE
`
`TTI-1001, Page 11
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`US 8,856,030 B2
`
`U.S. Patent
`
`4m27,a0
`
`hS
`
`52
`
`00
`
`306,58,
`
`m
`
`20.192:
`
`
`
`3558o_:n_<meozoo_
`
`'
`
`
`
`
`
`
`
`
`
`
`
`
`
`:5@Sm
`
`IHN—OZ
`
`Z<0Em2<
`
`SE9250E952UzoEmomE9.
`ezorzmzmg9m._<o_E<moom_o
`m5moma
`
`
`
`w20205308
`
`0aGE
`
`TTI-1001, Page 12
`
`
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 11 of 25
`
`US 8,856,030 B2
`
`
`
`
`
`
`
`CREATIVE DIRECTOR
`LISTENSIWATCHES
`CONTENT
`
`1000
`
`TAGTIME INDICES
`WITHAFFINITIES
`
`1010
`
`VY 1020
`
`EDTAFFINITIES
`COALESCE MEANING
`IDENTIFY MAPPING
`
`1030
`
`
`
`STORE TAGS
`
`1040
`
`
`
`
`
`
`
`CREATE INTERACTIONS M 1050
`
`
`
`DEVELOP COLLATERAL
`MATERIALS
`
`1060
`
`FIG. 10
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 12 of 25
`
`US 8,856,030 B2
`
`
`
`
`
`E:4_<_>_-m_omO>>mw<mn:mum:
`
`
`
`._.m_z.n_w_.1:55IwEOO38.5
`
`
`
`
`
`
`
`::moz<Eon=>=ZEOZXZQZEOZXEmsmd3:9:eE8
`
`
`
`
`
`228222672!mag
`
`
`
`a:xzém3;Emsmd3:9:9$8
`
`
`
`
`
`:92265807.35%mama
`
`zgozxz:8%£85
`
`
`
`IE65mmoz<Em<magImoz<mm_<zmmmagI5&52!was
`
`
`
`
`
`2Gt
`
`TTI-1001, Page 14
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 13 Of 25
`
`US 8,856,030 B2
`
`
`
`
`
`PERSON
`
`1200
`
`AFFINITY LIST
`GET AFFINITIES
`SETAFFINITIES
`
`
`
`1210
`
`
`
`
`
`
`
`
`
`
`
`PEOPLE GROUP
`
`SET OF PEOPLE
`
`1220
`
`EXPERIENCE
`
`GROUP OF PEOPLE
`RECORDEDAFFINITIES
`MEDIA
`
`1240
`
`
`
`AFFINITY
`NAME
`DESCRIPTION
`PARENT
`CHILD
`GROUP
`
`
`
`
`
`
`
`1230) RECORDED AFFINITIES
`TYPE
`AFFINITY
`
`
`
`
`
`
`
`
`
`
`
`FIG. 12
`
`

`

`U.S. Patent
`
`27,ao
`
`52MMamS
`
`06,588,SU
`
`2
`
`Eomnzoh
`
`
`
`wo_>mo.an._<_mmm
`
`
`
`
`
`Eomutok20min:Eomntoh
`
`
`
`
`
`$92202om<0m>mxmeEmz
`
`482$3528m9.
`
`
`
`m._.O._w_on_9.0.5<m_
`
`mag92:
`
`B.w2GE
`
`TTI-1001, Page 16
`
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 15 Of 25
`
`US 8,856,030 B2
`
`
`
`1410
`
`ST 1 H^ 1432
`
`BACKGROUND 1
`
`ST2 H^ 1434
`
`BACKGROUND 2
`
`FIG. 14
`
`

`

`U.S. Patent
`
`US 8,856,030 B2
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 16 0f 25
`
`US 8,856,030 B2
`
`
`
`
`
`5«I
`
`acts
`
`ELSE.
`
`
`
`W0m
`
`w.mH
`
`\
`
`\
`
`mw
`
`\x.
`
`15..
`
`umw
`
`\x
`
`mwon.
`
`
`
`83823.2.33D
`
`.QE
`
`TTI-1001, Page 18
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 17 Of 25
`
`US 8,856,030 B2
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 17 0f 25
`
`US 8,856,030 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`828E393® .mmo_OIoomzkmmomm<H<I._.b“.mv‘u‘m>:o<mm_._.z_EOmE
`Awecu»
`
`
`
`
`
`meEOmE20m:Qmw<mmmw:1mmsz_§mm._.wow_._.2m:.zooQwN3<zOmmmmo__>_<z>omi...
`
`
`
`
`
`
`mw.W
`
`.mm
`
`w.
`
`\.
`
`.flmuS
`
`.I0Ommus
`N
`
`\
`
`remoNemP
`
`TTI-1001, Page 19
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 18 of 25
`
`US 8,856,030 B2
`
`
`
`fervoritesy
`
`istory
`
`.
`
`Search
`
`is
`s'
`
`Patie Holder,
`
`Zl,
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 19 Of 25
`
`US 8,856,030 B2
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 19 0f 25
`
`US 8,856,030 B2
`
`
`
`
`
`
`
`aw»
`
`
`
`2‘GE
`
`TTI-1001, Page 21
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 20 of 25
`
`US 8,856,030 B2
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 20 0f 25
`
`US 8,856,030 B2
`
`
`
`
`
`
`
`Nom—
`
`TTI-1001, Page 22
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 21 of 25
`
`US 8,856,030 B2
`
`
`
`
`
`
`
`
`
`
`
`ANOI ANOI ÅNol J|}
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 22 of 25
`
`US 8,856,030 B2
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 22 of 25
`
`US 8,856,030 B2
`
`
`
`
`
`
`
`“was"«man”E835:szEgg
`
`.Eflssugfiw3a5333.29231i
`
`2&3ufi:3:955HEEfies
`
`TEMPEE83
`
`and
`
`TTI-1001, Page 24
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 23 Of 25
`
`US 8,856,030 B2
`
`ELVERJO
`
`
`
`
`
`

`

`U.S. Patent
`
`US 8,856,030 B2
`
`U.S. Patent
`
`cO
`
`1027,
`
`52f042teehS
`
`00SU
`
`2B030:658a
`
`t.2%
`
`ZO_.P<OE_ww<._04(.20EOHOMEO
`
`
`
`whzm§w4wwin—ONE>>OQ<Iw._<._._O_n_
`
`
`
`LOmech.mum;OFwhiz.ZO_._.<34<>w
`
`
`
`
`
`ZO_._.<.5n_OQmZEOmm._<DO_>_DZ_
`
`
`
`hDQZ_wszFuéwHZ—m02w53<
`
`
`
`wCEz.mm_._<s_mOz
`
`ommm
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`ZO_F<O_u=ww<._Uhmwmd‘.3505m.._.<34<>m
`
`
`
`
`
`
`
`
`
`<20“—0mm:...x=2mmm2.OPm0k0mw=o>m
`
`
`
`mozmmmEm—makwmw<
`
`
`
`
`
`min—ONE>>Oo<Im._<.:0_D“.0tEOEa
`
`mhzmfim—Jm.
`
`
`
`
`
`
`
`20:04.Hmww<(Ems.449.05“.0EEOEQ
`
`
`
`
`
`._<Do_>_oz_10$0...ZO_._.0<(5—0022
`
`oomm
`
`ommm
`
`ommm
`
`ommm
`
`mmmm
`
`mwmm
`98ZZ
`
`mmm6E
`
`TTL1001,Page26
`
`
`
`O...OZEmOOO<m..__u_0ma300.11%4<.:0_o
`
`
`
`
`
`
`
`
`
`
`
`mozmmmmmmam0“.ESE/«moanmeIQI
`
`
`
`
`
`zOF<o_u=HZmo_m02m_n_3<Dwoz<IzmDZ<
`
`
`
`
`
`

`

`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 25 0f 25
`
`US 8,856,030 B2
`
`m>_._bm._._oo
`
`m>_._.<m0m<._._oo
`
`m2;
`
`zO_._.<._.om¢xm_a050.2%\m>szmw
`
`oz_ozm30mw
`
`._<zmm:z_
`
`m>_._.<mm<z
`
`mtg:
`
`mozmmmummm
`
`>OO._On_O.r
`
`22
`
`.202
`
`Esmmommnz
`
`
`
`29:#szon
`
`2599:0
`
`>j<60w
`
`Em0<02m
`
`oszm—D
`
`._<_00wOz
`
`._.zms_m0<ozm
`
`momoz<>mjmm
`
`._<_oow
`
`HZmeo/‘ozm
`
`55.20002
`
`wmmIHoEOE
`
`Eamomafi
`
`5928
`
`m02m8mm
`
`23$62
`
`It;
`
`$558
`
`mo:96“.
`
`20.58%
`
`mmEOoFDO
`
`\wzo_.p<._.omn_Xm
`
`dmmam
`
`20:05.
`
`mEOEbOamoz<_m<>
`
`mm._m<_m<>
`
`hzm._.2_
`
`0252545
`
`
`
`Hzmpz.w
`
`mEIB:9:
`
`zofisoomm,‘
`
`mmo<ozm
`
`mmoz<~m>m
`
`mmEFZmQ
`
`mmxogmo
`
`._<o_mOIn_<._.ms_
`
`4<mm._._._
`
`oz:.<mm<z
`
`oz_z_<._.z_<_>_
`
`Hzmkzoo
`
`Oz
`
`HzmHZOo
`
`mEEmOmn—nz
`
`._.zm._.zOo
`
`5:;
`E:
`.zOz
`-ujmw
`5mm
`
`
`
`
`
`
` MmogmzémZOHH<DEHHZmQHZOmeUMmEBrigg/HiOE<UEBm<AUquOME
`
`
`
`
`
`
`
`wm_._._z_uE<
`
`4<_00m
`
`4<zo_._.OS_m
`
`mmEzEnE
`
`m>_._._szo
`
`mez_u_u_<
`
`
`
`
`
`HD<E>Em>EL<MM<Zmm<mMUZH
`
`“Q<00
`
`>>O._
`
`>>O._
`
`woz<o_¢_zw_m
`
`moz<o_..=29w
`
`sSEws.
`
`220m:
`
`moz<o_n__zo_m
`
`moz<or=zgw
`
`
`
`>>Ozx’52
`
`2262!
`
`.moL552206
`
`925%
`
`mC‘EmOw—mn—d‘
`
`._.zm._.zoo
`
`Basra”:
`
`65:58
`
`go0226:
`
`5528
`
`8528
`
`mod:$50
`
`
`
`20:3I._._>>
`
`
`
`m>0m<._._<
`
`IQ:
`
`IQ:
`
`moz<o_u=zo_w
`
`moz<o_..=zw_w
`
`mm.OE
`
`TTI-1001, Page 27
`
`
`
`
`
`
`
`
`
`
`

`

`US 8,856,030 B2
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`1.
`METHOD, SYSTEMAND SOFTWARE FOR
`ASSOCATING ATTRIBUTES WITHIN
`DIGITAL MEDIA PRESENTATIONS
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This patent application claims priority to the provisional
`patent application entitled, “Method and System for Digital
`Entertainment Asset Personalization.” filed Apr. 7, 2003,
`serial No. 60/460,998, which is incorporated by reference
`herein. This patent application is also related to the patent
`application entitled, “Method, System and Software for Digi
`tal Media Narrative Personalization.” filed Apr. 7, 2004.
`
`BACKGROUND OF THE INVENTION
`
`It has been stated that human thinking falls into one of two
`categories: reasoning and narrative, descriptive, contempla
`tive thinking. Reasoning is the process that permits the accep
`tance of facts followed by a logical process to conclude a
`result, opinion, conclusion or decision. Narrative, descrip
`tive, contemplative thinking corresponds to the storytelling
`process (including receiving of a story) and the thought pro
`cesses associated with the reception of a story.
`Text and stories (presented in various formats) can also be
`interpreted at various levels including the literal, ethical, his
`torical and mystical levels. It is generally understood that
`texts and stories are read and interpreted in different ways
`including at the different levels. A story or narrative can have
`different meanings at different levels, different interpreta
`tions at different levels, and will be interpreted differently by
`different individuals at each of the levels. Additionally, an
`individual, based on their emotional or physical state, may
`interpret each level of a story differently. A person may react
`differently to a story based on their current emotional state,
`which may in fact change while they are receiving the story.
`In some cases, it may be possible to alter their emotional state
`while receiving the story to make them more receptive to a
`particular interpretation at one or more levels.
`Advertising is typically a type of narrative in which the
`advertiser attempts to influence the recipient (viewer, reader,
`listener) in a way that creates a positive association with a
`productor in Some cases attempts to influence the recipient to
`purchase the product. Humoris frequently used inadvertising
`because it puts the recipient in a receptive state for a message
`regarding the product. As such, humor is already used to alter
`the emotional state of the recipient of the advertising. Such
`manipulation of the emotional state of the recipient is cur
`rently limited to simple manipulation through the presenta
`tion of the advertisement (narrative).
`With the advent of the digital era, the number of ways to
`transmit and present advertisements has increased dramati
`cally, with advertising being inserted into Web pages, being
`displayed as pop-up windows on Web pages, transmitted as
`streaming media over the Internet, presented as animation
`over the Internet, and inserted into television streams at cen
`tral or distributed insertion points. The advent of the digital
`era also threatens advertising in that viewers can avoid adver
`tisements by stopping pop-up windows, or skipping over the
`advertisements on a Personal Video Recorder (PVR).
`Due to the rapid changes in technology, advertisers are
`being forced to take new approaches to advertising including
`product placement, in which products are advertised during
`an entertainment narrative (e.g. a sit-com) as opposed to in a
`separate advertisement. Advertisers are also exploring ways
`to draw viewers (or listeners) to content that is essentially
`
`2
`digital media narrative storytelling, typically in the form of
`music or a short story that would appeal to viewers. What is
`required are ways to attract individuals to content that is
`personally more relevant and impactful for them and which
`may contain an advertising message (in the form of product
`placement), and have them receive that message in full, as
`opposed to skipping over all or a portion of the message. It
`would be an additional benefit to have the ability to under
`stand the individual’s likes and dislikes or current mood in
`order to adapt the message appropriately for the individual at
`the time that they are receiving it.
`Digital media narrative may include content, such as video,
`audio, and combinations thereof, that is distributed digitally
`and can be stored digitally. In some instances digital media
`narrative can be manipulated via computer without human
`interaction. Digital media narrative includes but is not limited
`to stories presented over the Internet, modifiable digital video
`including digital television and streaming media, presenta
`tions on web sites including animations, still images,
`sequences of still images, audio, textual presentations, and
`combinations of all of the abovementioned media.
`Digital media narrative can be distributed in a multitude of
`ways, such as broadcasting over cable and satellite, the Inter
`net, or on media such as optical disks (CDs or DVDs). Dis
`tribution of digital media narrative is a significant business
`and involves complex relationships between artists and
`record labels or film studios, producers, and a complete dis
`tribution chain that includes advertising, both in the form of
`advertising of the content itself as well as sponsored adver
`tising that may be inserted into, or become part of the content
`Such as product placement. Examples of sponsored advertis
`ing include the insertion of commercials into recorded mate
`rials or the use of sponsors products (e.g., automobiles) in the
`narrative.
`Digital media narratives may be directed at a segment of
`the market, but the segment that the digital media narrative
`material is directed to may be quite large. For example, an
`artist may create works that appeal to the 18-24 year old age
`group, but that group can include people with different inter
`ests from various countries and different economic classes.
`Furthermore, when digital media narratives are used as part
`of advertising, that digital media narrative and the advertising
`may be directed at Such a large market segment that it is
`difficult to keep the interest of individuals within that market
`segment. If a user goes to an Internet web site to view a
`recording of an artist, and in particular a sponsored work that
`contains advertising, that work (including the advertising)
`may be so generic that the user cannot respond to or relate to
`the work. Additionally, the user may experience different
`emotional states than another user, or the user may be in an
`emotional state that is different than the user's emotional state
`when the user previously viewed the material.
`For the foregoing reasons, there is a need for a method,
`system, and software to enable a distributor of sponsored or
`un-sponsored digital media narratives to change the content
`of the digital media narrative based on user demographics,
`psychodemographics, emotional states, affinities (cognitive,
`emotional and social), self-narrating content classification,
`internal narrative traits preference topology, time sensitive,
`episodic expectation sequencing, and collective/collaborate
`attributes.
`
`SUMMARY OF THE INVENTION
`
`65
`
`A method, system and software are presented which allow
`for customizing and personalizing content based on a com
`bination of the user's demographics, psychodemographics,
`
`

`

`3
`cognitive states, emotional States, social placement and group
`interaction dynamics within an online community, and/or
`affinity for certain content elements (images, Sounds, seg
`ments, graphics, video, text, dialog), self provided narrating
`content, internal narrative traits preference topology, and
`expectation level and temporal spacing of assets within the
`narrative. In one embodiment, a user selects material for
`presentation and is presented with the dynamic digital media
`narrative and optional embedded advertising presented in a
`customized format that gives the user a personalized experi
`CCC.
`The system, method and software may support product
`placement type advertising by modifying both the product
`placement material as well as the narrative Surrounding the
`product placement material. Appropriate advertising may be
`selected as well as selecting components for the narrative that
`Supports the product placement advertising.
`In one embodiment, the user accesses a website containing
`sponsored material and logs on to that website. Information
`regarding the demographics, psychodemographics, cognitive
`states, emotional states, social placement and group interac
`tion dynamics with the online community, and/or affinity for
`certain content elements of the user is retrieved from storage.
`If the user is a new user, he or she is asked a series of
`questions, preferably through a form of media narrative, that
`would creatively ask the questions, in order to obtain key
`information that the system would then classify and include
`into the user's profile. These questions can include yes/no.
`multiple choice, like? dislikes ratings or any other type of
`personal assessment or individual and/or group question.
`Alternatively, the user may interact with the media narrative
`and the interactions may be used to create a user profile.
`In one embodiment, the user requests that the material be
`played or displayed, and is simultaneously presented with
`that material and customized personalized graphics and
`video. In another embodiment, the user can interact with the
`graphics and video. By making certain choices, a profile of
`the user can be determined and updated and the content (in
`cluding background material, primary video, overlay graph
`ics and audio) modified appropriately, thus giving the user an
`enhanced narrative, and more meaningful and emotionally
`satisfying experience with respect to the viewed content. In
`both embodiments, the user profile can then be used to present
`the most appropriate digital assets to the Subscriber, namely
`those with which the user has the highest affinity, or those
`which map well to the user's tendencies and temperament,
`which may be included in the user's internal narrative per
`ception identification framework. The internal narrative per
`ception identification framework may include a collection of
`attributes, qualities, and measurements regarding the user that
`allow for matching, correlation and/or selection of digital
`media assets that are appropriate for that user and the effective
`communication of the message.
`In one embodiment, the user's demographics are used to
`determine personalized content. Demographics may include,
`and are not limited to, group affiliations, regional location,
`sex, sexual preference, age, family situation (married or
`single, children or no children), race or ethnic background,
`religion and Socioeconomic condition. The user's demo
`graphics may be used alone, or in combination with the pre
`viously discussed techniques, to create the individual internal
`narrative perception identification framework.
`One advantage of an embodiment of the method, system
`and software presented herein is that the user is presented
`with an enhanced experience of the creator's content that
`creates a greater emotional experience for the user and a more
`impactful narrative. This creates a more receptive state for
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 8,856,030 B2
`
`10
`
`15
`
`4
`product placed advertising inserted into the content, for a
`direct advertisement, or for entertainment containing an
`advertising message. Another advantage of the method, sys
`tem and Software presented herein is that it allows advertising
`to be inserted in Subtle ways and presented in a context in
`which users may be able to fully engulf themselves into the
`lifestyle being positioned and portrayed by the brand. By
`combining the personalized digital media narrative asset ele
`ments with product placement advertising, a more powerful
`media is created for the advertisers and the artists. Addition
`ally, since users typically Volunteer to see the content, includ
`ing advertising, they are much more likely to be receptive to
`the message presented, and less likely to skip over or fast
`forward through the content including the advertising.
`In an alternate embodiment, users are presented with ques
`tions or chat like textbased opportunities within a narrative to
`either share the emotional experience with other users or to
`comment on their own emotional experience. This enables a
`user to share an experience with friends or acquaintances or
`allows a user to “sit back and relax” while someone else
`controls the enhanced experience. In this embodiment, the
`personalized content is based on the learned social dynamics
`of the user.
`Another embodiment includes a developer toolkit that
`allows a creative director to create trigger points in the mate
`rial where digital assets such as overlay graphics, back
`grounds, icons, text, sound, and/or product placement images
`can be inserted into the digital media narrative. The developer
`toolkit allows the creative director to create a map of the
`elements of the user's profile and his or her internal narrative
`perception identification framework topology to digital assets
`that may be created. The developer toolkit also provides for
`the ability to insert interactive material Such as questions or
`interactive graphics, allowing for the updating and enhance
`ments of the user profile based on the users interactions. The
`toolkit further permits the creative director to create and link
`the digital assets to the material such that assets for which the
`user has a high degree of affinity or assets having a highest
`probability for matching a user's internal narrative perception
`identification framework topology can be selected at the trig
`ger point.
`One advantage of an embodiment of the method, system
`and Software presented herein is that they may enable a cen
`tral server or service to compile a user's internal narrative
`perception identification framework and to store the internal
`narrative perception identification framework as an internal
`narrative perception identification framework profile and
`allow this profile to track or follow the user as he or she
`peruses multiple digital media narratives. This enables the
`user to receive enhanced digital media assets and various
`narrative experiences and allows the service to be sold to
`advertisers that advertise on any web site or broadband IP
`based distribution point.
`These and other features and objects of the invention will
`be more fully understood from the following detailed descrip
`tion of the embodiments which should be read in light of the
`accompanying drawings.
`In this respect, before explaining at least one embodiment
`of the invention in detail, it is to be understood that the
`invention is not limited in its application to the details of
`construction and to the arrangements of the components set
`forth in the description or illustrated in the drawings. The
`invention is capable of other embodiments and of being prac
`ticed and carried out in various ways. Also, it is to be under
`stood that the phraseology and terminology employed herein,
`as well as the abstract, are for the purpose of description and
`should not be regarded as limiting.
`
`

`

`US 8,856,030 B2
`
`5
`As such, those skilled in the art will appreciate that the
`concepts upon which this disclosure is based may readily be
`used as a basis for designing other structures, methods, and
`systems for carrying out the several purposes of the present
`invention. It is important, therefore, that the claims be
`regarded as including Such equivalent constructions insofar
`as they do not depart from the spirit and scope of the present
`invention.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The accompanying drawings, which are incorporated in
`and form a part of the specification, illustrate embodiments of
`the present invention and, together with the description serve
`to explain the principles of the invention.
`FIG. 1 illustrates a user-relationship diagram for an
`embodiment of the system;
`FIG. 2 illustrates the representative components of a digital
`media narrative and the digital media assets;
`FIG. 3 illustrates the creation of a personalized digital
`media narrative;
`FIG. 4 illustrates the use of interactivity to create an
`enhanced user profile and an enhanced experience through a
`personalized digital media asset;
`FIG. 5A illustrates a context diagram for digital media
`narrative asset personalization;
`FIG. 5B illustrates potential databases that may be used in
`creation of the personalized digital media asset and the rela
`tionships between those databases;
`FIG. 6 illustrates exemplary relationships between the
`Social, emotional, and cognitive affinity elements;
`FIG. 7 illustrates exemplary components of the cognitive
`element;
`FIG. 8 illustrates exemplary components of the social ele
`ment,
`FIG. 9 illustrates exemplary components of affinity;
`FIG. 10 illustrates a flowchart for the development of the
`components for a personalized digital media narrative;
`FIG. 11 illustrates data structures for a relational database
`for linking components of a personalized digital media asset
`with affinities;
`FIG. 12 illustrates an object oriented approach for linking
`components of a personalized digital media asset with affini
`ties;
`FIG. 13 illustrates a computer on which the invention can
`be built;
`FIG. 14 illustrates the construction of an Interactive Musi
`cal Intersode (IMI);
`FIG. 15 illustrates a registration screen for an IMI;
`FIG. 16 illustrates a screen representing a personalized
`background in an IMI;
`FIG. 17 illustrates a screen with personalized content in
`digital video form:
`FIG. 18 illustrates a screen posing a question related to an
`IMI to a user;
`FIG. 19 illustrates a screen posing a multiple choice ques
`tion related to an IMI to a user;
`FIG. 20 illustrates an alternative screen for posing a ques
`tion related to an IMI to a user;
`FIG. 21 illustrates a screen having a text communication
`window related to an IMI;
`FIG. 22A illustrates a flow chart for the operation of the
`screens shown in FIGS. 15-21;
`FIG.22B illustrates a flow chart for the operation of a fuzzy
`logic based Enhanced Director Agent (EDA); and
`
`6
`FIG. 23 illustrates a narrative perception identification
`framework that enables matching of digital media assets to
`individuals.
`
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS
`
`In describing an embodiment of the invention illustrated in
`the drawings, specific terminology will be used for the sake of
`clarity. However, the invention is not intended to be limited to
`the specific terms so selected, and it is to be understood that
`each specific term includes all technical equivalents which
`operate in a similar manner to accomplish a similar purpose.
`With respect to the user, a number of terms are used that
`describe how the user is identified and profiled. As used
`herein the term “user identification' means a number, sym
`bol, or alphanumeric sequence that identifies a single Sub
`scriber, a group of Subscribers, a Subset of Subscribers, or a
`Subscriber in a specific location or on a specific network.
`The term “user profile' includes a stored or calculated
`profile that describes one or more aspects of the user, such as
`demographics, psychodemographics, and attributes. The pro
`file can be determined from questions answered by the user,
`forms filled out by the user, and/or interactions of the user
`with the digital media narrative. Alternatively, the profile can
`be determined from the user's web surfing characteristics,
`shopping habits, television viewing habits, and/or actual pur
`chases. Profiling of users based on these interactions, viewing
`habits, and purchases is well understood by those skilled in
`the art.
`“User attributes” include aspects, characteristics, and
`qualities of the user that are useful for determining (matching,
`correlating, and selecting) dig