`
`
`
`
`
`
`
`Exhibit A
`
`
`
`
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 2 of 39
`Case 612°‘CV'00810'ADA D°°“"‘|‘IIIIIIIIIII||flI|IIII|IIIIIIIIIflIIIIlIIIIIIIIIIIIIIIIIIIIIIIIIIII
`
`USOO8856030B2
`
`(12) United States Patent
`(10) Patent No.:
`US 8,856,030 B2
`
`Russek
`(45) Date of Patent:
`Oct. 7, 2014
`
`(54) METHOD, SYSTEM AND SOFTWARE FOR
`ASSOCIATING ATTRIBUTES WITHIN
`DIGITAL MEDIA PRESENTATIONS
`
`5,692,212 A
`5,725,472 A
`5,734,795 A
`5,751,953 A
`
`11/1997 Roach
`3/1998 Weathers
`3/1998 Rogers
`5/1998 Shiels
`
`(75)
`
`Inventor: David J. Russek, Maple Glen, PA (US)
`
`5,754,787 A *
`5,774,664 A
`
`5/1998 Dédmk """""""""""" 709/228
`6/1998 Hidary
`
`(73) Assignee: Sevenecho, LLC, Norristown, PA (US)
`
`5’805’784 A
`5,848,934 A
`
`9/1998 crfoord
`12/1998 Shiels
`
`( * ) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`USC 154 b b
`1841 da
`'
`'
`'
`( ) y
`W
`
`.
`(21) APP1~ N°~~ 10/819,514
`
`(22)
`
`Filed:
`
`Apr. 7, 2004
`
`2,3355%; 2
`’
`’
`5,913,310 A
`5,918,603 A
`5,935,003 A
`5,945,988 A
`
`51333 gifeinan
`le S
`6/1999 Brown
`7/1999 Brown
`8/1999 Stephens
`8/l999 Williams
`
`(Continued)
`
`US 2004/0199923 A1
`
`Oct. 7, 2004
`
`WC
`
`0177939 A1
`
`10/2001
`
`Related US. Application Data
`(60) Provisional application No. 60/460,998, filed on Apr.
`73 2003.
`
`(2012.01)
`(2012.01)
`(2006.01)
`
`51
`
`(
`
`)
`
`Int. Cl,
`G06Q 30/02
`G06Q 30/00
`G06F 17/30
`(52) US. Cl.
`CPC .......... G06F 1 7/30867 (2013.01); G06Q 30/00
`(2013.01); G06F17/30265 (2013.01)
`USPC ........................................ 705/14.66; 705/319
`(58) Field of Classification Search
`CPC ........................... G06
`50/01' G06
`30/0269
`USPC
`Q
`765/1 4 (124 66 319
`See application file for complete search history. ’
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5,119,474 A
`5,586,967 A
`5,604,855 A
`5,625,570 A
`
`6/1992 Beitel et a1.
`12/1996 Davis
`2/1997 Crawford
`4/1997 Vizireanu
`
`OTHER PUBLICATIONS
`_
`_
`_
`_
`_
`_
`Quintana, Y.: “DeSIgn of an Object-Oriented Multimedia Database
`for Personalize Multimedia News”, Proc. 1996 Canadian Conf. on
`Electrical and Computer Engg., May 26, 1996-May 29, 1996; pp.
`282-285, XP002460458, Calgary, Canada.
`
`Primary Examiner 7 Donald L. Champagne
`(74) Attorney, Agent, or Firm 7 Pepper Hamilton LLP
`
`(57)
`ABSTRACT
`Disclosed are a system, method and software to associate
`attributes with digital media assets. Digital media contains
`specific assets, such as images, that canbe replaced with other
`assets. The system, method and software permit the associa-
`tion of attributes with specific assets. The association of
`attributes and assets enables the provision of content that is
`enhanced and more impacting for a user.
`
`2 Claims, 25 Drawing Sheets
`
`
`
`
`
`
`
`e.
`‘1 “1090
`13“» 0025‘
`~
`I“
`T93”
`
`”01* cos?“
`
`(”hr
`f6“
`0'»
`‘35
`A"
`f '1‘ 541
`“‘5’ DIGITAL ASSET
`
`0 “‘23
`: II .A
`
`commumw USER m2“
`ONLINE
`”MMUN'TY fiTIVE v. INACTIVE-r
`521
`
`‘
`
`CONTENT
`
`531
`
`REPOSITORY
`
`Q
`
`
`
`STORED USER
`PROFILE
`
`
`
`
`
`
`
`MODIFIED CONTENT
`STORAGE
`
`551
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 3 of 39
`Case 6:20-cv-00810-ADA Document 1—1 Filed 09/02/20 Page 3 of 39
`
`US 8,856,030 B2
`
`Page2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,977,968 A
`5,987,415 A
`5,999,172 A
`6,029,195 A
`6078 936 A
`6’088’717 A
`6’120’846 A
`6:135:944 A
`6,175,772 B1
`6190314 Bl
`6’205’472 B1
`6’222’925 B1
`6’249’780 B1
`6:293:904 B1
`6306 077 B1
`6:311:194 B1
`6,317,127 B1
`6,330,595 131
`6,345,288 131
`6,357,042 B2
`6,421,669 B1
`6,425,825 B1
`6,466,232 B1
`6,499,741 B2
`
`11/1999 Le Blane
`11/1999 Breese
`12/1999 Roach
`2/2000 Herz
`6/2000 Martin
`7/2000 Reed
`9/2000 Hm
`10/2000 Bowman
`1/2001 Kami
`2/2001 Ark
`3/2001 Gilmour
`4/2001 Shiels
`@2001 Mizokawa
`9/2001 Blazey
`10/2001 Prab
`10/2001 Shed1
`“/2001 Daily
`12/2001 Ullman
`new Reed
`3/2002 Srin
`7/2002 Gilm
`7/2002 Sitrick
`10/2002 Newell
`12/2002 Yamazaki
`
`6,513,011 B1
`6,513,069 B1
`6,520,905 Bl
`6,526,395 B1
`6,527,700 Bl
`6,529,864 Bl
`6,545,209 B1
`6,581,037 Bl
`6,585,521 B1
`6,604,091 B2
`7,000,180 B2
`7,162,432 B2
`7,483,871 B2*
`8,635,649 B2*
`2001/0049588 A1
`2002/0013943 A1
`2002/0092019 A1
`2002/0122430 A1
`2002/0159640 A1
`2003/0001880 A1
`2003/0009078 A1
`2003/0036899 A1
`2003/0045957 A1
`2003/0131351 A1
`2003/0142689 A1
`2004/0267816 A1
`
`1/2003 Uwakubo
`1/2003 Abato
`2/2003 SurVe
`2/2003 Morris
`3/2003 Manico
`3/2003 Chase
`4/2003 Flannery
`6/2003 Pak
`7/2003 Obrador
`8/2003 Sadakuni
`2/2006 Balthaser
`1/2007 Mascarenhas
`1/2009 HerZ ..................................... l/l
`1/2014 Wardet al.
`...................... 725/40
`12/2001 Bausch etal.
`1/2002 Haberrnan
`7/2002 Marcus
`9/2002 Haberrnan
`10/2002 Vaithilinga
`1/2003 Holtz etal.
`1/2003 Fedorovskaya
`2/2003 Leslie et a1.
`3/2003 Haberman
`7/2003 Shapira
`7/2003 Haberman
`12/2004 Russek
`
`* cited by examiner
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 4 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 4 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 1 of 25
`
`30w,
`
`m
`
`____N:
`
`
`
`._.m_._.m<Hzmhzoo
`
`8,weMEEzooU:2an
`
`0hat
`
`
`
`E8:502$2828
`
`5&on2w:
`
`zo_._.3m_w_._.m_o
`
`oz<>._m=2m_wm<
`
`
`
` §__
`
`_
`
`
`
`m>:<mmo_so:
`
`memzommwe~55me
`
`on:
`
`mOHDmEHmE
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 5 of 39
`m20w0
`o.
`
`mmNONm$23:wQZDOmommOm
`
`U
`
`
`ozaomoxoé“Kab”_tllllllllmm3N"nwMmmozmafim$25:"”CommP89>
`
`
`uw
`
`1..n365
`
`ma2m
`
`mww6&2:8Nm.0230QObahm.Emgov:9DMmoiméocomAmozazémDaA7.
`
`9330,mw58,Mr00
`
`m
`
`0$82365
`
`
`
`mmmam?NE
`
`0NGE
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 6 of 39
`w0
`
`xaCPQMU
`
`2t6.n
`
`em808
`
`
`
`moi:.w0535.
`
`$05”.meWmmzozozmmészz;"msemoz<wBEA/283$mm$22w52mW52mxas"...........................................................................................................h"Hfm_IIIIIIu1«Mm_nA.....
`0%mhv_8.0“mm?mS%_rIIIII_43mm.......................................................H.
`......v1.mMAwEJ.
`
`8omoms............._......_......_m22%;:m.nuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuu._
`
`.......wmm........................................Dm555%?m1---}.7---}.7---}.1---}.m20.55Am32%;;mu_u_n_u_"Ann7,8N3<zo$mmmnu_u_u_"msvom9:mam_uu_u_u_m32;
`
`
`
`
`
`rm66me8
`
`30,
`
`mI9wmat
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 7 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 7 of 39
`
`US. Patent
`
`Oct. 7, 2014
`
`Sheet 4 of 25
`
`US 8,856,030 B2
`
`DEFAULT
`
`EXPERIENCE
`
`POINTS
`
`PERSONALIZED
`
`EXPERIENCE
`
`‘_TRIGGER
`
`
`ENHANCEDUSERPROFILE
`
`
`
`320
`
`FIG.4
`
`
`404
`
`04
`
`
`
`INTERACTIVE
`
`OPPORTUNITIES
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 8 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 8 of 39
`
`U.S. Patent
`
`41027.,aO
`
`52f05
`
`US 8,856,030 B2
`
`Em
`
`HZmfizOommm:
`
`m,
`
`
`«M(/9Sm>_._.o<z_>w>_.ro<>y:z:s=>_oo
`
`$33.4mmm:5222200mzjzo
`
`
`
`
`
`
`
`Hzmhzooown—EOE
`
`mo<mo._.m
`
`
`
`mmw:ammo;
`
`w.=n_omn_
`
`«6.6E
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 9 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 9 of 39
`
`U.S. Patent
`
`Co
`
`6ae
`
`US 8,856,030 B2
`
`m825a$8.”:2z_2,508%mm9mpzngooNEEm92mzszo
`
`5&8;mEmoE:
`
`
`
`.0252w30mwEwm<
`
`>tw50mwoz<02:2:
`
`
`
`Qz<zoFOEmszmmammpz
`
`
`
`oz_mn_<_2SuedQOSo
`
`mom
`
`Fm02m=n5<
`
`vowmzjzo
`
`EoszmEXmmmm:m:.:.m_OI>>
`
`It;ozfiomzzoo
`
`5mm?3.05mas
`
`Eoemonmm
`
`$20:
`
`%in«5com
`
`n5
`
`motzoz
`
`Hzmzmo<z<z
`
`oz_._m><m.r
`mimomavmm
`
`
`
`o5
`
`82m365
`
`to$522@2555
`
`3E05
`
`
`
`w;BEES
`
`oz<gnome
`
`._<_oOm
`
`am-
`
`meuoma
`
`Eiaowmmmm
`
`mmGE
`
`
`
`
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 10 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 10 of 39
`
`US. Patent
`
`Oct. 7, 2014
`
`Sheet 7 of 25
`
`US 8,856,030 B2
`
`0’)
`O
`
`O(
`
`LU
`
`2.
`
`:ZOOU
`
`NO(
`
`D
`
`.J
`<_:0
`)
`
`OU
`
`FIG.6
`
`5(D
`
`VOC
`
`O
`
`_|
`
`<Z 9I
`
`—O2L
`
`L]
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 11 of 39
`8
`
`mPs”U
`
`mm
`
`wMS0N5Non
`
`vm829Emt2:
`m.m:N02.3881
`
`0m2,7mm
`
`0,m6,
`
`muHM1m1..emm29258;m52towzoEmsco<mMo:m2
`e8NEWmzoEzooomm
`Hmmo:
`
`azazfimfioz:mEta;IQMEmwwon
`295502E20.580EmmmH2825
`
`2
`
`BlwwAGE
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 12 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 12 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 9 of 25
`
`US 8,856,030 B2
`
`33E;miss:
`
`m>¢<mm<z
`
`zo_Emommn_
`
`2920::25
`
`éoEEE
`
`ASEEi
`
`3m
`
`0%
`
`wow
`
`m5
`
`wn=Isz:.<._mm_
`
`m5
`
`”.0$902
`
`zo_._.o<mm.rz_
`
`
`
`w<zOmmmE._<_00m
`
`@202328832.:
`
`3:208:2
`
`wow
`
`91w:0;wmmEzE<
`
`mGE
`
`Now
`
`vow
`
`._<_oom
`
`20:30me
`
`zOF<o_n_:.zmn__
`
`xmogmiém
`
`mom
`
`
`
`
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 13 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 13 of 39
`
`U.S. Patent
`
`4m7,a0
`
`hS
`
`5
`
`US 8,856,030 B2
`
`N3
`
`._<o_mo.rw_I
`
`
`
`._<m3._..50o=._n_<meozoo_
`
`Sm
`
`r
`
`25pm?ééIEOZ
` 228%;Em5A822”;538m
`
`
`
`
`
`mV0gm._<o_mo_.m_Imom
`
`>moho<50e>mo_._o:<
`
`zo_._.n_mommn_
`
`angxmzmg05
`645.20.5686E
`
`a.GE
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 14 of 39
`Case 6:20-cv-00810-ADA Document 1—1 Filed 09/02/20 Page 14 of 39
`
`US. Patent
`
`Oct. 7, 2014
`
`Sheet 11 of 25
`
`US 8,856,030 B2
`
` CREATIVE DIRECTOR
`
` 1000
`
`LISTENSNVATCHES
`
`CONTENT
`
`
`
`
`TAG TIME INDICES
`WITH AFFINITIES
`” 1010
`
`
`
`vA1020
`
` EDIT AFFINITIES
`
`COALESCE MEANING
`
`1030
`
`IDENTIFY MAPPING
`
`STORE TAGS
`
`
`
`
`
`
`
`
`
`CREATE INTERACTIONS
`
`DEVELOP COLLATERAL
`
`MATERIALS
`
`FIG. 10
`
`1040
`
`1050
`
`1060
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 15 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 15 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 12 of 25
`
`US 8,856,030 B2
`
`
`
`S:.=<_>_-mom0>>mm<mQmum:
`
`
`
`._.m_z.n_w_.122wIwEOOmrmmnm
`
`
`
`
`
`3:moz<._.mon=>=2>>Ozxz22>>OzxHzmsmjmm.__u_ommn:mmm:
`
`
`
`
`
`
`
`
`
`
`
`
`
` ;2267:802/39:28?,Izgozxz:8%28%
`
`
`
`228222672!egg
`
`5:é!HzmgmdflmoEé
`
`
`IELEléééIELE
`
`
`
`
`
`
`2.GE
`
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 16 of 39
`Case 6:20-cv-00810-ADA Document 1—1 Filed 09/02/20 Page 16 of 39
`
`US. Patent
`
`Oct. 7, 2014
`
`Sheet 13 of 25
`
`US 8,856,030 B2
`
`
`PERSON
`
`1200
`
`AFFINITY LIST
`
`SET AFFINITIES
`
`PEOPLE GROUP
`
`1210
`
`
`
`
`
`
`GET AFFINITIES
`
`
`SET OF PEOPLE
`
`
`1220
`
`EXPERIENCE
`
`
`GROUP OF PEOPLE
`RECORDED AFFINITIES
`
`
`
`MEDIA
`
`1230 W
`
`
`
`TYPE
`
`AFFINITY
`
`
`AFFINITY
`1240
`
`
`NAME
`DESCRIPTION
`
`
`
`
`
`PARENT
`
`CHILD
`
`GROUP
`
`FIG. 12
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 17 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 17 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 14 of 25
`
`US 8,856,030 B2
`
`m>_mo92:
`
`82E46528
`
`Q.
`
`30.6_On_$0.6<m_
`
`2at
`
`EONEOP
`
`
`
`
`
`m_0_>m_n_.Eon_._<_mmw
`
`
`
`
`
`EomuzokEOMEOHEOEEH
`
`
`
`
`
`mo.:zo_>_om<0m>mxonEmz
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 18 of 39
`Case 6:20-cv-00810-ADA Document 1—1 Filed 09/02/20 Page 18 of 39
`
`US. Patent
`
`Oct. 7, 2014
`
`Sheet 15 of 25
`
`US 8,856,030 B2
`
`1410
`
`8T1
`
`|———————————————}» 1432
`BACKGROUND 1
`
`ST 2 k——k 1434
`BACKGROUND 2
`0
`
`FIG. 14
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 19 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 19 of 39
`
`U.S. Patent
`
`5
`
`US 8,856,030 B2
`
`
`
`
`
`a.“
`
`
`
`.22«:7;-25354
`
`7,Amm
`4t1mma.
`e\huSm
`2ufm0no.6c1waP
`.33.U
`_s=z<oz_>>w_>O._.«5?;mmfiwamm53—2mmmsmzmozm53<z<m“..Oxk
`
`
`
`
`
`
`
`.mm
`m
`
`wm
`
`‘n
`
`2.33232
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 20 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 20 of 39
`
`U.S. Patent
`
`7:
`
`52f
`
`US 8,856,030 B2
`
`4\1h0m2m
`awom
`0a7m1wta.mmsm
`
`
`
`
`
`
`
`
`E
`
`m0m‘
`
`A
`
`0Ommm
`
`
`
`
`
`scan32an:3an
`
`
`
`
`
`
`
`wfliOmm20m:omm<m”Em:mmEoszmmEom_._.2m:zooQmN_._<zOmmmn_o_s_<z>omI._.
`.mmo_OIomake/Ewe;EOWEowe/Emumm<H<I._.ms.Q‘n‘
`
`
`
`
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 21 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 21 of 39
`
`U.S. Patent
`
`LcO
`
`m
`
`US 8,856,030 B2
`
`512IfH.0a.8m1“Ht.We0eVP‘hmSat
`2m7N
`4w\
`
`
`
`
`2...:itEat;m$3:5.3%
`
`
`
`
`
`.5889>355z_2NFGE32,m<20:2moozazflmME8030;wigs__>__NE
`
`23Ecfifififl;S
`
`_
`
`' mm“ ”snow
`
`<m.cm.
`.‘h
`
`an
`
`D
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 22 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 22 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 19 of 25
`
`US 8,856,030 B2
`
`
`
`
`
`
`
`.2GE
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 23 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 23 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 20 of 25
`
`US 8,856,030 B2
`
`
`
`
`
`
`
`No2
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 24 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 24 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 21 of 25
`
`US 8,856,030 B2
`
`92:NEE..
`
`.,.L
`
`
`>ZOH>20»>20;(L.
`63553.10,
`jillllillllixW
`
`
`
`a.i.mmmmmInimIwEOWwmm:Z<U_>mI
`
`
`
`ww....,.__u.w.
`
`w,
`meow
`
`on.OE
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 25 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 25 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 22 of 25
`
`US 8,856,030 B2
`
`
`
`
`
`2&3fiépng?§9%.5E3
`
`.33233
`
`
`
`.BxxsuEEu«5535333315
`
`.umuE«sawuEaEHEEwQEggA
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 26 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 26 of 39
`
`U.S. Patent
`
`27,a0
`
`2w.hS
`
`5
`
`US 8,856,030 B2
`
`52m35%
`
`mimomm
`
`325mm
`
`mmozmimem
`
`
`
`05.._.<Hoémhz
`
`
`
`H312.moSOwE
`
`
`
`omb‘mmoIU_I>>
`
`m.__n_omn_
`
`<-GE
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 27 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 27 of 39
`
`U.S. Patent
`
`cO
`
`102—/a
`
`52f042teehS
`
`US 8,856,030 B2
`
`LmNNN
`
`ZO_.—.<O_n=mwSO4(5—DEOPOmmfi—
`
`
`
`
`
`w._.Zm_S_m._mw.__n_Omn.>>OD<IW._<._._0_D
`
`
`
`
`
`LOmeHXE‘mum;Op.WHDQZZO_._.<34<>w
`
`
`
`
`
`ZO_._.<43n_Oamemomm._<DO_>_DZ_
`
`8mm
`
`onmm
`
`
`
`mSmZMN_._<s_moz
`
`ommm
`
`
`
`
`
`#312.»sz_._.O<mw._.Z_m02w53<
`
`
`
`
`
`
`
`ZO_.r<O_n:ww<40Hmmw<._<H_O_n_m._.<34<>m
`
`
`
`ZO_.—.O<Hmww<(Ems.4/3205“.0>._._~._O_mn_
`
`
`
`
`
`
`
`._<Do_>_oz_:05OhZOFQ<<20as).
`
`
`
`
`
`mfiEOKn—>>Oo<Iw4.1205“.0>._._mO_mE
`
`9.2!).me
`
`
`
`<55nowmbhxsmmu2.OPNahum—«=0>m
`
`
`
`mozmmwummmFwww<
`
`owmm
`
`mwmm
`
`ommm
`
`mama
`
`mNNGE
`
`
`
`O...OZ_DmOOO<mEEOmn.>>OD<Iw4350.0
`
`
`
`
`
`
`
`
`
`
`
`mozmMmmmmam0“.>._._.:m<m0mn_meIQI
`
`
`
`
`
`zO_k<o_u=.rzmo_m02m_03<Dmoz<IzwDZ<
`
`
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 28 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 28 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 25 of 25
`
`US 8,856,030 B2
`
`m>_._.0m._._oo
`
`m>_._.<m0m<._._oo
`
`>j<_oOm
`
`Bm0<ozm
`
`oszm—o
`
`._<_00wOz
`
`._.zms_m0<ozm
`
`mowoz<>mdm
`
`._<_oow
`
`HZmeo/‘ozm
`
`HzmhzooOz
`
`
`
`wmmzho20mm
`
`Eggnog;
`
`$928
`
`ms:
`
`E>Emzmw
`
`zO_._.<._.omaxm_a03.09%
`
`oz_ozw30mw
`
`6.68%
`
`22£02
`
`IE,
`
`$558
`
`mo:82
`
`20.38%
`
`meOoHDO
`
`\szPEbmaXm
`
`dwmam
`
`20:02
`
`$200.50amoz<E<>
`
`mm._m<_m<>
`
`._.zm._.z_
`
`wz_ozm:<>
`
`
`
`Hzmfiz.w
`
`25
`
`.202
`
`EsEoEnz
`
`
`
`29:#szon
`
`22$515
`
`«$15205
`
`zo_k<_oomm<
`
`mmoz<._.m>m
`
`
`
`w:<~:.5mm
`
`._<o_mOIa<._.ms_
`
`._<mw._._._
`
`Ex;
`
`
`
`
`
`
`éogmzémZOP<OEEZmQHZOHHmmUMmEm>F<MM<Z”ZOHH<UEHmm<1—OmQEOME
`
`
`
`
`
`
`FD<E>Em>EL<MM<Zmm<mMUZH
`
`H€050
`
`
`
`
`
`._<zmm:2_-ujmw._<_oOm
`
`
`
`
`
`m>_._.<mm<zwzfi<mm<zwm_._._z_nE<
`
`4<zo_._.Os_m
`
`mmEer<
`
`m>:._zoOo
`
`wm=:2_u_u_<
`
`
`
`mozmmflmmmwz_z_<._.z_<_>_
`
`
`
`>wO._On_OHhzmkzoo
`
`
`
`
`
`mmo<ozmOz>>Ozx.52>>Ozx152>>Ozx.52
`
`HzEzoo
`
`
`
`m_.r<_w_n_omn_n_.a~£925..2>>Ozx
`
`
`
`mmEFZmQ220m:
`
`
`
`-zOz555w:
`
`8m:E15
`
`Baal/é
`
`amtsmam
`
`
`
` wMHdfi—DQOQEOI>>_.._._._>>E:m>0m<j<E58
`
`
`
`
`
`
`
`
`
`mmIoEmoIQ:IQ:
`
`“_0oz_z<m_>_
`
`
`
`moz<o_u=zo_wmoz<o_..=zw_m
`
`._.zm._.zoo
`
`
`
`mh<_w_n_omn_n_<woz<o_n=zo_w
`
`moz<0_n=zw_m
`
`.mo+V
`
`3023,;
`
`mmGE
`
`
`
`._.zm._.zoowoz<o_n=zo_wmoz<o_u=29mHz<o_m_zo_m
`
`
`
`
`
`
`
`
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 29 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 29 of 39
`
`US 8,856,030 B2
`
`1
`METHOD, SYSTEM AND SOFTWARE FOR
`ASSOCIATING ATTRIBUTES WITHIN
`DIGITAL MEDIA PRESENTATIONS
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This patent application claims priority to the provisional
`patent application entitled, “Method and System for Digital
`Entertainment Asset Personalization,” filed Apr. 7, 2003,
`serial No. 60/460,998, which is incorporated by reference
`herein. This patent application is also related to the patent
`application entitled, “Method, System and Software for Digi-
`tal Media Narrative Personalization,” filed Apr. 7, 2004.
`
`BACKGROUND OF THE INVENTION
`
`It has been stated that human thinking falls into one of two
`categories: reasoning and narrative, descriptive, contempla-
`tive thinking. Reasoning is the process that permits the accep-
`tance of facts followed by a logical process to conclude a
`result, opinion, conclusion or decision. Narrative, descrip-
`tive, contemplative thinking corresponds to the storytelling
`process (including receiving of a story) and the thought pro-
`cesses associated with the reception of a story.
`Text and stories (presented in various formats) can also be
`interpreted at various levels including the literal, ethical, his-
`torical and mystical levels. It is generally understood that
`texts and stories are read and interpreted in different ways
`including at the different levels. A story or narrative can have
`different meanings at different levels, different interpreta-
`tions at different levels, and will be interpreted differently by
`different individuals at each of the levels. Additionally, an
`individual, based on their emotional or physical state, may
`interpret each level of a story differently. A person may react
`differently to a story based on their current emotional state,
`which may in fact change while they are receiving the story.
`In some cases, it may be possible to alter their emotional state
`while receiving the story to make them more receptive to a
`particular interpretation at one or more levels.
`Advertising is typically a type of narrative in which the
`advertiser attempts to influence the recipient (viewer, reader,
`listener) in a way that creates a positive association with a
`product or in some cases attempts to influence the recipient to
`purchase the product. Humor is frequently used in advertising
`because it puts the recipient in a receptive state for a message
`regarding the product. As such, humor is already used to alter
`the emotional state of the recipient of the advertising. Such
`manipulation of the emotional state of the recipient is cur-
`rently limited to simple manipulation through the presenta-
`tion of the advertisement (narrative).
`With the advent of the digital era, the number of ways to
`transmit and present advertisements has increased dramati-
`cally, with advertising being inserted into Web pages, being
`displayed as pop-up windows on Web pages, transmitted as
`streaming media over the Internet, presented as animation
`over the Internet, and inserted into television streams at cen-
`tral or distributed insertion points. The advent of the digital
`era also threatens advertising in that viewers can avoid adver-
`tisements by stopping pop-up windows, or skipping over the
`advertisements on a Personal Video Recorder (PVR).
`Due to the rapid changes in technology, advertisers are
`being forced to take new approaches to advertising including
`product placement, in which products are advertised during
`an entertainment narrative (e.g. a sit-com) as opposed to in a
`separate advertisement. Advertisers are also exploring ways
`to draw viewers (or listeners) to content that is essentially
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`
`digital media narrative storytelling, typically in the form of
`music or a short story that would appeal to viewers. What is
`required are ways to attract individuals to content that is
`personally more relevant and impactful for them and which
`may contain an advertising message (in the form of product
`placement), and have them receive that message in full, as
`opposed to skipping over all or a portion of the message. It
`would be an additional benefit to have the ability to under-
`stand the individual’s likes and dislikes or current mood in
`
`order to adapt the message appropriately for the individual at
`the time that they are receiving it.
`Digital media narrative may include content, such as video,
`audio, and combinations thereof, that is distributed digitally
`and can be stored digitally. In some instances digital media
`narrative can be manipulated via computer without human
`interaction. Digital media narrative includes but is not limited
`to stories presented over the Internet, modifiable digital video
`including digital television and streaming media, presenta-
`tions on web sites
`including animations,
`still
`images,
`sequences of still images, audio, textual presentations, and
`combinations of all of the abovementioned media.
`
`Digital media narrative can be distributed in a multitude of
`ways, such as broadcasting over cable and satellite, the Inter-
`net, or on media such as optical disks (CDs or DVDs). Dis-
`tribution of digital media narrative is a significant business
`and involves complex relationships between artists and
`record labels or film studios, producers, and a complete dis-
`tribution chain that includes advertising, both in the form of
`advertising of the content itself as well as sponsored adver-
`tising that may be inserted into, or become part of the content
`such as product placement. Examples of sponsored advertis-
`ing include the insertion of commercials into recorded mate-
`rials or the use of sponsor’ s products (e.g., automobiles) in the
`narrative.
`
`Digital media narratives may be directed at a segment of
`the market, but the segment that the digital media narrative
`material is directed to may be quite large. For example, an
`artist may create works that appeal to the 18-24 year old age
`group, but that group can include people with different inter-
`ests from various countries and different economic classes.
`
`Furthermore, when digital media narratives are used as part
`of advertising, that digital media narrative and the advertising
`may be directed at such a large market segment that it is
`difficult to keep the interest of individuals within that market
`segment. If a user goes to an Internet web site to view a
`recording of an artist, and in particular a sponsored work that
`contains advertising, that work (including the advertising)
`may be so generic that the user cannot respond to or relate to
`the work. Additionally, the user may experience different
`emotional states than another user, or the user may be in an
`emotional state that is different than the user’ s emotional state
`
`when the user previously viewed the material.
`For the foregoing reasons, there is a need for a method,
`system, and software to enable a distributor of sponsored or
`un—sponsored digital media narratives to change the content
`of the digital media narrative based on user demographics,
`psychodemographics, emotional states, afiinities (cognitive,
`emotional and social), self-narrating content classification,
`internal narrative traits preference topology, time sensitive,
`episodic expectation sequencing, and collective/collaborate
`attributes.
`
`SUMMARY OF THE INVENTION
`
`A method, system and software are presented which allow
`for customizing and personalizing content based on a com-
`bination of the user’s demographics, psychodemographics,
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 30 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 30 of 39
`
`US 8,856,030 B2
`
`3
`cognitive states, emotional states, social placement and group
`interaction dynamics within an online community, and/or
`affinity for certain content elements (images, sounds, seg-
`ments, graphics, Video, text, dialog), self provided narrating
`content,
`internal narrative traits preference topology, and
`expectation level and temporal spacing of assets within the
`narrative. In one embodiment, a user selects material for
`presentation and is presented with the dynamic digital media
`narrative and optional embedded advertising presented in a
`customized format that gives the user a personalized experi-
`ence.
`
`The system, method and software may support product
`placement type advertising by modifying both the product
`placement material as well as the narrative surrounding the
`product placement material. Appropriate advertising may be
`selected as well as selecting components for the narrative that
`supports the product placement advertising.
`In one embodiment, the user accesses a website containing
`sponsored material and logs on to that website. Information
`regarding the demographics, psychodemographics, cognitive
`states, emotional states, social placement and group interac-
`tion dynamics with the online community, and/or affinity for
`certain content elements of the user is retrieved from storage.
`If the user is a new user, he or she is asked a series of
`questions, preferably through a form of media narrative, that
`would creatively ask the questions, in order to obtain key
`information that the system would then classify and include
`into the user’s profile. These questions can include yes/no,
`multiple choice, like/dislikes ratings or any other type of
`personal assessment or individual and/or group question.
`Alternatively, the user may interact with the media narrative
`and the interactions may be used to create a user profile.
`In one embodiment, the user requests that the material be
`played or displayed, and is simultaneously presented with
`that material and customized personalized graphics and
`video. In another embodiment, the user can interact with the
`graphics and video. By making certain choices, a profile of
`the user can be determined and updated and the content (in-
`cluding background material, primary video, overlay graph-
`ics and audio) modified appropriately, thus giving the user an
`enhanced narrative, and more meaningful and emotionally
`satisfying experience with respect to the viewed content. In
`both embodiments, the user profile can then be used to present
`the most appropriate digital assets to the subscriber, namely
`those with which the user has the highest affinity, or those
`which map well to the user’s tendencies and temperament,
`which may be included in the user’s internal narrative per-
`ception identification framework. The internal narrative per-
`ception identification framework may include a collection of
`attributes, qualities, and measurements regarding the user that
`allow for matching, correlation and/or selection of digital
`media assets that are appropriate for that user and the effective
`communication of the message.
`In one embodiment, the user’s demographics are used to
`determine personalized content. Demographics may include,
`and are not limited to, group affiliations, regional location,
`sex, sexual preference, age, family situation (married or
`single, children or no children), race or ethnic background,
`religion and socioeconomic condition. The user’s demo-
`graphics may be used alone, or in combination with the pre-
`viously discussed techniques, to create the individual internal
`narrative perception identification framework.
`One advantage of an embodiment of the method, system
`and software presented herein is that the user is presented
`with an enhanced experience of the creator’s content that
`creates a greater emotional experience for the user and a more
`impactful narrative. This creates a more receptive state for
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`product placed advertising inserted into the content, for a
`direct advertisement, or for entertainment containing an
`advertising message. Another advantage of the method, sys-
`tem and software presented herein is that it allows advertising
`to be inserted in subtle ways and presented in a context in
`which users may be able to fully engulf themselves into the
`lifestyle being positioned and portrayed by the brand. By
`combining the personalized digital media narrative asset ele-
`ments with product placement advertising, a more powerful
`media is created for the advertisers and the artists. Addition-
`
`ally, since users typically volunteer to see the content, includ-
`ing advertising, they are much more likely to be receptive to
`the message presented, and less likely to skip over or fast-
`forward through the content including the advertising.
`In an alternate embodiment, users are presented with ques-
`tions or chat like text based opportunities within a narrative to
`either share the emotional experience with other users or to
`comment on their own emotional experience. This enables a
`user to share an experience with friends or acquaintances or
`allows a user to “sit back and relax” while someone else
`
`controls the enhanced experience. In this embodiment, the
`personalized content is based on the learned social dynamics
`of the user.
`
`Another embodiment includes a developer toolkit that
`allows a creative director to create trigger points in the mate-
`rial where digital assets such as overlay graphics, back-
`grounds, icons, text, sound, and/or product placement images
`can be inserted into the digital media narrative. The developer
`toolkit allows the creative director to create a map of the
`elements of the user’s profile and his or her internal narrative
`perception identification framework topology to digital assets
`that may be created. The developer toolkit also provides for
`the ability to insert interactive material such as questions or
`interactive graphics, allowing for the updating and enhance-
`ments of the user profile based on the user’ s interactions. The
`toolkit further permits the creative director to create and link
`the digital assets to the material such that assets for which the
`user has a high degree of affinity or assets having a highest
`probability for matching a user’ s internal narrative perception
`identification framework topology can be selected at the trig-
`ger point.
`One advantage of an embodiment of the method, system
`and software presented herein is that they may enable a cen-
`tral server or service to compile a user’s internal narrative
`perception identification framework and to store the internal
`narrative perception identification framework as an internal
`narrative perception identification framework profile and
`allow this profile to track or follow the user as he or she
`peruses multiple digital media narratives. This enables the
`user to receive enhanced digital media assets and various
`narrative experiences and allows the service to be sold to
`advertisers that advertise on any web site or broadband IP-
`based distribution point.
`These and other features and objects of the invention will
`be more fully understood from the following detailed descrip-
`tion of the embodiments which should be read in light of the
`accompanying drawings.
`In this respect, before explaining at least one embodiment
`of the invention in detail, it is to be understood that the
`invention is not limited in its application to the details of
`construction and to the arrangements of the components set
`forth in the description or illustrated in the drawings. The
`invention is capable of other embodiments and of being prac-
`ticed and carried out in various ways. Also, it is to be under-
`stood that the phraseology and terminology employed herein,
`as well as the abstract, are for the purpose of description and
`should not be regarded as limiting.
`
`
`
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 31 of 39
`Case 6:20-cv-00810-ADA Document 1-1 Filed 09/02/20 Page 31 of 39
`
`US 8,856,030 B2
`
`5
`As such, those skilled in the art will appreciate that the
`concepts upon which this disclosure is based may readily be
`used as a basis for designing other structures, methods, and
`systems for carrying out the several purposes of the present
`invention.
`It
`is important,
`therefore,
`that
`the claims be
`regarded as including such equivalent constructions insofar
`as they do not depart from the spirit and scope of the present
`invention.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The accompanying drawings, which are incorporated in
`and form a part ofthe specification, illustrate embodiments of
`the present invention and, together with the description serve
`to explain the principles of the invention.
`FIG. 1 illustrates a user-relationship diagram for an
`embodiment of the system;
`FIG. 2 illustrates the representative components ofa digital
`media narrative and the digital media assets;
`FIG. 3 illustrates the creation of a personalized digital
`media narrative;
`FIG. 4 illustrates the use of interactivity to create an
`enhanced user profile and an enhanced experience through a
`personalized digital media asset;
`FIG. 5A illustrates a context diagram for digital media
`narrative asset personalization;
`FIG. 5B illustrates potential databases that may be used in
`creation of the personalized digital media asset and the rela-
`tionships between those databases;
`FIG. 6 illustrates exemplary relationships between the
`social, emotional, and cognitive affinity elements;
`FIG. 7 illustrates exemplary components of the cognitive
`element;
`FIG. 8 illustrates exemplary components of the social ele-
`ment;
`FIG. 9 illustrates exemplary compon