Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 1 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 1 of 39
`
`EXHIBIT 1
`EXHIBIT 1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 2 of 39
`ee= STATETETETAT
`
`US008856030B2
`
`US 8,856,030 B2
`(10) Patent No.:
`a2) United States Patent
`Russek
`(45) Date of Patent:
`Oct. 7, 2014
`
`
`(54) METHOD, SYSTEM AND SOFTWARE FOR
`ASSOCIATING ATTRIBUTES WITHIN
`DIGITAL MEDIA PRESENTATIONS
`
`(75)
`
`Inventor: David J. Russek, Maple Glen, PA (US)
`
`(73) Assignee: Sevenecho, LLC, Norristown, PA (US)
`(*) Notice:
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`USC. 154(b)
`by
`1841 da
`S.C.
`154(b) by
`ys.
`
`.
`(21) Appl. No.: 10/819,514
`(22)
`Filed:
`Apr. 7, 2004
`
`5,692,212 A
`5,725,472 A
`5,734,795 A
`5,751,953 A
`5,754,787 A *
`5,774,664 A
`5,805,784 A
`5,848,934 A
`ae A
`an
`5,913,310 A
`5,918,603 A
`5,935,003 A
`5,945,988 A
`
`11/1997 Roach
`3/1998 Weathers
`3/1998 Rogers
`5/1998 Shiels
`5/1998 Dedrick Peete neces enenenenens 709/228
`6/1998 Hidary
`9/1998 Crawford
`12/1998 Shiels
`ees ene
`e's
`6/1999 Brown
`7/1999 Brown
`8/1999 Stephens
`8/1999 Williams
`(Continued)
`
`US 2004/0199923 Al
`
`Oct. 7, 2004
`
`Wo
`
`0177939 Al
`
`10/2001
`
`Related U.S. Application Data
`(60) Provisional application No. 60/460,998, filed on Apr.
`7, 2003.
`
`51)
`(51)
`
`(2012.01)
`(2012.01)
`(2006.01)
`
`Int. Cl.
`G0630/02
`G0630/00
`G06F 1730
`(52) U.S.CL
`CPC oo. GO06F 1730867 (2013.01); G06Q 30/00
`(2013.01); GO6F 17/30265 (2013.01)
`USPC oiccceeeeeeeeseeseeseeceeeeeeeees 705/14.66; 705/319
`(58) Field of Classification Search
`CPC voecececceeceteeteeteeses G06Q
`50/01; GO6Q
`30/0269
`USPC
`Q
`705M14 2 66.319
`See application file for complete search history. ;
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5,119,474 A
`5,586,967 A
`5,604,855 A
`5,625,570 A
`
`6/1992 Beitel etal.
`12/1996 Davis
`2/1997 Crawford
`4/1997 Vizireanu
`
`OTHER PUBLICATIONS
`oo
`;
`;
`;
`;
`Quintana, Y.: “Design of an Object-Oriented Multimedia Database
`for Personalize Multimedia News”, Proc. 1996 Canadian Conf. on
`Electrical and Computer Engg., May 26, 1996-May 29, 1996; pp.
`282-285, XP002460458, Calgary, Canada.
`
`Primary Examiner — Donald L. Champagne
`(74) Attorney, Agent, or Firm — Pepper Hamilton LLP
`
`(57)
`ABSTRACT
`Disclosed are a system, method and software to associate
`attributes with digital media assets. Digital media contains
`specific assets, such as images, that can be replaced with other
`assets. The system, method and software permit the associa-
`tion of attributes with specific assets. The association of
`attributes and assets enables the provision of content that is
`enhanced and more impacting for a user.
`
`2 Claims, 25 Drawing Sheets
`
`
`
`541
`
`
` ys “9
`
`content|*534
`
`
`Sr”
`3 as gs
`SY
`os"
`
`%
`x
`g
`<<of o Oa
`6] piGtTAL ASSET
`
`RID “520 nesa
`
`ONLINE—|coMMUNITY USER ea
`COMMUNITY[aorivE Vv, INACTIVES
`521
`3
`
`
`
`REPOSITORY
`
`
`STORED USER
`PROFILE.
`
`
`
`
`
`551
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 3 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 3 of 39
`
`US 8,856,030 B2
`
`Page 2
`
`(56)
`
`References Cited
`
`1/2003 Uwakubo
`6,513,011 Bl
`1/2003 Abato
`6,513,069 Bl
`2/2003 Surve
`6,520,905 Bl
`2/2003 Morris
`6,526,395 Bl
`3/2003 Manico
`6,527,700 Bl
`11/1999 Le Blanc
`5,977,968 A
`3/2003 Chase
`6,529,864 Bl
`11/1999 Breese
`5,987,415 A
`4/2003 Flannery
`6,545,209 Bl
`12/1999 Roach
`5,999,172 A
`6/2003 Pak
`6,581,037 Bl
`2/2000 Herz
`6,029,195 A
`7/2003 Obrador
`6,585,521 Bl
`6/2000 Martin
`6.078.936 A
`8/2003 Sadakuni
`6,604,091 B2
`7/2000 Reed
`6.088.717 A
`2/2006 Balthaser
`7,000,180 B2
`9/2000 Hin
`6.120.846 A
`1/2007 Mascarenhas
`7,162,432 B2
`10/2000 Bowman
`6.135.944 A
`
`
`6,175,772 Bl 7,483,871 B2*=1/2009 Herz weecceccecccccsceseeereenenene V11/2001 Kami
`6.190.314 Bl
`2/2001 Ark
`8,635,649 B2*
`1/2014 Wardetal. ow. 725/40
`6.205.472 Bl
`3/2001. Gilmour
`2001/0049588 Al
`12/2001 Bauschetal.
`6.222.925 Bl
`4/2001. Shiels
`2002/0013943 Al
`1/2002 Haberman
`6.249.780 Bl
`6/2001. Mizokawa
`2002/0092019 Al
`7/2002 Marcus
`6,293,904 Bl
`9/2001 Blazey
`2002/0122430 Al
`9/2002 Haberman
`6.306.077 BL
`10/2001 Prab
`2002/0159640 Al
`10/2002 Vaithilinga
`6.311.194 Bl
`10/2001 Sheth
`2003/0001880 Al
`1/2003 Holtz etal.
`6,317,127 BL
`11/2001 Daily
`2003/0009078 Al
`1/2003 Fedorovskaya
`6,330,595 Bl
`12/2001 Ullman
`2003/0036899 Al
`2/2003 Leslie etal.
`
`6,345,288 Bl 3/2003_Haberman2/2002 Reed 2003/0045957 Al
`
`
`6,357,042 B2
`3/2002 Srin
`2003/0131351 Al
`7/2003 Shapira
`6,421,669 Bl
`7/2002 Gilm
`2003/0142689 Al
`7/2003 Haberman
`6,425,825 BL
`7/2002 Sitrick
`2004/0267816 Al
`12/2004 Russek
`6,466,232 Bl
`10/2002 Newell
`6,499,741 B2
`12/2002 Yamazaki
`
`U.S. PATENT DOCUMENTS
`
`* cited by examiner
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 4 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 4 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 1 of 25
`
`US 8,856,030 B2
`
`YOSNOdS80LYOLOAYIG
`
`Lor|
`
`
`
`
`
`FAILVSYO
`
`|
`
`
`
`LOLI
`
`001
`
`YOLNEIYLSIO
`
`
`
` yeeyLN3LNO9|ZLL
`
`ONVAlaWaASsy
`
`NOILNGIYLSIG
`
`TWNIDINO
`
`LNALNOO
`
`
`
`INSLNODmuad1s1GOWYAWNSNOD
`
`901
`
`L‘OW
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 5 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 5 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 2 of 25
`
`US 8,856,030 B2
`
`
`
`VIGAWIWLIDIO
`
`SLISSVZL
`
`@Old
`
`ONIGNVYS
`
`SOIHdVUD902
`
`GNnoYwoaYOS
`
`SaOvMI
`
`C0C
`
`W1dId
`
`OC
`
`
`
`"AWTHSAO)LXSL
`
`‘ONITIONDSLX3l
`
`(dSy¥SVW802
`
`r0zae!S3ONaNODISSS9VWI|,002
`
`O3QIAaNnowoMovEa|iL/
`
`
`
`|
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 6 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page6 of 39
`
`Sheet 3 of 25
`
`US 8,856,030 B2
`
`U.S. Patent
`
`SOIYAT8SISNW
`
`Oct. 7, 2014
`
`CAZMWNOSY3d||!|!!bg1ivp0eONY
`AMOLSYaSNFLLerlwTTT
`(QWNUSLNI)|!!by!} SNOILOWS
`
`gpg+-----PnbobotootTWNYSLNI
`ti!!]ytSAV
`
`wmwo————Tbe
`
`
`
`
`
`=aeAnen:oze160!)tee
`nD|Vv
`
`
`
`Acccncenncnneenecesceeconeeneenecennaneeeesonecenpooneeenecesseesneseneesescssenssceseeesceeseeseece!QoyctT@|forbes)
`
`G3ZIVNOSuad|MSN!SSOvMI
`Anmainl:' SNOILOWS
`
`
`(VNYSLNI)OLE.ONY
`
`MAN
`
`€‘OH
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 7 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 7 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 4 of 25
`
`US 8,856,030 B2
`
`Qa
`uy LU
`NI ©
`a Zz
`<< Ww
`=<
`
`© th
`
`wx
`LW lw
`oO
`
`DEFAULT
`
`EXPERIENCE
`
`ENHANCEDUSER
`
`“wt
`.
`)
`~—
`
`PROFILE
`
` N
`oO
`2
`st
`co
`oO
`ws
`oO
`oO
`=
`—t
`=
`
`INTERACTIVE
`
`OPPORTUNITIES
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 8 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 8 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 5 of 25
`
`US 8,856,030 B2
`
`bes
`
`INSLNODYasn
`
`YsAd3sS06S
`
`LOS
`
`2quY3aSNALINAWWOD
`
`
`ALINAWWOS
`
`SNNNO
`
`
`
`LSSLNALNOOCaisIGOW
`
`JOVYOLS
`
`YdSNGAYOLS
`
`ANdOud
`
`VS‘Sid
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 9 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 9 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 6 of 25
`
`US 8,856,030 B2
`
`CNYSNIINO31d03dSH40SUSI
`
`
`
`‘ONIONANDASLASSV
`
`ALINOASGNVONIWIL
`
`20S
`
`|SONaIany
`
`YOLINOW
`
`
`
`
`
`LHONOYE3dOLSLNSWNOOYISHL|SONAINAdXAYASNSHLSIOHM
`
`SINIOdH399IULSVNIHLIMONILOANNOO
`
`LaSSYWISICINGOWNdlmn|ONITSAVELves
`bbsZ4S00Sdf
`
`
`qasvgIvLloldoS
`dOSYNLVNONIATOAGNVdnow90zS
`
`(iv)AYOLSWId0SAurunoas
`
`
`
`81S“7soINWNAG7725
`r0sNINO
`
`
`9bS
`
`AYOLISOdSY
`
`d¥WOI
`
`gSOld
`
`
`
`ONVNOILOWSC3LSYdysLNI
`
`
`
`
`
`ONIddVWVLVGACISLNO
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 10 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 10 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 7 of 25
`
`US 8,856,030 B2
`
`oO
`oS
`CO
`
`LJ
`
`>E=©o
`
`O
`oO
`
`>CO
`
`aO
`
`S
`CO
`
`a|
`
`—z SEO
`
`o=W
`
`u
`
`FIG.6
`
`N©o
`
`O
`
`—_l
`
`a©
`
`O
`oO
`wo”
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 11 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 11 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 8 of 25
`
`US 8,856,030 B2
`
`NOILISINDOV
`
`MAN4O
`
`NOILVAOSNI
`
`8bZ
`
`9b
`
`~0¢2
`
`COL
`
`£09
`
`702
`
`‘Ways
`
`ONISSSOOUd
`
`N3LLIM
`
`JSVNONVI
`
`INOILISINDOV
`
`NOILINDOO3Y
`
`vb
`
`CbZ
`
`OZ
`
`/ONIONVLSYSaNNn
`
`NOILONGOYdSy
`
`HO33dS
`
`902
`
`802
`
`Z‘Ol
`
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 12 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 12 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 9 of 25
`
`US 8,856,030 B2
`
`YYOMANVYS
`
`
`
`(so63Jayy)
`
`SALLYUMYN
`
`028
`
`TWNYSLNISdLTA209
`
`NOILYOISILNZGI
`
`NOLLda0u3dc08
`
`
`
`819NOLd30uad
`
`NOILVOISILNZGI
`
`YYOMANVY
`
`08
`
`WIO0$
`
`918
`
`SdlHSNOILV1Sy
`
`ZL8
`
`
`
`SVNOSY3dTVIOOS
`
`
`
`anlgoiadajdoadMOH)
`
`(Ajjeloosnok
`
`
`
`4OS3GOW908
`
`p18
`
`NOLLOVYSLNI
`
`808
`
`O18®SAILINIdV
`
`VIL
`
`8‘Sid
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 13 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 13 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 10 of 25
`
`US 8,856,030 B2
`
`(SGNSI¥4)
`
`TWNOSYSd
`
`VL6
`
`WOIMOLSIH
`
`wantingJIHdVYDONOD!
`
`ateTWNOSYAad
`
`WaNLINO
`
`TWOIYOLSIH
`
`cb6806
`
`AYXOLOVIIO
`
`NOILda9u3desAMOLIGNY
`
`Ol
`6
`
`{vnanwHLMON
`
`WI908NVOMSNY
`
`916
`
`
`
`109Cissouna)606
`
`6‘SO
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 14 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 14 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 11 of 25
`
`US 8,856,030 B2
`
` CREATIVE DIRECTOR
`
`
`LISTENS/WATCHES
`CONTENT
`
` 1000
`
`
`TAG TIME INDICES
`WITHAFFINITIEs
`
` |~ 1010
`
` EDIT AFFINITIES
`
`
`
`
`
`
`
`COALESCE MEANING
`IDENTIFY MAPPING
`
`STORE TAGS
`
`
`
`CREATE INTERACTIONS
`
`DEVELOP COLLATERAL
`MATERIALS
`
`FIG. 10
`
`1030
`
`1040
`
`1050
`
`1060
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 15 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 15 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 12 of 25
`
`US 8,856,030 B2
`
`
`LLbLJONVLYOdINI|—NMONYANA/NMONY|LNAWST13311d08ddl¥aSn
`
`
` |AOLS|SSONWIddV|S1622E|pTSONVSSIVNS|SNOZLEPpELLY||Sb6C26|hejNW|IMWATSTISONd|cl43SNLNAWSTS
`
`LOLLAWWAQYOMSSVddi¥asn
`
`
`LANdSI@HLINSHSIS09GL6CZE
`
`
`WNIdaWNMONM|=—OSLWshZZE
`
`NMONYNNsauldGLEE
`
`bb‘O
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 16 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 16 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 13 of 25
`
`US 8,856,030 B2
`
`AFFINITY LIST
`GET AFFINITIES
`SET AFFINITIES
`
`1210
`
`EXPERIENCE
`
`
`
`PERSON
`1200
`
`
`
`
`
`PEOPLE GROUP
`SET OF PEOPLE
`
`
`1220
`
`
`
`
`GROUP OF PEOPLE
`RECORDED AFFINITIES
`MEDIA
`
`
`
`
`
`
`1230~'RECORDEDAFFINITIES
`
`
`TYPE
`
`AFFINITY
`
`
`AFFINITY
`
`
`NAME
`
`DESCRIPTION
`
`
`PARENT
`CHILD
`
`GROUP
` FIG. 12
`
`1240
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 17 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 17 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 14 of 25
`
`US 8,856,030 B2
`
`
`
`AAIGYVH
`
`O0€L
`
`YATIONLNOD
`
`
`
`ADIAAILYOdWId3as
`
`YOLINOW
`GuvOgAsy
`
`Wous/Ol
`
`WOUYs/OL
`WOUYs/OL
`
`WOus/OL
`
`MYOMLAN
`
`SLOTSVSI
`
`eb‘Sid
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 18 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 18 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 15 of 25
`
`US 8,856,030 B2
`
` 0
`
`2:56
`| sd 1430
`
`AUDIO
`
`ST 1Jro 142
`
`BACKGROUND 1
`
`ST2
`
`BACKGROUND 2
`
`1434
`
`IT 1| e -—___.-146
`
`TEXT1
`
`IT 2
`
`|_———_---__—_______—__—}- 1438
`
`TEXT 2
`
`FIG. 14
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 19 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 19 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 16 of 25
`
`US 8,856,030 B2
`
` 3&5aae
`
`
`
`
`
`
`
`IW]NVONIMSIAOLYOlddYSLSIOSYLSNWYSEWSWSONSIGNYNV
`
`St‘Old
`
`
`
`
`
`THWideiseinga
`
`43°:
`
`[=]£&a=&ax=3=
`
`le
`
`
`
`
`
`SUSEMiqaeUs(0907an)
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 20 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 20 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 17 of 25
`
`US 8,856,030 B2
`
`
` “SSOIOHDSAILOVESLNIWOdSGSLVSeOSevLVHLQL‘Old
`
`
`
`
`S3MlsdOdNOdNGASVYASNYadGANINYSLAGSILNALNODGaziIvNOSuadSINVNAGSHL
`
`
`
`
`
`GU02auyyseUWbeee)(al
`
`
`
`
`
`
`
`
`Saag
`
`Oey
`
`“#2&
`
`raz2a
`
`
`
` iRa3a=eza“a.2o2z5“|£5eww
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 21 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 21 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 18 of 25
`
`US 8,856,030 B2
`
`| Favorites History
`
`sy,
`‘ye Search”
`“
`
`383
`
`os,
`ay4
`
`“Pogeltolder; |
`
`mAiS
`
`
`
`RMWidFLgSetg
`
`
`
`
`
`
`
`
`
`TIAMSYNOILOV30ONIGN318SHLSSCNTONISAILLVSYOIISHL
`
`
`
`‘WeOdOF0IAWLISIGNISV
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 22 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 22 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 19 of 25
`
`US 8,856,030 B2
`
`Old
`
`
`
`
`
`CRB
`
`ON
`
`8h
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 23 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 23 of 39
`
`U.S. Patent
`
`US 8,856,030 B2
`
`Oct. 7, 2014
`
`Sheet 20 of 25
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 24 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 24 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 21 of 25
`
`US 8,856,030 B2
`
`0¢‘Old
`
`
`
`7C
`
`
`
` GavHaWLH—ANOLANOLANOLaud
`MSOT,4|i
`
`xasanianilo
`
`
`83Y39Hd1gH3WOS199|N¥DIA3H
`
` eenypenmeneame
`
`002
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 25 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 25 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 22 of 25
`
`US 8,856,030 B2
`
`
`
`
`
`
`
`puodsaraseaqy{QYIoyModareyeUyy,
`
`azogaq
`
`
`
`“puaqaamSng[TdayULM[Tum]auotianaapy
`
`
`
`
`
`“Seaptyear;autospyyianuaywoArepyf
`
` ifepirg
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 26 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 26 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 23 of 25
`
`US 8,856,030 B2
`
`ALVAYO
`
`MAN
`
`d11d0ud
`
`NI938
`
`JONAlYAdxS
`
`OVLLVLOVYSLNI
`
`
`
`LAdNIACIAO¥d
`
`
`
`Q3LVSYOHOIHM
`
`3114d0ud
`
`Vee9
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 27 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 27 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 24 of 25
`
`US 8,856,030 B2
`
`RAG
`
`VWYOLOSHIG
`
`NOILVOISISSV19D
`
`
`
`
`
`NOILWINdOd3114dOUdTYNGIAIONI
`
`
`
`
`
`dOSYNLXINYSSNtOLSLAANINOILWATWAZ
`
`
`
`
`
`LONI'?SNOILOVYSALNIJONZIGNY
`
`
`
`SLAANIAZITVWYON
`
`0S¢¢
`
`
`
`
`
`
`
`NOILVOIEISSW1DLASSYTWLISIGALWNIWAS
`
`
`
`NOILOVLASSVVIGAWTWLISICGSOALIMONd
`
`
`
`
`
`
`
`TWNGIAIGN!HOVAOLNOILLOVVIGdv
`
`
`
`
`
`J14OYUdMOAVHSTVLISIO40ALIMOldd
`
`SLNAW313
`
`
`
`VIG40SYNLXIWYASANIOLYOLOANIGAG
`
`
`
`AONSY3s3udLassv
`
`0822
`
`
`
`
`
`
`
`OLONIQYODOV31sOYUdMOGVHSTVLISIG
`
`
`
`
`
`AON3YS3AadHOSALIMGYdOUdLSAHSIH
`
`
`
`
`
`NOILVOISILNAGIAONAIGNYGSONVHNAONY
`
`
`
`
`
`
`
`SLNAW313A1IOYddMOUVHSIWLUSIG
`
`0922
`
`0222
`
`G8c¢
`
`0622
`
`G6CC
`
`22Dd
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 28 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 28 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 25 of 25
`
`US 8,856,030 B2
`
`
`
`
`
`WYOMANVAANOILVOMILNA]NOILdaOuddJAILVUAVN‘NOILVOMISSVTOAMAOUd
`
`
`ONIONSNDAS
`AQ010d0L
`
`LINSLNO9
`
`ATWIOO0S
`
`(39V9N3
`
`daANl43q
`
`
`
`(N/A)SNOW
`
`HLIM
`
`WwunLing
`
`LNALNOO
`
`YAONVLSAG
`
`ALVlddOuUddvNO)MOTNMON
`
`JONaNoDaS
`
`YADVONAONMONLONMONLONMON»LON
`
`LNSWS9VONA
`TWIDOSON
`NOILVL93dx3
`
`YOHOASd
`
`
`
`
`
`
`
`LINSLNOSJONVOISINSISJONVOIAINDISLNVOISINSIS
`
`SAWOOLNO
`
`
`
`alVlddOudd¥FONVIISINSIS
`
`JDNVOISINS!S
`
`
`
`YAlsILNAdlWnIdaw
`
`-NONWwnidsn
`
`(N/A)
`
`“NON
`
`AividdOUdd¥
`
`
`
`OudLNALNOO
`
`
`
`(N/A)SHSHLO
`
`INALNOO
`
`3LVIdOUdd
`AINSLNI8QNIDNAIWA
`
`SYSHLOWOU
`NOILVIOOSSYV
`
`IN3ILNI
`
`West
`
`
`
`Sd13l4YSHLO
`
`dayunidv9)
`
`N3HM
`
`(daLLIWaNS
`
`LINSW39VONA
`AWOOLNO2SONVINVA
`SATEVINVA
`WOMOHdVL3SW
`
`LVHLaA0gyTv
`
`WISOS
`
`LNALNOOON
`
`SYSHLOWONS
`
`4OSONVAI13Y
`ISNOILWLOAdxX3
`
`‘33dS
`
`NOLLOW
`
`LINSLNOOD
`
`
`
`YsHOVLIGHOIHHOIH
`
`JOONINVAN
`
`INALNOO
`
`
`
`JONVIISINSISJONVOISINDIS
`
`
`
`S3LV1NdOdWOHMHLIM
`
`-40+)
`
`(AONATWA
`
`&¢Sid
`
`TWNOILOWS
`
`SILINISSV
`
`JAILINDOO
`
`SAILINISSV
`
`JAILVeORVTIOO
`NOILVLOAdx33D100SId3fAAILISNAS
`SONIYAIIYd
`
`SLVaL4138
`ONINIVLNIV
`
`
`
`
`
`AAILVYYVNONILWYYVNS3ILINISIV
`
`AAILOSTION
`
`AWIL
`
`
`
`TWNYALNI“3148WI90S
`
`
`
`
`
`LOVdWIFAILVaaVNASVAAONI
`
`“IWOD
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 29 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 29 of 39
`
`US 8,856,030 B2
`
`1
`METHOD, SYSTEM AND SOFTWARE FOR
`ASSOCTIATING ATTRIBUTES WITHIN
`DIGITAL MEDIA PRESENTATIONS
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This patent application claimspriority to the provisional
`patent application entitled, “Method and System for Digital
`Entertainment Asset Personalization,” filed Apr. 7, 2003,
`serial No. 60/460,998, which is incorporated by reference
`herein. This patent application is also related to the patent
`application entitled, “Method, System and Software for Digi-
`tal Media Narrative Personalization,” filed Apr. 7, 2004.
`
`BACKGROUND OF THE INVENTION
`
`It has been stated that human thinking falls into one of two
`categories: reasoning and narrative, descriptive, contempla-
`tive thinking. Reasoning is the process that permits the accep-
`tance of facts followed by a logical process to conclude a
`result, opinion, conclusion or decision. Narrative, descrip-
`tive, contemplative thinking corresponds to the storytelling
`process (including receiving of a story) and the thought pro-
`cesses associated with the reception ofa story.
`Text and stories (presented in various formats) can also be
`interpreted at various levels including theliteral, ethical, his-
`torical and mystical levels. It is generally understood that
`texts and stories are read and interpreted in different ways
`includingat the different levels. A story or narrative can have
`different meanings at different levels, different interpreta-
`tionsat different levels, and will be interpreted differently by
`different individuals at each of the levels. Additionally, an
`individual, based on their emotional or physical state, may
`interpret each level of a story differently. A person may react
`differently to a story based on their current emotionalstate,
`which may in fact change while they are receiving the story.
`In somecases, it may be possible to alter their emotionalstate
`while receiving the story to make them more receptive to a
`particular interpretation at one or morelevels.
`Advertising is typically a type of narrative in which the
`advertiser attempts to influence the recipient (viewer, reader,
`listener) in a way that creates a positive association with a
`productor in some cases attempts to influencethe recipient to
`purchasethe product. Humoris frequently used in advertising
`becauseit puts the recipient in a receptive state for a message
`regarding the product. As such, humoris already usedto alter
`the emotional state of the recipient of the advertising. Such
`manipulation of the emotional state of the recipient is cur-
`rently limited to simple manipulation through the presenta-
`tion of the advertisement(narrative).
`With the advent of the digital era, the number of ways to
`transmit and present advertisements has increased dramati-
`cally, with advertising being inserted into Web pages, being
`displayed as pop-up windows on Webpages, transmitted as
`streaming media over the Internet, presented as animation
`overthe Internet, and inserted into television streamsat cen-
`tral or distributed insertion points. The advent ofthe digital
`era also threatens advertising in that viewers can avoid adver-
`tisements by stopping pop-up windows, or skipping over the
`advertisements on a Personal Video Recorder (PVR).
`Due to the rapid changes in technology, advertisers are
`being forced to take new approachesto advertising including
`product placement, in which products are advertised during
`an entertainmentnarrative (e.g. a sit-com) as opposedto ina
`separate advertisement. Advertisers are also exploring ways
`to draw viewers (or listeners) to content that is essentially
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`digital media narrative storytelling, typically in the form of
`music or a short story that would appeal to viewers. Whatis
`required are ways to attract individuals to content that is
`personally more relevant and impactful for them and which
`may contain an advertising message (in the form of product
`placement), and have them receive that message in full, as
`opposed to skipping overall or a portion of the message. It
`would be an additional benefit to have the ability to under-
`stand the individual’s likes and dislikes or current mood in
`order to adapt the message appropriately for the individualat
`the timethat theyare receivingit.
`Digital media narrative may include content, such as video,
`audio, and combinations thereof, that is distributed digitally
`and can be stored digitally. In some instances digital media
`narrative can be manipulated via computer without human
`interaction. Digital media narrative includes butis not limited
`to stories presented overthe Internet, modifiable digital video
`including digital television and streaming media, presenta-
`tions on web sites
`including animations,
`still
`images,
`sequences ofstill images, audio, textual presentations, and
`combinationsofall of the abovementioned media.
`
`Digital media narrative can be distributed in a multitude of
`ways, such as broadcasting over cable andsatellite, the Inter-
`net, or on media such as optical disks (CDs or DVDs). Dis-
`tribution of digital media narrative is a significant business
`and involves complex relationships between artists and
`record labels or film studios, producers, and a complete dis-
`tribution chain that includes advertising, both in the form of
`advertising of the content itself as well as sponsored adver-
`tising that maybeinserted into, or becomepart ofthe content
`such as product placement. Examples of sponsored advertis-
`ing includethe insertion of commercials into recorded mate-
`rials or the use of sponsor’s products (e.g., automobiles) in the
`narrative.
`
`Digital media narratives may be directed at a segment of
`the market, but the segment that the digital media narrative
`material is directed to may be quite large. For example, an
`artist may create works that appeal to the 18-24 year old age
`group, but that group can include people with different inter-
`ests from various countries and different economicclasses.
`
`Furthermore, when digital media narratives are used as part
`of advertising, that digital media narrative andthe advertising
`may bedirected at such a large market segment thatit is
`difficult to keep the interest of individuals within that market
`segment. If a user goes to an Internet web site to view a
`recording ofan artist, and in particular a sponsored work that
`contains advertising, that work (including the advertising)
`maybeso generic that the user cannot respondto orrelate to
`the work. Additionally, the user may experience different
`emotional states than another user, or the user may be in an
`emotionalstate that is different than the user’s emotionalstate
`whenthe user previously viewed the material.
`For the foregoing reasons, there is a need for a method,
`system, and software to enable a distributor of sponsored or
`un-sponsored digital media narratives to change the content
`of the digital media narrative based on user demographics,
`psychodemographics, emotionalstates, affinities (cognitive,
`emotional and social), self-narrating content classification,
`internal narrative traits preference topology, time sensitive,
`episodic expectation sequencing, and collective/collaborate
`attributes.
`
`SUMMARY OF THE INVENTION
`
`A method, system and software are presented which allow
`for customizing and personalizing content based on a com-
`bination of the user’s demographics, psychodemographics,
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 30 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 30 of 39
`
`US 8,856,030 B2
`
`3
`cognitive states, emotionalstates, social placement and group
`interaction dynamics within an online community, and/or
`affinity for certain content elements (images, sounds, seg-
`ments, graphics, video, text, dialog), self provided narrating
`content,
`internal narrative traits preference topology, and
`expectation level and temporal spacing of assets within the
`narrative. In one embodiment, a user selects material for
`presentation and is presented with the dynamic digital media
`narrative and optional embedded advertising presented in a
`customized format that gives the user a personalized experi-
`ence.
`
`The system, method and software may support product
`placement type advertising by modifying both the product
`placement material as well as the narrative surrounding the
`product placement material. Appropriate advertising may be
`selected as well as selecting componentsfor the narrative that
`supports the product placement advertising.
`Tn one embodiment, the user accesses a website containing
`sponsored material and logs on to that website. Information
`regarding the demographics, psychodemographics, cognitive
`states, emotionalstates, social placement and group interac-
`tion dynamics with the online community, and/oraffinity for
`certain content elements oftheuseris retrieved from storage.
`If the user is a new user, he or she is asked a series of
`questions, preferably through a form of media narrative, that
`would creatively ask the questions, in order to obtain key
`information that the system would then classify and include
`into the user’s profile. These questions can include yes/no,
`multiple choice, like/dislikes ratings or any other type of
`personal assessment or individual and/or group question.
`Alternatively, the user may interact with the media narrative
`and the interactions may be usedto create a userprofile.
`In one embodiment, the user requests that the material be
`played or displayed, and is simultaneously presented with
`that material and customized personalized graphics and
`video. In another embodiment, the user can interact with the
`graphics and video. By making certain choices, a profile of
`the user can be determined and updated and the content(in-
`cluding background material, primary video, overlay graph-
`ics and audio) modified appropriately, thus giving the user an
`enhancednarrative, and more meaningful and emotionally
`satisfying experience with respect to the viewed content. In
`both embodiments, the user profile can then be usedto present
`the most appropriate digital assets to the subscriber, namely
`those with which the user has the highest affinity, or those
`which map well to the user’s tendencies and temperament,
`which may be included in the user’s internal narrative per-
`ception identification framework. The internal narrative per-
`ception identification framework may include a collection of
`attributes, qualities, and measurements regardingthe userthat
`allow for matching, correlation and/or selection of digital
`media assets that are appropriate for that user and the effective
`communication of the message.
`In one embodiment, the user’s demographics are used to
`determine personalized content. Demographics mayinclude,
`and are not limited to, group affiliations, regional location,
`sex, sexual preference, age, family situation (married or
`single, children or no children), race or ethnic background,
`religion and socioeconomic condition. The user’s demo-
`graphics may be used alone, or in combination with the pre-
`viously discussed techniques, to create the individual internal
`narrative perception identification framework.
`One advantage of an embodiment of the method, system
`and software presented herein is that the user is presented
`with an enhanced experience of the creator’s content that
`creates a greater emotional experience for the user and amore
`impactful narrative. This creates a more receptive state for
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`product placed advertising inserted into the content, for a
`direct advertisement, or for entertainment containing an
`advertising message. Another advantage of the method, sys-
`tem and software presented herein is thatit allows advertising
`to be inserted in subtle ways and presented in a context in
`which users may be able to fully engulf themselves into the
`lifestyle being positioned and portrayed by the brand. By
`combining the personalized digital media narrative asset ele-
`ments with product placement advertising, a more powerful
`media is created for the advertisers and the artists. Addition-
`
`ally, since users typically volunteerto see the content, includ-
`ing advertising, they are much morelikely to be receptive to
`the message presented, and less likely to skip over or fast-
`forward through the content including the advertising.
`In an alternate embodiment,users are presented with ques-
`tionsor chatlike text based opportunities within a narrative to
`either share the emotional experience with other users or to
`comment on their own emotional experience. This enables a
`user to share an experience with friends or acquaintances or
`allows a user to “sit back and relax” while someone else
`controls the enhanced experience. In this embodiment, the
`personalized content is based on the learned social dynamics
`of the user.
`
`Another embodiment includes a developer toolkit that
`allowsa creative directorto create trigger points in the mate-
`rial where digital assets such as overlay graphics, back-
`grounds, icons, text, sound, and/or product placement images
`can be insertedinto the digital media narrative. The developer
`toolkit allows the creative director to create a map of the
`elements ofthe user’s profile and his or her internal narrative
`perception identification framework topologyto digital assets
`that may be created. The developer toolkit also provides for
`the ability to insert interactive material such as questions or
`interactive graphics, allowing for the updating and enhance-
`ments ofthe userprofile based on the user’s interactions. The
`toolkit further permits the creative director to create and link
`the digital assets to the material such that assets for which the
`user has a high degree of affinity or assets having a highest
`probability for matching a user’s internal narrative perception
`identification framework topology can beselectedatthe trig-
`ger point.
`One advantage of an embodiment of the method, system
`and software presented herein is that they may enable a cen-
`tral server or service to compile a user’s internal narrative
`perception identification framework andto store the internal
`narrative perception identification framework as an internal
`narrative perception identification framework profile and
`allow this profile to track or follow the user as he or she
`peruses multiple digital media narratives. This enables the
`user to receive enhanced digital media assets and various
`narrative experiences and allows the service to be sold to
`advertisers that advertise on any web site or broadband IP-
`based distribution point.
`These andother features and objects of the invention will
`bemorefully understood from the following detailed descrip-
`tion of the embodiments which should be read in light of the
`accompanying drawings.
`In this respect, before explaining at least one embodiment
`of the invention in detail, it is to be understood that the
`invention is not limited in its application to the details of
`construction and to the arrangements of the components set
`forth in the description or illustrated in the drawings. The
`invention is capable of other embodiments andof being prac-
`ticed and carried out in various ways. Also, it is to be under-
`stoodthat the phraseology and terminology employed herein,
`as well as the abstract, are for the purpose of description and
`should not be regardedas limiting.
`
`

`

`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 31 of 39
`Case 6:20-cv-00810-ADA Document 28-1 Filed 12/10/20 Page 31 of 39
`
`US 8,856,030 B2
`
`5
`As such, those skilled in the art will appreciate that the
`concepts upon whichthis disclosure is based may readily be
`used as a basis for designing other structures, methods, and
`systems for carrying out the several purposes of the present
`invention.
`It
`is important,
`therefore,
`that
`the claims be
`regarded as including such equivalent constructions insofar
`as they do not depart from the spirit and scopeofthe present
`invention.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The accompanying drawings, which are incorporated in
`and form a part ofthe specification, illustrate embodiments of
`the present invention and, together with the description serve
`to explain the principles of the invention.
`FIG. 1 illustrates a user-relationship diagram for an
`embodimentof the system;
`FIG.2 illustrates the representative components ofadigital
`media narrative and the digital media assets;
`FIG. 3 illustrates the creation of a personalized digital
`media narrative;
`FIG. 4 illustrates the use of interactivity to create an
`enhanceduser profile and an enhanced experience through a
`personalized digital media asset;
`FIG. 5A illustrates a context diagram for digital media
`narrative asset personalization;
`FIG.5Billustrates potential databases that may be used in
`creation of the personalized digital media asset andtherela-
`tionships between those databases;
`FIG. 6 illustrates exemplary relationships between the
`soci