Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 1 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 1 of 39
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`EXHIBIT 1
`EXHIBIT 1
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 2 of 39
`HeeSLESTATA
`
`US008856030B2
`
`US 8,856,030 B2
`(10) Patent No.:
`a2) United States Patent
`Russek
`(45) Date of Patent:
`Oct. 7, 2014
`
`
`4) METHOD, SYSTEM AND SOFTWAREFOR
`ASSOCIATINGATTRIBUTES WITHIN
`DIGITAL MEDIA PRESENTATIONS
`
`(75)
`
`Inventor: David J. Russek, Maple Glen, PA (US)
`
`(73) Assignee: Sevenecho, LLC, Norristown, PA (US)
`(*) Notice:
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b)
`by
`1841 da
`S.C.
`154(b)
`by
`YS.
`
`(21) Appl. No.: 10/819,514
`(22) Filed:
`Apr. 7, 2004
`
`5,692,212 A
`5,725,472 A
`5,734,795 A
`5,751,953 A
`5,754,787 A *
`5,774,664 A
`5,805,784 A
`5,848,934 A
`ee \
`Soin
`‘
`5,913,310 A
`5,918,603 A
`5,935,003 A
`5,945,988 A
`
`11/1997 Roach
`3/1998 Weathers
`3/1998 Rogers
`5/1998 Shiels
`5/1998 Dedrick Sete e enero eres 709/228
`6/1998 Hidary
`9/1998 Crawford
`12/1998 Shiels
`vo0 eee
`ee
`res
`6/1999 Brown
`7/1999 Brown
`8/1999 Stephens
`8/1999 Williams
`(Continued)
`
`(65)
`
`Prior Publication Data
`
`FOREIGN PATENT DOCUMENTS
`
`US 2004/0199923 Al
`
`Oct. 7, 2004
`
`wo
`
`0177939 Al
`
`10/2001
`
`Related U.S. Application Data
`(60) Provisional application No. 60/460,998, filed on Apr.
`7, 2003.
`
`(51)
`
`(2012.01)
`(2012.01)
`(2006.01)
`
`Int. Cl.
`G06E 30/02
`G060 30/00
`GO6F 17/30
`(52) U.S.CL
`CPC oo. GO6F 1730867 (2013.01); G06O 30/06
`(2013.01); GO6F 1730265 (2013.01)
`USPC licccccceccccssceteestescesesereesees 705/14.66; 705/319
`(58) Field of Classification Search
`
`.. GO6Q
`CPC ....
`50/01; GO6Q
`30/0269
`USPC vieccscccsessssesssssssseueen°vee 705/14 Tse 319
`See applicationfile for complete search history.
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`5,119,474 A
`5,586,967 A
`5,604,855 A
`5,625,570 A
`
`6/1992 Beitel et al.
`12/1996 Davis
`2/1997 Crawford
`4/1997 Vizireanu
`
`OTHER PUBLICATIONS
`
`oo
`;
`;
`;
`;
`Quintana, Y.: “Design of an Object-Oriented Multimedia Database
`for Personalize Multimedia News”, Proc. 1996 Canadian Conf. on
`Electrical and Computer Engg., May 26, 1996-May 29, 1996; pp.
`282-285, KP002460458, Calgary, Canada.
`
`Primary Examiner — Donald L. Champagne
`(74) Attorney, Agent, or Firm -~ Pepper Hamilton LLP
`
`(57)
`ABSTRACT
`.
`.
`Disclosed are a system, method and software to associate
`attributes with digital media assets. Digital media contains
`specific assets, such as images, that can be replaced with other
`assets. The system, method and software permit the associa-
`tion of attributes with specific assets. The association of
`attributes and assets enables the provision of content that is
`enhanced and more impacting for a user.
`
`2 Claims, 25 Drawing Sheets
`
`
`
`545
`
`a/
`
`
`
`RID AS:
`wa
`Cis
`ONUNE—[oonmuiry uses NITENS
`©of
`COMMINITY ACTIVE V. INACTIVES
`8)
`SY
`Bn
`LSoy PS)
`YY
`Ry,
`&
`
`
`
`
`Z
`“
`3
`&
`
`STORED USER
`
`PROFILE
`
`Ln ge4
`
`MODIFIED CONTENT
`STORAGE
`
`554
`
`10Tales0000001
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 3 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 3 of 39
`
`US 8,856,030 B2
`
`Page 2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,977,968 A
`5,987,415 A
`5,999,172 A
`6,029,195 A
`6,078,936 A
`6,088,717 A
`6,120,846 A
`6,135,944 A
`6,175,772 Bl
`6,190,314 Bi
`6,205,472 Bl
`6,222,925 Bl
`6,249,780 Bl
`6,293,904 Bl
`6,306,077 BL
`6,311,194 Bl
`6,317,127 Bl
`6,330,595 BL
`6,345,288 Bl
`6,357,042 B2
`6,421,669 Bl
`5,425,825 BL
`6,466,232 Bl
`6,499,741 B2
`
`11/1999 Le Blane
`11/1999 Breese
`12/1999 Roach
`2/2000 Herz
`6/2000 Martin
`7/2000 Reed
`9/2000 Hin
`10/2000 Bowman
`1/2001 Kami
`2/2001 Ark
`3/2001 Gilmour
`4/2001 Shiels
`6/2001 Mizokawa
`9/2001 Blazey
`10/2001 Prab
`10/2001 Sheth
`11/2001 Daily
`12/2001 Ullman
`2/2002 Reed
`3/2002 Srin
`7/2002 Gilm
`7/2002 Sitrick
`10/2002 Newell
`12/2002 Yamazaki
`
`6,513,011 Bl
`6,513,069 BI
`6,520,905 Bl
`6,526,395 Bl
`6,527,700 BL
`6,529,864 BL
`6,545,209 BL
`6,581,037 BL
`6,585,521 BI
`6,604,091 B2
`7,000,180 B2
`7,162,432 B2
`TA83,871 B2*
`8,635,649 B2*
`2001/0049588 Ai
`2002/0013943 Al
`2002/0092019 Al
`2002/0122430 Al
`2002/0159640 Al
`2003/0001880 Al
`2003/0009078 Al
`2003/0036899 Al
`2003/0045987 Al
`2003/0131351 Al
`2003/0142689 Al
`2004/0267816 Al
`
`1/2003 Uwakubo
`1/2003 Abato
`2/2003 Surve
`2/2003 Morris
`3/2003 Manico
`3/2003 Chase
`4/2003 Flannery
`6/2003, Pak
`7/2003 Obrador
`8/2003 Sadakuni
`2/2006 Balthaser
`1/2007 Mascarenhas
` VW2009 Herz ccc ccceeetseettees L/L
`1/2014 Wardetal. we 725/40
`12/2001 Bauschet al.
`1/2002 Haberman
`7/2002 Marcus
`2002 Haberman
`10/2002 Vaithilinga
`1/2003 Holtz et al.
`1/2003 Fedorovskaya
`2/2003 Leslie et al.
`3/2003 Haberman
`7/2003 Shapira
`7/2003 Haberman
`12/2004 Russek
`
`* cited by examiner
`
`10Tales0000002
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 4 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 4 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 1 of 25
`
`US 8,856,030 B2
`
`TWNIDIO
`
`LNALNOO
`
`v
`
`bL
`
`LSILav
`
`ZLL
`
`INSLNOO
`
`ONVATSWSSSV
`
`NOILAGIYLSIO
`
`001
`
`YHOLNEIYLSIC
`
`90}
`
`CaisIgon
`
`LNSLNOO
`
`YAWNSNOO
`
`vol
`
`b‘Old
`
`10Tales0000003
`
`LOL
`
`YOSNOdS
`
`
`
`801YOLOSYI
`
`AAILVSeO
`
`|
`
`~ObL
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 5 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 5 of 39
`
`U.S. Patent
`
`O3GIAaNNOYSyOVE|{
`
`p0zae|S3ONINOISSaovWI|;002
`
`
`
`Oct. 7, 2014
`
`Sheet 2 of 25
`
`US 8,856,030 B2
`
`VIGAWWLIDIG
`
`SassyChe
`
`ONIGNVYS
`
`SOIHdVED902
`
`GNNOYDsyOS
`
`SOVINI
`
`C02
`
`WoId
`
`OZ
`
`
`
`"AWTMSAO)LXSL
`
`‘ONITIONDSx31
`
`(qSySvW802
`
`éOld
`
`10Tales0000004
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 6 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 6 of 39
`
`U.S. Patent
`
`SOMA2OISNIN
`
`Oct. 7, 2014
`
`CAZMVNOSHAd||!|!!MyroroeNV|!!;SSSVWI
`AMOLSMASNyee-.OS\----.o-oo
`
`CYNUBLNI){||!|!;|i}SNOHOWA
`guei------poPoLoebtWNYALNI
`
`Sheet 3 of 25
`
`
`
`ZIEWinnSSos|2Oi|Gonedee)
`
`SWozeTi0']teomoilvOBoeeee|yee
`
`MAN
`
`GAZMWNOSYAd
`AYOLSHASN|SNOILOWS
`
`CYNYSLNI)OLEGN
`ad!SaOVNI
`
`US 8,856,030 B2
`
`
`
`10Tales0000005
`
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 7 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 7 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 4 of 25
`
`US 8,856,030 B2
`
`DEFAULT
`
`EXPERIENCE
`TRIGGER
`
`PERSONALIZED
`
`EXPERIENCE
`
`
`
` ~«—
`
`
`
`ENHANCEDUSERPROFILE
`
`10Tales0000006
`
`“
`®
`cq)
`L.
`
`—L
`
`<t
`©
`<t
`
`co
`on)
`<—t
`
`
` CN
`Q
`oS
`>
`—
`st
`
`INTERACTIVE
`
`OPPORTUNITIES
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 8 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 8 of 39
`
`bes
`
`LNSLNOO
`
`YSAdaSS06S
`
`U.S. Patent
`
`Oct. 7, 2014
`
`
`
`Sheet 5 of 25
`
`US 8,856,030 B2
`
`LSS
`
`
`
`LNSLNOOGaisIdOW
`
`AOVYOLS
`
`Ww
`a>
`
`
`
` YSSNALINNWAIOD
`
`SNNNO
`
`ALINNWINOS
`
`
`
`YdsGaYOLS
`
`ddOud
`
`VSOld
`
`10Tales0000007
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 9 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 9 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`
`
`ONYANMINO31dO3dSHL40SLSI1
`
`
`
`LHONOYE38OLSLNSWWOOYISHL
`
`
`
`SINIOdYSDSIYLSVNI
`
`
`
`‘ONIONSNDASLASSV
`
`ALIMNOASGNYONIAIL
`
`cOS
`
`LaONsaIanv
`
`bAONSIYSdxXS
`
`YaSNSHLS|OHM
`
`HLIMONLLOANNOO
`
`ANNNO
`
`
`
`ONVNOILOWSCa1auddaNi
`
`
`
`
`
`ONIddVWVLVGAGISLNO
`
`Sheet 6 of 25
`
`PLSZS
`
`dn
`
`YOLINON
`
`
`
`
`
`dlcSm|ONITSAVELbes
`LaSsvTLISICINBNBOYNYN
`
`US 8,856,030 B2
`
`OLS
`
`AYOLISOdaY
`
`
`
`dasvaTvul9idco
`4OSYNLVNONIATOASCNVdnOw90z5
`
`
`
`(lv)AMOLSWI00Seee
`
`81S“soIWWNAG724
`
`
`
`
`
`gSSid
`
`10Tales0000008
`
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 10 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 10 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 7 of 25
`
`US 8,856,030 B2
`
`603
`
`Lil
`
`oO
`
`==a
`
`me
`oO
`oO
`
`601
`
`604
`
`EMOTIONAL
`
`602
`
`SOCIAL
`
`FIG.6
`
`10Tales0000009
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 11 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 11 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 8 of 25
`
`US 8,856,030 B2
`
`
`RECOGNITION ACQUISITION OFNEWINFORMATION
`ACQUISITION/
`LANGUAGE
`WRITTEN
`
`
`UNDERSTANDING/
`REPRODUCTION
`
`712
`
`710
`
`
`
`708
`
`FIG.7
`
`10Tales0000010
`
`716
`
`718
`
`oO
`o
`
`©c
`
`720”
`
`702
`
` (vss
`
`PROCESSING
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 12 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 12 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 9 of 25
`
`US 8,856,030 B2
`
`
`
`WNYSLNIJId/LTINW
`
`FALLVYYYN
`
`NOLLda0uSd
`
`NOILYOISILNSGI
`
`YYOMANYYS
`
`
`
`(so63Jay)
`
`8L8SSCNLULLV
`
`40SAGOW
`
`NOLLOVYSLNI
`
`a
`
`4013A37
`
`LNSWSATOANI
`
`08
`
`WID0S
`
`
`
`NOILdS0uad
`
`NOILVOISILLNSGI
`
`MYOMAWVeS
`83s)
`
`908
`
`609
`
`
`
`TWID0S$SdNOwd
`
`208
`
`idSNl/id¥8SSLINISSV
`PLOVIML
`
`SdIHSNOILV13u
`
`18
`
`
`
`SVYNOSUSdTVIDOS
`
`
`
`anlgoiadajdoadMoH)
`
`econ
`
`8‘Old
`
`10Tales0000011
`
`
`
`
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 13 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 13 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 10 of 25
`
`US 8,856,030 B2
`
`O 2 o
`
` CULTURAL
`HISTORICAL
` PERSONAL
`
`2xa
`
`;<a
`
`2
`co)
`
`©zo
`
`914
`
`601
`
`902
`
`AUDITORY(acmOLFACTORY
` EUROPEAN
`GEOGRAPHICAL
`CraneSq
`
`PERCEPTION
`
`nl
`=<
`m—
`
`Oa
`
`w
`om
`Lu
`Qu.
`
`910
`
`CULTURAL
`
`HISTORICAL
`
`NORTH
`
`AMERICAN
`
`FIG.9
`
`10Tales0000012
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 14 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 14 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 11 of 25
`
`US 8,856,030 B2
`
`4000
`
`CREATIVE DIRECTOR
`LISTENS/(WATCHES
`
`CONTENT
`WITH AFFINITIES
`
`TAG TIME INDICES=|_, 4010
`
`N°
`
`~ 1020
`
`
`
`EDIT AFFINITIES
`COALESCE MEANING
`IDENTIFY MAPPING
`
`1030
`
`CREATE INTERACTIONS}~1050
`
`FIG. 10
`
`DEVELOP COLLATERAL
`MATERIALS
`
`1060
`
`10Tales0000013
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 15 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 15 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 12 of 25
`
`US 8,856,030 B2
`
`
`
`
`
`
`LELLSONVLYOdW|©NMONYNA/NMONY|LNSWS73311d0uddid¥3asn
`
`
`
`
`
`
` poAOS|SFONVIIddSL6CLESONVSSIVNSU|da|SDEZLELNSWS7SFNSOUd|GIYaSN| bbyNV]MWA]pea]SH6ZLEpA]
`
`LOLLVASGYOMSSVdQI¥3SN
`
`
`
` |MOTNMONYPoNMONYNNsaulgGL6ZZESSONWIIdd¥GL6ZZE
`
`
`
`
`LANdSIQHLINSHSIdO9GL6cZE
`
`WNIdaWNMON¥;siCSE
`
`
`
`bh‘Old
`
`10Tales0000014
`
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 16 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 16 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 13 of 25
`
`US 8,856,030 B2
`
`
`
`1200
`PERSON
`
`
`
`AFFINITY LIST
`GET AFFINITIES
`
`SET AFFINITIES
`
`PEOPLE GROUP
`SET OF PEOPLE
`
`
`
`
`
`GROUP OF PEOPLE
`RECORDED AFFINITIES
`MEDIA
`
`EXPERIENCE
`
`1220
`
`1210
`
`
`
`
`
`1230~RECORDEDAFFINITIES
`
`
`TYPE
`
`AFFINITY
`
`
`AFFINITY
`1240
`
`
`NAME
`
`
`DESCRIPTION
`PARENT
`CHILD
`
`GROUP
`
`
`FIG. 12
`
`10Tales0000015
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 17 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 17 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 14 of 25
`
`US 8,856,030 B2
`
`SACGaVH
`
`O0EL
`
`YATIOULNOD
`
`
`
`SOASLYOdWIYAS
`
`YOLINOW
`GYVOeAIy
`
`WOud/OL
`
`WOus/OL
`WOUs/OL
`
`WOYS/OL
`
`MYOMLAN
`
`SLOTSVSI
`
`cb‘Sid
`
`10Tales0000016
`
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 18 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 18 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 15 of 25
`
`US 8,856,030 B2
`
` 0
`
`2:56
`|nt/N1430
`
`AUDIO
`
`BACKGROUND 1
`
`BACKGROUND 2
`
`ST 1
`
`ST 2
`
`1432
`
`1434
`
`IT 1be 1436
`
`TEXT 1
`
`IT 2 |—__-------------— 1438
`
`TEXT 2
`
`FIG. 14
`
`10Tales0000017
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 19 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 19 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 16 of 25
`
`US 8,856,030 B2
`
` 3&3&
`
`
`
`
`
`aEGoody
`
`3e:
`
`le
`
`4>3=
`
`“
`
`sga
`
`o
`
`o2&&
`
`a
`
`
`
`ADEEBIBS[O90]e
`
`
`
`
`
`
`
`IW]NVONIMSIAOLYOlddYALSIOSYLSNWYSdWSWSONSICNVNV
`
`Sb‘Old
`
`10Tales0000018
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 20 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 20 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 17 of 25
`
`US 8,856,030 B2
`
`
` “SSOIOHOGSLVSuOSevLVHL9L‘O94SAILOVYSLNIWOXd
`
`
`
`
`
`SaWIOUdNOdNGaSVdYSSNYadCANINYSALSGS|LNSLNOOGaZIVNOSdadOINVNACSHL
`
`
`
`
`
`
`
`
`aug
`
`pare.apgy.
`
`“#$&£
`
`4zezz
`
`4
`
`
`
` ee32a2e2a/et.eog32&“4£5a
`
`hes
`
`
`
`
`
`
`
`SUG?SuyyseusbeoO)aa
`
`10Tales0000019
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 21 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 21 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 18 of 25
`
`US 8,856,030 B2
`
`ip
`
`TISMSVNOLLOV40ONIGNS18SHLSSQNTONISAILVSHOIWAHL
`
`
`
`(UYZO6SIES{GuayfFALMENSPSSM/HAIN984994OZOEUAL9FUCEELAQUUTASEEUEEAS
`
`
`
`‘WHOO3dIATWLIDICNISV
`taWiadHestaRm&<<oY@
`ALe
`
`gMew
`
`ww
`
`a
`
`olz2a“
`
`eI3.a$4ral.2&A
`Ce4
`
`3
`
`$3a==B
`
`10Tales0000020
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 22 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 22 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 19 of 25
`
`US 8,856,030 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`10Tales0000021
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 23 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 23 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 20 of 25
`
`US 8,856,030 B2
`
`6h
`
`
`
`
`
`COL
`
`10Tales0000022
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 24 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 24 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 21 of 25
`
`US 8,856,030 B2
`
`
`
`GavHaWLH,—ANOLANOLANGL
`
`xasanianilo
`
`10Tales0000023
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 25 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 25 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 22 of 25
`
`US 8,856,030 B2
`
`34Whatareyourthoughts?Pleaserespond
`
`
`
`FIG.21
`
`ce
`EE
`
`10Tales0000024
`
`’y:e
`
`eaz
`
`e
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 26 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 26 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 23 of 25
`
`US 8,856,030 B2
`
`JALVAYO
`
`MAN
`
`4Nsd0¥d
`
`OVILYLOVYSLNI
`
`
`
`LAdNISCIAOud
`
`
`
`QaLVadoHOIHM
`
`AWdOud
`
`NIO3d
`
`SONAlMAdXxS
`
`0€@2
`
`VeeDd
`
`10Tales0000025
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 27 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 27 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 24 of 25
`
`US 8,856,030 B2
`
`Glee
`
`VGYOLOSHIC
`
`NOLLVOISISSV1D
`
`NOLLOVLASSVVIGAWTIWLISIG4OALINOldd0622
`
`
`
`SONSYsd3edHOSALINEVEOddLSSHOIHG6ZZOLONIGUOSDOV
`
`
`
`
` VING4OSYNLXIWHANIOLNOLOauiaAad&*O8ZZ0122
`
`
`
`dOSYNLXIWYSSNIOLSLANENOLLYNTWAS
`NOLLVOISILNAG!]JONSIGAVGSONVHNAONY
`
`
`
`
`
`NOLLWOIEISSV1OLASSVTWLIDIGALWATWAa
`
`
`FNsOudMOAOVHSWLSIG
`S1dJOUdMOGVHSWLOIdJOALINOdd
`
`WNCGIAIGNIHOVAOLNOLLOWVINdv
`
`
`
`NOILWINdOd31dOudTWNGIAIGNI0922
`
`LNdNI3SNOLLOVYALNISJONZIOGNY
`
`SLAdNIAZMVAYONQSé2
`
`JONSYaIaudLass
`
`SLN3ZW313
`
`G8C¢
`
`GeOld
`
`10Tales0000026
`
`
`
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 28 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 28 of 39
`
`U.S. Patent
`
`Oct. 7, 2014
`
`Sheet 25 of 25
`
`US 8,856,030 B2
`
`
`
`
`WYOMANVANOLLVOMTEINA!NOLLdaOWddFAILVUAIVN‘NOILVOMISSVTOATIdOUd
`“d138Wid0S €¢Old
`
`NOILWLOadxXS39100SId3
`JONAdadaud
`ONINIVLNIVA
`
`SLiVaL
`
`d13a8
`
`ONIONANDAS
`AD010d01
`
`LNSLNOS
`
`AAILOATION
`
`SWIL
`
`WNYSLNI
`
`SAILVHORSVTIOD
`[AALLISNAS
`
`FALLEN
`
`
`
`ONILVYYVNSAILINGS
`
`TWNOILOWS
`
`SHILINISAV
`
`HALLINDOO
`
`SaiLINISSV
`
`
`
`
`
`LOVdWIAAILVAVNASVAUONI
`
`“TWOD
`
`ATWID0S
`
`/Q39VONA
`
`QSNId30
`
`3ONaNOSS
`
`
`
`(NIA)SNONIV
`
`HLIM
`
`WHNLIND
`
`LNSNZOVONA
`TWID0SON
`NOILWLO3dXS
`
`YOHOASd
`
`LNSALNOO
`
`YADVONA
`
`ONMONLON
`
`MONYLON
`
`MONLON
`
`YSCNVISAd
`dLlVlddOddd¥
`
`MOo1
`
`
`
`MO"NMONH
`
`LNALNOS
`JONVOISINSIS
`
`
`
`JONVOIIND'SAINV3IEINSIS
`
`SAWODLNO
`
`
`
`aLviddOddd¥FONVOISINDIS
`
`JONVOISINDIS
`
`YSislLNAdl
`
`“NONwniaan
`
`Whidan
`
`(N/A)
`
`“NON
`
`ALVIddOUdd¥
`
`
`
`WOUdLNILNOS
`
`
`
`(N/A)SHSHLO
`
`ALVIedOUdd¥
`ONIONSWA
`
`INSLNOO
`
`ANALNI®
`
`SYSHLOWOU
`NOLVIOOSSY
`
`LNALNI
`
`Wes
`
`
`
`S114YSHLO
`
`asunldvo)
`
`NSHM
`
`(CaLLWans
`
`LINSWA9VONA
`AWOOLNO8JONVINVA
`SATSVINVA
`WoMOKdVLAW
`
`LVHLaAO8vTv
`
`LNALNOOON
`
`
`
`SYSHLONOUS
`
`WIS
`
`40SONVASTSY
`ISNOILVLOAdxX3
`
`‘d33d$
`
`NOILOW
`
`LINSLNOO
`
`
`
`S3LViNdOdWOHMHLIM
`
`LNALNOD
`
`YSHOVSd
`
`HOIH
`
`HOIH
`
`
`
`JOONINVANJONVIISINDIS
`
`JONVOIINOIS
`
`-YO+)
`
`(AONATVA
`
`10Tales0000027
`
`
`
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 29 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 29 of 39
`
`US 8,856,030 B2
`
`1
`METHOD, SYSTEM AND SOFTWARE FOR
`ASSOCIATING ATTRIBUTES WITHIN
`DIGITAL MEDIA PRESENTATIONS
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This patent application claims priority to the provisional
`patent application entitled, “Method and System for Digital
`Entertainment Asset Personalization,” filed Apr. 7, 2003,
`serial No. 60/460,998, which is incorporated by reference
`herein. This patent application is also related to the patent
`application entitled, “Method, Systemand Software for Digi-
`tal Media Narrative Personalization,” filed Apr. 7, 2004.
`
`16
`
`15
`
`BACKGROUNDOFTHE INVENTION
`
`2
`digital media narrative storytelling, typically in the form of
`music or a short story that would appeal to viewers. Whatis
`required are ways to attract individuals to content that is
`personally more relevant and impactful for them and which
`maycontain an advertising message (in the form of product
`placement), and have thern receive that message in full, as
`opposed to skipping overall or a portion of the message. It
`would be an additional benefit to have the ability to under-
`stand the individual’s likes and dislikes or current mood in
`order to adapt the message appropriatelyfor the individualat
`the time that they are receiving it.
`Digital media narrative may include content, such as video,
`audio, and combinations thereof, that is distributed digitally
`and can be stored digitally. In some instances digital media
`narrative can be manipulated via computer without human
`interaction. Digital media narrative includes butis not limited
`to stories presented overthe Internet, modifiable digital video
`including digital television and streaming media, presenta-
`tions on web sites including animations,
`still
`images,
`sequences of still images, audio, textual presentations, and
`combinationsof all of the abovementioned media.
`
`Digital media narrative can be distributed in a multitude of
`ways, such as broadcasting over cable andsatellite, the Inter-
`net, or on media such as optical disks (CDs or DVDs). Dis-
`tribution of digital media narrative is a significant business
`and involves complex relationships between artists and
`record labels orfilm studios, producers, and a complete dis-
`tribution chain that includes advertising, both in the form of
`advertising of the content itself as well as sponsored adver-
`tising that may be inserted into, or becomepart of the content
`suchas product placement. Examples of sponsored advertis-
`ing include theinsertion of commercials into recorded mate-
`rials or the use of sponsor’ s products (e.g., automobiles) inthe
`narrative.
`
`Digital media narratives may be directed at a segment of
`the market, but the segment that the digital media narrative
`material is directed to may be quite large. For example, an
`artist may create works that appeal to the 18-24 year old age
`group, but that group can include people with different inter-
`ests from various countries and different economicclasses.
`
`Furthermore, when digital media narratives are used as part
`of advertising, that digital media narrative and the advertising
`may be directed at such a large market segment that it is
`ditticult to keep the interest of individuals within that market
`segment. If a user goes to an Internet web site to view a
`recording ofanartist, and in particular a sponsored work that
`contains advertising, Uhat work (including the advertising)
`maybe so generic that the user cannot respondtoorrelate to
`the work. Additionally, the user may experience different
`emotional states than another user, or the user may be in an
`emotionalstate that is different than the user’s emotionalstate
`when the user previously viewed the material.
`For the foregoing reasons, there is a need for a method,
`system, and software to enable a distributor of sponsored or
`un-sponsored digital media narratives to change the content
`of the digital media narrative based on user demographics,
`psychodemographics, emotionalstates, affinities (cognitive,
`emotional and social), self-narrating content classification,
`internal narrative traits preference topology, time sensitive,
`episodic expectation sequencing, and collective/collaborate
`attributes.
`
`SUMMARY OF THE INVENTION
`
`Amethod, system and software are presented which allow
`for customizing and personalizing content based on a com-
`bination of the user’s demographics, psychodemographics,
`
`10Tales0000028
`
`It has been stated that humanthinking falls into one of two
`categories: reasoning and narrative, descriptive, contempla-
`tive thinking. Reasoningis the process that permits the accep-
`tance of facts followed by a logical process te conclude a
`result, opinion, conclusion or decision. Narrative, descrip-
`tive, contemplative thinking corresponds to the storytelling
`process (including receiving of a story) and the thought pro-
`cesses associated with the reception of a story.
`Text and stories (presented in various formats) can also be
`interpreted at various levels includingtheliteral, cthical, his-
`torical and mystical levels. It is generally understood that
`texts and stories are read and interpreted in different ways
`includingat the different levels. A story or narrative can have
`different meanings at different levels, different interpreta-
`tions at different levels, and will be interpreted differently by
`different individuals at each of the levels. Additionally, an
`individual, based on their emotional or physical state, may
`interpret each level ofa story differently. A person may react
`differently to a story based on their current emotionalstate,
`which mayin fact change while theyare receiving the story.
`In some cases, it may be possible to alter their emotional state
`while receiving the story to make them more receptive to a
`particular interpretation at one or morelevels.
`Advertising is typically a type of narrative in which the
`advertiser attempts to influence the recipient (viewer, reader,
`listener) in a way that creates a positive association with a
`product or in some cases attempts to influence the recipientto
`purchasethe product. Humoris frequently usedin advertising
`because it puts the recipient in a receptive state for a message
`regarding the product. As such, humoris already used to alter
`the emotional state of the recipient of the advertising. Such
`manipulation of the emotional state of the recipient is cur-
`rently limited to simple manipulation through the presenta-
`tion of the advertisement(narrative).
`With the advent of the digital era, the number of ways to
`transmit and present advertisements has increased dramati-
`cally, with advertising being inserted into Web pages, being
`displayed as pop-up windows on Web pages, transmitted as 5
`streaming media over the Internet, presented as animation
`over the Internet, and inserted into television streamsat cen-
`tral or distributed insertion points. The advent ofthe digital
`era also threatens advertising in that viewers can avoid adver-
`tisements by stopping pop-up windows, or skipping over the
`advertisements on a Personal Video Recorder (PVR).
`Due to the rapid changes in technology, advertisers are
`being forced to take new approaches to advertising including
`product placement, in which products are advertised during
`an entertainmentnarrative (e.g. a sit-com) as opposedto in a
`separate advertisement. Advertisers are also exploring ways
`to draw viewers (or listeners) to content that is essentially
`
`20
`
`25
`
`35
`
`40
`
`45
`
`50
`
`60
`
`65
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 30 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 30 of 39
`
`US 8,856,030 B2
`
`3
`cognitive states, emotional states, social placement and group
`interaction dynamics within an online community, and/or
`affinity for certain content elements (images, sounds, seg-
`ments, graphics, video, text, dialog), self provided narrating
`content,
`internal narrative traits preference topology, and
`expectation level and temporal spacing of assets within the
`narrative. In one embodiment, a user sclects material for
`presentation and is presented with the dynamic digital media
`narrative and optional embedded advertising presented in a
`customized format that gives the user a personalized experi-
`ence.
`
`The system, method and software may support product
`placement type advertising by modifying both the product
`placement material as well as the narrative surrounding the
`product placement material. Appropriate advertising may be
`selected as well as selecting components for the narrative that
`supports the product placement advertising.
`In one embodiment, the user accesses a website containing
`sponsored material and logs on to that website. Information
`regarding the demographics, psychodemographics, cognitive
`states, emotional states, social placement and group interac-
`tion dynamics with the online community, and/or affinity for
`certain content elementsof the useris retrieved fromstorage.
`If the user is a new user, he or she is asked a series of
`questions, preferably through a form of media narrative, that
`would creatively ask the questions, in order to obtain key
`information that the system would then classify and include
`into the user’s profile. These questions can include yes/no,
`multiple choice, like/dislikes ratings or any other type of
`personal assessment or individual and/or group question.
`Alternatively, the user may interact with the media narrative
`and the interactions may be used to create a userprofile.
`In one embodiment, the user requests that the material be
`played or displayed, and is simultaneously presented with
`that material and customized personalized graphics and
`video. In another embodiment, the user can interact with the
`graphics and video. By making certain choices, a profile of
`the user can be determined and updated and the content(in-
`cluding background material, primary video, overlay graph-
`ics and audio) modified appropriately, thus giving the user an
`enhanced narrative, and more meaningful and emotionally
`satisfying experience with respect to the viewed content. In
`both embodiments, the user profile can then be used to present
`the most appropriate digital assets to the subscriber, namely
`those with which the user has the highest affinity, or those
`which map well to the user’s tendencies and temperament,
`which may be included in the user’s internal narrative per-
`ception identification framework. The internal narrative per-
`ception identification framework may include a collection of
`attributes, qualities, and measurements regardingthe userthat
`allow for matching, correlation and/or selection of digital
`media assets that are appropriate for that user and the effective
`communication of the message.
`In one embodiment, the user’s demographics are used to
`determine personalized content. Demographics may include,
`and are not limited to, group affiliations, regional location,
`sex, sexual preference, age, family situation (married or
`single, children or no children), race or ethnic background,
`religion and socioeconomic condition. The user’s demo-
`graphics maybe used alone, or in combination with the pre-
`viously discussed techniques, to create the individual internal
`narrative perception identification framework.
`One advantage of an embodiment of the method, system
`and software presented herein is that the user is presented
`with an enhanced experience of the creator’s content that
`creates a greater emotional experiencefor the user and a more
`impactful narrative. This creates a more receptive state for
`
`20
`
`35
`
`40
`
`45
`
`5
`
`4
`product placed advertising inserted into the content, for a
`direct advertisement, or for entertainment containing an
`advertising message. Another advantage of the method, sys-
`tem and software presented herein is thatit allows advertising
`to be inserted in subtle ways and presented in a context in
`which users may be able to fully engulf themselves into the
`lifestyle being positioned and portrayed by the brand. By
`combining the personalized digital media narrative asset ele-
`ments with product placement advertising, a more powerful
`media is created for the advertisers and the artists. Addition-
`
`ally, since users typically volunteerto see the content, includ-
`ing advertising, they are much morelikely to be receptive to
`the message presented, and less likely to skip over or fast-
`forward through the content including the advertising.
`Inan alternate embodiment, users are presented with ques-
`tionsor chat like text based opportunities within a narrative to
`either share the emotional experience with other users or to
`comment on their own emotional experience. his enables a
`user to share an experience with friends or acquaintances or
`allows a user to “sit back and relax” while someone else
`controls the enhanced experience. In this embodiment, the
`personalized content is based on the learned social dynamics
`of the user.
`
`Another embodiment includes a developer toolkit that
`allows a creative director to create trigger points in the mate-
`rial where digital assets such as overlay graphics, back-
`grounds, icons, text, sound, and/or product placement images
`can be insertedinto the digital media narrative. The developer
`toolkit allows the creative director to create a map of the
`elements of the user’s profile and his or her internal narrative
`perception identification framework topologyto digital assets
`that may be created. The developer toolkit also provides for
`the ability to insert interactive material such as questions or
`interactive graphics, allowing for the updating and enhance-
`ments of the user profile based on the user’s interactions. The
`toolkit further permits the creative director to create and link
`the digital assets to the material such that assets for which the
`user has a high degree of affinity or assets having, a highest
`probability for matching a user’s internal narrative perception
`identification framework topology can be selectedat the trig-
`ger point.
`Oneadvantage of an embodiment of the method, system
`and software presented herein is that they may enable a cen-
`tral server or service to compile a user’s internal narrative
`perception identification framework and to store the internal
`narrative perception identification framework as an internal
`narrative perception identification framework profile and
`allow this profile to track or follow the user as he or she
`peruses multiple digital media narratives. This enables the
`user to receive enhanced digital media assets and various
`narrative experiences and allows the service to be sold to
`advertisers that advertise on any web site or broadband IP-
`based distribution point.
`These and other features and objects of the invention will
`be more fully understood fromthe following detailed descrip-
`tion of the embodiments which shouldbe read in light of the
`accompanying drawings.
`In this respect, before explaining at least one embodiment
`of the invention in detail, it is to be understood that the
`invention is not limited in its application to the details of
`construction and to the arrangements of the componentsset
`forth in the description or illustrated in the drawings. The
`invention is capable of other embodiments and ofbeing prac-
`ticed and carried out in various ways. Also, it is to be under-
`stood that the phraseology and terminology employed herein,
`as well as the abstract, are for the purpose ofdescription and
`should not be regarded as limiting.
`
`10Tales0000029
`
`

`

`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 31 of 39
`Case 6:20-cv-00810-ADA Document 62-1 Filed 04/02/21 Page 31 of 39
`
`US 8,856,030 B2
`
`5
`As such, those skilled in the art will appreciate that the
`concepts upon whichthis disclosure is based mayreadily be
`used as a basis for designing other structures, methods, and
`systems for carrying out the several purposes ofthe present
`invention.
`It
`is important,
`therefore,
`that
`the claims be
`regarded as including such equivalent constructions insofar
`as they do not depart from the spirit and scope ofthe present
`invention.
`
`BRIEP DESCRIPTION OF THE DRAWINGS
`
`The accompanying drawings, which are incorporated in
`and forma part ofthe specification, illustrate embodiments of
`the present invention and, together with the description serve
`to explain the principles of the invention.
`FIG. 1 illustrates a user-relationship diagram for an
`embodiment of the system;
`FIG.2 illustrates the representative components ofa digital
`media narrative and the digital media assets;
`VIG. 3 illustrates the creation of a personalized digital
`media narrative;
`FIG. 4 illustrates the use of interactivity to create an
`enhanced userprofile and an enhanced experience through a
`personalized digital media asset;
`FIG. 5A illustrates a context diagram for digital media
`narrative