Case 6:20-cv-00810-ADA Document 73-6 Filed 04/23/21 Page 1 of 4
`
`EXHIBIT A
`(Part 5 of 6)
`
`

`

`Case 6:20-cv-00810-ADA Document 73-6 Filed 04/23/21 Page 2 of 4
`
`US 8,856,030 B2
`
`5
`
`13
`timing and security 506 may also play a role in detennining
`the final digital media asset which is presented as the experi(cid:173)
`ence 504 to the audience 502.
`A user profile monitor 500 may also work to understand
`outside emotion and data mapping 528 to detern1ine whom
`the user is connecting with online 526 and the traveling pro(cid:173)
`file management 524, which may ensure that an individual
`profile travels from program to program. Each of these ele(cid:173)
`ments may act to create a more complete stored user profile
`561 and thus a better customization of the experience 504.
`FIG. 6 illustrates exemplary relationships between social,
`emotional and cognitive elements, all of which may define a
`user's affinities. As is understood by those skilled in the art, a
`user's reality may be determined by a number of elements
`including: his or her emotional attraction 604 to certain affini- 15
`ties 601 and elements that are presented; his or her interaction
`with others during consumption of an experience (which is
`the user's social element 602); and the user's cognitive ele(cid:173)
`ment 603, which is his or her awareness and perception of the
`world around him or her. These three elements may form the 20
`basis for a user's reality as perceived by him or her.
`The three elements, as illustrated in FIG. 6, may include
`elements of an "internal narrative perception identification
`framework topology," which detern1ines the user's tenden(cid:173)
`cies, temperaments and provides a classification so as to 25
`increase the digital media narrative impact through providing
`media elements. Other examples of the elements of the inter(cid:173)
`nal narrative perception identification framework (see FIG.
`23) include social/collective attributes, time sensitive, epi(cid:173)
`sodic and expectation sequencing, self-narrating content 30
`classification, the user's tendency to be literal vs. metaphori(cid:173)
`cal, an identifier vs. a detacher, or an engager vs. a bystander.
`These elements of personality are well known by those skilled
`in the art and are exemplary only. Other attributes may be used
`to define the user's internal narrative perception identification 35
`framework topology.
`Referring to FIG. 7, exemplary components of the cogni(cid:173)
`tive 603 element may include: analytical skills 702 or the
`ability ofa user to become consciously aware of elements that
`are being delivered and to analyze elements that he or she 40
`perceives; verbal processing 704 or the ability to verbalize his
`or her reality; inferencing 706, in which individuals infer the
`meaning of certain elements based on other elements; visu(cid:173)
`alization 708 of elements of the reality; the understanding and
`reception of speech 710; the user's ability to compute things 45
`712; the user's ability to comnnmicate in written language
`and/or acquire new information 714; the user's preference in
`method ofacquisition ofinformation 716; the users' ability to
`reason by analogy 718; and the user's ability to quantify 770.
`FIG. 8 illustrates exemplary social elements 602 and their 50
`components therein. These components may include: the
`groups 802 that the user is affiliated with; a social perception
`identification framework 804, such as a user's on-line per(cid:173)
`sonality or alter ego; the social personas 808, which are how
`people perceive that user; a user's social affinities 806; the 55
`level ofinvolvement 810 that the user has with other individu(cid:173)
`als; the relationship 812 the user has with other individuals,
`the modes of interaction 814 or how the user communicates
`and interacts with other individuals; the ability of the user to
`be apt or, alternatively, inept in perilJrming functions, such as 60
`social interaction 816; the attitudes 818 the user has; and
`internal narrative perception identification frameworks 820.
`FIG. 9 illustrates exemplary components of affinity 601,
`including cultural affinities 902 in which the user is associ(cid:173)
`ated with a particular culture or race; artistic affinities 904; his 65
`or her digital media narrative (entertainment) likes and dis(cid:173)
`likes 906; his or her geographical affinities 908; the dates and
`
`14
`events that are important to that user910; his or her sensation
`and perception of the world 912, his or her iconographic
`perception of the world 914; and his or her individual affini(cid:173)
`ties 916. These affinities may determine how a user perceives
`the world and may represent the particular elements that
`allow a user to influence other human beings. Users perceive
`or receive a sensation, consciously or unconsciously create
`the basis for emotions, and provide a catalyst for thoughts and
`emotions that are stored in the brain through an n-Gram
`10 encoding, ultimately placing the experience in a user's
`memory. By having knowledge of a user's affinities, it may be
`possible to influence the user by creating a closer bond
`through personalization to make a narrative experience more
`meaningful to that user.
`FIG.10 illustrates a detailed flow chart for an embodiment
`of the development of the components for a personalized
`digital media asset. Referring to FIG.10, the creative director
`110 may watch and listen to content in step 1000 and subse(cid:173)
`quently tag the time indices with affinities in step l 010. A test
`l 020 may then be performed to determine if sufficient tagging
`has taken place. This step may include comparing the number
`of tags set to a minimum threshold value or determining if all
`of the tags determined by a test sequence have been set. If
`further tagging is needed, a return to step 1010 occurs. If
`sufficient tagging 1020 has occurred, the affinities may be
`edited and associated with certain digital media assets in step
`l 03 0. Step l 03 0 may represent the affiliation of tags to trigger
`points 320 having certain affinities, such that based on a
`user's profile and interaction it becomes possible to retrieve
`the appropriate components of the digital media asset 212 to
`create a personalized experience. Subsequent to the creation
`of the affinities, the tags may be stored in step 1040 and the
`interactions may be created in step 1050, which may result in
`the development of collateral materials for narrative market(cid:173)
`ing in step 1060.
`FIG. 11 illustrates exemplary tables for a user profile when
`the user profile is stored in a relational database. A first table
`1101 may contain a user ID, password, and e-mail address of
`a user. A second table 1111 may contain the user ID, a par(cid:173)
`ticular profile element, a known/unknown status field, and an
`importance field. The second table 1111 may tabulate profile
`elements pertaining to the user. Profile elements may include
`any number of affinities previously discussed, internal narra(cid:173)
`tive perception identification framework profile elements, or
`any other attributes of the user. A third tab le 1121 may contain
`the user ID, one or more profile elements and a value ranking
`for each profile element. The value may indicate that the user
`has an affinity for that particular type of profile element and
`the ranking of that value. As illustrated in FIG. 11, aspects of
`the user's afllnity for art may be known. These aspects may
`have been determined by one of the methods detailed earlier.
`In the example shown, it is known that the user has an affinity
`±ix abstract and renaissance art. In one embodiment, the rank
`column of the third table 1121 is used to represent the strength
`of a positive affinity. In this embodiment, the ranking indi(cid:173)
`cates that the user likes renaissance art more than abstract art.
`In another embodiment, the rank column of the third table
`1121 is used to illustrate how strong the affinity is with that
`particular value. In such an embodiment, higher rankings do
`not indicate stronger positive preferences, but rather indicate
`stronger impact data structures for relational databases used
`to link components for personalized digital media assets with
`a ffini ti es.
`FIG. 12 illustrates an object-oriented approach for storing
`user profile elements and linking components of a personal(cid:173)
`ized digital media asset with affinities. Using an object-ori(cid:173)
`ented approach, a person object 1200, containing a profile of
`
`10Tales0000034
`
`

`

`Case 6:20-cv-00810-ADA Document 73-6 Filed 04/23/21 Page 3 of 4
`
`US 8,856,030 B2
`
`5
`
`15
`elements and set element commands, may be created and
`related to a people group object 1210. A people group object
`1210 may also be related to an experience object 1220, which
`contains aspects of the profile element including the name
`and description of the profile element as well as its relation-
`ship to other objects, which allows retrieval of recorded
`afllnities 1230. Individual affinities 1240 may be separate
`objects that contain specific elements known to be important
`to that user.
`The database structures illustrated in FIGS. 11 and 12 are 10
`exemplary only, and illustrate certain aspects of the user
`profile and his or her afllnities for certain objects and shared
`experiences that are part of his or her social interactions.
`Regardless of the particular database structure used, some or
`all of the aspects of the user's reality described earlier may be
`captured in a database to pem1it a user's profile to determine
`digital media assets that have a strong impact on that user. The
`optimi?2tion process of finding the strongest or most appro(cid:173)
`priate affinities and best match to the user's internal narrative
`social perception identification framework 804 may be based
`on a number of algorithms. Exemplary algorithms may
`include look-up tables, in which values of profile elements are
`matched to digital media assets, and correlation algorithms,
`which correlate profile elements, values, and ranks with pro(cid:173)
`file elements, values, and ranks for a digital media asset to
`determine the best digital media asset to present. Other tech(cid:173)
`niques for matching the user profile to the digital media asset
`may include neural networks and fuzzy logic, wherein
`aspects of the user profile are used to train the network or as
`inputs to the fuzzy logic system to determine the best digital
`media asset. Other types of artificial intelligence techniques,
`well known to those skilled in the art, may also be used to find
`the digital media asset, or sets of digital media assets, that
`have the largest impact on that particular user.
`FIG. 13 illustrates a block diagram of a computer system
`for a realization of the server 590 based on the reception of
`multimedia signals from a bi-directional network. A system
`bus 1320 transports data among the CPU 1312, the RAM
`1308, Read Only Memory-Basic Input Output System
`(ROM-BIOS) 1324 and/or other components. The CPU 1312
`accesses a hard drive 1300 through a disk controller 1304.
`Standard input/output devices are connected to the system
`bus 1320 through the I/0 controller 1316. A keyboard may be
`attached to the I/0 controller 1316 through a keyboard port
`1336 and the monitor may be connected through a monitor
`port 1340. A serial port device may use a serial port 1344 to
`communicate with the I/0 controller 1316. Industry Standard
`Architecture (ISA) expansion slots 1332 and Peripheral
`Component Interconnect (PCI) expansion slots 1326 allow
`additional cards to be placed into the computer. In an embodi(cid:173)
`ment, a network card is available to interface a local area,
`wide area. or other network.
`Software to provide the functionality for a personalized
`digital media asset creation may be developed using a number
`of computer languages such as C, C++, Perl, Lisp, Java and
`other procedural or object oriented languages. Different pro(cid:173)
`gramming languages may be used for different aspects of the
`system, such that a first programming language may be used
`for the content creation process illustrated in FIG. 10 and a
`second programming language may be used for the determi(cid:173)
`nation of the digital media assets to present to the user.
`In one embodiment, the software may be a web-based
`application containing program modules. The program mod(cid:173)
`ules may include Java servlets, Java Server Pages (JSPs),
`HyperText Markup Language (HTML) pages, Joint Photo(cid:173)
`graphic Expert Groups (.JPEG) images, Macromedia Flash
`MX movies, and/or a reusable Macromedia Flash MX com-
`
`16
`ponent. The software may be executed on a compatible server
`environment including a web server, servlet container, Struc(cid:173)
`tured Query Language (SQL) database and .lava Database
`Connectivity (.IDBC) driver.
`The Macromedia Flash MX movies and the reusable Mac(cid:173)
`romedia Flash MX component may include multiple Macro(cid:173)
`media Flash MX source files. A programmer may supply a
`first file that contains code for a Time Frame component
`and/or a reusable Flash MX component that implements the
`user side of the trigger point 320. An implementation may
`include visually framing the image to be displayed and resiz-
`ing the image to be displayed to fit the frame, if necessary. For
`example, a programmer may supply a second file that
`includes code having two Time Frame instances and three
`15 buttons per Time Frame, the buttons including a "Wam1er"
`button, a "Colder" button and a "Reset" button. The
`"Warmer" button may set a variable indicative of an afllnity
`value to a lower value and load an image ( or images) from the
`server that correspond to the new variable value. Similarly,
`20 the "Colder" button may set the afllnity value variable to a
`higher value and load an image ( or images) from the server
`that correspond to the new variable value. The "Reset" button
`may reset the variable to a mid-range value or the previously
`stored value for the user. As an alternative to the second file,
`25 a third file may be stored including a Time Frame instance, a
`"Load preferred image" button and two or more text entry
`boxes. The user may utilize text entry boxes to enter, for
`example, an affinity group name and a username. When the
`user enters valid information into both text entry boxes and
`30 clicks on the "Load preferred image" button, the infomiation
`may be sent to the server. The server may use a database table
`to select an image based on the received infomiation and may
`return the selected image to the user.
`The application software may include multiple database
`35 tables such as tables of internal narrative perception identifi(cid:173)
`cation frameworks, current users, user specific social affini(cid:173)
`ties, user specific emotional affinities, and/or trigger points.
`In an embodiment, the application software may include a
`table that specifies an image that best represents the element
`40 for a specific afllnity element group and affinity element.
`The application soft\vare may include one or more HTML
`pages used to access the Macromedia Flash MX source files
`and to update the stored user Profile. The application may
`include one or more Java servlets. In an embodiment, a first
`45 Java servlet is utilized to find the afllnity elements having the
`maximum value for the specific user, among all afllnity ele(cid:173)
`ments in a specified group, and return the image correspond(cid:173)
`ing to that element having the maximum value. In the embodi(cid:173)
`ment, a second Java servlet is utilized to display the afllnity
`50 values for the user, the affinity type and the afllnity element
`group and to provide a means for the creative director 110 to
`update the afllnity values.
`The application software may include a plurality of JPEG
`image files that are provided from one or more sources. The
`55 sources may include any public source of image files, public
`copyrighted files with an appropriate copyright agreement or
`private files
`FIG.14 illustrates the construction ofan Interacting Musi(cid:173)
`cal Intersode (!MI). When used herein, Interactive Musical
`60 Intersode (IMI) may refer to an embodiment of a personalized
`digital media narrative program that is used to create a per(cid:173)
`sonalized internal narrative experience for the user. Referring
`again to FIG. 14, a window 1400 is presented which contains
`the IMI 1420 and a toolbar 1410 for the construction of the
`65 IMI 1420. In one embodiment, the creative director 110/
`distributor 102 views the elements which comprise the digital
`media asset repository. In an embodiment, these elements
`
`10Tales0000035
`
`

`

`Case 6:20-cv-00810-ADA Document 73-6 Filed 04/23/21 Page 4 of 4
`
`US 8,856,030 B2
`
`17
`include audio 1430, video option/graphic option track 11432,
`which comprises background option 1, video option/graph
`option track 2 1434, which comprises background option 2,
`overlay text option l 1436 and overlay text option 2 1438. In
`one embodiment, each of these digital media asset options is
`linked to an affinity, such that switching can occur between
`these elements at the appropriate trigger points 320. In one
`embodiment, the IMI is realized using Flash such that over(cid:173)
`lays are created and switching occurs between background
`overlays and the appropriate audio to create the IMI 1420.
`FIG. 15 illustrates an exemplary registration screen of an
`IMI in which an audience member is presented with a log on
`window 1520 in order to view the IMI. In an embodiment,
`having the user log on creates the ability to either retrieve
`information about that user from the database or create a new
`entry in the database about the user.
`FIG. 22A is a flow chart that illustrates an embodiment of
`the present system at work. A user may access a network or
`website at step 2200. The network or website may determine
`whether the user is a new user or a repeat user at step 2205.
`This check may include reviewing the user's cookies or sim(cid:173)
`ply asking the user to enter user identification ( e.g. username
`and password). If the user is a repeat visitor, the user may be
`asked to input a user name and password at the login step
`2210. The user profile may then be loaded into the system. If
`the user is a new user, a user profile may be created at step
`2220. The created user profile may then be loaded into the
`system. In an embodiment, the created profile contains at
`least a username and a password. The profile optionally
`includes demographic data such as the user's gender, age,
`regional location and ethnic background. At step 2230, the
`user may select a digital media presentation. As the digital
`media experience begins, the default digital media presenta(cid:173)
`tion may be presented for the user's viewing and/or listening
`pleasure. During the viewing of the digital media presenta(cid:173)
`tion, at step 2240 the trigger points 320 may be compared
`with the tags 1010 stored in the user's personal profile to
`determine if the default digital media asset video sequence
`300 should be changed. If no tag 1010 is present for the tested
`trigger point, the digital media asset video sequence 300 may
`be viewed unchanged until the next trigger point 320. If a tag
`is present, the content of the digital media may be changed
`according to the stored user's personal profile 561.
`In one example, an advertiser that manufactures various
`types of pet food, including dog food and cat food, forms an
`agreement with a record label that distributes music videos on
`the Internet for free viewing. A new music video of a popular
`artist may include a scene, segment or image having a dog or
`cat walk across the background to eat from a bowl of food or
`simply have a dog or cat graphic. Sitting next to the bowl of
`food is a bag labeled with one of the advertiser's brand name
`of pet food. Upon entering the website, the user's personal
`profile may be accessed. The profile may include inilJrmation
`that the individual is a dog lover and/or dog owner. During
`playback of the video, a selection of the species ofanimal may
`be determined at the trigger point 320 based on the viewer's
`profile. For example, a tag 1010 in the profile may indicate to
`insert a dog into the video. Insertion ofa dog into the media as
`opposed to a cat increases both the effectiveness of the adver(cid:173)
`tising, by allowing the advertiser to highlight dog food to a
`dog lover, and the enjoyment of the video, since a dog lover is
`more likely to enjoy a music-based digital media experience
`featuring a dog. Thus, both the advertiser and the artist may
`benefit from the enhanced digital media being presented to
`viewers. Furthern10re, the personal profile may further indi(cid:173)
`cate a preferred breed of dog, such as golden retriever or
`terrier. If such inforniation is specified, the specific breed of
`
`5
`
`18
`dog or cat may be inserted at the trigger point 320. The affinity
`of the user to the breed ofanimal may result in the user feeling
`more personally connected to the video.
`FIG. 16 illustrates exemplary digital media assets in the
`form of specialized background materials 200 illustrating one
`part of the world (the continent of South America) and back(cid:173)
`ground materials 200 including a woman's eye and face. In
`the event that it is determined from the user profile that the
`user has an appropriate ( or positive) affinity for South
`10 America, this background may be selected for presentation.
`Similarly, the woman's eye (or eye color) and face (or hair
`color) may be selected if it is determined that they would
`create a better emotional experience for the user. By provid-
`15 ing this material to that particular user an enhanced emotional
`experience may occur for the user.
`FIG. 17 illustrates a screen with personalized content in the
`itJrm of digital media in which an automobile segment pro(cid:173)
`duced by the creative director 110 is shown on the screen
`20 along with a young man and a young woman. In one embodi(cid:173)
`ment, these images are produced in conjtmction with the
`audio, such that the user hears the artist's song and sees this
`specialized content and background material to create an
`emotionally enhanced experience. In one embodiment, the
`25 digital assets used to create the personalized digital media
`asset are selected based on the user profile and the ability to
`optimize the emotional experience for the user. These assets
`may include the make and/or color of the automobile, the
`ethnic background of the young man and young women, and
`30 even the color of the eye in the background.
`FIG. 18 illustrates a screen posing a question relating to an
`IMI to a user. In this screen, the user is asked a question 1820
`and can respond in textual form. The system may use the
`35 response to determine the user's perception of the IMI and
`consequently the user's preferences within the media narra(cid:173)
`tive form. The system may further assess the user's potential
`affinity and add it to the user's profile.
`FIG. 19 illustrates a screen posing a multiple choice ques-
`40 tion 1902 relating to a media narrative experience. The user
`may respond to the question by selecting an answer. The
`selected answer may provide information to the system
`regarding the user's desires, preferred affinities and other
`internal narrative perceptual identification based attributes. In
`45 FIGS. 18 and 19, digital media assets are presented on the
`screen and may be customized according to the updated
`user's profile based upon the answers to those or previous
`questions posed to the user.
`FIG. 20 represents an alternative screen for posing ques-
`50 tions 2002 within a digital media asset based experience to a
`user in which the user is asked to respond to a particular
`question and a particular character 2004, in this case from a
`TV series. Upon responding, the system may store informa(cid:173)
`tion about the user and update the experience or presentation
`55 based on the current answer.
`FIG. 21 illustrates a screen having a "chat type" window
`2102 relating to an experience in which case the user can
`communicate with other users of the same experience. One
`advantage of this embodiment is that the users can share their
`60 comments on that experience with each other in either an
`anonymous or non-anonymous format. In another embodi(cid:173)
`ment, users can simultaneously log on to an experience in a
`customized format. In this embodiment, the users can com(cid:173)
`municate with each other through a chat type window, e-mail,
`65 instant messenger, or other communication mechanism to
`discuss their emotional experience and put themselves into
`the story or the narrative experience. Such communication
`
`10Tales0000036
`
`