throbber
Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 1 of 24
`Case 4:20-cv-01509—DMR Document 1-1 Filed 02/28/20 Page 1 of 24
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`EXHIBIT A
`
`EXHIBIT A
`
`
`
`

`

`USO095651 49B2
`
`(12) United States Patent
`Coatta et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 9,565,149 B2
`Feb. 7, 2017
`
`(54) MEDIA MESSAGING METHODS, SYSTEMS,
`AND DEVICES
`
`(56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`(71) Applicant: Brilliant Mobile, L.L.C., Chanhassen,
`MN (US
`(US)
`
`(72) Inventors: Jay David Coatta, Chanhassen, MN
`(US); John Mikkelsen, Wayzata, MN
`(US); Sergey Tolkachev, Minneapolis,
`MN (US); Robert Freidson, Saint
`Petersburg (RU); Max Freidson, Saint
`Petersburg (RU
`g (RU)
`(73) Assignee: Phoji, LLC. Minneapolis, MN (US)
`(*) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 285 days.
`(21) Appl. No.: 14/072,418
`(22) Filed:
`Nov. 5, 2013
`
`(65)
`
`Prior Publication Data
`US 2014/0129.650 A1
`May 8, 2014
`Related U.S. Application Data
`(60) Provisional application No. 61/722,560, filed on Nov.
`5, 2012, provisional application No. 61/835,975, filed
`on Jun. 17, 2013.
`(51) Int. Cl.
`G06F 5/16
`H04L 2/58
`(52) U.S. Cl.
`CPC ..................................... H04L 51/10 (2013.01)
`(58) Field of Classification Search
`CPC .......... H04L 51/10; H04L 12/58; H04L 29/06;
`G06Q 10/107
`USPC ................ 709/204, 205, 206, 217, 218, 219
`See application file for complete search history.
`
`(2006.01)
`(2006.01)
`
`7,035,803 B1 * 4/2006 Ostermann et al. .......... TO4,260
`2002.0137507 A1* 9, 2002 Winkler ................ HO4L 12.583
`455,412.1
`2006.0053386 A1* 3, 2006 Kuhl ..................... G06F 3,0236
`71.5/773
`2006/0129633 A1* 6/2006 Potluri et al. ................. TO9,203
`2007/0203995 A1* 8/2007 Wang et al. ...
`TO9,206
`(Continued)
`FOREIGN PATENT DOCUMENTS
`
`KR
`KR
`
`4/2005
`10-2005-0037070
`2, 2007
`10-0680O34
`(Continued)
`
`OTHER PUBLICATIONS
`
`“Notification of Transmittal of the International Search Report and
`the Written Opinion of the International Searching Authority, or the
`Declaration” for PCT/US2013/035209 filing date Nov. 5, 2013, 4
`pageS.
`
`(Continued)
`Primary Examiner — Liangche A Wang
`Assistant Examiner Kaylee Huang
`(74) Attorney, Agent, or Firm — Patterson, Thuente,
`Pedersen, P.A.
`ABSTRACT
`(57)
`A wireless communications system that allows a mobile
`phone, tablet or personal computer user the ability to initiate
`the sending of a text message or email whereby the sender
`is able to include photographs, graphs, pie charts and the like
`within the flow of the actual word by word texting or email
`writing process, without depending on the traditional nec
`essary step to “attach' the photograph.
`13 Claims, 14 Drawing Sheets
`
`
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 2 of 24
`
`

`

`US 9,565,149 B2
`Page 2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`2007/0266090 A1* 11/2007 Len ....................... GO6F 17,211
`TO9.204
`
`2008/0285071 A1 11, 2008 Mori et al.
`2009/0055484 A1* 2/2009 Vuong et al. ................. TO9,206
`2009.0156170 A1* 6/2009 Rossano et al. ........... 455,412.1
`2013/001224.5 A1
`1/2013 Choi ................. HO4M 3/42.382
`455,466
`2014/0101553 A1* 4/2014 Nagel ................... G06F 3.0482
`71.5/728
`2014/0143330 A1* 5, 2014 Shaw ..................... G06Q 50/01
`TO9.204
`
`
`
`FOREIGN PATENT DOCUMENTS
`
`KR
`WO
`
`10-2009-0057655
`WO 2004/017596
`
`6, 2009
`2, 2004
`
`OTHER PUBLICATIONS
`
`International Preliminary Report on Patentability from PCT Appli
`cation PCT/US2013/068509, dated May 14, 2015, 8 pgs.
`* cited by examiner
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 3 of 24
`
`

`

`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 4 of 24
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 4 of 24
`
`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 1 of 14
`
`US 9,565,149 B2
`
`1&3!
`
`3m .
`
`
`gygfigm
`fiemmaj
`
`
` @mwans-Commun (5555
`
`fierswaé Library
`
`
`
`
`
`
`
`
`
`Wei: zswgfiicafii‘ime Mime; agispzsiii‘miarzsa
`
`
`Fig. l
`
`

`

`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 2 of 14
`
`US 9,565,149 B2
`
`a
`
`28
`
`roo.
`8
`:
`8:38:
`83;xicatios:
`
`
`
`
`
`'-'-'-'-'-'-'-'-'.
`
`i:
`::
`$8:338
`:grw:“
`
`
`
`8:8:
`
`
`
`
`
`exti&essaging
`$8:::::8
`
`ixiefs
`88:
`i:8:y 3:
`88:338
`
`
`
`
`
`o
`it
`i.
`i-Mar Protocar Proteca.
`-MMP exting wet socket básed Protocol
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 5 of 24
`
`Fig. 2
`
`

`

`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 3 of 14
`
`US 9,565,149 B2
`
`
`
`
`
`
`
`E.
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 6 of 24
`
`

`

`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 4 of 14
`
`US 9,565,149 B2
`
`&
`
`Cloud
`wet Service
`
`
`
`8
`
`&
`&
`&
`&
`
`MMP organizer
`--
`
`
`
`
`
`Pisotocon's Es
`
`yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 7 of 24
`
`Fig. 4
`
`MMP exting
`
`

`

`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet S of 14
`
`US 9,565,149 B2
`
`
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 8 of 24
`
`Fig. 5
`
`

`

`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 6 of 14
`
`US 9,565,149 B2
`
`
`
`Exx: 8:888:
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 9 of 24
`
`Fig. 6
`
`

`

`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 7 of 14
`
`US 9,565,149 B2
`
`
`
`Scrossing
`
`is extreis
`
`Fig. 7
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 10 of 24
`
`

`

`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 8 of 14
`
`US 9,565,149 B2
`
`
`
`Organizer
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 11 of 24
`
`

`

`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 12 of 24
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 12 of 24
`
`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 9 of 14
`
`US 9,565,149 B2
`
`
`
`........................§
`QM? CLZEfiT ~ SERVER Wrofiac $3
`
`
`
`1 33“,.
`
`“= 3’?
`
`
` W3? mem’fw PmmmE
`
`
`
`
`WIMP ”filming? Megsagéng memi
`
`
`
`£358! 2
`
`Kisser "Z
`
`
`
`Texiing 2’
`{mapmder}
`Messaging
`{imfimer‘}
`
`
`Sewfie
`
`
`
`
`
`ding tr
`3
`‘.
`"- Til)”,
`.
`..
`.
`,
`.M"
`
`'= #Wfig‘m “33° m 3
`21. ire-{mm Sfimz‘é if:
`g
`_
`.
`x
`«i
`_
`_Q f5? .m
` :1
`
`, me an in ”'3‘
`3mm: 3,;er 22m {Jim
`scsaérsn smegma c;
`"
`
`
`
`22" “v 322 “1",x {
`x...
`
`Web Service
`
`233%?
`F‘hmewns
`
`
`
`{923mm}
`
`Library Same .
`
`3
`1
`3
`
`3
`:
`-'
`
` ;
`
`fihcfiocon refiners;
`
`
`mm fies: US$355;
`
`
`
`
`i? Mama? mats:
`
`
`
`
`
`
`éi Magma» deiiraemzs a";
`User 2 Er: £12m: {mt-ifs”: car
`E
`5
`if555E$5§§SF§21§ 1'0- Lésefl
`Memsagge send in
`
`Naiifimfinn farm?
`
`
`3 Mwasga pmgsarm
`2 mfléfig 35's,?em
`{m- sgmfiefizzgg in 5,5322?
`
`weperw,
`
`.'
`
`:
`
`_:
`
`if}. Emma
`
`
`
`
`
`Fig. 9
`
`

`

`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 10 of 14
`
`US 9,565,149 B2
`
`
`
`
`
`33.38:8:
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 13 of 24
`
`8.33:38:8:3
`
`Fig. 10
`
`

`

`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 11 of 14
`
`US 9,565,149 B2
`
`input
`
`*ressess
`
`stipat
`
`
`
`i. Pictestion ibrary Service
`
`
`
`
`
`
`
`-
`
`... ii:38 388: 8xxo:
`13. Assign proper it as -
`3.3, 88ogarize .ixaty
`3.8, sewirie views act existing
`toois 3 cc 3: 88.28
`if 888
`8, 88t: 838 &; gi:is
`18quiest
`
`2. exting Service
`
`
`
`3, .. 3i: 88f 3:{x}:8
`8.2. Esiasis gi:8::::::: xii:
`wis-à-vis
`2.3, 388v8 essages 388xi
`{:} :8388 888
`3.3. 388 88ssages 38
`{ei}: 8xxxxxis for itse
`*88si:ing
`
`3:38.8
`
`aw
`
`*88:3:38
`
`Photocons
`
`8:8° exting
`&ia:3888
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 14 of 24
`
`;
`
`3.3. iter-re: 88i set:328i
`3.3. *:::::::::::::::38
`
`
`
`:
`
`Fig. 11
`
`

`

`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 12 of 14
`
`US 9,565,149 B2
`
`
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 15 of 24
`
`

`

`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 13 of 14
`
`US 9,565,149 B2
`
`--------------------------------------
`
`3:33
`
`save picturer
`
`a---
`
`
`
`i&is
`*:::::::g::8
`p:rsora itsary
`
`ise
`
`Ste: i.
`xiak88 3: $888; it: 88 (3
`saw: it *ote:{x}; i.ittay
`
`
`
`Sir
`&pgicatio:
`
`Step: 2.
`8: 83:38::::::
`i:it:388
`with 8:8 cite register:
`&
`S:8:
`
`-
`
`Y
`
`&is
`{x}8:88:38:3:38
`Syster
`
`1.&
`&
`
`-
`323
`
`
`
`x8
`Step 3.
`s& X:
`yes 3r. 888 essages
`x8
`with MMP tags within 1
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 16 of 24
`
`323
`
`Fig. 13
`
`

`

`U.S. Patent
`
`Feb. 7, 2017
`
`Sheet 14 of 14
`
`US 9,565,149 B2
`
`Pair text MMP message
`
`Parsed and rendered essage
`
`ir
`1 hi 8c:38: working 838 of
`i&isió:38::::::.
`...
`
`be
`
`8: 8888t: viorking 8:38
`ieyek;88::: p33}.
`sixxix."
`X
`
`sight {xx' --> wic::ier where air fight
`3:::::::::::::::::::.
`
`(location)
`
`Food is great:
`
`So, tikirt; if sic:3 stay is: {xug;&
`ge cays (weeks (3: 38y 38 years.
`
`&c, 88king : : sixxix: stay
`if {x}{3:33:38: Riays
`weeks 33 83y 88 years.
`
`$8is is y ::::::::stats.3 tie at:
`:
`Sinai: isote: (cal 1)
`
`is is try caret state citi
`art smartphore:
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 17 of 24
`
`
`
`
`
`Your frienci - Yo: Si:{x:
`
`Fig. 14
`
`

`

`1.
`MEDIA MESSAGING METHODS, SYSTEMS,
`AND DEVICES
`
`US 9,565,149 B2
`
`CLAIM TO PRIORITY
`
`This application claims the benefit of U.S. Provisional
`Patent Application 61/772,560 entitled “RMS Communica
`tion Service' filed Nov. 5, 2012, and U.S. Provisional Patent
`Application 61/835,975 entitled “Reach Messaging Meth
`ods, System, and Service' filed Jun. 17, 2013, each of which
`is incorporated herein by reference.
`
`10
`
`TECHNICAL FIELD
`
`This application relates generally to electronic device
`communication, and more particularly, to systems, methods,
`and devices for the insertion of media data within electronic
`device communication.
`
`15
`
`BACKGROUND
`
`In a traditional texting or messaging environment, in
`order to insert media data other than text, a user must go
`through a cumbersome process of attaching the media data,
`select the media data through a series of menus, or otherwise
`go through similar cumbersome processes of attaching the
`media data. Further, such inserted data is typically generic
`and impersonal to the users.
`Therefore, there is a need for a messaging system that
`optimizes the information power of the communication
`process between users and further adds a personal feeling
`and experience, which in turn, Substantially increases the
`amount of information per message.
`
`SUMMARY
`
`Embodiments of the present application substantially
`meet the aforementioned needs of the industry. In an
`embodiment, a Media Messaging Platform (MMP) is a
`cloud-based communication platform, allowing mobile and
`tablet users to seamlessly integrate texts, images, photo
`graphs audio and video (Rich Media Data) within texting
`and messaging interactions with correspondents without
`having to interrupt the mobile user's texting flow skipping
`the laborious “step” of “attaching such rich media data into
`a mobile user's message.
`In embodiments, MMP provides users with a semantically
`organized multi-dimensional communication space for per
`Sonalizing rich media content (for example, images, photo
`graphs, audio, video, etc.) and integrating the content into a
`more emotionally enriched communication experience
`between MMP users.
`In embodiments, MMP can include a set of tools for
`content organization and integration of Such personalized
`rich media content within mobile related texts, chats and
`email communications.
`In embodiments, MMP’s “Content Organizer visually
`presents Rich Media Data Objects in the form of unique
`icons called “Photocons' or “Phoji’s.’ Photocons and Phojis
`are real photos of faces, brands, logos, animals and any other
`true photographic images (although the MMP shortcuts and
`“Content Organizer” can also be used with traditional non
`photographic emojis or emoticons, as discussed below).
`According to embodiments, a wireless communications
`platform allows a mobile phone, tablet or personal computer
`user the ability to initiate the sending of a text message or
`email whereby the sender is able to include photographs,
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 18 of 24
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`graphs, pie charts and the like within the flow of the actual
`word by word texting or email writing process, without
`depending on the traditional necessary step to “attach” said
`photograph (graph, pie chart and the like) as is currently
`experienced with all existing texting and email platforms.
`As the user types email or text message, user is able to
`incorporate photographs, graphs, pie charts and the like by
`tapping on, clicking on or dragging existing photographs
`stored on the user's device. User also having the option to
`use “qwerty symbol short-cuts' like the exclamation point,
`colon, semi colon and the like which are pre-assigned by
`user to user-identified photographs, pie charts, graphs and
`the like, giving user the ability to instantly “shortcut the
`inclusion of said photographs, images, pie charts graphs and
`the like within the user's texting flow experience.
`For example, user is writing an email explaining with
`words a particular “financial ownership break down. Rather
`than the user attaching an image of a "pie chart graph' to the
`bottom of the email (which must then be opened by receiver
`separately taking reader away from text), the user (sender)
`can simply type “pc' within the flow of their email typing
`and the image of said "pie chart graph will automatically
`“place itself within the paragraph of the words being typed,
`the symbol “pc' having been pre-assigned to said image of
`"pie chart graphic' within user's device or photograph
`library.
`A similar example can be made within a text messaging
`experience between users in a chat or texting experience. For
`example, the platform user can type a text message to a
`friend explaining, “We won the game!' Traditional texting
`platforms allow a user to add Small, animated "Smiley
`faces.” “sad faces.” “heart symbols and the like. These
`“non-rich media icons' have come to be known worldwide
`as ubiquitous yellow “smileys, otherwise known as “emoti
`cons” or “emojis.” These small animated icons are not real
`photographs or real images, yet add to the expression of
`texted words within a given message's context.
`According to embodiments, the platform allows the user
`to send the same expressive text message, “We won the
`game adding a real photograph of say the team cheering,
`or the individual smiling. Thus the emotional experience by
`the receiver of the text message is far more personal as it is
`a photo of the actual person who just “won the game' rather
`than a “small yellow smiley face' which may or may not
`represent the truest nature of the text message.
`A text message might be sent reading, "Let's have a beer
`. . .
`. The sender typing the symbol “beer allowing the
`pre-assigned photograph of a tall cold mug of beer to follow
`the text message without the extra step of “attaching” said
`photograph. User can insert or otherwise couple an unlim
`ited number of photographs within the texting cell, which
`cannot be accomplished by “attaching photos' within the
`traditional texting experience.
`In an embodiment, a communications system comprises a
`first user device presenting a first user interface; a second
`user device presenting a second user interface; a messaging
`cloud including: at least one database of rich media data, a
`messaging service configured to: receive a message, the
`message including a rich media data request from the first
`user, retrieve the requested rich media data from the at least
`one database of rich media data, insert the retrieved rich
`media data into the message, and present the message to the
`second user device via the second user interface; and a
`network operably coupling the first user device, the second
`user device, and the messaging cloud, wherein the message
`
`

`

`3
`is originally formatted on the first user device via the first
`user interface Such that the rich media data request is within
`the message.
`In an embodiment, a method of streamlined media inser
`tion on an electronic device comprises initiating, by the
`electronic device, a messaging session with a receiving user;
`receiving a message on the electronic device, the message
`including at least one media insertion request; evaluating the
`at least one media insertion request; retrieving media from
`at least one database of media based on the evaluated at least
`one media insertion request; and inserting the retrieved
`media into the message.
`In an embodiment, a wireless device in electronic com
`munication with a messaging service along a wireless net
`15
`work, the messaging service configured to evaluate at least
`one media insertion request, retrieve media from at least one
`database of media based on the evaluated at least one media
`insertion request, insert the retrieved media into the mes
`sage, and present the message to a receiving user, the
`wireless device comprises a memory; and a processor in
`electronic communication with the memory, the processor
`configured to: initiate a messaging session with a receiving
`user, receive a message including at least one media inser
`tion request, and transmit the message to the messaging
`service.
`The above summary of the invention is not intended to
`describe each illustrated embodiment or every implementa
`tion of the present invention. The figures and the detailed
`description that follow more particularly exemplify these
`embodiments.
`
`10
`
`25
`
`30
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The invention may be more completely understood in
`consideration of the following detailed description of vari
`ous embodiments of the invention in connection with the
`accompanying figures, in which:
`FIG. 1 is a block diagram of an MMP Architecture,
`according to an embodiment of the invention.
`FIG. 2 is a block diagram of an MMP Cloud Architecture,
`according to an embodiment of the invention.
`FIG. 3 is an illustration of an MMP Semantic Model,
`according to an embodiment of the invention.
`45
`FIG. 4 is a block diagram of components of an MMP
`architecture, according to an embodiment of the invention.
`FIG. 5 is an illustration of raw pictures in a picture library,
`according to an embodiment of the invention.
`FIG. 6 is an illustration of an MMP Organizer Layout
`page, according to an embodiment of the invention.
`FIG. 7 is a block diagram of the design structure of a
`mobile MMP application, according to an embodiment of
`the invention.
`FIG. 8 is a block diagram of generic Smartphone views
`and controls, according to an embodiment of the invention.
`FIG. 9 is a flow diagram of MMP information and
`communication, according to an embodiment of the inven
`tion.
`FIG. 10 is an example illustration a photocon library
`having images and the respective emotional tags, according
`to an embodiment of the invention.
`FIG. 11 is a diagram of interaction between MMP users in
`relation to one or more MMP databases, according to an
`embodiment of the invention.
`FIG. 12 is an MMP database class diagram, according to
`an embodiment of the invention.
`
`35
`
`40
`
`50
`
`55
`
`60
`
`65
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 19 of 24
`
`US 9,565,149 B2
`
`4
`FIG. 13 is a block diagram of interaction between MMP
`users in relation to Personal Environments and Services,
`according to an embodiment of the invention.
`FIG. 14 is a diagram of an example inserting Photocons/
`Phojis, according to an embodiment of the invention.
`While embodiments are amenable to various modifica
`tions and alternative forms, specifics thereof have been
`shown by way of example in the drawings and will be
`described in detail. It should be understood, however, that
`the intention is not to limit the invention to the particular
`embodiments described. On the contrary, the intention is to
`cover all modifications, equivalents, and alternatives falling
`within the spirit and scope of the invention as defined by the
`appended claims.
`
`DETAILED DESCRIPTION OF THE DRAWINGS
`
`According to embodiments of the invention, MEDIA
`MESSAGING PLATFORM (MMP) is an alternative Mobile
`Messaging Communication Service to Short Messaging
`Service (SMS) and Multimedia Messaging Service (MMS)
`implemented in such applications like Skype, Microsoft IM,
`ICO, Google Chat, etc.
`MMP improves Social messaging and texting services by
`increasing information power of communication process
`between participants by adding “personal feeling and expe
`rience, which should optimize an amount of information per
`message.
`In an embodiment, MMP uses two form of MMP of
`exchange objects: Messages and Texts. Message is HTML
`string, which constructed by the Sender using internal
`HTML format. Receiver will see them in traditional rich web
`format. Such Message when arrived at Receiver's environ
`ment will be properly rendered by Web browser or by MMP
`Application. Text is a plain string where tags presented as a
`combination of characters on the Sender's side. On the
`Receiver's side, these tags would be interpreted and substi
`tuted with rich HTML objects (pictures, audio, etc.).
`In an embodiment, MMP can operate in two service
`modes: Interactive Chat and Notification Service. MMP
`Interactive Chat allows participants to exchange with Mes
`sages and Texts while both participants are being online in
`real time. MMP Notification Service allows sending Texts
`and Messages without all participants having to be online
`simultaneously.
`MMP can comprise a family of protocols and applica
`tions, which allows Socially engaged users to include real
`photographs or another "rich objects within a text messages
`without having to “attach' the photo to the text or interrupt
`user’s “texting flow.”
`Similar to the Emoticon's libraries inserted inside the
`texting field, the MMP real photos icons (Photocons) are
`similarly inserted from the personal "emotional library
`within the user's texting field. The user can add his/her
`personalized photos and replace standard Emoticons in
`MMPUser's Photocon Library or alternatively the user can
`assign additional text symbols, numbers and letters to asso
`ciated photographs for future “easy insert” in texting field.
`During the chat session or preparing SMS message in the
`framework of Notification Service the user can open the
`MMP Photocon Library and click on a MMP Photocon or,
`alternatively the user can assign text symbols, numbers and
`letters to associated photographs.
`The assigned symbols, letters or numbers can be typed
`while texting, resulting in immediate and automatic place
`ment of the associated Photocon where the symbol, number
`or letter was typed. An example of the uninterrupted “Pho
`
`

`

`6
`fication includes ping as one of the variety of notifications,
`when one of MMP users enters MMP and his/her correspon
`dent is offline (hibernate).
`MMP Photocon Library System and Database/MMP Seman
`tic Model
`In embodiments, a Photocon Library 222 is a Cloud
`Service and Database System based on a MMP Semantic
`Model illustrated in FIG. 3. See also FIG. 10. This model
`allows categorizing of Personal Image Library and signifi
`cantly optimizes the image search during chatting and
`texting. It provides personal rich media content organization
`based on multi-dimensional Social and emotional content
`classification, preparing it for seamless integration into
`mobile messaging, comprising: semantically organized
`multi-dimensional communication space and multilayer rich
`content structure; content representation, which associated
`with every particular contact person in user's contact list,
`including types of interpersonal relationship (friendship,
`family, professional, romantic, etc.) and user's current emo
`tional state 810 (Happy 811, Excited 812, Angry 813, etc.).
`In an example, consider the following scenario:
`Step 1. A user selects the contact for MMP chat session. This
`selection will automatically select the corresponding layer
`with a unique set of Photocons.
`Step 2. The user is able to define his/her “Emotional State'
`or “Temperature' for this particular chat by using slider 810
`or sequential cyclic color setter for MMP Mobile applica
`tion. This selection will pick out one particular column
`(color) on the selected layer: green (Happy 811), yellow
`(Excited 812) or red (Angry 813).
`Step 3. During the dialogue, the user is able to dynamically
`change the current emotional state 810 of the dialogue and
`shortcut will select the Photocon from the correspondent
`column automatically.
`In embodiments, a particular Substitution of emoticon's
`symbol instead of the shortcut will depend on the current
`square of the Semantic Model 800, namely on contacts,
`dialogue “temperature' and current emotion 810.
`MMP Web Service
`In embodiments, MMP Cloud Web Service 221 connects
`users to MMP Photocons Library System 203 and Database
`204, and at the same time provides support for the Social
`Networks content share and other traditional Web services.
`FIG. 4 illustrates the components of MMP architecture at a
`system level.
`MMP Cloud Web Service 221 integrates different System
`components such as MMP Organizer 220, MMP Parser 230,
`MMP Application 240, the Photocon's IDs component 231
`and Main UI 250 to organize rich media personal content
`and to create individual layer for each person from user's
`contact list.
`MMP Parser 230 analyzes text string and substitute MMP
`shortcuts with appropriate Photocons. MMP Application
`240 is a platform dependent implementation of MMP client
`side functions such as photo transfer, asynchronous push
`notification and other. Photocons' IDs 231 is a part of MMP
`Database storing and accessing relations between Photocons
`and Rich Media Objects. Main UI 250 is client based
`integration of MMP Web based components and platform
`dependent applications 240.
`MMP Organizer 220 provides an ability to upload rich
`content items from the Web, desktop or from a Picture
`Library, such as that illustrated in FIG. 5 on smartphones or
`tablets. An example MMP Organizer layout page is illus
`trated in FIG. 6.
`MMP Organizer allows user to collect, classify and rear
`range Photocons, as well as to initiate chats and invite others
`
`Case 4:20-cv-01509-DMR Document 1-1 Filed 02/28/20 Page 20 of 24
`
`40
`
`45
`
`65
`
`US 9,565,149 B2
`
`5
`
`10
`
`15
`
`5
`tocon Placement within the user texting experience would
`be the ubiquitous “smiley face' emoticon. For example, the
`universal “colon symbol combined with a “right parentheti
`cal symbol' automatically places a “smiley face' emoticon
`within the user's text message where the combo of these two
`symbols is typed.
`MMP has the universal QWERTY keyboard symbols
`associated with all existing emoticons, e.g. Smiley, Sad,
`Angry, Nerd, Heart, Crying, Yawn and so forth. Upon a
`user's MMP application download to their mobile device,
`the MMP application can include a library of stock Photo
`COS.
`System Architecture
`According to an embodiment, the MMP general architec
`ture 100 incorporates multiple components, applications and
`environments, as illustrated in FIG. 1. Architecture 100 can
`include MMP Cloud 201, MMP Communication System
`202, and MMP Photocons' Personal Library 203 and further
`includes MMP Personal Environment 300, which in turn
`contains Mobile Applications 301 and Web applications 302.
`An embodiment of the System’s Cloud architecture is
`depicted in FIG. 2. It includes a representation of main MMP
`components 100 and a representation of the communication
`protocols 110, 111, 112 between structural elements of MMP
`25
`System.
`MMP Communication System
`According to an embodiment, MMP Communication Sys
`tem 202 comprises different modules performing system
`critical tasks and provides connection and command inter
`pretation received from users’ devices 300. In embodiments,
`system 202 can include MMP Web Service 221, MMP
`Photocons Library and Database 222 and MMP Text/Mes
`saging Service 223.
`35
`MMP Text/Messaging Service
`In embodiments, MMP Text/Messaging Service 223 com
`prises a module configured to handle the Texting/Messaging
`modality. The interaction can be handled both ways using
`duplex channels based on MMP Photocon Protocol 111 and
`HTML5 Web socket texting protocol 112, in an embodi
`ment. The System provides interface and command proces
`Sor to interpret tags and Support interaction with Photocon
`Library and Database System 222.
`MMP Notification System
`In embodiments, MMP Notification System 210 provides
`delivery and notification mechanism to users currently
`offline using MMP Notification Protocol 110. Notification
`System 210 checks if a user has subscribed for SMS
`notification and, if yes, will send SMS and/or e-mail with
`50
`URL to MMP formatted message. If not, Notification Sys
`tem 210 will periodically check user status and will deliver
`the message when it is possible. Notification System 210
`includes custom alert Sounds notification when it is received
`by a MMP user. A MMP user is able to transmit a photocon
`55
`to a non-MMP user's mobile phone number using MMP
`Demo account invitation, in embodiments.
`MMP Push Notifications System
`In embodiments, MMP Push Notification System com
`bines sound and vibration alerts with specific pictures and
`60
`automatic actions such as open chat with calling person,
`show a picture or play an audio or video clip. MMP Mobile
`Applications 310 installed on mobile device is capable to
`receive notifications even in a “hibernate” mode. MMP Push
`Notification System includes protocols, which are capable to
`send “push notifications” from MMP Text/Messaging Ser
`vice 223 to MMP Mobile Application 310. The push noti
`
`30
`
`

`

`8
`MMP Protocols can define the set of control commands
`and its flow that manages the users' texting/messaging
`session. FIG. 9 depicts interaction between users and Pho
`tocon Library 222 and the information flow between the
`modules of the System based on BMP Protocols. It also
`depicts the architecture of communication and connections
`between System components. The three major Services are
`the backbone of MMP system: Web Service 221, Photocons
`Library (for example, that shown in FIG. 10) and Database
`Service 222 and Texting/Messaging Service 223.
`An interaction between the Users through these major
`modules 221, 222 and 223 is established and controls by
`Photocons Protocol 111 and Texting/Messaging Protocol
`112, which shown in FIG. 9 and can be illustrated by the
`following steps and control commands according to the
`session scenarios:
`1. User 1 (initiator) initiates texting/messaging session with
`User 2 (responder) via module 223.
`2. Module 223 sends invitation to User 2 and Chat session
`is established or request for Notification sent to User 1
`3. User 1 sends Text or Message to User 2 via module 223.
`4. Module 223 sends text to the Parser of the module 221
`with the request: “Photocon included (yes/no'?)
`5. Module 221 sends Photocon request to the Library 222.
`6. Module 222 checks if Photocon URL exists in the Library
`(yes/no?) and sends respond to module 223.
`7. Module 223 wrapped Photocon into internal format and
`prepared Photocon for transferring to User 2 (responder).
`8. Message delivered to User 2 in Chat mode or Message
`sent to User 2 in Notification format.
`9. Message prepared for rendering in User 2 texting system
`and sent to module 222.
`10. Module 222 provides Image to User 2.
`11. User 1 sends request to module 222 to saves Image into
`Photocon Library.
`12. Image saved into Photocon Library by module 222.
`13. User 2 requests the Photocon Library 222 to show the list
`of available Photocons.
`14. Module 222 provides User 1 with the List of Photocons
`available for selection.
`MMP Integrated Environment
`In embodiments, MMP Integrated Environment top-level
`structure consists of three main services: Photocon Library
`Service, Texting Service and Presentation Service. FIG. 11
`depicts a diagram of interaction between MMP users in
`conjunction with MMP Database 204, including data struc
`tures for personalization database 204-1 and structure for
`texting and messaging database 204-2. As depicted, the
`interaction can comprise a number of Subsections.
`In an embodiment, the photocon library service can
`interface to MMP users 103 via Design Structure 320 and
`personalization database 204-1. In an embodiment, the pho
`tocon library service is configured to, among other pro
`cesses, initiate a user account, upload raw images, assign
`proper ID emotional tags to images, organize and reorganize
`the library, provide views and editing tools to crop and resize
`images, and return image(s) on the user's request.
`In an embodiment, the texting service can interface to
`MMP

This document is available on Docket Alarm but you must sign up to view it.


Or .

Accessing this document will incur an additional charge of $.

After purchase, you can access this document again without charge.

Accept $ Charge
throbber

Still Working On It

This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.

Give it another minute or two to complete, and then try the refresh button.

throbber

A few More Minutes ... Still Working

It can take up to 5 minutes for us to download a document if the court servers are running slowly.

Thank you for your continued patience.

This document could not be displayed.

We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.

Your account does not support viewing this document.

You need a Paid Account to view this document. Click here to change your account type.

Your account does not support viewing this document.

Set your membership status to view this document.

With a Docket Alarm membership, you'll get a whole lot more, including:

  • Up-to-date information for this case.
  • Email alerts whenever there is an update.
  • Full text search for other cases.
  • Get email alerts whenever a new case matches your search.

Become a Member

One Moment Please

The filing “” is large (MB) and is being downloaded.

Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.

Your document is on its way!

If you do not receive the document in five minutes, contact support at support@docketalarm.com.

Sealed Document

We are unable to display this document, it may be under a court ordered seal.

If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.


Access Government Site

We are redirecting you
to a mobile optimized page.





Document Unreadable or Corrupt

Refresh this Document
Go to the Docket

We are unable to display this document.

Refresh this Document
Go to the Docket