Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 1 of 27
`
`
`
`
`
`
`
`
`
`
`
`
`Exhibit 12
`to the Declaration of Catherine Hartman
`Public Redacted Version
`
`
`
`
`Exhibit 12
`
`to the Declaration of Catherine Hartman
`
`Public Redacted Version
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Exhibit 12
`to the Declaration of Catherine Hartman
`Public Redacted Version
`
`
`
`
`Exhibit 12
`
`to the Declaration of Catherine Hartman
`
`Public Redacted Version
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 2 of 27
`
`Autoplay DeepDive
`charleswu
`04/2021
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051950
`
`Autoplay DeepDive
`
`charleswu
`04/2021
`
`HIGHLY CONFIDENTIAL—ATTORNEYS'EYES ONLY
`
`GoocG-SCHNDR-00051950
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 3 of 27
`
`up nein invC
`
`• •I
`
`PILOT R
`
`Tesla Autopilot
`opilot Race Across The
`Episode 2
`Coun
`-
`Ryan Trahan
`
`3:46 15:04
`
`Pyanl
`you gonna keep g' in?
`
`#TeamTrahan
`Tesla Autopilot Race Acro s The
`Country! - Episode 1
`lbellb
`453K views • 1 month ago
`
`115
`33K
`
`S
`254
`
`i•
`A>
`Dl
`Live chat Share Dow...oad Sa
`
`gik, Ryan Trahan
`W
`2.79M subscribers
`
`Comment_
`
`View all 1.9K
`
`•
`
`Ryan's content is coming diffe eat in 2021
`
`Is this video recommend tion good
`or not good for you?
`Help us improve You Tube
`ILOT RIAL Tesla Autopilot Race
`Across The Country! -
`Eoisode 2
`
`Id.18
`
`CANCEL
`
`PLAY NOW
`
`ountDown
`
`SMOwancrinnin_WI
`
`yawn.. Weed
`
`Rom Rya
`
`Up r,en m
`
`nPILOT RARE
`
`1 •
`
`Tesla Autopilot Race Across The Country!
`Episode 2
`
`CANCEL
`
`The Pase CINNEnNe NA.. To
`
`1015
`
`14.1.0
`5 7M0mra • Am. MP
`
`11:24
`
`01100.I.
`Wands
`Smals1119.9
`an Mahan 0
`1M Mom • amen. Inge
`
`OW I SIJMNII On $0.01 Re 3 Dap .
`
`'
`
`.
`
`I 907
`
`IMan Inanan 0
`7 1M yin,. • 1 year ngo
`
`1.1
`
`.0
`
`IS 03,5 02
`
`Manmimnan
`Teals Autopilot Race Across The Countryl - Episode 1
`
`0 03
`
`El 0
`
`E3
`
`7...... • ), ,,ffellinher
`
`.r0,7:a4 ,0atnoMn10.rraart 0a0
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051951
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 4 of 27
`
`Core Metrics Share
`
`Rasta
`
`VWT%
`
`Views%
`
`Enabled%
`
`All
`
`Android
`
`IOS
`
`Web
`
`TVHTML5
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051952
`
`Core Metrics Share
`
`Rasta
`
`VWT%
`
`Views%
`
`Enabled%
`
`All
`
`Android
`
`lOS
`
`Web
`
`TVHTML5
`
` HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051952
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 5 of 27
`
`Caveats
`
`• Client stops autoplay after LACT
`
`Web/Mobile
`
`TVHTML5
`
`MWeb
`
`XXm
`
`Xh
`
`XXm
`
`• Logging
`o WATCH_NEXT_AUTOPLAY
`o Try go/rtseeker-user-guide
`• Loop: ...A->B->...->A
`o Algorithm: Ep3->Ep1->Ep2->Ep3->...
`o Watch History failure
`• Most autoplay trails are short
`o Could be as long as 90+ videos though
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051953
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 6 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051954
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051954
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 7 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051955
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051955
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 8 of 27
`
`Main App Flow
`
`• Nomination
`o WatchNext nominators
`o Music nomination when there's an "autoplay seed".
`• Scoring
`o WatchNext scoring
`o Extra XLQ filtering
`• Post-processing
`o Policies and priorities
`• Override
`o TVHTML5: automix on music watch
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051956
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 9 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051957
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051957
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 10 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051958
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051958
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 11 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051959
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051959
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 12 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051960
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051960
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 13 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051961
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051961
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 14 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051962
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051962
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 15 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCH N DR-00051963
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051963
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 16 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051964
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051964
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 17 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051965
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051965
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 18 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051966
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051966
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 19 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051967
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051967
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 20 of 27
`
`Autoplay Training Data Reweighting
`
`•
`
`(with SIR team) If user opted in autoplay, but watched a non-autoplay(lower-
`position) WN suggestion, then penalize the autoplay suggestion.
`o Negative results
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051968
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 21 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051969
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051969
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 22 of 27
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051970
`
`
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051970
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 23 of 27
`
`Thank You
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051971
`
`Thank You
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051971
`
`Thank You
`
`HIGHLY CONFIDENTIAL — ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051971
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 24 of 27
`
`Notes Summary:
`
`Slide 1: 'This deep dive is about Autoplay in main app backed by WatchNext
`suggestions server. If you are watching a playlist and it autoplays across
`the video list, that's not what we are talking about.'
`Slide 2: 'Android and web UI. Count-down.'
`Slide 3: 'Overall Autoplay drives
` of YT watch time, and
`have autoplay enabled.
`Usage is higher on TV > Web > IOS/Android.
`TV users have to disable autoplay on their personal settings page, while
`other platforms the knob is on watch page.
`
`of devices
`
`*User can disable autoplay in LR on settings page.
`
`Net gain: Web
`
`, Mobile
`
`at initial launch. No ablation is running.'
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051972
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 25 of 27
`
`Notes Summary:
`
`Slide 4: 'If user is inactive for some time, Autoplay will stop by the
`client. On web and mobile autoplay stops if user stays inactive for 75m.
`
`Timeout after LACT is not the same as "autoplay time limit", since it's
`based on "last activity".'
`Slide 5: 'Impressions'
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051973
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 26 of 27
`
`Notes Summary:
`
`Slide 8: 'The post-processing stage has 8 steps.
`0. Creator redirect
`"Good" next episode
`If a good next episode is not found, we filter watch videos.
`Then for embedded yt watch,
`
`go/shows-platform for SHOW_PLAYLIST/SERIES_PLAYLIST/SHOW etc.'
`Slide 9: 'Age-restricted
`Live Event that hasn't started yet or long-running Live Event.'
`Slide 10: 'Overall only 0.015% of watches have no autoplay returned per
`request.'
`Slide 12: 'Loop Suppression is a key feature in autoplay. As we have seen
`from the post-processing steps, loop suppression is done in 2 ways:'
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051974
`
`
`
`Case 3:20-cv-04423-JD Document 209-5 Filed 12/08/22 Page 27 of 27
`
`Notes Summary:
`
`Slide 13: 'Sometimes users complain about topic-drift over an autoplay
`trail.
`
`What's tricky is how to get the seed video at serving time.
`
`We don't do this if watch history is turned off.'
`Slide 14: 'Rationale:'
`Slide 17: 'We also tried a lot of other ideas but it turned out quite
`challenging.
`Paradox'
`Slide 18: 'These are some failed attempts.'
`Slide 21: 'Loop detection: watch history'
`
`HIGHLY CONFIDENTIAL - ATTORNEYS' EYES ONLY
`
`GOOG-SCHNDR-00051975
`
`
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
