( 12 ) United States Patent
`Scofield
`
`US 11,040,725 B2
`( 10 ) Patent No .:
`( 45 ) Date of Patent :
`Jun . 22 , 2021
`
`US011040725B2
`
`( 54 ) MANUAL VEHICLE CONTROL
`NOTIFICATION
`( 71 ) Applicant : INRIX Inc. , Kirkland , WA ( US )
`( 72 ) Inventor : Christopher L. Scofield , Seattle , WA
`( US )
`( 73 ) Assignee : INRIX Inc. , Kirkland , WA ( US )
`Subject to any disclaimer , the term of this
`( * ) Notice :
`patent is extended or adjusted under 35
`U.S.C. 154 ( b ) by 228 days .
`( 21 ) Appl . No .: 14 / 845,356
`Sep. 4 , 2015
`( 22 ) Filed :
`( 65 )
`Prior Publication Data
`US 2017/0066452 A1
`Mar. 9 , 2017
`Int . Ci .
`B60W 50/14
`B60K 28/00
`G08G 1/16
`GO8G 1/0967
`G05D 1/00
`( 52 ) U.S. CI .
`CPC
`
`B60W 50/14 ( 2013.01 ) ; B60K 28/00
`( 2013.01 ) ; G05D 1/0061 ( 2013.01 ) ; G08G
`1/096725 ( 2013.01 ) ; G08G 1/096791
`( 2013.01 ) ; G08G 1/163 ( 2013.01 ) ; G08G
`1/165 ( 2013.01 ) ; B60W 2050/143 ( 2013.01 ) ;
`B60W 2540/22 ( 2013.01 ) ; B60W 2554/00
`( 2020.02 ) ; G05D 2201/0213 ( 2013.01 )
`( 58 ) Field of Classification Search
`B60W 50/14 ; B60K 28/00 ; G05D 1/0061 ;
`CPC
`GO5D 1/0088 ; GO8G 1/096725 ; GOOG
`1/096791 ; GO8G 1/163 ; G08G 1/165
`See application file for complete search history .
`
`( 51 )
`
`( 2020.01 )
`( 2006.01 )
`( 2006.01 )
`( 2006.01 )
`( 2006.01 )
`
`( 56 )
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`8,718,861 B1
`9,690,292 B1 *
`
`5/2014 Montemerlo et al .
`6/2017 Chan
`( Continued )
`
`G06K 9/00845
`
`OTHER PUBLICATIONS
`Transition to manual : Driver behaviour when resuming control from
`a highly automated vehicle http://www.sciencedirect.com/science/
`article / pii / S1369847814001284 Transportation Research Part F :
`Traffic Psychology and Behaviour , vol . 27 , Part B , Nov. 2014 by
`Merat et al . ( Year : 2014 ) . *
`( Continued )
`Primary Examiner Donald J Wallace
`( 74 ) Attorney , Agent , or Firm — Cooper Legal Group ,
`LLC
`ABSTRACT
`( 57 )
`One or more techniques and / or systems are provided for
`notifying drivers to assume manual vehicle control of
`vehicles . For example , sensor data is acquired from on
`board vehicles sensors ( e.g. , radar , sonar , and / or camera
`imagery of a crosswalk ) of a vehicle that is in an autono
`mous driving mode . In an example , the sensor data is
`augmented with driving condition data aggregated from
`vehicle sensor data of other vehicles ( e.g. , a cloud service
`collects and aggregates vehicle sensor data from vehicles
`within the crosswalk to identify and provide the driving
`condition data to the vehicle ) . The sensor data ( e.g. , aug
`mented sensor data ) is evaluated to identify a driving
`condition of a road segment , such as the crosswalk ( e.g. ,
`pedestrians protesting within the crosswalk ) . Responsive to
`the driving condition exceeding a complexity threshold for
`autonomous driving decision making functionality , a driver
`alert to assume manual vehicle control may be provided to
`a driver .
`
`20 Claims , 7 Drawing Sheets
`
`500
`
`START
`
`502
`
`DENTIFY CURRENT LOCATION OF VEHICLE THAT IS IN
`AUTONOMOUS DRIVING MODE
`
`DETERMINE ROUTE OF VEHICLE
`
`ACQUIRE VEHICLE SENSOR DATA OF ONE OR MORE VEHICLES
`TRAVELING ROAD SEGMENT OF ROUTE
`
`AGGREGATE VEHICLE SENSOR DATA TO DETERMINE DRIVING
`CONDITION DATA
`
`PROVIDE INSTRUCTION TO VEHICLE TO DETERMINE WHETHER TO
`PROVIDE DRIVER WITH DRIVER ALERT
`
`END
`
`514
`
`504
`
`506
`
`508
`
`510
`
`512
`
`IPR2025-00943
`Tesla EX1022 Page 1
`
`

`

`( 56 )
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`10,241,509 B1 *
`2003/0050740 A1
`2012/0235819 A1 *
`
`3/2019 Fields
`3/2003 Fecher et al .
`9/2012 Watkins
`
`6/2014 Cullinane
`2014/0156133 A1 *
`2014/0303827 A1 * 10/2014 Dolgov
`2/2015 Schnieders
`2015/0051781 A1 *
`3/2015 Yopp
`6/2015 Attard et al .
`8/2015 Offenhaeuser
`
`2015/0175070 A1
`2015/0229885 A1 *
`
`2015/0066282 A1 *
`
`2015/0241878 A1 *
`
`2016/0026182 A1 *
`
`8/2015 Crombez
`1/2016 Boroditsky
`
`US 11,040,725 B2
`Page 2
`
`2016/0107655 Al *
`
`4/2016 Desnoyer
`
`2016/0185387 A1 *
`
`6/2016 Kuoch
`
`2016/0200317 A1 *
`
`7/2016 Danzl
`
`2016/0252903 A1 *
`
`9/2016 Prokhorov
`
`2016/0363935 A1 * 12/2016 Shuster
`2017/0021837 A1 *
`1/2017 Ebina
`2017/0220039 A1 *
`8/2017 Funakawa
`2018/02 16958 A1 *
`8/2018 Park
`
`B60W 50/14
`701/23
`BOOK 35/00
`701/41
`B60W 10/10
`701/25
`B60W 30/143
`701/23
`B60L 7/10
`B60W 50/082
`GO5D 1/0061
`GO6Q 30/02
`
`OTHER PUBLICATIONS
`Int . Search Report / Written Opinion cited in PCT Application No.
`PCT / US2016 / 050118 dated Feb. 13 , 2017 , 15 pgs .
`Corresponding Great Britain Patent Application No. GB1802953.8 ,
`Examination Report dated Jan. 21 , 2021 .
`* cited by examiner
`
`G07C 5/0808
`
`A61B 5/18
`340 / 573.1
`B60W 30/00
`701/23
`B60W 30/00
`701/23
`B60W 40/08
`701/23
`GO5D 1/0061
`701/24
`
`GO8G 1/165
`348/78
`GO5D 1/0088
`701/23
`HO4L 67/306
`701/23
`
`IPR2025-00943
`Tesla EX1022 Page 2
`
`

`

`U.S. Patent
`
`Jun . 22 , 2021
`
`Sheet 1 of 7
`
`US 11,040,725 B2
`
`100
`
`START
`
`102
`
`ACQUIRE SENSOR DATA FROM ONE OR MORE ON - BOARD
`VEHICLE SENSORS OF VEHICLE IN AUTONOMOUS DRIVING MODE
`
`EVALUATE SENSOR DATA TO DETERMINE DRIVING CONDITION OF
`ROAD SEGMENT
`
`GENERATE DRIVER ALERT TO ASSUME MANUAL VEHICLE
`CONTROL
`
`PROVIDE DRIVER ALERT TO DRIVER OF VEHICLE AT POINT OF
`TIME BASED UPON DRIVING CONDITION
`
`104
`
`106
`
`108
`
`110
`
`END
`
`112
`
`FIG . 1
`
`IPR2025-00943
`Tesla EX1022 Page 3
`
`

`

`U.S. Patent
`
`Jun . 22 , 2021
`
`Sheet 2 of 7
`
`US 11,040,725 B2
`
`200
`
`204
`
`202
`206
`
`SENSOR
`
`DRIVER ALERT
`COMPONENT
`
`C
`
`A
`
`210
`
`...
`
`{
`
`1
`
`208
`
`DRIVER , PLEASE CONSIDER
`MANUAL VEHICLE CONTROL
`FOR UPCOMING CROSSWALK IN
`200 METERS
`
`FIG . 2
`
`IPR2025-00943
`Tesla EX1022 Page 4
`
`

`

`U.S. Patent
`
`Jun . 22 , 2021
`
`Sheet 3 of 7
`
`US 11,040,725 B2
`
`300
`
`304
`
`306
`
`WEATHER SERVICE
`
`TRAFFIC SERVICE
`
`308
`INFORMATION
`SERVICE
`
`DRIVING CONDITION AGGREGATOR
`
`302
`
`VEHICLE SENSOR
`DATA
`
`324
`
`*
`
`INSTRUCTION
`
`301
`
`??
`
`314
`
`?? ?
`
`312
`
`310
`
`? ? ?? ?? ?? ?
`
`?? 318
`
`320
`
`316
`
`322
`
`326
`
`DRIVER , PLEASE CONSIDER
`MANUAL VEHICLE CONTROL
`FOR ICY ROAD CONDITIONS INA !
`QUARTER OF A MILE
`
`FIG . 3
`
`IPR2025-00943
`Tesla EX1022 Page 5
`
`

`

`U.S. Patent
`
`Jun . 22 , 2021
`
`Sheet 4 of 7
`
`US 11,040,725 B2
`
`400
`
`404
`
`ZZZ
`
`402
`406
`
`SENSOR
`
`DRIVER ALERT
`COMPONENT
`STUDENT DRIVER
`
`O
`
`410
`
`hr
`my
`
`000000000
`
`FIG . 4
`
`IPR2025-00943
`Tesla EX1022 Page 6
`
`

`

`U.S. Patent
`
`500
`
`Jun . 22 , 2021
`
`Sheet 5 of 7
`
`US 11,040,725 B2
`
`START
`
`502
`
`504
`
`506
`
`508
`
`510
`
`512
`
`IDENTIFY CURRENT LOCATION OF VEHICLE THAT IS IN
`AUTONOMOUS DRIVING MODE
`
`DETERMINE ROUTE OF VEHICLE
`
`ACQUIRE VEHICLE SENSOR DATA OF ONE OR MORE VEHICLES
`TRAVELING ROAD SEGMENT OF ROUTE
`
`AGGREGATE VEHICLE SENSOR DATA TO DETERMINE DRIVING
`CONDITION DATA
`
`PROVIDE INSTRUCTION TO VEHICLE TO DETERMINE WHETHER TO
`PROVIDE DRIVER WITH DRIVER ALERT
`
`514
`
`END
`
`FIG . 5
`
`IPR2025-00943
`Tesla EX1022 Page 7
`
`

`

`U.S. Patent
`
`Jun . 22 , 2021
`
`Sheet 6 of 7
`
`US 11,040,725 B2
`
`600
`
`602
`
`I 1
`
`1
`
`}
`}
`}
`}
`
`}
`}
`{
`}
`}
`}
`}
`
`}
`}
`
`1
`1
`1
`
`}
`
`604
`
`COMPUTER
`INSTRUCTIONS
`
`}
`}
`
`606
`
`01011010001010
`10101011010101
`101101011100 ...
`
`1
`
`608
`
`COMPUTER READABLE
`MEDIUM
`
`FIG . 6
`
`IPR2025-00943
`Tesla EX1022 Page 8
`
`

`

`U.S. Patent
`
`Jun . 22 , 2021
`
`Sheet 7 of 7
`
`US 11,040,725 B2
`
`700
`
`}
`}
`{
`}
`}
`}
`}
`
`}
`}
`}
`
`}
`}
`
`{
`}
`}
`}
`}
`
`714
`
`716
`
`PROCESSING
`UNIT
`
`MEMORY
`
`-718
`
`--
`
`|
`
`-
`
`712
`
`1
`1
`1
`
`1
`1
`1
`1
`1
`1
`1
`1
`
`1
`1
`1
`
`720
`
`722
`
`724
`
`726
`
`STORAGE
`
`OUTPUT DEVICE ( S )
`
`INPUT DEVICE ( S )
`
`COMMUNICATION
`CONNECTION ( S )
`
`NETWORK
`
`728
`
`COMPUTING
`DEVICE
`
`-730
`
`FIG . 7
`
`IPR2025-00943
`Tesla EX1022 Page 9
`
`

`

`US 11,040,725 B2
`
`20
`
`1
`MANUAL VEHICLE CONTROL
`NOTIFICATION
`
`2
`level of the driver , a mental state of the driver , etc. ) , a state
`of traffic , and / or potential alternate routes .
`In another embodiment of notifying a driver to assume
`manual vehicle control , a driving condition aggregator may
`BACKGROUND
`5 identify a current location of a vehicle that is in an autono
`mous driving mode . In an example , the driving condition
`Many autonomous driving systems support a spectrum of
`aggregator may be hosted by a server , such as a cloud
`vehicle autonomy . For example , an autonomous driving
`service having communication connections with various
`system may manage speed , distance to other vehicles by
`vehicles . The driving condition aggregator may determine a
`braking or adjusting speed , lane following , turning , etc. In
`this way , a user may be able to perform other tasks , such as 10 route of the vehicle ( e.g. , the route may be determined as a
`eating , working , engaging in a conversation , resting , etc.
`predicted route of the vehicle based upon one or more prior
`Unfortunately , the autonomous driving system may encoun-
`trips of the vehicle ) . The driving condition aggregator may
`ter situations that may be too complex ( e.g. , pedestrians may
`acquire vehicle sensor data from one or more vehicles
`be entering a crosswalk , leaving the crosswalk , approaching
`traveling a road segment , of the route , not yet traveled by the
`the crosswalk with a bike , stopped within the crosswalk , 15 vehicle . The driving condition aggregator may aggregate the
`etc. ) . Without the ability to provide a driver with an appro-
`vehicle sensor data to determine driving condition data for
`priately timed alert to take manual control of the vehicle
`the road segment ( e.g. , vehicle cameras , stability systems ,
`( e.g. , a timely alert so that the user can adequately resume
`braking patterns , and / or other sensor data may indicate black
`control of the vehicle ) , the autonomous driving system may
`ice on the road segment ) . The driving condition aggregator
`implement sub - optimal driving decisions .
`may provide an instruction , comprising the driving condi
`tion data ( e.g. , a location and icy condition of the road
`SUMMARY
`segment ) , to the vehicle . The instruction may specify that the
`vehicle ( e.g. , a driver alert component ) is to determine
`This summary is provided to introduce a selection of
`whether to provide a driver of the vehicle with a driver alert
`concepts in a simplified form that are further described 25 to assume manual vehicle control of the vehicle based upon
`below in the detailed description . This summary is not
`the driving condition data .
`intended to identify key factors or essential features of the
`To the accomplishment of the foregoing and related ends ,
`claimed subject matter , nor is it intended to be used to limit
`the following description and annexed drawings set forth
`the scope of the claimed subject matter .
`certain illustrative aspects and implementations . These are
`Among other things , one or more systems and / or tech- 30 indicative of but a few of the various ways in which one or
`niques for notifying a driver to assume manual vehicle
`more aspects may be employed . Other aspects , advantages ,
`control are provided . In an embodiment of notifying a driver
`and novel features of the disclosure will become apparent
`to assume manual vehicle control , a driver alert component
`from the following detailed description when considered in
`may be configured to acquire sensor data from one or more
`conjunction with the annexed drawings .
`on - board vehicle sensors of a vehicle that is in an autono- 35
`mous driving mode ( e.g. , information from a sonar detector ,
`DESCRIPTION OF THE DRAWINGS
`a radar detector , a camera , a stability system , an anti - lock
`braking system , a thermostat , etc. ) . In an example , the driver
`FIG . 1 is a flow diagram illustrating an exemplary method
`alert component may be integrated into the vehicle or into a
`of notifying a driver to assume manual vehicle control .
`computing device associated with the vehicle . In an 40
`FIG . 2 is a component block diagram illustrating an
`example , the driver alert component may evaluate the sensor
`exemplary system for notifying a driver to assume manual
`data to determine a driving condition of a road segment that
`vehicle control , where a driver alert component utilizes
`is to be traveled by the vehicle ( e.g. , a crosswalk , that is 300
`sensor data from on - board vehicles sensors for generating a
`meters from the vehicle , may be identified has having 4
`driver alert .
`pedestrians within the crosswalk ,
`1 cyclist and a dog 45
`FIG . 3 is a component block diagram illustrating an
`approaching the crosswalk , 2 pedestrians entering the cross-
`exemplary system for notifying a driver to assume manual
`walk , etc. ) . In another example , the driver alert component
`vehicle control , where a driving condition aggregator pro
`may transmit the sensor data to a remote computing device ,
`vides an instruction , comprising driving condition data , to a
`such as a cloud service hosting a driving condition aggre-
`vehicle .
`gator , for processing ( e.g. , the driving condition aggregator 50
`FIG . 4 is a component block diagram illustrating an
`exemplary system for notifying a driver to assume manual
`may evaluate the sensor data and / or vehicle sensor data from
`other vehicles to create , and transmit back to the driver alert
`vehicle control , where a cognitive load of a driver is
`component , driving condition data and / or an instruction as
`evaluated to determine if and how to provide a driver alert .
`FIG . 5 is a flow diagram illustrating an exemplary method
`to whether a driver alert should be provided ) .
`Responsive to determining that the driving condition 55 of notifying a driver to assume manual vehicle control .
`exceeds a complexity threshold for autonomous driving
`FIG . 6 is an illustration of an exemplary computer read
`decision making functionality ( e.g. , more than a threshold
`able medium wherein processor - executable instructions
`number of objects are within or near the crosswalk ) , a driver
`configured to embody one or more of the provisions set forth
`alert for the driver to assume manual vehicle control may be
`herein may be comprised .
`generated . The driver alert may be provided to the driver at 60
`FIG . 7 illustrates an exemplary computing environment
`a point in time , relative to encountering the road segment ,
`wherein one or more of the provisions set forth herein may
`based upon the driving condition ( e.g. , a point in time that
`be implemented .
`affords the driver enough time to assume manual vehicle
`control for adequate decision making ) . In an example , the
`DETAILED DESCRIPTION
`driver alert and / or the point in time may be determined based 65
`upon a cognitive load of the driver ( e.g. , a current activity of
`The claimed subject matter is now described with refer
`the driver , an awareness of the driver , a driving expertise
`ence to the drawings , wherein like reference numerals are
`
`IPR2025-00943
`Tesla EX1022 Page 10
`
`

`

`US 11,040,725 B2
`
`3
`4
`driving impediment of a group of parade attendees walking
`generally used to refer to like elements throughout . In the
`up the expressway onramp to enter a parade route along a
`following description , for purposes of explanation , numer-
`portion of the expressway .
`ous specific details are set forth to provide an understanding
`At 108 , responsive to determining that the driving con
`of the claimed subject matter . It may be evident , however ,
`that the claimed subject matter may be practiced without 5 dition exceeds a complexity threshold for autonomous driv
`ing decision making functionality , a driver alert to assume
`these specific details . In other instances , structures and
`manual vehicle control may be generated . For example , the
`devices are illustrated in block diagram form in order to
`complexity threshold may correspond to a number of pedes
`facilitate describing the claimed subject matter .
`trians in a crosswalk , a conditions and / or parameters that
`An embodiment of notifying a driver to assume manual
`vehicle control is illustrated by an exemplary method 100 of 10 autonomous driving decision making functionality is unable
`to interpret ( e.g. , an airplane flying towards the vehicle
`FIG . 1. At 102 , the method 100 starts . At 104 , sensor data
`during an emergency landing ) , conditions and / or parameters
`may be acquired from one or more on - board vehicles
`that are computationally too expensive to calculate within an
`sensors of a vehicle that is in an autonomous driving mode .
`adequate time for resolving a driving action before reaching
`For example , radar , sonar , cameras , and / or other sensors of 15 the road segment ( e.g. , autonomous driving decision making
`the vehicle may collect information regarding an express-
`functionality may be unable to timely resolve a driving
`way onramp road segment . In an example where the vehicle
`decision before reaching the expressway onramp ) , an inabil
`may have a communication connection with a remote com-
`ity to resolve a driving decision that provides a safe or
`puting device such as a cloud service hosting a driving
`desirable result ( e.g. , the ethical decision corresponding to
`condition aggregator , driving condition data for the express- 20 the car , unable to stop in time , that can either swerve out of
`way onramp road segment may be received from the remote
`a pedestrian's way into onc ncoming traffic or onto a sidewalk
`computing device over the communication connection . The
`with other pedestrians ) , etc.
`driving condition data may correspond to an aggregation of
`At 110 , the driver alert may be provided to a driver of the
`vehicle sensor data from one or more vehicles while the one
`vehicle at a point in time , relative to encountering the road
`or more vehicles were traveling the expressway onramp road 25 segment , based upon the driving condition . In an example ,
`segment ( e.g. , vehicles may have reported , to the remote
`a determination as to whether to provide the driver alert to
`computing device , information regarding braking patterns ,
`the driver may be performed based upon a cognitive load of
`speed , deceleration , vehicles being driven in reverse , and / or
`the driver . For example , the cognitive load of the driver may
`other information indicative of a traffic obstruction on the
`be determined using sensors ( e.g. , a camera or microphone
`expressway onramp road segment ) . In an example , the 30 may obtain audio visual information indicating that the user
`driving condition data may correspond to weather data ,
`is sleeping , eating , conversing with someone , working on a
`event data ( e.g. , a news service may indicate a parade on the
`laptop , etc. ) , evaluating user data ( e.g. , the user may post to
`expressway ) , vehicle sensor data ( e.g. , a vehicle changing
`a social network “ taking a nap now , I don't feel well ” , a
`into a reverse gear ) , driving impediment data ( e.g. , a traffic
`driver profile may indicate that the user is a student driver ,
`service may indicate that there is blocked traffic on the 35 etc. ) , etc. In this way , the cognitive load may be determined
`expressway onramp road segment ) , and / or data from a
`based upon a current activity of the driver , an awareness of
`service ( e.g. , social network posts describing the parade ) . In
`the driver , a driving expertise level of the driver , or a mental
`this way , the sensor data may be augmented with the driving
`state of the driver ( e.g. , the driver may be returning home
`from a bar , and thus should not be given control of the
`condition data .
`In an example , the driving condition aggregator ( e.g. , 40 vehicle , but the vehicle should instead merely pull over until
`the driving condition has been resolved ) .
`hosted by the remote computing device to provide driving
`condition data and / or instructions to vehicles ) and / or a
`In an example , the point in time may be determined so that
`driver alert component ( e.g. , hosted by the vehicle to provide
`the driver has adequate time to switch cognitive states into
`driver alerts ) may utilizing travel prediction functionality to
`a manual driving cognitive state and / or so that the driver is
`predict a route of the vehicle . For example , the route may be 45 not notified too early such that the driver losses attention by
`predicted based upon previous trips by the vehicle ( e.g. , the
`the time the road segment is reached . The point in time may
`driver may generally use the expressway to travel to soccer
`be determined based upon the cognition load of the driver ,
`practice on Saturday afternoons ) . A road segment , such as
`a state of traffic ( e.g. , a current speed of the vehicle , a time
`the expressway onramp road segment , may be identified
`to reach the road segment , a number of vehicles near the
`based upon the route and a current location of the vehicle . 50 vehicle , traffic congestion , etc. ) , a potential alternate route
`In this way , driver alerts may be generated for road segments
`( e.g. , so that the driver has adequate time to decide whether
`to use the potential alternate route to avoid the road seg
`not yet traveled by vehicles .
`At 106 , the sensor data ( e.g. , augmented with the driving
`ment ) , etc. In this way , the driver alert may be provided to
`condition data if available ) may be evaluated to determine a
`the driver .
`driving condition of the expressway onramp road segment 55
`In an example , a second driving condition , for a second
`that is to be traveled by the vehicle . The driving condition
`road segment that is to be traveled by the vehicle , may be
`may correspond to a road condition ( e.g. , icy , wet , an oil
`received from the remote computing device over the com
`spill , a pot hole , etc. ) , a weather condition ( e.g. , freezing
`munication connection . For example , the second road seg
`rain ) , a pedestrian ( e.g. , a biker , people in a crosswalk , a dog
`ment may comprising a city street , 5 miles from the vehicle ,
`in the street , ducks crossing the street , etc. ) , an accident , an 60 leading into a soccer field for Saturday soccer practice for
`ethical decision ( e.g. , a car , unable to stop in time , can either
`which the driver is predicted to be traveling to as a desti
`swerve out of a pedestrian's way into oncoming traffic or
`nation for the route . Vehicles may report erratic braking
`onto a sidewalk with other pedestrians ) , a driving impedi-
`patterns and / or weaving patterns indicative of surprise road
`ment ( e.g. , construction , a road barricade , a police car
`conditions . Such information may be aggregated to deter
`pulling over another car , an ambulance , etc. ) , etc. For 65 mine the second driving condition that is provided to the
`example , the expressway onramp road segment may be
`vehicle . Responsive to determining that the second driving
`determined to have a driving condition corresponding to
`condition exceeds the complexity threshold for autonomous
`
`IPR2025-00943
`Tesla EX1022 Page 11
`
`

`

`US 11,040,725 B2
`
`6
`5
`sensor data 314. In an example , the driving condition
`driving decision making functionality , a second driver alert
`aggregator 302 may augment the vehicle sensor data 314
`for manual vehicle control may be generated . The second
`with information from a weather service 304 ( e.g. , providing
`driver alert may be provided to the driver at a second point
`below freezing temperature data ) , a traffic service 306 ( e.g. ,
`in time based upon the second driving condition . For
`example , the second driver alert may be provided 0.25 miles 5 indicating accidents near the road segment 310 from ice ) ,
`and / or any other information service 308 ( e.g. , a news
`from the city street . At 112 , the method 100 ends .
`FIG . 2 illustrates an example of a system 200 , comprising
`service , a website , a social network , etc. ) . In this way , an
`a driver alert component 204 , for notifying a driver of a
`instruction 324 , comprising the driving condition data , may
`vehicle 202 to assume manual vehicle control . The driver
`be provided to the vehicle 322. The instruction 324 may
`alert component 204 may acquire sensor data from one or 10 specify that the vehicle 322 , such as the driver alert com
`more on - board vehicles sensors of the vehicle 202 that is in
`ponent 318 , is to determine whether to provide a driver of
`an autonomous ( e.g. , fully autonomous , semi - autonomous ,
`the vehicle 322 with a driver alert 326 to assume manual
`etc. ) driving mode . For example , a sensor 206 , such as a
`vehicle control of the vehicle 322 based upon the driving
`radar detector , a camera , a sonar detector , etc. , may capture
`condition data ( e.g. , provide an audible or visual alert
`sensor data corresponding to a crosswalk 210. The driver 15 “ driver , please consider manual vehicle control for icy road
`alert component 204 may evaluate the sensor data to deter-
`conditions in a quarter of a mile ” ) .
`mine a driving condition of a road segment , such as the
`FIG . 4 illustrates an example of a system 400 , comprising
`crosswalk 210 , which is predicted to be traveled by the
`a driver alert component 404 , for notifying a driver of a
`vehicle 202. For example , the driving condition may indi-
`vehicle 402 to assume manual vehicle control . The driver
`cate that 3 pedestrians are near the crosswalk 210 and 2 20 alert component 404 may acquire sensor data from one or
`pedestrians are within the crosswalk 210 .
`more on - board vehicles sensors of the vehicle 402 that is in
`The driver alert component 204 may evaluate the driving
`an autonomous ( e.g. , fully autonomous , semi - autonomous ,
`condition against a complexity threshold of autonomous
`etc. ) driving mode . For example , a sensor 406 , such as a
`driving decision making functionality ( e.g. , a predicted
`radar detector , a camera , a sonar detector , etc. , may capture
`calculation time to resolve an adequate autonomous driving 25 sensor data corresponding to a zone in front of the vehicle
`action for the driving condition ; an unknown constraint or
`402. The driver alert component 404 may evaluate the
`parameter unable to be processed by autonomous driving
`sensor data to determine a driving condition of a road
`decision making functionality ; an inability to resolve a safe
`segment , such as the zone , that is to be traveled by the
`autonomous driving action for the driving condition ; an
`vehicle . For example , the driving condition may indicate
`ethical decision , etc. ) . Responsive to the driving condition 30 that two cows 410 are obstructing the road segment .
`exceeding the complexity threshold , the driver alert com-
`The driver alert component 404 may determine that the
`ponent 204 may generate a driver alert 208 to assume
`driving condition exceeds a complexity threshold of autono
`manual vehicle control of the vehicle 202 ( e.g. , “ driver ,
`mous driving decision making functionality because the
`please consider manual vehicle control for upcoming cross-
`autonomous driving decision making functionality does not
`walk in 200 meters ” ) . The driver alert 208 may be provided 35 understand how to resolve an appropriate driving action for
`to the driver at a point in time , relative to encountering the
`avoiding the two cows 410. Before generating a driver alert
`road segment , based upon the driving condition .
`to assume manual vehicle control of the vehicle 402 , the
`FIG . 3 illustrates an example of a system 300 , comprising
`driver alert component 404 may evaluate a cognitive load of
`a driving condition aggregator 302 , for notifying a driver of
`the driver . For example , a camera may indicate that the
`a vehicle 322 to assume manual vehicle control . In an 40 driver is sleeping , a social network post by the user “ I am
`example , the driving condition aggregator 302 may be
`taking a nap in my vehicle ” may indicate that the driver is
`hosted on a remote computing device that is remote from the
`sleeping , and a driver profile may indicate that the driver is
`vehicle 322 ( e.g. , hosted by a cloud server ) . The driving
`a student driver . Because the cognitive load may indicate
`condition aggregator 302 may identify a current location of
`that the driver will be unable to safely assume manual
`the vehicle 322 ( e.g. , the vehicle 322 may provide global 45 control of the vehicle 402 due to the two cows 410 , the
`positioning system ( GPS ) coordinates to the driving condi-
`driver alert component 404 may pull off the road until the
`tion aggregator 302 ) that is in an autonomous driving mode .
`two cows 410 are gone or are capable of being avoided by
`For example , a driver alert component 316 may be utilizing
`autonomous driving functionality . In an example , the driver
`on - board sensors 318 to collect sensor data within a zone
`alert component 404 may provide a driver alert to the driver
`320 near the vehicle 322 for controlling speed , performing 50 that the vehicle 402 is being pulled over , and that the driver
`lane following , making turns , avoiding driving impedi-
`can take control of the vehicle 402 if so desired .
`ments , and / or for making other autonomous driving deci-
`An embodiment of notifying a driver to assume manual
`vehicle control is illustrated by an exemplary method 500 of
`sions .
`The driving condition aggregator 302 may determine a
`FIG . 5. At 502 , the method 500 starts . At 504 , a current
`route of the vehicle 322. For example , the route may be 55 location of a vehicle that is in an autonomous driving mode
`determined as a predicted route of the vehicle 322 based
`may be identified . At 506 , a route of the vehicle may be
`upon prior trips of the vehicle 322 ( e.g. , the driver generally
`determined . At 508 , vehicle sensor data may be acquired
`drives home 301 at 4:30 on Mondays ) . The driving condition
`from one or more vehicles traveling a road segment of the
`aggregator 302 may acquire vehicle sensor data 314 from
`route . The road segment may not yet be traveled by the
`vehicles 312 traveling a road segment 310 of the route . The 60 vehicle . At 510 , the vehicle sensor data may be aggregated
`road segment 310 may correspond to a portion of the route
`to determine driving condition data for the road segment . At
`not yet traveled by the vehicle 322. The vehicle sensor data
`512 , an instruction , comprising the driving condition data ,
`314 may be aggregated to determine driving condition data
`may be provided to the vehicle . The instruction may specify
`for the road segment 310. For example , the driving condition
`that the vehicle is to determine whether to provide a driver
`data may indicate black ice on the road segment based upon 65 of the vehicle with a driver alert to assume manual vehicle
`braking patterns , stability control system activation , imag-
`control of the vehicle based upon the driving condition data .
`ery , and / or other information provided through the vehicle
`At 514 , the method 500 ends .
`
`IPR2025-00943
`Tesla EX1022 Page 12
`
`

`

`US 11,040,725 B2
`
`7
`8
`cessor systems , consumer electronics , mini computers ,
`Still another embodiment involves a computer - readable
`mainframe computers , distributed computing environments
`medium comprising processor - executable instructions con-
`that include any of the above systems or devices , and the
`figured to implement one or more of the techniques pre-
`like .
`sented herein . An example embodiment of a computer-
`readable medium
`a computer - readable device is 5
`Although not required , embodiments are described in the
`or
`illustrated in FIG . 6 , wherein the implementation 600 com
`general context of “ computer readable instructions ” being
`prises a computer - readable medium 608 , such as a CD - R ,
`executed by one or more computing devices . Computer
`DVD - R , flash drive , a platter of a hard disk drive , etc. , on
`readable instructions may be distributed via computer read
`which is encoded computer - readable data 606. This com able media ( discussed below ) . Computer readable instruc
`puter - readable data 606 , such as binary data comprising at 10 tions may be implemented as program modules , such as
`least one of a zero or a one , in turn comprises a set of
`functions , objects , Application Programming Interfaces
`computer instructions 604 configured to operate according
`( APIs ) , data structures , and the like , that perform particular
`to one or more of the principles set forth herein . In some
`tasks or implement particular abstract data types . Typically ,
`embodiments , the set of computer instructions 604 are
`configured to perform a method 602 , such as at least some 15 the functionality of the computer readable instructions may
`be combined or distributed as desired in various en