(12) United States Patent
`Caveney
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 9.483,059 B2
`Nov. 1, 2016
`
`US009483059B2
`
`(54) METHOD TO GAIN DRIVER'S ATTENTION
`FOR AUTONOMOUS VEHICLE
`(71) Applicant: Toyota Motor Engineering &
`Manufacturing North America, Inc.,
`Erlanger, KY (US)
`Derek S. Caveney, Plymouth, MI (US)
`(72) Inventor:
`(73) Assignee: Toyota Motor Engineering &
`Manufacturing North America, Inc.,
`Erlanger, KY (US)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`(21) Appl. No.: 14/554,291
`(22) Filed:
`Nov. 26, 2014
`
`(*) Notice:
`
`(65)
`
`Prior Publication Data
`US 2016/O1466 18 A1
`May 26, 2016
`
`(2006.01)
`(2006.01)
`
`(51) Int. Cl.
`GOIC 21/34
`G05D I/06
`(52) U.S. Cl.
`CPC ....................................... G05D I/06 (2013.01)
`(58) Field of Classification Search
`None
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,813,993 A * 9/1998 Kaplan ................ A61B 5,0476
`600/26
`6,748,302 B2 * 6/2004 Kawazoe ............. GOSD 1.0246
`340/937
`
`3/2010 Beneker et al.
`7,679,496 B2
`8, 2012 Nieves
`8,248,270 B2
`8,554,408 B2 10/2013 Springer et al.
`8,698,639 B2 * 4/2014 Fung ...................... B6OK 28,06
`340,576
`8,909,428 B1* 12/2014 Lombrozo ........... B62D 15,025
`TO1/41
`2004/0164851 A1* 8, 2004 Crawshaw ............. B60Q 9/008
`340/.435
`
`
`
`116,...,
`
`116
`
`116
`
`2004/0262063 A1* 12/2004 Kaufmann ............. B62D 1,286
`180/169
`2005/O115753 A1* 6, 2005 Pemberton ............. G08G 1, 164
`18Of 167
`2008, 008074.0 A1* 4/2008 Kaufmann ............. G08G 1, 167
`382/104
`2008/0183342 A1* 7/2008 Kaufmann ........... B6OK 28,066
`TO1/1
`2008/0270.018 A1* 10, 2008 Citelli ................ B6OK 31,0008
`701,532
`4, 2009 Bolourch ............ B6OK 28,066
`340/.435
`2010/0172542 A1* 7, 2010 Stein .................. GO6K9/00798
`382,103
`
`2009/0091435 A1 *
`
`9/2010 Lee et al.
`2010/0228417 A1
`9/2012 Veen et al.
`2012fO226418 A1
`11/2012 Bobbitt, III
`2012,0283939 A1
`2013,0084847 A1* 4, 2013 TibbittS ................. HO4W 8.245
`455,419
`
`2013/0342365 A1 12/2013 Kiefer et al.
`2014/0309927 A1* 10/2014 Ricci........................ B60Q 1/00
`701 424
`
`* cited by examiner
`Primary Examiner — Truc M Do
`(74) Attorney, Agent, or Firm — Christopher G. Darrow;
`Darrow Mustafa PC
`
`ABSTRACT
`(57)
`A computer-implemented method for the automated driving
`of a vehicle. The method may include coordinating a
`planned vehicle path using a path planner application. The
`path planner application may receive information based on
`inputs to sensors disposed on the vehicle. The method may
`include sending a command to one or more vehicle systems
`to control the vehicle to follow the planned vehicle path.
`While the vehicle follows the planned vehicle path, the
`method may include receiving an indication that the path
`planner application is not meeting a threshold performance
`level. After receiving the indication that the path planner
`application is not meeting the threshold performance level.
`a command is sent to one or more vehicle systems to control
`the vehicle to follow a temporary and irregular full vehicle
`movement to alert a vehicle driver. The temporary and
`irregular full vehicle movement may be a full vehicle
`side-to-side wobbling movement.
`18 Claims, 4 Drawing Sheets
`
`Coordinate a planned vehicle path
`for use with an autonomous vehicle
`
`/n 410
`
`Direct the autonomous wehicle
`to follow the planned vehicle path
`
`/n 420
`
`Monitor and receive information from one or
`more sensors of the autonomous vehicle
`J.
`Periodically determine whether a Path Planner /n 440
`Application meets a threshold performance level
`
`/ra 430
`
`Receive an indication requesting manual
`control of the autonomous vehicle
`
`/n 450
`
`Initiate a temporary and irregular full vehicle /n 460
`movement to alert a driver of the wehicle
`
`IPR2025-00943
`Tesla EX1023 Page 1
`
`

`

`U.S. Patent
`
`Nov. 1, 2016
`
`Sheet 1 of 4
`
`US 9.483,059 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Exterial Storage
`
`
`
`Operating
`System
`
`114
`
`-10
`
`110
`
`Data Analyzer
`
`Sensors
`
`Path Piafer
`
`Vehicle
`Controle
`
`Applications
`
`Coalpuig Device
`
`Vehicle
`Systems
`
`FIG. 1
`
`IPR2025-00943
`Tesla EX1023 Page 2
`
`

`

`U.S. Patent
`
`Nov. 1, 2016
`
`Sheet 2 of 4
`
`US 9.483,059 B2
`
`rera in re r s
`
`:
`
`be retres in a s r. sent us r is is peg is an ess near sy
`
`
`
`2O2
`
`216
`
`FIG. 2B
`
`204
`
`206 208
`
`IPR2025-00943
`Tesla EX1023 Page 3
`
`

`

`U.S. Patent
`
`Nov. 1, 2016
`
`Sheet 3 of 4
`
`US 9.483,059 B2
`
`
`
`
`
`Z09
`
`IPR2025-00943
`Tesla EX1023 Page 4
`
`

`

`U.S. Patent
`
`Nov. 1, 2016
`
`Sheet 4 of 4
`
`US 9.483,059 B2
`
`
`
`Coordinate a planned vehicle path
`for use With an autonomous Vehicle
`
`Direct the autonomous vehicle
`to follow the planned vehicle path
`
`Monitor and receive information from One Or
`more SenSOrS Of the autonomous vehicle
`
`Periodically determine whether a Path Planner
`Application meets a threshold performance level
`
`Receive an indication requesting manual
`COntrol Of the autonomous Vehicle
`
`Initiate a temporary and irregular full vehicle
`movement to alert a driver Of the Vehicle
`
`410
`
`420
`
`430
`
`440
`
`450
`
`460
`
`FIG. 4
`
`IPR2025-00943
`Tesla EX1023 Page 5
`
`

`

`1.
`METHOD TO GAN DRIVERSATTENTION
`FOR AUTONOMOUS VEHICLE
`
`TECHNICAL FIELD
`
`The present disclosure generally relates to methods for
`autonomous driving and, more particularly, to methods for
`gaining the attention of a driver to indicate a need for manual
`control of a vehicle.
`
`BACKGROUND
`
`The background description provided herein is for the
`purpose of generally presenting the context of the disclo
`sure. Work of the presently named inventors, to the extent it
`may be described in this background section, as well as
`aspects of the description that may not otherwise qualify as
`prior art at the time of filing, are neither expressly nor
`impliedly admitted as prior art against the present technol
`Ogy.
`Partially-automated or monitored driving systems are
`designed to assist drivers in operating a vehicle safely and
`efficiently on the road. For example, they may use tech
`niques such as eye-tracking of the driver to send a warning
`when the driver becomes inattentive, lane tracking of the
`vehicle to send a warning to the driver when the vehicle is
`leaving its lane, and controlling vehicle Velocity based on
`distance to a vehicle ahead of the driver when adaptive
`cruise control is activated by the driver.
`Fully or highly automated driving systems are preferably
`designed to operate a vehicle on a road without driver
`interaction or other external control, for example, in self
`driving or autonomous vehicles. Advanced driver safety
`systems may monitor the situation of a vehicle, including its
`location, as well as a location of other vehicles in its vicinity.
`However, fully automated driving systems are not currently
`designed to notify the driver of upcoming vehicle operations
`in order to prepare the driver in terms of what to expect from
`the automated driving systems control of the vehicle
`While certain systems may provide haptic feedback in the
`form of a vibrating steering wheel, textual warnings within
`the interior of a vehicle, or audible warnings requesting
`attention from the driver, such warnings may, in fact, be
`insufficient to alert the driver. Still further, certain warnings
`may be overbearing or distracting to the point of causing
`alarm on the part of the driver.
`Accordingly, it would be desirable to provide improved
`and reliable warning means to gain the attention of a driver
`of an autonomous vehicle when the need arises for the
`manual control of the vehicle or for another action that needs
`input from the driver.
`
`SUMMARY
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`This section provides a general Summary of the disclo
`Sure, and is not a comprehensive disclosure of its full scope
`or all of its features.
`In various aspects, the present teachings provide a com
`puter-implemented method for the automated driving of a
`60
`vehicle. The method may include coordinating a planned
`vehicle path using a path planner application, and directing
`the vehicle to follow the planned vehicle path. In various
`aspects, the method includes receiving information from one
`or more sensors of the vehicle. Where it is determined that
`the information from the one or more sensors is not sufficient
`to meet a threshold performance level, the method includes
`
`65
`
`US 9,483,059 B2
`
`2
`initiating a temporary and irregular movement to alert a
`driver of the vehicle, such as a full vehicle side-to-side
`wobbling movement.
`In other aspects, the present teachings provide an auto
`mated driving system for a vehicle. The system may include
`one or more sensors disposed on a vehicle, and a computing
`device in communication with the one or more sensors. The
`computing device may include one or more processors for
`controlling the operations of the computing device, and
`memory for storing data and program instructions used by
`the one or more processors. The one or more processors may
`be configured to execute instructions stored in the memory.
`The system may determine, using a path planner application
`receiving information based on inputs to the one or more
`sensors, a planned vehicle path. The system may send a
`command to one or more vehicle systems to control the
`vehicle to follow the planned vehicle path. Periodically, the
`system may determine whether the path planner application
`is meeting a threshold performance level. If the path planner
`application is not meeting the threshold performance level.
`the system may send a command to the one or more vehicle
`systems to control the vehicle to initiate a temporary and
`irregular full vehicle movement to alert a driver of the
`vehicle.
`In still other aspects, the present teachings provide a
`computer-implemented method for the automated driving of
`a vehicle. The method may include coordinating a planned
`vehicle path using a path planner application. The path
`planner application may receive information based on inputs
`to one or more sensors disposed on the vehicle. The method
`may include sending a command to one or more vehicle
`systems to control the vehicle to follow the planned vehicle
`path. The method may also include monitoring the planned
`vehicle path, and periodically determining whether the path
`planner application is meeting a threshold performance
`level. If the path planner application is not meeting the
`threshold performance level, the method may include send
`ing a command to one or more vehicle systems to control the
`vehicle to initiate a temporary and irregular full vehicle
`movement to alert a driver of the vehicle.
`Further areas of applicability and various methods of
`enhancing mapping technology will become apparent from
`the description provided herein. The description and specific
`examples in this Summary are intended for purposes of
`illustration only and are not intended to limit the scope of the
`present disclosure.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The present teachings will become more fully understood
`from the detailed description and the accompanying draw
`ings, wherein:
`FIG. 1 is a block diagram of an exemplary computing
`device that may be used in assisting an automated driving
`system;
`FIG. 2A is a schematic illustration of a vehicle including
`the computing device of FIG. 1;
`FIG. 2B is a schematic representation of the vehicle of
`FIG. 2A traveling in a forward direction along a roadway
`containing multiple lanes;
`FIG. 3 illustrates an exemplary portion of a navigation
`route being traversed by the vehicle of FIG. 2A and an
`example planned vehicle path along the portion of the
`navigation route; and
`FIG. 4 is a high-level flow chart illustrating a system and
`method for gaining the attention of a driver of an autono
`mous vehicle.
`
`IPR2025-00943
`Tesla EX1023 Page 6
`
`

`

`US 9,483,059 B2
`
`3
`It should be noted that the figures set forth herein are
`intended to exemplify the general characteristics of materi
`als, methods, and devices among those of the present
`technology, for the purpose of the description of certain
`aspects. These figures may not precisely reflect the charac
`teristics of any given aspect, and are not necessarily intended
`to define or limit specific embodiments within the scope of
`this technology. Further, certain aspects may incorporate
`features from a combination of figures.
`
`DETAILED DESCRIPTION
`
`10
`
`15
`
`4
`control the vehicle to follow the planned vehicle path. In
`certain aspects, if an indication is received that the path
`planner application is not meeting a threshold performance
`level, the automated driving system can be configured to
`provide a notification to a driver of the vehicle to take
`control of the vehicle if the path planner application is not
`meeting the threshold performance level. For example, the
`present technology provides the means for a driver to
`receive timely, non-intrusive instruction from the autono
`mous vehicle that manual control or input will shortly be
`required. As discussed in more detail below, the instruction
`may be provided in the form of a wobbling motion of the
`entire vehicle sufficient to alert or get the attention of a driver
`while minimizing unnecessary alarm on the part of the
`driver. It is envisioned that the present teachings provide
`safety features that improve the overall operation of the
`automated driving system as compared to prior art autono
`mous systems that may simply cancel operation of the
`automated driving system if the path planner application
`does not meet a threshold performance level.
`According to various aspects of the present technology,
`once a system or computing device of the autonomous
`vehicle has been informed or has determined that either the
`path planner is operating below a predetermined threshold
`level, or that information regarding the Surrounding envi
`ronment has been degraded such that autonomous operation
`may not be possible or recommended for much longer, the
`vehicle can be configured to initiate an irregular full vehicle
`moVement.
`In various aspects, the vehicle may be commanded to
`steer in a slight back and forth, or wobbling manner, for a
`very brief period of time. This may provide a slight jiggling
`feeling to the driver in order to indicate that the driver may
`need to take over manual control of the vehicle, or that
`certain vehicle systems may need additional information.
`The irregular full vehicle movement, or wobbling, would
`preferably occur while the vehicle generally remains on its
`planned path. In one example, as will be explained in more
`detail below, lane lines or lane markers of a roadway may be
`occluded or partially removed from the pavement such that
`the vehicle may not be able to accurately recognize lane
`information in order to fully or adequately support autono
`mous driving. In another example, a vehicle controller or
`computing device may determine that certain locating algo
`rithms are taking too long to process, and localization may
`not be occurring at a sufficient interval to accurately place
`the vehicle with respect to its surroundings. In yet another
`example, various systems may rely on the receipt of signals
`from satellites, and it may be determined that the reception
`is becoming sporadic, or the strength, quality, and/or reli
`ability of the signals may be decreasing. In all of the above
`non-limiting examples, the vehicle may still be able to
`provide Sufficient control means, but a degraded quality or
`clarity of the available information may soon require manual
`intervention to ensure future control. Thus in various
`aspects, the present teachings may also utilize the vehicle
`systems, controllers, or computing devices to determine a
`probability that a future threshold performance level can be
`met based on a current ability to analyze data or information
`from one or more vehicle sensors.
`FIG. 1 is a block diagram of an exemplary computing
`device 100 that may be used for implementing the auto
`mated driving system. The computing device 100 can be any
`type of vehicle-installed, handheld, desktop, or other form of
`single computing device, or can be composed of multiple
`computing devices. The processing unit in the computing
`device can be a conventional central processing unit (CPU)
`
`The following description is merely illustrative in nature
`and is in no way intended to limit the disclosure, its
`application, or uses. As used herein, the phrase at least one
`of A, B, and C should be construed to mean a logical (A or
`B or C), using a non-exclusive logical “or.” It should be
`understood that the various steps within a method may be
`executed in different order without altering the principles of
`the present disclosure. Disclosure of ranges includes disclo
`Sure of all ranges and Subdivided ranges within the entire
`range.
`The headings (such as “Background” and "Summary”)
`and Sub-headings used herein are intended only for general
`organization of topics within the present disclosure, and are
`not intended to limit the disclosure of the technology or any
`aspect thereof. The recitation of multiple embodiments
`having stated features is not intended to exclude other
`embodiments having additional features, or other embodi
`ments incorporating different combinations of the stated
`features.
`As used herein, the terms “comprise' and “include” and
`their variants are intended to be non-limiting, Such that
`recitation of items in Succession or a list is not to the
`exclusion of other like items that may also be useful in the
`devices and methods of this technology. Similarly, the terms
`“can and “may” and their variants are intended to be
`non-limiting, Such that recitation that an embodiment can or
`may comprise certain elements or features does not exclude
`other embodiments of the present technology that do not
`contain those elements or features.
`The broad teachings of the present disclosure can be
`implemented in a variety of forms. Therefore, while this
`disclosure includes particular examples, the true scope of the
`disclosure should not be so limited since other modifications
`will become apparent to the skilled practitioner upon a study
`of the specification and the following claims. Reference
`herein to one aspect, or various aspects means that a
`particular feature, structure, or characteristic described in
`connection with an embodiment is included in at least one
`embodiment or aspect. The appearances of the phrase "in
`one aspect' (or variations thereof) are not necessarily refer
`ring to the same aspect or embodiment.
`The present technology generally relates to an automated
`driving system of a vehicle that is able to gain the attention
`of a driver by initiating a temporary and irregular full vehicle
`movement. As used herein, it should be understood that the
`term vehicle should not be construed narrowly, and should
`include all types of vehicles, including a passenger car,
`truck, motorcycle, off-road vehicle, bus, boat, airplane,
`helicopter, lawn mower, recreational vehicle, amusement
`park vehicle, farm vehicle, construction vehicle, tram, golf
`cart, train, or trolley.
`Automated driving systems can be configured to deter
`mine or otherwise follow a planned vehicle path using a path
`planner application and send commands, for example,
`through a vehicle controller, to various vehicle systems to
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`IPR2025-00943
`Tesla EX1023 Page 7
`
`

`

`US 9,483,059 B2
`
`10
`
`15
`
`25
`
`30
`
`5
`102 or any other type of device, or multiple devices, capable
`of manipulating or processing information. A memory 104
`in the computing device can be a random access memory
`device (RAM) or any other suitable type of storage device.
`The memory 104 can include data 106 that is accessed by the
`CPU 102 using a bus 108.
`The memory 104 can also include an operating system
`110 and installed applications 112, with the installed appli
`cations 112 including programs that permit the CPU 102 to
`perform the automated driving methods described below.
`The computing device 100 can also include secondary,
`additional, or external storage 114, for example, a memory
`card, flash drive, or any other form of computer readable
`medium. The installed applications 112 can be stored in
`whole or in part in the external storage 114 and loaded into
`the memory 104 as needed for processing.
`The computing device 100 can also be in communication
`with one or more sensors 116. The sensors 116 may be
`disposed on a vehicle and can capture data and/or signals for
`processing by an inertial measurement unit (IMU), a lane
`keeping assist (LKA) system, a dead-reckoning system, an
`adaptive cruise control (ACC) system, a global navigation
`satellite system (GNSS) or global positioning system (GPS),
`a light detection and ranging (LIDAR) system, a radar
`system, a Sonar system, an image-based sensor system,
`simultaneous localization and mapping (SLAM), visual
`SLAM (VSLAM), or any other type of system capable of
`capturing information specific to the environment Surround
`ing a vehicle, including information specific to objects Such
`as other vehicles proximate to the navigation route of the
`vehicle, pedestrians, features of the route being traveled by
`the vehicle, landmarks, or other localized position data
`and/or signals and outputting corresponding data and/or
`signals to the CPU 102.
`35
`The sensors 116 can also capture data representative of
`changes in X, y, and Z-axis position, Velocity, acceleration,
`rotation angle, and rotational angular rate for the vehicle and
`similar data for objects or landmarks proximate to the
`navigation route of the vehicle. If the sensors 116 capture
`data for a dead-reckoning system, data relating to wheel
`revolution speeds, travel distance, steering angle, and steer
`ing angular rate of change can be captured. If the sensors 116
`capture signals for a GNSS or GPS, a receiver can calculate
`vehicle position and Velocity estimated in global coordi
`nates. A plurality of satellites can be used to estimate the
`vehicle's position and Velocity using three-dimensional tri
`angulation and time estimation.
`If the sensors 116 capture data for a LIDAR system,
`ranging data relating to intensity or reflectivity returns of the
`environment Surrounding the vehicle can be captured. In
`various examples, the sensors 116 can capture, at least: data
`for a dead-reckoning system, ACC system, or other system
`that estimates vehicle Velocity, acceleration, deceleration,
`position, and orientation; signals for a GNSS or other system
`that determines vehicle position and Velocity; and data for a
`LIDAR system, LKA System, or other system that measures
`vehicle distance from lane lines (e.g., route surface markings
`or route boundaries), obstacles, objects, or other environ
`mental features including traffic lights and road signs. The
`computing device 100 can also be in communication with
`one or more vehicle systems 118, such as vehicle braking
`systems, vehicle steering systems, vehicle propulsion sys
`tems, etc. The vehicle systems 118 can also be in commu
`nication with the sensors 116, the sensors 116 being con
`65
`figured to capture data indicative of performance of the
`vehicle systems 118.
`
`55
`
`6
`With respect to the example computing device 100 as
`described with reference to FIG. 1, the applications 112
`stored in the memory 104 may include at least a data
`analyzer 120, a path planner 122, and a vehicle controller
`124. In general, data captured by the sensors 116 can be used
`by one or more of these applications 112 to understand the
`environment Surrounding the vehicle, plan one or more
`potential vehicle paths for autonomous operation of the
`vehicle along a navigation route for the vehicle, improve
`positional accuracy of the vehicle, and send commands to
`the various vehicle systems 118 to change the current
`operating characteristics of the vehicle.
`FIG. 2A shows a schematic of a vehicle 200 including the
`computing device 100 described in FIG. 1. The computing
`device 100 can be located within the vehicle 200 as shown
`in FIG. 2A or can be located remotely from the vehicle 200
`in an alternate location (not shown). If the computing device
`100 is located remotely from the vehicle 200, the vehicle
`200 can include the capability of communicating with the
`computing device 100.
`The vehicle 200 can also include a plurality of sensors,
`such as the sensors 116 described in reference to FIG.1. One
`or more of the sensors 116 shown can be configured to
`capture changes in Velocity, acceleration, wheel revolution
`speed, and distance to objects within the Surrounding envi
`ronment for use by the computing device 100 to estimate
`position and orientation of the vehicle 200, steering angle for
`a dead-reckoning system, images for processing by an image
`sensor, vehicle position in global coordinates based on
`signals from a plurality of satellites, or any other data and/or
`signals that could be used to determine the current state of
`the vehicle or determine the position of the vehicle 200 in
`respect to its environment.
`For example, if the sensors 116 are configured to capture
`data for use by a LIDAR system, the sensors 116 can capture
`data related to laser returns from physical objects in the area
`Surrounding the vehicle 200 with ranging distances calcu
`lated by measuring the time it takes for a signal to return to
`the sensor 116. Laser returns can include the backscattered
`light reflected by objects hit by a source of light, e.g. laser
`light, being emitted by the sensors 116 or another source on
`or proximate to the vehicle 200. Once the light is reflected
`by an object, the sensors 116 can capture intensity values
`and reflectivity of each point on the object to be used for
`analyzing and classifying the object, for example, by the
`data analyzer 120, one of the applications 112 stored within
`or accessible to the computing device 100.
`The data analyzer 120 briefly described in FIG. 1 can
`analyze data and/or signals captured by the one or more
`sensors 116 by, for example, filtering noise, extracting
`features for clustering, and/or classifying and tracking
`objects. The data analyzer 120 can also process data from the
`one or more sensors 116 such that the data is configured for
`use by the other various applications 112 used to implement
`the automated driving system, Such as the path planner 122.
`In certain aspects, at least one of the sensors 116 can
`capture signals for use with a global navigation satellite
`system, a global positioning system, or a receiver for use
`with one of the systems. As such, the data analyzer 120 may
`be configured to analyze the quality of signals received from
`satellites for use with a GNSS or GPS. The vehicle may
`include one or more processors configured to determine that
`the path planner application is not capable of meeting a
`threshold performance level when the global navigation
`satellite system or a global positioning system fails to
`properly perform or meet predetermined minimum stan
`dards. The vehicle may also have a processor configured to
`
`40
`
`45
`
`50
`
`60
`
`IPR2025-00943
`Tesla EX1023 Page 8
`
`

`

`7
`determine a probability whether a future threshold perfor
`mance level can be met based on a current ability to analyze
`data from the one or more sensors receiving satellite signals.
`In various other aspects, the data analyzer 120 may be
`configured to analyze the quality and clarity of digital
`images that may be obtained from a camera sensor 115. For
`example, the data analyzer may be able to analyze and
`review data or information pertaining to the sharpness,
`contrast, patterns, spatial frequencies, quality, resolution,
`noise, and other features or metrics of objects, particularly
`proximate objects, landmarks, lane marking information,
`and lane markers or lane indicators, which are located within
`an image. When LIDAR, SLAM, or VSLAM systems are
`used, the data analyzer 120 may be configured to ascertain
`the accuracy of the systems to map-match the vehicle 200 to
`the environment. For example, where landmarks 322 (FIG.
`3) in the environment may be used to determine location. In
`various aspects, the vehicle 200 includes at least one pro
`cessor configured to determine that the path planner appli
`cation 122 is not capable of meeting a threshold perfor
`mance level when the LIDAR, SLAM, or VSLAM system
`fails to map-match the vehicle with landmarks according to
`predetermined minimum standards.
`Thus, the data analyzer may not only detect the presence
`or absence of indicators such as lane markings, but may be
`configured to analyze the quality of the indicators or mark
`ings. For example, the quality of data may be monitored
`and/or periodically analyzed in order to detect any occlu
`Sion, degradation, or loss of quality of the indicators that
`may signal a likelihood of a future failure mode or the
`probability that the quality, accuracy, and/or reliability of the
`data may get worse or ultimately not be able to support
`features or systems that may be required for autonomous
`driving.
`The path planner 122 can be configured to determine the
`navigation route for the vehicle 200 to follow based on the
`vehicle's 200 current location in respect to the surrounding
`environment as well as any points of destination chosen, for
`example, by the driver of the vehicle 200. The path planner
`122 can thus determine the navigation route for the vehicle
`200 based on data received from the data analyzer 120.
`The vehicle controller 124 can be configured to send
`commands to one or more vehicle systems 118 in order to
`maintain the navigation route indicated by the path planner
`122. In one example, the vehicle controller 124 can be a
`propulsion controller configured to send a command to the
`engine throttle to move the position of a throttle plate based
`on the position of an accelerator pedal or a brake pedal. In
`another example, the vehicle controller 124 can send com
`mands to a traction control system to implement steering
`50
`changes or a hybrid control system to redistribute a power
`ratio between electric and gas power sources. As another
`example, the vehicle controller 124 can be an electronic
`stability controller configured to send a command to activate
`one of the front or rear brakes if either more or less yaw
`motion (rotation around the vehicle's 200 vertical axis) is
`detected than optimum for the current angle of the steering
`wheel. In yet another aspect, the vehicle controller 124 can
`be a steering controller configured to initiate the temporary
`and irregular full vehicle movement, Such as side-to-side
`wobbling, to alert a driver of the vehicle.
`In FIG. 2B, the vehicle 200 is illustrated moving in a
`forward direction along an exemplary roadway 202 that
`contains multiple side-by-side lanes, with three lanes 204,
`206, and 208 being shown by example. It shown be under
`stood that the roadway 202 may contain as few as one lane
`and up four or more lanes.
`
`30
`
`35
`
`40
`
`45
`
`55
`
`60
`
`65
`
`US 9,483,059 B2
`
`10
`
`15
`
`25
`
`8
`In one aspect, a sensor may include a camera 115 having
`a field of view directed to the front, to the left side and to the
`right side of the vehicle as respectively shown by the
`directional arrows 210, 212 and 214. This arrangement
`allows the camera 115 to detect surface features of the
`roadway 202, such as lane markers. The roadway 202 can be
`a highway or freeway with typical lane markers, such as a
`solid continuous lane marker 216 at the left edge (in the
`direction of vehicle travel) of the left most lane 204, dashed
`lane markers 218 and 220 respectively defining the right
`edge of the left most lane 204 and the right edge of the
`middle lane 206. The right most lane 208 is delimited at a
`right edge by a solid continuous lane marker 222. The
`camera 115 can have the field of view shown in FIG. 2B
`where a camera 115 can obtain an image of the lane marker
`type to the immediate left side and to the immediate right
`side of the vehicle, such as lane markers 218 and 220 for the
`position of the vehicle 200 in FIG. 2B in the middle lane
`206. Alternately, when the camera 115 has a larger field of
`view, lane marker types at the far edges of the adjacent lanes,
`such as the lane markers 202 or 220 can also be obtained by
`the camera 115 or another suitable sensor.
`The camera 115 may be a black and white or color camera
`capable of sending images of the lane markers detected
`within the fiel