I 1111111111111111
`
`11111 111111111111111 111111111111111 11111 lll111111111111111
`
`
`
`
`
`US008874305B2
`
`c12) United States Patent
`Dolgov et al.
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 8,874,305 B2
`Oct. 28, 2014
`
`(54) DIAGNOSIS AND REPAIR FOR
`AUTONOMOUS VEHICLES
`
`(75)
`
`Inventors: Dmitri A. Dolgov, Mountain View, CA
`(US); Christopher Paul Urmson,
`Mountain View, CA (US)
`
`(73) Assignee: Google Inc., Mountain View, CA (US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by O days.
`
`(21) Appl. No.: 13/248,674
`
`(22) Filed:
`
`Sep.29,2011
`
`(65)
`
`Prior Publication Data
`
`US 2012/0083959 Al
`
`Apr. 5, 2012
`
`(60)
`
`(51)
`
`(52)
`
`Related U.S. Application Data
`
`Provisional application No. 61/390,094, filed on Oct.
`5, 2010, provisional application No. 61/391,271, filed
`on Oct. 8, 2010.
`
`(2006.01)
`(2006.01)
`(2012.01)
`
`Int. Cl.
`B60T7/12
`G05D 1/02
`B60W 30/186
`U.S. Cl.
`CPC .............. G0SD 11021 (2013.01); G05D 1/0278
`(2013.01); G05D 1/0274 (2013.01); G05D
`1/0246 (2013.01); B60W 2550/22 (2013.01);
`G0SD 110214 (2013.01); B60W 30/186
`(2013.01); G05D 2201/0213 (2013.01); G05D
`1/0257 (2013.01); G05D 1/024 (2013.01);
`B60W 2530/14 (2013.01)
`701/31.9; 701/25; 701/28; 701/29.1;
`701/31.6; 701/32.3; 701/70
`
`USPC
`
`(58) Field of Classification Search
`USPC
`.............. 701/23, 25, 26, 28, 29.1, 31.6, 31.9,
`701/32.3, 32.5, 44, 70, 71, 77, 78
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`1,924,984 A
`3,186,508 A
`3,324,805 A
`3,596,728 A
`
`8/1933 Fageol
`6/1965 Lamont
`6/1967 Mulch
`8/1971 Neville
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`JP
`
`8/2010
`2216225 Al
`6/1997
`09-160643 A
`(Continued)
`OTHER PUBLICATIONS
`
`International Search Report and the Written Opinion for Application
`No. PCT/US 2011/054896, Apr. 25, 2012.
`
`(Continued)
`
`Primary Examiner - Thomas G Black
`Assistant Examiner - Peter D Nolan
`(74) Attorney, Agent, or Firm - Lerner, David, Littenberg,
`Krumholz & Mentlik, LLP
`
`ABSTRACT
`(57)
`A system and method of controlling a vehicle is provided. In
`one aspect, the system and method determines the amount of
`wear on a component of the vehicle and, based on the amount
`of wear and information derived from the environment sur(cid:173)
`rounding the vehicle ( e.g., another vehicle in the path of the
`vehicle or a requirement to stop at a particular location),
`maneuvers the vehicle to mitigate further wear on the com(cid:173)
`ponent.
`
`17 Claims, 6 Drawing Sheets
`
`101
`
`Vehicle
`
`180
`
`,_ __
`
`140
`
`___..,Monitored
`components
`
`Sensor,;
`
`I
`
`/
`
`□□
`
`Passengers
`
`~ 195
`
`Computer System
`
`Movement Control
`Components
`Steering
`
`Surrounding
`Environment
`
`130
`
`131
`
`132
`
`134
`
`135
`
`160
`
`133 =1==1t;===~
`
`Throttle
`
`Brakes
`
`Transmission
`
`150
`
`151
`
`152
`
`153
`
`154
`
`IPR2025-00943
`Tesla EX1032 Page 1
`
`

`

`US 8,874,305 B2
`Page 2
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`188/1.llL
`
`2/1983 Anderson et al.
`4,372,414 A
`6/1983 Carman
`4,387,783 A
`4/1987 Elpern
`4,656,834 A
`5/1990 Ottemann
`4,924,795 A
`1/1991 Takigami
`4,982,072 A
`5,187,666 A *
`....................... 701/79
`2/1993 Watanabe
`5/1995 Latarnik et al.
`5,415,468 A
`9/1995 Arai
`5,448,487 A
`5,470,134 A * 11/1995 Toepfer et al. ............... 303/9.61
`5,684,696 A * 11/1997 Rao et al. ........................ 701/25
`5,774,069 A
`6/1998 Tanaka et al.
`5,906,645 A
`5/1999 Kagawa et al.
`6,064,926 A
`5/2000 Sarangapani et al.
`6,070,682 A *
`180/167
`6/2000
`Isogai et al.
`6,151,539 A * 11/2000 Bergholz et al. ................ 701/25
`6,195,610 Bl
`2/2001 Kaneko
`6,321,147 Bl*
`11/2001 Takeda et al . ................... 701/23
`6,332,354 Bl * 12/2001 Lalor et al. ...................... 73/121
`6,343,247 B2
`1/2002 Jitsukata et al.
`6,438,472 Bl
`8/2002 Tano et al.
`6,438,491 Bl
`8/2002 Farmer
`6,470,874 Bl
`10/2002 Mertes
`6,504,259 Bl
`1/2003 Kuroda et al.
`6,516,262 B2 *
`2/2003 Takenaga et al. ............... 701/96
`6,591,172 B2
`7/2003 Oda et al.
`11/2003 0 Connor et al.
`6,643,576 Bl
`6,832,156 B2
`12/2004 Farmer
`6,836,719 B2
`12/2004 Andersson et al.
`6,847,869 B2 *
`1/2005 Dewberry et al. ........... 701/33.9
`6,862,524 Bl
`3/2005 Nagda et al.
`6,876,908 B2 *
`4/2005 Cramer et al. ............... 701/29.3
`6,934,613 B2
`8/2005 Yun
`7,011,186 B2 *
`3/2006 Frentz et al.
`7,031,829 B2
`4/2006 Nisiyama
`7,102,496 Bl
`9/2006 Ernst, Jr. et al.
`7,194,347 B2
`3/2007 Harumoto et al.
`7,207,304 B2
`4/2007
`I watsuki et al.
`7,233,861 B2
`6/2007 Van Buer et al.
`7,327,242 B2
`2/2008 Holloway et al.
`7,346,439 B2
`3/2008 Bodin
`7,394,046 B2
`7/2008 Olsson et al.
`7,486,802 B2
`2/2009 Hougen
`7,499,774 B2
`3/2009 Barrett et al.
`7,499,776 B2
`3/2009 Allard et al.
`7,499,804 B2
`3/2009 Svendsen et al.
`7,515,101 Bl
`4/2009 Bhogal et al.
`7,579,942 B2
`8/2009 Kalik
`7,656,280 B2
`2/2010 Hines et al.
`7,694,555 B2 *
`4/2010 Howell et al. ................... 73/129
`7,818,124 B2
`10/2010 Herbst et al.
`7,865,277 Bl
`1/2011 Larson et al.
`7,894,951 B2
`2/2011 Norris et al.
`7,908,040 B2
`3/2011 Howard et al.
`7,956,730 B2
`6/2011 White et al.
`8,050,863 B2
`11/2011 Trepagnier et al.
`8,078,349 Bl
`12/2011 Prada Gomez et al.
`8,190,322 B2 *
`5/2012 Lin et al. ...................... 701/31.5
`8,195,341 B2
`6/2012 Huang et al.
`8,244,408 B2
`8/2012 Lee et al.
`8,260,515 B2
`9/2012 Huang et al.
`8,280,601 B2
`10/2012 Huang et al.
`8,634,980 Bl
`1/2014 Urmson et al.
`2001/0037927 Al
`11/2001 Nagler et al.
`2003/0016804 Al
`1/2003 Sheha et al.
`2003/0055554 Al
`3/2003 Shioda et al.
`2003/0093209 Al
`5/2003 Andersson et al.
`2004/0243292 Al
`12/2004 Roy
`2005/0012589 Al
`1/2005 Kokubu eta!.
`2005/0273251 Al
`12/2005 Nix et al.
`2006/0037573 Al
`2/2006
`I watsuki et al.
`2006/0082437 Al
`4/2006 Yuhara
`2006/0089764 Al
`4/2006 Filippov et al.
`2006/0178240 Al
`8/2006 Hansel
`2006/0276942 Al
`12/2006 Anderson et al.
`2007/0165910 Al
`7/2007 Nagaoka et al.
`
`9/2007 Sakano
`2007 /0225909 Al
`10/2007 Kaplan
`2007 /0239331 Al
`10/2007 Shimomura
`2007/0247281 Al
`1/2008 Tryon
`2008/0021628 Al
`3/2008 Kessler et al.
`2008/0059048 Al
`4/2008 Kalik
`2008/0084283 Al
`5/2008 Naitou et al.
`2008/0120025 Al
`6/2008 Breed
`2008/0147253 Al
`7/2008 Breed
`2008/0161987 Al
`7/2008 Benzinger et al.
`2008/0183512 Al
`8/2008 Sheha et al.
`2008/0188246 Al
`11/2008 Huang et al.
`2008/0277183 Al
`12/2008 Aso et al.
`2008/0303696 Al
`12/2008 Mehta et al.
`2008/0306969 Al
`1/2009 Bargman et al.
`2009/0005959 Al
`3/2009 Dooley et al.
`2009/0082879 Al
`5/2009 Han
`2009/0115594 Al
`10/2009 Kamiya
`2009/0248231 Al
`11/2009 Subramanian et al.
`2009/0276154 Al
`11/2009 Salinger
`2009/0287367 Al
`2009/0287368 Al * 11/2009 Bonne ............................. 701/29
`2009/0319096 Al
`12/2009 Offer et al.
`2009/0319112 Al
`12/2009 Fregene et al.
`2009/0326799 Al
`12/2009 Crook
`2010/0017056 Al
`1/2010 Asakura et al.
`2010/0052945 Al
`3/2010 Breed
`2010/0076640 Al
`3/2010 Maekawa et al.
`2010/0179720 Al
`7/2010 Lin et al.
`2010/0205132 Al
`8/2010 Taguchi
`2010/0228419 Al
`9/2010 Lee et al.
`2010/0241297 Al
`9/2010 Aoki et al.
`2010/0253542 Al
`10/2010 Seder et al.
`2010/0256836 Al
`10/2010 Mudalige
`2011/0071718 Al
`3/2011 Norris et al.
`2011/0137520 Al
`6/2011 Rector et al.
`2011/0150348 Al
`6/2011 Anderson
`2011/0206273 Al
`8/2011 Plagemann et al.
`2011/0213511 Al
`9/2011 Visconti et al.
`2011/0254655 Al
`10/2011 Maalouf et al.
`2012/0053775 Al
`3/2012 Nettleton et al.
`2012/0157052 Al
`6/2012 Quade
`2012/0277947 Al
`11/2012 Boehringer et al.
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`JP
`WO
`WO
`WO
`WO
`
`11282530 A
`2000149188 A
`2000305625 A
`2000-338008 A
`2001-101599 A
`2002236993 A
`2002251690 A
`2003081039 A
`2003162799 A
`2005067483 A
`2005071114 A
`2005-339181 A
`2006322752 A
`2007001475 A
`2008117082 A
`2008152655 A
`2008170404 A
`2008290680 A
`2009053925 A
`0070941 Al
`0188827
`2009028558 Al
`2011021046 Al
`
`10/1999
`5/2000
`11/2000
`12/2000
`4/2001
`8/2002
`9/2002
`3/2003
`6/2003
`3/2005
`3/2005
`12/2005
`11/2006
`1/2007
`5/2008
`7/2008
`7/2008
`12/2008
`3/2009
`11/2000
`11/2001
`3/2009
`2/2011
`
`OTHER PUBLICATIONS
`
`PCT Notification of Transmittal of the International Search Report
`and the Written Opinion of the International Searching Authority for
`PCT/US2011/054899, Oct. 5, 2011.
`
`IPR2025-00943
`Tesla EX1032 Page 2
`
`

`

`US 8,874,305 B2
`Page 3
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`"Fact Sheet: Beyond Traffic Signals: A Paradigm Shift Intersection
`Control for Autonomous Vehicles", [online]. [Retrieved Apr. 27,
`2011]. Retrieved from the internet: <http://www.fhwa.dot.gov/
`advancedresearch/pubs/10023/index.cfm>, 3 pages.
`"Google Cars Drive Themselves, in Traffic" [online]. [Retrieved
`Aug. 19, 2011] Retrieved from the internet: <http://www.nytimes.
`corn/2010/10/10/science/ lOgoogle.html>, 4 pages.
`Carl Crane, David Armstrong, Antonio Arroyo, Antoin Baker, Doug
`Dankel, Greg Garcia, Nicholas Johnson, Jaesang Lee, Shannon
`
`Ridgeway, Eric Schwartz, Eric Thorn, Steve Velat, and Ji Hyun Yoon,
`Team Gator Nation's Autonomous Vehicle Development for the 2007
`DARPA Urban Challenge, Dec. 2007, 27 pages.
`Martin Schonhof, Martin Treiber, Arne Kesting, and Dirk Helbing,
`Autonomous Detection and Anticipation of Jam Fronts From Mes(cid:173)
`sages Propagated by Intervehicle Communication, 2007, pp. 3-12.
`Vincenzo DiLecce and Marco Calabrese, Experimental System to
`Support Real-Time Driving Pattern Recognition, 2008, pp. 1192-
`1199.
`International Search Report and the Written Opinion for Application
`No. PCT/US 2011/054154, Apr. 24, 2012.
`
`* cited by examiner
`
`IPR2025-00943
`Tesla EX1032 Page 3
`
`

`

`U.S. Patent
`
`Oct. 28, 2014
`
`Sheet 1 of 6
`
`US 8,874,305 B2
`
`Vehicle -
`- ]Erernal ID
`
`-----Monitored
`components
`
`101
`~
`
`180
`
`~
`
`- -
`
`140
`
`Surrounding
`Environment
`
`Passengers
`!Driver
`
`I
`
`·-
`
`Movement Control
`Components
`
`Steering
`
`Lr
`
`Throttle
`
`Brakes
`
`Transmission
`
`l
`I
`
`I
`I
`
`I
`I
`
`I
`1-
`
`
`
`1 95
`
`150
`
`
`
`1 51
`
`
`
`1 52
`
`
`
`1 53
`
`
`
`1 54
`
`1~~\
`
`Sensors
`
`,-
`
`"
`
`\
`
`I
`I
`I
`
`.... "
`
`130
`
`Computer System
`
`131
`
`132
`
`---
`
`133
`
`134
`
`135
`
`160
`
`170
`
`1Processor
`
`Memory
`
`1Instructions
`
`Data
`
`4Map Data
`
`..
`
`User
`Input
`
`User
`Output
`
`I
`
`•
`
`_,/
`
`FIGURE 1
`
`IPR2025-00943
`Tesla EX1032 Page 4
`
`

`

`U.S. Patent
`
`Oct. 28, 2014
`
`Sheet 2 of 6
`
`US 8,874,305 B2
`
`14
`0 ---
`2
`30--
`
`20
`9----
`
`10
`2
`
`2 11 -
`
`2 12
`
`2 13
`
`2 14
`
`0
`22
`
`1 -
`22
`
`2-
`22
`
`3
`22
`
`4
`22
`
`-
`
`Sensors
`
`- Vehicle Component Sensors
`- Environmental Sensors
`
`Geographic Location Component
`
`-JGPS Receiver
`
`- Inertial Guidance
`
`~Accelerometer
`
`~Gyroscope
`
`External Object Detector
`
`I
`I
`
`Laser
`
`~Radar
`
`~Cameras
`
`Sonar
`
`240
`
`250
`
`!Ambient Sensors
`
`I Passenger Sensors
`
`I
`
`I
`
`FIGURE 2
`
`IPR2025-00943
`Tesla EX1032 Page 5
`
`

`

`U.S. Patent
`
`Oct. 28, 2014
`
`Sheet 3 of 6
`
`US 8,874,305 B2
`
`180~ Monitored Components
`
`Vehicle Component Senso ,or
`
`181 - ~Gas tank
`
`182 --- ~Engine
`153 __..., ~!Brakes
`
`184-- ~ITires
`
`I
`I
`
`I
`I
`
`I
`I
`
`:Fuel Level
`
`: Oil Pressure
`
`:srake Wear
`
`I
`1~~ ..............
`Nrire
`
`:rread Wear
`
`Pressure
`
`L
`I
`
`I
`I
`
`~ v----
`L
`I
`
`I
`I
`
`230
`
`231
`
`232
`
`233
`
`234
`
`235
`
`FIGURE 3
`
`IPR2025-00943
`Tesla EX1032 Page 6
`
`

`

`U.S. Patent
`
`Oct. 28, 2014
`
`Sheet 4 of 6
`
`US 8,874,305 B2
`
`325
`
`330
`
`320
`
`315
`
`310
`
`390
`
`317
`
`FIGURE 4
`
`IPR2025-00943
`Tesla EX1032 Page 7
`
`

`

`U.S. Patent
`
`Oct. 28, 2014
`
`Sheet 5 of 6
`
`US 8,874,305 B2
`
`Brake wear
`detector·
`Good condition
`
`----------------------------------~
`
`External object detector
`
`0---,-----
`
`510
`
`101
`
`Detected distance from object
`
`520J
`
`Braking distance
`
`FIGURE 5
`
`530
`
`Brake wear
`detector·
`Worn
`
`___________
`
`External object detector ___________
`
`101
`
`L
`
`Detected distance from object
`
`520___,/~-~----------B-ra-l,-1n_g_d_1s-ta_n_ce-.----;__J-~
`
`-T 510
`
`FIGURE 6
`
`\__630
`
`Brake wear
`detector·
`Worn
`
`.,
`~710
`,,
`,,
`.,
`.,
`,,
`,,
`.,
`
`101
`
`Calculated distance from obJect
`
`5205
`
`Braking Distance
`
`1510
`
`I
`
`FIGURE 7
`
`~630
`
`IPR2025-00943
`Tesla EX1032 Page 8
`
`

`

`U.S. Patent
`
`Oct. 28, 2014
`
`Sheet 6 of 6
`
`US 8,874,305 B2
`
`Detemiine location of
`vehicle
`
`1,
`
`Detemiine location of
`geographic location
`specific traffic rules
`
`1'
`
`Is vehicle component #1
`damaged?
`
`----No
`
`Maneuver vehicle in
`accordance with normal
`operations
`
`I
`
`Yes '
`
`Select alternative
`maneuver to minimize
`damage to component #1
`by issuing instruction to
`component #2 at time
`required to satisfy traffic
`rule
`
`1'
`
`Autonomously maneuver
`vehicle to repair facility
`
`FIGURE 8
`
`IPR2025-00943
`Tesla EX1032 Page 9
`
`

`

`US 8,874,305 B2
`
`1
`DIAGNOSIS AND REPAIR FOR
`AUTONOMOUS VEHICLES
`
`CROSS REFERENCE TO RELATED
`APPLICATIONS
`
`The present application claims the benefit of the filing
`dates of U.S. Provisional Application No. 61/390,094,
`entitled "AUTONOMOUS VEHICLES," filed Oct. 5, 2010,
`and U.S. Provisional Application No. 61/391,271, entitled
`"AUTONOMOUS VEHICLES," filed Oct. 8, 2010, the entire
`disclosures of which are hereby incorporated herein by ref(cid:173)
`erence.
`
`BACKGROUND
`
`Autonomous vehicles may be configured to be driven in a
`manual mode (where the operator exercises a high degree of
`control over the movement of the vehicle) or in an autono(cid:173)
`mous mode (where the vehicle essentially drives itself).
`These vehicles use various computing systems to aid in the
`transport of passengers from one location to another. Some
`autonomous vehicles may require some initial input or con(cid:173)
`tinuous input from an operator, such as a pilot, driver, or
`passenger. Other systems, for example autopilot systems,
`may be used only when the system has been engaged, which
`permits the operator to switch from a manual mode to an
`autonomous mode and to modes that lie somewhere in
`between.
`A vehicle with autonomous driving capability may be con(cid:173)
`figured to receive signal inputs from the sensors that monitor
`the vehicle operations, surrounding objects and road condi(cid:173)
`tions in order to identify safety hazards and generate coun(cid:173)
`termeasures to deal with various driving situations. The
`autonomous vehicle may also collect and record data from
`various information sources such as cellular network, satel(cid:173)
`lites as well as user inputs such as users' identification, des(cid:173)
`tinations and routes of navigation requests and vehicle opera(cid:173)
`tion preferences.
`A vehicle with autonomous driving capability may further
`be adapted to detect various potential hazardous conditions
`and issue warnings to the user. The potential hazardous con(cid:173)
`dition may include, for example, the vehicle's approaching a
`sharp curve, nearby pedestrians, icy roads, etc. Such vehicle
`may also be configured with mechanisms of taking active
`steps to avoid these hazards, e.g., slowing down the vehicle,
`applying the brake, etc.
`
`SUMMARY
`
`5
`
`15
`
`2
`stopping proximate to the object.A memory contains instruc(cid:173)
`tions accessible by the processor. The instructions include
`changing the motion of the vehicle relative to the external
`object based on output received from the two sensors.
`In still another aspect, a method of controlling a vehicle
`includes: determining the wear on a component of the
`vehicle; determining the geographic position of the vehicle;
`determining traffic requirements in the path of the vehicle;
`and selecting, with a processor, between a first and second
`10 maneuver based on the determined wear and the determined
`distance. At least one difference between the first and second
`maneuver is selected from the group consisting of rate of
`acceleration, rate of deceleration and direction. The vehicle is
`moved in accordance with the selected maneuver.
`In a further aspect, a vehicle is provided that includes
`control components for controlling the movement of the
`vehicle. The components respond to commands from a pas(cid:173)
`senger and a processor. Environment sensors detect the envi(cid:173)
`ronment external to the vehicle and component sensors detect
`20 the physical characteristic of components internal to the
`vehicle, where such physical characteristic changes based on
`the operation of the vehicle. A processor is in communication
`with the control components, environment sensors and com(cid:173)
`ponent sensors, and executes instructions that include: receiv-
`25 ing output from the environment sensors, receiving output
`from the component sensors, selecting a first command or
`second command to be provided to the control components
`based on the output from the environment sensors and com(cid:173)
`ponent sensors, and providing the selected command to the
`30 control components.
`In another aspect, a system of controlling the movement of
`a vehicle includes a processor, a memory accessible by the
`processor, and instructions contained in the memory and
`executable by the processor. The instructions include: deter-
`35 mining the geographic location of the vehicle relative to loca(cid:173)
`tion-dependant vehicle movement restrictions, determining
`whether a first component of the vehicle is damaged, provid(cid:173)
`ing a first instruction to a second component of the vehicle to
`move the vehicle so as to comply with the location-dependant
`40 movement restriction when the first component is determined
`to be not damaged, and providing a second instruction to the
`second component to move the vehicle so as to comply with
`the location-dependant movement restriction when the first
`when the first component is determined to be damaged. The
`45 movement instructed by the second instruction is different
`than the movement instructed by the first command.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`In one aspect, a method of maneuvering a vehicle is pro(cid:173)
`vided. The method includes detecting a characteristic of the
`environment surrounding the vehicle and detecting the
`amount of damage to a component of the vehicle, wherein
`different maneuvers can be expected to affect the component, 55
`and thus the damage, differently. A processor selects between
`a first maneuver and a second maneuver based on the detected
`environment characteristic and detected damage amount. The
`vehicle is then moved in accordance with the selected maneu-
`ver.
`In another aspect, a system includes a processor and sen(cid:173)
`sors. One sensor is in communication with the processor and
`is configured to detect the presence of physical damage to a
`component of a vehicle. Another sensor is in communication
`with the processor and configured to detect the presence of 65
`objects external to the vehicle that require a change in the
`motion of the vehicle, e.g., maneuvering around the object or
`
`DETAILED DESCRIPTION
`
`Short Summary
`In one aspect, a system and method of autonomously mov(cid:173)
`ing a vehicle is provided wherein sensors detect whether a
`
`50
`
`FIG. 1 is a functional diagram of a system.
`FIG. 2 is a functional diagram of sensors of a vehicle.
`FIG. 3 is a functional diagram of monitored components
`and sensors of a vehicle.
`FIG. 4 is diagram of the interior of a vehicle.
`FIG. 5 is a diagram of vehicle movement in response to
`detection of a location-specific vehicle movement restriction.
`FIG. 6 is a diagram of vehicle movement in response to
`detection of a location-specific vehicle movement restriction.
`FIG. 7 is a diagram of vehicle movement in response to
`60 detection of a location-specific vehicle movement restriction.
`FIG. 8 is a flowchart.
`
`IPR2025-00943
`Tesla EX1032 Page 10
`
`

`

`US 8,874,305 B2
`
`3
`component is damaged and a processor maneuvers the
`vehicle differently based on the presence of damage and in
`order to mitigate further damage to the component.
`Vehicle Introduction
`FIG. 1 illustrates one possible aspect of an autonomous
`driving system of vehicle 101. Although certain aspects of the
`invention are particularly useful in connection with specific
`types of vehicles, vehicle 101 may be any type of vehicle.
`Possible vehicles include, by way of example only, cars,
`trucks, motorcycles, busses, boats, airplanes, helicopters,
`lawnmowers,
`recreational
`vehicles,
`amusement
`park
`vehicles, trams, golf carts, trains and trolleys.
`Vehicle 101 may include an autonomous vehicle computer
`system 130 that is in communication with sensors 140, com(cid:173)
`ponents 150 that control the movement of the vehicle, user
`input 160 and user indicators 170.
`Computer System
`Computer system 130 may comprise a computer contain-
`ing a processor 131, memory 132 and other components
`typically present in general purpose computers.
`The memory 132 stores information accessible by proces(cid:173)
`sor 131, including instructions 133 and data 134 that may be
`executed or otherwise used by the processor 131. The
`memory 132 may be of any type capable of storing informa(cid:173)
`tion accessible by the processor, including a computer-read(cid:173)
`able medium, or other medium that stores data that may be
`read with the aid of an electronic device, such as a hard-drive,
`memory card, ROM, RAM, DVD or other optical disks, as
`well as other write-capable and read-only memories. Systems
`and methods may include different combinations of the fore- 30
`going, whereby different portions of the instructions and data
`are stored on different types of media.
`The instructions 133 may be any set of instructions to be
`executed directly (such as machine code) or indirectly (such
`as scripts) by the processor. For example, the instructions may 35
`be stored as computer code on a computer-readable medium.
`In that regard, the terms "instructions" and "programs" may
`be used interchangeably herein. The instructions may be
`stored in object code format for direct processing by the
`processor, or in any other computer language including 40
`scripts or collections of independent source code modules
`that are interpreted on demand. Functions, methods and rou(cid:173)
`tines of the instructions are explained in more detail below.
`The data 134 may be retrieved, stored or modified by
`processor 131 in accordance with the instructions 133. For 45
`instance, although the system and method is not limited by
`any particular data structure, the data may be stored in com(cid:173)
`puter registers, in a relational database as a table having a
`plurality of different fields and records, XML documents or
`flat files. The data may also be formatted in any computer-
`readable format. By further way of example only, image data
`may be stored as bitmaps comprised of grids of pixels that are
`stored in accordance with formats that are compressed or
`uncompressed, lossless ( e.g., BMP) or lossy ( e.g., JPEG), and
`bitmap or vector-based (e.g., SVG), as well as computer 55
`instructions for drawing graphics. The data may comprise any
`information sufficient to identify the relevant information,
`such as numbers, descriptive text, proprietary codes, refer(cid:173)
`ences to data stored in other areas of the same memory or
`different memories (including other network locations) or 60
`information that is used by a function to calculate the relevant
`data.
`The processor 131 may be any conventional processor,
`such as processors from Intel Corporation or Advanced Micro
`Devices. Alternatively, the processor may be a dedicated
`device such as an ASIC. Although FIG. 1 functionally illus(cid:173)
`trates the processor, memory, and other elements of central
`
`4
`control 130 as being within the same block, it will be under(cid:173)
`stood by those of ordinary skill in the art that the processor
`and memory may actually comprise multiple processors and
`memories that may or may not be stored within the same
`5 physical housing. For example, rather than being stored in the
`same computer, processor 131 and memory 132 may be
`stored in separate devices. Although there may be advantages
`to locating the processor 131 and memory 132 within vehicle
`101, various processes may be performed external to the
`10 vehicle and various data may be stored outside of the vehicle.
`For example, if a processor or memory used or required by the
`vehicle 101 occurs in an external device, vehicle 101 may
`obtain the information it requires wirelessly. Accordingly,
`although references to a processor or memory herein will
`15 assume that the processor and memory are affixed to vehicle
`101, such references will be understood to include references
`to a collection of processors or computers or memories that
`may or may not operate in parallel and may or may not be
`located within affixed to vehicle 101.
`20 Map Data
`Data 134 may include map-related data 135. In addition to
`storing the geographic location of streets and their intersec(cid:173)
`tions, the map data may further include information relating
`to traffic rules and location-specific rules, such as the geo-
`25 graphic location of stop signs and speed limits. Yet further, the
`map data may include information relating to altitude, e.g.,
`information from which the grade of a hill may be deter(cid:173)
`mined.
`Navigation Controls
`As shown in FIG. 1, a vehicle 101 may include various
`components 150 relating to controlling the navigation of the
`vehicle. For example, the vehicle may include steering 151,
`throttle 152 ( e.g., operated by an accelerator), brakes 153 and
`transmission 154.
`Monitored Components
`As shown in FIG. 2, the vehicle may also include a variety
`of internal and external sensors 140 that provide data to the
`autonomous vehicle computer system 130.
`Whereas environmental sensors 209 provide data about the
`environment surrounding the vehicle and passenger sensors
`250 provide data about the passengers 195 in the vehicle,
`vehicle component sensors 230 provide information about
`the physical characteristics of the vehicle components. By
`way of example and as shown in FIG. 3, fuel level sensor 231
`may monitor the amount of gasoline held by gas tank 181, oil
`pressure sensor 23 2 may monitor the amount of oil pressure in
`engine 182, brake wear sensor 233 may monitor the amount
`of wear on brakes 153, air pressure sensor 235 may monitor
`the amount of pressure in tires 184 and tread wear sensors 234
`may monitor the treads of the tires. The vehicle may include
`other sensors as well, such as battery level, engine tempera-
`ture, weight distribution and steering alignment sensors. In
`some aspects, the system and method may store and rely on
`information representing changes in the output of the sensors
`over time, i.e., a log of the fuel level at different times or an
`average fuel level over some selected time period.
`A variety of the sensors may monitor components that are
`highly likely to become damaged based on the continued
`normal operation of the vehicle. For example, it may be
`expected that the brake pads will wear away and eventually
`require replacement. Data 134 may include detailed informa-
`tion regarding past vehicle maintenance repairs of such com(cid:173)
`ponents ( e.g., when brakes and tires were changed) as well as
`other components. In one aspect, sensors 230 detect changes
`65 in the physical characteristics of components that are within
`the car ( e.g., under the hood) and likely to change as the
`vehicle is operated.
`
`50
`
`IPR2025-00943
`Tesla EX1032 Page 11
`
`

`

`US 8,874,305 B2
`
`5
`
`Environmental Sensors
`Environmental sensors 140 allow the vehicle to understand
`and potentially respond to its environment in order to navigate
`and maximize safety for passengers as well as people or
`property in the surrounding environment. The sensors may be
`used to identify, track and predict the movement of objects as
`well, such as pedestrians and other vehicles. Yet further, the
`sensors may be used to provide recommendations. The
`vehicle may include other sensors that are not shown in FIG.
`2.
`Geographic Location Component
`Geographic location component 210 is used to determine
`the geographic location and orientation of the vehicle 101.
`For example, component 210 may include a GPS receiver 211
`to determine the vehicle's latitude, longitude and altitude.
`Although references herein to the location of the vehicle will
`often refer to a location in latitude/longitude/altitude coordi(cid:173)
`nates, the data representing the location of the vehicle may
`also be relative to other reference systems, such as the vehi(cid:173)
`cle's distance from objects. Some reference systems may be
`subject to less noise than others.
`Inertial Guidance
`The geographic location component may also include an
`inertial guidance system 212, which may in turn include an
`accelerometer 213 and gyroscope 214. The inertial guidance 25
`system 212 may determine the current orientation of the
`device and changes of speed in any direction. For example,
`the inertial guidance system 212 may detect when the vehicle
`is turning. It may also estimate the current location of the car
`relative to a starting location based on changes in speed and 30
`direction.
`Multiple Sources
`Other devices may also be used to determine the location of
`the vehicle 101. For example, if the external object detector
`220 identifies an object and its location relative to the vehicle, 35
`and if processor 131 can access pre-existing data that identi-
`fies the geographic location of the object, then processor 131
`can identify the geographic location of the vehicle. The sys(cid:173)
`tem may also triangulate its location based on cell phone
`tower transmissions or the presence of smaller wireless net- 40
`works. The processor may combine the information from the
`various components and detectors, or select the most accurate
`source, and determine the geographic location of the vehicle
`accordingly.
`External Object Detector
`The vehicle may include an external object detection sys(cid:173)
`tem 220 for detecting objects external to the vehicle such as
`other vehicles, obstacles in the roadway, traffic signals, signs,
`trees, etc. The detection system 220 may include a laser 221,
`radar 222, cameras 223, sonar 224 or and other detection 50
`devices.
`Laser
`Vehicle 101 may include a laser 221 mounted on the roofor
`other convenient location. In one aspect, the laser may mea(cid:173)
`sure the distance between the vehicle and object surfaces
`facing the vehicle by spinning on its axis and changing its
`pitch. The laser may also be used to identify changes in
`surface texture or reflectivity. Thus, the laser may be config(cid:173)
`ured to detect lane lines by distinguishing between the
`amount of light reflected by a painted lane line relative to
`unpainted dark pavement.
`Radar
`Sensors 140 may further include various radar detection
`units 222, such as those used for adaptive cruise control
`systems. The radar detection units may be located on the front 65
`and back of the car as well as on either side of the front
`bumper. In addition to using radar to determine the relative
`
`6
`location of external objects, other types of radar may be used
`for other purposes as well, such as a conventional speed
`detector. Short wave radar may be used to determine the depth
`of snow on the road and to determine location and condition
`5 of the road surface.
`Cameras
`One of the sensors may also include one or more cameras
`223. If multiple cameras are used and the distances from each
`other are known, the parallax from the different images may
`10 be used to compute the distance to various objects which are
`captured by the cameras. Content may also be extracted from
`images captured by a camera. For example, the vehicle may
`automatically slow down if its current speed is 50 mph and it
`detects, by using its cameras and using optical-character rec-
`15 ognition, that it will shortly pass a sign indicating that the
`speed limit is 35 mph. Yet further, pattern matching and other
`feature detection algorithms may be used to determine the
`type of the object. This may be combined with other infor(cid:173)
`mation to determine location specific information that is rel-
`20 evant to the maneuvering of the vehicle, such as determining
`the presence of a stop sign with cameras 223 and determining
`the location of the sign, and thus where the vehicle needs to
`stop, with laser 221.
`Ambient State Sensors
`Ambient sensors 240 may determine environmental
`aspects that do not specifically relate to external object detec(cid:173)
`tion, such as air quality sensors for detecting the surrounding
`air's temperature, humidity, or particulates.
`User's State Sensors
`Sensors 140 may also include sensors for determining the
`state of the user, such as the driver and other pa