`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 1 of 22 PageID #: 27
`
`
`
`
`
`EXHIBIT A
`
`EXHIBIT A
`
`
`
`(12) United States Patent
`Naboulsi
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 9,713,994 B2
`Jul. 25, 2017
`
`USOO9713994B2
`
`SAFETY CONTROL SYSTEM FOR APPS IN
`VEHICLES
`
`Applicant: ACT-IP, West Bloomfield, MI (US)
`
`Inventor:
`
`Mouhamad Ahmad Naboulsi, West
`Bloomfield, MI (US)
`Assignee: ACT-IP, West Bloomfield, MI (US)
`Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`Appl. No.: 15/352,206
`
`Filed:
`
`Nov. 15, 2016
`
`Prior Publication Data
`US 2017/O182956 A1
`Jun. 29, 2017
`
`Related U.S. Application Data
`Continuation of application No. 14/661,598, filed on
`Mar. 18, 2015, now Pat. No. 9,524,034.
`
`Int. C.
`G06F 3/02
`B60R I6/023
`G08B 2/06
`B6OR II/02
`B60O 9/00
`G06F 3/0
`
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`(2006.01)
`(Continued)
`
`U.S. C.
`CPC .......... B60R 16/0232 (2013.01); B600 9/008
`(2013.01); B60R II/0264 (2013.01); G06F
`3/017 (2013.01); G06F 3/0227 (2013.01);
`G08B 21/06 (2013.01); H04M I/6091
`(2013.01); H04M 1/72577 (2013.01); B60K
`
`2350/906 (2013.01); B60K 2350/967
`(2013.01); B60W 2.520/105 (2013.01)
`(58) Field of Classification Search
`CPC. B60R 16/0232; B60R 11/0264; B60Q 9/008:
`G06F 3/017: G06F 3/0227; G08B 21/06;
`H04M 1/6091; H04M 1/72577; B60K
`2350/906; B60K 2350/967
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,555,502 A * 9/1996 Opel .................. B6OH 1/OO985
`TO1/36
`5,949,149 A * 9/1999 Shitanaka ............ B60Q 1/0082
`2OOfS R
`
`(Continued)
`Primary Examiner — Richard Camby
`(57)
`ABSTRACT
`According to one aspect of one embodiment of the present
`invention, a safety control system for using applications in
`vehicles, includes, a communication device having at least
`one of an input accessible from within the vehicle and an
`output communicated within the vehicle, at least one sensor
`operable to sense at least one condition related to vehicle
`operation, and data about distraction features of a running
`application, to and a controller communicated with the
`sensor and the communication device to selectively suppress
`at least one of said input and said output in response to a
`sensed parameter of said at least one condition being outside
`of a threshold. When an input is suppressed, the driver is
`prevented from accessing or inputting information into the
`communication device. When an output is Suppressed, com
`munication between the device and the driver of a vehicle is
`Suppressed to, among other things, avoid distracting the
`driver during certain driving situations or conditions relating
`to the driver, vehicle and/or environment.
`
`10 Claims, 7 Drawing Sheets
`
`(54)
`
`(71)
`(72)
`
`(73)
`(*)
`
`(21)
`(22)
`(65)
`
`(63)
`
`(51)
`
`(52)
`
`
`
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 2 of 22 PageID #: 28
`
`
`
`
`
`-See- or sea see we see of.
`axe.
`N
`
`Y
`
`g
`
`
`
`US 9,713,994 B2
`Page 2
`
`(51) Int. Cl.
`H04M I/725
`H04M I/60
`
`(2006.01)
`(2006.01)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6,078.293 A * 6/2000 Yamamoto ............ B6OR 25.245
`307 10.2
`6,396,006 B1* 5/2002 Yokoji .................. G06F 3.0312
`200/11 R
`6.479,769 B1 * 1 1/2002 Barat ..................... HO1H 13,70
`2004
`6,529,125 B1* 3/2003 Butler .................... B6OK 37,06
`340/407.1
`
`* cited by examiner
`
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 3 of 22 PageID #: 29
`
`
`
`U.S. Patent
`
`Jul. 25, 2017
`
`Sheet 1 of 7
`
`US 9,713,994 B2
`
`
`
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 4 of 22 PageID #: 30
`
`
`
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 5 of 22 PageID #: 31
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 5 of 22 PageID #: 31
`
`U S. Patent
`
`Jul. 25, 2017
`
`Sheet 2 of 7
`
`US 9,713,994 B2
`
`.
`.Sm“\w-‘st-“wxst-mw-mxva- '
`‘
`
`:‘
`2-»
`Ag
`3
`§
`:‘
`§:
`§
`\
`1 x
`x
`9
`‘
`$
`
`x
`
`A‘g‘x
`s‘“&$¢‘
`§‘
`\
`‘\
`§
`c‘.
`~°
`
`’
`=:
`i-
`\‘X
`5“ ix\xxxx\xx\\\\\\\\\\\\\\\\\xxx\x\\«xxxxxaxxxm»‘
`x“\\\
`:~
`_
`,
`3 )\\m\\ywm\\\xx\\\wxxx\wngv-M‘\\
`1-
`.“
`.1"
`5:;
`\x
`
`
`’(K .‘
`:‘
`'
`S'
`“\‘ms'dxxxfix\\\\\\\\\\\\\\wxmmm\\x§
`\
`
`\
`\z Wxxxfi.:\'\\\\‘x\\\\\«««:«\\\;~((\'.u‘uq‘ufi
`
`
`
`“.5.-f
`
`
`
`
`xk\m
`
`I
`g
`,~\
`.\
`\{
`\v ‘
`M,
`«KKK ~39
`e‘ww.\w“WM“W‘WM‘N
`
`
`
`\v
`
`g!l-n‘n‘flv)
`
`
`
`~ ~\\ w “x -- u
`w:
`xxsfigx 3&1 ,‘V, \k
`
`
`
`xse :
`
`«\‘xm‘xw«\\\\W\\V\W\Wx\\\\\w~\\
`
`95
`
`élilillllW’yll’l’llilll’”;yfll'll
`
`.9.-
`x“
`
`
`
`“
`w .
`'
`..\
`
`
`v:;.;
`
`
`
`/"’a.IIV/l/I'I'I‘I‘Iua
`
`
`
`
`
`3,
`‘xm¢¢xx
`3
`x
`i
`a
`§
`:
`5Mxxwxxmxxwxxw‘xwxwwxxmmfi
`§
`3
`k
`.1 3
`
`
`
`\\‘\‘\H« «V wamwkg
`i
`WW
`S
`”“9 ~ 9‘
`
`1:‘-3~.\\\EIN§ \xfin ‘
`, k
`-:
`t
`: a
`$§fi§$~¥~§$§
`”\‘m‘w’
`“k
`~ «3.3% \o§
`gt
`k
`5.
`“ “
`hwwxsmxx~ww+mm>>>>w
`
`,
`¥\\\\\\\\\V\\\\\\\\\\\m\~\wm“.
`
`.r:
`‘
`
`i S
`~33“
`
`k
`>
`E
`\‘N
`§
`35.
`
`k
`xxx
`.
`u\\\m\\v\vx\\\mm\wvvwm
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
` igz
`
`
`
`
`
`
`
`
`nvmmnmvfimmxfi
`
`g
`
`
`
` .xwi/ir‘l/lmiWilli/Ifl
`
`.dx‘is‘Q0)‘1‘!iiil/iil/«IWfliiii/il/flmflwm-mU/liu
`
`
`
`
`. \
`V
`\‘
`\
`\é
`~
`3
`
`'
`”W ‘
`W
`\5
`\\§.\\\“;\.\
`{a -\
`saw -.c-\ w .w, xx
`$3\t\\w;3®\
`>9 :rs-‘aevox'r‘x 5'-
`.-u,-,-.»\~,~(W.-.v.-.w
`
`
`*
`
`§2
`\
`:
`..
`;
`1
`1:
`2
`
`
`
`§3
`
`
`
`
`:»
`-.
`§
`x
`
`
`WWW\V~W~\\\\3“\MV\
`\-
`.
`.
`.
`V
`x x u a. V
`1
`waw mwm
`:
`\\ ‘ y :\H 5
`
`9, >
`$
`'
`.
`mmnm
`
`'
`
`‘é1
`§
`3.
`§
`\
`%
`1;.
`
`§“x
`
`.‘\.‘
`:,
`Q
`3
`3
`'$\\\\\MNM\W5\WMM\NA\\R
`g
`:5
`z?
`§
`i
`5
`\
`,\
`(«Hams
`\
`
`Q
`\3
`1
`k
`i
`t
`\
`»
`g
`\\\\\\\\\\\\\\\\\\\\\\\Wmmfin
`lmxwAmmwamxmxmmmxwmwi
`‘
`i
`
`2*
`
`
`
`
`
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 6 of 22 PageID #: 32
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 6 of 22 PageID #: 32
`
`US. Patent
`
`Jul. 25, 2017
`
`Sheet 3 of 7
`
`US 9,713,994 B2
`
`S
`\Yex‘ax.
`\\¢ -.\ «.
`‘x
`Mix
`
`“ "“
`
`
`
`Hump.i-(-\'\\\WW\\\\\\\\\\\\\‘~3«\wfi‘.‘m
`
`T
`y
`§
`\
`‘g
`5»
`\
`x
`3
`s
`3
`\
`kQ
`:1;
`
`
`-.
`\z
`
`I:
`§
`\
`
`3'
`g
`3
`3
`'
`\
`\
`
`\
`'
`\
`“HQ-S
`§w«\:
`s
`- 3
`s
`S
`
`9.
` A
`
`a
`Wmmwwxw‘w‘mg
`IE
`
`
`
`
`Hat”,”lulu/”m
`
`
`~ \ \
`.
`‘
`* e
`
`‘4 4
`V 0m . 4‘“
`‘\“
`
`o
`.
`~
`‘. w
`«A
`\‘0 \_~‘
`4 ,-‘:~
`
`
`
`1 1 ‘1.
`.~
`.4ul“
`
`
`
`“‘0.
`
`
`
`221
`
`,6
`
`;fl
`
`
`
`
`”woman/”my-,
`'I01-ia‘mw/fl/x».
`
`I’Mulmww»;¢
` .«Inn/”caA
`
`r’.
`
`§
`
`4
`
`
`
`.«4cwVInt/(z/l/(ll
`
`
`
`S\
`-:
`
`ss§§§>
`
`CS,
`
`§Q
`sik
`
`ss\\‘x:
`
`1\
`§§
`34
`1;5fi
`3Ra
`3k
`%§
`3§
`Q$V
`
`a .
`
`E % §§3sx\§‘\§§§ ,9
`
`
`.wwan-”syn”;vw”.
`
`
`
`WWW 4%
`
`
`
`U.S. Patent
`
`Jul. 25, 2017
`
`Sheet 4 of 7
`
`US 9,713,994 B2
`
`
`
`
`
`
`
`& origcaisikiti dist:3ction; remind&: wis
`intertaixterisikaas view &isers
`Sisir irri's: stificatio Wis
`it:tertaif$xteira Searies vircr:
`
`&
`
`8
`
`9
`
`iss-Ostfag. Afte: etergencies
`& Frissess
`:
`For Busy
`Predictive Aigorit: its
`& Oi Ramp Fror Highway Speed
`y C. Rarots High say speed
`Bird Wies, Aaasi
`& Briwing riftetected
`:
`Straight Aeaci Vector
`cite care:
`east:eiata tegy
`ong as a ti distractor featurs
`t
`riser Bieristics airii; a.ki: activity
`8
`fias & AS Rigats seifatias its
`s
`e intelligest vehicle featies
`Se'ice of it
`& Natia: E:fircrers
`a vehicle peratic
`resis
`* Switches decision Algor:the
`Accident Cecil as Aigg:it: it
`s Sanity Check Algorithm
`*
`irre & Locatiof Aigarith: :
`is
`log Agoritiricircuit
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Sea; ratics Eclip rest Cositections
`{}
`aise fore
`{
`a gigastics
`'C
`is: rectory
`to vessage Pecci cer
`Page:
`to e-rai
`{
`i rites" et Bird's sirg
`{
`i teiiser it coisios Aias it's
`is rigtai srti isÉfactic:
`
`k
`
`as
`
`is ratics fairs erst
`A sing: cer arts f Fascinack f Aiair-s etc.
`
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 7 of 22 PageID #: 33
`
`Figure 5A
`
`
`
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 8 of 22 PageID #: 34
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 8 of 22 PageID #: 34
`
`US. Patent
`
`Jul. 25, 2017
`
`Sheet 5 of 7
`
`US 9,713,994 B2
`
`
`Ham: EUS at: SENSOR OE 131111121:me \P’AMJES
`S we
`
`OtiitlllfltoflOl
`
`{‘3211 1 Kfirsg-.183 f
`119.1113: Gear
`D five :1 e.ar
`C [8‘1E1 deg
`Tra ~15.1
`'
`8 Fa kiss
`6 ForczefT:1e'-o
`Thm‘f.EE 3811; {1:1}
`<3 35 $913139
`
`Tack?flEx‘EF-Eng RE 131.311 raw-Rime-
`1511113maticTrammi$513!: Figwhe-Q WHEEL"
`r139 121 Fr
`regensm Changein {9:11 t’ip’si
`3121111.
`S‘s-EH g member a rm
`
`[It’lltivl'll
`
`98.th
`
`-
`
`"‘99-"538 M 99:13 “ism
`*
`~
`Wee: 5-8530?
`
`Q;W,-Temperagure
`,. M113*er
`
`-rkm‘er
`
`32711551313? weigh! Sen-5m
`i511: W???at gasi‘tioz"
`
`.
`
`R $132121 51* 1:5 or
`Airbag [fiepayme‘rs‘t
`Reiiij‘aa 85K
`Tires PrESSure. 111 E1, RR, R1
`
`A 11113 Erma-r Agséstani 9181193115
`Sac:
`1m Signaiidme‘:
`11113911119813:n“
`
`~- ['1 11111117121 sg
`85¢ flapsrwre
`
`arm‘itian 11
`‘Wigyer Seififlg
`HEgh'seam E:gm
`Fag 13mg
`HVAE Setting
`Toi:
`'
`.
`behride 5p;ed
`Numm-r’u1‘Pa;51:11115114'
`
`Bea11n-gf-‘isir-g (1 ‘8-
`
`ngm EB
`
`
`
`U.S. Patent
`
`Jul. 25, 2017
`
`Sheet 6 of 7
`
`US 9,713,994 B2
`
`
`
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 9 of 22 PageID #: 35
`
`Figure 6
`
`
`
`U.S. Patent
`
`Jul. 25, 2017
`
`Sheet 7 Of 7
`
`US 9,713,994 B2
`
`
`
`
`
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 10 of 22 PageID #: 36
`
`FIGURE 7 c -
`
`
`
`1.
`SAFETY CONTROL SYSTEM FOR APPS IN
`VEHICLES
`
`US 9,713,994 B2
`
`REFERENCE TO CO.-PENDING APPLICATIONS
`
`The present application is a continuation of U.S. patent
`application Ser. No. 14/661,598 filed Mar. 18-2015, which
`claims the benefit of an priority from U.S. patent application
`Ser. No. 13/663,085, filed Oct. 29, 2012, which claims the
`benefit of an priority from U.S. patent application Ser. No.
`10/383,708, filed May 4, 2004, which claims the benefit of
`a priority from Ser. No. 10/287,299, filed Nov. 4, 2002,
`which claims the benefit of a priority from U.S. patent
`application Ser. No. 10/279.447, filed Oct. 24, 2002, Provi
`sional Application No. 60/336,293 filed Oct. 24, 2001, and
`Provisional Application No. 60/390,877 filed Jun. 21, 2002,
`the contents of which are incorporated herein by reference.
`
`10
`
`15
`
`FIELD OF THE INVENTION
`
`The present invention relates to the field of telematics,
`namely to the field of integrating information, communica
`tion, computing and entertainment technologies into
`vehicles for civilian or military use. The invention particu
`larly relates to safety control systems for vehicles to reduce
`driver distraction, avoiding potentially dangerous conditions
`tending to produce accidents.
`
`BACKGROUND OF THE INVENTION
`
`One potentially dangerous condition is the use of a
`vehicle telephone by the vehicle driver while driving the
`vehicle. The use of telematics in general and particularly
`cellular telephones by drivers while driving has been found
`to increase the possibility of an accident since Such a
`telephone not only diverts the drivers attention from driv
`ing, but also generally requires the use of at least one of the
`drivers hands and distract the driver's eyes from the road
`and traffic. In fact, many states and countries have enacted
`legislation requiring that telephones used in vehicles by
`drivers while driving must be of the “hands free” type and
`usually telematics equipment carries a warning to educate
`and discourage the driver about the risk of using these
`devices while driving. However, such legislation is difficult
`to enforce and education is not usually effective in assuring
`driver compliance. Moreover, even where the vehicle is
`equipped with a “hands free' telephone, drivers nevertheless
`still frequently use one hand for holding or dialing the
`telephone. When one hand is occupied by holding a tele
`phone, the danger of causing an accident in an emergency
`situation is increased because of the additional reaction time
`required to properly grip the steering wheel with both hands.
`Similar danger exists when the driver attempts to control
`audio and video equipment, e.g. Radio, Music CD, DVD,
`Books on tape etc., or when the driver attempts to change
`environmental controls like adjusting the heat or air condi
`tioning, or other vehicle settings.
`There are other potentially dangerous conditions and
`inherent risks in driving that depends on the driving act
`itself. Such as rapidly accelerating or decelerating, excessive
`maneuvering, merging to or exiting a freeway, passing,
`changing lanes, changing gears, depressing the clutch, driv
`ing at high speed, negotiating a turn, braking, reverse
`driving, or a stress condition on the part of the driver, that
`could increase the possibility of an accident should the
`driver be distracted by activation of the telephone or other
`signal or device. This inherent risk is also dependent on the
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 11 of 22 PageID #: 37
`
`2
`driving purpose as well, for example, the risk in driving a
`police cruiser is inherently riskier then in driving a sedan,
`and driving a delivery van has different risk than driving the
`family van.
`Herbert et al., U.S. Pat. No. 6,188,315 and Brown, U.S.
`Pat. No. 6,353,778, disclose systems for avoiding preset
`potentially dangerous conditions while operating a vehicle
`having a vehicle telephone, but the systems described in
`those patents are of relatively limited application, and do not
`provide for avoiding dangerous conditions or to managing
`risk and individualizing the warnings to individual driving
`skills or application and to combinations of events and
`environmental conditions.
`
`SUMMARY OF THE INVENTION
`
`An object of at least some presently preferred embodi
`ments of the present invention is to provide a safety control
`system for vehicles tending to reduce the possibility of
`accidents in one or more of the above respects. Another
`object of at least some presently preferred embodiments of
`the invention is to provide a method of reducing or avoiding
`driver distraction during potentially dangerous conditions
`encountered while operating a vehicle.
`According to one aspect of one embodiment of the present
`invention, a safety control system for vehicles, includes, a
`communication device having at least one of an input
`accessible from within the vehicle and an output commu
`nicated within the vehicle, at least one sensor operable to
`sense at least one condition related to vehicle operation, and
`a controller communicated with the sensor and the commu
`nication device to selectively suppress at least one of said
`input and said output in response to a sensed parameter of
`said at least one condition being outside of a threshold.
`When an input is suppressed, the driver is prevented from
`accessing or inputting information into the communication
`device. When an output is suppressed, communication
`between the device and the driver of a vehicle is suppressed
`to, among other things, avoid distracting the driver during
`certain driving situations or conditions.
`According to one aspect of one embodiment of the present
`invention, there is provided a safety control system for
`vehicles including a telephone or other input or output
`device and one or more sensors for sensing instantaneous
`driver Stimuli and/or a potentially dangerous condition and
`for automatically disabling or Suppressing the telephone or
`other input or output device when sensing such stimuli and
`condition. In one form, the sensors include two sensors
`mounted on a steering member to provide an indication of
`the presence of the drivers hands on the vehicle steering
`member, and effective to suspend use of the telephone or
`other input/output device when the two hands of the driver
`are not sensed as present on the steering member while the
`vehicle is in motion. This system is modular, dynamic,
`interactive, and adaptive to each individualized user. In one
`implementation, the invention employs a method for auto
`mated machine prioritizing to provide assistance to the to
`driver and optimize the functionality of telematics features
`accessibility by arranging them according to a user's needs
`and preferences based on usage frequency of individual
`features and/or application or as customized individually by
`the user preferences, skills and events. In another embodi
`ment, sensors on a steering member are used to measure
`changes in driver physiology. Other methods can be used for
`sensing driver physiology, e.g. via infrared detection, cam
`era and image/color recognition etc.
`
`
`
`4
`devices when a predetermined physiological condition is
`sensed. For example, the physiological conditions sensed
`could be a predetermined gripping force applied by a hand
`of the driver while gripping the steering wheel, or a prede
`termined pulse rate, temperature, blood pressure, blood
`oxygen level, and/or skin conductivity of the driver. Such
`physiological conditions may indicate a stress condition of
`the driver and, when sensed, can lead to disabling or
`Suppressing operation of the input/output devices to avoid
`aggravating the stressed condition.
`The system may also sense a drowsiness condition of the
`vehicle operator. For example, the system may include a
`steering direction sensor that actuates a drowsiness alarm
`when sensing a failure to change the steering direction
`within a predetermined time, distance interval while
`accounting for vehicle speed in indicating a possible drowsi
`ness condition in the driver. Additionally, Such sensor when
`monitored with respect to changes over time will indicate
`jerk reaction, which indicates that the driver was not paying
`attention and the system will temporarily suspend all tele
`matics to give the driver a chance to recover. Another
`application for Such a sensor is the monitoring of an OFF
`Zero angle for an extended period of time/distance which
`can indicate a blind curve or hard curve, and again, here the
`system will temporarily Suspend the telematics and/or input/
`output devices from interacting with the driver, and vice
`versa, until normal driving functions are restored.
`According to another aspect of the present invention,
`there is provided a method of avoiding potentially dangerous
`conditions while operating a vehicle having an input/output
`device and a steering mechanism including a steering mem
`ber to be manipulated by the driver, comprising: providing
`the steering member with two sensors for sensing the
`presence on the steering member of the two hands of the
`driver; and disabling the input/output device when the two
`sensors fail to sense the presence on the steering member of
`both hands of the driver while the vehicle is in motion.
`According to further features in the described preferred
`embodiment, the input/output devices may also be disabled
`when the vehicle is traveling in the reverse direction, or is
`being braked, or is within a predetermined proximity of
`another vehicle, or is traveling at a high Velocity, acceler
`ating, decelerating, merging onto or exiting a freeway,
`passing, changing lanes, changing gears, depressing the
`clutch, or a driver is occupied using other accessories in the
`vehicle or otherwise distracted. Since a high degree of
`attention of the driver is required under all the foregoing
`conditions, operation of the vehicle telephone, for example,
`even the ringing signal of an incoming telephone call, could
`be highly distracting to the driver and is therefore disabled
`to avoid the possibility of increasing the risk of an accident.
`To assure that the driver and the vehicle as well as on
`board communication devices as described above are work
`ing harmoniously together, one presently preferred embodi
`ment of the system includes the following:
`The Driving Systems, (Man, Machine, Environment,
`Regulation, and History)
`Man: the driver, the passengers, the pedestrians, Society;
`Machine: the car, the telematics, the infrastructure;
`Environment: the driving environment (in the car and
`outside the car and the infrastructure used)
`History: the personal driving experience, the equipment
`maintenance history
`Regulation: the existing laws and common safe driving
`etiquette into, Society and the infrastructure regulation.
`All of these elements will be harmonized by the system as
`it isolates the drivers from non driving related distractions
`
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 12 of 22 PageID #: 38
`
`40
`
`50
`
`60
`
`US 9,713,994 B2
`
`5
`
`10
`
`15
`
`25
`
`30
`
`35
`
`3
`Smart Speaker: Incoming calls are routed to a speaker that
`reflects and bounce sounds of front windshield at driver
`Look Ahead, Eye Level. Or simulate such action so that a
`driver focuses or has his/her attention directed toward the
`windshield just like he would do if he is carrying a conver
`sation with another person.
`According to further aspects in the described preferred
`embodiment, the steering member is a steering wheel, and
`the sensors include two sensors on opposite sides of the
`steering wheel located to sense the presence of the driver's
`hands on the steering wheel. Preferably, the two sensors are
`located approximately on or between the “two and “ten'
`and the “three' and “nine' clock positions of the steering
`wheel.
`It will thus be seen that such a system, requiring both
`hands to be on the steering wheel in order for the driver to
`operate the input/output devices, not only requires the
`vehicle to be equipped with a “hands free’ interface for the
`input/output devices, or a system that can be used as Such
`with an adapter or when docked to the system gateway, but
`also enforces the use of the “hands free” feature by sensing
`that the driver actually has both hands placed on the steering
`member before the input/output devices can be operated
`accessed or displayed to the driver. Disabling the operation
`of the device would preferably include not only disabling
`making outgoing and receiving incoming telephone calls,
`but also disabling the signal (typically audible tones, vibra
`tions, or visible light) of an incoming call, fax, e-mail, the
`display of non-urgent vehicle status or warning indicators,
`since Such signals, indicators or displays can distract the
`driver. Such distractions are problematic at times and con
`ditions wherein operation of the vehicle requires more than
`usual driver attention and interaction, or when other distrac
`tions are already present for the driver.
`According to further features in the described preferred
`embodiment, the vehicle may also include a computer or the
`driver may also use a portable multi-function telematics
`device in the vehicle allowing access to the Internet or other
`network for transmitting and/or receiving faxes or e-mail or
`browsing the web or accessing a WAN, with the sensors also
`disabling driver initiated access to such devices when the
`two hands of the driver are not sensed on the steering
`member while the vehicle is in motion.
`In most cases, the steering member would be a steering
`wheel as presently included in conventional vehicles. How
`45
`ever, in certain applications the steering member could be a
`joystick, or other type of steering member. In Such case, the
`sensors are placed in areas a driver is recommended or
`required to grip the steering member to safely control the
`vehicle.
`According to further optional features in the preferred
`embodiment of the invention described below, the sensors
`may further include devices for sensing vehicle acceleration,
`deceleration, merging onto or exiting a freeway, passing,
`changing lanes, changing gears, depressing the clutch, a
`reverse-drive condition of the vehicle, the braking of the
`vehicle, the undue proximity of the vehicle to another
`vehicle, excessive maneuvering, and/or an unduly high
`velocity of the vehicle, any one of which conditions, or
`combination of conditions, may also be effective to disable
`the operation of the telephone, computer, or other potentially
`distracting equipment, display or indicator within the
`vehicle.
`According to still further optional features in the preferred
`embodiment of the invention described below, at least one of 65
`the sensors on the steering member also senses a physiologi
`cal condition of the driver and disables the input/output
`
`55
`
`
`
`US 9,713,994 B2
`
`5
`and helps them comply with driving related laws and
`etiquette via reminders and passive assistance.
`Further features and advantages of at least some of the
`embodiments or implementations of the invention will be
`apparent from the description below.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`These and other objects, features and advantages of the
`present invention will be apparent from the following
`detailed description of the preferred embodiments and best
`mode, appended claims and accompanying drawings in
`which:
`FIG. 1 schematically illustrates one form of a safety
`control system for vehicles constructed in accordance with
`the present invention;
`FIG. 2 is an enlarged view illustrating the steering wheel
`in the vehicle of FIG. 1 and the sensors mounted thereon;
`FIG. 3 is a block diagram illustrating the main compo
`nents in the system of FIG. 1;
`FIG. 4 is a flowchart illustrating the operation of the
`system of FIG. 1; and
`FIGS. 5A and 5B show a block diagram illustrating the
`nature and the flow of signals and algorithms used in one
`presently preferred embodiment of the system of the present
`invention.
`FIG. 6 is a diagram illustrating steering wheel with all
`sensors described.
`FIGS. 7A, 7B and 7C are schematic diagrams illustrating
`various gesture capture sensors described.
`
`10
`
`15
`
`25
`
`30
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`6
`tivity (e.g. perspiration), loud Voice(s) or stressful sounds in
`the cabin, such as baby crying, dog barking etc., any
`combination of one or more of which conditions would also
`disable incoming telematics. A further condition sensed by
`the system is the possibility of drowsiness on the part of the
`driver, in which case an audio alarm would be activated to
`alert the driver to this condition. Examples without limita
`tion of other alarms to overcome driver drowsiness include
`vibration in the seat, changing HVAC temperature settings
`and blower speed to extremes, change of seat position, radio
`volume or station, CD-track etc. The system will restore
`operation of the input/output devices when conditions are
`normalized and will notify driver of all missed activities.
`Vehicle 2 illustrated in FIG. 1 is a conventional vehicle
`including a steering mechanism, generally designated 3.
`having a steering wheel 4, a propulsion device such as a
`motor or engine 5 for driving the vehicle via a transmission
`or other torque converting means Schematically indicated 6.
`an acceleration pedal 7, and a braking pedal 8 for controlling
`the vehicle. Vehicle 2 further includes one or more visual
`indicator and audio alarms 9. e.g. mounted within the
`forward-look ahead viewing or hearing by the driver.
`FIG. 1 further schematically illustrates a cellular tele
`phone 10 within the vehicle, and a computer 11 or other
`multifunction telematic device allowing access to the Inter
`net for transmitting and/or receiving faxes or e-mail, WAN
`and Web access, or other input/output device. Other input/
`output devices include vehicle fault/warning lights (battery,
`temperature, washer fluid, etc.) or other signal or alarm
`(open door, low fuel level, seatbelt monitor, etc.). Vehicle 2
`illustrated in FIG. 1 may also include many other compo
`nents conventionally provided on vehicles at the present
`time or to be provided in the future.
`The safety control system included in vehicle 2 illustrated
`in FIG. 1 includes a plurality of sensors for sensing various
`conditions with respect to the vehicle driver, the vehicle
`itself and/or the environment. These signals are collected via
`direct tapping to existing or added sensors or via vehicle bus
`and user specified values. These include sensors S1 and S2
`in FIGS. 2, 7A, 7B, 7C applied to the steering wheel 4 of the
`vehicle; sensor S3 applied to the steering mechanism 3 of the
`vehicle to sense changes in the steering direction and/or
`actuation of the turning indicator. The turning signal indi
`cator Switch/lever can also act as a blind spot collision
`avoidance actuator. When a driver actuates the turn signal
`indicator by moving the turning signal lever in advance of
`making a turn, Subsequent momentary pull up or momentary
`push down on the lever will move the corresponding mirror
`further out to scan the vehicle blind spot.
`Other sensors may include sensor S4 sensing the condi
`tion of the gas pedal 7 and/or vehicle speed or acceleration;
`sensor S5 sensing the condition of the braking pedal 8; and
`sensor S6 sensing the condition of the transmission or other
`type torque converter 6.
`Also schematically illustrated in FIG. 1 are sensors S7 and
`S8 carried to sense the proximity of the vehicle with respect
`to another vehicle; sensor S9 sensing darkness or alterna
`tively sensing the activation of the headlight; and sensor S10
`sensing weather conditions rain, sleet, Snow, ice, tempera
`ture and/or sensing the activation of the front or rear wipers
`or headlight wipers.
`As will be described more particularly below, the fore
`going sensors (or signals) are generally effective only when
`the vehicle is moving to sense their respective conditions
`and to execute certain control functions in order to decrease
`the possibility of an accident. One important control func
`tion is to disable an incoming call from ringing the telephone
`
`35
`
`45
`
`FIG. 1 schematically illustrates a vehicle, generally des
`ignated 2, equipped with a control system for sensing a
`variety of risk factors and potentially dangerous conditions
`and for automatically executing various responses when
`sensing Such conditions in order to avoid hazardous situa
`tions tending to increase the possibility of an accident. One
`40
`response is the disabling or Suppression of one or more input
`or output devices to avoid interaction between the devices
`and the driver in certain situations and conditions. Another
`response includes providing a signal to or requiring the
`driver to take some action to increase driver alertness and/or
`aWaSS.
`One example of a hazardous situation avoided by the
`control system illustrated in FIG. 1 is the use of the vehicle
`telephone in certain situations wherein a making of a tele
`phone call by the vehicle driver, or the receiving of an
`incoming call, particularly the ringing of Such a call, may
`distract the driver and increase the possibility of an accident
`when the driver is in a high-risk driving situation. Similar
`increased risk can result from the driver changing vehicle
`controls like temperature settings, or interacting (e.g. input
`ting or receiving output) with other telematics such as
`e-mail, radio, CD, DVD, navigation system, incoming page
`or the like. In such cases, the vehicle telephone, other
`telematics and/or other input/output devices are Suppressed
`and no incoming or outgoing signals are allowed to distract
`the driver. In case the driver is the party initiating the
`telematics, a visual indicator and audio feedback can be
`activated to indicate to the driver that telematics is disabled,
`Supply reason therefore, and even recommend driving modi
`fication to enable telematics. Another condition sensed by
`the system is undue stress in the driver, as indicated by the
`sensed pulse rate, temperature, blood pressure, skin conduc
`
`50
`
`55
`
`60
`
`65
`
`Case 1:21-cv-00064-UNA Document 1-1 Filed 01/21/21 Page 13 of 22 PageID #: 39
`
`
`
`US 9,713,994 B2
`
`5
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`7
`10, and the computer or other telematics portable or built in
`11 from accessing the Internet or announcing incoming
`signals, e.g. page, e-mail etc., and to indicate same by
`actuating a visual indicator and an audio feedback if a driver
`attempts to initiate telematics during an unsafe or a high risk
`condition, and may direct a driver to alternative driving habit
`to gain access to telematics. The system may also suppress
`delivery of unnecessary external signals such as certain
`vehicle warning lights or alarms, the system will restore
`function of the input/output devices when conditions are
`normalized and will notify driver of all missed activities. In
`Some cases, such as where a drowsiness condition is sensed,
`an audio alarm 9 is actuated. Other possible alarms to
`overcome driver drowsiness would include vibration in the
`seat, changing HVAC temperature settings and blower speed
`to eXtremes, etc.
`FIG. 2 more particularly illustrates the sensors S1, S2 and
`in FIGS. 2, 7A, 7B, 7C mounted on the steering wheel 4. As
`shown in FIG. 2, the two sensors are mounted on or between
`the “two and “ten