Case 6:20-cv-01001-ADA Document 1-4 Filed 10/28/20 Page 1 of 15
`
`
`
`
`Exhibit 3
`
`

`

`Case 6:20-cv-01001-ADA Document 1-4 Filed 10/28/20 Page 2 of 15
`I IIIII IIIIIIII Ill lllll lllll lllll lllll lllll lllll lllll lllll 111111111111111111
`US007793136B2
`
`c12) United States Patent
`Lutter
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 7,793,136 B2
`Sep.7,2010
`
`(54) APPLICATION MANAGEMENT SYSTEM
`WITH CONFIGURABLE SOFTWARE
`APPLICATIONS
`
`(75)
`
`Inventor: Robert Pierce Lutter, Tacoma, WA
`(US)
`
`(73) Assignee: Eagle Harbor Holdings LLC,
`Bainbridge Island, WA (US)
`
`5,045,937 A
`5,111,401 A
`5,115,245 A
`5,245,909 A
`5,287,199 A
`5,303,297 A
`5,339,086 A
`
`9/1991
`5/1992
`5/1992
`9/1993
`2/1994
`4/1994
`8/1994
`
`Myrick
`Everett, Jr. et al.
`Wen et al.
`Corrigan et al.
`Zoccolillo
`Hillis
`DeLuca et al.
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 923 days.
`
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`(21) Appl. No.: 11/616,650
`
`(22) Filed:
`
`Dec. 27, 2006
`
`(65)
`
`Prior Publication Data
`
`US 2007/0277175 Al
`
`Nov. 29, 2007
`
`Related U.S. Application Data
`
`(63) Continuation of application No. 10/132,886, filed on
`Apr. 24, 2002, now Pat. No. 7,178,049.
`
`(51)
`
`Int. Cl.
`G06F 11100
`(2006.01)
`(52) U.S. Cl. ........................................... 714/1; 718/100
`(58) Field of Classification Search ..................... 714/1,
`714/2, 3, 10, 13; 718/100, 101, 102, 103,
`718/104, 106, 107
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`2,995,318 A
`4,303,978 A
`4,528,563 A
`4,591,976 A
`4,829,434 A
`4,907,159 A
`5,008,678 A
`5,031,330 A
`
`8/1961 Cocharo
`12/1981 Shaw eta!.
`7/1985 Takeuchi
`5/1986 Webber et al.
`5/1989 Karmel eta!.
`3/1990 Mauge eta!.
`4/1991 Herman
`7/1991 Stuart
`
`DE
`
`3125151
`
`1/1983
`
`(Continued)
`
`OTHER PUBLICATIONS
`
`Stirling A: "Mobile Multimedia platforms" Vehicular Technology
`Conferene Fall 2000. IEEE VTS Fall VTC2000. 52nd Vehicular
`Technology Conference (CAT. No. OOCH37152).
`
`(Continued)
`
`Primary Examiner-Dieu-Minh Le
`(74) Attorney, Agent, or Firm-Stolowitz Ford Cowger LLP
`
`(57)
`
`ABSTRACT
`
`An application management system identifies a new device.
`The new device is configured into a multiprocessor system
`when a type of data used by the new device conforms with a
`type of data used in the multiprocessor system.An application
`in the multiprocessor system is identified that uses a same
`data type used on the new device. The stored application is
`then used to take over control of the new device and process
`data received from the new device. A security protocol can be
`optionally used to control what types of data, applications, or
`devices are allowed to access the multiprocessor system.
`
`31 Claims, 5 Drawing Sheets
`
`,---1 __
`
`- - _'\_ - - ,
`
`___ \_ __
`
`r·-·-·-·,
`I
`j
`
`!
`
`:
`
`I
`
`ll
`
`iS5NSORFUSION!
`j
`THREAD
`!
`"
`!_. ~·.-:.·.°=f·~·.:.-. -
`-·-·-··-·-·-·~
`/2.
`I
`I
`:
`:
`;
`i
`i __ 1._ --~~-·-·c~~--J __ ~-----~ ,_j .i
`
`SENSOR FUSION
`THREAC
`
`I
`
`aa..J
`
`JVM
`
`! El
`·{ ~ MESSAGE MI\NAGER
`
`j I
`
`CRITICAL DATA MANAGER
`
`SECURITY MANAGER
`
`DATA MANAGER
`
`TASK MANAGER
`
`CONFIGURATION MANAGER
`
`"
`..
`"'
`..
`..
`"
`
`

`

`Case 6:20-cv-01001-ADA Document 1-4 Filed 10/28/20 Page 3 of 15
`
`US 7,793,136 B2
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`5,341,301 A
`5,438,361 A
`5,471,214 A
`5,506,963 A
`5,532,706 A
`5,552,773 A
`5,572,201 A
`5,581,462 A
`5,585,798 A
`5,617,085 A
`5,646,612 A
`5,749,060 A
`5,751,211 A
`5,761,320 A
`5,786,998 A
`5,872,508 A
`5,907,293 A
`5,915,214 A
`5,943,427 A
`5,963,092 A
`5,964,822 A
`5,966,658 A
`5,969,598 A
`5,977,906 A
`5,983,092 A
`5,983,161 A
`6,009,330 A
`6,028,537 A
`6,028,548 A
`6,054,950 A
`6,060,989 A
`6,061,709 A
`6,097,285 A
`6,128,608 A
`6,148,261 A
`6,150,961 A
`6,154,123 A
`6,161,071 A
`6,163,711 A
`6,166,627 A
`6,167,253 A
`6,169,894 Bl
`6,175,728 Bl
`6,175,782 Bl
`6,181,994 Bl
`6,182,006 Bl
`6,202,027 Bl
`6,203,366 Bl
`6,204,804 Bl
`6,226,389 Bl
`6,233,468 Bl
`6,240,365 Bl
`6,243,450 Bl
`6,252,544 Bl
`6,275,231 Bl
`6,292,109 Bl
`6,292,747 Bl
`6,294,987 Bl
`6,297,732 B2
`6,298,302 B2
`6,326,903 Bl
`6,327,536 Bl
`6,362,748 Bl
`6,374,286 Bl
`6,389,340 Bl
`6,405,132 Bl
`6,408,174 Bl
`6,417,782 Bl
`6,421,429 Bl
`6,429,789 Bl
`6,429,812 Bl
`
`8/1994 Shirai et al.
`8/1995 Coleman
`11/1995 Faibish et al.
`4/1996 Ducateau et al.
`7/1996 Reinhardt et al.
`9/1996 Kuhnert
`11/1996 Graham
`12/1996 Rogers
`12/1996 Yoshioka et al.
`4/1997 Tsutsumi et al.
`7/1997 Byon
`5/1998 Graf et al.
`5/1998 Shirai
`6/1998 Farinelli et al.
`7/1998 Neeson et al.
`2/1999 Taoka
`5/1999 Tognazzini
`6/1999 Reece et al.
`8/1999 Massie et al.
`10/1999 VanZalinge
`10/1999 Alland
`10/1999 Kennedy et al.
`10/1999 Kimura
`11/1999 Ameen
`11/1999 Whinnett et al.
`11/1999 Lemelson et al.
`12/1999 Kennedy et al.
`2/2000 Suman eta!.
`2/2000 Farmer
`4/2000 Fontana
`5/2000 Gehlot
`5/2000 Bronte
`8/2000 Curtin
`10/2000 Barnhill
`11/2000 Obradovich et al.
`11/2000 Alewine
`11/2000 Kleinberg
`12/2000 Shuman et al.
`12/2000 Juntunen et al.
`12/2000 Reeley
`12/2000 Farris et al.
`1/2001 McCormick
`1/2001 Mitama
`1/2001 Obradovich et al.
`1/2001 Colson et al.
`1/2001 Meek
`3/2001 Alland et al.
`3/2001 Muller et al.
`3/2001 Andersson
`5/2001 Lebelson et al.
`5/2001 Chen
`5/2001 Bunn
`6/2001 Jansen et al.
`6/2001 Hoflberg
`8/2001 Obradovich et al.
`9/2001 Murano et al.
`9/2001 Amro eta!.
`9/2001 Matsuda et al.
`10/2001 Hsu et al.
`10/2001 Walgers et al.
`12/2001 Gross et al.
`12/2001 Tsuji et al.
`3/2002 Huang
`4/2002 Gee et al.
`5/2002 Rayner
`6/2002 Breed et al.
`6/2002 Steijer
`7/2002 Darnall
`7/2002 Merritt
`8/2002 Kiridena et al.
`8/2002 Hoflberg
`
`................ 340/435
`
`................ 701/24
`
`9/2002 Koike
`6,445,308 Bl
`9/2002 Drori
`6,452,484 Bl
`11/2002 Breed
`6,484,080 B2
`12/2002 Himmelstein
`6,496,107 Bl
`12/2002 Keller et al.
`6,496,689 Bl
`1/2003 Stuempfle et al.
`6,505,100 Bl
`2/2003 Obradovich et al.
`6,515,595 Bl
`2/2003 Dowling et al.
`6,522,875 Bl
`5/2003 Berry
`6,559,773 Bl
`9/2003 Lutter
`6,615,137 B2
`9/2003 Kitamura
`6,616,071 B2
`9/2003 Knockeart et al.
`6,622,083 Bl
`6,629,033 B2 * 9/2003 Preston et al.
`................ 701/70
`6,647,270 Bl
`11/2003 Himmelstein
`6,734,799 B2
`5/2004 Munch
`6,778,073 B2 * 8/2004 Lutter et al.
`6,778,924 B2
`8/2004 Hanse
`6,782,315 B2
`8/2004 Lu et al.
`6,785,551 Bl
`8/2004 Richard
`6,792,351 B2
`9/2004 Lutter
`6,901,057 B2
`5/2005 Rune
`6,952,155 B2
`10/2005 Himmelstein
`6,993,511 B2
`1/2006 Himmelstein
`7,006,950 Bl
`2/2006 Greiffenhagen et al.
`7,024,363 Bl
`4/2006 Comerford et al.
`7,079,993 B2
`7/2006 Stephenson et al.
`7,092,723 B2
`8/2006 Himmelstein
`7,120,129 B2
`10/2006 Ayyagari et al.
`7,123,926 B2
`10/2006 Himmelstein
`7,146,260 B2 * 12/2006 Preston et al.
`7,158,956 Bl
`1/2007 Himmelstein
`7,178,049 B2 * 2/2007 Lutter ........................... 714/1
`7,187,947 Bl
`3/2007 White et al.
`7,450,955 B2
`4/2007 Himmelstein
`7,249,266 B2
`7/2007 Margalit
`7,257,426 Bl
`8/2007 Witkowski et al.
`7,272,637 Bl
`9/2007 Himmelstein
`7,274,988 B2
`9/2007 Mukaiyama
`7,277,693 B2
`10/2007 Chen
`7,343,160 B2
`3/2008 Morton
`7,375,728 B2
`5/2008 Donath
`7,379,707 B2
`5/2008 Difonzo
`7,418,476 B2
`8/2008 Salesky
`7,587,370 B2
`9/2009 Himmelstein
`7,594,000 B2
`9/2009 Himmelstein
`7,596,391 B2
`9/2009 Himmelstein
`7,599,715 B2
`10/2009 Himmelstein
`7,614,055 B2 * 11/2009 Buskens et al.
`2001/0008992 Al
`7/2001 Saito et al.
`7/2001 L'Anson
`2001/0009855 Al
`2001/0018639 Al
`8/2001 Bunn
`2001/0041556 Al
`11/2001 Laursen et al.
`2001/0048749 Al
`12/2001 Ohmura et al.
`2001/0051853 Al
`12/2001 Evans et al.
`2002/0012329 Al
`1/2002 Atkinson et al.
`2002/0022927 Al
`2/2002 Lemelson et al.
`2002/0087886 Al
`7/2002 Ellis
`2002/0119766 Al
`8/2002 Bianconi et al.
`2002/0142759 Al
`10/2002 Newell et al.
`2002/0144010 Al
`10/2002 Younis et al.
`2002/0177429 Al
`11/2002 Watler et al.
`2002/0198925 Al
`12/2002 Smith et al.
`2003/0004633 Al
`1/2003 Russell et al.
`2003/0009270 Al
`1/2003 Breed
`2003/0011509 Al
`1/2003 Honda
`2003/0060188 Al
`3/2003 Gidron et al.
`2003/0065432 Al
`4/2003 Shuman et al.
`2003/0110113 Al
`6/2003 Martin
`10/2003 Nelson
`2003/0201365 Al
`2003/0201929 Al
`10/2003 Lutter et al.
`2004/0149036 Al
`8/2004 Foxlin et al.
`2004/0162064 Al
`8/2004 Himmel stein
`2004/0164228 Al
`8/2004 Fogg et al.
`
`............. 718/102
`
`

`

`Case 6:20-cv-01001-ADA Document 1-4 Filed 10/28/20 Page 4 of 15
`
`US 7,793,136 B2
`Page 3
`
`1/2005 Smolentzov
`2005/0009506 Al
`3/2005 Upton
`2005/0070221 Al
`4/2005 Lu et al.
`2005/0080543 Al
`6/2005 Chen
`2005/0130656 Al
`7/2005 Anderson
`2005/0153654 Al
`11/2005 Karabinis
`2005/0260984 Al
`12/2005 Himmelstein
`2005/0275505 Al
`2005/0278712 Al* 12/2005 Buskens et al.
`2007/0115868 Al
`5/2007 Chen
`2007/0115897 Al
`5/2007 Chen et al.
`2008/0092140 Al*
`4/2008 Doninger et al. ............ 718/102
`
`............. 717/148
`
`FOREIGN PATENT DOCUMENTS
`
`DE
`DE
`DE
`DE
`EP
`EP
`EP
`JP
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`
`3125161
`4237987
`19922608
`19931161
`0441576
`841648
`1355128
`2000207691
`9624229
`9908436
`9957662
`9965183
`WO 0029948
`0040038
`0130061
`0158110
`
`1/1983
`5/1994
`11/2000
`1/2001
`8/1991
`5/1998
`10/2003
`7/2000
`8/1996
`2/1999
`11/1999
`12/1999
`5/2000
`6/2000
`4/2001
`8/2001
`
`OTHER PUBLICATIONS
`
`Nusser R. et al.: "Bluetooth-based wireless connectivity in an auto(cid:173)
`motive environment" Vehicular Technoloty Conference Fall 2000.
`IEEE VTS Fall VTC2000 52nd Vehicular Techonlogy Conference
`(Cat. No. OOCH37152).
`Martins e fv et al. "design of an OS9 operating system extension for
`a message-passing multiprocesor" Microprocessors
`and
`Microsysetms, IPC Business Press LT. London, BG, vol. 21, No. 9,
`Apr. 1, 1998, pp. 533-543.
`Gutierrez Garcia JJ et al. "Minimizing the effects of jitter in distrib(cid:173)
`uted hard real-time systems" Journal of Systems Architecture,
`Elsevier Science Publishers BV., Amsterdam, NL, vol. 41, No. 6/7.
`Dec. 15, 1996, pp. 431-447.
`International Search Report for PCT/US02/020402; Mailing date
`Apr. 3, 2003.
`International Search Report for PCT/US02/020403; Mailing date
`Jan. 27, 2003.
`International Search Report for PCT/US02/016364; Mailing date
`Feb. 14, 2003.
`International Search Report for PCT/US02/016371; Mailing date
`Aug. 18, 2003.
`A. Das, R. Fierro, V. Kumar, J. Ostrowski, J. Spletzer, and C. Taylor,
`"A Framework for Vision Based Formation Control", IEEE Transac(cid:173)
`tions on Robotics and Automation, vol. XX, No. Y, 2001, pp. 1-13.
`Ada 95 Transition Support-Lessons Learned, Sections 3, 4, and 5,
`CACI, Inc.-Federal, Nov. 15, 1996, 14 pages.
`Boeing News Release, "Boeing Demonstrates JSF Avionics Multi(cid:173)
`Sensor Fusion", Seattle, WA, May 9, 2000, pp. 1-2.
`Boeing Statement, "Chairman and CEO Phil Condit on the JSF
`Decision", Washington, D.C., Oct. 26, 2001, pp. 1-2.
`
`Bluetooth Specification version 1.1: Feb. 22, 2001.
`Counterair: The Cutting Edge, Ch. 2 "The Evolutionary Trajectory
`The Fighter Pilot-Hereto Stay?" AF2025 v3c8-2, Dec. 1996, pp. 1-7.
`Counterair: The Cutting Edge, Ch. 4 "The Virtual Trajectory Air
`Superiority without an "Air" Force?" AF2025 v3c8-4, Dec. 1996, pp.
`1-12.
`Green Hills Software, Inc., "The AdaMULTI 2000 Integrated Devel(cid:173)
`opment Environment," Copyright 2002, 7 pages .
`H. Chung, L. Ojeda, and J. Borenstein, "Sensor Fusion for Mobile
`Robot Dead-reckoning with a Precision-calibrated Fiber Optic Gyro(cid:173)
`scope", 2001 IEEE International Conference on Robotics and Auto(cid:173)
`mation, Seoul, Korea, May 21-26, pp. 1-6.
`Hitachi Automated Highway System (AHS), Automotive Products,
`Hitachi, Ltd., Copyright 1994-2002, 8 pages.
`ISIS Project: Sensor Fusion, Linkoping University Division of Auto(cid:173)
`matic Control and Communication Systems in cooperation with
`SAAB (Dynamics and Aircraft), 18 pages.
`J. Takezaki, N. Ueki, T. Minowa, H. Kondoh, "Support System for
`Safe Driving-A Step Toward Its Autonomous Driving-", Hitachi
`Review, vol. 49, No. 3, 2000, pp. 1-8.
`Joint Strike Fighter Terrain Database, ets-news.com "Simulator
`Solutions" 2002, 3 pages.
`Luttge, Karsten; "E-Charging API: Outsource Charging to a Payment
`Service Provider"; IEEE; 2001 (pp. 216-222).
`M. Chantler, G. Russel, and R. Dunbar, "Probabilistic Sensor Fusion
`for Reliable Workspace Sensing", pp. 1-14.
`MSRC Redacted Proposal, 3.0 Architecture Development, pp. 1-43.
`Powerpoint Presentation by Robert Allen-Boeing Phantom Works
`entitled "Real-Time Embedded Avionics System Security and COTS
`Operating Systems", Open Group Real-Time Forum, Jul. 18, 2001,
`16 pages.
`Product description of Raytheon Electronic Systems (ES), Copyright
`2002, pp. 1-2.
`Product description of Raytheon RT Secure, "Development Environ(cid:173)
`ment", Copyright 2001, pp. 1-2.
`Product description of Raytheon RT Secure, "Embedded Hard Real(cid:173)
`Time Secure Operating System", Copyright 2000, pp. 1-2.
`Product description of Raytheon RT Secure, Copyright 2001, pp. 1-2.
`S.G. Goodridge, "Multimedia Sensor Fusion for Intelligent Camera
`Control and Human-Computer Interaction", Dissertation submitted
`to the Graduate Faculty of North Carolina State University in partial
`fulfillment of the requirements for the degree of Doctor of Philosophy
`in Electrical Engineering, Raleigh, NC, 1997, pp. 1-5.
`TNO FEL Annual Review 1998: Quality works, 16 pages.
`Vehicle Dynamics Lab, University of California, Berkeley, funded by
`BMW, current members: D. Caveney and B. Feldman, "Adaptive
`Cruise Control", 17 pages.
`Specification of the Bluetooth System vl.O.B Dec. 1, 1999.
`Specification of the Bluetooth System vl.l Feb. 22, 2001.
`MyGig.
`Embedded Bluetooth Lisbon-Seattle Jan. 23, 2008.
`AMIC. Architecture specification release 1, 2001.
`Bluetooth hands-free profile 1.5-Nov. 25, 2005.
`Bluetooth advance audio distribution profile specification-May 22,
`2003.
`Bluetooth audio/video remote control profile-May 22, 2003.
`IEEE Standard for
`Information Technology-POSIX Based
`Supercomputing Application Environment Profile; Jun. 14, 1995, 72
`pages.
`
`* cited by examiner
`
`

`

`Case 6:20-cv-01001-ADA Document 1-4 Filed 10/28/20 Page 5 of 15
`
`U.S. Patent
`
`Sep.7,2010
`
`Sheet 1 of 5
`
`US 7,793,136 B2
`
`2
`
`,/
`
`JAVA
`
`-2
`
`. -
`
`. l
`
`JAVA VIRTUAL MACHINE
`
`16
`
`12.
`
`JINI
`
`SECURE REAL TIME EXECUTIVE
`
`14
`
`--10
`
`.
`
`-·-·- .. -·-·-·-·-·-·-·-·-·-·-
`.
`
`FIG 1
`
`

`

`Case 6:20-cv-01001-ADA Document 1-4 Filed 10/28/20 Page 6 of 15
`
`15
`
`/
`
`I -
`
`. -
`
`. -
`
`. -
`
`. L16-. - . - . - . , . - . f._18-. -1
`
`20
`
`22
`
`,·-·-·-"\_·-r-·-·1·-·-,·-·-·-·1·-·-1
`
`24
`
`BRAKE
`CONTROL
`
`SECURITY
`CONTROL
`
`AUDIO
`CONTROL
`
`IR
`SENSOR 1
`
`IR
`SENSOR 2
`
`RADAR
`SENSOR
`
`28
`
`JAVA
`
`JAVA
`
`1Q
`
`JVM
`
`JVM
`
`26
`
`10
`
`14
`
`I
`I
`I
`I
`I
`
`I
`I
`
`I
`
`30
`
`1Q
`
`JAVA
`
`JVM
`
`· -
`
`I
`I
`I
`I
`I
`
`I
`I
`I
`I
`I
`
`32A
`
`1Q
`
`JAVA
`
`JVM
`
`SECURE REAL TIME EXECUTIVE
`
`L
`
`. -
`
`. -
`
`. -
`
`. -
`
`. -
`
`. -
`
`. -
`
`. -
`
`. -
`
`.
`. ~ . -
`
`. -
`
`. -
`
`. -
`
`. -
`
`.
`.
`.
`-·-·-·-·-·-··-·-·-·--~----·-·-·-·-'
`
`32C
`
`10
`
`JAVA
`
`JVM
`
`328
`
`1Q
`
`JAVA
`
`JVM
`
`I
`I
`I
`I
`I
`
`I
`
`I
`I
`I
`I
`I
`
`I
`
`I
`
`I
`
`.
`
`,I
`
`FIG 2
`
`

`

`Case 6:20-cv-01001-ADA Document 1-4 Filed 10/28/20 Page 7 of 15
`
`U.S. Patent
`
`Sep.7,2010
`
`Sheet 3 of 5
`
`US 7,793,136 B2
`
`14
`
`/
`
`50
`
`52
`
`54
`
`56
`
`58
`
`60
`
`MESSAGE
`MANAGER
`
`CRITICAL DATA
`MANAGER
`
`SECURITY
`MANAGER
`
`DATA MANAGER
`
`TASK MANAGER
`
`CONFIGURATION
`MANAGER
`
`FIG 3
`
`

`

`Case 6:20-cv-01001-ADA Document 1-4 Filed 10/28/20 Page 8 of 15
`
`80
`·-·-·-·j·-·-·-
`
`1
`
`82
`. - _\_ . -
`
`. -
`
`. -,
`
`r . -
`
`. -
`
`84
`
`\
`
`15
`~
`
`FIG4
`
`GPSTHREAD
`
`SENSOR FUSION
`THREAD
`
`,--·-·-·-,
`
`I
`I
`
`SENSOR FUSION
`THREAD
`
`66_J
`
`I
`
`I
`
`I
`
`JVM
`
`14
`
`I
`.f.
`.
`.
`I
`'-·-·-·--·-·---·~
`I
`I
`I
`I
`
`72
`
`/
`
`76
`
`I
`I
`j
`·---~~·-·
`I
`.
`._ . - . - . -i . ~ . - . - .1
`. I
`I
`
`I
`
`MESSAGE MANAGER
`
`CRITICAL DATA MANAGER
`
`SECURITY MANAGER
`
`DATA MANAGER
`
`TASK MANAGER
`
`CONFIGURATION MANAGER
`
`74
`
`50
`
`5.2
`
`54
`
`56
`
`58
`
`60
`
`d r.,;_
`
`-....l
`~
`\C w
`"' """' w
`O', = N
`
`

`

`Case 6:20-cv-01001-ADA Document 1-4 Filed 10/28/20 Page 9 of 15
`
`U.S. Patent
`
`Sep.7,2010
`
`Sheet 5 of 5
`
`US 7,793,136 B2
`
`0
`0)
`
`0 ?
`
`1
`
`N
`(D
`
`0
`~
`0::
`:c
`I-
`(/) a.
`
`<:)
`
`co
`
`0) .l.
`-· -·•
`
`(D
`C J ) - - ,
`
`-· -·-
`I
`I
`I
`I
`-· _,_J
`
`~ > "'")
`
`a:
`w
`(9
`<( z
`
`<(
`~
`::,c
`(f)
`<(
`I-
`
`L!)
`(9
`LL
`
`·-
`
`·r "'1"
`
`0)
`
`,-.
`
`'--
`
`~
`(.)
`0
`...J
`(.)
`
`

`

`Case 6:20-cv-01001-ADA Document 1-4 Filed 10/28/20 Page 10 of 15
`
`US 7,793,136 B2
`
`1
`APPLICATION MANAGEMENT SYSTEM
`WITH CONFIGURABLE SOFTWARE
`APPLICATIONS
`
`This application is a continuation of U.S. Pat. No. 7,178,
`049, filed Apr. 24, 2002 entitled: METHOD FOR MULTI(cid:173)
`TASKING MULTIPLE JAVA VIRTUAL MACHINES IN A
`SECURE ENVIRONMENT.
`This application incorporates by reference U.S. Pat. No.
`6,629,033, filed Apr. 24, 2001 entitled: OPEN COMMUNI(cid:173)
`CATION SYSTEM FOR REAL-TIME MULTIPROCES(cid:173)
`SOR APPLICATIONS and U.S. Pat. No. 7,146,260, filed
`Apr. 24, 2001 entitled: METHOD AND APPARATUS FOR
`DYNAMIC CONFIGURATION OF MULTIPROCESSOR
`SYSTEM.
`
`BACKGROUND OF THE INVENTION
`
`2
`The foregoing and other objects, features and advantages
`of the invention will become more readily apparent from the
`following detailed description of a preferred embodiment of
`the invention which proceeds with reference to the accompa-
`5 nying drawings.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a diagram showing ajava stack with an additional
`10 Secure Real-time Executive (SRE) layer.
`FIG. 2 is a diagram of a multiprocessor system that runs
`multiple Java Virtual Machines that each include a SRE.
`FIG. 3 is a detailed diagram of the managers in the SRE.
`FIG. 4 is a block diagram of how the SRE manages a
`15 multiprocessor system.
`FIG. 5 is a bock diagram showing how a task manager in
`the SRE operates the multiprocessor system in a lock-step
`mode.
`
`Java is a robust, object-oriented programming language
`expressly designed for use in the distributed environment of 20
`the Internet. Java can be used to create complete applications
`that may run on a single computer or be distributed among
`servers and clients in a network. A source program in Java is
`compiled into byte code, which can be run anywhere in a
`network on a server or client that has a Java virtual machine 25
`(NM).
`A NM describes software that is nothing more than an
`interface between the compiled byte code and the micropro(cid:173)
`cessor or hardware platform that actually performs the pro(cid:173)
`gram's instructions. Thus, the JVM makes it possible for Java
`application programs to be built that can run on any platform
`without having to be rewritten or recompiled by the program(cid:173)
`mer for each separate platform.
`Jini is a distributed system based on the idea of federating 35
`groups of users and the resources required by those users.
`Resources can be implemented either as hardware devices,
`software programs, or a combination of the two. The Jini
`system extends the Java application environment from a
`single virtual machine to a network of machines. The Java
`application environment provides a good computing platform
`for distributed computing because both code and data can
`move from machine to machine. The Jini infrastructure pro(cid:173)
`vides mechanisms for devices, services, and users to join and
`detach from a network. Jini systems are more dynamic than is
`currently possible in networked groups where configuring a
`network is a centralized function done by hand.
`However, the Java/Jini approach is not without its disad(cid:173)
`vantages. Both Java and Jini are free, open source applica(cid:173)
`tions. The Java application environment is not designed for
`controlling messaging between different machines. For
`example, the Java application is not concerned about the
`protocols between different hardware platforms. Jini has
`some built-in security that allows code to be downloaded and
`run from different machines in confidence. However, this 55
`limited security is insufficient for environments where it is
`necessary to further restrict code sharing or operation sharing
`among selected devices in a secure embedded system.
`
`SUMMARY OF THE INVENTION
`
`The present invention allows construction of a secure, real(cid:173)
`time operating system from a portable language such as Java
`that appears to be a Java virtual machine from a top perspec(cid:173)
`tive but provides a secure operating system from a bottom
`perspective. This allows portable languages, such as Java, to
`be used for secure embedded multiprocessor environments.
`
`DETAILED DESCRIPTION
`
`A java application stack includes a Java layer 5 for ruillllng
`any one of multiple different applications. In one example,
`the applications are related to different vehicle operations
`such as Infrared (IR) and radar sensor control and monitoring,
`vehicle brake control, vehicle audio and video control, envi(cid:173)
`ronmental control, driver assistance control, etc. A Java Vir-
`tual Machine (JVM) layer 16 provides the hardware indepen(cid:173)
`dent platform for running the Java applications 5. A Jini layer
`30 12 provides some limited security for the Java applications
`that run on different machines. However, the Jini layer 12
`does not provide the necessary reconfiguration and security
`management necessary for a distributed real-time multipro-
`cessor system.
`A Secure Real-time Executive (SRE) 14 provides an exten-
`sion to the NM 16 and allows Java to run on different pro(cid:173)
`cessors for real-time applications. The SRE 20 manages mes(cid:173)
`saging, security, critical data, file I/0 multiprocessor task
`control and watchdog tasks in the Java environment as
`40 described below. The NM 16, Jini 12 and SRE 14 can all be
`implemented in the same NM 10. However, for explanation
`purposes, the JVM 10 and the SRE 14 will be shown as
`separate elements.
`FIG. 2 shows a system 15 that includes multiple processors
`45 16, 18, 20, 22 and 24. Each processor includes one or more
`JVMs 10 that run different Java applications. For example,
`processor 16 includes one Java application 28 that controls a
`vehicle security system and another Java application 26 that
`controls the vehicles antilock brakes. A processor 18 includes
`50 a Java application 30 that controls audio sources in the
`vehicle. Other processors 20 and 22 may run different threads
`32A and 328 for the same sensor fusion Java application 32
`that monitors different IR sensors. Another thread 32C on
`processor 24 monitors a radar sensor for the sensor fusion
`Java application 32.
`The SRE 14 runs below the NMs 10 in each processor and
`control tasks, messaging, security, etc. For example, the Java
`application 26 controls vehicle braking according to the sen(cid:173)
`sor data collected by the sensor fusion Java application 32.
`60 The SRE 14 in one example prevents unauthorized data from
`being loaded into the processor 16 that runs brake control
`application 26. The SRE 14 also prevents other Java applica(cid:173)
`tions that are allowed to be loaded into processor 16 from
`disrupting critical braking operations, or taking priority over
`65 the braking operations, performed by Java application 26.
`For example, the SRE 14 may prevent noncritical vehicle
`applications, such as audio control, from being loaded onto
`
`

`

`Case 6:20-cv-01001-ADA Document 1-4 Filed 10/28/20 Page 11 of 15
`
`US 7,793,136 B2
`
`5
`
`10
`
`50
`
`3
`processor 16. In another example, noncritical operations,
`such as security control application 28, are allowed to be
`loaded onto processor 16. However, the SRE 14 assigns the
`security messages low priority values that will only be pro(cid:173)
`cessed when there are no braking tasks in application 26 that
`require processing by processor 16.
`The SRE 14 allows any variety ofreal-time, mission criti(cid:173)
`cal, nonreal-time andnonmission critical Java applications to
`be loaded onto the multiprocessor system 15. The SRE 14
`then automatically manages the different types of applica(cid:173)
`tions and messages to ensure that the critical vehicle applica(cid:173)
`tions are not corrupted and processed with the necessary
`priority. The SRE 14 is secure software that cannot be
`manipulated by other Java applications.
`The SRE 14 provides priority preemption on a message
`scale across the entire system 15 and priority preemption on
`a task scale across the entire system 15. So the SRE 14
`controls how the NMs 10 talk to each other and controls how
`the NMs 10 are started or initiated to perform tasks. The SRE
`14 allows programmers to write applications using Java in a 20
`safe and secure real time environment. Thus, viruses can be
`prevented by SRE 14 from infiltrating the system 15.
`While the explanation uses Java as one example of a pro(cid:173)
`gramming environment where SRE 14 can be implemented, it
`should be understood that the SRE 14 can be integrated into
`any variety of different programming environments that may
`run in the same or different systems 15. For example, SRE 14
`can be integrated into an Application Programmers Interface
`(API) for use with any programming language such as C++.
`FIG. 3 shows the different functions that are performed by
`the SRE 20. Any combination of the functions described
`below can be provided in the SRE 20. A message manager 50
`controls the order messages are received and transmitted by
`the different Java applications. A security manager 52 con(cid:173)
`trols what data and messages are allowed to be received or
`transmitted by different Java applications. A critical data
`manager 54 controls what data is archived by the different
`Java applications.
`A data manager 56 controls what data is allowed to be 40
`transferred between different processors. A task manager 58
`controls the order tasks are performed by the different NMs.
`A reconfiguration manager 60 monitors the operation of the
`different processors in the system and reassigns or reconfig(cid:173)
`ures Java applications and Java threads to different processors 45
`according to what processors have failed or what new proces(cid:173)
`sors and applications have been configured into system 15.
`The message manager 50 partially corresponds to the pri(cid:173)
`ority manager 44 shown in FIG. 2 of pending patent applica(cid:173)
`tion Ser. No. 09/841,753, the critical data manager 52 par(cid:173)
`tially corresponds with the logging manager 44 shown in FIG.
`2 of the copending '753 patent application, and the security
`manger 54 a least partially corresponds with the security
`manager 40 shown in the '753 patent application. The data
`manager 56 at least partially corresponds with the data man- 55
`ager 42 shown in FIG. 2 of pending patent application Ser.
`No. 09/841, 915, the task manager 58 partially corresponds to
`thedevicemanger46 shown in FIG. 2 of the '915 application,
`and the configuration manager 60 at least partially corre(cid:173)
`sponds to the configuration manager 44 shown in FIG. 2 of the 60
`'915 patent application. The descriptions of how the different
`managers 50-60 operate similarly to the corresponding man(cid:173)
`agers in the '753 and '915 patent applications are herein
`incorporated by reference and are therefore not described in
`further detail.
`However, some specific tasks performed by the managers
`50-60 are described below in further detail.
`
`4
`FIG. 4 shows in more detail how the SRE 14 operates. One
`of the operations performed by the task manager 58 is to
`control when different tasks are initiated on different proces(cid:173)
`sors. For example, a first Global Positioning System (GPS)
`thread 62 is running on a NM in a processor 80. Another
`sensor fusion thread 64 is running on a different processor 82.
`Block 74 represents the Java Virtual Machine operating in
`each of processors 80 and 82. A master JVM 74 may run on
`either processor 80, processor 82 or on some other processor.
`The task manager 58 sends an initiation command 66 to the
`GPS thread 62 to obtain location data. The task manager 58
`then directs the obtained GPS data 68 through a link to the
`sensor fusion thread 64 for subsequent processing of GPS
`15 data 68. The link maybe any bus, such as a PCibus, serial link
`such as a Universal Serial Bus, a wireless link such as blue
`tooth or IEEE 802 .11, or a network link such as Ethernet, etc.
`The configuration manager 60 acts as a watchdog to make
`sure that the GPS thread 62 and the sensor fusion thread 64 are
`each running correctly. In one example, separate configura(cid:173)
`tion managers 60 in each processor 80 and 82 sends out
`periodic signals to the other configuration managers 60 in the
`other processors. Any one of the configuration managers 60
`can detect a processor or application failure by not receiving
`25 the periodic "ok" signals from any one of the other processors
`for some period of time. If a failure is detected, then a par(cid:173)
`ticular master configuration manager 60 in one of the proces(cid:173)
`sors determines where the task in the failed processor is going
`to be reloaded. If the master configuration manager 60 dies,
`30 then some conventional priority scheme, such as round robin,
`is used to select another configuration master.
`If a failure is detected, say in the processor 82 that is
`currently performing the sensor fusion thread 64, a message is
`sent from the configuration manager 60 notifying the task
`35 manager 58 which processor is reassigned the sensor fusion
`thread. In this example, another sensor fusion thread 76 in
`processor 84 is configured by the configuration manager 60.
`The critical data manager 52 manages the retention of any
`critical data 72 that was previously generated by the sensor
`fusion thread 64. For example, the critical data manager 54
`automatically stores certain data and state information that
`was currently being used in the sensor fusion thread 64. The
`critical data may include GPS readings for the last 10 min-
`utes, sensor data obtained from sensors in other processors in
`the vehicle over the last 10 minutes. The critical data may also
`include any processed data generated by the sensor fusion
`thread 64 that identifies any critical vehicle conditions.
`The critical data manager 52 also determines which data to
`archive generally for vehicle maintenance and accident
`reconstruction purposes.
`The configuration manager 60 directs the critical data 72 to
`the new sensor fusion thread 76. The task manager 74 then
`redirects any new GPS data obtained by the GPS thread 78 to
`the new sensor fusion thread 76 and controls sensor fusion
`tasks from application 76. Thus, the configuration manager
`60 and the task manager 58 dynamically control how different
`Java threads are initialized, distributed and activated on dif(cid:173)
`ferent processors.
`The message manager 50 determines the priority of sent
`and received messages. If the data transmitted and received
`by the sensor fusion thread 76 is higher priority than other
`data transmitted and received on the processor 84, then the
`sensor fusion data will be given priority over the other data.
`65 The task manager 58 controls the priority that the sensor
`fusion thread 76 is giving by processor 84. If the sensor fusion
`thread 76 has higher priority than, for example, an audio
`
`

`

`Case 6:20-cv-01001-ADA Document 1-4 Filed 10/28/20 Page 12 of 15
`
`US 7,793,136 B2
`
`5
`application that is also being run by processor 84, then the
`sensor fusion thread 76 will be performed before the audio
`application.
`The SRE 14 can be implemented in any system that needs
`to be operated in a secure environment. For example, network 5
`servers or multiprocessors operating in a home environment.
`The multiprocessors in home appliances, such as washer and
`dryers, home computers, home security systems, home heat(cid:173)
`ing systems, can be networked together and operate Java
`applications. The SRE 14 prevents these multiple processors 10
`and the software that controls these processors from being
`corrupted by unauthorized software and also allows the appli(cid:173)
`cations on these different processors to operate as one inte(cid:173)
`grated system.
`The SRE 14 is a controlled trusted computing based that is 15
`not accessible by non-authorized application programmers
`and anyone in the general public. Therefore, the SRL 14
`prevents hacking or unauthorized control and access to the
`processors in the vehicle.
`
`TASK CONTROLLED APPLICATIONS
`
`6
`when and how often activation commands 94 are sent to GPS
`thread 62. In a similar manner, the task manager 58 can
`control when other tasks are performed by the system 89,
`such as when the sens