Chemical vapor detection with a multispectral
`thermal imager
`
`E
`
`Milt L G. Althous., MEMBER SPl£
`U.S. Army Chemical Research
`Development and Engineering Center
`SMCCR..()DT
`Aberdeen Proving Ground. Maryland
`21010
`
`Cheln-I Chang, MEMBER S~E
`University of Maryland
`Department of Electrical Engineering
`8altimo, • Maryland 21228
`
`Abstract. Detection of chemical vapOrs with I remote 5ensor is necessary
`for both military defense and civilisn pollution control. The therm,l imager
`is a nBtur,1 instrument from which to build I chemlcallensor since most
`chemical vapors of interest are spectr,lIy active in its operating wave(cid:173)
`length range. A system has been designed to place a chemical detection
`capability 18 en adjunct fundlon in 8milllary thermal imager. An additional
`detector array, which is s~rally filtered at the focal ptane. is added to
`the imager. Real-time autonomous detection and alarm i8 also required.
`A detection system model by Warren. based on 8 Gaussian vlpor con·
`centratio" distributton is the basis for detection elgorilhms. Algorithms
`recursive in both time and spectrel frequency have been derived using
`Kalman filter th80rv. Adaptive filtering il used for preprocessing clutter
`rejection. VeriouscOmponents of the detection system have been tested
`individually and In integrated system 1& now being fabricated.
`
`Sutljea 'firms: 'nfflred Im.glng ryst,ms; muJr1,pet:lr.1 rhemtallm.g;nQ: chemical
`dtJrectlon; "d.prive filretS; K./man filtering; ".PO' cloud m"de/.
`Optic,' Eng;ftHf;ng 30(" J. , 725..1733 (November' 99IJ
`
`CONTENTS
`1. IntrOduction
`1. Sensor CQn~ep(
`3.
`tfanjWlllt description
`4. Syscem model
`5. PnsProcasinl algorithms: backSJOUOd clutter suppression
`5. t. Uoconslnined MMSE adaptive filtering
`S.l. Linearly consuain«ld MMSE. adaptive Ultering
`5.3. Systolic arrIY I1goriUuns
`6. ~ 4erection alloritluo
`7. C Delusions
`B. Acblo'MledpnlS
`9. Rcfetellt'e5
`
`1. INTRODUrnON
`Dettdioo of chemical vapor douds has been a military coneem
`since the first u~ of chemical warfare. Orowing concern for
`coviroamcntal poUutioD has additionally driven development of
`~mote chemical ~apor detectors. I The R..IR (forward looking
`lnfrvcd) is an 8· to 12·f1m IhcnnaJ imager widely used in the
`military for !itgtt acquisirioD and night vision. Chemical vapors
`ate spectrally active in the 8- to 12-p.1D region. There have been
`5Cved1 militarY programs to idapt the FUR for chemical de(cid:173)
`rect.ioD.z-' Aaavy fUR has becD deployed for chemical sensing."
`II is I standard COIllDlOD module fUR modified with bandpass
`spc:c1l'8l ftlters. Agc.ot detection is made by the operator based
`OD the vicwed scenc.
`Army ftqlairemenu specify aD autonomous detection syst£m
`not dependeDt 00 an operator for an a1anD decision. To meet
`
`that Rqu..iremeol. the Chemical Research, Development, and
`Engineeriag Center (CRDEC), as pan of its detection program.
`is dC'ltloping automatic: image enhancement and derectiOQ aJ(cid:173)
`goriduns. A large body of work existS. on target detection and
`rtCognition in thcnnaJ images. But the targets of inteRst in this
`wort U! lIDks, tnlCks. a.i1mft. and buildings. aJ) objects thal
`are gcneraUy modeled by such deterministic MJctions IS 5bape
`and sw,. Vapor c:louds fit 0.0 fixed size, shape, or motioo eat·
`egories, but ratb« are modeled statistically based 00 their COD(cid:173)
`ccntration. The detection problem becoow ooe of deciding on
`the composite hypothesis of the image containing one or mo~
`distributions of various chemical vapors venus the hypothesIS
`of Ihe image containing ollly the dUtt!r bacqround. AdditiOll+
`any, any detection algorithm must operate in real time, thal
`being the 1/3()..s rime period between me arrival of frames of
`video. To meet \his constJlinl, adaptive and recunive ruJiza·
`tions are punued, as are parallel and pipeUned implemencaUous.
`Sincc lIlom sensitive remote detec:tion mc1hods elilt for a
`dcdieawt chemical seDsor, specifically IR interferometry' and
`laser spec:tJ'OSCopy,1 the 5U'eDgths ~f the FUR-based approach
`are the pictorial output and the large num.bers of staodan:l FURs
`in use in the militaly. Because mosl-taeUcal vehicles and ain:raft
`lie fitted with III FUR, the space- and COSl«ooonUcal way to
`provide them with I remote cbemical vaporseasing capability
`would be through iaclusioD of that capability in the FLlR. As
`the oexlgeneration of FURs is developed, ad adjuoc;t chemical
`vapor deteetioo apabilll)' could be ~rporalCd as III Opti~DaJ
`cou&guratioo.
`
`1. SENSOR CONCEPt'
`Turning an FUR iJ1to a speclrDmew usiPg Il8I1'Ow-bandpass
`filten'is a fairly slmple process. Placing a fi.lter ia l'root of the
`
`Of'nCAL ENGtNEERlNG I NCMmber 199' I VfA. 30 No.1' I
`
`'725
`
`FLIR Systems, Inc.
`Exhibit 1013-00001
`
`

`

`ALniOUSE. CHANG
`
`~t apeature of lhe fUR. much as ODe docs with a 61cu 00 a
`camera. is Ihe iaitiaJ soludoD. To gel more than one spectral
`band.rutuh'u the installation ofa iiilei' wheel or some mechaDicaJ
`device ro cbaDge fillerS. Ofcount when rile upensc: of IR lilrel1
`several &:eDUmcten in diameter becomes appmDt. a qUick move
`is madr. to aD iotemal focal point in tbe· AJR. This was the
`initial approach' at CRDEC and it performed weUfor proof of
`principle and initial ttiaJs. The stepper-D10IOf-Q)DO'OUed filter
`wheel could reliably switch filIUS every second. at its fastest.
`Thus an eatire sct of four bands could be ooUcete:d ill aboUlIO
`s. Such a coUcction rate wouJd be adequate for a fixed detector,
`i.e., nOM1obile. To ~ lbt daIa coUection rate the next
`iteration chemical 5eDiLng FlIR. has a coatiouously spinning
`filter wheel.8.9 This wheel bas three different passbands and is
`synclvoo.ized to the video frame rate or the FUR. thus yielding
`a full sct of spectral dam every tint frames or ./to s. A 0.1-5
`data collectioo time would be adequate in slowly moviDg ve·
`hides. But this system still JtqU~5 a fuUy dedicaltd FUR and
`would experience tOO much registration enor for use in ainnft.
`Dividing the 8- to 12-l1m region into smaller region.s iD an
`FUR. can ~accomplisbcd oDJy with fill£rs or a dispersion optit.
`both of whicb bavetnnsmissioD losses. lmagiPg spectrometers
`based on other principles exist but do not have the spatial res(cid:173)
`olution, compact sizto or bigh-dara-acquisitioo mte of the RlR.. 10
`In the cue of fihm. the passband may be 15 Darrow as 0.5 11m
`and still yield aD 80" peak tnnsmission. Filters smaller ill pass(cid:173)
`band quickly degrade in peak fJUSmi5sioD. Although Ibe filter
`decreases the total energy incident on lite detector. thus lowering
`ovetaU sensitiviry, it has the advlUitage that an absorption fearure
`located widUn the filter passbmd Comprises a much larger pet.
`cen.tage oC the energy incident on the detector. This yields I
`better Signal.to-DOise rario for the Object with that absorption
`reanue. FiguR 1Ulustrates (WO filter pusbaDds and the spcc:trum
`of SF6. whicb bas a 5pecttal feature in che lO.6-...m lilta' pass..
`bud.
`There lie two difficulties with using standard bandpass filters.
`To divide the 8.. to 12-tuD band fully infO O.S-ta-:m-wide segmCDts
`would req.m eight individual filem. These Deed 10 be me(cid:173)
`cbaDicalJy rotated mro the field of view sequentially to obtain
`spectral data over the full rIDge. TbeR could alremative~ be
`eight to tell individually &Jwed defector amys, eacb Wlth a
`diffnnlpusbaDd. IIld some mcdlod for SClnDiDg the field of
`vI~. over each delcdor amy. Either system requires ID optical
`~ip of the FUR. A secoDd difkulty ls that &he detector
`DOW vic", a "boa" (system opcratiIlg tanpera~) filter clement
`
`chat is opaque over much of the sensitivity range of the dclt'Ctor.
`CM-of-passband emissiOD from the filter represencs a ooise source_
`This may Dot be • problem if the scene backrround is signifi(cid:173)
`cantly warmer thaD the filter. but If a colder stene is viewed.
`the wanner &Iter causes a tOosiden.ble loss or sensitivity. An(cid:173)
`other concern specific to latucal miliwy FUR.s is the n:quirc(cid:173)
`tnalt for excellent spatial resoludon and good sensitivity for
`target acquisition and Rtognition. The image quality aod OJ)(cid:173)
`eratioaal availability of the taetitaJ sensor cannot be compro(cid:173)
`mised to lOy way by th! additioD of further missioa ~uirc.
`m~ts. such u chemical detection or hardware. sucb as filleB.
`To ~ss aU of the necessary elemenu for adjunct chemical
`deaection in a harmonious way leaves Uttle space for desig.n
`alternatives but does accommodate a solution.
`
`3. HARDWAIlE DESCIUFI10N
`There is an arrangement thal would penNt the modified FUR
`to deliver both 115 standard image and a filtered image simul(cid:173)
`Wleously, which we call CSFLIR for chemic.al sensing A..IR.
`The standard image would not be delJ1ded in any way. Tht
`ftlrmd image would be designed (0 have me highest seR5ibvit~·
`obtainable for the! Biven dettetor array and filter bands. Usc of
`bolb the fille~ lIIId unfiltered images allows sufficient spectraJ
`ttwacterization of the viewed scene to detect and classify chem(cid:173)
`ical vapor clouds.
`F"lJUfe 2 illustrates the focal plane fillering concepl fOf a
`tactical FLIR. There are two detector amys, the standard taetical
`one and I ftJten:d one for chemical detection. The tactical amy
`is a standard common module detector amy. which is a linear
`amy of semiconductor detector el!mcnts. genenilly I x 180.
`1)( 120. or I X60. The semiconductor material is HgCdTe for
`8- to 12-~ detecton. Individual element sizes are on the order
`of 4 x 10-1 by 6 x 10- 2 nun. The signals produced OD these
`detector elemeou. as a scene is scanned over them. are elcc(cid:173)
`IIOnically fonned into a video image. Dual detector implemen(cid:173)
`tation in the common module Fl.JR would requin a redesign of
`the Dewar assembly to accommodate additional electrooic con(cid:173)
`aecti.oDS.
`The chemical sensing detector Bmy would be identical in
`coDStrUCbon techniques and material to the COIlUDOD module
`(eM) amy. It could bave elements the same siu as or larger
`IbaD the eM amy. A larger detector clement could be. used
`because it is more sensitive (less noisy) chan the sma1J ooe. The
`tra.de"Off is a loss of spatial resolution iD the sceDe. This trade(cid:173)
`off bas been found Dot to be a great concern in vapor cloud
`seosing as vapor clouds tend to be large relative to the sce"e
`
`0.05
`
`0.00
`
`i -0.05
`SI ..0.10
`
`i
`
`-0."
`
`8DO
`
`'000
`WA\'EH\IU'P'G IC"-'»
`At. 1. sa"'''''nlIC*tfVIft wtdl ,..taend of ........,. .nd ott......
`tuNtpKtnI .....
`
`'ZOO
`
`,I
`
`~i
`
`'"I
`
`FLIR Systems, Inc.
`Exhibit 1013-00002
`
`

`

`CHEMICAL VAPOR OETECTlON YJ'In. AMULT\SPECTM.l THERMAL IMAGER
`
`,
`~ ~,
`
`,
`II
`J
`\:
`
`•
`
`\1
`
`.
`
`., wrNDOw MATERIAL
`c. 8ANOPASS FILTER PATCH'
`d. COMMON MOOUlE DmcroR El.EMENT
`e. OETECtOR ASSfMBL't S08STftATE
`f. ACCIPTANCE ANGLl
`Ag. 4. Side YIew '" • foul plln, Rtt. aamn, I common module
`detector Imy; four detector ,I,mlntl per fitt. pItI:tL
`
`fit. I. Effect an filter codn, laVIII . . to • "".n P1rtieut.,. lm(cid:173)
`
`purity.
`
`eo·
`
`;i
`
`1&1z
`8 30'
`i
`
`. IJId have iDd1stioc1 edges. High-spatial ftlSOlutioo is necessary
`to resDlve sma1J objcclS or sharp edges in a sceuc.
`A focal Diane filter is moun1ed aboH.Jhu.bemical detector
`anay \ViWD me c:001c:Q ClCreaor nouJJALJtconsiJrs of aWindow
`"'_im"~'DlpJIJQ"iLDSJIlWlve mUle II· 10 12-aim-bud, coated
`.-lib paU:he.s of DaItOW bandpass Olter coalings. '1'be baDdpaa
`filrer1 ~ made by vapor depositing mUltiple laym of l/4-wavc(cid:173)
`lenp thickness dielectric material on the window 5ubstralC. A
`wide vari«y of malerials U'C used and the deposition process is
`~1I. UDderstood. Adjaceot fiJler patches have different pass(cid:173)
`bands. The Dumber of distinct passbands required depeuds on
`dJe complclil)' and Ilumber of chemical vapoB being deteCt£d.
`A passbaDd should ~ WOCiltedwith each strong absorption
`ft.&lUJ1l and at least DOC band where lhete art 110 feanues for
`ba5cliDe or background refeteDCt. The filter patebes would be
`arraDgcd iD ordtt from I to II. repeating the order tiJllhc wbole
`del£Ctor amy is covered. aa shown in Fig. 3. The physical depth
`of some of the passband coatings is on the order of the CM amy
`dement dimellsioos. As sucb it may oot be poIsible to put a
`filter patcb over eacb element, but Ihc clements would be &JOUPCd;
`sAy four elelMuu under apatch, one element lost in the nnsition
`resiOD from one patch ro the out, then four mOte, and 50 on,
`asiD Fig. 4. Any Ilumber of d~tector elements couJd be placed
`Wldet the filler such chat the filter coating edge should Dot iD(cid:173)
`tufeR with the deteCtOr elemeDt acceptance angle. The smalleSt
`poupinC' and bence the largcsI number of filter strips is dcsinlble
`to get the most complete mlJltispcctnl coverage on the image.
`A custom array with largu detectors c:ould be matched to the
`design coustraints of the filter patches, as in Fig. 3. The filter
`pateh should be at least three times as wide u it is deep for
`saueturaJ stability. Atilter patch edge with no slope is preferable
`as the transition wililheo occupy less 5plte on the i~ge plaDc.
`Each filrer: clemenl cannot have aD edse of the filter patclt within
`irs KUptaDte angle whhout experienc:iDg serious focusing and
`spectral errors. The tilm should be mounted far enough from
`the ddcctor to avoid largc noise contributions due to forward
`scattaing from small defects, which generally exist i.D che filter
`patebes. This scatrmng is called the SU.rwalt effect. I 1-11
`SiJIcc the lilteB are fabricated from l'4-w8velengtb thkkness
`laym sequentially applied. any foreign material that adhera to
`the filter during fabrication will result in a fearun: as shown in
`FlI. 5. 11le bumP mated by the defect acts IS a leas to focus
`stray r1dWioo onto the detector.
`Commatial fnfrved ftlters geoerally contain some of these
`cttfects. Because the tilten are not generally mounted close to
`the detector. Che defects bave no effect. Through can:f\11 pro-
`
`- - - e. WINDOW MATERIAL
`
`Ag. 1 Top ¥Ww at I focal .,I.RI fillef covering I eLIltom d-,ec\o,
`emy; OM tIttM pMch PI' _ecta, ..,,,,em.
`
`10-3
`10-.
`TRANSMmANCE
`Ag.l. Off pellblnd trln""taInce from I spectral 111111 U • function
`
`10-1
`
`of the half...ne an". of the ~or.
`
`10-5
`
`10-1
`
`teSSiD.g, we tan g:ready reduce the lumber of d1esc defects and
`thus eliminate the consequent DOise of the StlCl\IIalt effect. fif
`UJe 6 sbows some data frol,D· Stierwalt dial illustRte lhe effecl. I
`The curve sbows au.smlnance of an out-of-panband frequency
`fOT various baIf~one ~epWlce angles of the detector. As we
`move a filter with fied apert\IR closer to tbe detector. the half(cid:173)
`cone angle inCJUStS. to a limit of 90 dea. MI'R 7 illustrates
`the change in half-cone ugle with me change in ofliet distance
`of the filler from the detector· for 8 tiled ape~_ Defects in
`the filter coating Dot only scatter tbt object beaJn away from the
`detector. thus reducinS dcsm:d signal st1'tIJg1h, but saner and
`focus stray radiation from outside &be normal optical path and
`outo(1f-band radi~tioD ooto the detector, thus lAcreasillg the noise
`l~"el. The larger the aceepWlcc angle of cbe deleCtOr the mon:
`pronounced the effed.
`Some recent measurements with bip-quality filten indlcaced
`no Stierwalt effect up 10 a ba1f~DC angle of 20 deB. lDitial
`designs for a focaJ-plane-filltred FUR include haIf-cooe aDgles
`greaLCr ItWJ 20 deg. As a result. pal ~ is being taken 10
`ensuxe I clean aanosphere and pure marmials for the ongoing
`iDitiai fabricatiOD of these filleR.
`
`OPTlCAL ENGINEERING I NOvember 1991 I Vol. 3) No.1 1 I 1127
`
`FLIR Systems, Inc.
`Exhibit 1013-00003
`
`

`

`AlTliOUSE. CHANG
`
`"peed of dall KquisitiOD, MOlt image-processiDg techniques
`eppfic:able CO cbe cloud detection problem require good image
`stability aod registtalioo over the sequence of lmagC$ used for
`analysis. Tactical FlIRJ rend to be mounted on moving vehicles.
`resulting in DXJtiOD iD the scene between frames. Corrcctiag lhc
`$ensorlDOtion is pOssible but computationally costI)'.1n the new
`desip both filaertd and unfilttral image! arc coUected simul.
`taDeOUslyll rbf! system frame rate. gtfteral.Jy 30 Hz. The aet\taJ
`tiltered p~el to corresponding unfiltered pixel COUectiOD lime
`difference will be on the order of 10- S 5 due to the pbysical
`separation of the deteCtor ana)'s on the focal plane. For systeau
`witba stePped filla wheel or continuouslY variable filter thal
`saJ:neplXel couecnObDJlleailTerencc is on Uie order or seconds.
`III the case of the condouously spinning filter wheel, evr:nlhis
`IJ10st ideal mechanically scanned system will have a pi~el col(cid:173)
`lection lime difference of at least 1130 I. A 10-' 5 time sepa·
`ration will keep uy scene bluniag at subpixel ~solution even
`lrthc FUR is mounted ill a high.speed airtraft. Thus this system
`is compatible wid! prtSeri.t dcttttion algorithms. The only im(cid:173)
`provement in the amount and usefulness of information coU!C(ed
`would· come from the addition of mtR detector arra),s, each
`fil~ at a single color, providing liimultaneous multispectral
`data for each pixel.
`The block diagram in Pig. 9 shows the major system com(cid:173)
`poDCDU. The FUR optics and tactical detector ~ as designed
`for a stal1dard CM FUR application. The CSFUR detector aM
`tactical dcteaor are iotepated into I detector/cooler wembly
`Ibat includes a cryogenic pump. A data line ftom each deteCtor
`caniea the signal to video electronics modulesthac transfer it {o
`RS-l10 format. Both RS·170 liJK.s feed a digital image protes(cid:173)
`5«. The image. processor curies OUI the image enhancement
`routiocs callediD the detection aJgori~, Prior to Ib actUaJ
`decision on d1e presence of a c:loud, the detection algoriUun
`co.bOls the proceasinJ and extraction of relevant iDIonnation
`contained ill the available data.
`Status of the chemical vapor detection OpentlOD is presented
`to (be system operator ~a aD audio or visual means. The operator
`could have manual aeJection of aD appropriate display mode in
`bia image display sa=1l. It may be a colored RI, shaded area.
`or pcrimtter overlay of the cletctttd vapor cloud..
`
`FU"
`OPT1CS
`
`~04ENIC
`COOLS
`
`I
`t
`OFFSET DISTANCE (A£LAUVE UHrTSl
`
`RI.1, aalngeln hatr-eane 1",11 with varytng affWI d1ttlnGlt 'ar
`• bed aperture 50'% 1a'01f titan Itt. detector ••. Unitt ar. nor·
`matlzed to the difft.Mioft at "'edetlC'tGf,
`
`The image resulliDl from the described chemical sensing
`detector would appear as ill Fig. 8. Horiloatal stripes of the
`image would be lilrercd by different pa5sbands in 8 repeating
`order venicaJly dowa the ima,e. Abuilding or solid feature that
`bas a flat emission spectnUn would .how ooJy minor ,hinges
`ill appearanteftom stripe to stripe due to slightly unequal band·
`pass functions. On Ibe other hand, the vapor cloud, which bas
`distinct spectral propettles, is suongly visible chrough filter I,
`weakly visible tluwgb lilter 3, and not visible through filters 2
`aCld 4. To an openlOr this would be:l very poor quality image,
`but to U1 image protC$5OI'. ics aoalysis poses flO problem. AJ·
`though cht system docs oot produce a tOmplete set of specttal
`data for each pixel, it does provide sufficieot spectral analysis
`of the scene. R$·170 standard video. is composed of 480 hori·
`mnral lines. Using Ihe assumption of four Wtl:ted elements, the
`aext element lost to iller pllCb transition. foUt mo~ fill~
`elemedtS, and SO on, we would get 96 stripes ill the image. The
`Dext seneration of ractical FURs will bave 48G-line.image dala,
`Given the ISO-line CM display, we would ICI36 stripes. With
`four distinct pusbends, 1bat would yie1dsix complete filter st·
`qutncres. Based OD t£515 wUb vaporclouds gencrared·io arcali5Lic
`manner,lhe cloud quickly fills a sipiliclnc portion of the image
`if it is closcenough to be det.ectI:d. The fURl used i.a the stud)'
`bad kids of view (FOV) on dac order of 20 x 30 deS. Many
`tKticaJ fI.JJU have a mucb smaller fOV and thus will have the
`cloud fill • larse partioo of the FOV.
`Aoodaer MvaDtage of."Jhis ,)'Steal over Cbe use of dJscme
`IUICn .in a W. wbec1 or a cootJAuously .variable Ilter is lis
`
`.
`
`-, --
`
`......... e.
`
`l
`
`".
`
`7
`
`'.
`
`i
`
`;
`
`,J.'Y
`
`""""
`~I '-= ~
`"-
`.....
`,
`
`~
`
`I
`
`1,
`At
`~
`
`~ .
`
`\,
`AI
`~
`.\c
`-',
`la
`-'I
`
`~ I
`
`t
`~l
`-'3
`A;t
`1,
`
`~
`
`, '128 I 0P1lCAL. 8tGIN&RING I ~rt., I y~. JO No. 1t
`
`FLIR Systems, Inc.
`Exhibit 1013-00004
`
`

`

`CHEMICAL VAPOR OETEC110N WI'" AMUlllSPEcmAL ~EAMAlIMAGER
`
`,. SYSTEM MODEL
`For a standard fArget detection problem in the FUR world cbc
`IIIIcl is a bard object that bas a relatively bed d.i.mension' IIld
`ptCfiJc,aAd ~utes a sbup.edged f~ in. the flIR image.
`()eIOCUOO allonthou of maay cypcs bave been devised to sepo
`atal. features wich those cballcleristics (roll) the scene batt(cid:173)
`grvuod or clutter,l•.lS CJUtte1 itself is modtled as a firsl~rder
`Markov process andslalistica1 methods have been devised for
`ilS suppression. 16 Rauch ct aI. b'caltd aonnalarmospherk dou.ds
`as cluncr and investigartd medlods lo suppress dlem. " MiDor
`and Sklansky presented a method for detectiDg blobs, a class
`inlO wtUcb clouds CID fall. in I:R images using edge detection
`and adler techniques designed for detennittistically ~pracDted
`feacures. 11 We have found no other work in which douds have
`beeR IRatcd as a target ror a detectioo algorithm. Since vapor
`douds have cenaio propenie$lhat differ from clutter, EveD cloud
`cluner, Ib_ propenies can be the buis of- a discriminatioa
`teChnique.
`10 o*r to design a chemical agcDt deaucor ror the multi·
`spectral FUR, a sufficiently accurate system model is requited.
`It must cDCOlDpISIlbe backgrouod, signal from the vapor cloud.
`and &he sensor. The model described iD this section is due to
`bri ft
`..:.. de'
`.
`W 19,10 UI
`ne 5Ul1Ul1tltW!
`e y lua
`nvauoD.
`111eO..
`The extinction coefficient at wavelcngth ~
`
`•
`
`(1)
`
`wbm: x-;;; (~,)') represents a two-dimensional spatiaJ image, 1M
`~ aD amb~nt aIIDOSpbcrit componcot, p). is the absorbtivlry of
`rhe vapor, and C(x.:) 15 the vapor concefttnlUon at locatio" I
`in the plane at z. nonnaJ to the instrument line of sight. A
`Gaussian model for the vapor concentration is u~:
`
`C(I~)=Co exp -
`[
`
`(.zZ + 1+(l
`y 2a~ -to
`
`)1)]
`
`,
`
`lrilb Co the pcak concentration and :0 the cloud center. Slncc
`an fLlR, u·a pusive instrUm.ent, collects a signal integrated
`over Ibc romIparh Itng1h, the patbainteglafCd concentralion CL(x.:)
`Ii
`.
`
`CL(J,r)~ , C(...·) dr'
`
`with Q(x) defined by
`
`Q{x)= [
`.
`J
`
`el.p( -fl2) eft ,
`
`(2)
`
`(3)
`
`ft'bm Ii is Cbe pilellocatioo OD me detector plaDe•• is Cbe field
`of vie~ diftdio~. of the pilei. I is the effective focal leagdl.
`~ 1'.. as lbc addiuve ddcdor aoise uncom1atcd with Ibe signal,
`S,guaI power P~ is coasidenld a nmdom. fuoctiOil s.iRc:C bodJ
`dcteetor aoise and the bac:kgJouod ICIDpe11lIftT,(8) III'C random
`funetiooa. Thus we CIIl cbaract!rizc I'd by ill fim· aDd second·
`order moments. rant we geoeralize Eq. (4) for a time (indel k)
`series of·moltisp!CUal (i1\dtl J) imaaes.
`
`PIII{I;.'t)~ f P,,(8,1t.)R(Jl/- 8) d2t+P¥-.) .
`
`(.5)
`
`wbere the entrance aperture po~er is
`
`Pt/.8Jt)-Ar[FJ(~)(B.(rcl+TA(~1 C1.p!-p.CL(e,/tll
`
`X(B.IT,(eJul-B.(T,lI) ] III .
`
`(6)
`
`Additional paramttm lit
`
`= system optical CODItaDt ,
`A
`B~(1) .. Plack function 81 tcmpeTlNJ'e T,
`FJ(~) ~ bandpass (unctioaof che opticaJ filter celltered ill
`wav~len&tb ~,
`• ~mpmture of the vapor cloud, and
`- craasmittanee of the atmOsphere II wave~Dgtb ~.
`
`Tc
`1'A().)
`
`The first and second moments of tbe signal power aR
`
`E(Pd~I;,',)lePdJIi.II) 15 f E[P,J8,lt)JR{ljl/- I) d2
`
`,
`
`(7)
`
`IDd
`ApyCI, .It.&r,IA')aE{~pct,{Ji ,It) • PJ.,., ,I.5)[pd(14','t')
`- ,..(Xr.~')}}
`-I[R(lIlj-8)A,,(8.lt:I',lr)
`
`xR(Ki'f!-8)1 d2t d28'
`
`+A,.J1'{X{.'t;ll',It').
`
`(8)
`
`(9)
`
`(10)
`
`"Ibm lli/~ll l~~M, and lf$j~L.
`Background or duRer telllperarurejs assumed to be wide
`
`sense Stationary sucb thalE(T.(I.'I»)~ T,. a CODSWll. Although
`
`background KeDeS do DOl geoenlly exhibit staliooaril)' I War·
`m. found that his algorithm performed weU with field data.
`Using these !\vo moments, die sipal modd is I multivarillC
`Ouassian deo.slty wilb signal power P.. elplUSCd as an
`.~ L-dimeusionaJ random 'lector Pct =(I'd" .... Ptit) with probability
`dcuiry fuDCtiOD gi~en by
`
`The detected signal power Pd is a convolutioD of P"
`the
`power at the eo.trance apcrtwe of the. seUOf. with R. the system
`poinl spread function,
`
`'rAP..)-
`
`#'d{Xi)~f P..(8)R(KiI/- 8) d1t +I',,(~) ,
`
`(.4)
`
`Itl
`
`1
`NJ~
`(2'11"-
`,ArJ)
`xexp{ -¥<P4-PifA;.U>rr.1I} ,
`
`(II)
`
`Of'llCAl. ENGtNEERING I N~mbe, 1991 I Vd, 30 No. '1 I
`
`11'2.9
`
`FLIR Systems, Inc.
`Exhibit 1013-00005
`
`

`

`ALTHOUSE. CHANG
`
`Using the probability density functioD. Eq. (1). yields lite fol·
`lowing likelihood ndo rest
`.
`
`Hr
`ta.i!4!,Hr) ~ P(Ha)
`d ='I,J,PdtHO) No P(Hr)s-, ,
`L(P)
`
`wbere No is the bypolhesLs thaI 00 vapor elists and Hr is the
`hypolhesis chat a vapor cloud target is present
`The above likelihood ratio can be t\anhec n:duted to the sum(cid:173)
`maIioo over the lime and spct1r8J dimensions of a Iuottion R~(I4).
`where RlJ(lI) is anivcd at by takiDg the product of the Fowier
`rransform ofeach lnPUI image ae each lime increment and speccraJ
`frequeoc:y with the invem of chc aUlocovariance 8t each spectraJ
`mquency, the target filter function, and &he sensor modulation
`b"aRsfer fuoctioQ (MTF). Here Rlj(lI) is the resuJl of the invent
`Fourier tranSform 01 the above product. Computing the inverse
`of the autotovariuc:e turned OUlao be Ute most time-consunling
`piocess. Wbeu implemented in fORTRAN on a MicroVAX
`computer, • 64-Crame 5equel)ce of two spectral bands ~uited
`lUore rhan ~ b to proceIs. The resultiog images bad a very
`significant increase in sr·to-noise ratio and targeu became
`easy to d~ visually.l
`
`5, PREPROCESSING ALGORITHMS: BACKCROUND
`CLV1TER SUPPRESSION
`FUR sensors image the rhermaI radiation emitted by a target.
`The target presence is identified by a difference io temperature
`bctweeD the target and lbe immediate surtOWldingS, Le., back(cid:173)
`potmd clutter and DOise. As a result. the loss of the coonst
`betwClCQ dle target and the background can seriously degrade
`Ihe detettabiJity of the targ~t. In order to alleviate this problem,
`Iftproccssing is generally mquUed prior to duesholdlng delC(o
`dOD. Two preprocessing techlliques are commooly used:
`(1) adaptive ~ enbaacemen~1 and (2) backgmUlld suppres·
`SiOD.16.17J2.23 Of pankuJar IntaUt iD multispectral images is
`the 1auet applO8C~ wbidl dcaips computationally efficieot
`adaptive tUfer8 of different types to mnove or suppress highly
`~ backgroUDd cllltter. SiDcc an adaptive filter baa the
`capability of adapeios llDknoWD SllciJtics and yields desired Ie(cid:173)
`lUlu over. wide range of eaviroDments, Ibc atudy of ldapdve
`. filren for I variety of apptica1iODl baa r=lved considerable
`infaeSt O"'f dle past yean.2A
`A wieldy used feclmique for adapcivc mulrispcctral filtering
`is to use che minimum meao-square-enor (MMSE) criterion to
`acbicve background clutter SUppressiOD, thus improving deleCt(cid:173)
`ability of IatJCtl in PLIR imagea.16.11.23 The adaptive filter to
`be designed coasists ofaset ofseaIar IUtercoefficients e1pRSsed
`by • welJbrinc ver;b.•. The 00tpUI of the filter, y is dte weigbftJd
`sum of the input signal vectOr I
`liveD by
`
`(12)
`where r is the transpose and " is a weighling vector. An optimal
`adaptive fill£r is Doe minimizing Ihe meaD·square error between
`abe desin=d signal and the output y.
`Two appmacbes tD findin& aD optiI:na1 weighting vector for
`a desired MMSE ldaplive multispectral 81rcr 1ft of ioterest.
`
`5.1, UIlClOllStralned MMSE ldapdve flIt.erIDg
`FUR image backgrouud SUpprESSion tID be e:ut as an UDCOo(cid:173)
`strained MMSB adapdlle filta' problca.l6.•1.Z) The ~uJcial
`
`1730 I OP'TlCAl £HQIN££fUHG I N~ ,., I Vol. 30 No. , ,
`
`·f
`
`optimal filler is generally rderred (0 as a Wieoa.Hopf filler.
`To be more specific. let c be the diff~oce between che desin!cl
`sipals • aod 1 expressed by
`
`e;5-7 .
`
`(131
`
`An ~timal filter minimizes the mean sqUaft. of Eq. (13), i.(.•
`E(~)-=£(S-1)T(I- y»). where Eis Ihe eapectatiDD taken t.ittl
`IUpetl to (be joint random vector (s,,). The resulting optimal
`
`weigbtinl vector w. is a solution to lhe well-known Weiner.
`
`Hopf equation:
`
`wopl~RJ) ,
`(14t
`where R. is the invene of M1 == £(QT). the covariance mllJU
`of x, and DcE(81 ii the cross covariante maw of x and 5.
`We use a window (or searcb boll La implement adaptive
`filtering to suppress the background clUller of FUR images. Tht
`window is moved and centered on all pi~els ill the lmage in rum
`to calculate all local.covariance matrices. The window consid(cid:173)
`ered here is an adaptive tilter specified by a weighting wctor
`thal puts weights on all pixels falling in the window. The details
`of implementation can be found in Refs. 16, 17, and 23.
`
`5.2, LlDearly CODJtrI1ned MMSE adaptive 8Iterlng
`An alternative approach to the Wlconstraincd MMSe adapdvt
`6lteriag is I coD5ttBincd MMSEadapdve ftJteriag,l' whicb is
`applied to Ibc case where prior knowledge of the desired sipaJ
`5 is DOl known. Due to the lack of I, a linear coosQ'1int vector
`c must be imposed on the filter to constrain the filter OUtpUl Let
`the linear constraiDt be givea by
`
`The resultina consO'1lined MMSE acSapative lUter is ebaraeteriz~
`by fiD.dinglD optimal weighting vector wapi' a solution to tbe
`following equatiou:
`
`~"'M,"l subject to etw=g ,
`.W
`
`(16
`
`w~ .op can be solved and given by
`
`f¥
`".=::VB .
`c ~
`
`.
`
`Using the fonnulatioD (16), the adaptive lihefiDg for FLI~
`background suppression CID be accomplished by choosinS a
`
`appropriate linear constralDt vector t and a laiD'r For i.nsraI1c~
`
`if we Inl lnterested in applying a 3x 3 squate lJTIy window I
`aD ~mage while maintaining the wiec signal WlChaDgcd.
`II
`gain I can be chosen to be uniry IPd the coDStraiD.1 vector
`cbo5eo 15 a nlne-4irnensional vector widl ODe 10 the centr
`compoaenl and uro in aU other componeots. AJ I multo I:
`pb~ fallinl in the cenrer of the wiodow is relained UDcbugt
`This fomuslatioo yields a tGnstr'ained MMSE probltm. The I
`8ultiDg oplima1 filter is geoenLIly ~femd to u aminirolllD \':
`iance distortionless IQponJe (MVDR) adaptive beamformef.
`
`5.3. S)'ItoIk am,l1gorithms
`A major dif6culry eocounfere4 in Ibe approaches described
`Sec. S.I and .5.2 is Ule computation of Ihe optima! weigbti
`vector ~uJtin& from adaptive fileers, which requites Ulvcni
`
`FLIR Systems, Inc.
`Exhibit 1013-00006
`
`

`

`OEIt1ICAL VAPOR OEliCT1ON wnN AMULllSPECTRAL Tl1ERMAlIMAGEA
`
`• sample covari&llCt rtIIIIU ... and tntel15M larBe-scale mabW
`rector multiplicarioas. II is very cosdy ifIhc filta' is implemented
`willi di*t maIril inveniOll. A1tboup WIDIUdeveloped. ft.
`cursive fonnula for generadAg Ihe opdmal weipa vector to ~.
`,du« tomputalioDuload. run advlIltqe of using maIriJ. sttuc.
`cuRS is IlOt eAploited.
`Recently. the lntroductioo of systoUc amy algorithms by
`Kung and LeisenonD revolutionized bardwuc design for car·
`rying out matril computations. With the help of systolic anay
`aJgorichms. malril tomputatiolls can be petfonncd in pnlleJ
`and implcsPented in 1UJ.rJme processing. BasicalJy. systolic u(cid:173)
`ray alBorilhms are Ottholoaal aiangulariulioD praocucs using
`QR.ompositioo or Qolcsky factoriutioD. whith allow uS to
`QiaDgulariZc the sample tGvariance matrix so that the resuJtiDS
`uiangularized malrice8 CID be inverted very eftleleDdy by fOf(cid:173)
`ward and backward substitutions. Most imponantJy, the systolic
`uray algorithms ItC DUmerital.Iy stable, lObUlI to 8nitc arilh-
`metic precision. and taD be designed to be panlIel and pipeliAed.
`Ibus·greatly improving d1e implemelltation of the optimal filter
`ill real-lime applicadoas. The derails of studying such ,systolic
`anay approach De ~poncd by Ret, 27.
`
`,. REAL-TIME DETECI10N ALGOlUTHMS
`Akr preprocessing FUR !maBes. I tbresboldins technique is
`applied to deteCt the barBel signa!, Many detecUOQ algorithms
`ba'IC been developed for Ibis purpose. Amoas them is the de..
`rection alsorithm S\lggcsted by Wanen, 1~.20 particularly BOte(cid:173)
`worthy because it is rccunive in time and bas been shown to be
`c£fec:tive.
`Radler than CoUowing Wanen's algorithm, which is based
`00 a fi~t-order autoregmsive (AR) time series. Kalman filter
`dlcotY is used to develop I similar I'eClIJIive detection algorithm
`. Ibat coveR WatmD'S algorithm IS a special cue IIld extends it
`ro include I,eCWlion m. spcctnI frequency. The idea of using a
`K.alman Iiltu is natural since it caD ~ recursively Unplemeoted
`in .w-time ptOCCSIUlI· Unlike Warrco's work. this approach
`uses a state equation 10 modtl background cluner of d.ifferent
`1ypeS. The backgroWid clutter in WarreD·S wort 9Ias asswned
`ro be a ftrst"'" au~gressivc mocle1lhat conesponds to lu
`counterpane a state equation ift Kalman filtering. Usilll the Mar(cid:173)
`kov property induced by the AR model, Wamn derived aM(cid:173)
`cursive Cannula for derecton described by I sequcnriallikelihood
`ratio leSt SlIdatic. The essence of Ka1maD 6Jtef theory is to
`iIlrroduce a DCW pmeess, cite iMovaliona process suggested by
`Kailath.21 lostud of diRctly dealing with an