THAT TARTANI A MAN IN A UN UO NAO VI VAR VI TIM
`US 20170236281A1
`( 19 ) United States
`( 12 ) Patent Application Publication ( 10 ) Pub . No . : US 2017 / 0236281 A1
`( 43 ) Pub . Date :
`Aug . 17 , 2017
`DACOSTA
`
`( 51 )
`
`2 )
`
`( 54 ) COLLECTION AND ANALYSIS OF DATA
`FOR DIAGNOSTIC PURPOSES
`( 71 ) Applicant : UNIVERSITY HEALTH
`NETWORK , Toronto , ( CA )
`Inventor : Ralph DACOSTA , Etobicoke , Ontario
`( CA )
`
`( 72 )
`
`( 73 ) Assignee : UNIVERSITY HEALTH
`NETWORK , Toronto , ON ( CA )
`( 21 ) Appl . No . :
`15 / 328 , 214
`( 22 ) PCT Filed :
`Jul . 24 , 2015
`( 86 ) PCT No . :
`PCT / CA2015 / 000444
`$ 371 ( c ) ( 1 ) ,
`Jan . 23 , 2017
`( 2 ) Date :
`
`Related U . S . Application Data
`( 60 ) Provisional application No . 62 / 028 , 386 , filed on Jul .
`24 , 2014 .
`
`Publication Classification
`
`Int . Ci .
`G06T 7700
`A61B 5 / 00
`U . S . CI .
`
`( 2006 . 01 )
`( 2006 . 01 )
`. . G06T 7 / 0016 ( 2013 . 01 ) ; A61B 5 / 0071
`( 2013 . 01 ) ; A61B 5 / 445 ( 2013 . 01 ) ; A61B
`5 / 0077 ( 2013 . 01 ) ; G06T 2207 / 10064
`( 2013 . 01 ) ; G06T 2207 / 10024 ( 2013 . 01 ) ; G06T
`2207 / 30096 ( 2013 . 01 )
`ABSTRACT
`( 57 )
`Systems and methods for determining bacterial load in
`targets and tracking changes in bacterial load of targets over
`time are disclosed . An autofluorescence detection and col
`lection device includes a light source configured to directly
`illuminate at least a portion of a target and an area around the
`target with excitation light causing at least one biomarker in
`the illuminated target to fluoresce . Bacterial autofluores
`cence data regarding the illuminated portion of the target and
`the area around the target is collected and analyzed to
`determine bacterial load of the illuminated portion of the
`target and area around the target . The autofluorescence data
`may be analyzed using pixel intensity . Changes in bacterial
`load of the target over time may be tracked . The target may
`be a wound in tissue .
`
`12 13 14
`
`15 16
`
`17
`
`18
`
`
`
`are the only
`
`the
`
`that
`
`* * * * * * * *
`
`prestatud
`
`Petitioner's Exhibit 1014
`Page 1 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 1 of 34
`
`US 2017 / 0236281 A1
`
`12 13 14 15 16 17 18
`lendemainlamla
`
`e
`
`L
`
`no
`
`20
`
`FIG . 1
`
`Petitioner's Exhibit 1014
`Page 2 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 2 of 34
`
`US 2017 / 0236281 A1
`
`*
`
`FIG . 2
`
`Petitioner's Exhibit 1014
`Page 3 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 3 of 34
`
`US 2017 / 0236281 A1
`
`Red fluorescence
`
`Bacterial porphyrins
`
`collagen
`
`: :
`
`:
`
`: : .
`
`3
`
`.
`
`rent
`
`FIG . 3
`
`Petitioner's Exhibit 1014
`Page 4 of 63
`
`

`

`Patent Application Publication Aug . 17 , 2017 Sheet 4 of 34
`
`US 2017 / 0236281 A1
`
`antara Viewing Screen
`
`Qeveria
`
`Camera
`
`a
`
`Western
`
`FIG . 4
`
`Petitioner's Exhibit 1014
`Page 5 of 63
`
`

`

`Patent Application Publication Aug . 17 , 2017 Sheet 5 of 34
`
`US 2017 / 0236281 A1
`
`wa
`
`Who
`oran
`EK
`ss
`
`* W
`Pre
`
`& Cosmos
`sambusogick
`8 . Ruos meios
`in WoW
`
`FIG . 5
`
`Petitioner's Exhibit 1014
`Page 6 of 63
`
`

`

`Patent Application Publication Aug . 17 , 2017 Sheet 6 of 34
`
`US 2017 / 0236281 A1
`
`Omission 91
`
`switch
`LED heart
`sin
`endosed
`
`. . . . . . . .
`
`Der
`
`$ acro
`
`. . . . . . .
`
`. .
`
`.
`
`Battery
`bolder
`
`. . . . .
`
`. . .
`
`Phone 5S
`
`FIG . 6A
`
`Vamosas
`
`FIG . 6B
`
`Petitioner's Exhibit 1014
`Page 7 of 63
`
`

`

`Patent Application Publication Aug . 17 , 2017 Sheet 7 of 34
`
`US 2017 / 0236281 A1
`
`START
`
`200
`
`ILLUMINATE TARGET
`
`COLLECT BACTERIAL
`AUTOFLUORESCENCE DATA
`
`220
`
`230
`
`ANALYZE DATA USING PIXEL
`INTENSITY
`
`ma 240
`
`DETERMINE BACTERIAL
`LOAD ON TARGET
`
`TRACK CHANGES IN
`BACTERIAL LOAD ON
`TARGET OVER TIME
`
`260
`
`FIG . 7A
`
`Petitioner's Exhibit 1014
`Page 8 of 63
`
`

`

`Patent Application Publication Aug . 17 , 2017 Sheet 8 of 34
`
`US 2017 / 0236281 A1
`
`START
`
`300
`
`ILLUMINATE TARGET
`
`COLLECT DATA INCLUDING
`RGB IMAGES
`
`320
`
`330
`
`SEPARATE RGB IMAGES
`INTO RGB CHANNELS
`
`340
`
`CONVERT GREEN AND RED
`IMAGE CHANNELS TO GRAY
`SCALE
`
`wwwmasti mention 350
`
`COUNT PIXELS HAVING
`GRAY - SCALE INTENSITY
`ABOVE THRESHOLD
`
`360
`
`FIG . 7B
`
`Petitioner's Exhibit 1014
`Page 9 of 63
`
`

`

`Patent Application Publication Aug . 17 , 2017 Sheet 9 of 34
`
`US 2017 / 0236281 A1
`
`a
`
`Day
`
`Day
`
`D ay 3
`
`Day
`
`FIG . 8
`
`Day
`
`Day 10
`
`Mouse
`
`.
`
`a
`
`? * * * * * * * * * * * * * * * * * * *
`
`Bateria '
`
`*
`
`09
`Daily taking of the mouse wound her .
`Ogreus inoculation from day to day 10 . :
`Red arrow jidicates Kuprin teatritest . . .
`
`Petitioner's Exhibit 1014
`Page 10 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 10 of 34
`
`US 2017 / 0236281 A1
`
`.
`3xxx 2
`
`s
`
`.
`
`.
`
`. . . .
`
`v
`
`.
`
`.
`
`.
`
`. .
`
`. . .
`
`.
`
`. .
`
`Rogaciun bandice scotas
`
`azadan daha esasis
`
`. :
`
`Judexterna x
`
`EEEEEEEEE FIG . 9
`
`BESTÉKKOnce intertøy
`
`. . . :
`
`1 . 08
`Anm )
`603
`
`: : : : : : : : : : : : :
`: : : : : : : : : :
`: : : : : :
`Osy 2
`Day
`Og
`
`Days
`
`: : : : : : : : : :
`
`: : :
`
`Oaxa
`
`2007 . . . .
`
`osv ?
`
`: onys
`
`KO
`
`* ico
`
`KO
`
`YAKI
`
`17
`
`.
`
`ums ass & a aBEISSRS
`
`Petitioner's Exhibit 1014
`Page 11 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 11 of 34
`
`US 2017 / 0236281 A1
`
`DEVIL MESURSES
`
`yenide
`
`)
`
`Agak mate
`
`t
`
`1 day
`
`FIG . 10
`
`Petitioner's Exhibit 1014
`Page 12 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 12 of 34
`
`US 2017 / 0236281 A1
`
`: :
`
`: . . .
`
`: :
`
`: BO FIG . 11
`
`Petitioner's Exhibit 1014
`Page 13 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 13 of 34
`
`US 2017 / 0236281 A1
`
`a
`
`. .
`
`green
`
`red
`
`red
`green
`
`tetekort
`
`FIG . 12
`
`Petitioner's Exhibit 1014
`Page 14 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 14 of 34
`
`US 2017 / 0236281 A1
`
`Debridement
`
`Monitor Outcomes by
`Real - time Fluorescence
`Imaging
`
`Which
`Nound
`will heal ?
`
`WWW
`
`w
`
`Evaluation by
`Fluorescence
`Imaging
`
`- mayo
`
`catenin
`
`Wwwwwwwwwww
`
`interese
`
`JGU
`
`was
`
`No healing
`
`Healing
`
`FIG . 13
`
`39
`
`WEBUXO
`
`wwwwwwwwwwwwwwwww
`
`Petitioner's Exhibit 1014
`Page 15 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 15 of 34
`
`US 2017 / 0236281 A1
`
`Red
`
`Green
`Green
`
`Sie
`
`CA
`Q m
`
`3
`
`x
`
`x
`
`x
`
`W
`
`* * *
`
`*
`
`FIG . 14
`
`Blue
`
`mm
`mmmm
`
`ratio ( a . u . ) ?????zed
`
`Red / Green
`
`Petitioner's Exhibit 1014
`Page 16 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 16 of 34
`
`US 2017 / 0236281 A1
`
`pred now
`
`red arrow
`
`blue arrow
`
`6 ) b )
`
`red arrow
`
`red arrow
`
`blue arrow
`blue arrow
`
`pink arTOWS
`
`e
`
`yellow arrow
`
`FIG . 15
`
`Petitioner's Exhibit 1014
`Page 17 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 17 of 34
`
`US 2017 / 0236281 A1
`
`blue arrow
`
`blue arrow
`
`FIG . 16
`
`FIG . 17
`
`Petitioner's Exhibit 1014
`Page 18 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 18 of 34
`
`US 2017 / 0236281 A1
`
`Pre - clean
`
`Post - clean
`
`yellow arrow
`
`yellow arrow
`
`Pre clean
`
`Post - clean
`
`FIG . 18
`
`Petitioner's Exhibit 1014
`Page 19 of 63
`
`

`

`Patent Application Publicatio
`
`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 19 of 34
`
`US 2017 / 0236281 A1
`
`White
`
`Fluorescence
`
`Fluorescence
`
`Week 1 ( March 30 ,
`
`Week 2 ( April 6 ,
`
`Week 3 ( April 13 ,
`
`Week 4 ( April 20 ,
`
`Week 6 ( May 4 ,
`
`FIG . 19
`
`Petitioner's Exhibit 1014
`Page 20 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 20 of 34
`
`US 2017 / 0236281 A1
`
`Week 1 ( March 30 .
`
`red arrow
`
`blue arrow
`
`Jorange arrow
`yellow arrow
`
`red arrow
`
`blue arrow
`
`orange arrow
`
`blue arrow
`
`red arrow
`
`orange arrow
`
`red arrow
`
`blue arrow
`
`Week 2 ( April 6 .
`
`Week 3 ( April 13 .
`
`Week 6 ( Mav 4 .
`
`Week 4 ( April 20 . MA
`
`FREE
`
`orange arrow
`
`Healing
`Wound
`wound Qred arrow boundary
`blue arrow
`
`orange arrow skin tissue
`
`FIG . 20
`
`Petitioner's Exhibit 1014
`Page 21 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 21 of 34
`
`US 2017 / 0236281 A1
`
`TY
`
`) 3 . 5°C
`
`31 . 5°C
`
`34 . 8 °C
`
`30°C
`
`34 . 1°C
`
`* *
`
`* * * * *
`
`* * * * * * * * * *
`
`yellow
`arrows
`
`FIG . 22
`
`FIG . 23
`
`Petitioner's Exhibit 1014
`Page 22 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 22 of 34
`
`US 2017 / 0236281 A1
`
`bo
`
`500
`nesty
`
`400
`
`Spcerat 7090jromont on Paien : 1 ( 10 )
`locatior 1 DA3109 Filename : Patient 20420SK01
`
`034
`
`. 00€
`
`630 670
`500 500
`wysté : 09 : hom
`TRRSS RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR .
`Spectral measurement 07 - atent 1XJI
`location 14 , 30 . 4
`09 , filonomo : Paricri1 20 prokOX
`wwwwwwwwwww
`XY
`
`2501
`
`& 300
`
`L
`
`* * *
`
`*
`
`* * * * * * * *
`
`* * * * *
`
`*
`
`, ,
`
`????
`
`???
`
`
`
`?? ??
`
`??????????
`
`to cart
`?
`
`?
`
`g intensiewing
`
`??
`
`FIG . 24
`
`Petitioner's Exhibit 1014
`Page 23 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 23 of 34
`
`US 2017 / 0236281 A1
`
`.
`
`. . .
`
`. . . . . .
`
`. : : : : : : :
`
`: : : :
`
`: :
`
`:
`
`: : :
`
`: : :
`
`. . .
`
`. . . . . . . .
`
`: :
`
`.
`
`. .
`
`. . . . . . . . .
`
`. . . : :
`
`. . . . . .
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`. .
`
`. .
`
`.
`
`. .
`
`. .
`
`. . . . . . . . . .
`
`: : : : : : : : : : : : : :
`
`. .
`
`. . . . . . . . . .
`
`. . . . .
`
`:
`
`! ! !
`
`! !
`
`. . .
`
`. .
`
`. .
`
`. . . . .
`
`. . . .
`
`. . .
`
`FIG . 25
`
`RRR
`
`Petitioner's Exhibit 1014
`Page 24 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 24 of 34
`
`US 2017 / 0236281 A1
`
`Measurement
`scale
`Information
`barcodes
`Multiplexed
`fluorescence dyes
`Multiple
`concentrations
`of various
`exogenous
`fluorescent
`dyes for
`multiplexed
`fluorescence
`intensity
`calibration
`
`1 cm
`
`- - -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`Disposable " calibration " target
`
`Place disposable ' calibration '
`strip near wound site
`
`: NNN
`
`Select optimal
`fluorescence filter set
`
`Administer exogenous
`fluorescent agents
`
`4 .
`Collect WL + FL images Target Bx using fluorescence
`image - guidance
`
`FIG . 26
`
`Petitioner's Exhibit 1014
`Page 25 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 25 of 34
`
`US 2017 / 0236281 A1
`
`Bacteria
`??
`{ " ??
`
`Gram
`
`r
`
`obot
`
`. . . .
`
`'
`
`' ' ' ' ' ' ' ' ' ' '
`
`. "
`
`.
`
`. . . .
`
`.
`
`.
`
`, ' ' , ' ' , ' .
`
`: : : : : : : : : : : : : : : : : : : :
`
`, ' '
`
`.
`
`. . . '
`
`: : :
`:
`
`: :
`
`. . .
`
`: : : : :
`: : : :
`
`:
`
`:
`
`:
`
`:
`
`: : :
`
`red green
`FIG . 27
`
`a
`
`White
`
`Fluorescence
`
`FIG . 28
`
`15 : : : :
`
`:
`:
`
`: :
`: :
`:
`: : : : : :
`
`Petitioner's Exhibit 1014
`Page 26 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 26 of 34
`
`US 2017 / 0236281 A1
`
`a ) White Light
`
`Y
`
`.
`
`. . ' . ' . ' ' ; .
`
`* *
`
`Ikea
`
`Day 3
`
`. . . . . . . . .
`
`. . . . . .
`
`. . . . . . . . . . . . . . . . . . .
`
`DEDITO : :
`
`:
`: :
`:
`: : :
`:
`:
`:
`: :
`: :
`:
`:
`:
`: :
`: :
`: :
`:
`: : :
`:
`:
`: :
`: : : :
`:
`:
`:
`:
`:
`:
`:
`:
`:
`:
`:
`:
`:
`: :
`:
`:
`: : :
`:
`
`22 : 27
`31332999999999
`
`3333333333333333
`: : : KOSTEN
`
`. . . . . . . . . . . .
`
`. . . . . . . . . . . . . . . . . . . . . . . .
`
`: : : : : : : : : : : : : : : : : : : : :
`
`. . . . . . . . . . . . . . . . . . . . . . . . . .
`
`: : : : : :
`
`: : :
`
`: : :
`
`:
`
`: : :
`
`: :
`: :
`
`:
`: : :
`:
`
`7
`: :
`
`FIG . 29
`
`Petitioner's Exhibit 1014
`Page 27 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 27 of 34
`
`US 2017 / 0236281 A1
`
`Bacterial culture
`
`2
`
`.
`
`X
`
`red
`green
`blue
`
`* *
`
`:
`
`
`
`PULVI VIII
`
`.
`
`.
`
`.
`
`.
`
`VA
`
`9
`
`Pig meat sample
`
`* * *
`
`:
`
`P . Aeruginosa
`
`torsiklus
`
`* * *
`
`w
`
`indo
`wood soos
`
`.
`
`.
`
`. yellow
`. blue
`
`. .
`
`.
`
`*
`
`*
`
`. . .
`
`??????????????????????????
`
`:
`
`vooby
`
`.
`
`.
`
`. .
`
`Tred
`green
`
`blue
`
`wawe na
`
`E . coli
`
`s : 6 : 35 * Xics
`
`inni
`
`W
`
`FIG . 30
`
`E . Coli
`
`.
`
`WWW
`
`.
`
`Petitioner's Exhibit 1014
`Page 28 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 28 of 34
`
`US 2017 / 0236281 A1
`
`White Light
`
`Blood Absorption
`
`Www
`
`WWW * * * *
`
`.
`
`White Light
`
`Blood Absorption
`
`wwwwwwwww
`
`W
`
`MUTE
`
`- -
`
`-
`
`- -
`
`-
`
`- -
`
`-
`
`- -
`
`-
`
`-
`
`FIG . 31
`
`Petitioner's Exhibit 1014
`Page 29 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 29 of 34
`
`US 2017 / 0236281 A1
`
`chronie Wound
`
`. .
`
`. . .
`
`Treat
`Cause
`
`38ssere
`Demaku NSSEE
`
`Take patient histog
`
`Chmical assessment for diagnosis
`nchising assessment of
`NERDS STONES for clinical
`diagnosis of infection status ( adapted
`by Sibbala G et al : Wer Joranto )
`
`Describe wound / issue type :
`
`Determine amount of exudate on
`discharge bedrote tissue ano
`signs of Healing
`
`Make thermal reading of wount
`and surrounding normal tissues ,
`Hote wound odor and color
`
`BESS FL
`Place disnasable W
`sosoolion and sløsisity
`cakovation stronger wound
`on an excitation toht and
`imaging camera
`
`Use aktari brams to optimize
`Mevice to wound distance
`Brale WEF Images
`
`Colec w images at woum
`Select luorescence in the
`collet store AF images or WOUTO
`
`Proprietary software used to
`determine Buorescence
`osties of anavga
`:
`
`Measure vyouno dimensions
`Wheth length and depth or tracing
`the periphery using conventional
`
`3 No
`
`Is An Image contrast sütticieni
`Yes
`Use Amage guidance to target
`wound culture specimen nollection
`le ofousy
`ine nende aspirates
`
`. . . . . . .
`
`. . .
`
`. . . . . .
`
`. . .
`
`. . . .
`
`. .
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`prepare the Wound Best
`
`Patiente .
`Centered
`Concerns
`*
`*
`*
`*
`??
`
`Determine
`Superiaat
`Interior
`ago
`CHOCO
`Blaston
`
`Use Imaging device or teal time nofk .
`vasive masassment wound
`and surrounding margins
`
`Administer exogerious
`uotest
`lecklackage
`contrast agent sowound stes
`Wak : for predetermined time
`beton Nuptescence : maguig
`
`Calibrate Collect Stare Fumage ,
`
`Ose Ab image guidance to :
`target wound cukure
`Specimens collection
`die biopsy or fine needle
`gspttste )
`
`Determine wound
`Healing / remodeling
`intection status , and
`Bolectilar blomarker
`expression :
`
`Send wound culture
`* specimens to
`bacteriology lab : { for
`standard care
`
`Determine treatment
`options ( ie , compression
`bandage therapya
`VBCUBMissisted closure
`( VAC ) therapy , and
`surgery , antibiotics
`
`FIG . 32
`
`Quantitative image
`analysis of corective :
`tissues :
`collagens :
`elastins )
`bacteria micrübes , and
`Wother fluorescent
`molecular biomarkers
`
`Ofrequired
`Send data via
`lele medicine
`för remote
`clinical
`consultation
`
`Follow Up
`Monitor
`Treatment
`Response
`
`Petitioner's Exhibit 1014
`Page 30 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 30 of 34
`
`US 2017 / 0236281 A1
`
`b )
`
`White light
`
`Fluorescence
`
`White
`
`Fluorescence
`
`FIG . 37
`
`:
`
`Tojiy
`
`tusz
`? ?
`?
`?
`
`my
`??????????????
`
`?
`
`.
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`? ?
`
`? ?
`
`* 13 * * *
`
`Mink
`
`* * * *
`Pristasy
`EXX Shakeyakiso
`
`ristiintiin
`De
`mwen
`
`{ » wako
`
`* * * * * * * * *
`* * * * *
`* * * * *
`Bar Os
`Bintiu
`vxandi
`line
`rrriiiiwiwiwwwwwwww !
`Nowa XXX
`M
`
`*
`
`* * *
`
`tysis
`
`cocco
`
`* 2 * 3
`M » zos
`IO
`167
`XX
`Ostinos
`Om
`2 : 30
`* * *
`exis
`too .
`* *
`*
`
`Ram
`mmm
`
`.
`
`* * * * * * * *
`
`. .
`
`* *
`
`X
`
`20
`
`*
`
`cocoooooo
`
`*
`
`WW
`
`Kot
`FRIK
`
`FIG . 33
`
`Petitioner's Exhibit 1014
`Page 31 of 63
`
`

`

`Patent Application Publicati0ll
`
`Aug . 17 , 2017 Sheet 31 0f 34
`
`US 2017 / 0236281A1
`
`'
`
`?????????????????????
`?
`?
`?
`?
`?
`?
`? ? ?
`?
`?
`?
`? ; .
`
`?
`
`?
`
`?
`
`?
`
`? ? ?
`? ? ? ? ?
`?
`? ? ?
`? ? ?
`?
`? ? ?
`?
`? ?
`? ? ? ? ?
`?
`?
`? ? ? ? ?
`?
`?
`?
`? ? ?
`? ? ?
`?
`?
`?
`?
`?
`?
`? ? ? ?
`? ?
`? ? ?
`
`?
`
`? ?
`
`? ? ? ? ? ?
`
`?
`?
`?
`
`? ? ? ?
`?
`?
`?
`?
`?
`? ?
`? ? ?
`? ?
`?
`?
`
`?
`
`?
`? ? ? ?
`
`?
`
`? ? ?
`
`?
`
`? ?
`
`?
`
`?
`
`?
`
`? ?
`
`?
`
`? ?
`
`?
`
`?
`
`?
`
`?
`???
`
`;
`
`?
`
`? ?
`
`?
`
`?
`?
`38 ?
`?
`?
`?
`?
`?
`
`?
`
`?
`?
`
`?
`
`?
`
`?
`
`? ?
`?
`?
`? ?
`?
`? ?
`?
`
`?
`
`; ?
`?
`
`
`
`???? ??? ?? ???? . ?
`
`? ? ? ?
`
`?
`
`? ? ?
`
`? ?
`
`? ? : . .
`
`? ?
`
`?
`
`?
`
`? ? ? ? ? ? ? ? ? ? ? ? ??? ?? . ? ? ??? ? ?
`
`.
`
`?
`
`.
`
`.
`
`? ? ?
`
`?
`
`?
`
`?
`
`? ? ?
`? ?
`'
`? ?
`
`? ? ? ? ?
`
`|
`
`|
`
`|
`
`.
`
`? ? ??? . ? ? ??? . ?
`
`???? ? ??? ? - ? ??? ? ?
`
`??? ?
`
`? ? ? ??? ? ? . ??
`
`?
`
`?
`
`?
`
`< - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
`
`FIG . 34
`
`Petitioner's Exhibit 1014
`Page 32 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 32 of 34
`
`US 2017 / 0236281 A1
`
`WOX
`
`. . . . . .
`
`X 0080 X
`
`wewe
`
`*
`
`XXX9 *
`
`YOON
`:
`
`wwwwwwwwwwww
`
`2009 : 03
`
`*
`
`W
`
`WKOXX
`
`WÁNKU
`
`XX
`
`
`
`* * AX838
`
`migos
`
`FIG . 35
`
`Petitioner's Exhibit 1014
`Page 33 of 63
`
`

`

`Aug . 17 , 2017 Sheet 33 of 34
`
`US 2017 / 0236281 A1
`
`Patent Application Publication
`
`0 : 00 3
`
`FIG . 36
`
`Petitioner's Exhibit 1014
`Page 34 of 63
`
`

`

`Patent Application Publication
`
`Aug . 17 , 2017 Sheet 34 of 34
`
`US 2017 / 0236281 A1
`
`Viewme
`screen
`
`sources
`
`handle
`
`Com
`
`Karco
`
`co
`
`Yooo0000 Power
`Xory
`
`FIG . 38
`
`Petitioner's Exhibit 1014
`Page 35 of 63
`
`

`

`US 2017 / 0236281 A1
`
`Aug . 17 , 2017
`
`COLLECTION AND ANALYSIS OF DATA
`FOR DIAGNOSTIC PURPOSES
`This is application claims benefit to U . S . Provi
`[ 0001 ]
`sional Application No . 62 / 028 , 386 , filed Jul . 24 , 2014 , the
`entire content of which is incorporated by reference herein .
`TECHNICAL FIELD
`[ 0002 ] Devices and methods for collecting data for diag
`nostic purposes are disclosed . In particular , the devices and
`methods of the present application may be suitable for
`evaluating and tracking bacterial load in a wound over time .
`
`BACKGROUND
`[ 0003 ] Wound care is a major clinical challenge . Healing
`and chronic non - healing wounds are associated with a
`number of biological tissue changes including inflammation ,
`proliferation , remodeling of connective tissues and , a com
`mon major concern , bacterial infection . A proportion of
`wound infections are not clinically apparent and contribute
`to the growing economic burden associated with wound
`care , especially in aging populations . Currently , the gold
`standard wound assessment includes direct visual inspection
`of the wound site under white light combined with indis
`criminate collection of bacterial swabs and tissue biopsies
`resulting in delayed , costly and often insensitive bacterio
`logical results . This may affect the timing and effectiveness
`of treatment . Qualitative and subjective visual assessment
`only provides a gross view of the wound site , but does not
`provide information about underlying biological and
`molecular changes that are occurring at the tissue and
`cellular level . A relatively simple and complementary
`method that collects and analyzes ' biological and molecular
`information in real - time to provide early identification of
`such occult change and guidance regarding treatment of the
`same is desirable in
`clinical wound management . Early
`recognition of high - risk wounds may guide therapeutic
`intervention and provide response monitoring over time ,
`thus greatly reducing both morbidity and mortality due
`especially to chronic wounds .
`SUMMARY
`[ 0004 ] In accordance with various exemplary embodi
`ments , a method of determining bacterial load of a target
`from fluorescent image data of the target is provided . The
`method comprises identifying a region of interest in a
`fluorescent image of a target , separating RGB images into
`individual channels , converting individual green and red
`image channels from the RGB image to gray scale , and
`counting pixels whose gray scale intensity was above a
`given threshold .
`[ 0005 ]
`In accordance with another aspect of the present
`teachings , a method of obtaining diagnostic data regarding
`a target is provided . The method comprises directly illumi
`nating at least a portion of a target with a homogeneous field
`of excitation light emitted by at least one light source
`connected to a housing of a handheld device , the housing
`including an enclosure for receiving a wireless communi
`cation device having a digital camera . The at least one light
`source emits at least one wavelength or wavelength band
`causing at least one biomarker in the illuminated portion of
`the target to fluoresce . The method further comprises col
`lecting bacterial autofluorescence data regarding the illumi
`
`nated portion of the target with an image sensor of the digital
`camera of the wireless communication device . The wireless
`communication device is secured in the housing . The
`method also comprises analyzing the collected bacterial
`autofluorescence data using pixel intensity to determine
`bacterial load of the illuminated portion of the target .
`[ 0006 ]
`In accordance with a further aspect of the present
`disclosure , a system for acquiring data regarding a wound in
`tissue is disclosed . The system comprises at least one light
`source configured to directly illuminate a target surface with
`a homogeneous field of excitation light . The target surface
`includes at least a portion of a wound and an area around the
`wound . An optical sensor is configured to detect signals
`responsive to illumination of the illuminated portion of the
`wound and the area around the wound . Each detected signal
`is indicative of at least one of endogenous fluorescence ,
`exogenous fluorescence , absorbance , and reflectance in the
`illuminated portion of the wound and the area around the
`wound . A processor is configured to receive the detected
`signals and to analyze the detected signal data using pixel
`intensity and to output data regarding the bacterial load of
`the illuminated portion of the wound and area around the
`wound . The system further comprises a display for display
`ing the output data regarding the illuminated portion of the
`wound and the area around the wound output by the pro
`cessor .
`[ 0007 ]
`In accordance with yet another aspect of the pres
`ent disclosure , a portable , handheld device for imaging and
`collection of data relating to a wound in tissue is disclosed .
`The device comprises a housing comprising an enclosure
`configured to receive a mobile communication device and at
`least one light source coupled to the housing and configured
`to directly illuminate at least a portion of a wound and an
`area around the wound with a homogeneous field of light . A
`mobile communication device is secured in the enclosure of
`the housing , the mobile communication device comprising
`an embedded digital camera and having a touchscreen
`display disposed on a first side of the device and a lens of the
`camera disposed on a second side of the device opposite the
`first side . The mobile communication device is received in
`the housing such that an image sensor of the digital camera
`is positioned to detect optical signals responsive to illumi
`nation of the portion of the wound and the area around the
`wound with the homogeneous field of light , each of the
`optical signals being indicative of at least one of endogenous
`fluorescence , exogenous fluorescence , reflectance , and
`absorbance in the illuminated portion of the wound and the
`area around the wound . When the mobile communication
`device is secured in the enclosure , at least a portion of the
`touchscreen display is accessible and viewable by a user .
`The device further comprises a processor configured to
`receive the detected optical signals , to analyze detected
`signal data using pixel intensity , and to output data regarding
`the bacterial load of the illuminated portion of the wound
`and area around the wound .
`[ 0008 ] .
`In accordance with another aspect of the present
`disclosure , a method of obtaining diagnostic data regarding
`a target is provided . The method comprises directly illumi
`nating at least a portion of a target and an area around the
`target with a homogeneous field of excitation light emitted
`by at least one light source connected to a housing of a
`handheld device . The housing includes an enclosure for
`receiving a wireless communication device having a digital
`camera . The at least one light source emits at least one
`
`Petitioner's Exhibit 1014
`Page 36 of 63
`
`

`

`US 2017 / 0236281 A1
`
`Aug . 17 , 2017
`
`wavelength or wavelength band causing at least one bio -
`marker in the illuminated portion of the target and area
`around the target to fluoresce . The method further comprises
`collecting bacterial autofluorescence data regarding the illu
`minated portion of the target and the area around the target
`with an image sensor of the digital camera of the wireless
`communication device . The wireless communication device
`is secured in the housing . The method further comprises
`analyzing the collected bacterial autofluorescence data to
`determine bacterial load of the illuminated portion of the
`target and area around the target , and tracking changes in
`bacterial load of the target over time .
`[ 0009 ] Additional objects and advantages of the disclosure
`will be set forth in part in the description which follows , and
`in part will be obvious from the description , or may be
`learned by practice of the disclosure . The objects and
`advantages of the disclosure will be realized and attained by
`means of the elements and combinations particularly pointed
`out in the appended claims .
`[ 0010 ]
`It is to be understood that both the foregoing
`general description and the following detailed description
`are exemplary and explanatory only and are not restrictive of
`the disclosure , as claimed .
`[ 0011 ] The accompanying drawings , which are incorpo
`rated in and constitute a part of this specification , illustrate
`embodiments of the disclosure and together with the
`description , serve to explain the principles of the disclosure .
`BRIEF DESCRIPTION OF DRAWINGS
`0012 ] At least some features and advantages will be
`apparent from the following detailed description of embodi
`ments consistent therewith , which description should be
`considered with reference to the accompanying drawings ,
`wherein :
`[ 0013 ] FIG . 1 is a schematic diagram of a device for
`fluorescence - based monitoring ;
`[ 0014 ] FIG . 2 shows an example of a clinical wound care
`facility using a device for fluorescence - based monitoring ;
`[ 0015 ] FIG . 3 shows images of a muscle surface of a pig
`meat sample , demonstrating the use of a device for fluores
`cence - based monitoring for autofluorescence detection of
`connective tissues and bacteria ;
`[ 0016 ]
`FIG . 4 shows images of a hand - held embodiment
`of a device for fluorescence - based monitoring ;
`[ 0017 )
`FIG . 5 shows an alternate embodiment of a hand
`held device for obtaining white light and fluorescent light
`data from a target ;
`[ 0018 ] FIGS . 6A and 6B show another alternative embodi
`ment of a handheld device for obtaining data regarding a
`target , the handheld device incorporating an iPhone ;
`[ 0019 ] FIGS . 7A and 7B illustrate exemplary methods of
`determining bacterial load of a target ;
`[ 0020 ]
`FIG . 8 shows representative white light ( WL ) and
`fluorescent ( FL ) images for a single mouse tracked over 10
`days ;
`[ 0021 ] FIG . 9 illustrates preclinical data which show that
`pathogenic bacterial autofluorescence ( AF ) intensity corre
`lates with bacterial load in vivo ;
`10022 ] FIG . 10 shows images of live bacterial cultures
`captured using a device for fluorescence - based monitoring ;
`[ 0023 ] FIG . 11 shows an example of bacterial monitoring
`using a device for fluorescence - based monitoring ;
`
`[ 0024 ] FIG . 12 shows images of a simulated animal
`wound model , demonstrating non - invasive autofluorescence
`detection of bacteria using a device for fluorescence - based
`monitoring ;
`[ 0025 ] . FIG . 13 illustrates an example of monitoring of a
`chronic wound ;
`10026 ] FIGS . 14 - 28 show examples of the use of a device
`for fluorescence - based monitoring for imaging wounds and
`conditions in clinical patients ;
`[ 0027 ]
`FIG . 29 shows images of a skin surface of a pig
`meat sample , demonstrating non - invasive autofluorescence
`detection of collagen and various bacterial species using a
`device for fluorescence - based monitoring ;
`[ 0028 ]
`FIG . 30 shows images and spectral plots demon
`strating the use of a device for fluorescence - based monitor
`ing to detect fluorescence from bacteria growing in agar
`plates and on the surface a simulated wound on pig meat ;
`[ 0029 ]
`FIG . 31 shows images demonstrating use of a
`device for fluorescence - based monitoring for imaging of
`blood and microvasculature ;
`[ 0030 ] FIG . 32 is a flowchart illustrating the management
`of a chronic wound using a device for fluorescence - based
`monitoring ;
`[ 0031 ]
`FIG . 33 illustrates the phases of wound healing
`with time ;
`[ 0032 ]
`FIG . 34 is a table showing examples of tissue ,
`cellular and molecular biomarkers known to be associated
`with wound healing ;
`[ 0033 ]
`FIG . 35 is a diagram comparing a healthy wound to
`a chronic wound ;
`[ 0034 ] FIG . 36 shows images demonstrating the use of a
`device for fluorescence - based monitoring in imaging a
`mouse model ; and
`[ 0035 ]
`FIG . 7 shows an example of the use of a device for
`fluorescence - based monitoring for imaging small animal
`models ;
`[ 0036 ] FIG . 38 shows an example of a kit including a
`device for fluorescence - based monitoring .
`[ 0037 ] Although the following detailed description makes
`reference to illustrative embodiments , many alternatives ,
`modifications , and variations thereof will be apparent to
`those skilled in the art . Accordingly , it is intended that the
`claimed subject matter be viewed broadly .
`DETAILED DESCRIPTION
`[ 0038 ] Reference will now be made in detail to various
`embodiments , examples of which are illustrated in the
`accompanying drawings . The various exemplary embodi
`ments are not intended to limit the disclosure . To the
`contrary , the disclosure is intended to cover alternatives ,
`modifications , and equivalents .
`[ 0039 ] Conventional clinical assessment methods of acute
`and chronic wounds continue to be suboptimal . Such assess
`ment methods usually are based on a complete patient
`history , qualitative and subjective clinical assessment with
`simple visual appraisal using ambient white light and the
`“ naked eye , ' and can sometimes involve the use of color
`photography to capture the general appearance of a wound
`under white light illumination . Regular re - assessment of
`progress toward healing and appropriate modification of the
`intervention is also necessary . Wound assessment terminol
`ogy is non - uniform , many questions surrounding wound
`assessment remain unanswered , agreement has yet to be
`
`Petitioner's Exhibit 1014
`Page 37 of 63
`
`

`

`US 2017 / 0236281 A1
`
`Aug . 17 , 2017
`
`reached on the key wound parameters to measure in clinical
`practice , and the accuracy and reliability of available wound
`assessment techniques vary .
`[ 0040 ] Visual assessment is frequently combined with
`swabbing and / or tissue biopsies for bacteriological culture
`for diagnosis . Bacterial swabs are collected at the time of
`wound examination and have the noted advantage of pro
`viding identification of specific bacterial / microbial species .
`However , multiple swabs and / or biopsies often are collected
`randomly from the wound site , and some swabbing tech
`niques may in fact spread the microorganisms around with
`the wound during the collection process thus affecting
`patient healing time and morbidity . This may be a problem
`especially with large chronic ( non - healing ) wounds where
`the detection yield for bacterial presence using current
`swabbing and biopsy protocols is suboptimal ( diagnostically
`insensitive ) , despite many swabs being collected .
`[ 0041 ] Thus , current methods for obtaining swabs or tis
`sue biopsies from the wound site for subsequent bacterio
`logical culture are based on a non - targeted or ' blind ' swab
`bing or punch biopsy approach , and have not been optimized
`to minimize trauma to the wound or to maximize the
`diagnostic yield of the bacteriology tests . In addition , bac
`teriological culture results often take about 2 - 3 days to come
`back from the laboratory and can be inconclusive , thus
`delaying accurate diagnosis and treatment . Thus , conven
`tional methods of obtaining bacterial swabs do not neces
`sarily provide r