I 1111111111111111 1111111111 lllll lllll lllll 111111111111111 lll111111111111111
`
`US011266345B2
`
`c12) United States Patent
`Saiko et al.
`
`(IO) Patent No.: US 11,266,345 B2
`Mar.8,2022
`(45) Date of Patent:
`
`(54) APPARATUS FOR VISUALIZATION OF
`TISSUE
`
`(71) Applicant: Swift Medical Inc., Toronto (CA)
`
`(58) Field of Classification Search
`CPC ....... A61B 5/445; A61B 5/742; A61B 5/6898;
`A61B 5/0013; A61B 5/0077;
`(Continued)
`
`(72)
`
`Inventors: Guennadi Saiko, Mississauga (CA);
`Kenneth Macko, Toronto (CA); Andrei
`Betlen, Pickering (CA)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`(73) Assignee: Swift Medical Inc., Toronto (CA)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by O days.
`
`(21) Appl. No.:
`
`17/260,664
`
`(22) PCT Filed:
`
`Jul. 16, 2019
`
`(86) PCT No.:
`
`PCT /CA2019/050981
`
`§ 371 (c)(l),
`(2) Date:
`
`Jan. 15, 2021
`
`(87) PCT Pub. No.: WO2020/014779
`
`PCT Pub. Date: Jan. 23, 2020
`
`(65)
`
`Prior Publication Data
`
`Aug. 26, 2021
`US 2021/0259625 Al
`Related U.S. Application Data
`
`(60)
`
`Provisional application No. 62/698,799, filed on Jul.
`16, 2018.
`
`(51)
`
`(52)
`
`(2006.01)
`
`Int. Cl.
`A61B 5100
`U.S. Cl.
`CPC ............ A61B 51445 (2013.01); A61B 510013
`(2013.01); A61B 510077 (2013.01); A61B
`516898 (2013.01); A61B 51742 (2013.01)
`
`5,074,306 A * 12/1991 Green
`
`6,251,070 Bl*
`
`6/2001 Khazaka .
`
`(Continued)
`
`A61B 5/445
`600/317
`A61B 5/0059
`600/306
`
`FOREIGN PATENT DOCUMENTS
`
`WO
`WO
`
`WO-2017/01267 5 Al
`WO-2017/155265 Al
`
`1/2017
`9/2017
`
`OTHER PUBLICATIONS
`
`International Searching Authority, International Search Report and
`Written Opinion for International Application No. PCT/CA2019/
`0 50981; dated Oct. 9, 2019, ( 6 pages), Canadian Intellectual Prop(cid:173)
`erty Office, Quebec, Canada.
`
`Primary Examiner - May A Abouelela
`(74) Attorney, Agent, or Firm -Alston & Bird LLP
`
`ABSTRACT
`(57)
`A tissue imaging system comprising a computing device,
`tissue visualization application, image capturing unit, and an
`illumination unit, is configured to capture measurement data.
`The visualization application extracts visualizations of tis(cid:173)
`sue health indicators from the measurement data. The appli(cid:173)
`cation generates an interface with one or more interface
`elements corresponding to the visualization of tissue health
`indicators.
`
`20 Claims, 10 Drawing Sheets
`
`Positioning mobile device
`402
`
`Capturing image(s) ~
`
`Pre-processing image(s) 406
`
`Fluorescence
`
`Reflectance
`
`Processing image(s) 408
`
`Extracting map 410
`
`Generating visualization 412
`
`Capturing data 414
`
`Storing/transmitting files
`ill
`
`Petitioner's Exhibit 1001
`Page 1 of 26
`
`

`

`US 11,266,345 B2
`Page 2
`
`(58) Field of Classification Search
`CPC . A61B 5/7264; A61B 5/14551; A61B 5/1455;
`A61B 5/14546; A61B 5/4878; A61B
`5/4875; A61B 5/0075; G03B 33/08;
`G03B 2215/0514; G03B 15/05
`USPC .......................................................... 600/306
`See application file for complete search history.
`References Cited
`
`(56)
`
`U.S. PATENT DOCUMENTS
`
`6,571,003 Bl*
`
`6,685,635 B2 *
`
`6,792,137 B2 *
`
`7,179,227 B2 *
`
`5/2003 Hillebrand .
`
`2/2004 Shani .
`
`9/2004 Kenet
`
`2/2007 Shirai
`
`7,454,046 B2 * 11/2008 Chhibber .
`
`7,657,101 B2 *
`
`8,026,942 B2 *
`
`8,116,852 B2 *
`
`9/2011 Payonk .
`
`2/2012 Baker, Jr.
`
`8,591,413 B2 * 11/2013 Kruglick .
`
`A61B 5/0064
`382/100
`A61B 5/0059
`356/425
`Gl6H 10/20
`382/128
`A61B 5/0059
`600/306
`A61B 5/442
`382/118
`2/2010 Christiansen, II ........ G06T 7/90
`382/218
`H04N 5/2354
`348/77
`A61B 5/0059
`600/476
`A61B 5/441
`600/306
`A61B 6/08
`A61B 5/0066
`A61B 5/0075
`A61B 5/442
`382/128
`A45D44/00
`424/401
`A61B 5/0064
`382/162
`A61B 5/0079
`359/892
`G06K 9/2018
`382/118
`A61B 5/0059
`600/306
`A61B 5/442
`600/306
`
`9,662,076 B2 *
`5/2017 Jia .
`10,127,661 B2 * 11/2018 Wang.
`10,182,757 B2 *
`1/2019 Gareau
`2004/0125996 Al*
`7/2004 Eddowes
`
`2004/0202685 Al * 10/2004 Manzo .
`
`2004/0218810 Al* 11/2004 Momma
`
`2007/0002479 Al*
`
`2007 /0064979 Al *
`
`2008/0161661 Al*
`
`2008/0194928 Al*
`
`2008/0214907 Al *
`
`1/2007 Menke.
`
`3/2007 Chhibber .
`
`7/2008 Gizewski .
`
`8/2008 Bandic .
`
`9/2008 Gutkowicz-Krusin
`
`2008/0319283 Al* 12/2008 Cotton
`
`2009/0080727 Al *
`
`2009/0177051 Al*
`
`2009/0220415 Al*
`
`2010/0185064 Al*
`
`A61B 5/0059
`600/306
`A61B 5/0059
`600/306
`3/2009 Cotton ...................... G06T 7/42
`382/128
`A61B 5/445
`600/306
`A61K 49/0004
`424/1.11
`A61B 5/415
`600/306
`
`7/2009 Arons
`
`9/2009 Shachaf .
`
`7/2010 Bandic .
`
`2010/0271470 Al* 10/2010 Stephan
`
`2011/0117025 Al*
`
`5/2011 Dacosta
`
`2011/0288385 Al* 11/2011 Stamatas .
`
`2011/0301441 Al* 12/2011 Bandic .
`
`2012/0041283 Al*
`
`2/2012 Krishnan
`
`2012/0041284 Al*
`
`2/2012 Krishnan
`
`2012/0172685 Al*
`
`7/2012 Gilbert .
`
`2013/0094730 Al*
`
`4/2013 Segman
`
`2013/0096392 Al *
`
`4/2013 Adams.
`
`2013/0204101 Al*
`
`8/2013 Rumberg
`
`2014/0088380 Al*
`
`3/2014 Sprigle
`
`2014/0378779 Al* 12/2014 Freeman .
`
`2015/0099947 Al *
`
`4/2015 Qu
`
`2015/0105635 Al *
`
`4/2015 Yoshida .
`
`A61B 5/445
`348/77
`G0lN 21/6456
`424/9.6
`A61B 5/0059
`600/306
`A61B 5/0059
`600/306
`A61B 5/448
`600/306
`A61B 5/448
`600/306
`A61B 5/411
`600/306
`A61B 5/445
`382/128
`A61B 5/0064
`600/301
`A61B 5/0062
`600/306
`A61B 5/0077
`600/306
`A61B 5/1032
`600/301
`A61B 5/442
`600/306
`A61B 8/0858
`600/306
`2015/0297130 Al * 10/2015 Stanmes ................... G06T 7/44
`600/306
`A61B 5/0013
`600/306
`A61B 5/742
`600/306
`A61B 5/0077
`A61B 5/444
`A61B 5/447
`A61B 5/445
`
`2015/0374277 Al* 12/2015 Patwardhan
`
`2016/0135730 Al *
`
`5/2016 Arai .
`
`2018/0333053 Al* 11/2018 Verkruijsse .
`2018/0333589 Al* 11/2018 Kim.
`2019/0069836 Al*
`3/2019 Hettrick .
`2019/0082998 Al*
`3/2019 Nowroozi
`2019/0090751 Al
`3/2019 Hwang et al.
`2019/0125197 Al*
`5/2019 Fukuda ................. G06T 7/0016
`2019/0216340 Al*
`7/2019 Holz
`A61B 5/7221
`2019/0231249 Al*
`8/2019 Dascalu
`A61B 5/0095
`2019/0388022 Al* 12/2019 Talgorn .
`A61B 5/02055
`2020/0003683 Al *
`1/2020 Haddad.
`A61B 5/445
`2020/0046999 Al *
`2/2020 Lim
`A61B 5/0077
`2020/0092534 Al *
`3/2020 Eckhouse
`A61B 5/443
`2020/0113438 Al *
`4/2020 Bourquin .
`A61B 18/203
`2020/0113441 Al *
`4/2020 Varghese
`A61B 5/7278
`2020/0121243 Al*
`4/2020 Anderson
`A61B 5/00
`2020/0121262 Al*
`4/2020 De Haan.
`A61B 5/0245
`2020/0129069 Al*
`4/2020 Inglese
`A61B 5/1077
`2020/0176099 Al *
`6/2020 Weiss .
`A61B 5/443
`2020/0179713 Al *
`6/2020 Subhash
`A61N 5/0616
`2020/0281512 Al*
`9/2020 Grubb .
`A61B 5/489
`2020/0281513 Al*
`9/2020 Grubb .
`A61B 5/489
`2020/0320683 Al* 10/2020 Horiuchi .
`A61B 5/444
`2020/0383630 Al* 12/2020 Xu
`A61B 5/0075
`2021/0052212 Al*
`2/2021 Yaroslavsky
`A61B 5/4836
`
`* cited by examiner
`
`Petitioner's Exhibit 1001
`Page 2 of 26
`
`

`

`UI = N
`0--, w
`0--,
`'N
`'"""'
`'"""'
`d r.,;_
`
`~
`
`Illumination
`
`unit 104
`
`unit 103
`Imaging
`
`unit 103
`Imaging
`
`Illumination
`
`unit 104
`
`108a I 108b
`
`108b
`
`108a
`
`unit 103
`Imaging
`
`108a 108b
`
`' ..
`
`Illumination
`
`unit 104
`
`....
`0 ....
`....
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`0
`
`visualization
`
`app 112
`
`Tissue
`
`visualization
`
`app 112
`
`Tissue
`
`visualization
`
`app 112
`
`Tissue
`
`Device~
`
`Mobile
`
`Mobile Device
`
`108
`
`Device~
`
`Mobile
`
`IP
`
`N
`N
`0
`N
`~CIO
`
`~ :-:
`~
`
`106
`
`Systems
`
`External
`
`~ = ~
`
`~
`~
`~
`•
`00
`
`e •
`
`100
`
`System
`
`Visualization
`
`Tissue
`
`.,,--,
`
`~ .
`
`110
`
`r
`
`-
`
`,
`
`Network
`
`unit!!!!
`
`Illumination
`
`108b
`
`108a
`
`103
`
`108b
`
`108a
`
`Illumination
`
`unit 104
`
`~
`
`Imaging unit
`
`Imaging unit
`
`app 112
`
`visualization
`
`Tissue
`
`Device 108
`
`Mobile
`
`'
`
`app 112
`
`visualization
`
`Tissue
`
`Device 108
`
`Mobile
`
`I'll .5
`'&
`C:
`a
`
`I ;:;
`
`.,,
`
`...,),
`
`.,, -C,
`
`Petitioner's Exhibit 1001
`Page 3 of 26
`
`

`

`UI = N
`0--, w
`0--,
`'N
`"'""'
`"'""'
`d r.,;_
`
`~
`
`....
`0 ....
`N
`.....
`rJJ =(cid:173)
`
`('D
`('D
`
`0
`
`104
`
`Imaging un1t
`
`App 112
`
`Visualization
`
`Tissue
`
`108
`
`Mobile devic
`
`105
`
`N
`N
`0
`N
`~CIO
`
`~ :-:
`~
`
`106
`
`External Systems
`
`222
`
`218
`
`Interface Unit
`
`Data 1/0 Unit
`
`~ = ~
`
`~
`~
`~
`•
`00
`
`e •
`
`216
`
`Database
`External
`
`~~--:===-
`
`220
`Unit
`
`226
`Unit
`
`Data Processing
`
`Aggregation
`
`-.....
`
`102
`
`User Device
`
`N
`
`,, -G)
`
`224
`
`Database(s)
`
`Tissue Visualization System 100
`
`.--
`
`Petitioner's Exhibit 1001
`Page 4 of 26
`
`

`

`UI = N
`0--, w
`0--,
`'N
`""""
`""""
`d r.,;_
`
`~
`
`0
`....
`0 .....
`
`~
`
`('D a
`rJJ =(cid:173)
`
`N
`N
`0
`N
`
`~
`
`~ :-:
`~
`
`QO
`
`I
`
`I
`I
`I
`
`~ = ~
`
`~
`~
`~
`•
`00
`
`e •
`
`103
`Image capturing unit
`
`j Lens 307
`
`filter
`
`305
`Emission
`
`I Connector 302
`I 1/0 unit 304
`I Lighting unit 300 I I Controller 301
`
`illumination unit 104
`
`/105
`
`j Connector 301 1~
`
`I
`
`interface
`
`I App 1/0 unit 322
`318
`Display
`
`processing
`
`unit 330
`App
`
`unit
`
`328
`Calibration
`
`unit 326
`Pre-processing
`
`unit
`
`324
`Positioning
`
`112
`visualization app
`Tissue
`
`device
`
`108
`Mobile
`
`w
`)
`-n
`
`-G
`
`■
`
`Petitioner's Exhibit 1001
`Page 5 of 26
`
`

`

`U.S. Patent
`
`Mar.8,2022
`
`Sheet 4 of 10
`
`US 11,266,345 B2
`
`Positioning mobile device
`402
`
`!
`
`Capturing image(s) 404
`
`+
`
`Pre-processing image(s) 406
`
`l
`
`Fluorescen ce
`
`Reflectance
`
`Processing image(s) 408
`
`,
`
`Extracting map 410
`
`!
`
`Generating visualization 412
`
`+
`
`!
`
`Capturing data 414
`
`i
`
`Stori ng/transm itti ng files
`416
`
`FIG. 4
`
`Petitioner's Exhibit 1001
`Page 6 of 26
`
`

`

`U.S. Patent
`
`Mar.8,2022
`
`Sheet 5 of 10
`
`US 11,266,345 B2
`
`Hello
`
`Jane Doe
`
`Save
`
`501
`
`tool
`
`Color bar 502
`
`0
`
`50%
`
`100%
`
`FIG. 5
`
`Petitioner's Exhibit 1001
`Page 7 of 26
`
`

`

`U.S. Patent
`U.S. Patent
`
`Mar.8, 2022
`Mar.8,2022
`
`Sheet 6 of 10
`Sheet 6 of 10
`
`US 11,266,345 B2
`US 11,266,345 B2
`
`PROCESSOR
`
`MEMORY
`
`UO INTERFACE
`
`Seatac
`
`FIG. 6
`
`Petitioner's Exhibit 1001
`Page 8 of 26
`
`Petitioner's Exhibit 1001
`Page 8 of 26
`
`

`

`U.S. Patent
`
`Mar.8,2022
`
`Sheet 7 of 10
`
`US 11,266,345 B2
`
`Time
`
`T
`
`FIG. 7
`
`Petitioner's Exhibit 1001
`Page 9 of 26
`
`

`

`U.S. Patent
`U.S. Patent
`
`Mar.8, 2022
`Mar.8,2022
`
`Sheet 8 of 10
`Sheet 8 of 10
`
`US 11,266,345 B2
`US 11,266,345 B2
`
`104
`
`~802
`
`104
`
`
`
`804
`✓
`
`FIG. 8
`
`Petitioner's Exhibit 1001
`Page 10 of 26
`
`Petitioner's Exhibit 1001
`Page 10 of 26
`
`

`

`U.S. Patent
`
`Mar. 8, 2022
`
`Sheet 9 of 10
`
`us 11,266,345 B2
`
`904.--.___,.
`
`-~
`903 --------------.i~ -1/.%. '.%1.__,..;,,r
`
`-
`
`901
`
`FIG. 9
`
`Petitioner's Exhibit 1001
`Page 11 of 26
`
`

`

`U.S. Patent
`
`Mar.8,2022
`
`Sheet 10 of 10
`
`US 11,266,345 B2
`
`E,p.wound
`
`~Target.,.., lmeQed
`by the de\lice
`(1001)
`
`Self-reference objed.
`
`·•'~=:\~
`Q
`
`1002
`
`1002
`
`1002
`
`0
`
`Mobile device
`screen
`
`•
`
`·
`
`Predetermlned
`markers on the
`ftaeen to guide user
`on a proper distance
`(1003}
`
`Seit-reference ob;ed
`
`FIG. 10
`
`Petitioner's Exhibit 1001
`Page 12 of 26
`
`

`

`US 11,266,345 B2
`
`1
`APPARATUS FOR VISUALIZATION OF
`TISSUE
`
`CROSS REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a National Stage Application, filed
`under 35 U.S.C. § 371, of International Application No.
`PCT/CA2019/050981, filed Jul. 16, 2019, which claims
`priority to U.S. Provisional Application No. 62/698,799,
`filed Jul. 16, 2018; the contents of both of which as are
`hereby incorporated by reference in their entireties.
`
`BACKGROUND
`
`Related Field
`
`The improvements generally relate to the field of medical
`devices.
`
`Description of Related Art
`
`5
`
`20
`
`2
`In some embodiments, a small self-reference can be used
`to position the device properly.
`In some embodiments, a small self-reference can be used
`to calibrate the measurement data based on an intensity of
`illumination.
`In some embodiments, an illumination unit independent
`of the mobile device can be used for calibration and capture
`measurements together with a camera, laptop, or tablet.
`In accordance with an aspect, there is provided a tissue
`10 imaging system comprised of a user device with a visual(cid:173)
`ization application, an image capturing unit, and an illumi(cid:173)
`nation unit. The illumination unit is configured to illuminate
`the target area; the image capturing unit captures measure-
`15 ment data, the visualization application extracts visualiza(cid:173)
`tions of tissue health indicators from the measurement data
`and generates an interface with one or more interface
`elements corresponding to the visualization of tissue health
`indicators.
`In accordance with an aspect, there is provided a tissue
`visualization system connected to a tissue imaging system
`(user device with a visualization application, an image
`capturing unit, and an illumination unit). The illumination
`unit illuminates the target area; the image capturing unit
`25 captures measurement data. The visualization application
`extracts visualization of tissue health indicators from the
`measurement data and transmits the visualization of tissue
`health indicators or underlying data to the tissue visualiza(cid:173)
`tion system. The tissue visualization system processes and
`stores the visualization of tissue health indicators or under(cid:173)
`lying data, and displays them on user devices.
`In accordance with an aspect, there is provided a portable
`illumination apparatus for facilitating visualizations of tis(cid:173)
`sue. The apparatus comprises: a portable housing for detach-
`35 able attachment proximal to an image capturing unit; and an
`illumination unit comprising one or more narrow band light
`sources configured to shine m flashes at n predetermined
`wavelengths, wherein n/4smsn.
`In accordance with a further aspect, the illumination unit
`40 further comprises a lens covering the one or more light
`sources, and having a focal length that is 80%-120% of a
`working distance between the illumination unit and a target
`area of tissue.
`In yet a further aspect, the one or more light sources is
`45 configured to provide flashes that are at least one of: (i)
`405±10 nm wavelength, and having at least one of(a) a long
`pass filter with a cut-on wavelength of 450±25 nm or (b) a
`bandpass filter with transmission in a 425 nm-1000 nm
`range; (ii) two wavelengths in a 450 nm-750 nm range, at
`50 least one of which in the green range; (iii) three wavelengths
`in a 450 nm-750 nm range, at least one of which in the green
`range; or (iv) 970±10 nm wavelength.
`In accordance with a further aspect, the illumination unit
`further comprises at least one of (i) a controller to control
`55 illumination of the one or more light sources, and (ii) a
`rechargeable battery for powering the apparatus.
`In accordance with another aspect, the one or more light
`sources are arranged along a central aperture having a radius
`of 0.5-3 cm.
`In accordance with a further aspect, the one or more light
`sources are arranged in a ring having a radius of 1.5-6 cm.
`In accordance with an aspect, the portable housing com(cid:173)
`prises a compression clip for mounting the apparatus on a
`mobile device along at least one edge of the mobile device
`65 and proximal to a camera of the mobile device.
`In accordance with another aspect, the portable housing
`comprises a spring clip for mounting the apparatus on a
`
`People suffer from chronic and compromised wounds
`with debilitating pain and reduced quality of life for those
`whose health is already compromised. Patients with this
`condition often present to a doctor at late stages of the
`disease, which leads to many amputations, which may be
`avoidable. Moreover, proper diagnostics requires special(cid:173)
`ized vascular labs, which precludes these types of tests from
`being performed outside major hospitals and in an expedited 30
`fashion.
`The wound is considered chronic if it is not healed within
`four weeks. The tissue health and wound healing process can
`be compromised by various factors, including insufficient
`blood supply, edema, and the presence of bacteria. These
`factors ( oxygenation/perfusion, subepidermal moisture, and
`bacteria presence) among others will be referred to as tissue
`health indicators.
`Multispectral (hyperspectral) imaging is a promising non(cid:173)
`invasive optical modality for early detection of problematic
`wounds.
`Visualization of skin distribution of oxyhemoglobin and
`deoxyhemoglobin can give insight into perfusion and oxy(cid:173)
`genation of the tissue. It can be used for assessment of tissue
`health (for example, ischemia).
`Such as elevated levels of subepidermal moisture are
`typical for pressure injuries, visualization of water distribu(cid:173)
`tion in tissue can be used for early (pre-ulcer) diagnostics of
`pressure injuries.
`Fluorescence imaging is a promising non-invasive optical
`modality for detection of bacterial burden. Visualization of
`bacterial burden can be used to assess bacterial burden and
`guide swabbing and cleansing.
`
`BRIEF SUMMARY
`
`In accordance with an aspect, there is provided a process
`for generating visualizations of tissue. The process captures
`measurement data by a user device (e.g., smartphone), and
`processes the measurement data using the visualization 60
`application. The process extracts indications of tissue health
`from the processed measurement data, and stores or trans(cid:173)
`mits the underlying data. The process generates interface
`elements corresponding to the visualization tissue health
`indicators.
`In some embodiments, the process involves calibrating
`the visualization application using a reference object.
`
`Petitioner's Exhibit 1001
`Page 13 of 26
`
`

`

`US 11,266,345 B2
`
`3
`mobile device along at least one edge of the mobile device
`and proximal to a camera of the mobile device.
`In accordance with a further aspect, there is provided a
`tissue imaging system for visualization of tissue health
`indicators comprising a portable computing device, an 5
`image capture unit, and an illumination unit. The illumina(cid:173)
`tion unit comprises one or more narrow band light sources
`configured to shine m flashes at n predetermined wave(cid:173)
`lengths, wherein n/4smsn. The image capture unit and the
`illumination unit are configured to capture measurement
`data for a target area of tissue. The computing device
`comprises a processor configured to access and execute
`instructions in accordance with a tissue visualization appli(cid:173)
`cation stored in a non-transitory computer-readable memory
`of the computing device, for capturing measurement data,
`and pre-processing and processing the measurement data to
`generate tissue health indicators.
`In accordance with an aspect, the computing device
`comprises a mobile device and the image capture unit is a
`camera integrated with the mobile device.
`In accordance with a further aspect, the illumination unit
`of the tissue imaging system comprises any of the embodi(cid:173)
`ments of the illumination apparatus described above.
`In accordance with yet a further aspect, the portable
`illumination unit further comprises a wireless communica(cid:173)
`tion module for receiving commands from the computing
`device.
`In accordance with a further aspect, there is provided a
`tissue visualization system operatively connected to one or
`more tissue imaging systems (such as any of the tissue 30
`imaging systems described above), comprising a communi(cid:173)
`cations module for communicating with the one or more
`tissue imaging systems, a system processor, and system
`non-transitory computer-readable memory thereon, config(cid:173)
`ured to receive measurement data and tissue health indica- 35
`tors from the one or more tissue imaging systems and to
`generate a visualization of tissue health indicators of tissue
`images received from the one or more tissue imaging
`systems, for display to a user display unit.
`In accordance with a further aspect, there is provided a 40
`method for generating visualizations of tissue. The method
`comprises: positioning a computing device at a proper
`distance from a target area of the tissue for capturing an
`image of the target area, the computing device comprising a
`processor and a non-transitory computer-readable memory 45
`storing computer-executable instructions comprising a tis(cid:173)
`sue visualization application; capturing measurement data
`using an image capturing unit and an illumination unit, the
`image capturing unit and the illumination unit communica(cid:173)
`tively coupled to the computing device and the illumination 50
`unit configured to shine m flashes at n predetermined
`wavelengths during capturing of the measurement data,
`wherein n/4smsn; pre-processing the measurement data
`using the tissue visualization application to obtain normal(cid:173)
`ized images; extracting indications of tissue health indica- 55
`tors from the pre-processed measurement data; generating
`interface elements corresponding to the visualization tissue
`health indicators; and storing and/or transmitting the indi(cid:173)
`cations of the tissue health indicators.
`In accordance with an aspect, the method further com- 60
`prises, prior to capturing the measurement data: capturing a
`reference image, wherein the positioning the computing
`device for the reference image capturing comprises posi(cid:173)
`tioning the computing device using a reference object.
`In accordance with a further aspect, the illumination unit 65
`and the computing device are configured to provide a
`working distance of 15±5 cm from the target area of tissue.
`
`4
`In accordance with yet a further aspect, the positioning of
`the computing device for capturing the measurement data
`comprises positioning the computing device using a self(cid:173)
`reference object.
`In accordance with another aspect, pre-processing com-
`prises at least one of (i) registering images to avoid camera
`motion artifacts, (ii) subtracting images with no illumination
`from the illumination unit from images with illumination
`from the illumination unit to account for the presence of
`10 ambient light, (iii) recalibrating each measurement accord(cid:173)
`ingly to control parameters related to intensity of illumina(cid:173)
`tion using a self-reference object positioned within the target
`area, (iv) dividing the intensity images on reference images
`to obtain normalized images, and/or (v) flattening the
`15 obtained images to account for reflections from curved
`surfaces.
`In accordance with an aspect, camera exposure time is T
`and a flash time is T or any whole number multiple of T.
`In accordance with another aspect, the camera exposure
`20 time is 50 ms.
`In accordance with a further aspect, the measurement data
`comprises wound-related data.
`Many further features and combinations thereof concern(cid:173)
`ing embodiments described herein will appear to those
`25 skilled in the art following a reading of the instant disclo-
`sure.
`
`BRIEF DESCRIPTION OF THE FIGURES
`
`Embodiments will now be described, by way of example
`only, with reference to the attached figures, wherein in the
`figures:
`FIG. 1 depicts a view of an example of the overall system
`architecture with various configurations of tissue imaging
`systems according to some embodiments;
`FIG. 2 depicts a view of an example of a tissue visual(cid:173)
`ization system according to some embodiments;
`FIG. 3 depicts a view of an example of the illumination
`unit and a computing device according to some embodi(cid:173)
`ments;
`FIG. 4 depicts a flowchart of an example method for
`capturing measurements and visualizing tissue according to
`some embodiments;
`FIG. 5 depicts a view of an example interface for visu(cid:173)
`alizing tissue according to some embodiments;
`FIG. 6 depicts a diagram of an example architecture of a
`computing device according to some embodiments;
`FIG. 7 depicts an example of illumination and image
`capturing scheme according to some embodiments;
`FIG. 8 depicts a view of example illumination units
`according to some embodiments;
`FIG. 9 depicts an example of the workflow used to take
`reference images; and
`FIG. 10 depicts a view of a schematic of imaging tissue
`and self-reference objects.
`
`DETAILED DESCRIPTION OF VARIOUS
`EMBODIMENTS
`
`Some clinical-grade tools can only be used in specialized
`medical establishments. They can be large, require special
`training, and are mostly suitable for the use in inpatient
`settings only. For example, they cannot be easily carried to
`a patient's home or remote communities. Thus, these solu(cid:173)
`tions cannot be used as early diagnostic tools as a patient
`would have to be referred to a hospital having one of these
`tools.
`
`Petitioner's Exhibit 1001
`Page 14 of 26
`
`

`

`US 11,266,345 B2
`
`5
`Many people suffer from diabetes. Diabetic foot ulcers
`(DFU) and the resulting lower extremity amputations are a
`frequent, disabling and costly complication of diabetes.
`Many diabetics can develop a foot ulcer. DFU is a cause of
`non-traumatic below knee amputation. In addition to the 5
`reduced quality of life, amputees might not survive for that
`long after amputation. Consequently, early detection ofDFU
`can lead to better outcomes, thus saving limbs and lives.
`Peripheral vascular disease (PVD) affects arteries (periph(cid:173)
`eral arterial disease, PAD) and veins (chronic venous insuf- 10
`ficiency, CVI). PAD is of particular importance, as it affects
`about eight million Americans and is responsible for 10% of
`all leg ulcers.
`Pressure ulcers (PU) or pressure injuries represent a 15
`serious health problem to patients impacting up to 25-50%
`of patients across acute and long-term care settings.
`The cost of treatment of diabetic foot ulcer, pressure ulcer,
`and leg ulcer is high. Diagnosing these conditions at an
`earlier stage (e.g., before actual ulceration) might result in 20
`significant financial savings for healthcare systems and
`patients.
`Other clinical indications associated with abnormal blood
`perfusion and/or oxygenation, such as skin cancer (angio(cid:173)
`genesis), port-wine stains, and skin disorders, can benefit
`from a system for tissue imaging.
`Subepidermal moisture, a measure of localized edema, is
`associated with erythema, Stage I and II PUs [Bates-Jensen
`2007, Bates-Jensen 2008, Guihan 2012, Ching 2011], and
`can (ii) differentiate between healthy skin and skin with
`pressure-induced tissue damage [Harrow 2014] and (iii)
`serve as a predictor of imminent ulceration (PU s, sDTls) in
`various populations [Bates-Jensen 2007, Bates-Jensen 2008,
`Bates-Jensen 2009]. Thus, changes in measures of subepi(cid:173)
`dermal moisture could be utilized for both prevention and
`detection of PUs. Radiofrequency impedance measurement
`with spatially separated electrodes is a current standard way
`to measure skin moisture including subepidermal moisture.
`However, it is a contact single-point measurement tech(cid:173)
`nique, which may suffer from operator inconsistency.
`Near-Infrared spectroscopy (NIR) reflectance can be used
`to determine water content in the skin. Water spectrum
`dominating NIR spectra with overtone bands of the O-H
`bonds with peak absorption at 760 nm, 970 nm (due to the
`second overtone of the O-H stretching band), 1190 nm (the 45
`combination of the first overtone of the O-H stretching and
`the O-H bending band), 1450 nm (first overtone of the
`OH-stretching band and a combination band), and 1940 nm
`( combination of the O-H stretching band and the O-H
`bending band). [Luck 1974].
`Water absorption at 1440 nm is 30 times stronger than at
`1190 nm, which in turn is more than two times stronger than
`absorption at 970 nm. Thus, 1440 nm and 1920 nm wave(cid:173)
`lengths are suitable for imaging of water content in upper(cid:173)
`most skin layers (stratum comeum), while 970 nm and 1190 55
`nm can be used for water content determination and imaging
`in deeper skin layers, including epidermis, dermis (1190 nm)
`and even subcutaneous tissues (970 nm).
`Bacteria presence can significantly impact tissue health
`and wound healing progress. Bacteria are always present in 60
`the wound. There are several distinct levels of bacterial
`burden in the wound: contamination, colonization, and
`infection.
`Wound contamination is the presence of non-replicating
`organisms in the wound. All chronic wounds are contami(cid:173)
`nated. These contaminants come from the indigenous micro(cid:173)
`flora and/or the environment.
`
`6
`Wound colonization is the presence of replicating micro(cid:173)
`organisms adherent to the wound in the absence of injury to
`the host. Most of these organisms are normal skin flora; such
`as Staphylococcus epidermidis, another coagulase negative
`Staph., Corynebacterium sp., Brevibacterium sp., Propri(cid:173)
`onibacterium acnes, and Pityrosporum sp.
`Wound Infection is the presence of replicating microor-
`ganisms within a wound that cause host injury. Primarily,
`pathogens are of concern here, such as Staphylococcus
`aureus, Beta-hemolytic Streptococcus (S. pyogenes, S. aga(cid:173)
`lactiae ), E. coli, Proteus, Klebsiella, anaerobes, Pseudomo-
`nas, Acinetobacter, and Stenotrophomonas (Xanthomonas).
`Contamination and colonization by low concentrations of
`microbes are considered normal and are not believed to
`inhibit healing. However, critical colonization and infection
`are associated with a significant delay in wound healing.
`Clinical testing for bacterial presence includes analysis of
`swabs from the tissue. In addition to long processing time
`(several days), these tests suffer from possible contamina(cid:173)
`tion during swabbing and randonmess in the selection of
`swabbing sites. Thus, current clinical diagnostics techniques
`are sub-optimal.
`Portable fluorescence imaging can be used for visualiza-
`25 tion of bacterial presence. It was found that while excited at
`405 nm, S. aureus, S. epidermidis, Candida, S. marcescens,
`Viridans streptococci, Corynebacterium diphtheriae, S. pyo(cid:173)
`genes, Enterobacter, and Enterococcus produced red (610-
`640 nm) fluorescence from porphyrin [Kjeldstad 1985]
`30 while P. aeruginosa produced a bluish-green ( 450-520 nm)
`fluorescence from pyoverdin [Cody 1987]. Thus, fluores(cid:173)
`cence imaging can be used to assess bacterial burden and
`guide swabbing and wound cleansing.
`reflectance
`Thus, multispectral/hyperspectral-based
`35 imaging, fluorescence imaging or their combination can
`provide valuable insights on tissue health and wound healing
`potential.
`Embodiments described herein can provide a tool for
`tissue imaging.
`FIG. 1 depicts a view of an example tissue visualization
`system 100 that connects to tissue imaging systems 105 via
`network 110.
`Tissue imaging system 105 is a device for visualization of
`abnormalities of blood circulation, moisture distribution,
`and bacterial burden in surface tissues (skin or mucosa). For
`example, the device can be used for identification of isch-
`emic or angiogenic conditions. It can be used by primary
`care physicians, nurses, or even patients themselves in any
`type of settings: inpatient, outpatient, long-term facilities,
`50 patient's home, and so on, thus allowing earlier identifica(cid:173)
`tion of problematic wounds. Tissue imaging system 105 may
`comprise a computing device 108 which may comprise a
`mobile device 108, processor(s) 108a, non-transitory com-
`puter readable storage medium or memory 108b, image
`capturing unit 103, and illumination unit 104. Memory 108b
`may comprise computer executable instructions comprising
`tissue visualization app 112.
`Computing device 108 may be an off-the-shell computing
`device (for example, a mobile device, smartphone, tablet,
`laptop, a personal computer) or a custom-built computing
`device. In an example embodiment, computing device 108
`comprises a smartphone.
`Tissue visualization app 112 coordinates image capturing
`unit 103 and illumination unit 104 during data capturing,
`65 process images, display results on computing device 108,
`and store and/or transmit data to tissue visualization system
`100.
`
`40
`
`Petitioner's Exhibit 1001
`Page 15 of 26
`
`

`

`US 11,266,345 B2
`
`7
`Image capturing unit 103 may comprise an internal (built-
`in to computing device 108) or external device capable of
`capturing images. In an example embodiment, image cap(cid:173)
`turing unit 103 comprises a 3 channel (RGB) or 4 channel
`(RGB-NIR) camera.
`Illumination unit 104 may comprise an internal (built-in
`to computing device 108) or external device (e.g., multi(cid:173)
`spectral flash) capable of illuminating a target area with
`required intensity, wavelengths, and duration.
`Example tissue imaging system 105 architectures are 10
`presented on FIG. 1. In some embodiments, the tissue
`imaging system 105 can be a single device. In some embodi(cid:173)
`ments, the tissue imaging system 105 can have two separate
`parts (e.g., image capturing unit 103 built-in to computing 15
`device 108 and a separate illumination unit 104, or illumi(cid:173)
`nation unit 104 built-in to computing dev