Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 1 of 26
`Case 3:20-cv-02246—DMR Document 1-2 Filed 04/02/20 Page 1 of 26
`
`EXHIBIT B
`
`EXHIBIT B
`
`

`

`THAT ARE NOT AVAT TO DO T
`
`US009820698B2
`
`A NO LO
`
`( 12 ) United States Patent
`Fonseca et al .
`
`( 10 ) Patent No . :
`( 45 ) Date of Patent :
`
`US 9 , 820 , 698 B2
`Nov . 21 , 2017
`
`( * ) Notice :
`
`( 54 ) ACTIGRAPHY METHODS AND
`APPARATUSES
`( 71 ) Applicant : KONINKLIJKE PHILIPS N . V . ,
`Eindhoven ( NL )
`( 72 ) Inventors : Pedro Miguel Fonseca , Borgerhout
`( BE ) ; Reinder Haakma , Eindhoven
`( NL ) ; Ronaldus Maria Aarts , Geldrop
`( NL ) ; Xi Long , Eindhoven ( NL )
`( 73 ) Assignee : Koninklijke Philips N . V . , Eindhoven
`( NL )
`Subject to any disclaimer , the term of this
`patent is extended or adjusted under 35
`U . S . C . 154 ( b ) by 0 days .
`( 21 ) Appl . No . : 14 / 934 , 255
`( 22 ) Filed :
`Nov . 6 , 2015
`Prior Publication Data
`( 65 )
`US 2016 / 0128641 A1 May 12 , 2016
`Related U . S . Application Data
`( 60 ) Provisional application No . 62 / 076 , 693 , filed on Nov .
`7 , 2014 , provisional application No . 62 / 101 , 408 , filed
`on Jan . 9 , 2015 .
`Int . Ci .
`A61B 5 / 02
`A61B 5 / 00
`( 52 ) U . S . CI .
`CPC .
`
`( 51 )
`
`( 2006 . 01 )
`( 2006 . 01 )
`( Continued )
`. . A61B 5 / 7278 ( 2013 . 01 ) ; A61B 5 / 0245
`( 2013 . 01 ) ; A61B 5 / 0402 ( 2013 . 01 ) ;
`( Continued )
`( 58 ) Field of Classification Search
`None
`See application file for complete search history .
`
`( 56 )
`
`References Cited
`U . S . PATENT DOCUMENTS
`2005 / 0131288 A1 *
`6 / 2005 Turner . . . . . . . . . . . . . . . . . A61B 5 / 0006
`600 / 391
`2005 / 0245790 Al * 11 / 2005 Bergfalk . . . . . . . . . . . . . . A61B 5 / 0002
`600 / 300
`( Continued )
`FOREIGN PATENT DOCUMENTS
`2428159 A2
`3 / 2012
`2011013132 AL
`2 / 2011
`4 / 2014
`2014053538 A1
`
`EP
`WO
`WO
`
`OTHER PUBLICATIONS
`Long et al , “ Analyzing Respiratory Effort Amplitude for Automated
`Sleep Stage Classification ” , Biomedical Signal Processing and
`Control , vol . 14 , 2014 , pp . 197 - 205 .
`( Continued )
`
`Primary Examiner — Ankit D Tejani
`
`ABSTRACT
`( 57 )
`An actigraphy method includes receiving a physiological
`parameter signal as a function of time for a physiological
`parameter other than body motion ( such as electrocardiog
`raphy or a respiration monitor ) , computing a body motion
`artifact ( BMA ) signal as a function of time from the physi
`ological parameter signal ( for example , using a local signal
`power signal , a local variance signal , a short - time Fourier
`transform , or a wavelet transform over epochs of duration on
`order a few minutes or less ) , and computing an actigraphy
`signal as a function of time from the BMA signal , for
`example by applying a linear transform to the BMA signal
`and optionally applying filtering such as median removal
`and / or high - pass filtering .
`
`20 Claims , 16 Drawing Sheets
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 2 of 26
`
`wwwmmwwwwwwwwxnxwwwxnmwwwmmwwwxxxwwwxnew
`
`4
`
`ECG or
`Respiratory monitor
`
`.
`
`Sensor
`data storage
`
`20
`
`Actigraphy synthesis module
`
`Bocy Movement Artifac ? ( M4 )
`versus time signal extraction
`
`w
`
`*
`
`221
`
`W
`
`{ BMA signal - - > > actigraphy signal y 24
`converter
`
`ECG or RR
`analysis module
`
`19
`
`Processed
`data storage
`
`*
`
`-
`
`- -
`
`Median removal filter
`va ! Filter
`
`High pass filter
`
`-
`
`-
`
`-
`
`-
`
`- - -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`*
`
`L2 - 26
`
`28 *
`
`*
`
`*
`
`

`

`US 9 , 820 , 698 B2
`Page 2
`
`Pawar et al , “ Analysis of Ambulatory ECG Signal ” , Proceedings of
`the 28TH IEEE EMBS Annual International Conference , 2006 , pp .
`3094 - 3097 .
`Pawar et al , “ Transition Detection in Body Movement Activities for
`Wearable ECG ” , IEEE Transactions on Biomedical Engineering ,
`vol . 54 , No . 6 , 2007 , p . 1149 - 1152 .
`Pawar et al , “ Impact Analysis of Body Movement in Ambulatory
`ECG ” , Proceedings of the 29th Annual International Converence of
`the IEEE EMBS , 2007 , pp . 5453 - 5456 .
`Pawar et al , “ Body Movement Activity Recognition for Ambulatory
`Cardiac Monitoring ” , IEEE Transactions on Biomedical Engineer
`ing , vol . 54 , No . 5 , 2007 , pp . 874 - 882 .
`Singh et al , “ Optimal Selection of Wavelet Basis Function Applied
`to ECG Signal Denoising ” , Digital Signal Processing , vol . 16 , 2006 ,
`pp . 275 - 287 .
`Addison , “ Wavelet Transforms and the ECG : A Review ” , Physi
`ological Measurement , vol . 26 , 2005 , pp . R155 - R199 .
`Popov et al , “ Computation of Continuous Wavelet Transform of
`Discrete Signals With Adapted Mother Functions ” , Proceedings of
`SPIE , vol . 7502 , 2009 , pp . 1 - 6 .
`Pinheiro et al , “ Stationary Wavelet Transform and Principal Com
`ponent Analysis Appilcation on Capacitive Electrocardiography ” ,
`The International Conference on Signals and Electronics Systems ,
`2010 , pp . 37 - 40 .
`Sakoe et al , “ Dynamic Programming Algorithm Optimization for
`Spoken Work Recognitino " , IEEE Transactions on Acoustics ,
`Speech , and Signal Processing , vol . ASSP - 26 , No . 1 , 1978 , pp .
`43 - 49 .
`Myers et al , “ A Comparative Study of Several Dynamic Time
`Warping Algorithms for Connected - Word Recognition ” , The Bell
`System Technical Journal , vol . 60 , No . 7 , 1981 , pp . 1389 - 1409 .
`* cited by examiner
`
`( 51 )
`
`( 52 )
`
`( 2006 . 01 )
`( 2006 . 01 )
`( 2006 . 01 )
`( 2006 . 01 )
`( 2006 . 01 )
`
`Int . Ci .
`A61B 5 / 0245
`A61B 5 / 0402
`A61B 5 / 11
`A61B 5 / 113
`A61B 5 / 08
`U . S . CI .
`CPC . . . . . . . . . A61B 5 / 0816 ( 2013 . 01 ) ; A61B 5 / 1118
`( 2013 . 01 ) ; A61B 5 / 1135 ( 2013 . 01 ) ; A61B
`5 / 4812 ( 2013 . 01 ) ; A61B 5 / 725 ( 2013 . 01 ) ;
`A61B 5 / 726 ( 2013 . 01 ) ; A61B 5 / 7257
`( 2013 . 01 ) ; A61B 5 / 0809 ( 2013 . 01 ) ; A61B
`5 / 4806 ( 2013 . 01 ) ; A61B 5 / 7203 ( 2013 . 01 ) ;
`A61B 5 / 7242 ( 2013 . 01 )
`References Cited
`U . S . PATENT DOCUMENTS
`2007 / 0213620 A1 *
`9 / 2007 Reisfeld . . . . . . . . . . . . . . . A61B 5 / 0402
`600 / 484
`2009 / 0149770 A1 *
`6 / 2009 Sing . . . . . . . . . . . . . . . . . . . A61B 5 / 04017
`600 / 544
`2011 / 0082355 A1 4 / 2011 Eisen et al .
`A61B 5 / 11
`2012 / 0302926 A1 * 11 / 2012 Tanaka . . .
`600 / 595
`2014 / 0088373 A1 *
`3 / 2014 Phillips . . . . . . . . . . . . . . . . A61B 5 / 113
`600 / 301
`
`( 56 )
`
`OTHER PUBLICATIONS
`Karlen et al , “ Sleep and Wake Classification With ECG and Respi
`ratory Effort Signals ” , , IEEE Transactions on Biomedical Circuits
`and Systems , vol . 3 , No . 2 , 2009 , pp . 71 - 78 .
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 3 of 26
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 1 of 16
`
`US 9 , 820 , 698 B2
`
`g
`
`OOO
`
`OOO
`
`OOO
`
`OOO
`O
`OO
`OOO
`22 i
`
`OOO
`
`OOO
`
`OOO
`
`OOO
`
`O
`
`OOO
`
`OO
`
`O
`
`OOO
`
`OOO
`
`OOO
`
`OOO
`
`OOO
`
`OOO
`
`OOO
`
`WOO
`
`CO
`
`O
`
`
`
`
`
`ooox . 000 ooo coox 600
`
`
`
`oooo ooo
`
`
`
`oooo ooo 000 0000 ooo ooo
`
`OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO
`Body Movement Artifact ( BMA )
`
`
`
`
`
`versus time signal extraction
`
`
`
`
`
`Actigraphy synthesis module
`
`gooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
`
`
`
`BMA signal - - > actigraphy
`
`signal r 24 converter
`
`
`
`
`
`Median removal filter
`
`
`
`
`
`High pass filter
`
`mooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
`
`Processed data
`storage
`
`pouvoivupuuuuuuuuuuu
`
`hun www ww www www ww www www mm ww ww ww www www mm ww www ww www ww ww ww ww www www www www ww ww www ww un www www ww www ww ww www ww ww www . med
`
`FIG . 1
`
`
`
`
`
`
`
`
`
`ooo oooo ooo 0000 000 ooo oooo ooo ooo
`
`
`
`
`
`
`
`
`
`000DDDDDDDDDDDDD
`
`OOOOOOOOOOOO
`
`oo
`
`ooo
`
`ooo ooo ooo
`
`20
`
`12
`
`
`
`ooo ooo ooo oooo ooo odes 2000 ooo ooo
`
`
`
`
`
`Sensor data
`storage
`
`
`
`ECG or RR analysis module
`
`14
`
`
`
`
`
`pe oooo ooor 2004 ooo ooo
`
`
`
`
`
`W www ww ww ww ww ww www www walu www ww www ww ww ww www www ww www www www
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 4 of 26
`
`
`ECG or Respiratory monitor
`
`Respeto content
`
`10
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 2 of 16
`
`US 9 , 820 , 698 B2
`
`01 : 38 minutes
`
`01 : 38 minutes
`
`01 : 38 minutes
`
`
`
`Hitt tilft
`
`* * * *
`
`
`
`hifth latest and attitude the
`
`
`
`
`
`Actigraphy
`
`
`
` . . . ripremu . .
`
`
`
`
`
`Respiratory effort ( thoracic )
`
`01 : 37
`
`01 : 37
`
`ECG
`
`FIG . 2
`
`01 : 37
`
`??? ? + + + ?fff????il????????????? + + + + ???????f + + + + +
`
`V
`
`Vy
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 5 of 26
`
`LI
`01 : 36
`
`a . u .
`
`?? ?
`?
`; ;
`POLX ATY
`
`LALU
`? ? ? ( * e
`
`MV x103
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 3 of 16
`
`US 9 , 820 , 698 B2
`
`Sapnulu
`
`minutes
`
`VVV
`
`minutes
`
`01 : 39
`
`- -
`
`01 : 39
`
`II .
`
`01 : 39 07 : 39
`
`1
`
`D
`
`O
`
`1
`
`O
`
`O
`
`O
`
`O
`
`O
`
`O
`
`Actigraphy
`
`01 : 38
`
`. . .
`
`
`
`Respiratory effort ( thoracic )
`
`01 : 38
`
`01 : 38
`
`
`
`power Local signal power
`Local signal
`
`
`FIG . 3
`07 : 37 * 01 : 38
`
`01 : 37
`
`- - - - - -
`
`- -
`
`- - - -
`
`01 : 36
`
`- - -
`
`- -
`
`01 : 37
`
`01 : 37
`
`-
`
`-
`
`-
`
`•
`
`-
`
`-
`
`VU
`
`w
`
`22 , 4
`
`2
`
`01 : 36
`
`* 01 : 38 01 : 36
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 6 of 26
`
`LV
`N
`
`2 . u .
`
`VWVO
`
`
`
`UV X104
`
`L . . .
`
`N
`
`
`uV8 x10
`
`

`

`
`
`
`
`BORGCONDOR CONDOR CONDOR CONDOR CONDORONGORONGOR
`
`
`
`
`
`atent
`
`Nov . 21 , 2017
`
`Sheet 4 of 16
`
`US 9 , 820 , 698 B2
`
`hours
`
`* &
`
`. .
`
`.
`
`. . .
`
`hours
`
`03 : 00
`
`* *
`
`. . .
`
`. .
`
`. . .
`
`
`
`
`
`Local signal power
`
`hours
`
`delo
`
`03 : 00
`
`SRSRSREIRO
`
`FIG . 4 ( a )
`
`Lu 03 : 00
`
`. . . . NGONGO .
`
`
`
`Respiratory effort ( thoracic )
`
`Actigraphy
`
`to as
`
`homem
`
`D
`
`BA
`
`00 : 00
`
`00 : 00
`
`00 : 00 00 : 00
`
`RSS
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 7 of 26
`
`500F
`
`a . u .
`
`W
`
`C
`
`. . . . . . .
`
`. . . . . . . . .
`
`. . . . . .
`
`?
`
`?
`
`
`
`UV X104
`
`Liit
`å ?G
`
` u V2 x108
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 5 of 16
`
`US 9 , 820 , 698 B2
`
`hours
`
`hours
`
`hours
`
`. . . .
`
`. . . . . .
`
`06 : 00
`
`Puu
`
`. . . . .
`
`. . . . . pour
`
`06 : 00
`
`06 : 00
`
`min9999 . . ' . . ' . ' . ' . ' . ' .
`11
`
`- - - - - - - -
`
`. . . . .
`
`. ' . ' . ' . ' . ' . '
`
`* * * * * *
`
`* *
`
`
`
`Respiratory effort ( thoracic )
`
`03 : 00
`
`03 : 00
`
`Actigraphy
`
`
`
`
`
`Local signal power
`
`03 : 00 FIG . 4 ( b )
`
`HHHHHHHHHHHHHHHHHHH
`
`00 : 00
`
`* *
`
`* * * * *
`
`00 : 00
`
`00 : 00
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 8 of 26
`
`500
`
`0001
`
`WRRRR
`. .
`vo
`
`COLX ARI
`
`L
`
`obstenti
`orot
`GOLX7AD
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 6 of 16
`
`US 9 , 820 , 698 B2
`
`minutes
`
`• - - -
`
`-
`
`?
`
`?
`
`-
`
`?
`
`-
`
`?
`
`-
`
`?
`
`-
`
`3 34
`
`-
`
`?
`
`-
`
`?
`
`-
`
`;
`
`•
`
`-
`
`-
`
`-
`
`-
`
`minutes
`
`6
`
`,
`
`{ } , ' < ' 1 - - - -
`
`
`
`
`
`
`
`
`
`-
`
`3 : 38
`
`11 : 38 11 : 38
`
`
`
`Respiratory effort ( thoracic )
`
`?? { 1 s
`
`- - -
`
`1 : 39
`
`- - - - - -
`
`.
`
`- - -
`
`- - - { } { : 38
`
`Actigraphy
`
`-
`
`=
`
`=
`
`?
`
`?
`
`=
`
`?
`
`1 : 3 11 : 37
`
`(
`- -
`-
`-
`-
`-
`-
`
`{ { 3 }
`
`-
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 9 of 26
`
`1 : 36
`
`~ ~ ~ ~ ~ ~ ~
`
`~
`
`? . .
`
`3g ' } { }
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`??
`* * E
`
`-
`
`-
`
`-
`
`-
`
`?????
`??
`* ???
`
`} X { f
`
`
`
`
`
`Local signal var
`
`FIG . 5
`
`12 } }
`
`3
`
`1 . ••••••••••••••••••••••• } }
`
`of
`
` 11 \ 2 x17
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 7 of 16
`
`US 9 , 820 , 698 B2
`
`minutes
`
`minutes
`
`minutes
`
`RLU
`
`1 : 12
`
`?? LEK
`
`-
`
`- -
`
`- -
`
`-
`
`-
`
`- -
`
`- -
`
`-
`
`- -
`
`-
`
`-
`
`1
`
`01 : 39 01 : 39
`
`-
`
`-
`
`-
`
`-
`
`01 : 39
`
`01 : 39
`
`Actigraphy
`
`* IAIS 01 : 38 01 : 38
`
`- -
`
`-
`
`-
`
`ECG
`
`01 : 37 01 : 37
`
`01 : 36 01 : 36
`
`-
`
`w
`
`R
`
`- . -
`
`.
`
`11 DEH
`ti :
`
`
`
`
`
`
`
`
`
`
`
`efter e t peintre Montes pertinent that time that they are not the internet et non pertinent
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`01 : 38
`
`01 : 38
`
`
`
`
`
`Local signal var
`
`FIG . 6
`
`*
`
`T
`
`01 : 37
`
`Lunnnnnnnnnnnnnnnnnn
`
`01 : 37
`
`. .
`
`We
`
`UTERI
`titi
`
`P
`
`textfit *
`
`d
`
`.
`
`01 : 36
`
`01 : 36
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 10 of 26
`
`* * * * * * * * *
`
`Amanda
`|
`
`?
`
`?
`a . u .
`
`-
`
`-
`
`Lien
`po
`po o
`COLX477
`
`www .
`mine
`- NEW
`
`
`u V2x105
`
`

`

`U . S . Paten
`
`atent
`
`Nov . 21 , 2017
`
`Sheet 8 of 16
`
`US 9 , 820 , 698 B2
`
`hours
`
`23 : 08
`
`??????????????????????
`
`? ????
`
`?? 23 : 28
`
`Actigraphy
`
`
`
`
`
`Loca siginal var { C { 3 }
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`' .
`
`' .
`
`. ' . ' . ' .
`
`. ?
`
`. ? . ? . ? . ?
`
`5808
`
`. .
`
`03 : 08
`
`FIG . 7
`
`? ??
`
`{ } { } { } { }
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 11 of 26
`
`?????
`
`!
`
`????????????
`??????????????
`
`089
`
`200
`
`400
`
`
`
`e ???
`
`?
`*
`
`.
`
`CA
`Have x105
`
`N
`
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 9 of 16
`
`US 9 , 820 , 698 B2
`
`minutes
`
`01 : 39
`
`Sainoiu
`
`01 : 39
`
`01 : 38
`
`01 : 38
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`
`
`Respiratory effort ( thoracic )
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`.
`
`
`
`STFT ( respiratory effort )
`
`FIG . 8
`
`mmmmmlung
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`01 : 37
`
`9810
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 12 of 26
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`regla
`
`e
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`V
`
`e
`
`lho
`
`
`
`UV X104
`
`WWVVVWVO
`
`01 : 37
`
`01 : 36
`
`ño
`Hz HZ
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 10 of 16
`
`US 9 , 820 , 698 B2
`
`hours
`
`hours
`
`03 : 00
`
`03 : 00
`
`
`
`STFT ( respiratory effort ) ( dB )
`
`FIG . 9
`
`Actigraphy
`
`W
`
`00 : 00
`
`00 : 00
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 13 of 26
`
`1
`CA
`
`istiwa
`
`a . u .
`
`180
`
`160
`
`De
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 11 of 16
`
`US 9 , 820 , 698 B2
`
`minutes
`
`01 : 39
`
`Seinul
`
`01 : 39
`
`01 : 38
`
`01 : 38
`
`-
`
`
`
`• wwwwwwwww
`
`-
`
`RESIDE . . .
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`?
`
`??
`
`mm
`
`
`
`Respiratory effort ( thoracic )
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`ho
`
`01 : 37
`
`
`
`CWT ( respiratory effort )
`
`FIG . 10
`
`01 : 37
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`banimivim
`VVVVUVVV
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`0
`
`UV X104
`
`V
`
`@
`
`8
`
`8
`Scale
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`01 : 36
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`01 : 36
`
`BERSERKERS . ES
`
`So . . . PRESSO
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`.
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 14 of 26
`
`+
`
`0
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 12 of 16
`
`US 9 , 820 , 698 B2
`
`01 : 40 minutes
`
`01 : 40 minutes
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`
`
`ull tits ( ????????
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`01 : 39
`
`01 : 39
`
`u
`
`uu . iuuu .
`
`-
`
`-
`
`-
`
`-
`
`to p
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`Y
`
`. ' -
`
`u
`
`-
`
`-
`
`.
`
`-
`
`-
`
`-
`
`.
`
`-
`
`-
`
`- 01 : 38
`
`
`
`Respiratory effort ( thoracic )
`
`WO
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`01 : 38
`
`-
`
`- -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`- -
`
`
`
`
`
`
`
`Max CWT ( respiratory effort ) per scale epoch
`
`FIG . 11
`
`01 : 37
`
`LELO
`
`01 : 36 01 : 36
`
`001
`
`??
`
`-
`
`- -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`.
`
`-
`
`-
`
`-
`
`.
`
`-
`
`-
`
`-
`
`-
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 15 of 26
`
`01 : 37
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`1
`
`01 : 36
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`-
`
`y
`
`-
`
`WWWWWWWWWWWW
`
`o TV
`OLX A7
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 13 of 16
`
`US 9 , 820 , 698 B2
`
`hours
`
`ARCH
`
`hours
`
`03 : 00
`
`03 : 00
`
`
`
`
`
`Max CWT ( respiratory effort ) per scale / epoch
`
`FIG . 12
`
`Actigraphy
`
`00 : 00
`
`00 : 00
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 16 of 26
`
`?
`
`o
`
`ADE Ary
`
`Ñ
`
`8
`
`8
`
`s
`8
`Scales
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 14 of 16
`
`US 9 , 820 , 698 B2
`
`minutes
`
`01 : 39
`
`minutes
`minutes
`
`01 : 39
`01 : 39
`
`VVVVV
`
`01 : 38 01 : 38
`
`
`
`?? ?? ??? ????
`
`01 : 38
`
`
`
`musand the serien 07 : 37
`
`?? ???
`
`01 : 37
`
`XXYYYYYYYYY
`
`{ { cs
`
`01 : 39
`
`01 : 38
`
`01 : 37
`
`FIG . 13
`
`
`
`
`
`Body movement estimation
`
`- -
`
`-
`
`- -
`
`
`
` 2 . 0 , X105
`
`Actigraphy
`
`
`
`Respiratory effort ( thoracic )
`
`01 : 37 01 : 37
`
`01 : 36
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 17 of 26
`
`01 : 36
`
`01 : 36
`
`eo
`r
`he _
`2 . 1 .
`
`
`OLX A11
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 15 of 16
`
`US 9 , 820 , 698 B2
`
`Sing?
`
`03 : 00
`
`Landbou
`ideal 00 : 00
`
`Actigraphy
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 18 of 26
`
`.
`
`2 . 1
`
`sinou
`
`00 : 00
`
`
`
`moyo huyo pin periyopie upisanya
`
`
`
`
`
`
`
`
`
`Body movement estimation
`
`DDDDDDT
`
`FIG . 14
`
`
`
`
`
`pas long ang 00 : 00
`
`-
`
`19 ini
`
`n n
`
`nonoonnnnnnnnnnnnnnnnnnnn
`
`
`
` a . 0 . X105
`
`

`

`atent
`
`Nov . 21 , 2017
`
`Sheet 16 of 16
`
`US 9 , 820 , 698 B2
`
`hours
`
`hours
`
`( 36 : ( (
`
`{ } { } { }
`
`???? ??
`??????????? : : ??
`
`hours
`
`{ : { } { }
`
`?
`- - - -
`
`?
`
`{ { }
`
`
`
`
`
`local signal power
`
`?????????
`
`{ } { } { } { }
`
`500
`1000
`
`??
`
`Actigraphy
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 19 of 26
`
`( 03 : 42
`
`:
`
`:
`
`:
`
`:
`
`:
`
`:
`
`:
`
`:
`
`"
`
`"
`
`"
`
`' , '
`
`?????????
`???????????????????????
`? ????
`
`} { }
`
`?
`)
`x105 ??
`
`
`
`
`
`
`
`Local signal power filtered
`
`33 : { }
`
`FIG . 15
`
`:
`
`:
`
`:
`
`:
`
`:
`
`:
`
`:
`
`:
`
`:
`
`:
`
`{ } { } { } { }
`
`x105
`
`

`

`US 9 , 820 , 698 B2
`
`physiological parameter signal as a function of time for a
`ACTIGRAPHY METHODS AND
`physiological parameter other than body motion ; and com
`APPARATUSES
`puting an actigraphy signal as a function of time from the
`BMA signal .
`The following relates generally to the medical monitoring
`The invention may take form in various components and
`arts , actigraphy arts , sleep assessment arts , and related arts . 5
`arrangements of components , and in various steps and
`Actigraphy is a relatively unobtrusive method of moni -
`arrangements of steps . The drawings are only for purposes
`toring human rest / activity / sleep cycles . The subject being
`of illustrating the preferred embodiments and are not to be
`monitored wears a small device which comprises an accel -
`construed as limiting the invention .
`erometer and which is used to measure gross motor activity .
`FIG . 1 diagrammatically illustrates an ambulatory subject
`Typically worn at the location of the wrist , the actigraphy 10
`monitoring system including an actigraphy synthesis mod
`device is mostly deployed in a wrist - watch - like form factor ,
`ule as disclosed herein .
`which is familiar , and relatively comfortable to the user .
`FIG . 2 illustrates an example of a simultaneously
`Actigraphy is gaining acceptance for ambulatory and home -
`recorded actigraphy , respiratory effort ( thoracic ) and ECG
`based sleep assessment , in the healthcare as well as the
`consumer domain . Actigraphy devices such as the Actiwatch 15 signals , with an artifact .
`product line ( available from Koninklijke Philips N . V . ,
`FIG . 3 illustrates an example of the computed local signal
`Eindhoven , the Netherlands ) are accepted clinical tools for
`power for a segment of a respiratory effort signal .
`monitoring sleep / wake patterns and to help identify and
`FIGS . 4 ( a ) and 4 ( b ) illustrate the local signal power
`monitor Circadian Rhythm Disorders , Insomnia , Shift work
`computed for two full night recordings .
`disorders , and so forth . These devices may be worn on 20
`FIG . 5 illustrates a short respiratory effort segment along
`mid - to long - term investigations , typically spanning weeks
`with simultaneously acquired accelerometer - based actigra
`or months . Actigraphy advantageously provides a time log
`phy and the local variance .
`of activity over the investigation period .
`FIG . 6 illustrates a computed local signal variance on an
`In some situations , body movements cannot be measured
`ECG signal .
`using displacement , velocity or acceleration sensors placed 25
`FIG . 7 illustrates a local signal variance on a full night
`on the person ' s body or on a support system such as a chair
`ECG recording .
`or a bed . For example , such a situation may arise for
`FIG . 8 illustrates a spectrogram representation of a seg
`monitoring systems that do not include an actigraphy device
`ment of a respiratory effort signal with a Body Movement
`on - board or readily incorporated . For example , a Holter
`Artifact ( BMA ) , along with simultaneously acquired accel
`monitor ( also known as an Ambulatory Electrocardiography 30 erometer - based actigraphy .
`device ) uses electrocardiography ( ECG ) to monitor cardiac
`FIG . 9 illustrates a spectrogram of a respiratory effort
`activity during extended periods of 24 hours or longer .
`signal for a full night recording , together with a simultane
`Based on these measurements , cardiologists or other physi -
`ously recorded accelerometer - based actigraphy signal .
`FIG . 10 presents a scalogram illustrating continuous
`cians can diagnose the presence of cardiac disorders .
`The following discloses a new and improved systems and 35 wavelet transform ( CWT ) values obtained with a db4 wave
`methods that address the above referenced issues , and oth -
`let on 128 scales for each sample of a respiratory effort
`signal segment with a BMA .
`ers .
`In accordance with one aspect , a physiological monitoring
`FIG . 11 illustrates the values obtained after taking the
`maximum CWT value for each scale within the boundaries
`device comprises a sensor configured to generate a physi -
`ological parameter signal as a function of time for a physi - 40 of each epoch ( where each epoch is delineated with dashed
`ological parameter other than body motion , and an elec -
`vertical lines in the respiratory effort plot ) .
`tronic digital signal processing ( DSP ) device configured to
`FIG . 12 illustrates CWT - based BMA versus time signal
`perform operations including : computing a body motion
`extraction results for a whole - night recording .
`artifact ( BMA ) signal as a function of time from the physi
`FIG . 13 plots an example of an accelerometer - based
`ological parameter signal , and computing an actigraphy 45 actigraphy signal ( top plot ) , respiratory effort signal with a
`signal as a function of time from the BMA signal .
`BMA ( middle plot ) and body movement estimation obtained
`In accordance with another aspect , a physiological moni -
`with the Maximum CWT coefficients for each epoch ( bot
`toring method comprises : receiving a physiological param -
`tom plot ) .
`eter signal as a function of time for a physiological param -
`FIG . 14 plots an example of an accelerometer - based
`eter other than body motion ; computing a body motion 50 actigraphy signal ( top plot ) and body movement estimation
`artifact ( BMA ) signal as a function of time from the physi
`( bottom plot ) for a full night recording .
`ological parameter signal ; and computing an actigraphy
`FIG . 15 plots an example of an accelerometer - based
`signal as a function of time from the BMA signal . The
`actigraphy signal ( top plot ) , the body motion estimation by
`computing operations are suitably performed by an elec -
`local signal power from respiratory effort ( middle plot ) and
`tronic data processing device . In some embodiments , the 55 the body motion estimation signal after filtering by a median
`operation of computing a BMA signal as a function of time
`removal filter ( bottom plot ) .
`from the physiological parameter signal comprises comput
`With reference to FIG . 1 , an ambulatory subject moni
`ing a local signal variance signal from the physiological
`toring system includes one or more physiological sensors
`parameter signal , computing a Short - Time Fourier Trans -
`10 , each of which sense a physiological parameter other than
`form ( STFT ) signal from the physiological parameter signal , 60 body movement ( displacement , velocity , acceleration ) . For
`or computing a wavelet transform signal from the physi -
`example , the one or more physiological sensors 10 may
`ological parameter signal .
`include one or more of the following sensors : an ECG
`In accordance with another aspect , a non - transitory stor -
`sensor ; an inductance plethysmography sensor ; a photopl
`ethysmography sensor ; a ballistocardiography sensor ; a
`age medium stores instructions readable and executable by
`an electronic data processing device to perform a physi - 65 nasal pressure sensor ; a thoracic impedance sensor , or so
`ological monitoring method comprising : computing a body
`forth . Each of the one or more physiological sensors 10 is
`motion artifact ( BMA ) signal as a function of time from a
`configured to measure a physiological process other than
`
`Case 3:20-cv-02246-DMR Document 1-2 Filed 04/02/20 Page 20 of 26
`
`

`

`US 9 , 820 , 698 B2
`
`versus time signal , and should be short enough that the
`body movement . For example , the physiological sensors
`temporal resolution is high , e . g . a few minutes , a few tens of
`may measure one or more of the following : cardiac activity ;
`seconds , or better .
`thoracic respiratory effort ; abdominal respiratory effort ;
`In general , body movements can influence measured
`respiratory flow ; or so forth . In the illustrative embodiment
`the one or more physiological sensors 10 include an ECG 5 physiological signals . Such influence can arise as a conse
`quence of mechanical limitations of sensing systems . For
`sensor , a respiratory sensor , or both .
`example , when measuring an ECG , body movements will
`With continuing reference to FIG . 1 , the ambulatory
`cause the skin to deform , changing its capacitance and
`subject monitoring system further includes an electronic
`impedance . ECG electrodes will sense these changes which
`data processing device 12 , for example a microprocessor ,
`microcontroller , or the like , that is programmed to by 10 will result in artifacts corresponding to large amplitude
`signals on the signal .
`suitable software or firmware to acquire samples from the
`As another example , Respiratory Inductive Plethysmog
`one or more physiological sensors 10 , store the acquired
`raphy ( RIP ) is a method for measuring respiratory effort
`sensor data in a sensor data storage 14 ( for example , a flash
`( thoracic or abdominal ) . A RIP sensor suitably includes
`memory , magnetic disk or other magnetic memory , or so 1
`elastic wires coated with conductive material , which are
`forth ) , perform optional post - acquisition sensor data pro
`sewn on elastic bands that are placed around the ribcage and
`the abdomen . The cross - sectional area of these body parts
`cessing 16 ( . e . digital signal processing , “ DSP ” ) such as
`computing ECG lead signals from electrode voltages , com
`expands and contracts due to respiratory excursion , but also
`puting heart rate ( HR ) from ECG data , computing respira -
`due to body movements . The inductance of the conductive
`tory rate ( RR ) from respiratory sensor data , or so forth , and 20 elements of the RIP is proportional to the cross - sectional
`store the post - acquisition processed data ( e . g . ECG signal
`area or the body part they enclose , and hence small and large
`lead traces , HR , RR , et cetera ) in a processed data storage 18
`body movements will both result in artifacts in the measured
`( for example , a flash memory , magnetic disk or other mag -
`respiratory effort signal .
`netic memory , or so forth ; the data storages 14 , 18 may
`Due to the mechanical properties of these sensors , certain
`optionally comprise a single physical data storage element , 25 properties of the artifacts are closely related to the intensity
`e . g . a single flash memory , configured to have logical
`or amplitude of the body movements . Typically , in the time
`mal
`domain , a higher transitional signal power is observed in the
`storage structures for the acquired sensor data and post
`presence of large movements . In the frequency domain , the
`acquisition processed data ) .
`presence of wide - band noise is observed , with an substantial
`The electronic data processing device 12 is further pro
`in 30 low - frequency component . Suitable signal processing is
`grammed to by the software or firmware to implement an 30
`employed by the BMA signal actigraphy converter pro
`actigraphy synthesis module 20 , including performing a
`cess 24 quantifies these artifacts into a measure of body
`Body Movement Artifact ( BMA ) versus time signal extrac
`movement .
`tion process 22 , performing a BMA signal to actigraphy
`In the following , some illustrative embodiments of the
`sensor signal process 24 ( where the generated actigraphy 35 BMA versus time signal extraction process 22 are described
`sensor signal is again a function of time ) , and performing
`in additional detail
`optional further processing such as illustrative median
`FIG .
`2 illustrates an example of a simultaneously
`removal filtering 26 , high pass filtering 28 , or so forth . The
`recorded actigraphy , respiratory effort ( thoracic ) and ECG
`signals , with an artifact . Note that the signals have different
`resulting BMA signal is suitably stored in the processed data
`storage 18 .
`40 sampling rates . The period of the actigraphy signal is 30
`The ambulatory subject monitoring system of FIG . 1 may
`seconds . The peak in the actigraphy signal corresponds to a
`optionally include various other features not illustrated in
`body movement that took place within a 30 - second interval
`diagrammatic FIG . 1 , such as a wired or wireless commu
`centered around the temporal location of that peak . This
`nication interface ( e . g . a USB port , Bluetooth wireless
`body movement produced observable artifacts in the respi
`interface , et cetera ) , an on - board LCD or other display 45 ratory effort and in the ECG signals as seen in the middle and
`component , buttons or other user interface features to enable
`lower plots , respectively , of FIG . 2 .
`a user to perform configuration options such as inputting
`Body movement artifacts ( BMA ) in a physiological signal
`subject identification , choosing parameters to measure ( in
`typically have different time and frequency characteristics
`embodiments in which the one or more sensors 10 include
`than the expressions of physiological processes measured by
`more than one sensor ) , choosing post - acquisition processing 50 the different sensors 10 . As such , these characteristics can be
`exploited to distinguish artifacts from the physiological
`options , et cetera .
`The BMA versus time signal extraction process 22 may
`signal being measured in process 22 , and also to quantify
`use various processing to derive this signal , such as com -
`them as a measure of body movements using the processing
`puting the local signal power in the time domain , computing
`24 . Some suitable embodiments of such processing are
`the regularity of the signal in the time domain , computing 55 described in the sequel . In general , the actigraphy signal is
`signal power in the time - frequency domain ( for example by
`derived by detecting artifacts in the sensor signal ( process
`means of a Wavelet Transform ) , computing local signal
`22 ) and performing transformation processing 24 to generate
`power in the frequency domain ( for example by means of
`the actigraphy signal . The following illustrative examples
`Discrete Fourier Transform ) , or so forth . The output of the
`process a single sensor signal , but generalization to multiple
`BMA versus time signal extraction process 22 is a BMA 60 sensor signals is straightforward : for example , given a
`signal versus time . In embodiments in which the process 22
`multi - lead ECG signal , from every lead an actigraphy signal
`employs frequency domain processing ( e . g . local signal
`is derived , and these signals are combined using suitable
`power ) , this can be achieved by performing the frequency
`data fusion techniques such as addition or averaging of the
`domain processing over a small time window ( also called an
`signals .
`" epoch ” herein ) which is of sufficiently short duration to 65
`In one illustrative example , the process 22 generates the
`approximate a signal versus time . Said another way , the time BMA signal versus time by computing local signal power .
`window or epoch affects the temporal resolution of the BMA This approach is ba