United States Patent
`Jastrzebski et nl.
`
`[19]
`
`IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII
`US005218530A
`[11] Patent Number:
`[45] Date of Patent:
`
`5v218v530
`Jun. 8, 1993
`
`[56]
`
`[76]
`
`[54] METHOD OF DISPLAYING AND
`ANALYZING NONLINEAR, DYNAMIC
`BRAIN SIGNALS
`Inventors: George B. Jastrzebski, 824 McGuire
`Dr., Modesto, Calif. 95355; Lowell R.
`Wedemeyer, 3112 Thatcher Ave.,
`Marina Del Rey, Calif. 90292
`21} Appl. Noc 405,331
`FEed:
`22
`Sep. 11, 1989
`Int. CLi .............................................. G06F 15/42
`51]
`!
`[52] U.S. CL ............................... 364/413.05; 395/119;
`395/125; 364/413.06
`[58] Field of Search ...................... 364/413.05, 413.06,
`364/413.07 521; 395/119, 125, 128
`References @ted
`U.S. PATENT DOCUMENTS
`8/1986 McGil! et aL ....................... 128/731
`4,603,703
`4,736,307 4/1988 Satb ................................... 364/518
`5/1988 Raviv et al..................... 364/413.05
`4,744,029
`1/1988 Freeman ................,........... 128/731
`4,753,246
`5,003,986 4/1991 Finitzo et al........................ 128/731
`8/199i Lebron et al
`5,041,973
`................. 364/413.05
`OTHER PUBLICATIONS
`Packard, N. Hu Crutchfield, J. Pu Farmer, J. Doyne;
`and Shaw R. Sq Geometry of a Time Series, Physical
`Review Letters, voL 47 p. 712 (1980).
`Wadlinger, E. Alan, A computer program to fit hype-
`rellipses to a set of phase space points in as many as six
`dimensions, Department of Energy, Los Alamos Scien-
`tific Laboratory, 1980. For sale by the National Techni-
`cal Information Services, Series Title L.Ac 8271-MS,
`Springfield, Va.
`Takens, F., Detecting Strange Arrracrars in Tiirbuleuce,
`Lecture Notes in Mathematics S98, Dynamical Systems
`and Turbulence, Warwick 1980, D. A. Rand and L. S.
`Young, Eds., (Berlin, Springer-Verlag, 1981.).
`Froehling, Hq Crutchfield, J. Pq Farmer, J. Doyne;
`Packard, N. Hq and Shaw, R. S., On Determining the
`Dimension of Chaotic Flows, Physics 3D, pp. 605-617,
`(1981).
`Farmer, J. Doyne; Ott, Ec Yorke, J. A., The Dimension
`of Chaotic Arrracrars, Physics 7D (1983) 153-180.
`Albano, A. Mc Smilowitz, La Rapp, P. Ec de Guzman,
`G. Cq Bashore, T. R., Dimension Calculariaus lu a Mim-
`ma! Embedding Space: Law-Dimensional Arrracrovs for
`Human Eleciraeacephalogrami, m Lecture Notes m
`Physics 278, The Physics of Phase Space, University of
`
`Maryland, May 20-23, 1986, Springer-Verlag, Berlin:
`New York. (Note additional references in bibliography
`of this article.).
`Ritter, H. Ju Martinez, T. Mu Schulten, K.
`J.,
`Topology— Conserving Maps for Learning Lqsua-Marar-
`Coordination, Neura! Networks, vol. 2, pp. 159-168
`(1989) (received Sep. 16, 1988; revised and accepted
`Oct. 18, 19SS).
`Is Ii Healthy ia Be Chaailc? and The Faaiprinisaf Chaos,
`Science, voL 243, pp. 604-607, Feb. 8, 1989.
`Goldberger, Ary Lc Cardiac Chaos, Science, Mar. 17,
`1989, Letters, p. 1419.
`Chaos Theory: Haw Big an Advance, Science, voL 245,
`pp 26-28, Jul 7, 1989.
`Primary Examiner—Roy N. Envall, Jr.
`Assisranr Examiner—A, Bodendorf
`Arrarney, Agent, or Firm—Lowell R. Wedemeyer
`ABSTRACf
`[57]
`The invention is a method to aid analysis of signals, such
`as electroencephalograms, pursuant to modern mathe-
`matical
`theories of nonlinear dynamical processes,
`sometimes referred to as chaotic dynamics or chaos
`theory. It employs graphic display and visual inspection
`of relatively less fdtered, non-averaged raw test data,
`including raw data heretofore considered random or
`asynchronous 'noise'. The invention enables reversible
`decomposition of selected elements of graphic portraits
`of raw signal data to identify subsets of the depicted raw
`data which correspond to visually-identified, manually-
`selected patterns from within the graphic portrait. The
`identified subsets of raw data can be segregated even
`though a precise mathematical description of the visu-
`ally identified pattern is unknown. The invention fur-
`ther comprises a variety of techniques for displaying
`four or more variables and for enhancing visual discrim-
`ination of patterns within computer generated graphic
`phase space portraits, and conceptions for overlaying
`symbols onto graphic points representing stimulus and
`response events concurring with particular signal sam-
`ples in the phase space portrait. The invention also
`comprises subsets of pattern-generating signal data
`identified by the method of the invention and thus made
`available for further computer or other operations sepa-
`rately from the full data set.
`
`27 Claims, 1 Drawing Sheet
`
`Google Exhibit 1069
`Google v. VirtaMove
`
`

`

`U,S, Patent
`U.S. Patent
`
`June 8, 1993
`June 8, 1993
`
`5,218,530
`5,218,530
`
` SAPLADNNO?
`
`WLIOQL9O7bVE
`
`WOHALNOINIG?
`
`ONYLS2L
`
`
` NOILYIOS/
`
`SHAIdWEF0/WPOBLNOI
`
`SADAVIWELON
`
`LIRBLGOS
`
`spoeisNIVSONY
`
`SINVO/LIINVOD
`
`BPIUOHLID?77P
`
`SOTVOWLS
`
`SOLVARINFEP
`
`1°Old
`
`
`
`NOMLIDSDWLLWee
`
`
`
`
`

`

`5,218,530
`
`METHOD OF DISPLAYING AND ANALYZING
`NONLINEAR, DYNAMIC BRAIN SIGNALS
`
`5
`
`There are no related applications. No federally-spon-
`sored research and development is involved.
`SUMMARY OF THE INVENTION
`The invention is a method for applying the new theo-
`ries of chaotic dynamics to brain signals. It enables a 1o
`computer operator to automatically identify in a re-
`corded stream of brain signal data the subsets of signal
`data which correspond to manually-selected patterns
`observed in multi-variable,
`reversibly-transformable,
`phase space portraits. The data selection is enabled even 15
`though no algorithm describing the observed pattern is
`known. This data selection capability is combined with
`enhanced capabilities to display and manipulate multi-
`variable phase space portraits, thereby increasing the
`ability to visually discrimmate patterns in the stream of 30
`data for selection.
`The invention draws a large number of variables into
`a graphic phase space portrait for display on a computer
`monitor. It then enables manual selection of subsets of
`drawing elements from within the displayed portrait 35
`corresponding to visually identified patterns, and auto-
`matically decomposes the selected drawing elements to
`identify the subset of signal data corresponding to the
`visually-identified patterns. Each phase space portrait
`can simultaneously display multiple variables, including 30
`three spatial coordinates corresponding to three sepa-
`rate signal detectors, plus scalar magmtude, direction of
`change, and color-coded time sequence. In principle,
`such portraits also could display another variable corre-
`sponding to drawing line type and could superpose 35
`symbols to depict stimulus and response events relative
`to the time sequence of recorded signals, though the
`current model does not implement these features.
`The invention also creates and depicts composite
`phase space portraits of larger sets of four or more 40
`signal detectors by means of overlays of simultaneous
`phase space portraits from a plurality of three-detector
`subsets. Variable colors serve to visually distinguish the
`contributions of each layer to the composite, thus en-
`abling visual discrimination of the particular subsets of 45
`detectors which provide the signals of most interest. In
`principle, variable line types also could be used to dis-
`tinguish layers.
`One aspect of the invention may be viewed as a com-
`puter-assisted manual sieve to identify from a stream of 50
`data the subsets of data which correspond to an ob-
`served pattern where no algorithm describing the ob-
`served pattern is known. The resulting subsets of pat-
`tern-containing data then are available for more inten-
`sive analysis and other operations, including efforts to 55
`define a descriptive algorithm for the observed pattern
`and efforts to create a 'template'or automated comput-
`er-aided pattern recognition.
`BACKGROUND OF THE INVENTION
`The field of this invention is analytic methods for
`depiction and analysis of brain signals. Electric poten-
`tials have been measured on scalps or with implanted
`electrodes for decades. The subjects of such measure-
`ments have been both human and animal. The current 65
`method, commonly called an electroencephalogram or
`EEG, measures and records a data stream comprised of
`the electric potentials between each of a plurality of
`
`detection electrodes distributed across the subject's
`scalp and a reference electrode attached to some other
`portion of the subject's head. A common reference
`electrode, such as one on an earlobe, may be used.
`The relationship between a given electrode and the
`reference electrode is customarily called a "channel". It
`has been recognized that the placement of the plurality
`of electrodes affects the signal collection. Therefore,
`conventions, such as the "ten-twenty", have been
`agreed upon to guide the geographic distribution of
`electrodes about the scalp for collection of EEGs for
`clinical use.
`The data stream of electric potentials from each chan-
`nel is electronically amplified to strengths suitable for
`computer analysis and recorded. In modern practice the
`data stream usually is digitized for use in digital comput-
`ers, but the data stream can be analyzed by analog com-
`puter if desired. The amplified electric potentials usu-
`ally are electronically 'filtered'o limit the collected
`to selected electronic frequencies. The data
`signal
`stream also frequently is 'averaged'o eliminate more-
`random or asynchronous data that has been assumed to
`be meaningless 'noise'.
`In digital computer analysis the data stream some-
`times is passed through a Fast Fourier Transform, or
`FFT. Less commonly,
`the data may be analyzed
`through a Mellin or 'double'ourier Transform.
`The Fourier Transform and related transforms as-
`sume as a mathematical premise that the collected signal
`can be represented as the sum of a series of sine waves.
`The Fast Fourier Transform is a computer-imple-
`mented technique employing this mathematics! prem-
`ise.
`The electroencephalogram has a number of charac-
`teristics which limit its discrimination among brain sig-
`nals. The 'filtering'nd 'averaging'oth eliminate por-
`tions of the collected signal. This elimination process
`employs the unveritied assumpuon that the data which
`is thereby eliminated is meaningless 'noise'. That is, it
`commonly is assumed that the signal of interest is 'non-
`random'nd that data which is "random" is meaning-
`less 'noise'. The filtering and averaging have as their
`purpose the extraction of the 'non-random'ignal from
`'noise'. This assumption has been driven by the practical
`necessity that no better way of analyzing and depicting
`the entire data set has existed. That is, the discarded
`data was arbitrarily treated as 'meaningless'ue to the
`lack of a method of ascertaining its meaning, but with
`no practical method of evaluating whether or not the
`discarded data in fact is intrinsically meaningless.
`A compelling practical reason for use of filtering and
`averaging is the lack of computer-implemented algo-
`rithms which can represent a mathematically true trans-
`form of the entire data set of collectible electric poten-
`tials without such filtering and averaging.
`Some filtering is designed to eliminate signals exter-
`
`nal to the subject. An example is the 60 hertz'notch'terwhich is intended to eliminate radiated signals
`
`from electric transmission lines and devices that trans-
`mit about the 60 hertz frequency. However, other filter-
`ing is employed to eliminate portions of the collected
`signal which truly emanate from the human or animal
`subject.
`An analytic disadvantage of the current computer
`techniques is that discarding portions of the raw data set
`through filtering and averaging renders the transform
`of the raw data into the displayed signal irreversible.
`That is, a depicted EEG image cannot be reversibly
`
`

`

`5,218,
`
`3
`re-transformed into the original raw data from which it
`was drawn due to the destruction of part of the original
`data set by "filtering" and "averaging". Consequently,
`mathematically precise, reproducible decompositions of
`differing graphic depictions of mathematical transforms 5
`of the identical raw data stream can not readily be ana-
`lytically verified to be true equivalents.
`In recent years new mathematical techniques have
`been developed for analysis of nonlinear dynamical
`processes, sometimes referred to as the mathematics of to
`chaos or chaotic dynamics. These new techniques
`search for patterns, and frequently for those patterns
`mathematically defined as strange attractors. These
`new techniques employ computer generated 'phase
`space portraits'o visually depict a data stream, and 15
`then attempt to infer an appropriate mathematical de-
`scription of the data stream from patterns visually de-
`tected in the graphic 'phase space portraits'. Early ex-
`amples of this technique are as follows: N. H. Packard,
`J. P. Crutchfield, J. Doyne Farmer, and R. S. Shaw, 20
`"Geometry of a Time Series", Physical Review Letters,
`47 (1980), p, 712; F. Takens, "Detecting Strange Attrac-
`tors in Turbulence" in Lecrure Notes in Marhemaries
`898, D. A. Rand and L. S. Young, eds., (Berlin: Spring-
`er-Verlag, 1981), p. 336; J. P, Crutchfield, J. Doyne 25
`Farmer, N. H. Packard, and R. S. Shaw, "On Determin-
`ing the Dimension of Chaotic Flows", Physica 3D,
`(1981), pp. 605-17.
`Recent publications disclose efforts to apply phase
`space portraits to both electrocardiograms, EKGs, and 30
`electroencephalograms, EEGs. "Is it Healthy to be
`Chaotic" and "The Footprints of Chaos", Science, Vol.
`243, pp. 604—607, 8 Feb. 1989. "Chaos Theory: How Big
`an Advance", Science, Vol. 245, pp. 26-28, 7 Jul. 1989.
`THE PROBLEMS ADDRESSED BY THE
`INVENTION
`A problem experienced in efforts to apply the new
`mathematics of nonlinear or chaotic dynamics to brain
`signals is that the signals must be taken at extremely 40
`short time intervals, on the order of milliseconds or less,
`resulting in an extremely high volume of data in a very
`short period of time. This necessitates use of a computer
`to collect and record such signals and to correlate the
`data stream with stimulus and response events.
`The signals recorded from scalp electrodes appear to
`be a complex composite of several different biological
`processes of poorly defined origin. Signals reflecting
`muscular processes, including heartbeat, breathing and
`voluntary muscle contraction, are mixed in with and to 50
`a large extent obscure other signals of interest concern-
`ing brain function. In addition, signals from multiple
`processes within the brain itself may form a portion of
`the composite signaL Furthermore, the electrodes, or
`the subj«ct's body, may also be acting antennas collect- 55
`ing signals from the environment.
`Prior methods of extracting signals from the back-
`ground have gen«ral!y employed some form of averag-
`ing to limit the signal to nonrandom patterns, thereby
`deliberately discarding more random data from the 60
`signal. These prior methods also employ 'artifact rejec-
`tion'hich in practice means that signals exceeding a
`pre4efined amplitude are assumed to be "artifact", such
`as the signal of a muscular movement, and are "re-
`jected" or deleted from the data stream before averag- 65
`ing. According to the new mathematics of nonlinear or
`chaotic dynamics it may be postulated that such 'ran-
`dom'nd 'artifact'ata is in fact part of the genuine
`
`35
`
`45
`
`530
`
`brain signal, the meaning of which must be deciphered
`to understand the entire signal. Under these postulates,
`"averaging" to eliminate non-random portions of the
`signal eliminates chaotic data which is in fact a part of
`the true signal of brain function. In particular, averag-
`ing may obscure the transitions from chaotic to ordered
`states, and vice versa. However, programmable a)go-
`rithms describing such chaotic data sufficient to employ
`automated computer pattern recognition or signal anal-
`ysis have not yet been found.
`Thus, the problem is to identify nonlinear dynamic
`characteristics of cognitive and other brain signals
`which distinguish such elements from the composite
`signal when no descriptive, programmable algorithms
`are known. Since the characteristic attributes of signals
`denoting cognitive brain functions are not yet known,
`empirical tools are needed to search for such distinctive
`characteristics so that such signals can be extracted
`from the background data. The invention is conceived
`as such a tool.
`It is known that patterns sometimes can be visually
`recognized in phase space portraits even though such
`patterns cannot be described with mathematical preci-
`sion suAicient to defme an algorithm for automatic
`computer recognition of such patterns. The invention is
`designed to allow manual screening of raw data for
`visually recognizable patterns, manual selection of the
`drawing elements which form such patterns, and identi-
`fication of the raw data which is reflected in the se-
`lected patterns.
`Because the subset of raw data so identified contains
`the visually recognized pattern extracted to some de-
`gree from the background data, that subset of data can
`be subjected to more intensive and more eflicient pro-
`cessing and analysis to find the best mathematical de-
`scription to describe the observed pattern.
`Once a pattern has been visually recognized and the
`data including that pattern segregated, it is possible to
`guide the search for an algorithm describing the pattern
`by reference to the known phase space portraits of a
`variety of mathematical formulae. See "Geometry from
`a Time Series".
`It is known that patterns sometimes may appear in
`graphic phase space portraits of data reflecting nonlin-
`ear dynamic processes only when a sufficiently large
`number of variable dimensions is
`reflected in the
`graphic portrait. For this reason it is desirable in phase
`space portraits to visually depict as many variable di-
`mensions as can be achieved.
`OBJECTIVES AND FEATURES OF THE
`INVENTION
`A feature of the invention is the methods employed to
`increase the number of visually distinctive dimensions
`which can be displayed in a mathematically precise
`graphic portrait. In addition to the three physical di-
`mensions which can be disp)ayed in Cartesian coordi-
`nates, the invention enables other visually4istinctive
`dimensions by use of layers to create composite phase
`space portraits, use of colors to denote time sequence,
`use of colors to distinguish between the layers in a com-
`posite portrait, and use of time-linked color sequences
`to visually display time sequence and to seek periodicit-
`ies within graphic phase space drawings. In principle,
`distinctive line types also could be employed to distin-
`guish between layers in a composite portrait.
`The power of the invention to point out subsets of
`raw data corresponding to visually-identified patterns
`
`

`

`5,218,530
`
`thus is combined with enhanced ability to display pat-
`terns which anses from the capacity to graphically
`depict a large number of variable dimensions.
`It is known that patterns sometimes can be made to
`appear more visibly distinctive when the phase space
`diagram is rotated in three dimensions, or otherwise
`manipulated through mathematically precise transfor-
`mations. It is a feature of the invention that the graphic
`portraits can be passed into commercially available
`computer-aided drawing, design and engineering pro-
`grams wherein they can be rotated, viewed in mirror
`image, and otherwise viewed after mathematically pre-
`cise, reversible transformations.
`It is an objective of the invention to enable analytic
`depiction of the 'raw'mplified stream of electric po-
`tentials collected from a plurality of electrode channels,
`while reducing filtering of the raw data stream and
`eliminating the mathematical processing called 'averag-
`ing'. It is a feature of the invention that it enables ana-
`lytic depiction of the entire data stream,
`including
`within the analytically depicted data set so-called 'ran-
`dom'ata which prior analytic methods discarded as
`non-analyzable.
`It is an objective of the invention to enable testing of
`the assumption that apparently 'random'rain signal
`'noise'r
`data, heretofore eliminated as
`'artifact'hroughfiltering and averaging, are meaningless, It is a
`
`feature of the invention that it reflects visually detecti-
`ble patterns in the data stream, without first imposing
`patterns on the data by the assumption that it can prop-
`erly be represented by a Fourier transform. It also dis-
`plays signals of large magnitude heretofore rejected as
`"artifact".
`It is a further objective of the invention to enable the
`depiction of brain signal data in more than three dimen-
`sions, e.g, more than wave amplitude and phase over
`time as previously enabled by the continuous sine-like
`wave in an electroencephalogram. It is a feature of the
`invention that it enables depiction of change over time
`of a unique point in two or three dimensional space
`defined by simultaneously-recorded electric potentials,
`respectively, of two or three different electrode chan-
`nels. It is a further feature of Applicant's invention that
`a single drawing line entity can simultaneously depict
`three drawing coordinates, a direction, an amplitude
`corresponding to line length, a unique color and a
`unique line type. Two successive drawing line entities
`also form an angle, wluch may be employed in pattern
`characterization. Each drawing line entity can be
`placed within a color-coded time sequence in the phase
`space portrait. In principle, symbols denoting stimulus
`and response events could also be superposed upon the
`time sequence. In principle, another variable dimension
`could be depicted by use of unique line types. See Auto-
`CAD Manual, Sec. 7.9, et seq., pp. 192-195.
`It is a further feature of the invention that it enables
`composite depiction of a plurality of layers wherein
`each such layer reflects the data stream from a unique
`different subset of two or three signal channels drawn
`from a larger set of channels. For example, from a set of
`four electrodes, layer one can depict a three~lectrode
`combination 1,2,4; layer 2 can depict electrode combi-
`nation 3,2,4; layer 3 depict electrode combination 2,1,3;
`and layer 4 depict electrode combination 4,1,3. In this
`example, a compound visual image can be assembled by
`overlays of layers 1 through 4, or any subset of them.
`A particular layer might be thought of as a "slice"
`through the skull on a plane defined by the physical
`
`placement of the three detection electrodes, relative to
`the reference electrode, whose data streams are em-
`ployed to define that particular layer. The composite
`graphic phase space portrait formed by the overlay of
`two or more layers might be thought of as a series of
`slices cut at different angles through the skull.
`Each layer in a composite image can be assigned a
`different color so that the comparative contributions of
`different layers to the composite image can be visually
`10 distinguished on a color computer monitor. This ena-
`bles rapid visual focus on those layers which produce
`the most dramatic display of distinctive patterns. Be-
`cause layers can be turned on and off in any combina-
`tion, the most dramatic presentations can quickly be
`identified, while parsimony in data presentation can be
`achieved by turning off those layers which contribute
`least to visual discrimination of patterns.
`The limitation on the number of electrode combina-
`tions which can be so depicted in a composite image is
`practically limited by the number of channels recorded,
`by the capacity of the computer, and by the graphic
`capacity of its monitor, but not by the analytic method.
`The permutations of electrode combinations which can
`be depicted rises as a function of the number of elec-
`trodes recorded.
`It is a feature of the invention that it enables visual
`inspection for empirically reproducible patterns in a
`stream of signal data, with less restrictive assumptions
`30 than previously employed concerning the mathematical
`formula which will best describe the data stream. That
`is, patterns are allowed to manifest themselves in the
`graphic portrait even though algorithms which describe
`the visible patterns are unknown.
`It is a feature of the invention that it enables mathe-
`matically reversible transformations of the raw data set,
`without discarding any of the data set. It is a feature of
`the invention that it enables reversible transforms of the
`identical raw data set into a wide variety of mathemati-
`cally-comparable, visually-inspectible graphic portraits.
`The transformations available in commercial CAD soft-
`ware include mirror imaging, and three-dimensional
`rotations. See AutoCAD Manual, Sec. 5.25, p. 117 and
`Chap. 14, pp, 309-311; AutoCAD Reference Manual
`45 Supplement, Release 9.0, Sec. 1.13, p. 9. This enables
`identification of those transforms which provide more
`distinctive depictions of unique features or patterns in
`the data set.
`It is a feature of the invention that distinctive drawing
`50 "entities" depicted in the visual image can be "selected"
`and decomposed with mathematical precision into the
`subset of the raw data from which the selected drawing
`entity was created. For example, a drawing structure
`from within a phase space portrait displayed on the
`5» computer monitor can be selected by pointing with a
`computer mouse to the drawing entities which compose
`the structure. A computer program specially developed
`by the Inventors for this purpose then identifies the
`precise subset of raw data points upon which the se-
`60 lected drawing structure is based. See the PICK.LSP
`program listing appended hereto.
`Visually identified distinctive patterns can be ex-
`tracted as a "block" from the displayed phase diagram
`and decomposed into the subset of raw data which
`65 produced that block,
`thus enabling visually&irected
`identification of pattern-producing subsets of the raw
`data. See the GRAB.LSP program listing appended
`hereto.
`
`

`

`5,218,530
`
`Identification of the raw data point which produced a
`pattern enables segregation of that subset of raw data
`for more eflicient, intensive analysis to find a descrip-
`tive algorithm. This feature of the invention enhances
`efforts to focus the search for a descriptive algorithm on
`classes of mathematical formulae whose graphs are
`known to approximate the pattern segregated from the
`phase space portrait. See "Geometry from a Time Se-
`ries", above.
`A feature of the invention allows correlation of a
`subset of raw data with presentation of an external stim-
`ulus to the brain being monitored. "Event flags" or
`markers, indicating by their content the type of stimulus
`and by their location in the data sequence the time of
`presentation of stimuli to the human or animal whose
`brain signals are being recorded, could be inserted into
`the recorded stream of raw data. For example, if there
`were four detector channels being simultaneously re-
`corded in parallel, the recorded computer data file
`would contain sets of four data points each time the
`detectors are simultaneously sampled, one point for
`each detector. The four-point data sets would be iter-
`ated for as many sampling times as desired. The data
`sets could be expanded to six points per set, assigning
`one additional point to stimulus events and the other
`additional point to response events. When a distinctive
`drawing structure displayed on screen was decomposed
`into the raw data it then could be temporally related
`with mathematical precision to stimulus and response
`events through inspection of the stimulus and response
`data points embedded in the raw data stream. These
`same flags embedded m the data stream could also be
`employed to place a distinctive marker in the graphic
`phase portrait on or close to the point entity formed
`from the signals in the data set in which the flag ap-
`pear's.
`A feature of the invention is that the elements of the
`graphic display of the data may be considered virtual
`vectors calculated from the data streams of electrodes
`which are geographically dispersed over the subject's
`scalp. That is, a given drawing line entity possesses both
`a scalar magnitude and a direction. This allows infer-
`ences to be drawn from such virtual vectors about the
`geographic distribution within the head of the electric
`phenomena which are being graphically depicted.
`A further feature of the invention is that a series of
`permutations, comprised of combinations of different
`sets of three electrodes drawn from a larger sct of elec-
`trodes distributed over the whole head, can be depicted
`as overlays in a composite image. The resulting com-
`posite image reflects the virtual vectors of a broader
`geography of the bead than can a subset of only three
`electrodes, thus enabling the drawing of more sophisti-
`cated inferences about the geographic distribution of
`phenomena within the head.
`A feature of the invention is that the data stream can
`be depicted as a 'tree'tructure emanating radially from
`the center of a three dimensional phase space. Alterna-
`tively, the data stream can be depicted as emanations
`from corners of a cube along the cube's walls and within
`the interior space of the cube. The 'tree'r center-based
`structure tends to reduce overlapping of the larger mag-
`nitude elements of the drawing structure making such
`larger magnitude images more readily distinguishable.
`The cube corner structure tends to reduce overlapping
`near the origins at the corners of the cube thus making
`lower amplitude drawing structures near the origins
`more readily discernable.
`
`BRIEF DESCRIPTION OF DRAWINGS
`FIG. I is a schematic diagram of a signal collection
`system.
`
`THE INVENTION
`The invention employs specially developed computer
`programs to format a stream of raw electronically-
`recorded brain signal data into ASCII DXF, or Draw-
`lo ing eXchange Files. See the DXFROW.EXE and
`DXFBOX.EXE computer program listings appended
`hereto. A data "point entity" in three dimensional space
`is defined by assigning from the raw data stream the
`recorded substantially simultaneously from
`signals
`35 three different detection electrodes as the 'x', 'y', and
`
`'z'oordinates,respectively. For example, the potential
`
`is the 'x'imension, that
`recorded from electrode I
`from electrode 2 is the 'y'imension, and that from
`electrode 4 is the 'z'imension,
`thereby defining a
`20 unique point in the three dimensional phase space. A
`"line entity" then is defined as the line connecting two
`successive data "point entities". These point and line
`entities are defined within recognized conventions for
`graphic computer displays, such as the Drawing eX-
`25 change Format standard or DXF convention. Alterna-
`tively, various curved drawing entities instead of lines
`the data points, The
`could be employed to connect
`drawing "line entity" so detined can be thought of as a
`virtual vector reflecting the scalar value and the direc-
`30 tion of the change over time of the electric potential
`from one data point to the succeeding data point. It is,
`of course, possible in principle to transform the drawing
`structures from Cartesian coordinates to polar coordi-
`nates, though the current software does not do so.
`35 When so formatted the signal data can be imported
`into commercially available computer-aided design or
`CAD computer software programs. Formatting in ac-
`cord with the ASCII DXF definition system enables
`mathematically reversible graphic display of the raw
`40 data in a wide variety of engineering computer-aided
`design programs such as AutoCad (R). It further ena-
`bles manipulations of the raw data stream through three
`dimensional rotations and other image transformations
`using the capabilities of commercially available CAD
`45 programs. In principle,
`the formatted data could be
`translated into Initial Graphics Exchange Standard
`fiGES) files for use on other systems. See AutoCAD
`Manual, Sec. C.3, p. 383.
`A variety of capabilities within such commercial
`50 CAD software is employed to enhance the number of
`visually displayed variable dimensions within the phase
`space portrait. From within the commercial CAD pro-
`gram a computer mouse is employed to manually point
`out the drawing entities within a graphic portrait which
`55 form visually identified patterns. A set of commands
`within the CAD program then forms a 'selection-set'f
`the identified drawing entities.
`The invention then employs specially developed
`computer programs to decompose the 'selection-set'f
`60 drawing entities to identify the subset of raw signal data
`from which the manually-selected patterns were con-
`structed. See the PICK.LSP computer program listing
`appended hereto.
`DETAILED DESCRIPTION
`"Signal detector" means a device for detection of
`phenomena of interest such that signal data can be col-
`lected through the device. A signal detection device in
`
`

`

`5,218,530
`
`5
`
`10
`
`the case of electroencephalograms means metallic elec-
`trodes; and the signal is the electric potential measured
`between a pair of electrodes, one attached to the scalp
`and compared to a reference electrode attached else-
`wh