(12) United States Patent
`MOOre et al.
`
`USOO66787OOB1
`(10) Patent No.:
`US 6,678,700 B1
`(45) Date of Patent:
`Jan. 13, 2004
`
`(54) SYSTEM OF AND METHOD FOR
`TRANSPARENT MANAGEMENT OF DATA
`OBJECTS IN CONTAINERSACROSS
`DISTRIBUTED HETEROGENOUS
`RESOURCES
`
`(75) Inventors: Reagan W. Moore, San Diego, CA
`S. (EE Sea
`s
`s
`s
`(US)
`9.
`
`rr. A
`(73) Assignee: General Atomics, San Diego, CA (US)
`-
`(*) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 09/559,862
`(22) Filed:
`Apr. 27, 2000
`
`(56)
`
`2- Y-
`
`allad . . . . . . . . . . .
`
`5,819,296 A 10/1998 Anderson et al. ........... 707/204
`5,832,500 A 11/1998 Burrows ..................... 707/103
`5,832,506 A 11/1998 Kuzma ....................... 707/200
`5,864,843 A 1/1999 Carino, Jr. et al. .
`... 707/102
`5,864,864 A
`1/1999 Lerner ................
`5,870,742 A
`2/1999 Chang et al. .................. 707/8
`s: A : 3.
`st et al. ................7C.8.
`5,940,827 A * 8/1999 Hapner et al. ......
`... 707/8
`5,943,666 A 8/1999 Kleewein et al. .............. 707/2
`6,003,044 A 12/1999 Pongracz et al. ........... 707/204
`6,009,428 A 12/1999 Kleewein et al. ............. 707/10
`6,014,667 A 1/2000 Jenkins et al. .........
`... 707/10
`6,016,495 A 1/2000 McKeehan et al. ......... 707/103
`6,178,440 B1
`1/2001 Foster et al. ................ 709/201
`6.253,253 B1
`6/2001 Mason et al. ............... 709/315
`6,336,147 B1 * 1/2002 Brownell et al. ........... 709/310
`6,349,342 B1 * 2/2002 Menges et al. ............. 709/316
`6,601,195 B1 * 7/2003 Chirashnya et al. .......... 714/43
`y
`OTHER PUBLICATIONS
`Brochure on The Spring Implementation Conference; by The
`Data Warehousing Institute TM, 849-J Quince Orchard Bou
`levard, Gaithersburg, Maryland 20878, May 7-12, 2000, 3
`pages.
`Briefing. The Age of Petabytes, Red Herring, Mar. 2000,
`Selected pages (42 total).
`* cited by examiner
`Primary Examiner Shahid Alam
`(74) Attorney, Agent, or Firm-Howrey Simon Arnold &
`White, LLP
`(57)
`
`7
`(51) Int. Cl." ................................................ G06F 17/30
`(52) U.S. Cl. ..................... 707/200; 707/10; 707/103 R;
`709/201
`(58) Field of Search .......................... 707/3, 8, 10, 100,
`707/102, 103 R, 203, 205; 709/201, 203,
`310,316, 200; 714/43
`References Cited
`U.S. PATENT DOCUMENTS
`... 395/600
`5,325,524 A 6/1994 Black et al. ...
`... 395/200
`5,329,619 A 7/1994 Pagéet al. .........
`A System of and method for maintaining data objects in
`... 395/650
`5,345,586 A 9/1994 Hamala et al. ...
`containers acroSS a network of distributed heterogeneous
`... 395/600
`5,483.652 A 1/1996 Sudama et al. ...
`i. A 3.
`Ms. - - - - - - - - - 3.38 resources in a manner which is transparent to a client. A
`2 - - -a-
`a CCC. C. a.
`lient
`t
`taining t
`tai
`lved b
`5,495,607 A 2/1996 Pisello et al. ............... 395/600 Spring E.A.R.E.
`5,506.984. A 4/1996 Miller ..........
`... 395/600
`E. h one or more copies of the Sir Finant O
`5,596,744. A 1/1997 Dao et al.......
`... 395/610
`9.
`p
`5,606,693 A 2/1997 Nilsen et al. ............... 305610
`the E. play th meta data for the star SE
`5,721,916 A
`2/1998 Pardikar ..................... 305,617
`any changes made to the container as a result processing the
`5,727,145 A * 3/1998 Nessett et al. .
`... 713/200
`request, and, if a copy of the container has changed, chang
`5,727.203 A * 3/1998 Hapner et al. ...
`707/103 R
`ing the Status of the copy to indicate dirty status or Synchro
`5.737,607 A * 4/1998 Hamilton et al. ........... 709/316
`nizing the copy to one or more other copies that may be
`5,793.965 A * 8/1998 Vanderbilt et al. .......... 709/203
`present on the System.
`5,809,507 A 9/1998 Cavanaugh, III ........... 707/103
`5,819,275 A 10/1998 Badger et al. .............. 707/100
`
`ABSTRACT
`
`
`
`17 Claims, 27 Drawing Sheets
`
`-130
`(EE
`ANDF008JECT
`
`- 31
`
`i32
`
`-133
`
`-136
`
`WRITEEAA
`
`o; To
`
`upon SADT
`
`CANER
`
`-134
`
`SYNCHR032E
`WIfANY
`OTHER CPS
`
`35
`
`ARKCCPY
`AS
`BRiy T
`
`HPE, Exh. 1005, p. 1
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 1 of 27
`
`US 6,678,700 B1
`
`
`
`2.
`ars H
`9.
`2
`t
`-
`
`d
`
`d
`O
`a.
`
`Ysses
`
`V
`
`t
`e
`
`95
`
`O
`s
`
`C
`l
`2
`C
`
`HPE, Exh. 1005, p. 2
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jan. 13, 2004
`Jan. 13, 2004
`
`Sheet 2 of 27
`Sheet 2 of 27
`
`US 6,678,700 B1
`US 6,678,700 B1
`
`LYOW
`
`
`
`NOWWWOMIdd¥
`
`
`
`(LN3IT19aus)
`
`
`
`HPE, Exh. 1005, p. 3
`
`HPE, Exh. 1005, p. 3
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jan. 13, 2004
`Jan. 13, 2004
`
`Sheet 3 of 27
`Sheet 3 of 27
`
`US 6,678,700 B1
`US 6,678,700 B1
`
`
`
`xe
`
`> oo
`a 32 He
`oeFE i
`a Sc
`a_
`- 2
`aa
`a 92
`
`=2
`
`FIG.3
`
`HPE, Exh. 1005, p. 4
`
`HPE, Exh. 1005, p. 4
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 4 of 27
`
`US 6,678,700 B1
`
`d
`
`AWOW
`
`6
`
`auS3LOWSY
`
`XINNxn|SSdH|39¥INN|H3HLO|zea|a770v"0
`ablerIELGELeel
`(LNaIT9aus)\26NOMWONdd¥
`
`
`LI
`
`OL
`
`YAHLWdSid
`
`
`
`
`
`YATONVHLS3NDIYTSAI1HOH
`
`
`
`
`
`YITONVHLSANDIYT4A37MOI
`
`et
`
`
`
`SYSAIHGJdAL-8G
`
`HPE, Exh. 1005, p. 5
`
`HPE, Exh. 1005, p. 5
`
`
`
`
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 5 of 27
`
`US 6,678,700 B1
`
`Client
`
`20
`
`lsSue a Create COntainer
`Call - SrbContainerCreate().
`Input: Container Name,
`logical reSOUrce
`
`
`
`
`
`Server
`Interpret the request - Call
`SvrContainerCreate()
`
`21
`
`
`
`
`
`22
`
`Generate Container metadata
`and Store them in MCAT
`
`AllOCate reSOUrCe in the
`"permanent" resources
`
`
`
`23
`ReSOUrCe 1
`
`ReSOUrCe2
`
`
`
`
`
`
`
`Reply to client on
`SUCCeSS/failure
`
`FIG. 5
`
`HPE, Exh. 1005, p. 6
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 6 of 27
`
`US 6,678,700 B1
`
`30
`Client
`issue a Create Object Call
`SrbObjCreate().
`input. InContainer Object
`Name, Container Name
`
`Server
`Interpret the request - Call
`inContainerObjCreate()
`
`3.
`
`Query MCAT for Container
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`33
`
`
`
`An up-to-date Cache
`copy exists?
`NO
`34N Stage from "permanent”
`copy to Cache resourse
`
`Update container metadata
`35 N with MCAT to include Cache
`Copy
`
`SRB Server -
`"permanent
`COpy
`
`SRB Server 2 -
`
`reSOUCe
`
`Create metadata for
`inCOntainter object and
`register with MCAT
`
`Seek to the end of the Cache
`Copy. Allocate and fill in a
`SRB file descriptor
`
`
`
`Reply to Client
`Success - SRB File Descriptor
`Failure - error COde
`
`FIG. 6
`
`HPE, Exh. 1005, p. 7
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 7 of 27
`
`US 6,678,700 B1
`
`40
`Client
`SSUe a Create Object Write
`call - SrbObjWrite().
`Input. SRB File descriptor
`WriteSize
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`issue a SRB Object Close
`call - SrbObjectClose().
`Input: SRB File descriptor
`
`
`
`SerWer
`interpret the request - Call
`SvrObjWrite()
`
`41
`
`42
`
`SRB Server 2
`
`SOC
`
`43
`
`46
`
`SRB Server 2 -
`Cache
`eSOUCe
`
`LOCate the Opened File DeSC
`Use the parameters
`aSSOciated with the Opened
`File DeSC.
`Write to the Cache Copy
`Keep track of the object size
`and COntainer Size
`
`Reply to Client
`Success - Bytes Written
`Failure - error COde
`
`issues Call
`
`Locate the Opened File DeSC.
`• Close the Cache Copy
`Update MCAT On Object and
`COntainer sizes
`• Mark Cache Copy dirty
`
`Reply to Client
`SUCCeSS - 0
`Failure - error COde
`
`FIG. 7
`
`HPE, Exh. 1005, p. 8
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 8 of 27
`
`US 6,678,700 B1
`
`50
`Client
`issue a Sync Container call
`- SrbSyncContainer().
`input: Container Name
`Purge Flag
`
`SerWer
`Interpret the request - call
`SVrSyncContainer()
`
`5
`
`Query MCAT for Container
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`ls there a "dirty" copy
`for this COntainer?
`
`YES
`53N Replicate the "dirty" Copy
`to archiva resOurces
`
`
`
`SRB Server 1 -
`"Dirty" Cache
`COpy
`
`SRB Server 2 -
`Archival
`reSOurce 1
`
`
`
`
`
`54
`
`Update MCAT On the archival
`COpies
`Mark all copies "Not-dirty"
`
`SRB Server 3
`Archival
`reSOUrce 2
`
`Purge Flag on?
`
`Purge the "cache" Copies
`
`Reply to Client
`Success - 0
`Failure - error COde
`
`FIG. 8
`
`HPE, Exh. 1005, p. 9
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 9 of 27
`
`US 6,678,700 B1
`
`60
`Client
`issue a Open Object Call-
`SrbObjOpen().
`input: Object Name to be
`Opened
`
`Server
`Interpret the request - call
`isrvObjOpen()
`
`61
`
`Query MCAT for Object's
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Handle
`Opening Of a
`nOrmal
`object
`
`64
`
`An inCOntainer
`Object?
`
`A Cache COpy
`exist?
`NO
`Stage from "permanent" to
`Cache reSOUrCe
`
`Update Container metadata
`With MCAT to include
`Cache Copy
`
`Seek to the offset in the
`cache copy where the object
`is located.
`AIOCate and fill in a SRB
`file descriptor.
`Reply to Client
`Success - SRB File descripto
`Failure - error COde
`
`
`
`
`
`B
`
`SRB Server 1 -
`permanent
`Copy
`
`SRB server 2
`Cache
`eSOCC
`
`FIG. 9
`
`HPE, Exh. 1005, p. 10
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 10 of 27
`
`US 6,678,700 B1
`
`70
`
`Server
`Interpret the request - call
`SVrObjRead()
`
`71
`
`Client
`issue a SRB Object Read
`Call - Srb0bjRead().
`Input: SRB File descriptor
`Read size
`
`-is "
`
`More
`Read
`NO
`
`
`
`
`
`
`
`
`
`
`
`
`
`• Locate the Opened File DeSC.
`Use the parameters
`aSSOciated with the Opened
`File DeSC.
`• Read from the Cache Copy
`
`Reply to Client
`SuCCeSS-Bytes read
`Failure - error COde
`
`72
`
`SRB Server 2
`Cache
`CSOUCe
`
`73
`
`76
`
`75
`issue a SRB Object Close
`call - SrbObjectClose().
`input. SRB File descriptor
`
`
`
`Interpret the request- call
`SVrObjClose()
`
`
`
`SRB Server 2
`Cache
`Locate the opened File Desc. "
`77N Close the Cache COpy
`MCAT
`
`
`
`
`
`
`
`Reply to Client-
`Success - 0
`Failure - error Code
`
`78
`
`FIG. O
`
`HPE, Exh. 1005, p. 11
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 11 of 27
`
`US 6,678,700 B1
`
`£8
`
`98
`
`&8
`
`8
`
`88
`
`8
`
`LB
`
`08
`
`Ge’21S1-92-10-000¢
`
`dAVLSSWILASIGOW
`
`GeLb'SL-91-10-0002
`
`AWVLSSWIL3LV3u0
`
`9€°L)SL-9¢-10-0002
`
`9°21'S1-91-40-000¢
`
`0
`
`Auld3zI§3OHNOSIYWINMO3ZISXV
`Z]U0903)/OSpS'qUs/19U!e]U09/
`
`Q}u0d-]sa)
`
`qus
`
`ooo000002
`
`
`
`JWYNYANIVINOD
`
`9€°21'S1-92-10-0002
`
`
`9E21'S1-92-10-0002
`
`L2'Sb'LE-92-10-0002
`
`LeGb'91-92-10-0002
`
`L
`
`PEL8¢
`
`LtOld
`
`HPE, Exh. 1005, p. 12
`
`HPE, Exh. 1005, p. 12
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 12 of 27
`
`US 6,678,700 B1
`
`
`
`
`
`
`
`
`
`#6
`
`86Z6
`
`8
`
`1
`
`HPE, Exh. 1005, p. 13
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 13 of 27
`
`US 6,678,700 B1
`
`
`
`
`
` 9SIM-XIUN-]U09|_fealyoue|yoeyeaAuuyol|waisAsayssdy]—yoayeo-ssdyyuoo||_—_309-189)|Psswiaousu[oS3QNOOT]——3dALOSHSINVNOHSHAHd|WANH|JOUNOS3H|901SO!pOl£0LckLO}
`
`[ayes[soempede|—uraistsayyun|pueyCeu-ayaeo-1woa|2
`
`
`[ayesArea|OpsTeAOBI
`Aewud|ospsgp-spw|welshsajssdy|asps-ssdy-ud-ju09
`
`
`
`
`
`ayoeogunyna| waysdsayyxiun
`LU9}SASallyXIUN
`
`Qsps-xIuNn-Ud-Ju0d||
`
`~UISUODS|M
`
`UOSIPBU
`
`001
`
`jEAIYDIE
`
`€}“Old
`
`HPE, Exh. 1005, p. 14
`
`HPE, Exh. 1005, p. 14
`
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jan. 13, 2004
`Jan. 13, 2004
`
`Sheet 14 of 27
`Sheet 14 Of 27
`
`US 6,678,700 B1
`US 6,678,700 B1
`
`
`
`
`
`MCAT
`
`zS H
`
`S
`
`()
`
`)
`
`g
`
`s
`
`g) CL/
`
`w”
`f
`ocus
`
`>i
`
`o
`R
`2
`:
`f
`wn
`Lu
`c
`S
`O
`e
`=ia
`oS
`o
`tu
`
`2 O
`
`CY
`
`zc
`_
`2
`S5
`Se
`Sg
`<5
`So
`or
`—
`ao
`a 94.
`=z
`
`C
`St
`s
`
`
`
`FIG.14A
`
`CD
`-
`
`PE, Exh. 1005, p. 15
`
`HPE, Exh. 1005, p. 15
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jan. 13, 2004
`Jan. 13, 2004
`
`Sheet 15 of 27
`Sheet 15 Of 27
`
`US 6,678,700 B1
`US 6,678,700 B1
`
`
`
`les
`B}-
`
`MCAT
`
`coe
`
`£9)
`
`2)
`
`FIG.14B
`
`HPE, Exh. 1005, p. 16
`
`oD
`c
`ocbij
`obus
`e
`it
`wn
`2
`oe
`wo
`VO
`hu
`c
`l
`5
`Oo
`s
`=ca
`C
`O
`L
`2
`o
`
`aw
`
`i=Q
`
`o
`
`=c
`
`=
`9
`=
`x
`a.
`Qo
`x!
`2
`re al
`a
`<
`
`(SRBCLIENT)
`
`_<
`
`
`
`HPE, Exh. 1005, p. 16
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jan. 13, 2004
`Jan. 13, 2004
`
`Sheet 16 of 27
`Sheet 16 0f 27
`
`US 6,678,700 B1
`US 6,678,700 B1
`
`
`
`
`
`
`
`uo C
`
`
`ONEORMORESRBSERVERS
`
`
`t
`9
`u—!
`oO
`2
`co
`-
`Cc
`A
`wn~—
`a
`se
`
`t
`
`-oL
`
`i
`
`Wo)
`Co
`
`—_
`
`o
`
`&u
`
`APPLICATION
`
`HPE, Exh. 1005, p. 17
`
`HPE, Exh. 1005, p. 17
`
`

`

`MCAT FIG.
`ONEORMORESRBSERVERS
`
`
`2 S5 3
`<azt
`C2 ed
`Soco
`a ge
`a 92
`c
`
`s
`
`
`
`US 6,678,700 B1
`US 6,678,700 B1
`
`14D
`
`HPE, Exh. 1005, p. 18
`
`U.S. Patent
`U.S. Patent
`
`Jan. 13, 2004
`Jan. 13, 2004
`
`Sheet 17 of 27
`Sheet 17 of 27
`
`2c
`
`ax
`
`
`
`HPE, Exh. 1005, p. 18
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jan. 13, 2004
`Jan. 13, 2004
`
`Sheet 18 of 27
`Sheet 18 of 27
`
`US 6,678,700 B1
`US 6,678,700 B1
`
`
`
`Se
`S32
`ee
`
`co
`
`5s
`
`=
`
`0
`
`
`
`a
`
`ie
`2
`Gb.
`oe
`
`(s)
`y sy
`825
`
`_/
`
`
`
`LLJ
`+
`
`C5
`i
`
`ONEORMORESRBSERVERS
`
`op]
`
`
`
`
`
`
`
`x2 1
`
`=o
`2.
`' 62
`os
`ew<5
`a
`20a|
`C 5
`- s
`ofOo.
`R a 9
`
`HPE, Exh. 1005, p. 19
`
`HPE, Exh. 1005, p. 19
`
`

`

`—
`
`>a
`
`
`
`US 6,678,700 B1
`ONEORMORESRBSERVERS
`
`
`US 6,678,700 B1
`
`wo
`
`(3)
`Ors
`G
`
`LL
`~
`-
`J co
`Li
`
`HPE, Exh. 1005, p. 20
`
`U.S. Patent
`U.S. Patent
`
`Jan. 13, 2004
`Jan. 13, 2004
`
`Sheet 19 of 27
`Sheet 19 of 27
`
`
`
`
`
`iEwo
`=
`
`™
`
`ON
`
`
`
`a C 32
`S&
`be Ww
`Sa—_t
`C2 CS
`- a 9 is
`e
`e S
`
`aa
`
`
`
`
`
`oe
`
`st
`la
`
`
`
`
`
`
`(SRBCLIENT)
`APPLICATION
`
`
`HPE, Exh. 1005, p. 20
`
`

`

`m™
`ONEORMORESRBSERVERS
`
`
`s
`1a
`
`
`(SRBCLIENT)
`APPLICATION
`
`
`~
`
`@
`
`WS
`
`uw
`
`>
`
`C5
`i
`
`HPE, Exh. 1005, p. 21
`
`U.S. Patent
`U.S. Patent
`
`Jan. 13, 2004
`Jan. 13, 2004
`
`Sheet 20 of 27
`Sheet 20 Of 27
`
`US 6,678,700 B1
`US 6,678,700 B1
`
`
`
`
`
`nm
`
`:
`
`ze
`ofre
`ct
`eoco
`
`2x
`
`HPE, Exh. 1005, p. 21
`
`

`

`e
`
`ONEORMORESRBSERVERS
`
` APPLICATION
`(SRBCLIENT)
`
`(o)
`
`(3 (2)
`(82 -
`(&
`(o)
`(;
`<
`(s
`
`
`
`FIG.14H
`
`S.
`SD
`-
`
`HPE, Exh. 1005, p. 22
`
`U.S. Patent
`U.S. Patent
`
`Jan. 13, 2004
`Jan. 13, 2004
`
`Sheet 21 of 27
`Sheet 21 of 27
`
`US 6,678,700 B1
`US 6,678,700 B1
`
`
`
`s
`CD
`s
`
`c
`
`
`
`a
`zc
`ein
`s
`<5
`Oe
`ao
`a go
`
`ac
`
`
`
`s
`la
`
`
`
`HPE, Exh. 1005, p. 22
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jan. 13, 2004
`Jan. 13, 2004
`
`Sheet 22 of 27
`Sheet 22 of 27
`
`US 6,678,700 B1
`US 6,678,700 B1
`
`
`
`
`
`
`
`~N
`
`2c
`2 32 t g
`2a
`eS
`EO_I wo
`S2 C
`- A 2 5
`2 ox
`ce
`a St.
`
`nm FIG.141
` ™m
`la
`ONEORMORESRBSERVERS
` APPLICATION
`(SRBCLIENT)
`
`
`
`
`
`
`
`
`HPE, Exh. 1005, p. 23
`
`HPE, Exh. 1005, p. 23
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jan. 13, 2004
`Jan. 13, 2004
`
`Sheet 23 of 27
`Sheet 23 Of 27
`
`US 6,678,700 B1
`US 6,678,700 B1
`
`
`
`
`
`=<
`
`req>
`=
`
`-
`
` ~
`
`zeoF
`= w
`Sa
`a
`
`x
`
`~
`
`i,
`2 a. 32
`SE
`eS
`Sg
`Ooo
`- a ge
`x
`O a 92
`
`|C
`
`—
`=
`
`.SL
`
`L.
`
`(0)
`Outs
`(2
`
`Yeas
`oS
`
`(0)
`gi
`(2
`
`Ts)
`
`HPE, Exh. 1005, p. 24
`
`ed
`
`e
`
`f)
`s
`C
`O
`LU
`
`ONEORMORESRBSERVERS
`
`O
`S
`
`O
`
`e
`O
`
`HPE, Exh. 1005, p. 24
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 24 of 27
`
`US 6,678,700 B1
`
`LBS-HBO
`HZIS
`
`89 O
`
`89
`
`1. (po e o X X- N
`
`
`
`| HV8TOO+
`
`HPE, Exh. 1005, p. 25
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 25 Of 27
`
`US 6,678,700 B1
`
`120
`
`121
`
`2
`22
`
`GENERATE
`META-DATA FOR
`CONTAINER
`
`STORE IN MCAT
`
`ALLOCATE
`RESOURCES
`FOR CONTAINER
`
`FIG. 16A
`
`
`
`125
`
`OUERY MEA-DATA
`FOR
`CONTANER
`
`REPLICATE DRY
`COPY TO ALL OTHER
`RESOURCES
`
`123
`
`24
`
`FIG. 160
`
`HPE, Exh. 1005, p. 26
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 26 of 27
`
`US 6,678,700 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`130
`
`QUERY MEA-DAA
`FOR CONANER
`AND/OR OBJECT
`
`LOCATE COPY
`ON SELECTED
`RESOURCE
`
`
`
`32
`
`SAGE COPY TO
`SELECTED
`RESOURCE
`
`
`
`
`
`WRITE DATA
`OBJECT TO
`COPY
`
`136
`
`UPDATE META-DAA
`FOR
`CONTAINER
`
`
`
`SYNCHRONIZE
`WTH ANY
`OTHER COPES
`
`135
`
`MARK COPY
`AS
`DRTY
`
`FIG. 16B
`
`HPE, Exh. 1005, p. 27
`
`

`

`U.S. Patent
`
`Jan. 13, 2004
`
`Sheet 27 Of 27
`
`US 6,678,700 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`140
`
`QUERY META-DATA
`FOR
`OBJECT
`
`N
`CONTAINER2
`
`YES
`
`14
`
`NO
`
`43
`
`QUERY META-DATA
`FOR CONTANER
`AND/OR OBJECT
`
`NORMAL
`OBJECT READ
`
`142
`
`145
`
`STAGE COPY TO
`SELECTED
`RESOURCE
`
`146
`
`READ OBJECT
`FROM COPY
`
`FIG. 16D
`
`
`
`
`
`
`
`
`
`
`
`LOCATE COPY ON
`SELECTED
`RESOURCE
`
`
`
`HPE, Exh. 1005, p. 28
`
`

`

`1
`SYSTEM OF AND METHOD FOR
`TRANSPARENT MANAGEMENT OF DATA
`OBJECTS IN CONTAINERSACROSS
`DISTRIBUTED HETEROGENOUS
`RESOURCES
`
`The U.S. Government has a paid-up license in this
`invention and the right in limited circumstances to require
`the patent owner to license others on reasonable terms as
`provided for by the terms of W7405-ENG-48 awarded by
`Department of Energy.
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`This invention generally relates to managing data objects
`in a distributed, heterogenous network environment, and,
`more Specifically, to managing aggregate forms of Such data
`objects acroSS distributed heterogenous resources Such that
`the aggregate forms of the data objects are transparent to the
`USC.
`2. Background
`Many applications require access to data objects distrib
`uted acroSS heterogeneous network resources. Examples of
`Such data objects include office automation products,
`drawings, images, and electronic E-mail. Other examples
`include Scientific data related to digital images of croSS
`Sections of the human brain, digital Sky Survey image files,
`issued patents, protein Structures, and genetic Sequences. In
`a typical Scenario, data objects are generated at multiple Sites
`distributed around the country. Data objects related to a
`common topic or project are organized into a collection for
`access. If the data Sets are located at different Sites, efficient
`access usually requires gathering the data Sets at a common
`location. The resulting collection must then be archived to
`guarantee accessibility in the future. The management of
`data objects is typically complicated by the fact that the data
`objects may be housed in diverse and. heterogeneous
`computer-based Systems, including database management
`Systems, archival Storage Systems, file Systems, etc. To
`efficiently make use of these data objects, a unified frame
`work is needed for accessing the data objects from the
`numerous and diverse Sources.
`Conventional Systems for managing data include those
`depicted in U.S. Pat. Nos. 6,016,495; 5,345,586; 5,495,607;
`5,940,827; 5,485,606; 5,884,310; 5,596,744; 6,014,667;
`5,727.203; 5,721,916; 5,819,296; and 6,003,044.
`U.S. Pat. No. 6,016,495 describes an object-oriented
`framework for defining Storage of persistent objects (objects
`having a longer life than the process that created it). The
`framework provides Some core functionalities, defined in
`terms of several classes (e.g., Access Mode, Cached Entity
`Instance, TransactionManager, Distributed ThreadContext,
`and ConnectionManager) and user extensible functionalities
`that can be modified to provide access according to the
`persistent Storage being used. The concept of a “container'
`as discussed in the patent Simply refers to a logical grouping
`of class Structures in a persistent Storage environment, and
`is different from the concept of “container of the subject
`invention as can be seen from the embodiment, later
`described.
`U.S. Pat. No. 5,345,586 describes a data processing
`System consisting of multiple distributed heterogeneous
`databases. The System uses a global data directory to provide
`a logical data model of attributes and domains (type, length,
`Scale, precision of data) and a mapping (cross-reference) to
`physical attributes (and tables) residing in multiple (possibly
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,678,700 B1
`
`2
`remote) databases. The global data directory stores route (or
`location) information about how to access the (remote)
`databases. The cross-reference information is used to con
`Vert the values from the physical databases into a consistent
`and uniform format.
`U.S. Pat. No. 5,495,607 describes a network administrator
`System that uses a virtual catalog to present an overview of
`all the file in the distributed system. It also uses a rule-based
`monitoring System to monitor and react to contingencies and
`emergencies in the System.
`U.S. Pat. No. 5,940,827 describes a method by which
`database Systems manage transactions among competing
`clients who seek to concurrently modify a database. The
`method is used for maintaining cache coherency and for
`copying the cache into the persistent State.
`U.S. Pat. No. 5,485,606 describes a method and system
`for backing up files into an archival Storage System and for
`retrieving them back into the same or different operating
`System. To facilitate this function, the System writes a
`directory file, for each data file, containing information that
`is Specific to the operating System creating the file as well as
`information common to other operating Systems that can be
`utilized when restoring the file later.
`U.S. Pat. No. 5,884,310 describes a method for integrat
`ing data Sources using a common database Server. The data
`Sources are organized using disparate formats and file Struc
`tures. The method extracts and transforms data from the
`disparate data Sources into a common format (that of the
`common database server) and stores it in the common
`database for further access by the user.
`U.S. Pat. No. 5,596,744 describes a method for sharing of
`information dispersed over many physical locations and also
`provides a common interface for adapting to incompatible
`database Systems. The patent describes a Federated Infor
`mation Management (FIM) architecture that provides a
`unified view of the databases to the end user and shields the
`end user from knowing the exact location or distribution of
`the underlying databases.
`The FIM uses a Smart Data Dictionary (SDD) to perform
`this integration. The SDD contains meta-data Such as the
`distribution information of the underlying databases, their
`schema and the FIM configuration. The SDD is used to
`provide information for parsing, translating, optimizing and
`coordinating global and local queries issued to the FIM.
`The SDD uses a Cache Memory Management (CMM) to
`cache meta-data from SDD into local Sites for Speeding up
`processing. The patent describes Several Services that use the
`FIM architecture. The patent also describes methods for
`SQL query processing (or DBMS query processing).
`U.S. Pat. No. 6,014,667 describes a system and method
`for caching directory information that may include identi
`fication information, location network addresses and replica
`information for objects stored in a distributed system. These
`directory caches are located locally and used for Speeding up
`acceSS Since directory requests need not be referred to a
`remote site. The patent deals with caching of directory
`information in order to reduce traffic. The patent also allows
`for replicated data addresses to be Stored in the cache.
`U.S. Pat. No. 5,727,203 is similar to U.S. Pat. No.
`5,940,827 but is restricted to object-oriented databases.
`U.S. Pat. No. 5,721,916 describes a method and system
`for making available a Shadow file System for use when a
`computer gets disconnected from a network which allowed
`it to access the original file System. The System transparently
`copies the file from the original file System to a local System
`
`HPE, Exh. 1005, p. 29
`
`

`

`US 6,678,700 B1
`
`3
`whose structure is recorded in a local file database. When no
`longer connected to the network, the access to the file is
`redirected to the shadow file.
`U.S. Pat. No. 5,819,296 describes a method and apparatus
`for moving (migrating) large number of files (volumes) from
`one computer System to another. Included are methods for
`moving files from primary Storage to Secondary Storage and
`from one System to another System. In this latter case, the
`System copies the directory information, and the files that
`need to be migrated are manually copied. Then, the directory
`Structure merged with the new storage System. The patent
`discusses moving files residing in Volumes which are physi
`cal Storage partitions created by System administrators.
`U.S. Pat. No. 6,003,044 describes a system and method to
`back up computer files to backup drives connected to
`multiple computer Systems. A controller System allocates
`each file in a backup Set System to one or more multiple
`computers. Each of the multiple computer Systems is then
`directed to back up files in one or more Subsets, which may
`be allocated to that computer System. The allocation may be
`made to optimize or load balance acroSS the multiple com
`puter Systems.
`A problem which plagues Such Systems is the overhead
`involved in accessing archived individual data objects from
`a remote site. Remote accesses Such as this are typically
`fraught with delay caused primarily by the high latency of
`archival resources Such as tape and, to a lesser degree, the
`network latency and system overhead. This delay limits the
`effectiveness of Such Systems. To overcome the delay, the
`user might manually aggregate data objects using tools
`provided by the operating Systems or third parties, and copy
`the data to a nearby facility. However, this requires the user
`to be familiar with the physical location of the data objects
`and manner in which they are aggregated and Stored, a factor
`which further limits the effectiveness of the system.
`Consequently, there is a need for a System of and method
`for managing data objects distributed acroSS heterogenous
`resources which reduces or eliminates the delay or latency
`characteristic of conventional Systems.
`There is also a need for a system of and method for
`managing data objects distributed acroSS heterogeneous
`resources in which the physical location of and manner in
`which the data objects are Stored is transparent to the user.
`There is also a need for a system of and method for
`providing a data aggregation mechanism which transpar
`ently reduces overhead and delay caused by the high latency
`of archival resources.
`There is further a need for a system of and method for
`managing data objects distributed acroSS heterogenous
`resources which Overcomes one or more of the disadvan
`tages of the prior art.
`The objects of the subject invention include fulfillment of
`any of the foregoing needs, Singly or in combination. Further
`objects and advantages will be set forth in the description
`which follows or will be apparent to one of ordinary skill in
`the art.
`
`15
`
`25
`
`35
`
`4
`executable on a Server, for responding to a request issued by
`a client; a meta-data catalog maintained on a Server, and
`accessible by the broker, for defining data objects and
`containers, and associating data objects with containers, and
`at least one data resource maintained on one or more Servers
`for Storing data objects in containers, wherein the broker,
`responsive to a request, is configured to access the meta-data
`catalog, process the request using the meta-data catalog, and
`then update the meta-data catalog to reflect changes inci
`dental to the request, whereby data objects, once aggregated
`into containers, are maintained therein transparent to users.
`Also provided is a method of creating a logical resource
`comprising the Steps of associating one or more physical
`resources with the logical resource; for each physical
`resource, Specifying a type thereof from the group compris
`ing an archive, a cache, a primary archive, and a primary
`cache; and for each physical resource, also specifying Size
`and access control information.
`A method of creating a container is also provided com
`prising the Steps of Specifying, in response to a user request,
`a name of a container and a logical resource to be allocated
`to the container, the logical resource being associated with
`one or more physical resources, including at least one
`archive and at least one cache; creating meta-data for the
`container, including meta-data Specifying the container
`name, the logical resource to be allocated to the container,
`and the one or more physical resources associated with the
`logical resource, Storing the meta-data for the container in a
`meta-data catalog, and reserving one or more of the archives
`allocated to the container.
`The invention further includes a method of importing a
`data object into a container comprising the Steps of Speci
`fying a container; querying meta-data for the container,
`including an offset within the container, finding or Staging to
`a Selected resource a current cache copy of the container;
`Writing the data object into the cache copy at the Specified
`offset, updating the meta-data for the container to reflect
`introduction of the data object into the container; and
`marking the cache copy as dirty or Synchronizing it with any
`other copies.
`A method of Synchronizing a plurality of copies of a
`container is also included comprising the Steps of: if no
`copies of the container are marked as dirty, ending the
`method; if a cache copy of the container is marked as dirty,
`Synchronizing Such to one or more archival copies that are
`not marked as dirty; if all archival copies are thereby written
`over, resetting the dirty flags of all Such archival copies, and
`if one or more but not all archival copies are thereby written
`over, Setting the dirty flags of the one or more archives that
`are written over.
`The invention also includes a method of reading a data
`object from a container comprising the Steps of querying
`meta data for the container, including an offset where the
`data object is Stored within the contain; finding or Staging to
`a Selected resource a current cached copy of the container;
`and using the offset to retrieve the data object from the
`cached copy of the container.
`DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a simplified view of Software Resource Broker
`(SRB) middleware.
`FIG. 2 illustrates the SRB process model.
`FIG. 3 illustrates federated SRB operation.
`FIG. 4 illustrates the SRB agent design.
`FIG. 5 illustrates an implementation of a method of
`container creation in accordance with the Subject invention.
`
`40
`
`45
`
`50
`
`55
`
`SUMMARY OF THE INVENTION
`In accordance with the purpose of the invention as
`broadly described herein, there is provided a system for
`transparent management of data objects in containers acroSS
`distributed heterogeneous resources comprising: a client
`configured to issue requests relating to data objects in
`containers in response to user commands, at least one server
`accessible by the client over a network, a broker process,
`
`60
`
`65
`
`HPE, Exh. 1005, p. 30
`
`

`

`S
`FIG. 6 illustrates an implementation of a method of
`importing objects into a container, including creating an
`object in a container, in accordance with the Subject inven
`tion.
`FIG. 7 illustrates an implementation of a method of
`importing objects into a container, including writing data
`objects into a pre-existing container, in accordance with the
`Subject invention.
`FIG. 8 illustrates an implementation of a method of
`Synchronizing copies of a container in accordance with the
`Subject invention.
`FIG. 9 illustrates a first implementation of a method of
`reading data objects in a container in accordance with the
`Subject invention.
`FIG. 10 illustrates a second implementation of a method
`of reading data objects in a container in accordance with the
`Subject invention.
`FIG. 11 illustrates an implementation example of meta
`data describing a container in accordance with the Subject
`invention.
`FIG. 12 illustrates an implementation example of meta
`data describing the data objects in a container in accordance
`with the subject invention.
`FIG. 13 illustrates an implementation example of meta
`data describing the physical resources associated with a
`logical resource in accordance with the Subject invention.
`FIG. 14a illustrates multiple containers maintained in a
`System configured in accordance with one embodiment of
`the Subject invention.
`FIG. 14b illustrates multiple physical resources allocated
`to a container in a System configured in accordance with one
`embodiment of the subject invention.
`FIG. 14c illustrates replication of a container acroSS
`multiple archival resources in a System configured in accor
`dance with one embodiment of the subject invention.
`FIG.14d illustrates purging one or more cached copies of
`a container in a System configured in accordance with one
`embodiment of the subject invention.
`FIG. 14e illustrates system configured in accordance with
`one embodiment of the subject invention in which different
`client requests are handled through different archived copies
`of a container.
`FIGS. 14f14g illustrates the process of caching one or
`more of the archived container copies in the system of FIG.
`14e.
`FIG. 14h illustrates the process of writing one or more
`data objects into one of the cached container copies in the
`system of FIG. 14.g.
`FIG. 14i illustrates the process of synchronizing a dirty
`cached copy of a container with one or more archival copies
`in the system of FIG. 14h.
`FIG. 14-i illustrates transparent replication of a container
`acroSS the network in a System configured in accordance
`with one embodiment of the subject invention.
`FIG. 15a illustrates an example implementation of con
`tainer meta data associating a container with one or more
`data objects Stored in the container.
`FIG. 15b illustrates an example implementation of con
`tainer meta data associating a container with one or more
`physical resources.
`FIG. 16a illustrates a method