United States Patent (19)
`Kauffman et al.
`
`USOO5857203A
`Patent Number:
`11
`(45) Date of Patent:
`
`5,857,203
`Jan. 5, 1999
`
`54 METHOD AND APPARATUS FOR DIVIDING,
`MAPPING AND STORING LARGE DIGITAL
`
`5,454,103 9/1995 Coverston et al. ..................... 707/205
`5,497,463 3/1996 Stein ..............
`395/200.33
`
`OBJECTS IN A CLIENTISERVER LIBRARY
`
`SYSTEM
`
`75 Inventors: Steven Victor Kauffman, San Jose;
`Lara Marie Lewis, Campbell; Ronald
`Elliott Parrish, San Jose, all of Calif.
`
`73 ASSignee: stations ins.thines
`
`21 Appl. No.: 688,116
`
`Jul. 29, 1996
`22 Filed:
`(51) Int. Cl. ................................................ G06F 17/30
`52 U.S. Cl. ..................
`707/200; 707/10; 707/103
`58 Field of Search ................... 707/103, 10; 395/610,
`395/614
`
`56)
`
`References Cited
`U.S. PATENT DOCUMENTS
`4,092,732 5/1978 Ouchi ................................. osso
`5.175.851 12/1992 Johnson ...................................... 707/8
`5,367,698 11/1994 Webber ...
`395/200.33
`5,371,885 12/1994 Letwin .................................... 707/205
`5,375,128 12/1994 Menon et al. ...
`371/40.15
`5,448,727 9/1995 Annevelink ............................. 707/101
`
`
`
`14
`
`5,522,077 5/1996 Cuthbert - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 395/683
`
`370/422
`7/1996 Nilakantan et al.
`5,541,911
`5,555,388 9/1996 Shaughnessy ........................... 711/100
`5,649,185 7/1997 Antognini et al. .......................... 707/9
`5,649,196 7/1997 Woodhill ................................. 707/204
`OTHER PUBLICATIONS
`Abraham Silberschatz et al., “Operating System Concepts”,
`Addison-Wesley Publishing Company, Apr., 1989.
`Primary Examiner Wayne Amsbury
`ASSistant Examiner Diane D. Mizrahi
`Attorney, Agent, or Firm Sughrue, Mion, Zinn, Macpeak
`& Seas, PLLC
`57
`
`ABSTRACT
`-
`The present invention is directed to a method and apparatus
`for Supporting large digital objects. More particularly it
`relates to a client/server library System in which a large
`digital object is divided into Smaller pieces which are Stored
`in the client/server library System. By dividing the large
`digital object into Smaller pieces, System limitations regard
`ing object size are avoided. In one embodiment of the
`present invention the pieces are Stored as files of variable
`size. Moreover, the pieces can be Stored on Separate object
`Servers which can be located remotely from one another.
`
`36 Claims, 5 Drawing Sheets
`
`-1-
`
`Amazon v. Audio Pod
`US Patent 9,729,907
`Amazon EX-1015
`
`

`

`U.S. Patent
`
`Jan. 5, 1999
`
`Sheet 1 of 5
`
`5,857,203
`
`
`
`FIG.1
`
`Object Servers
`
`C
`
`%
`CLENT
`/é
`/ ( S) \
`d
`
`Neity
`
`30
`
`-2-
`
`

`

`U.S. Patent
`
`Jan. 5, 1999
`
`Sheet 2 of 5
`
`5,857,203
`
`
`
`-- Request Response
`
`=> Blob
`
`-3-
`
`

`

`5,857,203
`5,857,203
`
`U.S. Patent
`U.S. Patent
`
`Jan. 5, 1999
`Jan. 5, 1999
`
`Sheet 3 of 5
`Sheet 3 of 5
`
`FIG.3
`
`g
`
`
`
`-4-
`
`-4-
`
`

`

`U.S. Patent
`U.S. Patent
`
`
`
`Jan. 5, 1999
`Jan. 5, 1999
`
`5,857,203
`
`Sheet 4 of 5
`Sheet 4 of 5
`
`5,857,203
`
`-5-
`
`-5-
`
`

`

`Jan. 5, 1999
`Jan. 5, 1999
`
`Sheet 5 of 5
`Sheet 5 of 5
`
`5,857,203
`5,857,203
`
`U.S. Patent
`U.S. Patent
`
`
`
`O)
`
`s
`FIG.5
`
`2 to do b (b. c.
`LG
`3. Ca
`or
`
`wa
`
`8m
`
`e)
`On
`
`BSco
`
`Aon
`
`-6-
`
`

`

`5,857,203
`
`1
`METHOD AND APPARATUS FOR DIVIDING,
`MAPPING AND STORING LARGE DIGITAL
`OBJECTS IN A CLIENTISERVER LIBRARY
`SYSTEM
`
`BACKGROUND OF THE INVENTION
`
`2
`Objects stored in the library system can be checked out by
`a patron for a specified period of time. This feature can be
`used to ensure that one patron's updates to an object are not
`overwritten by another. While an object is checked out by a
`patron, other patrons can retrieve the object and view it, but
`they cannot update it. In typical implementations, there are
`groups of individuals who need access to the Same objects.
`Therefore, to Simplify the process of granting access to
`objects a System administrator can define patrons as mem
`bers of a group. When a patron is defined as a member of a
`group, that patron is able to acceSS any object for which the
`group has been granted privileges. Additionally, patrons can
`access objects for which they have been Specifically granted
`individual privileges. A patron can Set default groups whose
`members will have access to the objects the patron Stores.
`When patrons Store objects, they have the option to use this
`default group, to grant specific privileges to groupS and
`individual patrons, or to do both.
`If a library client request involves the Storage, retrieval, or
`update of an object, library server 10 forwards the request to
`the object server 20 that contains or will store the object(s)
`referred to in the request based upon information provided
`by library catalog 12. If the library client request is a query
`of the information stored in library catalog 12, library server
`10 will interact only with library catalog 12 and will not
`contact object server 20.
`The library catalog is analogous to a conventional
`library's card catalog. It is a single Set of database tables
`which contain an index of all the objects stored in the library
`System. In addition, it can Store information Such as textual
`descriptions for each object, information on the type of
`object (e.g., image object, spreadsheet, text document),
`library patron names and privileges, access authorization
`data for each object, links between objects. The library
`catalog can also Store a virtually unlimited number of
`property type/property value pairs for each object (e.g.,
`name/John, Age/35, Address/1 Greenway Drive). These
`property type/property value pairs are known as an object's
`properties.
`The library server contains a parts table 14, as shown in
`FIG. 3, which resides in the library catalog 12. For each part
`in the library system library server 10 stores information
`about that part. As shown in the parts table 14 in FIG. 3, the
`information Stored for a part includes an item identifier (ID),
`a part number, a representation type (REP type) and an
`object server ID.
`When a part is stored in the conventional client/server
`library system, library server 10 assigns an item ID and a
`part number, which are 16 bytes and 4 bytes long, respec
`tively. The item ID is a unique identifier for an item to which
`the part belongs. For example, an item could represent a
`folder in which the part represents a document within that
`folder. Likewise, the part number is a unique identifier for
`that part.
`The REP type field can be used to indicate the type or
`class in which the part is classified. For example, if the part
`is an image stored in a TIFF format, the REP type for that
`part could indicate that the part is a TIFF formatted image.
`An object server 20 maintains objects stored within the
`library system. Objects are stored or retrieved from an object
`store 22 by object server 20. Object server 20 receives
`requests from library server 10 and communicates with
`library client 30 to complete the requests. Such a library
`System can contain Several distributed object Servers. The
`object server field in the library server's parts table indicates
`the identifier for the object server which owns the part. For
`
`1O
`
`15
`
`25
`
`1. Field of the Invention
`This invention relates to the field of information and
`Storage management and, more particularly, to Storage and
`retrieval of large binary objects.
`2. Description of the Related Art
`Client/server object Storage Systems have been used to
`Store and manage a wide variety of digital objects Such as
`documents, graphics, audio, Video, Spread sheets and word
`processing text. Such digital objects are known generally as
`binary lo large objects (blobs).
`A conceptual view of a conventional client/server System
`is shown in FIG. 1 and includes a library server 10, one or
`more object servers 20 and a library client 30. Each of the
`library and object Servers and the library client includes an
`information store. That is, the library server 10 includes a
`library catalog 12, the is object server 20 includes an object
`store 22 and the library client 30 includes a client cache 32,
`as shown in FIG. 2. Also, a communications isolator (not
`shown) is included which allows the library server, object
`Server and library client to communicate with one another
`without concern for complex communications protocols.
`The library server, object servers and library clients are
`connected by a communications network, Such as a wide
`area network (WAN), but also can be locally connected via
`a local area network (LAN).
`In the conventional library client/server system the library
`client is typically embodied in a WorkStation, Such as a
`personal computer, and the library and object Servers are
`typically embodied in a host processor: generally a main
`frame computer.
`The library clients 30 each send requests to the library
`Server 10 to Store, retrieve, and update objects Stored in one
`or more of the object Servers, and to update and query the
`object indices and descriptive information Stored in library
`catalog 12. Library client requests are generated by library
`patrons. These patrons are users who have been granted
`privileges for the use of the library System.
`Two types of library servers have been used, a host based
`library server (HBLS) and a LAN based library server
`(LBLS). The HBLS, is a program which can be implemented
`in a mainframe computer in an IBM MVS/ESA (Multiple
`Virtual Storage/Enterprise Systems Architecture) environ
`50
`ment running under the Customer Information & Commu
`nication System (CICS). The library catalog with which it
`interacts can be implemented with an IBM DATABASE 2
`(DB2) database.
`Before a library client request is processed, library Server
`10 checks library catalog 12 to ensure that the patron's name
`and password are valid. Next, the library Server ensures that
`the patron has been granted the appropriate privileges to
`perform the requested action. Each patron is assigned a Set
`of privileges by a System administrator. An example of a
`library privilege is the ability to delete objects.
`Finally, the library server checks to ensure that the
`objects owner has granted the patron the privileges needed
`to do what is requested (e.g., update the object). The owner
`of an object is the patron who first stored the object. When
`an owner Stores an object that owner must Specify which
`other patrons are to have access to the object.
`
`35
`
`40
`
`45
`
`55
`
`60
`
`65
`
`-7-
`
`

`

`5,857,203
`
`15
`
`25
`
`3
`example, if the part is Stored on object Store 22 of object
`server 20, the object server ID field will contain the identifier
`for object server 20.
`Each object Server contains an object Server table 24 as
`shown in FIG. 3. The object server 20 uses object server
`table 24 to manage Storage of parts in its Storage areas, Such
`as the object store 22. Object server table 24 also contains
`the same item ID, part number and REP type for the part as
`does the library server parts table 14. The object server table
`also contains a file name for the part 28, which indicates the
`location in object store 22 of stored part 28.
`In the conventional library client/server System, object
`server 20 communicates with the library client 30 via the
`client cache 32. That is, when an object Server retrieves an
`object from library client 30, it retrieves the object from the
`client cache 32. Similarly, when Sending an object to library
`client 30, object server 20 places a copy of the object in
`client cache 32.
`Two types of object Servers have been used, a host based
`object server (HBOS) and a LAN based object server
`(LBOS). The HBOS is a program implemented on a main
`frame computer, for example in a MVS/ESA environment
`running under CICS. It interacts with the IBM Object Access
`Method (OAM) to provide object storage. The LBOS is a
`program implemented in a WorkStation, Such as in an OS/2
`environment, and provides object Storage on a LAN.
`When a library patrons’s privileges are defined a default
`object Server can be set for the patron. When a patron Stores
`an object, it will be stored in the default object server for that
`patron. If it is later determined that an object or a group of
`objects should be relocated to a different object server, a
`client application can cause those objects to be moved from
`one object Server to another.
`An LBOS can be located on any workstation having
`Sufficient hardware resources and is connected to the library
`server. Furthermore, an LBOS can be located at a site remote
`from the library server and local to the user. This allows
`Selected objects to be Stored close to a remote group of
`library patrons who will frequently use these objects. This
`capability is called distributed object storage. Distributed
`object Storage helps to reduce the costs associated with
`Sending objects over telecommunications lines and provides
`better performance in Storing and retrieving objects.
`The HBOS interacts with the IBM OAM to implement an
`object store that is maintained as a set of IBM DB2 tables,
`such as the object server table 24 discussed above. These
`DB2 tables can be monitored, backed up, and recovered
`using Standard database utilities. OAM is capable of man
`aging its information Store using a S combination of direct
`50
`access storage devices (DASD) and write once read many
`(WORM) optical storage.
`LBOS implements its object store by using a combination
`of the LBOS workstation hard drives and an optional optical
`library subsystem (often called an optical jukebox). The
`optical library supported by LBOS is capable of storing
`optical cartridges internally. Shelf-resident optical cartridge
`Support is also provided, thus greatly expanding the Storage
`capacity of the optical Server. LBOS controls the migration
`of objects between the workstation hard drive, which func
`tions as a staging area, and optical Storage. Because a
`WorkStation's hard drive can access Stored information faster
`than an optical jukebox, LBOS ensures that newly stored
`objects and objects that have recently been retrieved are
`maintained on the WorkStation hard drive. AS the WorkSta
`tion hard drive becomes full, LBOS removes those objects
`to optical Storage that has been least recently accessed to free
`
`4
`Storage Space for new objects. A Single drive optical drive
`can also be attached to LBOS to provide a transaction log as
`a backup mechanism for the optical library.
`LBOS includes a variety of storage administration
`functions, Such as transaction logging and the ability to write
`out duplicate copies of images and files to Support full
`backup and recovery.
`The library client 30 is the interface through which
`application programs can Submit requests to the library
`System. These can include requests to Store objects, update/
`add descriptors to objects, delete objects and query infor
`mation in the library catalog. Library requests can be Sub
`mitted through the library client either individually or in
`batches.
`The client cache 32 is a specialized function, imple
`mented on a user's WorkStation. The cache is used to locally
`hold copies of objects that have been stored to or retrieved
`from the object server. These local copies allow very fast
`access to objects and provide a means for communicating
`between the library client and the servers. When a library
`client requests a copy of an object from the library Server,
`the library Server causes a copy of that object to be sent from
`the object server which owns it to the library client that
`requested it. The object is Stored in the client cache of the
`requesting library client. When library request orders are
`submitted by library client 30 to library server 10 a copy of
`the request is also Stored in client cache 32.
`FIG. 2 illustrates the data flow in a conventional digital
`client/server library System. A library client, Such as library
`client 30, can be located remotely from the library server 10
`and object server 20. Typically, the library client 30 is
`connected to library server 10 and object server 20 via a
`WAN. Moreover, object server 20 may be connected to
`library server 10 via a WAN.
`When a requesting library client 30 requests an object, or
`blob, it sends a request 1 to library server 10. Upon receipt
`of the request library server 10 consults the parts table 14,
`among other tables, in the library catalog 12 and determines
`which object Server owns and has the requested object Stored
`in its object Store 22. Here, the owning object Server is
`shown as object server 20 to which library server 10 issues
`a request 2 which contains the itemID, part number and REP
`type of the requested part. Upon receiving the request, object
`server 20 retrieves the blob from object store 22 by con
`Sulting its object Server table 24 and Sends a copy of it to
`client 30. The double line 3 shown in FIG. 2 indicates a copy
`of the blob which is transmitted from object server 20 to
`requesting client 30. Object server 20 stores the blob in
`client cache 32. When the blob is successfully transmitted to
`client cache 32 object server 20 sends a response 4 to library
`server 10 upon successful transfer of the blob to client cache
`32. Library server 10, in turn, sends a response 5 to
`requesting library client 30 indicating to the client that the
`blob was successfully transferred, which allows the client to
`retrieve the blob from client cache 32 for use by a client
`application.
`When an application program Submits a request for Stor
`age of an object in the conventional library System library
`client 30 creates a copy of the object in its client cache 32
`to allow the appropriate object server 20 to retrieve the
`object. The library client then sends a Storage request 1 to
`library Server 10. Included in the Storage request is a handle
`to the object stored in the client cache 32. The handle is an
`identifier which is used to locate the object in the client
`cache.
`Upon receiving the Storage request 1, library Server
`updates tables in library catalog 12, including the parts table
`
`35
`
`40
`
`45
`
`55
`
`60
`
`65
`
`-8-
`
`

`

`S
`14 as shown in FIG. 3, to identify the object server 20 in
`which the object is to be stored. Typically, the object server
`is selected by default based on the user's identity. Library
`server 10 then sends a request 2 to object server 20 to
`retrieve the blob from the client cache 32 and store it in the
`object store 22, in which library server 10 passes the handle
`of the object stored in client cache 32 and the item ID, part
`number and REP type of the part.
`The object Server 20, upon receiving the request 2 to
`retrieve a copy of the object Stored in client cache 32 then
`retrieves a copy of the object 3, or blob, and Stores that copy
`in object Store 22, generally for archival purposes and
`updates its object Server table 24 accordingly to indicate a
`file name for the blob Stored in object store 22. As shown in
`FIG. 3 the file name uniquely identifies the location of the
`blob stored in object store 22.
`Upon Successfully Storing a copy of the blob, object
`server 20 sends a response 4 to library server 10 to notify it
`that the object was successfully stored. Library server 10
`then updates its tables including the parts table 24 to indicate
`that the object is successfully stored in object server 20. The
`library server 10 sends a response 5 to library client 30
`indicating that the object was Successfully Stored So that the
`library client can take further action based on the fact that
`the object was Successfully Stored in object Store 22, Such as
`deallocating memory resources for that object in client cache
`32.
`The conventional client/server library system has
`mechanisms, as described above, to efficiently Store and
`retrieve parts that range up to about 10 Mb. However, in the
`conventional library client and object Server an object, or
`blob, must be totally constituted in memory in Several places
`within the System as it is being processed on its path to file
`Storage. When objects get too big for a given configuration
`of available memory and network access, they cannot be
`processed because not enough System resource is available
`to create an in-storage copy. The external Symptom when
`these resource constraints are reached is a timeout of the
`library client that is waiting for a response from the Server.
`Even in cases where the timeout values are Set very high and
`the object is Successfully Stored the response time to move
`it is unacceptably slow.
`File systems are known which divide a file into blocks
`which are Smaller than the entire file in order to store the file
`information on a storage device, Such as a disk drive. For
`example, the Unix file System Stores a file on a disk in data
`blocks of fixed size (see A. Silberschatz, Operating System
`Concepts, pp. 488-96, 1989). These data blocks, however,
`have a fixed size, such as 8,192 bytes. Since these blocks
`have a fixed size, Some known file Systems employ two
`difference block sizes: a large block size for most of the
`blocks used, and a Smaller block size, Such as 1024, in order
`to minimize fragmentation. Accordingly, the Unix file SyS
`tem stores a file in Small blocks which are stored on the disk
`device. The Unix System employs an indeX node which is
`called an inode, which contains a record concerning the
`particular data blocks in which the file is stored. The inode
`contains pointers to the data blocks which comprise a file.
`A problem with known file systems, such as the Unix file
`System, is that data blocks are limited to fixed sizes and
`cannot be readily changed after a file has been Stored.
`Accordingly, changes in Stored files can cause a change to
`data within a particular block which causes a ripple effect
`throughout the blocks stored, such that the file data for the
`entire file needs to be Stored again in order to have the data
`redistributed throughout the allocated blocks.
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`5,857,203
`
`6
`Other file Systems, Such as the high performance file
`system (HPFS), disclosed by Letwin in U.S. Pat. No. 5,371,
`885, store data in variable length record forms. For example,
`Letwin shows in FIG. 5c, and discusses in column 7 through
`9 an FNODE which contains an allocation structure 546
`which points to a plurality of Sector runs. Sector runs are
`defined to be sets of contiguous Sectors on a disk. The
`allocation structure of the FNODE points to the beginning of
`a Sector run and indicates the length in Sectors corresponding
`to the number of contiguous Sectors in that Sector run.
`Accordingly, in the HPFS system a file is still allocated in
`terms of fixed size units, namely Sectors, and is limited to
`defining units of Storage which are physically contiguous on
`a device, such as a disk drive. The HPFS system does not
`allow for arbitrarily varying the length of the Sector runs
`Since the Sector run representation in the allocation Structure
`546 is related to the Sectors which are physically contiguous
`with one another on a disk. Accordingly, even the HPFS
`system would suffer from the above described problems.
`
`SUMMARY OF THE INVENTION
`It is an object of the present invention to solve the
`problems of the conventional client/server library system
`relating to Storage of large objects.
`The present invention relates to a number of Strategies
`that increase the Size of the objects that the System can
`handle. Some of these Strategies affect the way the objects
`are Stored, and others affect how the System processes the
`objects. All of the changes are made in a way that do not
`affect current users of existing Small objects. Most of these
`Strategies allow existing programs to process much larger
`objects without requiring recompilation.
`The present invention operates to divide a large file into
`Smaller pieces. That is, the present invention divides a large
`file into Smaller pieces which are themselves files, and
`creates a list of those pieces. A unique aspect of the present
`invention is in how the techniques are adapted in the
`conventional client/server digital library System.
`To achieve the above object the present invention pro
`vides a method for Storing and managing objects in a
`distributed information Storage management System includ
`ing a client, an object Server for Storing an object and a
`centralized Server having information Stored therein regard
`ing the location of the object and the client, the method
`comprising the Steps of:
`a) dividing a digital object into a plurality of pieces,
`wherein each of Said pieces is Smaller than Said digital
`object;
`b) generating a piece map for storing piece identification
`information identifying each of Said pieces,
`c) transmitting each of Said pieces and said piece map
`from the client to the object Server, Storing the transmitted
`pieces and piece map in the object Server and Storing
`information about each of Said pieces and Said piece map in
`Said centralized Server.
`Also to achieve the above object the present invention
`provides a System for Storing and managing objects, com
`prised of: a client for dividing a digital object into a plurality
`of pieces; one or more object Servers for Storing a piece map
`and Said plurality of pieces, wherein Said piece map includes
`piece identifying information identifying each of Said plu
`rality of pieces, and
`a centralized Server having a parts table for Storing Said
`piece identifying information and piece location information
`identifying which one of Said object Servers each of Said
`
`-9-
`
`

`

`5,857,203
`
`7
`plurality of pieces is Stored, and Storing piece map identi
`fication information identifying Said piece map and piece
`map location information identifying one of Said object
`Servers within which said piece map is Stored.
`BRIEF DESCRIPTION OF THE DRAWINGS
`The above and other objects and advantages of the
`invention will become more apparent and more readily
`appreciated from the following detailed description of the
`presently preferred embodiments of the invention taken in
`conjunction with the accompanying drawings, of which:
`FIG. 1 is a conceptual view of a conventional client/server
`library System;
`FIG. 2 is a diagram showing flows in a conventional
`client/server library System;
`FIG. 3 shows a portion of the parts table in the library
`Server, and the object Server table in the conventional
`client/server library System;
`FIGS. 4 shows a portion of the parts table of the library
`server and the object server table of the object server in the
`client/server library System according to the present inven
`tion; and
`FIG. 5 shows a portion of the parts table of the library
`server and the object server table of the object server in the
`client/server library System according to the present inven
`tion according to another embodiment of the present inven
`tion.
`
`15
`
`25
`
`8
`server library system, which is 32 bits long. Dividing the
`large object into a plurality of Smaller pieces also allows an
`intermediary part of a large object to be modified without
`accessing the beginning parts of the object.
`In the conventional client/server library System all parts
`have a part number. In the present invention large parts are
`Stored as a Sequence of Smaller pieces which have the same
`part number as the large part. Another field (i.e., the REP
`type field) is used to link the Smaller parts together. The part
`Stored in parts table 14 corresponding to the piece map
`contains the original user-provided REP type value.
`However, the REP type field for the piece entries in the parts
`table 14 is used as a Secondary, or piece counter to identify
`the piece within the Sequence of pieces for the large object.
`Users are unaware that the large objects are Stored as a
`Sequence of Smaller pieces. The part behavior of the client/
`Server library System remains unchanged from that in the
`conventional client/server library System. That is, the
`present invention allows most of the conventional client/
`Server library System to manage the parts therein without the
`knowledge that they may constitute a larger object.
`The present invention divides the large object into pieces
`when it is initially accessed and processes and Stores those
`pieces. The pieces are reconstituted into the complete object
`when the object is delivered to an end user. Preferably an
`environment variable indicates the largest size object to be
`processed on a particular client/server library System.
`Objects larger than this value will always be broken up into
`Smaller pieces.
`The method to break up the large object, or blob involves
`creating a piece map in a data structure which is Stored as the
`first piece of the multi-piece blob. The pieces of the blob are
`Stored as Subsequent parts for the particular itemID and part
`number related to the object. When the large object is
`retrieved, the map is retrieved first and the large object is
`reconstituted from the pieces, based on the piece numbers
`and the piece lengths obtained from the piece map.
`FIG. 4 shows a Schematic diagram of a piece map 26
`which contains information for each piece of a large object.
`Each entry in piece map 26 corresponds to a piece and
`includes a piece number and a length of that piece. The piece
`number references a unique entry in parts table 14, as shown
`in FIG. 4.
`From an external view, the Stored parts have only one part
`number. This maintains for the end user the integrity of part
`numbers for the large objects. Internally this is accomplished
`by using the REP type field as an additional counter for the
`pieces. Since the ItemID/PartNum/REP type triple is a
`unique identifier, multiple parts per item ID can be stored
`which all have the same part number. The entry in parts table
`14 for piece map 26 includes a REP type which is initially
`assigned to the large object, and all other pieces for that large
`object have the piece counter value stored in the REP type
`field of the particular piece.
`A PartLoc field of the parts table is the link between the
`pieces and the map. In the conventional client/server library
`System when a part is Stored in the object Server, the value
`in the PartLoc field is set to a predetermined value, for
`example, DOCSSMANAGED. In the present invention the
`PartLoc value is different for the stored pieces of the large
`blob. For example, the value stored in the PartLoc field for
`a piece is “FFFF' which is concatenated with the ItemID/
`PartNum/RepType of the map. The map contains sufficient
`information to find all the other pieces.
`The pieces cannot be individually processed by any
`existing application programming interface (API) unless the
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`The embodiments of the present invention are described
`below in more detail with reference to the above drawings,
`in which like reference numerals refer to the Same or similar
`elements.
`Most operating Systems have limits on the size of an
`individual file. Multimedia files can approach and exceed
`these limits. For example, a full length motion picture when
`digitized can approach 100 gigabytes (Gb). If a part corre
`sponding to a multimedia object is Stored in a client/server
`library System as Set of files, Such limits can be avoided.
`The conventional client/server library System has mecha
`nisms to efficiently Store and retrieve parts that range up to
`about 10 Mb. As these parts move within the system, copies
`are made on disk and in Virtual memory for performance and
`to preserve the System State in the event of a failure. The
`mechanisms that create and manage these copies are com
`pleX and not easily modified. Any change to the conven
`tional client/server library System to enable larger objects
`should not disrupt or change existing objects Stored in the
`System or the applications programs that use them.
`The present invention Solves the problem of Storing large
`objects by Storing a large binary object, or blob, as an
`ordered set of Smaller pieces. Further, the present invention
`employs a directory object, or piece map, that points to the
`set of smaller objects which can be reconstituted into the
`Single large object. The piece map is itself an object Stored
`in the client/server library system and Substitutes for the
`entire Set of Smaller pieces when referencing the large object
`via the library client. The piece map contains an ordered
`Sequence of identifiers and lengths of the actual content
`objects or pieces. In a preferred embodiment the piece
`identifiers are assigned a piece number. The length of each
`piece is included in the piece map to allow the Sys