USOO776.1655B2
`
`(12) United States Patent
`Mizushima et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,761,655 B2
`Jul. 20, 2010
`
`(54) STORAGESYSTEMAND METHOD OF
`PREVENTING DETERIORATION OF WRITE
`PERFORMANCE IN STORAGE SYSTEM
`
`75
`(75) Inventors: NE May, Miss (JP):
`uji Nakamura, Machida (JP)
`O
`O
`(73) Assignee: Hitachi, Ltd., Tokyo (JP)
`
`(*) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 416 days.
`
`(21) Appl. No.: 11/968,218
`
`(22) Filed:
`
`Jan. 2, 2008
`
`(65)
`
`Prior Publication Data
`US 2008/0229OO3 A1
`Sep. 18, 2008
`
`Foreign Application Priority Data
`(30)
`Mar. 15, 2007 (JP)
`............................. 2007-067142
`
`(51) Int. Cl.
`(2006.01)
`G06F 12/00
`711/103
`(52) U.S. Cl. ..........................................
`(58) Field of Classification Search ....................... None
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`5,359,569 A * 10/1994 Fujita et al. ................. 365,229
`5,483,491 A *
`1/1996 Yoshioka et al. ............ 365,200
`5,936,971 A
`8, 1999 Harari et al.
`6,388,919 B2 *
`5/2002 Terasaki ................ 365,185.09
`6.826,081 B2 * 1 1/2004 Nagashima et al. ... 365/185.09
`2002/0099995 A1
`7/2002 Furukawa ................... T14f766
`* cited by examiner
`Primary Examiner Hiep T Nguyen
`(74) Attorney, Agent, or Firm Brundidge & Stanger, P.C.
`
`ABSTRACT
`(57)
`Provided is a storage system capable of inhibiting the dete
`rioration of its write performance. This storage system
`includes a flash memory, a cache memory, and a controller for
`controlling the reading, writing and deletion of data of the
`flash memory and the reading and writing of data of the cache
`memory, and detecting the generation of a defective block in
`the flash memory. When the controller detects the generation
`of a defective block in the flash memory, it migrates pre
`scribed data stored in the flash memory to the cache memory
`and, even upon receiving from the host computer a command
`for updating the migrated data, disables the writing of data in
`the flash memory based on the command.
`
`17 Claims, 12 Drawing Sheets
`
`
`
`123
`
`321
`
`4OO
`
`HPE, Exh. 1005, p. 1
`
`

`

`U.S. Patent
`
`Jul. 20, 2010
`
`Sheet 1 of 12
`
`US 7,761,655 B2
`
`FIG.1
`
`
`
`
`
`ADAPTER
`
`ADAPTER
`
`1 O
`100- 110 111 112113 collies 114 115 116117
`COMPUTER
`Q
`VI C
`(
`121 127 120 122
`TTI /
`INTER. an
`ADAPTER M NETWORK N ADAPTER
`(
`\)
`124
`() is
`|)
`MEMORY
`EEAS, NSEA SSAs;
`130A----------Nesca,
`125 140 144 55;
`
`
`
`K
`
`
`
`192-y-
`
`HPE, Exh. 1005, p. 2
`
`

`

`U.S. Patent
`
`Jul. 20, 2010
`
`Sheet 2 of 12
`
`US 7,761,655 B2
`
`FIG.2
`
`121
`
`CHANNEL
`ADAPTER
`
`21 O
`
`11 O 111 112 113
`
`PROCESSOR
`
`HOST CHANNEL
`INTERFACE
`
`NETWORK
`INTERFACE
`
`PROCESSOR
`PERPHERAL
`CONTROLLER
`
`LOCAL
`MEMORY
`
`CACHE MEMORY
`INTERFACE
`
`FIG.3
`
`
`
`STORAGE
`ADAPTER
`
`22O
`
`PROCESSOR
`
`CACHE MEMORY
`INTERFACE
`
`NETWORK
`INTERFACE
`
`PROCESSOR
`PERPHERAL
`CONTROLLER
`
`LOCAL
`MEMORY
`
`STORAGE
`CHANNEL
`INTERFACE
`
`14O 141 142 143
`
`HPE, Exh. 1005, p. 3
`
`

`

`U.S. Patent
`
`Jul. 20, 2010
`
`Sheet 3 of 12
`
`US 7,761,655 B2
`
`FIG.4
`
`
`
`14O
`
`144
`
`FLASH
`MEMORY
`CHIP
`FLASH
`MEMORY
`CHIP
`FLASH
`MEMORY
`CHIP
`
`FLASH
`MEMORY
`CHIP
`FLASH
`MEMORY
`CHIP
`FLASH
`MEMORY
`CHIP
`
`MEMORY
`CHIP
`FLASH
`MEMORY
`CHIP
`FLASH
`MEMORY
`CHIP
`
`FIG.5
`
`34O
`
`350 351
`
`
`
`321
`
`HPE, Exh. 1005, p. 4
`
`

`

`U.S. Patent
`
`Jul. 20, 2010
`
`Sheet 4 of 12
`
`US 7,761,655 B2
`
`FIG.6
`
`
`
`HPE, Exh. 1005, p. 5
`
`

`

`U.S. Patent
`
`Jul. 20, 2010
`
`Sheet 5 of 12
`
`US 7,761,655 B2
`
`FIG.8
`
`123
`
`321
`
`FIG.9
`
`T41
`
`T42
`
`T43
`
`T44
`
`T4
`
`
`
`
`
`
`
`
`
`
`
`REDUNDANCY OF LOGICAL
`PAGE OF FLASH MEMORY
`INVALID PAGE CONTENT
`AVERAGE) DURING
`GARBAGE COLLECTION
`VALID PAGE MIGRATION
`VOLUME (AVERAGE) DURING
`GARBAGE COLLECTION
`
`WRITE PERFORMANCE
`OF FLASH MEMORY
`
`HIT PROBABILITY
`
`24/12
`200%
`
`2012
`167%
`
`8/20
`40%
`
`4/16
`25%
`
`20/8
`250%
`
`8/16
`50%
`
`DECREASED IMPROVED
`
`UNCHANGED DECREASED
`
`OF FLASH MEMORY t
`
`HPE, Exh. 1005, p. 6
`
`

`

`U.S. Patent
`
`Jul. 20, 2010
`
`Sheet 6 of 12
`
`761,655 B2
`US 7
`
`9
`
`FIG.10
`
`CACHE MEMORY
`
`SHARED MEMORY
`
`51O151O2 51O
`
`313
`
`CD
`
`52O 52O1 52O2
`
`
`
`
`
`
`
`HPE, Exh. 1005, p. 7
`
`

`

`U.S. Patent
`
`Jul. 20, 2010
`
`Sheet 7 of 12
`
`US 7,761,655 B2
`
`FIG.11
`
`CACHE MEMORY
`
`
`
`
`
`313
`
`SHARED MEMORY
`
`51 O151O2 51O
`
`HPE, Exh. 1005, p. 8
`
`

`

`U.S. Patent
`
`Jul. 20, 2010
`
`Sheet 8 of 12
`
`US 7,761,655 B2
`
`FIG. 12
`
`
`
`S704
`NO
`
`
`
`DiRTY SUB?
`
`YES
`
`
`
`S
`THERE ENTRY OF
`DIRTY - NO2
`
`
`
`
`
`NO
`
`WRITE BACK
`ENTRY OF
`RTY EYESTO FM
`D
`
`HPE, Exh. 1005, p. 9
`
`

`

`U.S. Patent
`
`Jul. 20, 2010
`
`Sheet 9 of 12
`
`US 7,761,655 B2
`
`FIG.13
`
`START
`
`RECEIVE READ REQUEST
`
`S711
`
`S72O
`
`TRANSFER DATATO CM
`
`S719
`
`READ STORED DATA OF
`PPA CORRESPONDING
`O READTARGET LPA
`
`YES
`
`stanc
`
`NO
`
`
`
`
`
`
`
`
`
`
`
`
`
`IS
`THERE ENTRY OF
`VALID = OFF?
`
`
`
`IS
`THERE ENTRY OF
`DIRTY NO2
`
`
`
`WRITE BACK
`ENTRY OF
`DIRTY YESTO FM
`
`
`
`S78
`
`ORTY-->NO
`
`READ IDATA
`FROM CM
`
`SEND READ DATA
`TO HOST COMPUTER
`
`S713
`
`S714.
`
`HPE, Exh. 1005, p. 10
`
`

`

`U.S. Patent
`
`Jul. 20, 2010
`
`Sheet 10 of 12
`
`US 7,761,655 B2
`
`FIG.14
`
`START
`
`ACGURE WRITE-BACKTARGET LPA
`AND ITS DATA FROM CM
`
`SELECT PAGE OF STATUS EF
`AND WRITE DATA
`
`S804
`PERFORM DEFECTIVE BLOCK
`SUBSTITUTE PROCESSING
`
`
`
`
`
`ERRORT
`
`NO
`
`PAGE STATUS OF PPA CORRESPONDING S805
`TO TARGET LPA -> 8
`
`SET PPA OF CURRENTLY WRITTEN PAGE S806
`NPPA COLUMN CORRESPONDING
`TOTARGET PA
`
`PAGE STATUS OF PPA -> 0
`
`PERFORMGARBAGE
`COLLECTION PROCESSING
`
`HPE, Exh. 1005, p. 11
`
`

`

`U.S. Patent
`
`Jul. 20, 2010
`
`Sheet 11 of 12
`
`US 7,761,655 B2
`
`FIG.15
`
`C START D
`START
`
`S90O
`
`COMPARE CHANGES INWRITE PERFORMANCE
`BASED ON DIFFERENCE OF SUBSTITUTE DESTINATION
`S901
`IS
`NO
`SUBSTITUTE TO CMMORE
`ADVANTAGEOUS?
`
`YES
`
`S902
`
`S912
`
`SELECT ONE BLOCKWORTH
`OF VALID PAGE FROM FM
`
`
`
`YES
`
`S
`THERE ENTRY OF
`WALD OFF?
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IS
`THERE ENTRY OF
`DIRTY NO2
`
`
`
`NO
`
`
`
`WRITE BACKENTRY
`OF DIRTY YESTO FM
`
`
`
`
`
`DIRTY-->NO
`
`
`
`MGRATE DATA OF ONE
`SELECTED WALID PAGE
`TO ENTRY
`
`
`
`DRTY-->SUB
`
`S
`DATA MIGRATION OF
`ALL SELECTED PAGES
`COMPLETE?
`
`
`
`PERFORM GARBAGE
`COLLECTION PROCESSING
`
`S913
`MGRATE WALID PAGE DATA
`N DEFECTIVE BLOCK
`TO ERASED BLOCK
`S914
`STATUS OF MIGRATION
`DESTINATION PAGEN
`PAGE STATUSTABLE->0
`
`
`
`DETECT PPA OF
`EACH WALD PAGE IN DEFECTIVE
`BLOCK FROMPPA COLUMN OF
`ADDRESSTRANSLATION TABLE
`S916
`
`SET PPA OF MIGRATION
`DESTINATION PAGE
`NPPA COLUMN
`
`DETECT PPA OF EACH VALID PAGEN
`DEFECTIVE BLOCK FROM PPA COLUMN
`OF ADDRESSTRANSLATIONTABLE
`
`PAGE STATUS OF DETECTED PPA->8
`PPA COLUMN->"NO CORRESPONDENCE"
`
`
`
`S917
`
`STATUS OF EACH PAGE
`N DEFECTIVE BLOCK->9
`
`HPE, Exh. 1005, p. 12
`
`

`

`U.S. Patent
`
`Jul. 20, 2010
`
`Sheet 12 of 12
`
`US 7,761,655 B2
`
`FIG.16
`
`
`
`NO
`
`
`
`REMAINING NUMBER
`OF UNWRITTEN PAGES LESS
`THAN PRESCRIBED
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SELECT BLOCKWITH MOST
`INVALID PAGES OF STATUS = 8,
`AND DETECT ALL VALID PAGES
`OF STATUS OTHEREIN
`
`
`
`
`
`COPY DATA OF ONE DETECTED WALD
`PAGE TO PAGE IN UNWRITTEN BLOCK
`
`STATUS OF COPY
`DESTINATION PAGE --> 0
`
`
`
`STATUS OF WALID PAGE --> 8
`
`SEARCH FOR PPA OF WALID PAGE
`IN ADDRESSTRANSLATIONTABLE,
`AND SET PPA OF COPY DESTINATION
`PAGE IN THAT COLUMN
`
`
`
`IS COPY OF
`ALL, DETECTED WALID PAGES
`COMPLETE?
`
`
`
`YES
`
`DELETE SELECTED BLOCKS
`
`STATUS OF EACH PAGE OF
`SELECTED BLOCK -> 9
`S1 OO8
`
`S1 OO9
`
`ERROR7
`N
`O
`STATUS OF ALL PAGES
`OF SELECTED BLOCK -> F
`
`S1 O11
`
`HPE, Exh. 1005, p. 13
`
`

`

`US 7,761,655 B2
`
`2
`SUMMARY
`
`1.
`STORAGE SYSTEMAND METHOD OF
`PREVENTING DETERIORATION OF WRITE
`PERFORMANCE IN STORAGE SYSTEM
`
`CROSS REFERENCES
`
`This application relates to and claims priority from Japa
`nese Patent Application No. 2007-067142, filed on Mar. 15,
`2007, the entire disclosure of which is incorporated herein by
`reference.
`
`10
`
`BACKGROUND
`
`In a storage system that uses a flash memory as the memory
`medium, the size of the backup extent to be used in the
`foregoing stored data update control will gradually decrease
`pursuant to the increase in the number of defective blocks in
`the flash memory. The smaller the ratio of the backup extent
`size for update control in relation to the net stored data vol
`ume, the more inferior the operating efficiency (write count of
`the flash memory per rewriting unit of the host system) during
`the update process of the stored data of the storage system.
`This is because the average work rate upon saving the other
`valid data remaining in the erased block will increase in the
`foregoing stored data update control.
`As a result, there is a problem in that the write performance
`of the storage system will gradually deteriorate pursuant to
`the increase in the number of defective blocks.
`The present invention was made in view of the foregoing
`points. Thus, an object of the present invention is to propose
`a storage system and a method for preventing deterioration of
`write performance in a storage system capable of inhibiting
`the deterioration of write performance in a storage system.
`The storage system of the present invention comprises a
`flash memory for writing data in page units and erasing the
`data in units of a block configured from a plurality of pages,
`and having a plurality of blocks and requiring the deletion of
`the blocks containing the pages in order to update the data, a
`cache memory for writing and temporarily storing data to be
`written in the flash memory at a speed that is faster than the
`flash memory, a controller for controlling the reading, writing
`and deletion of data of the flash memory and the reading and
`writing of data of the cache memory, and detecting the gen
`eration of a defective block in the flash memory, and a host
`computer for issuing a command requesting the write pro
`cessing of the data. The controller migrates prescribed data
`stored in the flash memory to the cache memory upon detect
`ing the generation of the defective block in the flash memory
`and, even upon receiving from the host computer a command
`for updating the migrated data, disables the writing of data in
`the flash memory based on the command.
`In other words, as a result of comprising a flash memory for
`writing data in page units and erasing the data in units of a
`block configured from a plurality of pages, and having a
`plurality of blocks and requiring the deletion of the blocks
`containing the pages in order to update the data, a cache
`memory for writing and temporarily storing data to be written
`in the flash memory at a speed that is faster than the flash
`memory, a controller for controlling the reading, writing and
`deletion of data of the flash memory and the reading and
`writing of data of the cache memory, and detecting the gen
`eration of a defective block in the flash memory, and a host
`computer for issuing a command requesting the write pro
`cessing of the data, and the controller migrating prescribed
`data stored in the flash memory to the cache memory upon
`detecting the generation of the defective block in the flash
`memory and, even upon receiving from the host computer a
`command for updating the migrated data, disabling the writ
`ing of data in the flash memory based on the command, it is
`possible to inhibit the deterioration of the operating efficiency
`during the update process of the stored data pursuant to the
`increase in the number of defective blocks of the flash
`memory. Thereby, even under circumstances where the hit
`rate of the cache memory is low, it is possible to inhibit the
`deterioration of the write performance in the storage system
`in comparison to conventional storage systems.
`Further, by making the cache memory in the foregoing
`storage system a nonvolatile random access memory, for
`
`The present invention generally relates to a storage system
`and a method for preventing deterioration of write perfor
`mance in a storage system using an electrically rewritable
`nonvolatile memory, and in particular relates to a storage
`system that uses a flash memory as the nonvolatile memory
`and a cache memory as the random access memory that is
`faster than the flash memory and which uses the cache
`memory in substitute for a defective flash memory block.
`A flash memory is a nonvolatile memory that cannot be
`overwritten directly. Thus, in order to rewrite data, it is nec
`essary to erase an erasure unit (block) configured from a
`plurality of write units (pages), and restore such pages to an
`unwritten state.
`Therefore, with a standard storage system that uses a flash
`memory as its primary memory medium, when the host sys
`tem does not update the stored data, it writes and validates
`new data in a backup extent that has been previously allocated
`to a part of the flash memory on the one hand, invalidates old
`data on the other hand, and thereafter erases the block con
`taining the old data to make it a new backup extent. Inciden
`tally, when there is other valid data remaining in the block to
`be erased, such valid data must be saved in an unwritten page
`of a different block prior to the foregoing erasure.
`Generally speaking, the backup extent in the flash memory
`is also used as a substitute extent when another block
`becomes defective. Since a block of a flash memory is only
`guaranteed a write cycle of roughly 100,000 times, defective
`blocks will be generated on a daily basis if the host system
`repeatedly updates the stored data of the storage system, and
`the number of defective blocks will increase gradually. In
`addition, when the number of defective blocks increases to a
`point of filling the backup extent, it will become difficult to
`perform the foregoing stored data update control, and the
`storage system will become incapable of rewriting.
`U.S. Pat. No. 5,936,971 describes a method of prolonging
`the rewriting lifespan of a data file storage system using a
`flash memory as the memory medium based on the following
`procedures using a cache memory: (1) Temporarily storing a
`flash memory data file from a host system in a cache memory
`capable of tolerating writeferasure cycles in a number that is
`significantly greater than a flash memory; (2) Writing a new
`data file in the cache memory in substitute for the flash
`memory according to a write request from the host system; (3)
`Storing in a tag memory the time from the last time the
`identifier of the data file and the respective data files were
`stored in the cache memory; and (4) When it is necessary to
`create additional space in the cache memory for a new data
`file, preferentially migrating the data file, in which the longest
`time has lapsed from the last writing by referring to the tag
`memory, from the cache memory to the flash memory. As a
`result of performing the processing described in (1) to (4)
`above, it is possible to significantly reduce the actual number
`of write counts and related stress in the flash memory.
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`HPE, Exh. 1005, p. 14
`
`

`

`US 7,761,655 B2
`
`4
`DETAILED DESCRIPTION
`
`3
`instance, a phase-change RAM (Random Access Memory),
`data substituted from the flash memory to the cache memory
`can be retained without any auxiliary power. Thus, the present
`invention yields an effect of reducing the electrical power
`consumption of the storage system, and protecting the loss of
`data caused by a failure Such as a Sudden power shutdown.
`Accordingly, the present invention yields the effect of
`inhibiting the deterioration of the write performance in a
`Storage System.
`
`DESCRIPTION OF DRAWINGS
`
`FIG. 1 is a diagram showing the configuration of a storage
`system according to the present invention;
`FIG. 2 is a diagram showing the internal configuration of a
`channel adapter configuring the storage system according to
`the present invention;
`FIG. 3 is a diagram showing the internal configuration of a
`storage adapter configuring the storage system according to
`the present invention;
`FIG. 4 is a diagram showing the internal configuration of a
`flash memory module configuring the storage system accord
`ing to the present invention;
`FIG. 5 is a diagram showing the configuration of a flash
`memory chip mounted on the flash memory module config
`uring the storage system according to the present invention;
`FIG. 6 is a diagram explaining the influence caused by the
`difference in the method of substituting the defective block
`generated in the flash memory according to the present inven
`tion;
`FIG. 7 is a diagram explaining the influence caused by the
`difference in the method of substituting the defective block
`generated in the flash memory according to the present inven
`tion;
`FIG. 8 is a diagram explaining the influence caused by the
`difference in the method of substituting the defective block
`generated in the flash memory according to the present inven
`tion;
`FIG. 9 is a diagram explaining the influence caused by the
`difference in the method of substituting the defective block
`generated in the flash memory according to the present inven
`tion;
`FIG. 10 is a diagram explaining the management of the
`cache memory and the flash memory chip according to the
`present invention;
`FIG. 11 is a diagram explaining the management of the
`cache memory and the flash memory chip according to the
`present invention;
`FIG. 12 is a flowchart showing the processing to be per
`formed by the storage controller and the memory controller
`based on a data write request from a host computer according
`to the present invention;
`FIG. 13 is a flowchart showing the processing to be per
`formed by the storage controller and the memory controller
`based on a data read request according to the present inven
`tion;
`FIG. 14 is a flowchart showing the write-back processing
`of cache data to be performed by the memory controller
`according to the present invention;
`FIG. 15 is a flowchart showing the defective block substi
`tute processing to be performed by the storage controller and
`the memory controller according to the present invention; and
`FIG. 16 is a flowchart showing the garbage collection
`processing to be performed by the memory controller accord
`ing to the present invention.
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Embodiments of the present invention are now explained
`with reference to the attached drawings.
`(1) Configuration of Storage System
`
`5
`
`FIG. 1 is a diagram showing a simplified hardware con
`figuration of a storage system 10 applying the present inven
`tion.
`The storage system 10 comprises a storage controller 120
`and flash memory modules (FMM) 151 to 154, 161 to 164,
`171 to 174, and 181 to 184. The storage controller 120 com
`prises channel adapters 121, 122, cache memories 123, 124,
`storage adapters 125, 126, a shared memory 129, and inter
`connection networks 127, 128.
`Incidentally, although the illustrated example shows a case
`where the storage controller 120 has the channel adapters
`121, 122, the cache memories 123, 124, the storage adapters
`125, 126, and the shared memory 129, the quantity of these
`components is not limited thereto.
`The interconnection networks 127 and 128, for instance,
`are Switches or the like, and mutually connect the respective
`components configuring the storage controller 120. Specifi
`cally, the interconnection networks 127 and 128 mutually
`connect the channel adapter 121, the cache memory 123, the
`storage adapter 125, and the shared memory 129. Similarly,
`the interconnection networks 127, 128 mutually connect the
`channel adapter 122, the cache memory 124, the storage
`adapter 126, and the shared memory 129.
`The channel adapter 121 is connected to the host computer
`100 via channels 110, 111, 112, 113. Similarly, the channel
`adapter 122 is connected to the host computer 100 via chan
`nels 114, 115, 116, 117. The host computer 100 is a computer
`Such as a personal computer, workstation, mainframe com
`puter or the like, and requests the storage controller 120 to
`read and write data from and in the storage system 10. The
`storage controller 120 uses the channel adapters 121, 122 to
`interpret the foregoing requests, and uses the storage adapters
`125, 126 to read and write data of the flash memory modules
`151 to 154, 161 to 164, 171 to 174, 181 to 184 in order to
`satisfy the requests.
`Thereupon, the cache memories 123, 124 are used to tem
`porarily store data received from the channel adapters 121,
`122 or the storage adapters 125, 126, and to permanently store
`specific received data as needed. The cache memories 123,
`124, for example, are dynamic random access memories, and
`are able to read and write data at high speed. The shared
`memory 129 stores a table for managing the stored data of the
`cache memories 123, 124, and the channel adapters 121, 122
`or the storage adapters 125, 126 can refer to and set such table.
`The shared memory 129, for example, is a dynamic random
`access memory, and is able to read and write data at high
`speed.
`The storage adapter 125 is connected to the flash memory
`modules 151 to 154, 161 to 164, 171 to 174, 181 to 184.
`Specifically, the storage adapter 125 is connected to the flash
`memory modules 151 to 154 via the channel 140. Further, the
`storage adapter 125 is connected to the flash memory mod
`ules 161 to 164 via the channel 141. Moreover, the storage
`adapter 125 is connected to the flash memory modules 171 to
`174 via the channel 142. In addition, the storage adapter 125
`is connected to the flash memory modules 181 to 184 via the
`channel 143.
`Similarly, the storage adapter 126 is connected to the flash
`memory modules 151 to 154, 161 to 164, 171 to 174, 181 to
`184. Specifically, the storage adapter 126 is connected to the
`
`HPE, Exh. 1005, p. 15
`
`

`

`5
`flash memory modules 151 to 154 via the channel 144. Fur
`ther, the storage adapter 126 is connected to the flash memory
`modules 161 to 164 via the channel 145. Moreover, the stor
`age adapter 126 is connected to the flash memory modules
`171 to 174 via the channel 146. In addition, the storage
`adapter 126 is connected to the flash memory modules 181 to
`184 via the channel 147.
`The channel adapters 121, 122 and the storage adapters
`125, 126 are connected to the maintenance terminal 130. The
`maintenance terminal 130 sends the configuration informa
`tion input by the administrator of the storage system 10 to the
`channel adapters 121, 122 and/or the storage adapters 125,
`126.
`Incidentally, the storage system 10 may also comprise one
`adapter in substitute for the storage adapter 125 and the chan
`nel adapter 121. Here, the one adapter will perform the pro
`cessing of the storage adapter 125 and the channel adapter
`121.
`Reference numerals 190 to 193 represent RAID (Redun
`dant Arrays of Inexpensive Disks) groups. For instance, the
`RAID group 190 is configured from the flash memory mod
`ules 151,161, 171, 181. When an error occurs in one of the
`flash memory modules; for instance, the flash memory mod
`ule 151 belonging to the RAID group 190 and it is not pos
`sible to read data from such defective flash memory module,
`data can be reconfigured from the other memory modules
`161, 171, 181 belonging to the flash RAID group 190.
`FIG. 2 is a diagram showing the hardware configuration of
`the channel adapter 121. The channel adapter 121 comprises
`a host channel interface 214, a cache memory interface 215, a
`network interface 211, a processor 210, a local memory 213,
`and a processor peripheral controller 212.
`The host channel interface 214 is an interface for connect
`ing the channel adapter 121 to the host computer 100 via the
`channels 110, 111, 112, 113. The host channel interface 214
`mutually converts the data transfer protocol in the channels
`110, 111, 112, 113 and the data transfer protocol in the stor
`age controller 120.
`The cache memory interface 215 is an interface for con
`necting the channel adapter 121 to the interconnection net
`works 127, 128.
`The network interface 211 is an interface for connecting
`the channel adapter 121 to the maintenance terminal 130.
`Incidentally, the host channel interface 214 and the cache
`memory interface 215 are connected via a signal line 216.
`The processor 210 performs various types of processing by
`executing programs stored in the local memory 213. Specifi
`cally, the processor 210 controls the transfer of data between
`the host computer 100 and the interconnection networks 127,
`128.
`The local memory 213 stores programs to be executed by
`the processor 210. Further, the local memory 213 storestables
`that are referred to by the processor 210. The tables referred to
`by the processor 210 contain configuration information for
`controlling the operation of the channel adapter 121, and are
`set or changed by the administrator. In the foregoing case, the
`administrator inputs information concerning the setting or
`changing of the table into the maintenance terminal 130. The
`maintenance terminal 130 sends the input information to the
`processor 210 via the network interface 211. The processor
`210 creates or changes the tables based on the received infor
`mation. The processor 210 further stores the tables in the local
`memory 213.
`The processor peripheral controller 212 controls the trans
`fer of data among the host channel interface 214, the cache
`memory interface 215, the network interface 211, the proces
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 7,761,655 B2
`
`10
`
`15
`
`6
`sor 210, and the local memory 213. The processor peripheral
`controller 212, for instance, is a chipset or the like.
`Incidentally, since the hardware configuration of the chan
`nel adapter 122 and the hardware configuration of the channel
`adapter 121 are the same, explanation of the hardware con
`figuration of the channel adapter 122 is omitted.
`FIG. 3 is a diagram showing the hardware configuration of
`the storage adapter 125. The storage adapter 125 comprises a
`cache memory interface 224, a storage channel interface 225.
`a network interface 221, a processor 220, a local memory 223,
`and a processor peripheral controller 222.
`The cache memory interface 224 is an interface for con
`necting the storage adapter 125 to the interconnection net
`works 127, 128.
`The storage channel interface 225 is an interface for con
`necting the storage adapter 125 to the channels 140,141,142.
`143. The storage channel interface 225 mutually converts the
`data transfer protocol in the channels 140,141, 142,143 and
`the data transfer protocol in the storage controller 120.
`Incidentally, the cache memory interface 224 and the stor
`age channel interface 225 are connected via a signal line 226.
`The network interface 221 is an interface for connecting
`the storage adapter 125 to the maintenance terminal 130.
`The processor 220 performs various types of processing by
`executing programs stored in the local memory 223.
`The local memory 223 stores programs to be executed by
`the processor 220. Further, the local memory 223 storestables
`that is referred to by the processor 220. The tables referred to
`by the processor 220 contain configuration information for
`controlling the operation of the storage adapter 125, and are
`set or changed by the administrator. In the foregoing case, the
`administrator inputs information concerning the setting or
`changing of the table into the maintenance terminal 130. The
`maintenance terminal 130 sends the input information to the
`processor 220 via the network interface 221. The processor
`220 creates or changes the tables based on the received infor
`mation. The processor 220 further stores the tables in the local
`memory 223.
`The processor peripheral controller 222 controls the trans
`fer of data among the cache memory interface 224, the Stor
`age channel interface 225, the network interface 221, the
`processor 220, and the local memory 223. The processor
`peripheral controller 222, for instance, is a chipset or the like.
`Incidentally, since the hardware configuration of the stor
`age adapter 126 and the hardware configuration of the storage
`adapter 125 are the same, explanation of the hardware con
`figuration of the storage adapter 126 is omitted.
`FIG. 4 is a diagram showing the hardware configuration of
`the flash memory module 151. The flash memory module 151
`comprises a memory controller 310 and a flash memory 320.
`The flash memory 320 stores data. The memory controller
`310 controls the “reading.” “writing,” and “deletion of data
`of the flash memory 320.
`The memory controller 310 comprises a processor 312, an
`interface 311, a data transfer unit 315, a RAM 313, and a
`ROM 314. The flash memory 320 comprises a plurality of
`flash memory chips 321.
`FIG. 5 is a diagram showing the internal configuration of
`the flash memory chip 321. The flash memory chip 321
`includes a plurality of blocks 330, and stores data in the
`respective blocks 330. The block330 is the unit (basis) for the
`memory controller 310 to delete data. The block 330 includes
`a plurality of pages 340. The page 340 is the unit (basis) for
`the memory controller 310 to read and write data. With the
`flash memory 320, data is read in the amount of time of
`roughly 20 LS per page. Further, data is deleted in the amount
`of time of roughly 1.5 ms per block. The time required to write
`
`HPE, Exh. 1005, p. 16
`
`

`

`US 7,761,655 B2
`
`5
`
`10
`
`15
`
`30
`
`40
`
`7
`data in the pages of the flash memory 320 is longer than the
`time required to write equal-sized data in the cache memories
`123, 124. Incidentally, the writing and deletion of data will
`gradually deteriorate the memory cells, and an error may
`occur when rewriting is performed numerous times (for
`example, several ten thousand times).
`The pages 340 are classified as a valid page, an invalid
`page, an unwritten page, or a defective page by the memory
`controller 310. A valid page is the page 330 storing valid data
`that needs to be stored in the storage system 10. An invalid
`page is the page 340 storing invalid data (garbage) that no
`longer needs to be stored in the storage system 10. An unwrit
`ten page is the page 340 that has not stored data since the
`block 330 to which it belongs has been erased. A defective
`page is the page 340 that cannot be physically rewritten due to
`reasons such as the memory elements in the page 340 being
`damaged. There are three factors for the pages 340 to become
`a defective page.
`The first factor is the rejection in the inspection at the chip
`manufacturing stage. The second factor is that an error occurs
`during the writing in the page 340. Incidentally, it will only be
`possible to read data from the defective page thereafter. A
`block 330 containing even one such page is referred to as a
`defective block, and the erasure of blocks and writing of
`pages are disabled. The third factor is that an error occurs
`25
`during the erasure of the block 330. Incidentally, all pages in
`this block will become defective pages. This block is referred
`to as a defective block, and the erasure of blocks and writing
`of pages are disabled.
`The interface 311 is connected to the storage adapter 125 in
`the storage controller 120 via the channel 140. Further, the
`interface 311 is connected to the storage adapter 126 in the
`storage controller 120 via the channel 144. The interface 311
`receives commands from the storage adapter 125 and the
`storage adapter 126. Commands from the storage adapter 125
`35
`and the storage adapter 126, for example, are SCSI com
`mands.
`Specifically, the interface 311 receives data from the stor
`age adapter 125 and the storage adapter 126. Then, the inter
`face 311 buffers the received data in the RAM 313. Further,
`the interface 311 sends the data buffered in the RAM 313 to
`the storage adapter 125 and the storage adapter 126.
`Moreover, the interface 311 has an interfacefunction that is
`compatible with hard disk drives. Thus, the storage adapters
`125, 126 recognize the flash memory modules 151 to 184 as
`hard disk drives. The storage system 10 may be equipped with
`a combination of flash memory modules and hard disk drives
`as the recording medium for storing data.
`The RAM 313, for instance, is a dynamic random access
`memory, and is able to read and write data at high speed. The
`RAM313 temporarily stores data to be sent and received by
`the interface 311. Meanwhile, the ROM 314 is a nonvolatile
`memory, and stores programs to be executed by the processor
`312. The programs to be executed by the processor 312 are
`loaded from the ROM 314 into the RAM 313 when the
`storage system 10 is booted so that they can be executed by
`th