USOO784926OB2
`
`(12) United States Patent
`Okuno et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,849.260 B2
`Dec. 7, 2010
`
`(54) STORAGE CONTROLLER AND CONTROL
`METHOD THEREOF
`
`(75) Inventors: Takahide Okuno, Odawara (JP);
`Mitsuhide Sato, Oiso (JP); Toshiaki
`Minami, Odawara (JP); Hiroaki Yuasa,
`Ninomiya (JP): Kousuke Komikado,
`Odawara (JP); Koji Iwamitsu, Odawara
`(JP); Tetsuya Shirogane, Odawara (JP);
`Atsushi Ishikawa, Minami-ashigara (JP)
`(73) Assignee: Hitachi, Ltd., Tokyo (JP)
`(*) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 514 days.
`(21) Appl. No.: 11/698,022
`
`(22) Filed:
`
`Jan. 26, 2007
`
`(65)
`
`Prior Publication Data
`US 2008/O126668A1
`May 29, 2008
`
`Foreign Application Priority Data
`(30)
`Nov. 28, 2006
`(JP)
`............................. 2006-3 19807
`(51) Int. Cl.
`(2006.01)
`G06F 3/10
`(52) U.S. Cl. ............... 711/113: 711/114; 711/E12.025;
`711/E12.035: 714/6: 714/9: 714/E11.092;
`710/22
`(58) Field of Classification Search ................. 711/100,
`711/112
`See application file for complete search history.
`References Cited
`
`(56)
`
`U.S. PATENT DOCUMENTS
`5,790.775 A *
`8/1998 Marks et al. ................... T14/9
`
`
`
`5,822,251 A * 10/1998 Bruce et al. ............ 365,185.33
`5.975,738 A * 1 1/1999 DeKoning et al. ............ 7OO/79
`7,603,485 B2 * 10/2009 Komikado et al. ............. 71O/5
`2001/0049774 A1* 12/2001 Otterness et al. ............ 711 148
`2006/O161707 A1* 7/2006 Davies et al. ............... T10/268
`
`FOREIGN PATENT DOCUMENTS
`
`JP
`JP
`
`09-146842
`11-312O58
`
`11, 1995
`4f1998
`
`* cited by examiner
`Primary Examiner Reginald G. Bragdon
`Assistant Examiner Daniel J Bernard
`(74) Attorney, Agent, or Firm Volpe and Koenig, P.C.
`
`(57)
`
`ABSTRACT
`
`Proposed is a storage controller and its control method for
`speeding up the processing time in response to a command in
`a simple manner while reducing the load of a controller that
`received a command targeting a non-associated logical Vol
`ume. This storage controller includes a plurality of controllers
`for controlling the input and output of data to and from a
`corresponding logical unit based on a command retained in a
`local memory, and the local memory stores association infor
`mation representing the correspondence of the logical units
`and the controllers and address information of the local
`memory in each of the controllers of a self-system and
`another-system. Upon receiving a command sent from a host
`computer, the controller determines whether the target logical
`unit is associated with the controller of a self-system or
`another-system based on the association information, and,
`when the logical unit is associated with the other-system
`controller, the controller transfers and stores the command to
`and in the corresponding other-system controller based on the
`address information.
`
`12 Claims, 18 Drawing Sheets
`
`ear
`
`29A
`PIElsa
`IHOSENSONowancN
`to CD, SNAEN
`
`---
`
`Host
`
`HST
`cgi-SS.
`
`HostcoMMUNICATION
`PROTOCOL.cHIP
`
`se INFORAN
`s: Rear ARAS
`
`SP96
`
`CoNTolunt
`
`A
`
`t
`
`coMMAND
`
`s
`
`MMNAIN
`focochip
`
`ENSE
`control
`
`SEAERS's
`SE ch
`33.6
`STRAGEAPPARAUS
`NTR UNIT
`
`HPE, Exh. 1017, p. 1
`
`

`

`U.S. Patent
`
`
`
`US 7,849.260 B2
`
`
`
`/(_OEZII, 2, LZL (ZI
`
`HPE, Exh. 1017, p. 2
`
`

`

`90€goeWOE
`
`
`
`
`
`
`
`
`
`
`
`(©)o1CF)NOISNVdX3ONIGNIONDAMOWANSHOVSP0719|(ce)O1(1)NOISNYdX3ONIGNIONI)AHOWJWSHOVO120LOdddddddZ4X%0-00000000"1x0
`
`
`
`
`
`
`
`
`HILSIDSYdiHD1090LOUdNOLLYDINNANIODSDVHOLSLL|HaLSIDSYSRIVH¥dd¥dIHD1090L0HdNOLLVOINNWNODSNWuVdd¥S9VHOLSOLD
`
`
`
`
`
`
`
`(BOISdDTODOLOUdNOLLVOINNININODLSOH1719)dWINAHOWAIN17.19|GalsdH109010HdNOLIVOINNKINODISOH019)dVINANOWANO1L9
`
`
`
`
`
`
`
`
`
`
`((E)O1(1)NOISNVdXaSNIGNTONI)AYOWANSHOVDPR07L9|((€)O1(1)NOISNVdX3DNIGNTONI)ANOWAINSHOWSL807-L9
`
`
`
`
`
`
`
`
`
`SID3udIHO100L0UdNOLLVOINNINIIODSPVUVddySVHOLSO1L9|HALSIDSYdIHd1090L0UdNOLLYOINNNINODSNivuvddyJOvVUOLSFLO
`
`
`
`
`
`
`HALSIDSYdIHD10D0LOHdNOLLYDINNWWODJSOH1119|HSLSID3YdIHDTODOLOWdNOLLVOINNININODLSOH015
`
`
`
`
`
`HSLSIDSYdIHO1OD0LOHdNOLLVOINNWWODISOHOTLD|YSLSID3HdIHDTODOLOUdNOHLVIINNWWODLSOH|19
`
`
`
`
`
`
`
`
`Y3.1SID3HLINNTOWLNODHAISNVHLVLV00119HALSIOIYLINNTOHLNODH34SNVHLVIVO11|ddddddaz2x0-000000082x0
`
`
`
`
`
`HALSIDAYLINNTOHLNODYSISNVELVIVO119Y3ISIDSYLINNTOHLNODHadSNVULVIVO019
`
`
`
`
`
`
`
`(AISHOSSSDOUdF190)dvANOWSIN17.19(3qiSHOSSSDOWd0119)dVAHOWSIN01.19
`
`
`
`
`
`AYOWSW1V9071119AYOWAW0x0-000000000x0AHOWZ1V¥9070119AMOWAIN1YOOTL19|44444441V90710LO|dddt4d4Z0%0-000000090x0
`
`
`
`
`
`
`H31SIDaYJDCIUE+719Y31SID3H3DGIHG0LO
`
`
`
`Ha1SIN3H3Ddlus0119Ha1SINayaDdlua|LO
`
`
`AHOWSW1V907FLDAYOWAW1¥9010LO
`
`
`
`
`AMOWSW1V¥9070119AMOINSIN1Y90TFLO
`
`
`
`
`
`dIHDTODOLOUdNOLLVIINNWIWOSLSOHWOudG2Ma3lA2DVdSAYOWNAN
`
`
`
`ual
`
`
`
`
`
`
`
`ddddddsd$x0-000000002x0
`
`
`
`ddddd4dt9X%0-000000009x0
`
`
`
`dddaddad9X0-00000008"9x0
`
`
`
`dj4ds43Z2x0-000000002x0
`
`aeSia
`
`HPE, Exh. 1017, p. 3
`
`
`
`
`
`444444450%0-000000080x0
`
`
`
`dadajajZ0%0-000000000x0
`
`dddddddZ1X0-000000001x0
`
`
`
`
`
`d4jddsjd+x0-00000008"1x0
`
`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 2 of 18
`
`US 7,
`
`849,260 B2
`
`
`
`
`
`HOSS300UdWOEGZM3IA39VdSAUOWSNEE
`
`VeoDid
`
`HPE, Exh. 1017, p. 3
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 3 of 18
`
`US 7,849.260 B2
`
`FIG.3A
`
`FIG.3B
`
`14A
`4OA
`-40AX
`
`41 A
`41 AX
`
`4AY
`
`
`
`CONTROL
`NFORMATION
`STORAGE AREA
`SELF-SYSTEM
`CTL RECEIVE
`COMMAND
`SORAGE AREA
`
`CONTRO
`INFORMATION
`STORAGE AREA
`EXECUTION
`PROCESSING
`STORAGE AREA
`OTHER-SYSTEM CTL
`RECEIVE COMMANO
`STORAGE AREA
`
`CONTROL
`INFORMATION
`STORAGE AREA
`SELF-SYSTEM
`CTL RECEIVE
`COMMAND
`STORAGE AREA
`
`CONTROL
`INFORMATION
`SORAGE AREA
`EXECUTION
`PROCESSING
`STORAGE AREA
`OTHER-SYSTEM CTL
`RECEIVE COMMAND
`STORAGE AREA
`
`LOGICAL
`UNT/PROCESSOR
`ASSOCATIONTABLE
`
`LOCAL MEMORY
`INFORMATIONTABLE
`
`CACE MEMORY
`NFORMATIONTABLE
`
`43A
`
`44A
`
`45A
`
`LOGICAL
`UNIT/PROCESSOR
`ASSOCATION TABLE
`
`LOCAL MEMORY
`INFORMATIONTABLE
`
`CACHE MEMORY
`INFORMATION TABLE
`
`SHARED AREA
`
`SHARED AREA
`
`HPE, Exh. 1017, p. 4
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 4 of 18
`
`US 7,849.260 B2
`
`FG4A
`
`FG4B
`
`
`
`DATA
`MANAGEMENT
`INFORMATION
`STORAGE AREA
`
`12A
`5OA
`
`5OAX
`
`DATA
`MANAGEMENT
`INFORMATION
`STORAGE AREA
`
`12B
`5OB
`
`5OBX
`
`DATA
`STORAGE AREA
`
`5OAY
`
`DATA
`STORAGE AREA
`
`5OBY
`
`SELF-SYSTEM
`PROCESSOR AREA
`
`SELF-SYSTEM
`PROCESSOR AREA
`
`MANAGEMENT
`NFORMATION
`STORAGE AREA
`
`51A
`
`51AX
`
`MANAGEMENT
`INFORMATION
`STORAGE AREA
`
`51 B
`
`51BX
`
`DATA
`STORAGE AREA
`
`51AY
`
`DATA
`STORAGE AREA
`
`51 BY
`
`OTHER-SYSTEM
`PROCESSOR AREA
`
`OTHER-SYSTEM
`PROCESSOR AREA
`
`HPE, Exh. 1017, p. 5
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 5 of 18
`
`US 7,849.260 B2
`
`FIG.5
`
`
`
`
`
`
`
`
`
`
`
`
`
`CONTROLLER if PROCESSOR if
`
`FLAG
`
`
`
`Host 0-0
`MPO
`cTLo
`o
`Host 0-1
`MP1
`cTL1
`1
`Host 1-0
`MPO
`cTLo
`2
`Host 0-2
`MP1
`cTL1
`3
`Host 0-3
`MPO
`cTLo
`4
`Host 1-1
`MP1
`cTL1
`5
`N-- N -- N-N-N--
`43
`43A(43B)
`43C
`43D
`4.3E
`
`
`
`
`
`
`
`HPE, Exh. 1017, p. 6
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 6 of 18
`
`US 7,849.260 B2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIG.6
`
`MEMORY
`
`
`
`MEMORY
`ADDRESS
`AREA
`OxO OOOOOOOO
`0x07FFFFFFF
`
`TOP
`SEGMENT SEGMENT
`BOCK
`ADDRESS
`a
`b
`
`LA000
`LA100
`
`Ox6 OOOOOOOO
`
`0x6 7FFFFFFF
`
`LBOOO
`
`LB100
`
`BOO
`
`44C
`
`44D
`
`A4E
`
`44F
`
`OPEN
`FLAG
`
`
`
`
`
`
`
`1
`
`44G
`44A(448)
`
`FG.7
`
`TOP
`MEMORY
`MEMORY ADDRESS SEGMENT SEGMENT TIME
`AREA
`BLOCK ADDRESS STAMP
`
`DIRTY
`FLAG
`
`A
`B
`c
`D
`E
`F
`
`CA000
`cAloo
`cA200
`CB000
`CB100
`cB200
`
`aaaa
`bbbb
`cc.cc
`ddidd
`eeee
`ffif
`
`1
`
`O
`
`45C
`
`45D
`
`45E
`
`45F
`
`45G
`
`45H
`45A(458)
`
`HPE, Exh. 1017, p. 7
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 7 of 18
`
`US 7,849.260 B2
`
`
`
`
`
`JlINT TOELNOO
`
`
`
`EEBISN\/bl1 VIVOJ
`
`ISOH
`
`WELLSÅS
`
`HETTOELLNOKO
`
`\79||
`
`WE.1SAS-0
`
`
`
`BIETTO HINOO
`
`HPE, Exh. 1017, p. 8
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 8 of 18
`
`US 7,849.260 B2
`
`0-SYSTEM CT PROCESSING
`
`1-SYSTEM CL PROCESSING
`
`START
`
`SP1
`
`
`
`
`
`O-SYSTEM HOST
`COMMUNCATION
`PROTOCOL CHIP RECEIVE
`WRITE COMMAND
`
`SP3
`
`
`
`
`
`
`
`
`
`
`
`SP7
`
`
`
`SP8
`
`
`
`SP9
`
`
`
`SP1 O
`
`
`
`
`
`
`
`
`
`
`
`
`
`O-SYSTEM HOST
`COMMUNICATION
`PROTOCOL CHP
`ACOURES OMA LIST
`FROM 1-SYSTEM LM
`
`
`
`
`
`O-SYSTEM DATA
`COMMUNICATION
`CONTROL UNIT RECEIVES
`WRITE DAA ACCORDING TO
`DMA ADDRESS AND
`STORES IT IN O-SYSTEM CM
`
`O-SYSTEM DATA
`TRANSFER CONTROL UNT
`REDUNDANTLY WRITES
`WRITE DATAN-SYSTEM CV
`
`O-SYSTEM HOST
`COMMUNICATION
`PROTOCOL CHP NOTIFIES
`COMPLETON OF DATA
`TRANSFER TO 1-SYSTEMMP
`
`
`
`
`
`1-SYSTEM MP
`RECOGNIZES
`RECEPTION OF
`WRE COMMAND
`
`CREATE DMA LIST AND
`STORE IT IN 1-SYSTEM LM
`
`BOOT -SYSTEM
`HOST COMMUNICATION
`PROTOCOL CHP
`
`SP4
`
`SP5
`
`SP6
`
`REPORT COMPLETON
`OF WRITE PROCESSING
`FROM SYSTEM
`MP WAO-SYSTEM
`HOST COMMUNICATION
`PROCESSOR
`
`DESTAGE WRITE DATA
`FRON 1-SYSTEM CM
`
`SP 11
`
`SP 12
`
`HPE, Exh. 1017, p. 9
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 9 of 18
`
`US 7,849.260 B2
`
`FIG.10
`
`TRANSFERWRITE COMMAND
`TO O-SYSTEM LM
`
`O-SYSTEM MP RECOGNIZES
`RECEPTION OF WRITE COMMAND
`
`CREATE DMA LST AND
`STORE T N 0-SYSTEM LM
`
`BOOTO-SYSTEM HOST
`COMMUNICATION PROTOCOL CHIP
`
`O-SYSTEM HOST COMMUNICATION
`PROTOCOL CHIP ACOURES DMA
`FROM O-SYSTEM LM
`
`OO-SYSTEM DATA COMMUNICATION
`CONTROL UNIT RECEIVES
`WRITE DATA ACCORDING TO DMA ADDRESS
`AND STOREST IN O-SYSTEM CM
`
`O-SYSTEM DATA TRANSFER CONTROL UNIT
`REDUNDANTLY WRITESWRITE DATA
`N 1-SYSTEM CM
`
`
`
`REPORT COMPLETION OF WRITE PROCESSING
`FROM O-SYSTEM MP VAO-SYSTEM HOST
`COMMUNICATION PROCESSOR
`
`PERFORM DESTAGING FROM O-SYSTEM CM
`
`SP13
`
`SP 14
`
`SP15
`
`SP16
`
`SP17
`
`SP 18
`
`SP 19
`
`SP2O
`
`SP21
`
`HPE, Exh. 1017, p. 10
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 10 of 18
`
`US 7,849,260 B2
`
`
`
`diIHd1090LOUd
`
`LL'Sid
`
`relaigNrndel
`Z+~||woe||+—~oeds
`
`NOLLVOINNWANOD
`LINNJOULNOD
`
`dIH1090L0ud
`
`N
`
`Hseenenannese
`
`
`
`OLLVOINAWAOD1SNOLLVOINNWWODLSOH0
`
`
`
`
`SiilvuVdd¥S9VHOLBSNLVuVdd¥SOVYOLS
`NOLYSINTIANOD|NOLLYSINAWWOD
`dIHD1O90LO"ddiHd10901L0Ud
`
`
`[LINNTO8LNOOp]BSHSNVerVive
`
`
`SNIWVEVddVSDVHO.SNLVYVddVSOVHOLS
`
`NOLLVOINONIWWODNOILVOINNWINOD
`LNAIOYLNODLINDJOHLNOS
`
`
`
`
`
`HPE, Exh. 1017, p. 11
`
`HPE, Exh. 1017, p. 11
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 11 of 18
`
`US 7,849.260 B2
`
`F.G. 12
`
`O-SYSTEM CTL PROCESSING
`
`1-SYSTEM CTL PROCESSING
`
`SP3O
`
`
`
`
`
`
`
`O-SYSTEM HOST
`COMMUNCATION
`PROTOCOL CHIP
`RECEIVES
`WRITE COMMAND
`
`SP31
`YESY is AssociaTEDMP OF N
`TARGET LU A 0-SYSTEM?/
`
`CB)
`SP32
`
`1-SYSTEV MP
`NO
`STORE WRITE COMMAND : - REcodNZESEEPTION
`IN 1-SYSTEM LM
`OF WRITE COMMAND
`
`SP4O
`
`
`
`
`
`SP41
`
`
`
`
`
`SP42
`
`
`
`
`
`
`
`
`
`SP43
`
`
`
`SP44
`
`
`
`O-SYSTEM HOST
`COMMUNICATION
`sist
`ACOUR
`FROM 1-SYSTEM LM
`
`O-SYSTEM HOST
`EMS
`SENESBAADSESs :
`TO O-SYSTEM DAA
`TRANSFER conTROL UNIT
`
`O-SYSTEM DATA
`COMMUNICATION CONTROL
`UNIT ACOURES AND
`CONFIRMS DATA
`MANAGEMENT
`INFORMATION ACCORDING
`TODMA ADDRESS
`
`
`
`O-SYSTEM DATA TRANSFER
`CONTROL UNIT TRANSFERS
`READ DATA FROM
`0-SYSTEM CM TO O-SYSTEM
`HOST COMMUNICATION
`PROTOCOL CHP
`
`TRANSFER DAATO
`HOST COMPUTER
`
`| |SSESAR.EgM
`TQAG5A
`TO 1-SYSTEM CM
`
`
`
`|| PERFORRENDANT
`FROM SISM
`TO O-SYSTEM
`iSSEE
`MANAGEMENT
`NFORMATION
`IN 1-SYSTEM CM
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`REDUNDANTLY WRITE
`DATA MANAGEMENT
`NFORMATION FROM
`1-SYSTEM CM
`TO O-SYSTEM CM
`
`CREATE DMAST AND
`STORE IT IN 1-SYSTEM LM
`
`BOOT O-SYSTE HOST
`COMMUNICATION
`PROTOCOL CHIP
`
`SP33
`
`SP34
`
`SP35
`
`SP36
`
`SP37
`
`SP38
`
`SP39
`
`HPE, Exh. 1017, p. 12
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 12 of 18
`
`US 7,849.260 B2
`
`COPY READ COMMAND TO O-SYSTEM LM
`
`O-SYSTEM MP RECOGNIZES RECEPTION OF COMMAND
`
`STAGE READ OATA FROM
`STORAGE APPARATUS TO O-SYSTEM CM
`
`PERFORM REDUNDANT WRITING FROM
`O-SYSTEM CMTO 1-SYSTEM CM
`
`O-SYSTEM MP SETS DATA MANAGEMENT
`INFORMATION IN 0-SYSTEM CM
`
`REDUNDANTLY WRITE DATA MANAGEMENT
`NFORMATION FROM O-SYSTEM CMTO 1-SYSTEM CM
`
`CREATE DMA LIST AND STORE IT IN O-SYSTEM LM
`
`BOOT O-SYSTEM HOST COMMUNICATION PROTOCOL CHIP
`
`O-SYSTEM HOST COMMUNICATION PROTOCOL CHIP
`ACOURES DMA LIST FROM O-SYSTEM LM
`
`SEND OMA ADDRESS TO O-SYSTEM
`HOST COMMUNICATION PROTOCOL CHIP
`AND O-SYSTEM DATA TRANSFER CONTROL UNIT
`
`O-SYSTEM DATA TRANSFER CONTROL UNIT ACORURES
`AND CONFIRMS DATA MANAGEMENT INFORMATION
`ACCORDING TO OMA ADDRESS
`
`O-SYSTEM DATA TRANSFER CONTROL UNIT
`TRANSFERS READ DATA FROM O-SYSTEM CM TO
`O-SYSTEM HOST COMMUNICATION PROTOCOL CHIP
`
`SEND DATA TO HOST COMPUTER
`
`SP45
`
`SP46
`
`SP47
`
`SP48
`
`SP49
`
`SP50
`
`SP51
`
`SP52
`
`SP53
`
`SP54
`
`SP55
`
`SP56
`
`SP57
`
`HPE, Exh. 1017, p. 13
`
`

`

`U.S. Patent
`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 13 of 18
`
`US 7,849.260 B2
`US 7,849,260 B2
`
`
`
`09
`
`
`
`YALNdWOD1SOH
`
`
`
`LINMTOULNOSD
`
`LINNTOULNOD
`
`
`
`SNLIWHVddVADVHOLS
`
`NOLLVOINNWIIOD
`
`dIHDJODOLONd
`
`NOILVOINNWNOD biSis
`YS4SSNV¥YLVIVOUSISNVEL
`
`LINNAOWLNODYATIOKLNOS
`
`NOLLVSINOINOD
`
`
`
`SNIVHVddVADVHOLSVIP
`
`HPE, Exh. 1017, p. 14
`
`HPE, Exh. 1017, p. 14
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 14 of 18
`
`US 7,849.260 B2
`
`FIG.15
`O-SYSTEM CTL PROCESSING
`1-SYSTEM CTL PROCESSING
`
`SP60
`
`O-SYSTEM HOST
`COMMUNICATION
`PROTOCOL CHIP RECEIVES
`WRITE COMMAND
`
`SP61
`YES
`
`C)
`SP62
`
`IS ASSOCATED MP OF
`TARGET LUA O-SYSTEM
`NO
`- STORE COMMAND
`
`SP66
`
`SP67
`
`SP68
`
`SP69
`
`SP7O
`
`SP71
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`COPY OMA LIST TO
`SELF-SYSTEM CTL RECEIVE
`COMMAND SORAGE AREA
`OF O-SYSTEM LM
`
`BOOT O-SYSTEM
`HOST COMMUNICATION
`PROTOCOL CHIP
`
`O-SYSTEM
`HOST COMMUNICATION
`PROTOCOL CHIP
`ACOUIRES DMA LIST
`FROM O-SYSTEM LM
`
`O-SYSTEM DATA
`COMMUNICATION
`CONTROL UNIT RECEIVES
`WRITE DATA ACCORDING
`TO DMA ADDRESS AND
`STOREST IN O-SYSTEM CM
`
`O-SYSTEM DATA TRANSFER
`CONTROL UNIT
`REDUNDANTLY WRITES
`WRITE DATAN 1-SYSTEM CM
`
`O-SYSTEM
`HOST COMMUNICATION
`PROTOCOL CHIP NOTIFIES
`COMPLETION OF DATA
`TRANSFERTO 1-SYSTEM MP
`
`
`
`
`
`SP72
`
`REPORT COMPLETION
`OF WRITE PROCESSING
`FROM O-SYSTEMMP
`VAO-SYSTEM HOS
`COMMUNICATION PROCESSOR
`
`SP63
`
`SP64
`
`SP65
`
`
`
`
`
`1-SYSTEMMP
`RECOGNIZES
`RECEPTION OF
`WRITE COMMAND
`
`CREATE OMA LIST
`AND STORE IT
`IN 1-SYSTEM LM
`
`COPY DMA LIST TO
`OTHER-SYSTEM CTL
`RECEIVE COMMAND
`STORAGE AREA
`OF O-SYSTEM LM
`
`SP73
`PERFORM DESAGNG
`FROM 1-SYSTEM CM
`
`HPE, Exh. 1017, p. 15
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 15 of 18
`
`US 7,849.260 B2
`
`FIG.16
`
`TRANSFER COMMAND TO O-SYSTEM LM
`
`O-SYSTEM MP RECOGNIZES RECEPTION
`OF WRITE COMMAND
`
`CREATE DMA LIST AND STORE IT IN 0-SYSTEM LM
`
`BOOT 0-SYSTEM HOST
`COMMUNICATION PROTOCOL CHIP
`
`O-SYSTEM HOST COMMUNICATION PROTOCOL CHIP
`ACGUIRES DMA FROM O-SYSTEM LM
`
`0-SYSTEM DATA COMMUNICATION CONTROL UNIT
`RECEIVES WRITE DATA ACCORDING
`TO DMA ADDRESS AND STOREST IN 0-SYSTEM CM
`
`O-SYSTEM DATA TRANSFER CONTROL UNIT
`REDUNDANTLY WRITES WRITE DATA IN 1-SYSTEM CM
`
`
`
`REPORT COMPLETION OF WRITE PROCESSING
`FROM O-SYSTEM MPVA 0-SYSTEM
`HOST COMMUNICATION PROCESSOR
`
`PERFORM DESTAGING FROM 0-SYSTEM CM
`
`SP74
`
`SP75
`
`SP76
`
`SP77
`
`SP78
`
`SP79
`
`SP8O
`
`SP81
`
`SP82
`
`HPE, Exh. 1017, p. 16
`
`

`

`U.S. Patent
`
`
`
`US 7,849.260 B2
`
`
`
`
`
`DOEZ, TOEZIE), Dae
`
`
`
`
`
`LINT TOELLNOS)
`
`HPE, Exh. 1017, p. 17
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 17 of 18
`
`US 7,849.260 B2
`
`O-SYSTEM CTL PROCESSING
`
`-SYSTEM CTL PROCESSING
`
`SP90
`
`SP91
`YES
`
`GD)
`SP92
`
`O-SYSTEM HOS
`COMMUNICATION
`PROTOCOCHIP RECEIVES
`REAO COMMAND
`
`IS ASSOCATED MP OF
`TARGE UA O-SYSTEM
`NO
`
`STORE READ COMMAND
`N -SYSTEM LM
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SPOO
`
`SP1 O1
`
`SP 102
`
`SP 103
`
`SPO4
`
`SP1 O5
`
`SP 106
`
`SP1 O7
`
`SP1 O8
`
`COPY DMA IST TO
`SELF-SYSTEM CTL
`RECEIVE COMMAND STORAGE
`AREA OF O-SYSTEM LM
`
`BOOT O-SYSTEM
`HOST COMMUNICATION
`PROTOCO CHIP
`
`O-SYSTEM HOST
`COMMUNICATION PROTOCOL
`CHP ACQUIRES OMA LIST
`FROM O-SYSTEM LM
`
`O-SYSTEM HOST
`COMMUNICATION
`PROTOCOL CHP SENDS
`DMA ADDRESSO
`O-SYSTEM DATA TRANSFER
`CONTROL UNIT
`system DAAtransfer
`cSESSEs
`MAAENNERYAgN
`
`O-SYSTEM DATA TRANSFER
`CONTROL UNIT SENDS
`READ DATAFROMOSYSTEM
`CMTO O-SYSTEM
`HOST COMMUNICATION
`PROTOCOL CHiP
`
`TRANSFER DATA TO
`HOST COMPUTER
`
`
`
`O-SYSTEM HOST
`COMMUNICATION
`PROTOCOL CHP
`STORES COMPLETION
`INFORMATION IN 0-SYSTEM LM
`
`COPY COMPETION
`NFORMATION TO 1-SYSTEM LM
`
`
`
`
`
`1-SYSTEM MP
`RECOGNIZES
`RECEPTION OF
`WRITE COMMAND
`
`issista Right
`
`TO 1-SYSTEM CM
`
`PERFORM, REDUNDAN
`
`SP93
`
`4.
`
`SP 95
`
`Sg
`| listers
`
`O-SYSTEM CM
`
`
`
`NFORMATION
`IN 1-SYSTEM CM
`
`REDUNDANTLY WRITE
`OAA MANAGEMENT
`NFORMATION FROM
`-SYSTEM CM
`TO O-SYSTEM CM
`
`CREATE OMA LIST AND
`LSTOREIT INSYSTEMLM
`c3EEso
`
`STORAGE AREA
`OF O-SYSTEM LM
`
`SP97
`
`SP98
`
`SP99
`
`SP 109
`VONITOR 1 -SYSTEM LM
`AND EXECUTE
`COMPLETION PROCESSING
`
`HPE, Exh. 1017, p. 18
`
`

`

`U.S. Patent
`
`Dec. 7, 2010
`
`Sheet 18 of 18
`
`US 7,849.260 B2
`
`COPY COMMAND TO O-SYSTEM LM
`
`O-SYSTEM MP RECOGNIZES RECEPTION OF COMMAND
`
`STAGE READ OATA FROM
`STORAGE APPARATUS TO O-SYSTEM CM
`
`PERFORM, REDUNDANT WRITING
`FROM O-SYSTEM CMO 1-SYSTEM CM
`
`O-SYSTEM MP SET DATA MANAGEMENT
`INFORMATION IN 0-SYSTEM CM
`
`REDUNDANTLY WRITE DATA MANAGEMENT
`INFORMATION FROM O-SYSTEM CMTO 1-SYSTEM CM
`
`CREATE OMA LIST AND STORE T N 0-SYSTEM LM
`
`BOOT O-SYSTEM HOST COMMUNICATION PROTOCOL CHP
`
`O-SYSTEM HOST COMMUNICATION PROTOCO CHP
`ACOURES DMA FROM O-SYSTEM LM
`
`SEND OMA LIST TO O-SYSTEM HOST
`COMMUNICATION PROTOCOL CHIP
`AND O-SYSTEM DATA TRANSFER CONTROL UNIT
`
`O-SYSTEM DATA TRANSFER CONTROL UNIT
`ACGURES AND CONFERNS DATA MANAGEMENT
`INFORMATION ACCORDING TO OMA ADDRESS
`
`O-SYSTEM DATA TRANSFER CONTROL UNIT
`SENDS READ DATA FROM O-SYSTEM CMTO
`O-SYSTEM HOST COMMUNICATION PROTOCOL CHIP
`
`SEND DATA TO HOST COMPUTER
`
`SP1 1 O
`
`SP 111
`
`SP 112
`
`SP 113
`
`SP 114
`
`SP 115
`
`SP1 16
`
`SP 117
`
`SP118
`
`SP119
`
`SP 12O
`
`SP 121
`
`SP 122
`
`HPE, Exh. 1017, p. 19
`
`

`

`US 7,849,260 B2
`
`1.
`STORAGE CONTROLLER AND CONTROL
`METHOD THEREOF
`
`CROSS REFERENCES
`
`This application relates to and claims priority from Japa
`nese Patent Application No. 2006-319807, filed on Nov. 28,
`2006, the entire disclosure of which is incorporated herein by
`reference.
`
`BACKGROUND
`
`The present invention generally relates to a storage con
`troller and its control method, and in particular is suitable for
`application in a storage apparatus comprising a plurality of
`microprocessors for controlling I/O requests of information
`from a host computer.
`Conventionally, a storage apparatus has been demanded of
`higher reliability and faster response. In light of this, technol
`ogy is known for redundantly storing data between a plurality
`of controllers, and executing data processing through distri
`bution. Japanese Patent Laid-Open Publication No.
`H9-146842 discloses technology for executing processing
`based on a command received by one controller with both
`processors; namely, a processor comprising the controller
`that received the command, and a processor comprising the
`other controller.
`
`10
`
`15
`
`25
`
`SUMMARY
`
`2
`processing time in response to a command in a simple manner
`while reducing the load of the controller that received a com
`mand targeting a non-associated logical Volume.
`In order to achieve the foregoing object, the present inven
`tion provides a storage controller for controlling the input and
`output of data to and from a plurality of logical units between
`a host computer as an upper-level device, and a storage appa
`ratus that provides the logical units configured from a storage
`extent for reading and writing data from and in the host
`computer. The storage controller comprises a plurality of
`controllers having a local memory for retaining a command
`given from the host computer, and which control the input and
`output of data to and from the corresponding logical unit
`based on the command retained in the local memory, and an
`inter-controller connection path for connecting the plurality
`of controllers in a communicable state. The local memory
`stores association information representing the correspon
`dence of the logical units and the controllers, and address
`information of the local memory in each of the controllers of
`a self-system and another-system. Upon receiving a com
`mand sent from the host computer, the controller determines
`whether the target logical unit is associated with the controller
`of a self-system or another-system based on the association
`information, and, when the logical unit is associated with the
`other-system controller, the controller transfers and stores the
`command to and in the corresponding other-system controller
`based on the address information.
`Thereby, with this storage controller, when the controller
`receives a command targeting a logical unit associated with
`another-system controller, the processing of the controller
`that received the command communicating with the other
`system controller and requesting the other-system controller
`to perform processing according to Such command will no
`longer be required. As a result, upon the transfer of the com
`mand, it is possible to effectively prevent an unnecessary load
`from arising in the controller that received the command
`based on the communication, and a delay from occurring in
`the processing time in response to the command. Further, in
`the foregoing case, there is no need to configure any setting in
`the host computer.
`The present invention also provides a control method of a
`storage controller for controlling the input and output of data
`to and from a plurality of logical units between a host com
`puter as an upper-level device, and a storage apparatus that
`provides the logical units configured from a storage extent for
`reading and writing data from and in the host computer. The
`storage controller comprises a plurality of controllers having
`a local memory for retaining a command given from the host
`computer, and which control the input and output of data to
`and from the corresponding logical unit based on the com
`mand retained in the local memory, and an inter-controller
`connection path for connecting the plurality of controllers in
`a communicable state. The local memory stores association
`information representing the correspondence of the logical
`units and the controllers, and address information of the local
`memory in each of the controllers of a self-system and
`another-system. The control method comprises a first step of
`the controller, upon receiving a command sent from the host
`computer, determining whether the target logical unit is asso
`ciated with the controller of a self-system or another-system
`based on the association information, and a second step of the
`controller, when the logical unit is associated with the other
`system controller, transferring and storing the command to
`and in the corresponding other-system controller based on the
`address information.
`Thereby, with this control method, when the controller
`receives a command targeting a logical unit associated with
`
`30
`
`35
`
`40
`
`45
`
`Meanwhile, the storage Subsystem disclosed in Japanese
`Patent Laid-Open Publication No. H9-146842 is a storage
`Subsystem comprising multiplexed controllers. The proces
`sor of each controller is associated with a logical volume, and,
`upon receiving a command targeting a non-associated logical
`Volume, request processing to the processor associated with
`Such logical Volume is executed.
`Nevertheless, with this storage subsystem, even when the
`controller receives a command targeting a non-associated
`logical Volume from the host computer, the data transfer
`between the host computer and the cache memory based on
`Such command is processed with the processor in the control
`ler that received the command. Thus, with the foregoing
`storage Subsystem, when the number of commands received
`between a plurality of multiplexed controllers becomes
`biased, there is a problem in that the load will be concentrated
`on the controller that received numerous commands.
`Further, with the foregoing storage Subsystem, when the
`controller receives a command targeting a non-associated
`logical volume, that controller will have to analyze the com
`50
`mand and communicate with the controller associated to Such
`logical Volume so as to request Such controller to perform
`processing according to the command, and there is a problem
`in that the controller that received the command targeting the
`non-associated logical volume will be burdened with an
`unnecessary load, and the processing time of the overall Stor
`age Subsystem in response to the command will be delayed.
`Accordingly, with the foregoing storage Subsystem, in
`order to speed up the processing time of the overall Subsystem
`in response to the command, it is necessary to set the host
`computer to issue commands to microprocessors and control
`lers associated with the target logical Volume, and there is a
`problem in that much labor and time are required for such
`Setting process.
`The present invention was made in view of the foregoing
`points. Thus, an object of the present invention is to propose
`a storage controller and its control method for speeding up the
`
`55
`
`60
`
`65
`
`HPE, Exh. 1017, p. 20
`
`

`

`US 7,849,260 B2
`
`3
`another-system controller, the processing of the controller
`that received the command communicating with the other
`system controller and requesting the other-system controller
`to perform processing according to such command will no
`longer be required. As a result, upon the transfer of the com
`mand, it is possible to effectively prevent an unnecessary load
`from arising in the controller that received the command
`based on the communication, and a delay from occurring in
`the processing time in response to the command. Further, in
`the foregoing case, there is no need to configure any setting in
`the host computer.
`According to the present invention, it is possible to realize
`a storage controller and its control method capable of speed
`ing up the processing time in response to a command in a
`simple manner while reducing the load of the controller that
`received a command targeting a non-associated logical Vol
`le.
`
`5
`
`10
`
`15
`
`4
`FIG. 16 is a flowchart explaining write command process
`ing in a storage system according to another embodiment of
`the present invention;
`FIG. 17 is a block diagram explaining read command pro
`cessing in a storage system according to another embodiment
`of the present invention;
`FIG. 18 is a flowchart explaining read command process
`ing in a storage system according to another embodiment of
`the present invention; and
`FIG. 19 is a flowchart explaining read command process
`ing in a storage system according to another embodiment of
`the present invention.
`
`DETAILED DESCRIPTION
`
`An embodiment of the present invention is now explained
`with reference to the attached drawings.
`
`DESCRIPTION OF DRAWINGS
`
`(1) First Embodiment
`
`25
`
`30
`
`35
`
`40
`
`FIG. 1 is a block diagram showing the configuration of a
`storage system according to an embodiment of the present
`invention;
`FIG. 2A is a chart showing an example of a memory space
`to be recognized by the 0-system and 1-system microproces
`sors, and FIG. 2B is a chart showing an example of a memory
`space to be respectively recognized by the 0-system and
`1-system host communication protocol chips 10A, 10B;
`FIG. 3A is a conceptual diagram showing the memory
`configuration of a 0-system local memory, and FIG. 3B is a
`conceptual diagram showing the memory configuration of a
`1-system local memory;
`FIG. 4A is a conceptual diagram showing the memory
`configuration of a 0-system cache memory, and FIG. 4B is a
`conceptual diagram showing the memory configuration of a
`1-system cache memory;
`FIG. 5 is a conceptual diagram showing the configuration
`of a logical unit/processor association table;
`FIG. 6 is a conceptual diagram showing the configuration
`of a local memory information table;
`FIG. 7 is a conceptual diagram showing the configuration
`of a cache memory information table;
`FIG. 8 is a block diagram explaining write command pro
`cessing in a storage system according to an embodiment of
`45
`the present invention;
`FIG.9 is a flowchart explaining write command processing
`in a storage system according to an embodiment of the present
`invention;
`FIG. 10 is a flowchart explaining write command process
`ing in a storage system according to an embodiment of the
`present invention;
`FIG. 11 is a flowchart explaining read command process
`ing in a storage system according to an embodiment of the
`present invention;
`FIG. 12 is a flowchart explaining read command process
`ing in a storage system according to an embodiment of the
`present invention;
`FIG. 13 is a flowchart explaining read command process
`ing in a storage system according to an embodiment of the
`present invention;
`FIG. 14 is a block diagram explaining write command
`processing in a storage system according to another embodi
`ment of the present invention;
`FIG. 15 is a flowchart explaining write command process
`ing in a storage system according to another embodiment of
`the present invention;
`
`50
`
`55
`
`60
`
`65
`
`(1-1) Configuration of Storage System in Present Embodi
`ment
`FIG. 1 shows the overall storage system 1 according to the
`present embodiment. The storage system 1 is configured by
`host computers 2A, 2B being connected to a plurality of
`storage apparatuses 4A to 4D via a storage controller 3.
`The host computers 2A, 2B, for instance, are computers
`comprising information processing resources such as a CPU
`(Central Processing Unit) and a memory, and are specifically
`configured from a personal computer, a workStation, a main
`frame or the like. The host computers 2A, 2B are provided
`with a communication port (for example, a port provided to a
`LAN card or a host bus adapter) for accessing the storage
`controller 3, and are able to send a data I/O request command
`to the storage controller 3 via this communication port.
`The storage controller 3 is configured from 0-system and
`1-system controllers 6A, 6B respectively connected to differ
`ent host computers, and an inter-controller connection path 5
`for connecting these controllers 6A, 6B in a communicable
`State.
`As the inter-controller connection path 5, for instance, a
`bus based on a PCI (Peripheral Component Interconnect)-
`Express standard for realizing high-speed data communica
`tion where the data transfer volume per direction of one lane
`(maximum of 8 lanes) is 2.5 Gbit/sec is used. The transfer of
`data and various information between the 0-system and 1-sys
`tem controllers 6A, 6B as described above is all conducted via
`the inter-controller connection path 5.
`The respective controllers 6A, 6B are used for controlling
`the reading and writing of data from and in the storage appa
`ratuses 4A to 4D according to a request from the host com
`puters 2A, 2B respectively connected to a self controller, and
`comprise host communication control units 10A, 10B, data
`transfer control units 11A, 11B, cache memories 12A, 12B,
`bridges 13A, 13B, local memories 14A, 14B, microproces
`sors 15A, 15B, storage apparatus communication control
`units 16A, 16B, storage apparatus-side switches 17A, 17B.
`and the like.
`Among the above, the host communication control units
`10A, 10B are interfaces for performing communication con
`trol with the host computers 2A, 2B, and have a plurality of
`communication ports 20A, 20B, and host communication
`protocol chips 21A, 21B.
`The communication ports 20A, 20B are used for connect
`ing the controllers 6A, 6B to a network or the host computers
`2A, 2B, and, for instance, are respectively allocated with a
`
`HPE, Exh. 1017, p. 21
`
`

`

`5
`unique network address such as an IP (Internet Protocol)
`address or a WWN (World Wide Name).
`The host communication protocol chips 21A, 21B perform
`protocol control during communication with the host com
`puters 2A, 2B. Thus, as the host communication protocol
`chips 21A, 21B, for example, a fibre channel conversion
`protocol chip is used when the communication protocol with
`the host computers 2A, 2B is a fibre channel (FC: Fibre
`Channel) protocol, and an iSCSI protocol chip is used when
`the communication protocol is an iSCSI protocol. In other
`words, an adequate protocol chip is applied to the communi
`cation protocol with the host computers 2A, 2B.
`Further, the host communication protocol chips 21A, 21B
`are equipped with a multi microprocessor function for
`enabling the communication between a plurality of micropro
`cessors, and the host communication protocol chips 21A, 21B
`are thereby able to communicate with both the microproces
`sor 15A in the 0-system controller 6A and the microprocessor
`15B in the 1-system controller 6B.
`The data transfer control units 11A, 11B have a function for
`controlling the data transfer between the 0-system and 1-sys
`tem controllers 6A, 6B and the data transfer between the
`respective elements in the 0-system controller 6A or the
`1-system controller