(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2003/0093509 A1
`(43) Pub. Date:
`May 15, 2003
`Li et al.
`
`US 20030093509A1
`
`(54) STORAGE AREA NETWORK METHODS
`AND APPARATUS WITH COORDINATED
`UPDATING OF TOPOLOGY
`REPRESENTATION
`(76) Inventors: Raymond M. Li, Oakland, CA (US);
`William Roy Yonker, Rochester, MN
`(US)
`Correspondence Address:
`NUTTER, McCLENNEN & FISH, LLP
`ATTORNEYS AT LAW
`ONE INTERNATIONAL PLACE
`BOSTON, MA 02110-2699 (US)
`(21) Appl. No.:
`09/972,542
`(22) Filed:
`Oct. 5, 2001
`
`Publication Classification
`
`(51) Int. Cl." ............................................. G06F 15/173
`(52) U.S. Cl. .............................................................. 709/223
`
`(57)
`
`ABSTRACT
`
`A digital data processing apparatus of the type that manages
`a SAN includes a first queue with entries representing tasks
`and a Second queue with entries representing data that
`correspond to those tasks. Data in the Second queue is
`grouped in accord with the task to which they correspond. A
`manager Service updates the internal representation of the
`SAN (e.g., the representation of the SAN topology) by
`executing the tasks in the first queue one at a time, for
`example, atomically using a single-threaded process.
`
`O N
`
`
`
`12a
`
`6a
`
`Fabric interconnect
`components
`
`16
`
`
`
`
`
`
`
`Unintelligerit
`Oisks
`-
`
`RADController
`
`
`
`Disk/tape Storage
`Subsystems
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 1 of 96
`
`

`

`Patent Application Publication May 15, 2003. Sheet 1 of 47
`
`US 2003/0093.509 A1
`
`
`
`12a
`
`18
`
`16a
`
`Fabric interconnect
`components
`
`
`
`
`
`16
`
`
`
`Unintelligert
`Disks
`
`RAD Controller
`
`
`
`Disk/tape Storage
`Subsystems
`
`
`
`14b
`
`4C
`
`Figure 1
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 2 of 96
`
`

`

`Patent Application Publication May 15, 2003. Sheet 2 of 47
`
`US 2003/0093.509 A1
`
`TSNM Console
`
`SNMP traa/EC console
`
`s Er
`NN // s
`
`N
`
`20
`
`I
`FE
`Managed Host
`
`
`
`
`
`Switch
`
`C C
`JBOD
`
`
`
`TSNM Manager
`
`W
`
`L
`sEE
`Managed Host
`
`Unmanaged Host
`
`7
`
`Printer
`
`Figure 2
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 3 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 3 of 47
`
`US 2003/0093.509 A1
`
`
`
`quêuoduop 136eue.W
`
`£ ?an61+
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 4 of 96
`
`

`

`Patent Application Publication May 15, 2003. Sheet 4 of 47
`
`US 2003/0093.509 A1
`
`# Bun614
`
`No.kigoaa5eko A
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 5 of 96
`
`

`

`Patent Application Publication May 15, 2003. Sheet 5 of 47
`
`US 2003/0093.509 A1
`
`
`
`G ?un614
`
`X?JONA?Uueu
`
`16WunTNWS
`
`
`
`quæuuæ6eue.W NOT
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 6 of 96
`
`

`

`Patent Application Publication May 15, 2003. Sheet 6 of 47
`
`US 2003/0093.509 A1
`
`2O
`
`64
`
`52
`
`E Jaya client
`RS:
`
`NetView Console is
`
`"S.O.
`
`i.
`:
`NetView Serve
`El Jacent 74-se
`Jet
`essewer
`54a
`
`P & s p 283's
`
`.
`
`.
`.
`.
`.
`NetView
`Requester
`8
`
`A B B A
`is sits spy
`endor Provided
`Migmt App
`basis is sees
`
`a sy
`
`Q1
`
`N, NaNs
`N2 N3 ...
`t-2-
`02 ss
`58
`
`56
`n
`Netview Daemony
`.
`.
`.
`.
`.
`in
`.
`.
`.
`.
`is a
`Console Req Handler
`62
`h :: EE",Israel L.1
`Emper sang us
`a esses is
`
`N.
`
`Directory Service
`
`Persistence &
`Activation Service
`Composite
`
`cs
`
`SAN Manager
`Service
`eig compare
`toueues) process
`
`50
`
`Event service
`EventService
`
`Discover Engine
`
`s: b c s vs 1
`
`B vs B say
`
`: Agent Monitor
`
`3 s is k
`
`s p as is ss
`
`sea A.
`
`Centrol
`
`Pine Air
`
`Conreate
`
`40
`
`A8
`
`4Ob
`
`
`
`Event Correlator
`
`etc.
`
`se
`X-Reference
`
`40a
`
`Query Engine
`Agent
`
`48
`
`-------
`i Event i
`Agent
`!------
`
`Figure 6
`
`is: vir's
`SNMP Gateway
`:
`E
`44
`is O.C." Scanner
`SNMP trap,
`is
`SubAgent
`: SNMP Agent?
`-------
`Subagent
`Execute
`
`-------
`
`
`
`assa
`
`is is is
`
`A bass
`
`-------- -
`Scanner
`...Nc42
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 7 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 7 of 47
`
`US 2003/0093.509 A1
`
`
`
`
`
`6u?ss320ad GEGEHOXHTCITOHSHHHITWELSÅSETIH
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 8 of 96
`
`

`

`Patent Application Publication May 15, 2003. Sheet 8 of 47
`
`US 2003/0093.509 A1
`
`J©IpUIBH|× 8
`
`9
`
`uo?ov
`
`8/ Ðunbl
`
`
`
`
`
`Œ> | .uonbuo nv
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 9 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 9 of 47
`
`US 2003/0093.509 A1
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 10 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 10 of 47 US 2003/0093509 A1
`
`Page 83h
`Standard
`
`
`
`
`
`Storage Subsystem. A
`
`
`
`Standard
`
`Page 83h
`Standard
`
`
`
`as
`
`14d E1 P
`
`Storage Subsystem C
`
`i
`
`14c 1.
`Storage Subsystem B
`
`Figure 9
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 11 of 96
`
`

`

`tnetaP
`
`P
`
`May 15, 2003 Sheet 11 0f 47
`
`US 2003/0093.509 A1
`US 2003/0093509 A1
`
`
`
`mo.3..
`
`
`
`aahimEa:
`
`
`
`NfiMu3mm
`
`
`
`ammcms.z<m
`
`.2839.En:a.233.926
`
`
`
`WE(imtmu‘ucu22.29293
`
`
`23%Eao.as:2=5...._23.2.2.2“”23.85.35.223
`
`
`ucmmzv3023n3%
`
`U05-2:202....20mini32335.2?minim
`mm:2323..__u23.25
`
`
`
`
`
`5.8..6mm5a.:3=50.m__.%sm.2waamfimmfilfimuss1353:52in5053522%
`
`p822m;,‘
`
`
`35065c:3llw50225.283
`mmmfim
`.l”2335.35.293E.”San...2.5m"mason.Bn=_u3.uar_a_=g=2u::
`
`R25,523"..9.86220.525>«on3mas—833530:mum:
`
`m25822522.9€32".
`
`
`.l.395).323255..5&9.35t“23823,355....3.22%»;2:0sagesw35%a3%82343235483?..2.523:8so.2322am
`
`.l«(ltrllt.1M50Emmcms.V65conmE23<@9305u‘y‘-131I}‘ill
`
`SN
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`LW>_._Dhwy—ELoom—.55—xwa.5CCm0w£39t3<memcmfiZ<m
`
`
`
`
`
`
`
`mm\
`?????????????
`
`
`nzfij€5.20:EBbmgmzvsfifica.32:».\.553.333Eu2325.8%..
`ON:2923ugna_§§§53.=8=Emo3.nEB.m92...:
`
` we23.:£3a.3
`??????????????????????v^ —
`
`..e23or5:332
`
`
`
`
`
`
`
`
`Baa—E:xnfi.EB.no.234.3E»flaw23:2»95:mm632.3522582.61am333wE8.—9234rap—Ea$20an
`
`
`
`852995some83%826958.6“..coszo_E_.mamn293.531
`
`a:a.23as5EcoEma=52:D.23205..a:«man.5228”2.2%3a.22:83.33322%«=5
`
`<m¢
`
`CISCO SYSTEMS, INC. / Page 12 of 96
`
`EX.1021
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 12 of 96
`
`
`
`
`
`
`

`

`Patent Application Publication May 15, 2003 Sheet 12 of 47 US 2003/0093.509 A1
`
`82 YN
`SANAdminClient
`
`88
`
`78
`
`LU group information
`
`UN assignment
`Request
`
`SANS to Auto
`Policy Engine
`
`N.
`
`N &
`host
`assignment
`foration
`
`Host & LJN.
`informatic
`
`Extend Request
`m -: SANAgent
`84
`FSExtend
`
`
`
`80
`
`Monitoring Policy
`86
`A
`threshold Ewent
`SANAgentScheduler
`SANAgent i
`FSMonitor
`
`72
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SANAgent
`Scanner
`.
`
`- - - - - - - - - - - m as
`
`Attribute Scanner
`
`SANAgent
`HostQuery
`
`Figure 11
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 13 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 13 of 47 US 2003/0093.509 A1
`
`90 N
`
`92
`
`Abstract 1
`int a1
`
`
`
`94
`
`Abstract 2
`int a2
`
`
`
`96
`
`Concrete 1
`int a 3
`
`Figure 12
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 14 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 14 0f 47
`
`US 2003/0093509 A1
`
`mgHBum038mequ
`nosofiomcoogeo\.EE;m
`
`HmpcflHSSE?mm
`
`"98BOwEOBmoB\.maEm
`
`NRoomofififlmbm
`
`mmpcflmBurma?
`
`vm
`
`\“QEsommBoomoESmEom
`080.EQwEOBmBQ
`
`llllllllllllllllllll
`
`mm“CHM822808
`
`fl959”.
`
`CISCO SYSTEMS, INC. / Page 15 of 96
`
`EX.1021
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 15 of 96
`
`
`
`
`

`

`Patent Application Publication May 15, 2003 Sheet 15 of 47 US 2003/0093509 A1
`
`transacto
`ompersist
`
`
`
`
`
`Apparent
`rampersist
`
`
`
`estAStact
`foresterss
`
`Figure 14a
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 16 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 16 of 47 US 2003/0093509 A1
`
`
`
`Associaret
`forapersis
`
`eFeer
`
`entespersis
`
`geS
`
`soc
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 17 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 17 of 47 US 2003/0093.509 A1
`
`
`
`PeerObject
`(fornperss
`
`setPeTimeOfSave)
`
`SetupSchenna
`(ost persst).
`- E.
`
`
`
`
`
`
`
`DScheila
`tempt risk ty
`EE
`aScte a0
`registerClasses0
`
`Figure 14c
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 18 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 18 of 47 US 2003/0093.509 A1
`
`false
`Newter. boclear
`deleted term 'bocean - false
`USE OPJMSICLOCKNG boolean= true
`USE BES OPTIMSTCLOCKING: boolean false
`
`Efosaeo
`d)
`seiPeertimeOfSave)
`EpiSaved
`setUrque
`PeerParent)
`PeerParent
`save)
`deleteO
`retreweby Eg:
`Ef converture 38tectido
`rettewegeneral
`retnevealObjecis0.
`
`
`
`a nerfaces
`RPersistake
`(from rub)
`registerschema)
`
`Peer Assoc
`(from persist
`
`
`
`setUnique
`getPeeTimeOSave()
`setPeemed Saves
`setPeersmeofSave
`
`PersistException
`(Sompensists
`
`
`
`eststExc E8
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 19 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 19 of 47
`
`US 2003/0093.509 A1
`
`
`
`
`
`
`
`
`
`
`
`Je^ues InÐ|
`
`
`
`• 86
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 20 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 20 of 47
`
`US 2003/0093.509 A1
`
`(
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 21 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 21 of 47 US 2003/0093.509 A1
`
`
`
`ZL e Inôl
`
`
`
`„ ........ yzwo48079001 OZEGO?z W8||
`
`
`
`
`
`
`
`Haynoq w <
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 22 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 22 of 47
`
`US 2003/0093509 A1
`
`
`
`9 | eun61-I
`
`
`
`
`
`0 || ||
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 23 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 23 of 47 US 2003/0093.509 A1
`
`
`
`
`
`6), ?Inôl
`
`6 si sselppº di IsoH
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 24 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 24 of 47
`
`US 2003/0093.509 A1
`
`SASIN/m ?
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 25 of 96
`
`

`

`Patent Application Publication
`
`US 2003/0093.509 A1
`
`
`
`?z eun61
`
`-20343404$.
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 26 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 26 of 47
`
`US 2003/0093.509 A1
`
`
`
`
`
`ZZ ?un61
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 27 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 27 of 47 US 2003/0093.509 A1
`
`
`
`Adapter 1
`
`Adapter2
`
`; Switch 1
`
`Figure 23
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 28 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 28 of 47
`
`US 2003/0093.509 A1
`
`315
`
`
`
`delete all
`virtual san
`references for every
`port in that
`Collection
`
`
`
`create collections of
`ports thatform
`potential virtual
`SANS from discover
`info
`
`Yes
`
`Any portin
`collection have a
`reference to a real
`SAN2
`
`311
`
`313
`
`317
`
`39
`
`create new virtual
`SAN for collection
`
`remove virtual SANs
`with no ports
`
`
`
`327
`
`Figure 24
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`current virtual SAN
`assigned to
`collection?
`
`renOWe
`SanPortRelationship
`for every port in the
`SAN not in
`collection
`
`remove all
`SanPortRelationship
`except for those
`that reference
`tempSan
`
`Create a new
`SanPortRelationshipo
`from tempSan to each
`port in collection that
`does not already have
`a relationship
`
`YY fa
`
`323
`
`325
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 29 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 29 of 47
`
`US 2003/0093.509 A1
`
`/
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 30 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 30 of 47
`
`US 2003/0093.509 A1
`
`
`
`109dsnS
`
`AA@N
`
`• • • • • • • • •• • • •* • ***
`
`? ? ? ? ?• • • • • • • • • • ** = &
`
`:T
`
`| |
`
`– – – – – r – – – – †
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 31 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 31 of 47 US 2003/0093.509 A1
`
`152
`
`
`
`54
`
`Request for File System
`Extension
`
`Identify from among accessible
`storage devices those having
`storage capacities in a range
`between a lower bound and an
`upper bound
`
`Was any storage
`device identified?
`
`
`
`
`
`
`
`
`
`56
`
`Assign the storage
`device from among
`the identified
`devices with
`-
`maximum capacity
`to the host
`
`
`
`
`
`
`
`Select a plurality of a accessible
`storage devices having a
`combined storage capacity that
`equals or exceeds the lower
`bound
`
`
`
`
`
`160
`
`Assign the selected
`storage devices to
`the host.
`
`Figure 27
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 32 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 32 of 47 US 2003/0093.509 A1
`
`
`
`S
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 33 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 33 of 47
`
`US 2003/0093.509 A1
`
`
`
`6Z ?un61-I
`
`
`
`Z/ |
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 34 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 34 of 47
`
`US 2003/0093.509 A1
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 35 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 35 of 47
`
`US 2003/0093.509 A1
`
`
`
`ZOZ
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 36 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 36 of 47 US 2003/0093509 A1
`
`
`
`>
`
`s
`
`S.
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 37 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 37 of 47 US 2003/0093.509 A1
`
`
`
`
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 38 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 38 of 47 US 2003/0093.509 A1
`
`008
`
`
`
`
`
`(gÐ) punoque MOT
`
`NOT – (gÐ)
`~]
`
`
`
`punoq 13.ddn
`
`sedK!
`
`/
`
`dno.16 ) SOH
`
`
`
`
`
`sæpuedoud dno.19 %soH
`
`
`
`
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 39 of 96
`
`

`

`May 15, 2003 Sheet 39 of 47
`
`US 2003/0093.509 A1
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 40 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 40 of 47 US 2003/0093509 A1
`
`Application Program
`
`File System
`
`A
`
`s
`
`
`
`:
`; :-----------------
`350
`
`Y ; :
`; :
`; :
`i: ;--
`;
`:
`:
`:
`; : :
`; ; ;
`; : :
`t
`
`386
`
`Manager ---...----...--- y
`(Win 2000)
`
`------------
`
`Port Driver
`
`:
`;
`
`--------------------- )
`
`Adapter
`
`356
`
`SCSI Class Driver
`
`352
`
`354
`
`Host
`
`LUND 1 Data
`LUN D2 Data
`LUND 3Data
`
`LUNDs
`
`Manager
`
`76
`
`Disk
`Manager
`
`:
`
`:
`
`:
`:
`
`Storage Devices
`
`1.
`
`FIGURE 36
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 41 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 41 of 47
`
`US 2003/0093.509 A1
`
`
`
`
`
`
`
`
`
`Zº ?un614
`
`[]
`
`
`
`
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 42 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 42 of 47
`
`US 2003/0093.509 A1
`
`[×
`
`
`
`Asanbeg M??A}êNCINS
`
`
`
`
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 43 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 43 of 47
`
`US 2003/0093.509 A1
`
`6€ ?an513
`
`
`
`
`
`
`
`#798
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 44 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 44 of 47 US 2003/0093.509 A1
`
`
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 45 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 45 of 47 US 2003/0093509 A1
`
`N
`
`
`
`
`
`
`
`S
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 46 of 96
`
`

`

`Patent Application Publication
`
`May 15, 2003 Sheet 46 of 47
`
`US 2003/0093.509 A1
`
`| | | | | | | | |
`
`------
`
`|
`
`|ºk-------- | | | | | | | |
`| | || | |
`
`
`DOETIFIE-FIEFJEI] DE] DOET EFF| No.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 47 of 96
`
`

`

`Patent Application Publication May 15, 2003 Sheet 47 of 47
`
`US 2003/0093.509 A1
`
`JeuueOS
`
`0,9
`
`
`
`
`
`
`
`
`
`
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 48 of 96
`
`

`

`US 2003/0093.509 A1
`
`May 15, 2003
`
`STORAGE AREANETWORK METHODS AND
`APPARATUS WITH COORDINATED UPDATING
`OF TOPOLOGY REPRESENTATION
`BACKGROUND OF THE INVENTION
`0001. The invention pertains to digital data processing
`and, more particularly, to Storage area networks and methods
`of operation thereof. The invention has application, for
`example, in managing access by a plurality of digital data
`processors (e.g., web or file servers, graphical workstations
`and So forth) to a plurality of disk drives, disk arrays and
`other Storage devices.
`0002.
`In early computer Systems, long-term data storage
`was typically provided by dedicated Storage devices, Such as
`tape and disk drives, connected to a data central computer.
`Requests to read and write data generated by applications
`programs were processed by Special-purpose input/output
`routines resident in the computer operating System. With the
`advent of “time sharing” and other early multiprocessing
`techniques, multiple users could simultaneously store and
`access data-albeit only through the central Storage devices.
`0003 With the rise of the personal computer (and work
`station) in the 1980's, demand by business users led to
`development of interconnection mechanisms that permitted
`otherwise independent computers to acceSS data on one
`another's Storage devices. Though computer networks had
`been known prior to this, they typically permitted only
`communications, not storage sharing.
`0004. The prevalent business network that has emerged is
`the local area network, typically comprising "client' com
`puters (e.g., individual PCs or workStations) connected by a
`network to a "server” computer. Unlike the early computing
`Systems in which all processing and Storage occurred on a
`central computer, client computers usually have adequate
`processor and Storage capacity to execute many user appli
`cations. However, they often rely on the Server computer
`and its associated battery of disk drives and Storage
`devices-for other than Short-term file Storage and for
`access to shared application and data files.
`0005. An information explosion, partially wrought by the
`rise of the corporate computing and, partially, by the Inter
`net, is spurring further change. LeSS common are individual
`Servers that reside as independent hubs of Storage activity.
`Often many Storage devices are placed on a network or
`Switching fabric that can be accessed by Several Servers
`(Such as file servers and web servers) which, in turn, Service
`respective groups of clients. Sometimes even individual PCS
`or WorkStations are enabled for direct access of the Storage
`devices (though, in most corporate environments Such is
`province of Server-class computers) on these So-called “Stor
`age area networks.’
`0006 A drawback in prior art storage area networks
`arises in managing the proliferation of hosts and Storage
`devices. Current Solutions focus on Setting Switches or
`Switch-like interfaces on the network or interconnect fabric
`between the hosts and Storage device, electrically “block
`ing” certain hosts certain Storage devices and So forth. A
`problem with these Solutions is that they permit only Zoning
`or Switch-like control. Another problem is that, by their very
`nature, these Solutions tend to be provider Specific.
`0007 An object of this invention is to provide improved
`Storage area networks and methods of operation thereof.
`
`0008 Further objects of the invention provide such meth
`ods and apparatus as facilitate access to multiple Storage
`devices, e.g., of varied types, from a plurality of Servers or
`other host digital data processors, e.g., running a variety of
`platforms.
`0009 Still further objects of the invention are to provide
`Such methods and apparatus for managing administrator
`defined and other policies for Storage networks, e.g., to
`facilitate acceSS by multiple hosts to multiple Storage
`devices in a manner consistent with network administrators
`wishes and without risk of unwanted access conflicts.
`0010 Yet still further objects of the invention are to
`provide Such methods and apparatus as facilitate the persis
`tence of Status and other data pertaining to Storage area
`networks regardless of the metaphors under which that data
`is used and/or stored (e.g., object-oriented, relational, and So
`forth).
`0011) Another object of the invention is to provide such
`methods and apparatus as facilitate automated handling of
`events that occur with respect to Storage area networks and
`their componentry.
`0012 Yet other objects of the invention are to provide
`Such methods and apparatus as facilitate Visual representa
`tion of the Storage area network topology, componentry and
`Status.
`0013 Still yet another object of the invention is to
`provide Such methods and apparatus as facilitate adminis
`trator (or other operator) definition of storage area network
`policy (e.g., vis-a-vis assignment of Storage devices to hosts)
`and as facilitate notification of events occurring with respect
`thereto.
`0014. These and other objects of the invention are evident
`in the drawings and in the description that follow.
`
`SUMMARY OF THE INVENTION
`0.015 LUN Management
`0016. The foregoing are among the objects attained by
`the invention which provides, in one aspect, novel Storage
`area networks (SANs) and methods of operation thereof. For
`example, in one aspect, the invention provides improve
`ments on a SAN of the type having a plurality of hosts
`coupled via a network or other interconnect with one or
`more Storage units. The improvement, according to this
`aspect of the invention, comprises a manager process, device
`or other functionality in communication with a plurality of
`agent processes, devices or other functionality, each of
`which is associated with a host. The agents identify
`attributes of (i) their associated hosts, (ii) the interconnect
`(or portion thereof) to which that host is coupled, and/or (iii)
`Storage units to which that host is coupled via the intercon
`nect. The manager responds to these attributes identified by
`the agents to manage the SAN.
`0017. The manager according to related aspects of the
`invention can be implemented on a first digital data proces
`Sor, while the hosts are implemented on further digital data
`processors. These digital data processors can be coupled Via
`a first network, e.g., an IP or other network, to Support
`communications between the manager and the agents. Such
`communications can be further effected, according to one
`aspect of the invention, utilizing an object request broker
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 49 of 96
`
`

`

`US 2003/0093.509 A1
`
`May 15, 2003
`
`(ORB). The interconnect, according to further related
`aspects of the invention, comprises a Second network, e.g.,
`SCSI and/or fiber channel based fabric, separate from the
`first network.
`0.018. According to still further aspects of the invention,
`the manager provides one or more management functions
`including, by way of non-limiting example, interfacing with
`a SAN administrator, resolving SAN topology, managing
`Storage device logical unit number assignment, and manag
`ing extension of host file Systems. The agents can Serve as
`proxies (or agents) for the manager, effecting functionality
`on its behalf at the host level. This functionality can include
`SAN component attribute collection, LUN masking control,
`host file System monitoring, and file System extension imple
`mentation.
`0.019
`Further aspects of the invention provide systems as
`describe above in which one or more agents utilize their
`asSociated hosts to query and otherwise gather information
`regarding Storage devices to them (the hosts) via the inter
`connect. This information can include the number of logical
`units present on each physical Storage device, the identifi
`cation of the physical Storage device and its respective
`logical units, and/or the Storage capacity of each logical unit.
`Queries from the hosts to the devices can be effected via
`using the protocol of the interconnect, e.g., a SCSI protocol
`for a fiber channel interconnect.
`0020. In related aspects of the invention, the manager
`correlates information collected by the agents from their
`respective hosts, e.g., disambiguating identifies of logical
`units in the Storage devices and, more typically, on the SAN,
`from potentially only partial (or incomplete) information
`Supplied by each agent. In accord with policies established
`by the SAN administrator (and entered into the manager,
`e.g., via its graphical interface), the manager assigns logical
`units to the hosts. According to related aspects of the
`invention, the manager communicates those assignments to,
`and effects them via, the agents.
`0021
`Further related aspects of the invention provide
`SAN systems as described above in which each agent
`imposes logical unit number (LUN) assignments on their
`respective agents, e.g., via filters at the adapter layer. This
`facilitates communication between the host and its assigned
`Storage devices by obviating the need for it (the host) to
`consult the manager for each read/write operation to those or
`other (e.g., unassigned) Storage devices.
`0022. In still further aspects, the invention provides
`SANs as described above in which the manager includes a
`graphical user interface (GUI) for display of SAN topology
`and/or for input of administrator-defined SAN “policy,” by
`way of non-limiting example, LUN assignment, un-assign
`ment, and file extension policy. The GUI can provide a
`plurality of views, each for example with icons or text
`representations (collectively, "icons” or “graphical objects”)
`representing hosts, storage devices (or logical units), asso
`ciations therebetween (e.g., assignment or accessibility),
`and/or properties thereof.
`0023 Assignment of a LUN to a host is permitted
`through administrator/operator-Selection of a host icon and
`a LUN icon on the GUI display. This is beneficially facili
`tated, according to one aspect of the invention, by Selec
`tively activating the icons representing the LUNs only after
`
`the icon for a Specific host has been Selected and, then, only
`activating icons for those LUN that are accessible to the
`Selected host and otherwise Suitable for assignment.
`0024.
`In related aspects of the invention, the GUI pro
`vides icons representing SAN operations, Such as assign
`ment, unassignment, and So forth. These icons are benefi
`cially activated, for example, only when icons for
`corresponding hosts, Storage units and/or other SAN com
`ponents have been Selected. For example, an icon for
`executing a LUN-to-host assignment operation is activated
`only after both a host and a LUN are selected. This can
`likewise be true of a LUN-to-host unassignment operation.
`A GUI with Such features advantageously facilitates admin
`istrator action, minimizing the number input decisions on
`the part of an administrator as well as the number of key
`Strokes, “mouse’ clicks, or other operator input device
`operations.
`0025. In further related aspects of the invention, a topo
`logical, hierarchical or enumerated (i.e., listing) display of
`SAN components can be accompanied by a display of
`component properties (e.g., identity of LUNs in a physical
`Storage device, and So forth). The latter display, too, is
`beneficially generated only upon Selection of a specific
`component in the former display. In a related aspect, data
`necessary for generating the latter (i.e., a component prop
`erty) display is retrieved, for example, from a local or remote
`database, only upon Selection of a specific component in the
`former display.
`0026 Further related aspects of the invention provide a
`system as described above in which the GUI provides for
`Selective display of Storage devices, or logical units, depend
`ing upon their Storage capacity or other quantitative
`attributes. In this regard, the GUI permits operator/admin
`istrator Specification of a numerical range for use by the
`manager in filtering Storage device display. This aspect of
`the invention can be used to display, for example, logical
`units having a storage capacity, Say, of between four and Six
`gigabytes or, for example, greater than ten gigabytes.
`0027 According to further aspects of the invention, the
`manager of a SAN as described above notifies the operator/
`administrator of SAN events such as, by way of non-limiting
`example, failure or disconnection of a storage device from
`the SAN. The manager permits specification (e.g., by the
`administrator) of a delay interval (or “alert interval”)
`between a first and Subsequent notifications of an event.
`Upon receipt of an event notification from an agent, for
`example, the manager can implement this mechanism by
`determining, e.g., from a database or otherwise, whether a
`previous notification of was made to the administrator. If So,
`further notification is made only if the current time follows
`that of the previous notification by the specified alert inter
`val.
`0028. In further aspects, the invention provides a SAN as
`described above in which the manager maintains policies for
`handling events pertaining to (i) attributes of at least Selected
`hosts and/or (ii) establishment of relationships of at least
`Selected hosts with one or more Storage units. A policy
`engine included within the manager responds to notification
`of at least a Selected event by effecting execution of an
`action according to the policy maintained therefor.
`0029. In a related aspect, the policy engine includes a
`module, herein referred to as an automation module, that
`
`Ex.1021
`CISCO SYSTEMS, INC. / Page 50 of 96
`
`

`

`US 2003/0093.509 A1
`
`May 15, 2003
`
`receives events from the agents and associates each event
`with a policy applicable to that event to form an event,
`policy pair. For example, as discussed in more detail below,
`when an agent file System monitor detects that the utilized
`portion of a file System associated with a managed host has
`exceeded a pre-defined threshold, it transmits an event
`notification to the policy engine. The policy engine deter
`mines, based on a pre-defined policy, whether the file System
`of this managed host should be extended. If the pre-defined
`policy calls for the extension of the file System, the policy
`engine identifies which LUN should be utilized and requests
`that a LUN manager assign the identified LUN to that host.
`0030) Further aspects of the invention provide systems as
`described above in which the manager maintains in a
`relational database a topological or other representation of
`the Storage area network, or aspect thereof. In response, for
`example, to notification from an agent of addition of a
`component to the SAN, the manager instantiates an object
`oriented programming (OOP) object reflecting attributes of
`the component. This object, referred to below as a “man
`ager object can also include, for example, method members
`for collecting those attributes (e.g., from other databases or
`Stores in the manager, or elsewhere). The manager instan
`tiates one or more further objects, referred to as "peer’
`objects, that Store persistable data from a corresponding
`manager object. These peer object are mapped into the
`relational database and, thereby, facilitate transfer of the
`persistable data to and from it.
`0031) Event Processing
`0.032 The invention provides in other aspects improve
`ments on a digital data processing apparatus of the type that
`manages a SAN and maintains an internal representation
`thereof, e.g. of the topology of the SAN. The improvements
`include providing a first queue with entries representing
`tasks and a Second queue with entries representing data for
`processing in connection with those tasks, where the data in
`the Second queue is grouped in accord with the task to which
`it corresponds. A manager Service updates the internal
`representation of the SAN (e.g., the representation of the
`SAN topology) by executing the tasks in the first queue one
`at a time, for example, atomically using a Single-threaded
`proceSS.
`0033. Further aspects of the invention provide improved
`apparatus as described above in which the data contained in
`the Second queue constitute event notifications, e.g., gener
`ated by a detection Service in response to changes in the
`SAN. That Service can receive, for example, from agents
`asSociated with host digital data processors on the SAN,
`information regarding the hosts and Storage devices to which
`they are connected via an interconnect. In related aspects of
`the invention, the detection Service discerns changes in the
`SAN and generates notifications by comparing information
`or “Scans' from the agents with previously Stored Scans. One
`or more notifications can be generated corresponding to each
`change and transmitted to the manager for placement on the
`queues. The notifications can reflect, for example, that a new
`host or storage device has been added to the SAN, that the
`attributes of Such a device have been modified, that a device
`is missing, and/or that a relationship between a Storage
`device and host has changed.
`0034) Further aspects of the invention provide improved
`apparatus as described above in which the manger Service
`
`Selectively adds notifications received from the detection
`Service to the Second queue until receipt of a Selected
`notification, e.g., indicating that the underlying Scan is
`complete. The Service manger can, upon Such receipt, gen
`erate for addition to the Second queue a