a mui uiu imi uiu iiui iiui iiui mu iiui mu IIIII 1111111111 mi
`I III IIIIIIII
`
`US 20100070725A1
`
`(19) United States
`(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2010/0070725 A1
`(12) Patent Application Publication (10) Pub. No.: US 2010/0070725 Al
`Mar. 18, 2010
`(43) Pub. Date:
`Mar. 18, 2010
`Prahlad et al.
`Prahlad et al.
`(43) Pub. Date:
`
`(54) SYSTEMS AND METHODS FOR
`(54) SYSTEMS AND METHODS FOR
`MANAGEMENT OF VIRTUALIZATION DATA
`MANAGEMENT OF VIRTUALIZATION DATA
`
`(76) Inventors:
`Inventors:
`(76)
`
`Anand Prahlad, East Brunswick,
`Anand Prahlad, East Brunswick,
`NJ (US); Rahul S. Pawar,
`NJ (US); Rahul S. Pawar,
`Marlboro, NJ (US); Prakash
`Marlboro, NJ (US); Prakash
`Varadharajan, Oceanport, NJ
`Varadharajan, Oceanport, NJ
`(US); Pavan Kumar Reddy
`(US); Pavan Kumar Reddy
`Bedadala, Ocean, NJ (US)
`Bedadala, Ocean, NJ (US)
`Correspondence Address:
`Correspondence Address:
`PERKINS COE LLP
`PERKINS COIE LLP
`PATENTSEA
`PATENT-SEA
`P.O. BOX 1247
`P.O. BOX 1247
`SEATTLE, WA 98111-1247 (US)
`SEATTLE, WA 98111-1247 (US)
`
`(21) Appl. No.:
`(21) Appl. No.:
`
`12/553,294
`12/553,294
`
`(22) Filed:
`(22) Filed:
`
`Sep. 3, 2009
`Sep. 3, 2009
`
`Related U.S. Application Data
`Related U.S. Application Data
`(60) Provisional application No. 61/094,753, filed on Sep.
`(60) Provisional application No. 61/094,753, filed on Sep.
`5, 2008, provisional application No. 61/121,383, filed
`5, 2008, provisional application No. 61/121,383, filed
`on Dec. 10, 2008, provisional application No. 61/169,
`on Dec. 10, 2008, provisional application No. 61/169,
`515, filed on Apr. 15, 2009.
`515, filed on Apr. 15, 2009.
`
`Publication Classification
`Publication Classification
`
`(51) Int. Cl.
`Int. Cl.
`(51)
`(2006.01)
`G06F 9/455
`(2006.01)
`G06F 9/455
`(2006.01)
`G06F 2/02
`(2006.01)
`G06F 12/02
`(2006.01)
`G06F 7/30
`(2006.01)
`G06F 17/30
`(52) U.S. Cl. ..................... 711/162: 718/1707/E17.014:
`711/162; 718/1; 707/E17.014;
`(52) U.S. Cl.
`711 FE12.OO2
`711/E12.002
`
`ABSTRACT
`(57)
`ABSTRACT
`(57)
`Described in detail herein is a method of copying data of one
`Described in detail herein is a method of copying data of one
`or more virtual machines being hosted by one or more non
`or more virtual machines being hosted by one or more non-
`virtual machines. The method includes receiving an indica
`virtual machines. The method includes receiving an indica-
`tion that specifies how to perform a copy of data of one or
`tion that specifies how to perform a copy of data of one or
`more virtual machines hosted by one or more virtual machine
`more virtual machines hosted by one or more virtual machine
`hosts. The method may include determining whether the one
`hosts. The method may include determining whether the one
`or more virtual machines are managed by a virtual machine
`or more virtual machines are managed by a virtual machine
`manager that manages or facilitates management of the Vir
`manager that manages or facilitates management of the vir-
`tual machines. If so, the virtual machine manager is dynami
`tual machines. If so, the virtual machine manager is dynami-
`cally queried to automatically determine the virtual machines
`cally queried to automatically determine the virtual machines
`that it manages or that it facilitates management of. If not, a
`that it manages or that it facilitates management o£ If not, a
`virtual machine host is dynamically queried to automatically
`virtual machine host is dynamically queried to automatically
`determine the virtual machines that it hosts. The data of each
`determine the virtual machines that it hosts. The data of each
`virtual machine is then copied according to the specifications
`virtual machine is then copied according to the specifications
`of the received indication.
`of the received indication.
`
`1005
`1005
`
`storage
`storage
`manager
`manager
`r
`index I
`
`1031
`
`mg mt.
`agent
`
`i
`
`jobs
`agent
`
`interface
`agent
`
`A
`/ 1020
`
`1050
`1050
`
`1025
`
`1130
`(,) 1095
`
`client
`
`data
`agent
`
`1070
`>
`C
`meta
`base
`
`1062
`
`SC
`
`SC
`
`1411
`
`1030
`(,) 1095
`
`1070 client
`
`C
`
`meta
`base
`
`data
`agent
`
`1060
`
`primary
`primary
`data
`data
`
`primary storage
`Secondary storage
`secondary storage
`
`1061
`1061
`
`
`
`
`
`SS
`MB
`
`secondary storage
`secondary storage
`computing device
`computing device
`
`1065
`
`1061
`
`1065
`1065
`
`>
`
`SS
`MB
`
`secondary storage
`computing device
`
`1023
`
`1015
`
`A
`
`storage
`storage
`device
`device
`
`A
`
`storage
`device
`
`1015
`
`sidb
`
`WIZ, Inc. EXHIBIT - 1053
`WIZ, Inc. v. Orca Security LTD.
`
`

`

`Patent Application Publication
`
`IZ Jo 1 WIN OIOZ `81 *JUN
`
`IV SZLOLOO/OIOZ SR
`
`data store
`storage
`
`secondary
`
`170
`
`175
`
`FIG. IA
`
`network
`
`SAN
`
`180
`
`130
`
`100
`
`data store
`
`165
`
`virtual disk analyzer
`
`component 160
`
`component 157
`
`integration
`
`data agent 155
`
`virtual machine mount
`
`component 154
`
`VLUN driver 152
`
`operation component 150
`virtual machine storage
`
`manager 145
`
`virtual machine storage
`
`virtual disk
`
`C
`
`virtual disk
`
`140a
`
`data storg_---'
`storages
`
`OS
`
`app
`
`virtual machine 110b
`
`OS
`
`app
`
`135
`
`virtual machine 110a
`
`virtual machine host 105
`
`125
`
`120b
`
`115b
`
`120a
`
`115a
`
`

`

`Patent Application Publication
`
`IZ Jo Z WIN OIOZ `81 'JUN
`
`IV SZLOLOO/OIOZ SR
`
`data store
`storage
`
`secondary
`
`170
`
`175
`
`FIG. 1B
`
`•
`
`network
`
`101
`
`data store
`
`165
`
`virtual disk analyzer
`
`component 160
`
`component 157
`
`integration
`
`data agent 155
`
`virtual machine mount
`
`component 154
`
`VLUN driver 152
`
`operation component 150
`virtual machine storage
`
`manager 145
`
`virtual machine storage
`
`180
`
`125 N.
`
`virtual disk
`
`virtual disk
`
`140b
`
`140a
`
`135 ,..___/-?-TDrimary storage\
`
`data store
`
`os
`
`app
`
`virtual machine 110b
`
`OS
`
`app
`
`120b
`
`115b
`
`120a
`
`115a
`
`virtual machine 110a
`
`virtual machine host 105
`
`

`

`Patent Application Publication
`
`IZ Jo £ WIN OIOZ `81 *JUN
`
`IV SZLOLOO/OIOZ SR
`
`data store
`storage
`
`secondary
`
`175
`
`165
`
`FIG. 2
`
`network
`
`180
`
`SAN
`
`=81
`E===i
`
`170
`
`215
`
`130
`
`125b
`
`125a
`
`data store
`
`virtual disk analyzer
`
`component 160
`
`component 157
`
`integration
`
`data agent 155
`
`virtual machine mount
`
`component 154
`
`VLUN driver 152
`
`operation component 150
`virtual machine storage
`
`manager 145
`
`virtual machine storage
`
`—o
`
`nent
`compo-
`
`API
`
`component
`
`ment
`
`manage-
`machine
`virtual
`
`manager 202
`virtual machine
`
`virtual disk.,
`
`210
`
`virtual disk
`
`virtual disk
`
`virtual disk
`
`205
`
`e--
`data store 135
`primary storage
`
`140d
`
`140c
`
`140b
`
`140a
`
`OS
`
`120d
`
`OS
`
`120b
`
`app
`
`115d
`
`app
`
`115b
`
`vm 110d
`
`vm 110b
`
`OS
`
`app
`
`120c
`
`115c
`
`OS
`
`app
`
`120a
`
`115a
`
`vm 110c
`host 105b
`
`vm 110a
`host 105a
`
`virtual machine
`
`virtual machine
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 4 of 21
`
`US 2010/0070725 Al
`
`325
`Query virtual machine manager
`to determine virtual machines
`
`Copy
`
`305
`Receive indication to perform
`copy
`
`310
`Is there a virtual
`machine manager?
`
`N
`
`315
`Select next virtual machine
`host
`
`320
`Query virtual machine host to
`determine virtual machines
`
`330
`
`Select next virtual machine
`
`335
`Copy virtual machine data
`according to indication
`
`337
`Perform other processing of
`virtual machine data
`
`340
`
`More virtual
`machines?
`
`345
`If there
`is not a virtual
`machine manager,
`are there other virtual
`machine hosts?
`
`Done
`
`FIG. 3
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 5 of 21
`
`US 2010/0070725 Al
`
`400
`
`405
`
`410a
`
`410b
`
`Enable Auto Discovery
`
`Match virtual machine names by regular expression
`
`Virtual machine host affinity
`
`Configure
`
`Configure
`
`415a
`
`415b
`
`OK
`
`Cancel
`
`Help
`
`L
`
`418
`
`FIG. 4
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 6 of 21
`
`US 2010/0070725 Al
`
`530
`
`540
`
`500
`
`Backupset Property of ManualBS
`
`510
`
`•-_,
`
`General
`
`Virtual Machine Configuration
`
`Security
`
`514
`
`
`
`
`
`
`
`-Virtual Machine Storage Manager: burnout
`
`-Application:
`
`
`
`Instance Name:
`
`Virtual Machine Host
`
`vsDefaultInstance
`
`5181
`
`Backup Set Name:
`
`ManualBS
`
`_g Automatically add new virtual machines that don't
`qualify for membership in any of the sub-client
`
`Rule-based discovery
`
`522
`524a^-,(:) Match virtual machine names by regular expression
`Configure
`
`524b--- -,O Virtual machine host affinity
`
`Configure
`
`OK
`
`Cancel
`
`Help
`
`Y
`528
`
`FIG. 5A
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 7 of 21
`
`US 2010/0070725 Al
`
`530
`
`540
`
`500
`r-)
`
`Backup Set Property of defaultBackupSet
`
`510
`"--N- , General
`
` 1
`2
`
`Virtual Machine Configuration
`544
`
`Security
`
`6
`
`Virtual machine host
`
`Virtual machine
`
`Sub-client name
`
`Subclient_test
`Subclient_test
`
`Subclient_test
`Subclient_test
`
`\\I
`
`a
`
`......._.—
`
` 570
`
`560 f
`
`562
`
`564
`
`rack0102
`rack0102
`rack0102
`rack0102
`rack0102
`rack0102
`rack0102
`rack0202
`rack0202
`rack0202
`rack0202
`rack0202
`rack0202
`rack0202
`rack0202
`{rack0302
`rack0302
`rack0302
`rack0302
`rack0302
`rack0302
`rack0302
`
`TESTVM79
`TESTVM99
`TESVM91
`rack0102-rh3
`TESTVM92
`TESTVM92
`rack0102rh4x64
`rack0102vm2
`TESTVM2459
`VM1
`TESTVM - ESX1
`rack0202su10x64
`VM2
`VM3
`VM4 x64
`SG1I1
`SG3(1)
`VM6
`SG W2K8_X64
`W2K8_SC
`W2K8SC_X64(1)
`rack0302vm1
`
`Subclient_test
`Default
`
`Marketing_Sales_SC El
`
`Subclient_test
`
`Database SC--
`Filesrv_SC—
`
`------
`
`Marketing_Sales_SC ----------
`Marketing_Sales_SC
`
`
`
`Filesrv_SC—
`
`572
`574
`
`- 576
`
`574
`
`_
`v
`
`Discover
`
`552
`
`Apply
`
`Change all selected virtual machines to sub-client:
`
`554
`
`OK
`
`Cancel
`
`Help
`
`578
`
`FIG. 5B
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 8 of 21
`
`US 2010/0070725 Al
`
`Virtual machine name
`
`userName
`
`password
`
`600
`
`Add
`
`Edit
`
`y -'612
`
`......./"614
`
`Remove
`
`616
`
`ler
`
`OK
`
`Cancel
`
`Help
`
`Y
`618
`
`FIG. 6
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 9 of 21
`
`US 2010/0070725 Al
`
`700
`
`CCopy virtual machine data)
`
`according to indication
`
`Quiesce virtual machine file systems
`
`Create snapshot of virtual machine
`
`705
`
`710
`
`715
`
`Unquiesce virtual machine file systems
`
`720
`Determine how to copy according to indication:
`file-level, volume-level of disk-level
`
`file-level
`
`volume-level
`
`disk-level
`
`722
`Determine mount point on
`virtual machine storage
`manager
`
`722
`Determine mount point on
`virtual machine storage
`manger
`
`746
`Determine copy point on virtual
`machine storage manager
`
`724
`
`724
`
`Determine volumes of virtual
`machine
`
`Determine volumes of virtual
`machine
`
`726
`Mount determined volumes at
`mount point on virtual machine
`storage manager
`
`726
`Mount determined volumes at
`mount point on virtual machine
`storage manager
`
`728
`Copy files on determined
`volumes at mount point on
`virtual machine storage
`manager to secondary storage
`data store
`
`Unmount determined volumes
`
`730
`
`754
`
`Extract metadata
`
`734
`Copy determined volumes at
`mount point on virtual machine
`storage manager to secondary
`storage data store
`
`732
`
`730
`
`Remove snapshot of virtual
`machine
`
`Unmount determined volumes
`
`732
`
`Remove snapshot of virtual
`machine
`
`748
`Determine virtual disk and
`configuration files for virtual
`machine
`
`750
`
`Copy virtual disk and
`configuration files to virtual
`machine storage manager
`
`♦
`
`752
`Remove snapshot of virtual
`machine
`
`Extract metadata
`
`754
`
`756
`
`Copy virtual disk and
`configuration files to secondary
`storage data store
`
`758
`Remove copied virtual disk and
`configuration files from virtual
`machine storage manager
`
`4,
`
`Done
`
`FIG. 7
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 10 of 21
`
`US 2010/0070725 Al
`
`800
`
`Extract metadata)
`
`805
`Access configuration files to
`determine parent-child
`relationships between virtual
`disk files
`
`,I.
`
`810
`
`Determine relationships
`between virtual disk files
`
`815
`
`Determine how volumes are
`structured on virtual disks
`
`4,
`
`820
`
`Determine location of master
`file table
`
`825
`Store determined relationships,
`determined structure of
`volumes and determined
`location of master file table
`
`Done
`
`C
`
`)
`
`FIG. 8
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 11 of 21
`
`US 2010/0070725 Al
`
`Restore
`
`
`
`)
`
`905
`Receive indication to restore
`file, volume, virtual disk or
`virtual machine
`
`4
`
`910
`Determine how copy was
`performed: file-level,
`volume-level or disk-level
`
`I—
`
`
`— Ifile-level
`915
`
`I volume-level I
`
`915
`
`I
`disk-level
`
`Mount copy set
`
`Mount copy set
`
`Mount copy set
`
`900
`
`915
`
`945
`
`920
`
`Restore file
`
`945
`Access stored
`metadata
`
`960
`Reconstruct files
`using master file
`table
`
`930
`Restore
`volume
`
`920
`Restore
`file
`
`Access stored metadata
`
`950
`Reconstruct virtual disk using
`relationships between virtual
`
`disk files 4.
`
`955
`Reconstruct volumes using
`determination of how volumes
`are structured on virtual disks
`
`960
`
`Reconstruct files using
`master file table
`
`965
`Restore
`virtual
`machine
`
`I
`
`3
`
`Restore
`volume
`
`920
`Restore
`file
`
`I 970
`Restore one
`or more
`virtual disks
`
`Y
`Done
`
`
`
`)
`
`FIG. 9
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 12 of 21
`
`US 2010/0070725 Al
`
`1005
`
`storage
`manager
`r
`I
`
`index
`
`jobs
`agent
`
`LL
`
`I 1020
`
`1050
`
`1031
`
`mgmt.
`agent
`
`interface
`agent
`
`1025
`
`1130
`1095
`
`Pt'
`
`•
`
`client ()
`
`1070
`
`data
`agent
`
`meta
`base
`
`1
`
`1062
`
`SC
`
`SC
`
`1411
`
`1030
`1095
`
`1070
`
`client ()
`
`meta
`base
`
`data
`agent
`
`r) 1060
`primary
`data
`
`primary storage
`secondary storage
`
`1061
`(._ _)
`SS
`MB
`
`1065
`
`1061
`
`1
`41
`
`1065
`
`secondary storage
`computing device
`
`SS
`MB
`
`secondary storage
`computing device
`
`1023
`
`1015
`
`storage
`device
`
`1015
`
`sidb
`
`V
`storage
`device
`
`FIG. 10
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 13 of 21
`
`US 2010/0070725 Al
`
`Browse Options
`
`1105
`
`0 Browse the Latest Data
`
`0 Specify Browse Time
`
`Browse Data Before:
`
`1100
`
`I X
`
`Wed 01/21/2009 w
`
`05:24 PM H
`
`Time Zone:
`
`(GMT-05:00) Eastern Time (US & Canada)
`
`.
`
`Virtual Machine
`
`
`Storage Manager:
`
`I Jordan
`
`Secondary Storage
`Computing Device: I<Any secondary storage computing device>
`
`1 1
`r -
`10
`
`I•
`
`lj
`
`r
`I -N
`
`
`
`11 15
`
`Ig Image Browsing
`
`-Type of Intended Restore
`
`0 Individual files/folders
`
`1
`
`20
`
`0 Entire Volume
`
`0 Virtual Machines/Virtual Disks
`
`OK
`
`Cancel
`
`Advanced
`
`List Media
`
`Help
`
`FIG. 11
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 14 of 21
`
`US 2010/0070725 Al
`
`/
`
`1205
`
`6 * REGULAR X
`
`aq Virtual Machine Storage
`40 Manager: jordan (Latest D...
`1'4 Virtual Machine Storage Manager: jordan (Latest Data)
`
`Job Controller
`
`1200
`
`Current Selected: \502088ad-e01e-d723-0dc4-79927234bd3bWolume1\Documents and Settings
`
`INST007
`q defaultBackupSet
`q PERFM3
`q PERFOM2
`E PERFORMANCE
`
`t REGULAR
`
`1208
`
`TESTVM 111
`Volumel
`Documents and
`+
`El Volume2
`El Volume3
`
`Name
`
`Size
`0 Bytes
`Administrator
`Administrator.GALAX... 0 Bytes
`0 Bytes
`Ol All Users
`0 Bytes
`Default User
`0 Bytes
`LocalService
`0 Bytes
`NetworkService
`0 Bytes
`wmpub
`
`Type
`Folder
`Folder
`Folder
`Folder
`Folder
`Folder
`Folder
`
`1250
`
`Restore Options for All Selected Items
`
`1210
`
`1215
`
`General Job Initiation
`
`I Restore ACLs
`
`■ Unconditional Overwrite
`
`Restore Destination
`
`
`
`12 20
`
`Destination Computer:
`
`jordan
`
`Destination folder:
`
`Preserve Source Paths
`
`v
`
`Browse
`
`
`
`12 25
`
`0 Preserve
`
`0 Remove
`
`I:level from the end of the source path
`0 N level from the beginning of the source path
`
`OK
`
`Cancel
`
`Advanced
`
`Help
`
`FIG. 12
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 15 of 21
`
`US 2010/0070725 Al
`
`Restore Options for All Selected Items
`
`General I Job Initiation
`
`[
`
`Restore as
`
`0 Physical volume
`
`1305
`
`0 VHD files
`
`0 VMDK files
`
`1300
`
`Type
`File
`File
`File
`File
`
`Destination Computer: I amber
`
`1310
`
`1315
`
`Source Volume
`8-h TESTVM111
`Volume1
`Volume2
`
`Volume3
`
`f`j Volume4
`
`Destination Volume
`
`1340
`
`Browse mount points on amber
`
`1320
`
`Select a mount point and click OK
`c:\[14.65 GB]
`d:\[1.95 GB]
`e:\[117.2 GB]
`f:417.11 GB]
`h:\[136.73 GB]
`j:\[30.01 GB]
`IN19.23 GB]
`m:\[19.53 GB]
`
`1325
`
`_
`
`OK
`
`Cancel
`
`OK
`
`Cancel
`
`Help
`
`FIG. 13A
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 16 of 21
`
`US 2010/0070725 Al
`
`Restore Options for All Selected Items
`
`General I Job Initiation
`
`[
`
`Restore as
`
`0 Physical volume
`
`1305
`
`0 VHD files
`
`0 VMDK files
`
`
`
`Destination Computer:
`
`I amber
`
`Destination Folder:
`
`I
`
`1300
`
` 1310
`
`I v Fi330
`
`17)I3j
`
`Browse
`
`OK
`
`Cancel
`
`Advanced
`
`Help
`
`FIG. 13B
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 17 of 21
`
`US 2010/0070725 Al
`
`Restore Options for All Selected Items
`
`General I Job Initiation
`
`[
`
`Restore
`
`as
`
`1405
`
`0 Virtual Machines
`
`O Virtual Disks
`
`Destination Computer:
`
`I jordan
`
`Destination Folder:
`
`I D:\MountFolder
`
`1400
`
`I X
`
`1410
`
`I. 17)1430
`
`Fl435
`
`Browse r
`
`—
`
`Virtual Machine Restore
`
`Options
`
`1415
`
`I TESTVM100
`
`1420
`1 J
`I VirtualCenterServer.dom
`
`I
`1440
`r )
`I ESXServer.commvault.c
`
`Virtual Machine Name
`
`Server Name
`
`0 Virtual Machine Manager
`
`1425
`
`Virtual Machine Host managed by
`Virtual Machine Manager
`
`O Virtual Machine Host
`
`'- Authentication
`
`User Name:
`
`root
`
`1445 -<
`
`Password:
`
`*************
`
`Confirm Password:
`
`*************
`
`OK
`
`Cancel
`
`Advanced
`
`Help
`
`FIG. 14A
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 18 of 21
`
`US 2010/0070725 Al
`
`1400
`
` 1410
`J
`I.1 1430
`
`F1435
`
` J
`
`Restore Options for All Selected Items
`
`General I Job Initiation
`
`r
`
`Restore as
`
`O Virtual Machines
`
`® Virtual Disks
`
`1405
`
`
`
`Destination Computer:
`
`I amber
`
`Destination Folder:
`
`I D:\MountFolder
`
`Browse
`
`OK
`
`Cancel
`
`Advanced
`
`Help
`
`FIG. 14B
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 19 of 21
`
`US 2010/0070725 Al
`
`Sub-client Properties of default
`
`Storage Device
`General
`
`I Activity Coptrol
`Filters
`
`I
`
`I Content
`I Encryption
`Pre/Post Process
`
`I Auto Discover
`Security
`
`Virtual Machine Storage Manager:
`
`jordan
`
`1500
`
`iDataAgent:
`
`
`
`150 i -.I
`
`Backup Set:
`s_ Sub-client name:
`
`Virtual Server
`
`REGULAR
`
`default
`
`1 B...../--- 1510
`
`Number of Data Readers:
`
`Note: This is a default sub-client.
`
`—
`
`Copy Type
`
`O
`
`
`
`File level
`
`151
`5
`
`C)
`
`
`
`Volume level
`
`O
`
`Disk
`
`Level
`
`M
`
`Keep snaps between failed attempts for Disk level restartability
`
`Use Virtual Machine Storage Manager
`
`-Description
`
`amber
`jordan
`sg-vm2
`sg9-vm3
`sogo24
`water
`wind
`
`1520—
`)
`
`r
`
`I v
`
`—
`
`OK
`
`Cancel
`
`Help
`
`FIG. 15
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 20 of 21
`
`US 2010/0070725 Al
`
`1600
`
`Copy
`
`1605
`
`Access virtual disk internal
`data structures
`
`1610
`
`Determine blocks used in
`virtual disk
`
`1615
`
`Access block identifier data
`structure
`
`1620
`
`Generate identifier for used
`block
`
`1625
`
`Identifier
`of block different
`from previous
`identifier?
`
`Copy block
`
`1630
`
`1635
`
`Update block identifier data
`structure with new identifier
`
`1640
`
`More blocks?
`
`Done
`
`FIG. 16
`
`

`

`Patent Application Publication Mar. 18, 2010 Sheet 21 of 21
`
`US 2010/0070725 Al
`
`Block Identifier
`
`Substantially
`Unique Identifier
`
`1700
`
`1706
`
`-1
`
`490
`
`491
`
`492
`
`...
`
`OxA1B3FG
`
`OxFG329A
`
`OxC1D839
`
`1702
`
`FIG. 17
`
`1704
`
`

`

`US 2010/0070725 Al
`
`1
`
`Mar. 18, 2010
`
`SYSTEMS AND METHODS FOR
`MANAGEMENT OF VIRTUALIZATION DATA
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`[0001] This application claims priority to U.S. Provisional
`Patent Application No. 61/094,753 filed Sep. 5, 2008 (entitled
`SYSTEMS AND METHODS FOR MANAGEMENT OF
`VIRTUALIZATION DATA, Attorney Docket No. 60692-
`8050US00), U.S. Provisional Patent Application No. 61/121,
`383 filed Dec. 10, 2008 (entitled SYSTEMS AND METH-
`ODS FOR MANAGEMENT OF VIRTUALIZATION
`DATA, Attorney Docket No. 60692-8050US01) and U.S.
`Provisional Patent Application No. 61/169,515 filed Apr. 15,
`2009 (entitled SYSTEMS AND METHODS FOR MAN-
`AGEMENT OF VIRTUALIZATION DATA, Attorney
`Docket No. 60692-8050US02), each of which is incorporated
`by reference herein in its entirety.
`
`BACKGROUND
`
`In general, virtualization refers to the simultaneous
`[0002]
`hosting of one or more operating systems on a physical com-
`puter. Such virtual operating systems and their associated
`virtual resources are called virtual machines. Virtualization
`software sits between the virtual machines and the hardware
`of the physical computer. One example of virtualization soft-
`ware is ESX Server, by VMware, Inc. of Palo Alto, Calif.
`Other examples include Microsoft Virtual Server and
`Microsoft Windows Server Hyper-V, both by Microsoft Cor-
`poration of Redmond, Wash., and Sun xVM by Sun Micro-
`systems Inc. of Santa Clara, Calif
`[0003] Virtualization software provides to each virtual
`operating system virtual resources, such as a virtual proces-
`sor, virtual memory, a virtual network device, and a virtual
`disk. Each virtual machine has one or more virtual disks.
`Virtualization software typically stores the data of virtual
`disks in files on the filesystem of the physical computer,
`called virtual machine disk files (in the case of VMware
`virtual servers) or virtual hard disk image files (in the case of
`Microsoft virtual servers). For example, VMware's ESX
`Server provides the Virtual Machine File System (VMFS) for
`the storage of virtual machine disk files. A virtual machine
`reads data from and writes data to its virtual disk much the
`same way that an actual physical machine reads data from and
`writes data to an actual disk.
`[0004] Traditionally, virtualization software vendors have
`enabled the backup of virtual machine data in one of two
`ways. A first method requires the installation of backup soft-
`ware on each virtual machine having data to be backed up and
`typically uses the same methods used to back up the data of
`physical computers to back up the virtual machine data. A
`second method backs up the files that store the virtual disks of
`the virtual machines, and may or may not require the instal-
`lation of backup software on each virtual machine for which
`the data is to be backed up.
`[0005] As an example of the second method, VMware Con-
`solidated Backup (VCB), also by VMware, Inc., enables the
`backup of the data of virtual machines on ESX Server without
`having to install backup software on the virtual machines.
`VCB consists of a set of utilities and scripts that work in
`conjunction with third-party backup software to backup vir-
`tual machine data. VCB and the third-party backup software
`are typically installed on a backup proxy server that uses the
`
`Microsoft Windows Server 2003 operating system by
`Microsoft Corporation. VCB supports file-level backups
`(backups at the level of files and directories) for virtual
`machines using Microsoft Windows operating systems. In a
`file-level backup, the granularity of the backup is at the level
`of individual files and/or directories of the virtual machine. A
`file-level backup allows copies of individual files on virtual
`disks to be made. File-level backups can be full backups,
`differential backups, or incremental backups.
`[0006] VCB also supports image-level backups for virtual
`machines using any operating system (e.g., Microsoft Win-
`dows operating systems, Linux operating systems, or other
`operating systems that may be installed upon ESX Server). In
`an image-level backup, the granularity of the backup is at the
`level of a virtual machine (i.e., the entire virtual machine,
`including its current state is backed up). For an image-level
`backup, typically the virtual machine is suspended and all
`virtual disk and configuration files associated with the virtual
`machine are backed up, and then the virtual machine is
`resumed.
`[0007] An administrator would typically choose to perform
`a file-level backup of a Microsoft Windows virtual machine
`because of the potential need to restore individual files or
`directories from the backed-up Microsoft virtual machine.
`However, VCB may not perform a file-level backup of a
`Microsoft Windows virtual machine as quickly as an image-
`level backup. Accordingly, a system that enables a backup of
`a Microsoft Windows virtual machine to be performed at least
`as quickly as a file-level backup and enables granular resto-
`ration of any data (e.g., individual files or directories) from
`the backed-up Microsoft virtual machine would have signifi-
`cant utility.
`[0008] Because VCB only supports file-level backups for
`virtual machines using Microsoft Windows operating sys-
`tems, a file-level backup cannot be performed using VCB for
`virtual machines using operating systems other than
`Microsoft Windows (e.g., Linux operating systems). An
`administrator must back up a non-Microsoft Windows virtual
`machine using an image-level backup. Therefore, in order to
`granularly restore data (e.g., an individual file or directory)
`from the backed-up non-Microsoft Windows virtual machine,
`the entire non-Microsoft Windows virtual machine must be
`restored. This may require overwriting the original virtual
`machine with the backed-up virtual machine, or re-creating
`the original virtual machine on a different physical machine.
`This may be a laborious and time-intensive process, and may
`result in loss of virtual machine data. Accordingly, a system
`that enables the granular restoration of any data (e.g., indi-
`vidual files or directories) within a virtual machine using any
`type of operating system would have significant utility.
`[0009] Another challenge posed by the use of VCB to per-
`form backups of virtual machines is that such backups require
`an administrator to manually identify or specify the virtual
`machines that are to be backed up, typically via a script
`created in advance of the backup operation. However,
`because virtual machines may be easily set up and torn down,
`virtual machines may be less permanent in nature than actual
`physical machines. Due to this potential transience of virtual
`machines, it may be more difficult for the administrator to
`identify all of the virtual machines which are to be backed up
`in advance of the backup operation. Accordingly, a system
`that provides automatic identification of virtual machines that
`are to be backed up at the time of the backup operation would
`have significant utility.
`
`

`

`US 2010/0070725 Al
`
`Mar. 18, 2010
`
`2
`
`[0010] The need exists for a system that overcomes the
`above problems, as well as one that provides additional ben-
`efits. Overall, the examples herein of some prior or related
`systems and their associated limitations are intended to be
`illustrative and not exclusive. Other limitations of existing or
`prior systems will become apparent to those of skill in the art
`upon reading the following Detailed Description.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0011] FIGS. 1A and 1B are block diagrams illustrating
`environments in which aspects of the invention may be con-
`figured to operate.
`[0012] FIG. 2 is a block diagram illustrating another envi-
`ronment in which aspects of the invention may be configured
`to operate.
`[0013] FIG. 3 is a flow diagram illustrating a process for
`discovering one or more virtual machines.
`[0014] FIGS. 4-6 are display diagrams illustrating example
`interfaces provided by aspects of the invention.
`[0015] FIG. 7 is a flow diagram illustrating a process for
`copying virtual machine data.
`[0016] FIG. 8 is a flow diagram illustrating a process for
`extracting metadata from virtual volumes and/or virtual disk
`and configuration files.
`[0017] FIG. 9 is a flow diagram illustrating a process for
`restoring virtual machine data.
`[0018] FIG. 10 is a block diagram illustrating an example of
`a data storage enterprise that may employ aspects of the
`invention.
`[0019] FIGS. 11-15 are display diagrams illustrating
`example interfaces provided by aspects of the invention.
`[0020] FIG. 16 is a flow diagram illustrating a process for
`copying virtual machine data.
`[0021] FIG. 17 is a diagram illustrating a suitable data
`structure that may be employed by aspects of the invention.
`
`DETAILED DESCRIPTION
`
`[0022] The headings provided herein are for convenience
`only and do not necessarily affect the scope or meaning of the
`claimed invention.
`
`Overview
`
`[0023] Described in detail herein is a method of copying
`data of one or more virtual machines being hosted by one or
`more non-virtual machines. The method includes receiving
`an indication that specifies how to perform a copy of data of
`one or more virtual machines hosted by one or more virtual
`machine hosts. The method further includes determining
`whether the one or more virtual machines are managed by a
`virtual machine manager that manages or facilitates manage-
`ment of the virtual machines. If so, the virtual machine man-
`ager is dynamically queried to automatically determine the
`virtual machines that it manages or that it facilitates manage-
`ment of. If not, a virtual machine host is dynamically queried
`to automatically determine the virtual machines that it hosts.
`The data of each virtual machine is then copied according to
`the specifications of the received indication.
`[0024] Under one example of the method, a file-level, vol-
`ume-level or disk-level copy of a virtual machine is per-
`formed. Performing a file-level copy involves determining
`volumes of the virtual machine, mounting the volumes on a
`proxy server, and copying files from the volumes mounted on
`the proxy server to a secondary storage data store. Performing
`
`a volume-level copy involves determining volumes of the
`virtual machine, mounting the volumes on a proxy server, and
`copying the volumes mounted on the proxy server to the
`secondary storage data store. Performing a disk-level copy
`involves determining virtual disk and configuration files of
`the virtual machine, copying the virtual disk and configura-
`tion files to the proxy server, extracting metadata from the
`virtual disk and configuration files, and copying the virtual
`disk and configuration files and the extracted metadata to the
`secondary storage data store.
`[0025] Various examples of aspects of the invention will
`now be described. The following description provides spe-
`cific details for a thorough understanding and enabling
`description of these examples. One skilled in the relevant art
`will understand, however, that aspects of the invention may be
`practiced without many of these details. Likewise, one skilled
`in the relevant art will also understand that aspects of the
`invention may have many other obvious features not
`described in detail herein. Additionally, some well-known
`structures or functions may not be shown or described in
`detail below, so as to avoid unnecessarily obscuring the rel-
`evant description.
`[0026] The terminology used below is to be interpreted in
`its broadest reasonable manner, even though it is being used in
`conjunction with a detailed description of certain specific
`examples of aspects of the invention. Indeed, certain terms
`may even be emphasized below; however, any terminology
`intended to be interpreted in any restricted manner will be
`overtly and specifically defined as such in this Detailed
`Description section.
`[0027] Unless described otherwise below, aspects of the
`invention may be practiced with conventional data processing
`systems. Thus, the construction and operation of the various
`blocks shown in FIGS. 1A, 1B and 2 may be of conventional
`design, and need not be described in further detail herein to
`make and use aspects of the invention, because such blocks
`will be understood by those skilled in the relevant art. One
`skilled in the relevant art can readily make any modifications
`necessary to the blocks in FIGS. 1A, 1B and 2 (or other
`embodiments or figures) based on the detailed description
`provided herein.
`[0028] Aspects of the invention will now be described in
`detail with respect to FIGS. 1 through 17. FIGS. 1A, 1B and
`2 are block diagrams illustrating various environments in
`which aspects of the invention may be configured to operate.
`FIG. 1A illustrates aspects of the invention interacting with
`virtual machines (e.g., VMware virtual machines or
`Microsoft virtual machines) storing data on a storage device
`connected to the virtual machine via a Storage Area Network
`(SAN), and FIG. 1B illustrates aspects of the invention inter-
`acting with virtual machines storing data locally. FIG. 2 illus-
`trates aspects of the invention interacting with a virtual
`machine manager (e.g., a VMware Virtual Center server or a
`Microsoft System Center Virtual Machine Manager), which
`manages virtual machines. FIG. 3 is a flow diagram illustrat-
`ing a process for discovering one or more virtual machines in
`one or more of the enviro