Table of Contents
Script: PowerCLI: Automatically Upgrade Hardware VMXNET3 and Paravirtual SCSI Controller for vSphere 4
Functionality
This script does the following:
- Upgrade hardware
- Replaces the NIC by the VMXNET3 NIC
- Replaces the SCSI Controller for the paravirtual SCSI controller
- Checks if the VM can be resolved and pinged after processing all VMs
- See the information in the script for more information, there are several checks and progress can get emailed to you, so you don't have to watch the screen all the time.
- The script only works on Windows Server 2003 and Windows XP, and I tested both with 32 bits editions.
Inspiration
The script was inspired by two other scripts: YoungTech Blogs His script also works on Windows Server 2008 but it made my powershell crash, it had to be run from vCenter server (which I don't like) and it doesn't work on 32 bits systems without modifications. It also made the data disks go offline after changing the SCSI controller! See the issues below.
Jase's Place His script got me on the right track for grabbing and restoring network information.
Issues
The last step may require manual intervention because of this issue. I had that several times and I don't know how to prevent this from happening when changing the primary SCSI controller through a script. I decided to send an email when there are multiple disks (the issue does not occur when there's only one disk) and leave the VM powered off. Just manually change the primary SCSI controller to the paravirtual SCSI controller and everything should be fine.
When testing YoungTech's script I fell into this issue. That's one of the reasons I'm not focusing on Windows Server 2008 right now, I really don't like issues like this.
The Script
# Automated upgrade of VMs in vSphere 4.1 # Script created by Sjoerd Hooft # # This script only works completely for Windows XP and Windows 2003 # That is because Get-VMGuestNetworkInterface only works wth XP and 2003 (and red hat, but I don't have red hat so I can't test it) # If you get the error message that the server cannot be resolved you have to use the host credentials. # # This script uses a input file with VM names, this can easily be created using the CSV creation script (see below) # Credentials need to be the same for all VMs # You should make an inventory of the VMs before using this script to be sure you got all data to restore functionality # You can use http://www.warmetal.nl/powerclicsvvminfo to create a CSV with all required information # Best is, that script uses the same techniques for gathering the data so if that works, this script will work # This script is not supported on VMs with multiple NICs and multiple SCSI controllers. It will check for multiple NICs but not for multiple SCSI controllers. # Again, use http://www.warmetal.nl/powerclicsvvminfo to find VMs with multiple SCSI controllers. # # Setting email functionality # Set this variable to $false if you don't want to receive emails regarding progress $SendEmail = $true # Set the SMTP Server address $MailServer = "10.10.10.25" # Set the email address to receive and send email from $email = "sjoerd _at_ warmetal.nl" Function PowerOn-VM($vm){ Start-VM -VM $vm -Confirm:$false -RunAsync | Out-Null Write-Host (get-date -uformat %I:%M:%S) "$vm is starting!" -ForegroundColor Green sleep 5 $time = 1 do { $vmview = Get-VM $VM | Get-View $getvm = Get-VM $vm $powerstate = $getvm.PowerState $toolsstatus = $vmview.Guest.ToolsRunningStatus Write-Host (get-date -uformat %I:%M:%S) "$vm is starting, powerstate is $powerstate and toolsstatus is $toolsstatus!" -ForegroundColor Green sleep 5 $time++ }until((($powerstate -match "PoweredOn") -and ($toolsstatus -match "guestToolsRunning")) -or ($time -eq 120)) if ($toolsstatus -match "guestToolsRunning"){ Write-Host (get-date -uformat %I:%M:%S) "$vm is started and has ToolsStatus $toolsstatus" -ForegroundColor Green } else{$Startup = "ERROR"} return $Startup } Function PowerOff-VM($vm){ Shutdown-VMGuest -VM $vm -Confirm:$false | Out-Null Write-Host (get-date -uformat %I:%M:%S) "$vm is stopping!" -ForegroundColor Green sleep 5 $time = 1 do { $vmview = Get-VM $VM | Get-View $getvm = Get-VM $vm $powerstate = $getvm.PowerState $toolsstatus = $vmview.Guest.ToolsStatus Write-Host (get-date -uformat %I:%M:%S) "$vm is stopping with powerstate $powerstate and toolsStatus $toolsstatus!" -ForegroundColor Green sleep 5 $time++ if($time -eq 120){ Write-Host (get-date -uformat %I:%M:%S) "$vm is taking more than 10 minutes to shutdown. Hard powering off the VM." -ForegroundColor Red Stop-VM -VM $vm -Confirm:$false | Out-Null } }until(($powerstate -match "PoweredOff") -or ($time -eq 180)) if ($powerstate -match "PoweredOff"){ Write-Host (get-date -uformat %I:%M:%S) "$vm is powered-off" -foregroundcolor green } else{$Shutdown = "ERROR"} return $Shutdown } Function Upgrade-VMHardware($vm){ $vmview = Get-VM $VM | Get-View $vmVersion = $vmView.Config.Version $v4 = "vmx-04" $v7 = "vmx-07" if ($vmVersion -eq $v4){ Write-Host (get-date -uformat %I:%M:%S) "Upgrading Hardware on" $vm -ForegroundColor Yellow; Get-View ($vmView.UpgradeVM_Task($v7)) | Out-Null } } Function CheckEverything($vm){ $vmview = Get-VM $VM | Get-View $getvm = Get-VM $vm $powerstate = $getvm.PowerState $GuestOS = $vmview.Guest.GuestFullname $Tools = $vmview.Guest.ToolsStatus $nics = $vmview.Guest.Net.Count Write-Host (get-date -uformat %I:%M:%S) "Evaluating $vm..."`n`t"GuestOS: $GuestOS"`n`t"Guest PowerState: $powerstate"`n`t"Guest ToolsStatus: $Tools"`n`t"Guest nr of NICs: $nics" -ForegroundColor Green if(($null -eq (get-scsicontroller -vm $vm | Where {$_.Type -ne "ParaVirtual"})) -and ($null -eq (Get-NetworkAdapter -vm $vm | Where {$_.Type -ne "Vmxnet3"}))) { Write-Host (get-date -uformat %I:%M:%S) "Bypassing VM with VMXNET3 and Paravirtual Controller: $vm" -ForegroundColor Red; Return "NOK"} elseif ((($GuestOS -match "2003") -or ($GuestOS -match "XP")) -and ($powerstate -eq "PoweredOn") -and ($Tools -eq "ToolsOK") -and ($nics -eq "1")){ Write-Host (get-date -uformat %I:%M:%S) "Prerequisites are met. Continuing the script for $vm" -ForegroundColor Green } else{ Write-Host (get-date -uformat %I:%M:%S) "Prerequisites are not met for $vm. Skipping VM." -ForegroundColor Red; Return "NOK"} } Function Send-Email ($vm, $subject, $info){ if ($SendEmail){ Send-MailMessage -To $email -From $email -SmtpServer $mailserver -Credential $mailcredentials -Subject $subject -Body $info } } Function UpgradeVM($vm){ # Check Prerequisites $check = CheckEverything $vm if ($check -eq "NOK"){ $info = "Prerequisites not met or $vm already has a VMXNET3 NIC and paravirtual SCSI controller." Send-Email $vm "$vm : Skipping! " $info Return } # Getting network information for Guest VM $GuestInterface = Get-VMGuestNetworkInterface -VM $vm -GuestCredential $credentials -HostCredential $hostcredentials $GuestIPpolicy = $GuestInterface.IPPolicy $GuestIP = $GuestInterface.Ip $GuestSubnetMask = $GuestInterface.SubnetMask $DefaultGateway = $GuestInterface.DefaultGateway $GuestDNSPolicy = $GuestInterface.DnsPolicy $GuestDNSServers = [string]$GuestInterface.Dns $SplitDNS = $GuestDNSServers.split() $pdns = $SplitDNS[0] $sdns = $SplitDNS[1] $currentadapter = Get-NetworkAdapter -vm $vm $networkname = $currentadapter.NetworkName $info = "--Current virtual machine network settings--`n`tIP Policy: $GuestIPpolicy`n`tDNS Policy: $GuestDNSPolicy`n`tIP: $GuestIP SM: $GuestSubnetMask DG: $DefaultGateway`n`tDNS: $pdns,$sdns`n`tNetworkname: $networkname" Write-Host `t(get-date -uformat %I:%M:%S) "$info" -ForegroundColor Green Send-Email $vm "$vm : Starting upgrade script! " $info if(($GuestIP -eq $null) -or ($GuestIPPolicy -ne "Static")){ $info = "$vm is skipped. This could have the following reasons: `n`tThe VM does not have static IP addresses configured. `n`tProvided credentials are wrong. `n`tThe network connection has a name with non-alphanumeric characters like - (dash)." Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Red; Send-Email $vm "$vm : Skipping! " $info Return } # Power off VM $poweroffvm = PowerOff-VM $vm if($poweroffvm -eq "ERROR"){ $info ="$vm is taking more than 15 minutes to shutdown. Hard powering off the VM also failed. Skipping VM" Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Red; Send-Email $vm "$vm : Skipping! " $info Return } # Upgrade hardware, NIC and SCSI controller Upgrade-VMHardware $vm Write-Host (get-date -uformat %I:%M:%S) "Removing legacy network adapter and adding VMxNet3 adapter" -ForegroundColor Green Get-NetworkAdapter -vm $vm | remove-networkadapter -confirm:$False New-NetworkAdapter -vm $vm -Type vmxnet3 -NetworkName $networkname -StartConnected | Out-Null Write-Host (get-date -uformat %I:%M:%S) "Adding temporary hard drive on paravirtual SCSI controller" -ForegroundColor Green New-HardDisk -VM $vm -CapacityKB 1024 -StorageFormat Thin | New-ScsiController -Type paravirtual | Out-Null # Power on VM $poweronvm = PowerOn-VM $vm if($poweronvm -eq "ERROR"){ $info = "$vm took more than 10 minutes to startup. Skipping VM." Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Red; Send-Email $vm "$vm : Skipping! " $info Return } # Setting the Guest Network settings $NewGuestInterface = Get-VMGuestNetworkInterface -VM $vm -GuestCredential $credentials -HostCredential $hostcredentials -ErrorAction SilentlyContinue while ($NewGuestInterface -eq $null) { Write-Host (get-date -uformat %I:%M:%S) "Waiting now for 15 seconds so the NIC and SCSI controller can be installed." -ForegroundColor Yellow sleep 15 $NewGuestInterface = Get-VMGuestNetworkInterface -VM $vm -GuestCredential $credentials -HostCredential $hostcredentials -ErrorAction SilentlyContinue } Write-Host (get-date -uformat %I:%M:%S) "Restoring network configuration to $vm. This might take a while." -ForegroundColor Green Set-VMGuestNetworkInterface -VMGuestNetworkInterface $NewGuestInterface -GuestCredential $credentials -HostCredential $hostcredentials -IPPolicy $GuestIPpolicy -IP $GuestIP -Netmask $GuestSubnetMask -Gateway $DefaultGateway -DnsPolicy $GuestDNSPolicy -DNS $pdns,$sdns | Out-Null # Power off VM $poweroffvm = PowerOff-VM $vm if($poweroffvm -eq "ERROR"){ $info = "$vm is taking more than 15 minutes to shutdown. Hard powering off the VM also failed. Skipping VM" Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Red; Send-Email $vm "$vm : Skipping! " $info Return } # Remove temporary changes Write-Host (get-date -uformat %I:%M:%S) "Removing temporary hard drive from virtual machine" -ForegroundColor Green Get-HardDisk -vm $vm | Where {$_.CapacityKB -eq 1024} | Remove-HardDisk -DeletePermanently -Confirm:$False | Out-Null # Change primary SCSI controller # This is only possible if the VM has only 1 disk, see: # http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1023592&sliceId=1&docTypeID=DT_KB_1_1&dialogID=221762954&stateId=0%200%20221764988 $vmdisks = Get-HardDisk -VM $vm $nrofdisks = (@($vmdisks).Count) if($nrofdisks -eq 1){ Write-Host (get-date -uformat %I:%M:%S) "Changing Primary SCSI controller to paravirtual" -ForegroundColor Green Get-HardDisk -VM $vm | Select -First 1 | new-scsicontroller -type paravirtual | Out-Null } else{ $info = "Now manually change the primary SCSI Controller to paravirtual. Skipping automatic startup of $vm." Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Red; Send-Email $vm "Manual interaction required on $vm" $info Return } # Power on VM $poweronvm = PowerOn-VM $vm if($poweronvm -eq "ERROR"){ $info = "$vm took more than 10 minutes to startup. This was the last step, so please check the VM manually." Write-Host `t(get-date -uformat %I:%M:%S) "$info" -ForegroundColor Red; Send-Email $vm "$vm : Manual interaction required! " $info Return } else{ $info = "Please check running services on $vm. Every step completed succesfully" Send-Email $vm "$vm : Done processing! " $info } } # Creating Source List $sourcename = "D:\adminshf\upgradevmlist.txt" $list = Get-Content $sourcename | Foreach-Object {Get-VM $_ | Get-View | Where-Object {-not $_.config.template} | Select Name } $vms = $list # Getting credentials for Guest VM while ($credentials -eq $null) { Write-Host (get-date -uformat %I:%M:%S) "Please provide authentication credentials for VM Guest Operating System(s)" -ForegroundColor Green; $credentials = $Host.UI.PromptForCredential("Please enter credentials", "Enter Guest credentials", "Administrator", "") } # Getting credentials for hosts while ($hostcredentials -eq $null) { Write-Host (get-date -uformat %I:%M:%S) "Please provide authentication credentials for Host(s)" -ForegroundColor Green; $hostcredentials = $Host.UI.PromptForCredential("Please enter credentials", "Enter ESX host credentials", "root", "") } # Getting Credentials for email while (($mailcredentials -eq $null) -and ($SendEmail)){ Write-Host (get-date -uformat %I:%M:%S) "Please provide authentication credentials for sending email" -ForegroundColor Green; $mailcredentials = $Host.UI.PromptForCredential("Please enter credentials", "Enter email credentials", "domain\sjoerd", "") } # Handle each VM foreach($item in $vms){ $vm = $item.Name UpgradeVM $vm } # Check for DNS resolving (removing the NIC sometimes breaks the DNS record) Write-Host (get-date -uformat %I:%M:%S) "Starting resolve checks!" -ForegroundColor Yellow; ipconfig /flushdns foreach($item in $vms){ $vm = $item.Name $getvm = Get-VM $VM $vmview = Get-VM $VM | Get-View $hostname = $vmview.Guest.HostName $ipaddress = [String]$getvm.Guest.IPAddress if (($hostname -eq $null) -or ($ipaddress -eq $null)){ $info = "$vm does not have a hostname $hostname or an ip address $ipaddress." Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Green; Send-Email $vm "$vm : DNS error! " $info Return } if ((Test-Connection -computername $hostname -Count 1 -Quiet) -eq $true){ $info = "$vm with hostname $hostname succesfully resolved to an IP address and could be pinged." Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Green; } elseif ((Test-Connection -computername $ipaddress -Count 1 -Quiet) -eq $true){ $info = "$vm with hostname $hostname and ip address $ipaddress could not resolve but could ping the ip address." Send-Email $vm "$vm : DNS error! " $info Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Green; } else{ Test-Connection -computername $hostname -Count 1 -ErrorVariable Err -ErrorAction SilentlyContinue $info = "$vm with hostname $hostname failed to resolve or could not be pinged.`n$Err" Send-Email $vm "$vm : DNS error! " $info } }
Creating the Input File
With this command you can create the required input file for a complete cluster with the least chance on skipped VMs:
get-cluster "Production" | Get-VM | get-view | where-object {($_.Guest.Net.Count -eq 1) -and ($_.Guest.ToolsStatus -eq "ToolsOK") -and ($_.Guest.GuestFullName -match "2003")} | Select Name | sort -property name | out-file D:\sjoerd\maintenance-prd-vms.txt