SHIFT

--- Sjoerd Hooft's InFormation Technology ---

User Tools

Site Tools


Sidebar

Recently Changed Pages:

View All Pages


View All Tags


LinkedIn




WIKI Disclaimer: As with most other things on the Internet, the content on this wiki is not supported. It was contributed by me and is published “as is”. It has worked for me, and might work for you.
Also note that any view or statement expressed anywhere on this site are strictly mine and not the opinions or views of my employer.


Pages with comments

View All Comments

upgradevmhardware

Script: PowerCLI: Automatically Upgrade Hardware VMXNET3 and Paravirtual SCSI Controller for vSphere 4

Functionality

This script does the following:

  • Upgrade hardware
  • Replaces the NIC by the VMXNET3 NIC
  • Replaces the SCSI Controller for the paravirtual SCSI controller
  • Checks if the VM can be resolved and pinged after processing all VMs
  • See the information in the script for more information, there are several checks and progress can get emailed to you, so you don't have to watch the screen all the time.
  • The script only works on Windows Server 2003 and Windows XP, and I tested both with 32 bits editions.

Inspiration

The script was inspired by two other scripts: YoungTech Blogs His script also works on Windows Server 2008 but it made my powershell crash, it had to be run from vCenter server (which I don't like) and it doesn't work on 32 bits systems without modifications. It also made the data disks go offline after changing the SCSI controller! See the issues below.
Jase's Place His script got me on the right track for grabbing and restoring network information.

Issues

The last step may require manual intervention because of this issue. I had that several times and I don't know how to prevent this from happening when changing the primary SCSI controller through a script. I decided to send an email when there are multiple disks (the issue does not occur when there's only one disk) and leave the VM powered off. Just manually change the primary SCSI controller to the paravirtual SCSI controller and everything should be fine.

When testing YoungTech's script I fell into this issue. That's one of the reasons I'm not focusing on Windows Server 2008 right now, I really don't like issues like this.

The Script

# Automated upgrade of VMs in vSphere 4.1
# Script created by Sjoerd Hooft
#
# This script only works completely for Windows XP and Windows 2003
# That is because Get-VMGuestNetworkInterface only works wth XP and 2003 (and red hat, but I don't have red hat so I can't test it)
# If you get the error message that the server cannot be resolved you have to use the host credentials.
#
# This script uses a input file with VM names, this can easily be created using the CSV creation script (see below)
# Credentials need to be the same for all VMs
# You should make an inventory of the VMs before using this script to be sure you got all data to restore functionality
# You can use http://www.warmetal.nl/powerclicsvvminfo to create a CSV with all required information
# Best is, that script uses the same techniques for gathering the data so if that works, this script will work
# This script is not supported on VMs with multiple NICs and multiple SCSI controllers. It will check for multiple NICs but not for multiple SCSI controllers.
# Again, use http://www.warmetal.nl/powerclicsvvminfo to find VMs with multiple SCSI controllers.
#
# Setting email functionality
# Set this variable to $false if you don't want to receive emails regarding progress
$SendEmail = $true
# Set the SMTP Server address
$MailServer = "10.10.10.25"
# Set the email address to receive and send email from
$email = "sjoerd _at_ warmetal.nl"
 
Function PowerOn-VM($vm){
   Start-VM -VM $vm -Confirm:$false -RunAsync | Out-Null
   Write-Host (get-date -uformat %I:%M:%S) "$vm is starting!" -ForegroundColor Green
   sleep 5
   $time = 1
   do {
      $vmview = Get-VM $VM | Get-View
      $getvm = Get-VM $vm
      $powerstate = $getvm.PowerState
      $toolsstatus = $vmview.Guest.ToolsRunningStatus
      Write-Host (get-date -uformat %I:%M:%S) "$vm is starting, powerstate is $powerstate and toolsstatus is $toolsstatus!" -ForegroundColor Green
      sleep 5
      $time++
    }until((($powerstate -match "PoweredOn") -and ($toolsstatus -match "guestToolsRunning")) -or ($time -eq 120))
 
    if ($toolsstatus -match "guestToolsRunning"){
      Write-Host (get-date -uformat %I:%M:%S) "$vm is started and has ToolsStatus $toolsstatus" -ForegroundColor Green
    }
    else{$Startup = "ERROR"}
    return $Startup
}
 
Function PowerOff-VM($vm){
   Shutdown-VMGuest -VM $vm -Confirm:$false | Out-Null
   Write-Host (get-date -uformat %I:%M:%S) "$vm is stopping!" -ForegroundColor Green
   sleep 5
   $time = 1
   do {
      $vmview = Get-VM $VM | Get-View
      $getvm = Get-VM $vm
      $powerstate = $getvm.PowerState
      $toolsstatus = $vmview.Guest.ToolsStatus
      Write-Host (get-date -uformat %I:%M:%S) "$vm is stopping with powerstate $powerstate and toolsStatus $toolsstatus!" -ForegroundColor Green
      sleep 5
      $time++
      if($time -eq 120){
        Write-Host (get-date -uformat %I:%M:%S) "$vm is taking more than 10 minutes to shutdown. Hard powering off the VM." -ForegroundColor Red
        Stop-VM -VM $vm -Confirm:$false | Out-Null
      }
   }until(($powerstate -match "PoweredOff") -or ($time -eq 180))
 
   if ($powerstate -match "PoweredOff"){
      Write-Host (get-date -uformat %I:%M:%S) "$vm is powered-off" -foregroundcolor green
   }
   else{$Shutdown = "ERROR"}
   return $Shutdown
}
 
Function Upgrade-VMHardware($vm){
   $vmview = Get-VM $VM | Get-View
   $vmVersion = $vmView.Config.Version
   $v4 = "vmx-04"
   $v7 = "vmx-07"
 
   if ($vmVersion -eq $v4){
      Write-Host (get-date -uformat %I:%M:%S) "Upgrading Hardware on" $vm -ForegroundColor Yellow;
      Get-View ($vmView.UpgradeVM_Task($v7)) | Out-Null
   }
}
 
Function CheckEverything($vm){
  $vmview = Get-VM $VM | Get-View
  $getvm = Get-VM $vm
  $powerstate = $getvm.PowerState
  $GuestOS = $vmview.Guest.GuestFullname
  $Tools = $vmview.Guest.ToolsStatus
  $nics = $vmview.Guest.Net.Count
  Write-Host (get-date -uformat %I:%M:%S) "Evaluating $vm..."`n`t"GuestOS: $GuestOS"`n`t"Guest PowerState: $powerstate"`n`t"Guest ToolsStatus: $Tools"`n`t"Guest nr of NICs: $nics" -ForegroundColor Green
  if(($null -eq (get-scsicontroller -vm $vm | Where {$_.Type -ne "ParaVirtual"})) -and ($null -eq (Get-NetworkAdapter -vm $vm | Where {$_.Type -ne "Vmxnet3"}))) {
    Write-Host (get-date -uformat %I:%M:%S) "Bypassing VM with VMXNET3 and Paravirtual Controller: $vm" -ForegroundColor Red;
    Return "NOK"}
  elseif ((($GuestOS -match "2003") -or ($GuestOS -match "XP")) -and ($powerstate -eq "PoweredOn") -and ($Tools -eq "ToolsOK") -and ($nics -eq "1")){
    Write-Host (get-date -uformat %I:%M:%S) "Prerequisites are met. Continuing the script for $vm" -ForegroundColor Green
  }
  else{
    Write-Host (get-date -uformat %I:%M:%S) "Prerequisites are not met for $vm. Skipping VM." -ForegroundColor Red;
    Return "NOK"}
}
 
Function Send-Email ($vm, $subject, $info){
  if ($SendEmail){
      Send-MailMessage -To $email -From $email -SmtpServer $mailserver  -Credential $mailcredentials -Subject $subject -Body $info
  }
}
 
Function UpgradeVM($vm){
  # Check Prerequisites
  $check = CheckEverything $vm
  if ($check -eq "NOK"){
    $info = "Prerequisites not met or $vm already has a VMXNET3 NIC and paravirtual SCSI controller."
    Send-Email $vm "$vm : Skipping! " $info
    Return
  }
 
  # Getting network information for Guest VM
  $GuestInterface = Get-VMGuestNetworkInterface -VM $vm -GuestCredential $credentials -HostCredential $hostcredentials
  $GuestIPpolicy = $GuestInterface.IPPolicy
  $GuestIP = $GuestInterface.Ip
  $GuestSubnetMask = $GuestInterface.SubnetMask
  $DefaultGateway = $GuestInterface.DefaultGateway
  $GuestDNSPolicy = $GuestInterface.DnsPolicy
  $GuestDNSServers = [string]$GuestInterface.Dns
  $SplitDNS = $GuestDNSServers.split()
  $pdns = $SplitDNS[0]
  $sdns = $SplitDNS[1]
  $currentadapter = Get-NetworkAdapter -vm $vm
  $networkname = $currentadapter.NetworkName
  $info = "--Current virtual machine network settings--`n`tIP Policy: $GuestIPpolicy`n`tDNS Policy: $GuestDNSPolicy`n`tIP: $GuestIP SM: $GuestSubnetMask DG: $DefaultGateway`n`tDNS: $pdns,$sdns`n`tNetworkname: $networkname"
  Write-Host `t(get-date -uformat %I:%M:%S) "$info" -ForegroundColor Green
  Send-Email $vm "$vm : Starting upgrade script! " $info
  if(($GuestIP -eq $null) -or ($GuestIPPolicy -ne "Static")){
    $info = "$vm is skipped. This could have the following reasons: `n`tThe VM does not have static IP addresses configured. `n`tProvided credentials are wrong. `n`tThe network connection has a name with non-alphanumeric characters like - (dash)."
    Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Red;
    Send-Email $vm "$vm : Skipping! " $info
    Return
  }
 
  # Power off VM
  $poweroffvm = PowerOff-VM $vm
  if($poweroffvm -eq "ERROR"){
    $info ="$vm is taking more than 15 minutes to shutdown. Hard powering off the VM also failed. Skipping VM"
    Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Red;
    Send-Email $vm "$vm : Skipping! " $info
    Return
  }
 
  # Upgrade hardware, NIC and SCSI controller
  Upgrade-VMHardware $vm
  Write-Host (get-date -uformat %I:%M:%S) "Removing legacy network adapter and adding VMxNet3 adapter" -ForegroundColor Green
  Get-NetworkAdapter -vm $vm | remove-networkadapter -confirm:$False
  New-NetworkAdapter -vm $vm -Type vmxnet3 -NetworkName $networkname -StartConnected | Out-Null
  Write-Host (get-date -uformat %I:%M:%S) "Adding temporary hard drive on paravirtual SCSI controller" -ForegroundColor Green
  New-HardDisk -VM $vm -CapacityKB 1024 -StorageFormat Thin | New-ScsiController -Type paravirtual | Out-Null
 
  # Power on VM
  $poweronvm = PowerOn-VM $vm
  if($poweronvm -eq "ERROR"){
    $info = "$vm took more than 10 minutes to startup. Skipping VM."
    Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Red;
    Send-Email $vm "$vm : Skipping! " $info
    Return
  }
 
  # Setting the Guest Network settings
  $NewGuestInterface = Get-VMGuestNetworkInterface -VM $vm -GuestCredential $credentials -HostCredential $hostcredentials -ErrorAction SilentlyContinue
  while ($NewGuestInterface -eq $null) {
    Write-Host (get-date -uformat %I:%M:%S) "Waiting now for 15 seconds so the NIC and SCSI controller can be installed." -ForegroundColor Yellow
    sleep 15
    $NewGuestInterface = Get-VMGuestNetworkInterface -VM $vm -GuestCredential $credentials -HostCredential $hostcredentials -ErrorAction SilentlyContinue
  }
  Write-Host (get-date -uformat %I:%M:%S) "Restoring network configuration to $vm. This might take a while." -ForegroundColor Green
  Set-VMGuestNetworkInterface -VMGuestNetworkInterface $NewGuestInterface -GuestCredential $credentials -HostCredential $hostcredentials -IPPolicy $GuestIPpolicy -IP $GuestIP -Netmask $GuestSubnetMask -Gateway $DefaultGateway -DnsPolicy $GuestDNSPolicy -DNS $pdns,$sdns | Out-Null
 
  # Power off VM
  $poweroffvm = PowerOff-VM $vm
  if($poweroffvm -eq "ERROR"){
    $info = "$vm is taking more than 15 minutes to shutdown. Hard powering off the VM also failed. Skipping VM"
    Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Red;
    Send-Email $vm "$vm : Skipping! " $info
    Return
  }
 
  # Remove temporary changes
  Write-Host (get-date -uformat %I:%M:%S) "Removing temporary hard drive from virtual machine" -ForegroundColor Green
  Get-HardDisk -vm $vm | Where {$_.CapacityKB -eq 1024} | Remove-HardDisk -DeletePermanently -Confirm:$False | Out-Null
 
  # Change primary SCSI controller
  # This is only possible if the VM has only 1 disk, see:
  # http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1023592&sliceId=1&docTypeID=DT_KB_1_1&dialogID=221762954&stateId=0%200%20221764988
  $vmdisks = Get-HardDisk -VM $vm
  $nrofdisks = (@($vmdisks).Count)
  if($nrofdisks -eq 1){
    Write-Host (get-date -uformat %I:%M:%S) "Changing Primary SCSI controller to paravirtual" -ForegroundColor Green
    Get-HardDisk -VM $vm | Select -First 1 | new-scsicontroller -type paravirtual | Out-Null
  }
  else{
    $info = "Now manually change the primary SCSI Controller to paravirtual. Skipping automatic startup of $vm."
    Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Red;
    Send-Email $vm "Manual interaction required on $vm" $info
    Return
}
 
  # Power on VM
  $poweronvm = PowerOn-VM $vm
  if($poweronvm -eq "ERROR"){
    $info = "$vm took more than 10 minutes to startup. This was the last step, so please check the VM manually."
    Write-Host `t(get-date -uformat %I:%M:%S) "$info" -ForegroundColor Red;
    Send-Email $vm "$vm : Manual interaction required! " $info
    Return
  }
  else{
    $info = "Please check running services on $vm. Every step completed succesfully"
    Send-Email $vm "$vm : Done processing! " $info
  }
}
 
# Creating Source List
$sourcename = "D:\adminshf\upgradevmlist.txt"
$list = Get-Content $sourcename | Foreach-Object {Get-VM $_ | Get-View | Where-Object {-not $_.config.template} | Select Name }
$vms = $list
 
# Getting credentials for Guest VM
while ($credentials -eq $null) {
  Write-Host (get-date -uformat %I:%M:%S) "Please provide authentication credentials for VM Guest Operating System(s)" -ForegroundColor Green;
  $credentials = $Host.UI.PromptForCredential("Please enter credentials", "Enter Guest credentials", "Administrator", "")
}
 
# Getting credentials for hosts
while ($hostcredentials -eq $null) {
  Write-Host (get-date -uformat %I:%M:%S) "Please provide authentication credentials for Host(s)" -ForegroundColor Green;
  $hostcredentials = $Host.UI.PromptForCredential("Please enter credentials", "Enter ESX host credentials", "root", "")
}
 
# Getting Credentials for email
while (($mailcredentials -eq $null) -and ($SendEmail)){
  Write-Host (get-date -uformat %I:%M:%S) "Please provide authentication credentials for sending email" -ForegroundColor Green;
  $mailcredentials = $Host.UI.PromptForCredential("Please enter credentials", "Enter email credentials", "domain\sjoerd", "")
}
 
# Handle each VM
foreach($item in $vms){
   $vm = $item.Name
   UpgradeVM $vm
}
 
# Check for DNS resolving (removing the NIC sometimes breaks the DNS record)
Write-Host (get-date -uformat %I:%M:%S) "Starting resolve checks!" -ForegroundColor Yellow;
ipconfig /flushdns
foreach($item in $vms){
  $vm = $item.Name
  $getvm = Get-VM $VM
  $vmview = Get-VM $VM | Get-View
  $hostname = $vmview.Guest.HostName
  $ipaddress = [String]$getvm.Guest.IPAddress
  if (($hostname -eq $null) -or ($ipaddress -eq $null)){
    $info = "$vm does not have a hostname $hostname or an ip address $ipaddress."
	Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Green;
	Send-Email $vm "$vm : DNS error! " $info
	Return
  }
  if ((Test-Connection -computername $hostname -Count 1 -Quiet) -eq $true){
    $info = "$vm with hostname $hostname succesfully resolved to an IP address and could be pinged."
    Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Green;
  }
  elseif ((Test-Connection -computername $ipaddress -Count 1 -Quiet) -eq $true){
    $info = "$vm with hostname $hostname and ip address $ipaddress could not resolve but could ping the ip address."
    Send-Email $vm "$vm : DNS error! " $info
	Write-Host (get-date -uformat %I:%M:%S) "$info" -ForegroundColor Green;
  }
  else{
    Test-Connection -computername $hostname -Count 1 -ErrorVariable Err -ErrorAction SilentlyContinue
    $info = "$vm with hostname $hostname failed to resolve or could not be pinged.`n$Err"
    Send-Email $vm "$vm : DNS error! " $info
  }
}

Creating the Input File

With this command you can create the required input file for a complete cluster with the least chance on skipped VMs:

get-cluster "Production" | Get-VM | get-view | where-object {($_.Guest.Net.Count -eq 1) -and ($_.Guest.ToolsStatus -eq "ToolsOK") -and ($_.Guest.GuestFullName -match "2003")} | Select Name | sort -property name | out-file D:\sjoerd\maintenance-prd-vms.txt
You could leave a comment if you were logged in.
upgradevmhardware.txt · Last modified: 2021/09/24 00:25 (external edit)