An overview of all sorts of commands and script usage.

• Azure Graph Explorer
• DNS Info
• Subnet cheat sheet
• One Time Secret - Share passwords secure

Useful links:

• Best practices and Styling
• Common parameters

Get-Help with -Parameter is a quick way to examine the data type a given parameter expects:

PS> Get-Help Get-ChildItem -Parameter Path
 
-Path <String[]>
    Specifies a path to one or more locations. Wildcards are permitted. The default location is the current directory (`.`).
 
    Required?                    false
    Position?                    0
    Default value                Current directory
    Accept pipeline input?       True (ByPropertyName, ByValue)
    Accept wildcard characters?  false

View history of powershell commands and invoke them using “r #”

Get-History
<# Output #>
r 1

See all saved commands in vscode

code (Get-PSReadLineOption).HistorySavePath

If statement

if (($alllogs -eq "True") -AND ($result)){
    Write-Host "Both statements are true"
}elseif (($buildreason -eq "IndividualCI") -or ($buildreason -eq "BatchedCI") -or ($buildreason -eq "PullRequest")){
    Write-Host "At least one of the statements is true"
}else {
    Write-Host "None of the statements is true"
}

Count number of strings / objects

# string
$een = "een"
$een.Count 
1
 
# Array
$twee = @("een","twee")
$twee.count
2

Length

# string
$een = "een"
$een.Length 
3
 
# Array
$twee = @("een","twee")
$twee.Length
2
$twee[0].Length
3

Measure-Object - Sometimes counting when there is only one item can fail, use measure-object instead.

$een = "een"
($een | Measure-Object).Count
1
$twee = @("een","twee")
($twee | Measure-Object).Count
2

# Dates
$timestamp = Get-Date -format "yyyyMMdd-HH.mm" # 20211011-14.27
$readdate = Get-Date -format "d MMM yyyy" # 11 Oct 2021
$weekdate = Get-Date -uformat %V # Week number # 41
$monthdate = Get-Date -format "MMMM-yyyy" # May-2021

add or subtract days

[datetime]$endDate = (Get-Date).AddDays(+90)
[datetime]$endDate = (Get-Date).AddDays(-90)

Error handling explained

Try Catch Error Finally with LineNumber for troubleshooting

function Get-ScriptLineNumber { 
   return $MyInvocation.ScriptLineNumber }
try{
    Write-Host "Do something that you want to catch the error of"
}catch{
    # Catch the error
    Write-Host "Full Error: $_"
    # Or error out after all
    Throw "Failed on: $actionname; Status: Failed; Line: $(Get-ScriptLineNumber); Error: $($_.Exception.Message)"
}finally {
    <#Do this after the try block regardless of whether an exception occurred or not#>
    # For example, clean up remote ps sessions
}

Exception.options

try{
    # Do something that you want to catch the error of
    Get-ChildItem \\prox-hyperv -ErrorAction Stop
 
    # [System.Net.DNS]::GetHostByName("server99")
 
    # Connect-AzAccount
    # Get-AzResourceGroup -Name "exampleGroup" -ErrorAction Stop
}catch [System.Management.Automation.ItemNotFoundException] {
    Write-Host "Cannot find item."
    Write-Host -NoNewLine "Exception Message        : "
    Write-Host -Foreground Red -Background Black "$($_.Exception.Message)"
}catch{
    # Catch the error
    Write-Host -Foreground Red -Background Black "Error found. Please try again, or if the error persists, contact support for evaluation. Please provide the following diagnostics:"
    Write-Host -NoNewLine "Exception Message        : "
    Write-Host -Foreground Red -Background Black "$($_.Exception.Message)"
 
    # Exceptions can contain inner exceptions. This is often the case when the code you are calling catches an exception and throws a different exception. They will place the original exception inside the new exception.
    Write-Host -NoNewLine "Inner Exception Message  : "
    Write-Host -Foreground Red -Background Black "$($_.Exception.InnerException.Message)"
 
    # The FullyQualifiedErrorId is the .Message property of the exception object along with the the fully-qualified name of the class where the exception originated.
    Write-Host -NoNewLine "Fully Qualified Error Id : "
    Write-Host -Foreground Red -Background Black "$($_.FullyQualifiedErrorId)"
 
    # Output provides the error type you need to catch this specific error type
    Write-Host -NoNewLine "Exception type           : "
    Write-Host -Foreground Red -Background Black "$($_.Exception.GetType().fullname)"
 
    # Shows the  powershell command. Does not work for .Net exceptions
    Write-Host -NoNewLine "Command                  : "
    Write-Host -Foreground Red -Background Black "$($_.InvocationInfo.MyCommand)"
 
    # Position of the error in the scriptblock or file
    Write-Host -NoNewLine "Position                 : "
    Write-Host -Foreground Red -Background Black "At line: $($_.InvocationInfo.ScriptLineNumber) char:$($_.InvocationInfo.OffsetInLine)"
 
    # Shows the faulty line. Also works for .Net exceptions
    Write-Host -NoNewLine "Full line                : "
    Write-Host -Foreground Red -Background Black "$($_.InvocationInfo.Line)"
 
    # Shows the full error. Useful for troubleshooting. Provides output so use careful when returning output.
    Write-Host "Full Error:"
    Get-Error
 
    # Shows the full error for AZ cmdlets. Useful for troubleshooting. Provides output so use careful when returning output. Usually also works with non AZ cmdlets.
    Write-Host "Full Azure Error:"
    Resolve-AzError -Last
}finally {
    #Do this after the try block regardless of whether an exception occurred or not. For example, clean up remote ps sessions, clear variables or cleanup files.
}

Enable logging of script

### Script Variables ###
$scriptname = [System.IO.Path]::GetFilenameWithoutExtension($MyInvocation.MyCommand.Path.ToString())
$scriptlocation = Split-Path $myinvocation.mycommand.path
$date = (Get-Date).ToString('yyyyMMdd-HHmm')
 
### Start transcript for full logging capabilities ###
start-transcript -path "$scriptlocation\$scriptname-$date-logtranscript.txt"
 
### Stop transcript
Stop-transcript

Script output and Write-Host options

# Inside a function use return or write-output
return "value"
Write-Output $value
# Output an object without adding it to the function output
Write-Host "color" -ForegroundColor Red  -BackgroundColor green -noNewLine 
$object | Out-Host

Input password

$password = Read-Host "Enter password" -asSecureString

Use padright to fill a line with one symbol

Write-Host ''.PadRight(78, '=')
# Or with a variable
$wide = 32
Write-Host ''.PadRight($wide, '=')

About Preference Variables

Verbose Preference

# Default
$VerbosePreference = 'SilentlyContinue'
# Override: -Verbose
 
# Verbose output
$VerbosePreference = 'Continue'
# Override: -Verbose:$false

Debug Preference

# Default
$DebugPreference = 'SilentlyContinue'
# Debug output
$DebugPreference = 'Continue'

Whatif Preference

# Default
$WhatIfPreference = $false
# Override: -WhatIf
 
# When WhatIf is enabled, the cmdlet reports the expected effect of the command, but doesn't execute the command.
$WhatIfPreference = $true
# Override: -WhatIf:$false

Set system (environment) variable

$env:var="value"

Get all system (environment) variables, sorted by name

get-childitem -path env:* | Sort Name

Variables in function
Declare a variable with script scope to use it in a function

$script:orgUrl = "https://dev.azure.com/getshiftingcom"
$script:apiversion = "api-version=6.0"
function SetBuildTag {
  # Load Azure DevOps API settings
  AzureDevOpsAPISettings
  foreach ($tag in $tags){
      $tagurl = "$orgUrl/$env:System_Teamproject/_apis/build/builds/$env:BUILD_BUILDID/tags/$($tag)?$apiversion"
  }
}

set variable with a dash/hyphen

New-Variable -Name "MODULE-SiteName" -Value "Incomingapi"

set system variable with a dash/hyphen

[Environment]::SetEnvironmentVariable('MODULE-SiteName','Incomingapi')

Set system variables with a dash/hyphen

${env:test-app_user} = "svc-test-dev"
$user = ${env:test-app_user}

Set variables

$AdminCredentials = Get-Credential
$remotePort = 5986
$pso = New-PSSessionOption -SkipCACheck -SkipCNCheck -SkipRevocationCheck
$Culture = "en-US"
$pso.Culture = $Culture
$pso.UICulture = $Culture 
$sessionParams = @{}
$sessionParams.Add('UseSSL', $true) 
$sessionParams.Add('Port', $remotePort) 
$sessionParams.Add('Credential', $AdminCredentials)

Start session to restart computer

$session = New-PSSession -ComputerName webserverdmz.domain -EnableNetworkAccess -SessionOption $pso @sessionParams
Invoke-Command -Session $session -ScriptBlock {Restart-Computer}
Remove-PSSession $session

See here for guidelines.

function Use-AzureDevOpsApi{ #Use Get-Verb for a list of approved verbs
    <#
    .SYNOPSIS
        Short description
 
    .DESCRIPTION
        Long description
 
    .PARAMETER Pat
        Explain the parameter Pat. Repeat for additional parameters.
 
    .OUTPUTS
        Explain the powershell output, if any
 
    .EXAMPLE
        Pipeline example:
        - task: PowerShell@2
          displayName: "Use Default SystemAccessToken" 
          env:
            SYSTEM_ACCESSTOKEN: $(System.AccessToken) 
          inputs:
            pwsh: true
            targetType: 'inline'
            script: |
                # Load Azure DevOps API settings
                Use-AzureDevOpsApi
 
        Script example:
        Use-AzureDevOpsApi -Pat $Pat
 
    .LINK
        System access token: https://docs.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml#systemaccesstoken
 
    .NOTES
        Author : Sjoerd Hooft
        Version: 2022-08-03 
 
    #>
    [CmdletBinding()]
    param (
        [Parameter()]
        [ValidateLength(52,52)] # See for more paramater validation https://poshcode.gitbook.io/powershell-practice-and-style/style-guide/function-structure 
        [ValidateSet("dev", "tst", "acc")]
        [string]
        $Pat
    )
 
    begin {
        Write-Host "##[section]Function: $($MyInvocation.MyCommand)"
        Write-Verbose "Input: $($PSBoundParameters | Out-String)"
 
        # Set Verbose switch based on PSBoundParameters
        $CmdLetOutput = @{Verbose = $($PSBoundParameters.Verbose)}
        # Collect information
    }
 
    process {
        # Do stuff
    }
 
    end {
        # Cleanup
    }
}

• Not null or empty: [ValidateNotNullOrEmpty()]
• Latest or YYYY-MM: [ValidatePattern("latest|20\d{2}[-]\d{2}")]
• Email: [ValidatePattern('(.+@getshifting\.com)$')]
• Specific server name: [ValidatePattern("web(dev|tst|acc)dmz0[12]")]
• Environment: [ValidateSet("dev", "tst", "acc", "prd")]
• Number between: [ValidateRange(8,100)]
• Length range: [ValidateLength(8,12)]
• Length exactly: [ValidateLength(8,8)]

Check Azure powershell module depending on PS version

Write-Host "Check required PowerShell Modules `n" 
# The "new" PS module for Azure requires PowerShell 7
if ($($PSVersionTable.PSVersion).Major -eq 7){
  if (Get-Module -ListAvailable -Name AZ) {
    Write-Host "PowerShell module Azure exists"
  } else {
    Write-Host "PowerShell module Azure does not exist. Start installation. "
    Install-Module -Name Az -Scope CurrentUser -Repository PSGallery -Force
  }
}else{
    Throw "Powershell needs to be version 7"
}

PowerShell DNS Module

if (Get-Module -ListAvailable -Name DnsServer) {
    Write-Host "PowerShell module DnsServer exists"
} else {
    Write-Host "Module DNSServer does not exist. Starting installation"
    Import-Module ServerManager
    Add-WindowsFeature -Name "RSAT-DNS-Server"
}

PowerShell AD Module

# Because ActiveDirectory is a 5.1 module you need to add -All 
if (Get-Module -ListAvailable -All -Name ActiveDirectory) {
    Write-Host "PowerShell module ActiveDirectory exists"
} else {
    Write-Host "Module ActiveDirectory does not exist. Starting installation"
    Import-Module ServerManager
    Add-WindowsFeature -Name "RSAT-AD-PowerShell" –IncludeAllSubFeature
}

PowerShell Pester module, remove shipped version and install latest version (Needs PS 7)

if (Get-Module -ListAvailable -Name Pester) {
  # Check for version 3.4 and if available remove it
  Get-Module -ListAvailable -Name Pester
  if (((((Get-Module -ListAvailable -Name Pester).Version).Major) -eq 3) -AND ((((Get-Module -ListAvailable -Name Pester).Version).Minor) -eq 4)) {
    Write-Host "PowerShell Module Pester is version 3.4 which is shipped along with Win10/Windows 2016. Start removal:"
    $module = "C:\Program Files\WindowsPowerShell\Modules\Pester"
    takeown /F $module /A /R
    icacls $module /reset
    icacls $module /grant "*S-1-5-32-544:F" /inheritance:d /T
    Remove-Item -Path $module -Recurse -Force -Confirm:$false
  }
  # Verifieer of Pester al aanwezig met minimaal versie 5
  if ((((Get-Module -ListAvailable -Name Pester).Version).Major) -ne 5) {
    Write-Host "PowerShell module Pester is not up to date"
    Install-Module -Name Pester -Force -Scope CurrentUser
  }else{
    Write-Host "PowerShell Module Pester is available with minimal version of 5:"
    Get-Module -ListAvailable -Name Pester
  }
} else {
  Write-Host "PowerShell module Pester does not exist. Start installation. "
  Install-Module -Name Pester -Force -Scope CurrentUser
}

IIS Administration Module, install extra version, see the version and the available commandlets. Note that this needs to be done in PowerShell 5 Administrative Shell

Install-Module -Name IISAdministration -Scope AllUsers -Force #Installs latest version next to it (1.1.0.0)
Import-Module IISAdministration -RequiredVersion 1.1.0.0
$mod = Get-Module -Name IISAdministration
$mod.Version
$mod.ExportedCmdlets | Format-Table

Import PS Module

Import-Module Pester

Get commands within a module

Get-Command -module Pester

Get module for a command

Get-Command Get-Service

Remove / Unload a PowerShell module

Remove-Module Pester

#Requires -Version 7.0
#Requires -PSEdition Core
# Requires for modules is slow because it also imports them
# Requires does not work for non-PS Core modules like DNS and ActiveDirectory
#Requires -Modules @{ ModuleName="Pester"; ModuleVersion="5.0" }
#Requires -Modules Az
#Requires -Modules AzureRmStorageTable

With custom PS Object and export to csv

$myCol = @()
foreach ($vm in (get-vmhost esxprd101.intranet | get-vm )){
  $VMInfo = "" | Select-Object VMName,VMHostName,NICCount
  $VMInfo.VMName = $vmview.Name
  $VMInfo.VMHostName = $vmview.Guest.HostName
  $VMInfo.NICCount = $vmview.Guest.Net.Count
  $myCol += $VMInfo
}
$myCol |Export-csv -NoTypeInformation $csvfile

Overview

# logical and comparison 
# -and, -or, -not , ! : Connect expressions 
# -eq, -ne : Equal, not equal 
# -gt / -lt, -ge / -le : Greater/less than, greater or equal / less or equal 
# -replace 
# -match / -notmatch : Regular expression match 
# -like / -notlike : wilcard matching 
# -contains / -notcontains : check for value in array: $array -contains $value
# -in / -notin : reverse syntax from contains: $value in $array 

Change directory (alias cd)

Set-Location c:\

Change directory and back

PS C:\Users\sjoer> Push-Location c:\
PS C:\> Pop-Location
PS C:\Users\sjoer>

Stop Windows Service

# Stop the service while checking the state for x² seconds and kill it if that doesn't work
Write-Host "Stopping Service: $servicename; Status: Initiating"
$sleeper = 1
while (((Get-Service -Name "$servicename").Status -ne "Stopped") -AND ($sleeper -lt 128)){
    if ($sleeper -eq 1){
        Stop-Service -Name "$servicename"
    } elseif ($sleeper -eq 32){
        Stop-Service -Name "$servicename" -Force -NoWait
    } elseif ($sleeper -eq 64){
        # Try to kill the process now
        $servicepid = (Get-CimInstance win32_service -Filter "Name = '$servicename'").ProcessId
        try{
            Stop-Process -id $servicepid -Force
        }catch{
            Throw "Stopping Service: $servicename; Stopping process: $servicepid; Status: Failed; Error: $($_.Exception.Message)"
        }
    }
    if ($alllogs -eq "True"){Write-Host "Stopping Service: $servicename; Sleeptime: $sleeper seconds"}
    Start-Sleep -Seconds $sleeper
    $sleeper = $sleeper * 2
}

Restart all Windows Services

get-service -name grn* -ComputerName appprd02 | Restart-Service -Verbose

Start all stopped Windows Services

Get-Service -Name grn* -ComputerName appprd01,appprd02,appprd03,appacc01,appacc02,apptst01,apptst02,appdev01,appdev02 | Where-Object {$_.Status -eq 'Stopped'} | Start-Service -Verbose
Get-Service | where {($_.Status -eq 'Stopped') -and ($_.name -like 'Grn*') -and ($_.StartType -eq 'Automatic')} | Start-Service

Note: -ComputerName only works in PS 5

Install Windows Service depends on Powershell version

if ($($PSVersionTable.PSVersion).Major -eq 7){
    New-Service -Name $serviceName -BinaryPathName $binaryAppExecutable -StartupType "AutomaticDelayedStart" -Credential $appuserCredentials
}else {
    New-Service -Name $serviceName -BinaryPathName $binaryAppExecutable -StartupType "Automatic" -Credential $appuserCredentials
    sc.exe config $serviceName start= delayed-auto
}

Note: PS 5 does not understand the startuptype AutomaticDelayedStart

Delete Windows Service

sc.exe delete windows.service

Note: In cmd you can use sc, in powershell sc is an alias for set-content, so you need to use sc.exe

Restart all App Pools

& $env:windir\system32\inetsrv\appcmd list apppools /state:Started /xml | & $env:windir\system32\inetsrv\appcmd recycle apppools /in

Note: This does not start stopped app pools

Restart all App Pools Remotely

$servers = 'web01','web02','web03'
Invoke-Command -ComputerName $servers { & $env:windir\system32\inetsrv\appcmd list apppools /state:Started /xml | & $env:windir\system32\inetsrv\appcmd recycle apppools /in }

Note:This does not start stopped app pools

Get the process for a IIS Site (always running under w3wp):

# Site and Appool have the same name:
$site = "customerapi"
processId = (Get-WmiObject -Class win32_process -filter "name='w3wp.exe'" | Where-Object { ($_.CommandLine).Split("`"")[1] -eq $site } ).ProcessId

Note: This one is also listed under Windows Process Management

View cert info

certutil -v -dump <path to cert>

Open Current User Certificate store:

certmgr.msc

Open Local Machine Certificate store:

certlm.msc

Get specific details from specific certificates from a list of servers and export to csv

$servers = @("server1","server2","server3")
Invoke-Command -ComputerName $servers {Get-ChildItem Cert:\LocalMachine\My | where-object {(($_.DnsNameList -like  "*rabobank*") -OR ($_.Subject -like "*rabobank*"))} |select-object @{name="hostname";expression={$(hostname)}},DnsNameList,NotAfter,Thumbprint,Subject,Issuer,FriendlyName} | Export-Csv certs.csv -NoTypeInformation

Get all processes

$processes = Get-CimInstance -ClassName Win32_Process | Select-Object ProcessName,ProcessId,CommandLine,Path | sort processid

Kill a process

try{
    $processid = (Get-CimInstance win32_process -Filter "Name = 'mmc.exe'").ProcessId
    if ($null -ne $processid){
        Stop-Process -id $processid -Force
    }
}catch{
    Write-Host "Error killing mmc: $_"
}

Get the process for a IIS Site (always running under w3wp):

# Site and Appool have the same name:
$site = "customerapi"
processId = (Get-WmiObject -Class win32_process -filter "name='w3wp.exe'" | Where-Object { ($_.CommandLine).Split("`"")[1] -eq $site } ).ProcessId

Get the process and child processes:

$id = (Start-Process cmd.exe -PassThru).id
Get-WmiObject -Class Win32_Process -Filter "ParentProcessId = '$id' or ProcessId ='$id'" | Select-Object ParentProcessId,ProcessId,CommandLine

Get all process from a specific path

$dir = "appdir"
Write-Host "Alle processen in Path -like $dir"
Get-CimInstance win32_process -Property * | Where-Object {$_.path -like "*$dir*"}

Kill process and child processes

$id = (Start-Process cmd.exe -PassThru).id
Get-WmiObject -Class Win32_Process -Filter "ParentProcessId = '$id' or ProcessId ='$id'" | Select-Object ParentProcessId,ProcessId,CommandLine | ForEach-Object {Stop-process $_.processId}

Using taskkill

$id = (Start-Process cmd.exe -PassThru).id
# Killing the PID
taskkill /pid $id /t /f
# Or using the image name (executable name without path)
taskkill /im cmd.exe /t /f

Collect uptime, patches, windows version and more

systeminfo 

Set an environment variable until the end of the current command prompt session

set AWS_DEFAULT_REGION eu-west-1

Set an environment variable in both the current command prompt session and all command prompt sessions that you create after running the command

setx AWS_DEFAULT_REGION eu-west-1

Group policy result to html

gpresult /h gpreport.html

Local Group Policy editor

gpedit.msc

GPO Management console:

gpmc.msc

Force GPO update

gpupdate /force

Allow incoming icmp rule to windows firewall

netsh advfirewall firewall add rule name="ICMP Allow incoming V4 echo request" protocol=icmpv4:8,any dir=in action=allow

Reboot in 1 second

shutdown -r -t 1

Shutdown in 0 seconds (force option implied)

shutdown -s -t 0

Count all users in all departments

Get-ADUser -SearchBase "OU=Users,DC=GetShifting,DC=local" -filter * -Properties name,department,enabled  | where {$_.enabled -eq "true"} | group department | select name, count

All enabled users that have never logged

Get-ADUser -Filter {(lastlogondate -notlike "*" -AND enabled -eq "True")} -SearchBase "OU=Users,DC=GetShifting,DC=local"

All enabled users that have not logged on for more than 90 days

$90Days = (get-date).adddays(-90)
Get-ADUser -Filter {(lastlogondate -le $90days -AND enabled -eq "True")} -SearchBase "OU=Users,DC=GetShifting,DC=local"

Get the most common reasons on a user for not being able to logon

Get-ADUser User123 -Properties * | Select-Object enabled,passwordexpired,lockedout

All servers

Get-ADComputer -Filter {(operatingSystem -like "*windows*Server*")}

Note: Does not select Windows 2000 Servers.

Select servers on properties

Get-ADComputer -filter {(description -like "*financial*" -AND name -like "SQL*")} -Properties * | select name, description

Get all AD DNS zones with all properties

Get-ChildItem "AD:CN=MicrosoftDNS,DC=DomainDNSZones,DC=getshifting,DC=local" | Select-Object *

Get all records in a DNS zone with all properties

Get-ChildItem "AD:DC=getshifting.local,CN=MicrosoftDNS,DC=DomainDNSZones,DC=getshifting,DC=local" | Select-Object *

Get creation and modify date of AD DNS records

Get-ChildItem "AD:DC=getshifting.local,CN=MicrosoftDNS,DC=DomainDNSZones,DC=getshifting,DC=local" | Where-Object {$_.name -like "build*"} | Get-ADObject -Properties Created,Modified | Select-Object Name,Created,Modified

Get the max password age from the policy

(Get-ADDefaultDomainPasswordPolicy -ErrorAction Stop).MaxPasswordAge.Days

Reverse loopup with ping

ping -a 10.10.10.10

NSlookup with default nameserver

nslookup servername

NSlookup with non-default nameserver ns01

nslookup servername ns01

Remote desktop console

mstsc.exe /v:servername /admin 

Test connection

Test-NetConnection -InformationLevel "Detailed" -Computername $host

Note: This needs (portable) tcpdump, the created file can be opened by (portable) wireshark for analysis. See here for more info on tcpdump.

dump traffic with host 10.10.10.10 on interface 1 to file

\tcpdump.exe -i 1 -vv host 10.10.10.10 -w tcpdumpfile.pcap 

dump traffic with host 10.10.10.10 on interface 1 to file with a readable timestamp format

\tcpdump.exe -i 1 -vv host 10.10.10.10 -tttt -nr tcpdumpfile.pcap 

Renew Ip config remotely without losing access

ipconfig /release && ipconfig /renew

Logging output

# Logging output in azure devops
##[group]Beginning of a group - white 
##[warning]Warning message - orange 
##[error]Error message - red 
##[section]Start of a section - green 
##[debug]Debug text - purple 
##[command]Command-line being run - blue 
##[endgroup] - green 
Write-Host "`n##[section]Function: $($MyInvocation.MyCommand).`n"

Log error or warning

Write-Host "##vso[task.logissue type=error]Something went very wrong."
Write-Host "##vso[task.logissue type=warning]Found something that could be a problem."

Generate extra output if debug mode is set

if ($env:SYSTEM_DEBUG -eq "True"){$headers | Out-Host}

Pipeline variable in yaml - without PowerShell

variables:
  - name: BuildConfiguration
    value: 'Release'
- task: DotNetCoreCLI@2
  displayName: "Build Solutions"
  inputs:
    command: 'build'
    projects: '**/*.sln'
    arguments: '--no-restore --configuration $(BuildConfiguration)'

Pipeline parameter in yaml

if ( "${{ parameters.alllogs }}" -eq "True" ){write-Host "Parameter from pipeline"}

System variable in yaml
Variables can be used like system environment variables

if (Test-Path "$env:Build_SourcesDirectory\$env:Build_Repository_Name\appsettings.Development.json"){Write-Host "System / Environment Variable"}
$sites = get-childitem -path env:* | Where-Object {$_.name -like "*-SiteName"} | Select-Object -ExpandProperty value # Added through Variable Group

Variables can also be used like pipeline variables

get-childitem $(Pipeline.Workspace) 
$dbserver = "$(serverDb)" + "." + "$(dnsZone)" # from Azure App Configuration

Variables in PS scriptfile

$adserver = $env:adServer + "." + $env:domain # from Azure App Configuration
$projects = Get-ChildItem -Path "$env:Build_SourcesDirectory\$env:Build_Repository_Name\deploy\" -Directory | Select-Object Name

name: $(Build.DefinitionName)-$(Build.BuildId)
appendCommitMessageToRunName: false

variables:
  buildConfiguration: 'Release'
  deploymentSuffix: '$(Build.DefinitionName)-$(Build.BuildId).$(System.StageAttempt).$(System.JobAttempt)'

parameters:
  - name: alllogs
    displayName: "Enable all logging"
    type: boolean
    default: true
 
# https://learn.microsoft.com/en-us/azure/devops/pipelines/process/scheduled-triggers?view=azure-devops&tabs=yaml
schedules:
- cron: "05 23 * * *" # 23.05h
  displayName: Nightly build on master
  branches:
    include:
    - master
  always: true

pool:
  name: ReleasePool # Self hosted
  vmImage: ubuntu-latest # MS Hosted
  vmImage: windows-latest # MS Hosted

trigger:
  branches:
    include:
      - release/*
      - master
trigger: none

resources:
  repositories:
  - repository: templates
    type: git
    name: DevOps/templates
  - repository: self
  pipelines:
  - pipeline: TA
    source: TestAutomation # Pipeline name that produces the artifact (must be unique)

stages:
- stage: build
  displayName: "Stage: Build"
  pool:
    name: BuildPoolDotNet

  jobs:
  - job: build
    displayName: "Job: Build & Test"
    steps:
    - checkout: self
      clean: true # If true, run git clean -ffdx followed by git reset --hard HEAD before fetching.
    - checkout: templatses
      clean: true

    - ${{ if not(eq(variables['Build.Reason'], 'PullRequest')) }}:
      - template: /pipelines/log-information.yml@operations
        parameters:
          environment: "environment: build"
          alllogs: ${{ parameters.alllogs }}
 
# Stage with template that has steps (see deploy.yml)
- stage: release
  displayName: "Stage: release"
  condition: and(succeeded(), ne(variables['Build.SourceBranchName'], 'master'))

  variables:
    environment: 'tst'

  jobs:
  - deployment: Deploy
    workspace:
      clean: all
    environment: ${{ variables.environment }}

  - template: /pipelines/deploy.yml@templates
    parameters:
      environment: ${{ variables.environment }}
      alllogs: ${{ parameters.alllogs }}
 
# Stage with template as step (see prod.yml)
- stage: production
  displayName: "Stage: prod"
  dependsOn: release

  variables:
    tag: 'Production'

  jobs:
  - job: prd
    displayName: "production"
    condition: and(succeeded(), eq( ${{ parameters.deployProduction }}, true))
    variables:
      environment: 'prd'

    steps:
    - template: /deploy/production.yml
      parameters:
        environment: ${{ variables.environment }}
        alllogs: ${{ parameters.alllogs }}

parameters:
  environment: ""
  alllogs: ""
  dependsOn1: []
  dependsOn2: []
  dependsOn3: []
  dependsOn4: []

jobs: 
- job: deploy 
  displayName: "Deploy steps"
  dependsOn: 
    - ${{ parameters.dependsOn1 }}
    - ${{ parameters.dependsOn2 }}
    - ${{ parameters.dependsOn3 }}
    - ${{ parameters.dependsOn4 }}
  workspace:
    clean: all
  steps:

  - checkout: operations
    clean: true # If true, run git clean -ffdx followed by git reset --hard HEAD before fetching.
 
  #- ${{ if eq(parameters.alllogs, true) }}:
  - template: /pipelines/log-information.yml@operations
    parameters:
      environment: ${{ parameters.environment }}
      alllogs: ${{ parameters.alllogs }}
      input1: "adGroups: ${{ parameters.adGroups }}"
  
  - task: PowerShell@2
    displayName: "PowerShell task"
    inputs:
      pwsh: true
      targetType: 'inline'
      script: |
        Write-Host "`n##[section]PowerShell task `n"

parameters:
  environment: ""
  alllogs: ""
  spokeEntity: ""

steps:
- task: PowerShell@2
  displayName: "Powershell Core task"
  inputs:
    pwsh: true
    targetType: 'inline'
    script: |
      Write-Host "`n##[section]Start PS Core task`n"
      Get-ChildItem -path env:* | Sort Name

- task: Bash@3
  displayName: "Bash task"
  inputs:
    targetType: 'inline'
    failOnStderr: true
    script: |
      echo '##[Section]Start Bash task'
      printenv | sort

- task: AzureCLI@2
  displayName: "Azure CLI task"
  condition: and(succeeded(), or(eq(variables.runAzureTask, true), eq(variables.runAzureAll, true)))
  inputs:
    azureSubscription: $(azureServiceConnection)
    scriptType: bash
    scriptLocation: inlineScript
    inlineScript: |
      set -e
      echo '##[Section]Start Azure CLI task'
      az account set --subscription $(subId)

- task: AzurePowerShell@5
  displayName: "Azure Powershell task"
  inputs:
    azureSubscription: $(azureServiceConnection)
    azurePowerShellVersion: 'LatestVersion'
    pwsh: true
    ScriptType: 'InlineScript'
    Inline: |
      if ( "${{ parameters.alllogs }}" -eq "True" ){Write-Host "`n##[section]Start Azure Powershell task`n"}
      # Set Azure Context for environment subscription
      Set-AzContext -Name ${{ parameters.environment }} -Subscription $env:azureSubscriptionName

Show running container

docker ps
# or
docker container ls

Show all containers

docker ps -a

docker stop/start/restart containers

# Stop
docker stop container_id
# default grace time = 10
docker stop --time=5 container_id 
# Start
docker start container_id
# Restart
# Restart apache container by name
docker container restart httpd_web

Get docker container info

docker inspect container_id

Get docker container compose info

docker inspect container_id | grep com.docker.compose

Copy a file from or to a container

docker cp tool_db:/var/lib/postgresql/data/pgdata/postgresql.conf ./postgresql.conf

Get logging from docker container

docker logs --tail 50 --follow --timestamps httpd_web

grep in logging

docker logs <container_name> 2>&1 | grep <string>

Note that docker logs to stderr do you need to redirect this to stdout

Get container stats

docker top <container_name>

Get docker stats

docker stats

Work in a container

docker exec -it container_id /bin/bash

As a different user

docker exec -u postgres -it container_id /bin/bash

Docker compose files are used to start containers. By default, a docker-compose.yml or a docker-compose.yaml file is used.

Start and stop containers from docker-compose file

docker compose up -d
docker compose down

Note the -d switch to start the containers in the background.

See logging

# the logs from each of the services interleaved into a single stream
docker compose logs -f
# the logs just for app
docker compose logs -f app

Docker build, run and show logs of a custom app called privacy

# Build an image called privacyimage
docker build -t privacyimage .
# Run a container called privacycontainer from the image
docker run -d --name privacycontainer privacyimage
# Show the logs from the container
docker logs privacycontainer

# Show all steps in plain output and redo every step (don't use cached steps)
docker build --progress=plain --no-cache  -t consoleappdbtestimage .

Save an image to disk and import somewhere else

# Create a tar file from the consoleappimage
docker save -o ./consoleappimage.tar consoleappimage
# import the image
docker load -i /consoleappimage.tar

A Dockerfile is used to create images:

Default .net app dockerfile:

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build-env
WORKDIR /App
 
# Copy everything
COPY . ./
# Restore as distinct layers
RUN dotnet restore
# Build and publish a release
RUN dotnet publish -c Release -o out
 
# Build runtime image
FROM mcr.microsoft.com/dotnet/aspnet:6.0
WORKDIR /app
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "DockerConsoleApp.dll"]

  - task: Docker@2
    displayName: Build and push an image to container registry
    inputs:
      command: buildAndPush
      repository: $(repository)
      dockerfile: $(dockerFilePath)
      containerRegistry: "containerRegistry"
      tags: |
        $(imageName)
        $(Build.BuildId)
        latest

grep file without empty lines and comments

grep -v '^\s*$\|^\s*\#' postgresql.conf

find file in current and subdirectories

sudo find . -name main.cf

display all crontabs for all users as root sudo su -

for user in $(cut -f1 -d: /etc/passwd); do echo $user; crontab -u $user -l; done

Note this is especially useful in pipelines

set

# -e : exit the script if any command fails
# -u : treat unset variables as errors and exit
# -o pipefail : ensure the entire pipeline fails if any command within it fails
# -x : print all the commands to the terminal
# -v : print all lines without variable resolution
# - For all, to disable use +, for example `set +v`
set -euo pipefail

Compare two files

diff file file2

Compare two files next to each other, only differences

diff -y --suppress-common-lines file1 file2

See release

cat /etc/os-release

• Turn off syntax :syntax off
• Delete all below lines, including current: dG
• Past text from windows clipboard through putty: SHIFT + INS

Create csr with new private key (preferred)

openssl req -nodes -newkey rsa:2048 -keyout *.shiftwiki.nl.key -out *.shiftwiki.nl.key.csr

Create csr with existing key

openssl req -new -key /data/keys/httpd-wild.shiftwiki.nl/oeserver.pem  -out *.shiftwiki.nl.key.csr

View certificate

openssl x509 -text -noout -in certificate.crt

View csr

openssl req -text -in request.csr

Truncate a logfile that has grown too big

sudo truncate -s 0 odoo.log

See link between device and filesystem for AWS EBS disks. From the instance, go to Storage tab and see the device name, then use these commands:

[sjoerd@server ~]$  df -hT /dev/sdf
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/nvme1n1   xfs    20G   14G  6.8G  67% /var/lib/docker
[sjoerd@server ~]$  df -hT /dev/sdg
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/nvme2n1   xfs    20G   15G  5.8G  72% /data

NOTE: Module AZ only works in PS 7

TLS version

# Set TLS version to use TLS 1.2 
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

Login and context

Connect-AzAccount
Get-AzContext -ListAvailable
Set-AzContext -Name $dtapEnvironment -Subscription $azureSubscriptionName

Get TenantId

$tenantId = (Get-AzContext).Tenant.Id

Get secret from KeyVault

$adadminpass = (Get-AzKeyVaultSecret -VaultName kv-operations-global -Name "adAdminPass").SecretValue

Set secret to KeyVault

$userpasswordsecure = ConvertTo-SecureString -String $userpassword -AsPlainText -Force
Set-AzKeyVaultSecret -VaultName kv-operations-global -Name $username -SecretValue $userpasswordsecure

((Invoke-AzRestMethod -Path "/subscriptions/$subId/resourcegroups?api-version=2020-06-01&`$expand=createdTime" -Method GET).Content | ConvertFrom-Json).value | Select-Object name,createdTime

az login
az account set --subscription xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
az account show

# Check if a resourcegroup exists and set a variable
RgExists=$(az group exists -n $(rgName))
echo "$(rgName) exists: $RgExists"
 
# Get firewall Id
firewallId=$(az network firewall show --name $(azureFirewallName) --resource-group $(rgName) --query id --output tsv)
 
# Get Log Analytics Workspace Id
workspaceId=$(az monitor log-analytics workspace show --workspace-name $(lawName) --resource-group $(rgName) --subscription $(subId) --query id -o tsv)
 
# Get Virtual Network Gateway (VPN) Id
vngId=$(az network vnet-gateway show --name $(virtualNetworkGatewayName) --resource-group $(rgName) --query id --output tsv)
 
# Get Keyvault Id
keyvaultId=$(az keyvault show --name $(keyVaultName) --resource-group $(rgName) --query id --output tsv)

resourceGroup=$(az group show -n poc-$(team)-$(pocnaam) --query id --output tsv)
echo 'The resource group has the following tags'
az tag list --resource-id $resourceGroup
echo 'add a tag'
az tag update --resource-id $resourceGroup --operation Merge --tags DateOne="$(firstdate)" DateTwo="$(seconddate)"

# check if a resource has a specific tag set
tag=$(az graph query -q "resourcecontainers | where type == 'microsoft.resources/subscriptions/resourcegroups' and name == 'poc-$(team)-$(pocnaam)' | where tags contains 'startdate'" | jq '.data[].name')
 
echo "Tag date: $tag"
 
# do something if the tag is empty
if [ -z "$tag" ];
then
  echo "do something, the tag is not set"
else
  echo "do something else, the tag is set"
fi
 
# do something if the tag is not empty
if [ -n "$bedrijf" ];
then
  echo "do something, the tag is set"
else
  echo "do something else, the tag is not set"
fi

# deploy resourcegroup in a subscription
az deployment sub create \
  --location $(location) \
  --name "hub-$(deploymentSuffix)" \
  --template-file ./modules/resourcegroup.bicep \
  --parameters ./hub/hub-rg-parameterfile.jsonc \
  --parameters location=$(location)
 
# deploy resources in resourcegroup
az deployment group create \
  --resource-group $(rgName) \
  --name "hub-vnet-$(deploymentSuffix)" \
  --template-file ./hub/hub-vnet.bicep \
  --parameters deploymentSuffix=$(deploymentSuffix) \
  --parameters subId=$(subId) \
  --parameters vnetName=$(vnetName)

These are just a few examples, see Kusto Queries in Azure for more.

Verify incoming logs per table

Perf
| where TimeGenerated > ago(10m)
| order by TimeGenerated desc
 
Event
| where TimeGenerated > ago(10m)
| order by TimeGenerated desc
 
RabbitMQ_CL
| where TimeGenerated > ago(10m)
| order by TimeGenerated desc
 
W3CIISLog
| where TimeGenerated > ago(10m)
| order by TimeGenerated desc

Log errors every 10 minutes in a stacked graph

APP_LOG
| where RawData contains "error"
| summarize count() by Computer, bin(TimeGenerated, 10m)
| order by TimeGenerated
| render areachart

Week overview of errors per service account in a column graph

APP_LOG
| where LogType_CF == '[error]'
| summarize count() by UserName_CF, Computer, bin(TimeGenerated, 1d)
| order by TimeGenerated, UserName_CF
| render columnchart

SQL timeouts in a column graph

exceptions
| where outerMessage has "timeout" and outerType == "Microsoft.Data.SqlClient.SqlException"
| summarize count() by bin(timestamp, 1d), cloud_RoleName
| order by timestamp
| render columnchart

All exceptions per 30 seconds per application

exceptions
| where outerMessage contains "timeout"
| where cloud_RoleName != ""
| summarize count() by bin(timestamp, 30s), cloud_RoleName
| order by timestamp

All high severity level exceptions of a specific type

exceptions
| where severityLevel > 2 and type == 'MassTransit.RequestTimeoutException'
| summarize count() by cloud_RoleName, bin(timestamp, 1d)
| render columnchart

Get unique logitems per rolename/roleinstance in an appi log

requests
| distinct cloud_RoleName, cloud_RoleInstance
| sort by cloud_RoleInstance

Get all resources, type, location, subName and rgName

resources
| join kind=inner (
    resourcecontainers
    | where type == 'microsoft.resources/subscriptions'
    | project subscriptionId, subscriptionName = name)
    on subscriptionId
| project name, type, location, subscriptionName, resourceGroup
| order by type asc

Get all application insights resources without a workspace

resources
| where type == "microsoft.insights/components" 
| where properties['WorkspaceResourceId'] == ""
| join kind=inner (
    resourcecontainers
    | where type == 'microsoft.resources/subscriptions'
    | project subscriptionId, subscriptionName = name)
    on subscriptionId
| project name, type, subscriptionName, resourceGroup
| order by subscriptionName asc

Get all storage accounts with an old tls version

resources
| where type == "microsoft.storage/storageaccounts"
| where properties['minimumTlsVersion'] != "TLS1_2"
| project name, resourceGroup, properties.minimumTlsVersion

Get backup job status (for multiple laws, change the scope)

AddonAzureBackupJobs
| parse VaultName with 'bvault-euw-' Spoke '-' Environment '-' Service
| extend JobDurationInSecsApprox = datetime_diff('second', TimeGenerated, JobStartDateTime)
| where JobOperation == 'Backup'
| project BackupItemFriendlyName, JobOperation, JobStartDateTime, JobDurationInSecsApprox, JobStatus, JobFailureCode, Environment, Spoke, Service, BackupItemUniqueId, VaultName, JobUniqueId, _ResourceId

• Audit Stream to Log Analytics
• Auditing event list

yaml and classic release that were not successful

AzureDevOpsAuditing
| where OperationName == "Pipelines.DeploymentJobCompleted" or OperationName == "Release.DeploymentCompleted"
| where Data.DeploymentResult != "Succeeded"
| extend deployment = strcat(Data.PipelineName, "-", Data.StageName)
| project TimeGenerated, deployment
| summarize count() by bin(TimeGenerated, 10m), deployment
| render columnchart

yaml releases to production that were successful

AzureDevOpsAuditing
| where OperationName == "Pipelines.DeploymentJobCompleted" and Data.DeploymentResult == "Succeeded" and Data.StageName contains "Production"
| extend deployment = strcat(Data.PipelineName, "-", Data.StageName)
| project TimeGenerated, deployment
| summarize count() by bin(TimeGenerated, 1d), deployment
| render columnchart

classis releases that were not successful

AzureDevOpsAuditing
| where OperationName == "Release.DeploymentCompleted" and Data.DeploymentResult != "Succeeded"
| extend deployment = strcat(Data.PipelineName, "-", Data.StageName)
| project TimeGenerated, deployment
| summarize count() by bin(TimeGenerated, 1d), deployment
| render columnchart

Bicep object vs array

param objectToTest object = {
  one: 'a'
  two: 'b'
  three: 'c'
}
param arrayToTest array = [
  'one'
  'two'
  'three'
]

• At any resource you can see which RBAC roles apply to this specific resource and what the allowed actions are: After selecting the resource, go to IAM - Roles → You'll now notice the roles that apply. Now, for any role, click on Details → This will show you the allowed actions and switch between the control and data plane for any data actions (useful with for example storage accounts).

Check git repository size

git count-objects -vH 

Combine fetch and merge to completely update a repository

git pull 

Show git log

git log

Show the files which have changed between the branch and master

git diff --name-only master..branchtocheck

Create and immediately checkout a branch

git checkout -b "#123456-add-dns-resolving"

Show all branches

git branch -a

Show all active branches

git ls-remote --heads origin

Merge changes from master branch, perform from inside the branch

git merge master

Show remote

git remote -v

Update the remote “origin” with a new url in case of a rename of the repo or project

git remote set-url origin https://getshifting@dev.azure.com/getshifting/Infra/_git/infraplayground

Add and commit a new file

git add .
git commit -m "Added new file"

Add and commit in 1 go for changed files

git commit -am "changed a file"

Co-Authoring / Pair coding commit message

Changed a couple of files

Co-Authored-By: Sjoerd <sjoerd @ getshifting.com>

Do not trigger CI in Azure DevOps

***NO_CI***

Start MS SQL Management Studio as a user from a different domain to a different database server

runas /netonly /user:getshifting.local\admin-sjoerd "c:\Program Files (x86)\Microsoft SQL Server Management Studio 18\Common7\IDE\Ssms.exe -S dbdev01.getshifting.local"

# get postgresql user and password
cd /data/docker
cat .env
docker container ps
docker exec -it container_db /bin/bash
psql -U <user> -W

See connections

SHOW max_connections;
SHOW shared_buffers;

See current activity

SELECT pid AS process_id, 
       usename AS username, 
       datname AS database_name, 
       client_hostname, 
       application_name,
       backend_start,
       state,
       wait_event,
       state_change
FROM pg_stat_activity;

See all databases

\l
-- See all databases with more information
\l+

- task: PythonScript@0
  displayName: "Sort and print all system environments "
  inputs:
    scriptSource: 'inline'
    script: |
      import os
      for name, value in sorted(os.environ.items()):
        print("   " + name + "=" + value)

- task: PythonScript@0
  displayName: "Python version "
  inputs:
    scriptSource: 'inline'
    script: |
      import subprocess
      import os
      if os.environ['AGENT_OS'] == "Linux":
        version = subprocess.getoutput('python3 --version')
      elif os.environ['AGENT_OS'] == "Windows_NT":
        version = subprocess.getoutput('py -3 --version')
      else:
        version = "unknown"
      print(f"Python version is {version}")

- task: PythonScript@0
  displayName: "Pip version "
  inputs:
    scriptSource: 'inline'
    script: |
      import subprocess
      pip = subprocess.getoutput('pip --version')
      print(f"Pip version is {pip}")

- task: PowerShell@2
  displayName: "Python virtual environment"
  inputs:
    pwsh: true
    targetType: 'inline'
    script: |
      Write-Host "`n##[section]Start python script`n"
      # Create virtual environment "env" by running module venv (https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments/)
      py -m venv env
      .\env\Scripts\activate.ps1 -Verbose
      Write-Host "`n##[section]Verify virtual environment`n"
      Get-Command python.exe
      Write-Host "`n##[section]Install packages`n"
      $scriptDir = "$(BUILD.SourcesDirectory)\$(BUILD.REPOSITORY.NAME)\src\$module"
      Write-Host "Locatie: $(Get-Location); Script dir: $scriptDir"
      if (Test-Path $scriptDir\requirements.txt){
          Write-Host "Directory: $scriptDir; Requirements file: Present"
          py -m pip install -r $scriptDir\requirements.txt
      }else {
          Write-Host "Directory: $scriptDir; Requirements file: Not present. Installing from code."
          # install a package 
          py -m pip install requests
          # install a specific version 
          py -m pip install pandas==1.5.2
      }
      Write-Host "`n##[section]Run python script`n"
      if (Test-Path $scriptDir\main.py){
          Write-Host "Script: $scriptDir\main.py; Status: Present"
          py $scriptDir\main.py
      }else{
          Throw "Script: not found in $scriptDir"
      }
 
      Write-Host "`n##[section]Install packages`n"
      # install a package 
      py -m pip install requests
      # install a specific version 
      py -m pip install pandas==1.5.2

const AWS = require("aws-sdk");
const dynamodb = new AWS.DynamoDB();
 
// Get Time in the Netherlands
var dutchTime = new Date().toLocaleString('en-US', {timeZone: "Europe/Amsterdam"});
var requestTime = new Date(dutchTime);
var year = requestTime.getFullYear();
var month = requestTime.getMonth() + 1; //Months go from 0 - 11
// Set months to get logs for
if (month === 1){
    var fullMonth = year + "-0" + month;
    var lastMonth = (year - 1 ) + "-12";
}
if (month > 1 && month < 10){
    var fullMonth = year + "-0" + month;
    var lastMonth = year + "-0" + (month - 1);
}
if (month === 10){
    var fullMonth = year + "-" + month;
    var lastMonth = year + "-0" + (month - 1);
}
if (month > 10){
    var fullMonth = year + "-" + month;
    var lastMonth = year + "-" + (month - 1);
}
 
const params = {
  TableName: 'LogTable',
  FilterExpression: 'begins_with(RequestTime, :RequestTime) OR begins_with(RequestTime, :RequestTime2)',
  ExpressionAttributeValues: {
    ':RequestTime': {S: fullMonth},
    ':RequestTime2': {S: lastMonth},
  },
};
 
exports.handler = (event, context, callback) => {
    console.log(fullMonth + lastMonth);
    dynamodb.scan(params, (err, data) => {
        if(err){
            console.log('Error on retrieving log data ' + err);
        }
        callback(null, data['Items']);
    });
};

//jshint esversion:6
 
/* IAM-Access
Description: Sends the end email to the customer with all required information. 
 
Links:
- AWS Javascript SDK for SES: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SES.html
- Util.inspect and JSON.stringify: https://medium.com/@reginald.johnson/testing-aws-lambda-functions-the-easy-way-41cf1ed8c090 
 
*/
 
// AWS Variables
var aws = require('aws-sdk');
aws.config.update({region: 'eu-west-1'});
// SES const & vars
const ses = new aws.SES();
var util = require('util');
var emailfrom = 'info@shiftwiki . nl'; 
 
// Workflow Variables
// Declaring the customerData variable
var customerData;
 
// Send email to customer
function sendMail(params){
  console.log("Start function sendMail. Email address: " + params.email );
  if (customerData.discountCode === "testmodus"){
    emailto = 'sjoerd@shiftwiki .nl';
    emailfrom = 'info@shiftwiki .nl';
  } else {
    emailto = params.email;
  }
  console.log("Email address: " + emailto );
  //email
  var mailparams = {
      Destination: {
          ToAddresses: [emailto]
      },
      Message: {
          Body: {
              Html: { 
                  Charset: "UTF-8",
                  Data: "Dear " + params.firstname + " " + params.lastname +
                  "<br><br>" +
                  "All is ready. " +
                  "<br><br>" +
                  "Kind regards, <br>" +
                  "Shift"
              }
          },
          Subject: { Data: "[SHIFT]Ready. "}
      },
      Source: emailfrom
  };
  return ses.sendEmail(mailparams).promise();
}
 
// Error Handling
function errorResponse(message) {
  this.name = "email-to-customer-end";
  this.message = message;
}
errorResponse.prototype = new Error();
 
exports.handler = (event, context, callback) => {
    console.log("Start email-to-customer-end");
    //console.log(event);
    //console.log(context);
    customerData = event;
 
    // Display available information
    console.log(customerData);
 
    // Invoke function sendmail
    sendMail(customerData).then(() => {
      customerData.status = customerData.status + "Customer end email sent out. ";
      callback(null, customerData); 
    }).catch((err) => {
      // Error to CloudWatch
      console.error(err); 
      // Add Error to Task Output
      const error = new errorResponse(err.message);
      callback(error);
    });       
};

//jshint esversion:6
 
/* S3-Buckets-Add
Description: Creates two S3 buckets, configure access policy, website hosting and redirect.
 
Original PowerShell Script:
### S3 Bucket Creation as a Static Website ###
New-S3Bucket -BucketName $bucketname -PublicReadOnly 
Write-S3BucketWebsite -BucketName $bucketname -WebsiteConfiguration_IndexDocumentSuffix index.html -WebsiteConfiguration_ErrorDocument error.html
$s3policy = '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":"*","Action":"s3:GetObject","Resource":"arn:aws:s3:::bucketnameplaceholder/*"}]}'
$s3policy = $s3policy -replace "bucketnameplaceholder", "$bucketname"
Write-S3BucketPolicy -BucketName "$bucketname" -Policy $s3policy
write-host "S3 Bucket $bucketname was created on $((Get-S3Bucket -BucketName $bucketname).creationdate) "
### S3 Bucket Creation Redirect ###
New-S3Bucket -BucketName $wwwbucketname
Add-S3PublicAccessBlock -BucketName $wwwbucketname `
-PublicAccessBlockConfiguration_BlockPublicAcl $true `
-PublicAccessBlockConfiguration_BlockPublicPolicy $true `
-PublicAccessBlockConfiguration_IgnorePublicAcl $true `
-PublicAccessBlockConfiguration_RestrictPublicBucket $true
Write-S3BucketWebsite -BucketName $wwwbucketname -RedirectAllRequestsTo_HostName $bucketname
write-host "S3 Bucket $wwwbucketname was created on $((Get-S3Bucket -BucketName $wwwbucketname).creationdate) "
 
Links:
- AWS Javascript SDK for S3: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html
 
*/
 
// AWS Variables
var aws = require('aws-sdk');
aws.config.update({region: 'eu-west-1'});
var s3 = new aws.S3();
 
// Workflow Variables
// Declaring the customerData variable
var customerData;
 
//function s3BucketCreate
function s3BucketCreate(params) {
  console.log("Start function s3BucketCreate. Input: " + params.domainname);
  return s3.createBucket({
    Bucket: params.domainname, 
    CreateBucketConfiguration: {
     LocationConstraint: aws.config.region
    },
    ACL: "public-read"
  }).promise();
}
 
//function s3wwwBucketCreate
function s3wwwBucketCreate(params) {
  console.log("Start function s3wwwBucketCreate. Input: " + params.wwwdomainname);
  return s3.createBucket({
    Bucket: params.wwwdomainname, 
    CreateBucketConfiguration: {
     LocationConstraint: aws.config.region
    }
  }).promise();
}
 
//function s3BucketWebsite
function s3BucketWebsite(params){
  console.log("Start function s3BucketWebsite. Input: " + params.domainname);
  return s3.putBucketWebsite({
    Bucket: params.domainname,
    WebsiteConfiguration: { 
      ErrorDocument: {
        Key: 'error.html' 
      },
      IndexDocument: {
        Suffix: 'index.html' 
      }
    }
  }).promise();
}
 
//function s3BucketPolicy
function s3BucketPolicy(params){
  console.log("Start function s3BucketPolicy. Input: " + params.domainname);
  var s3policyplaceholder = "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"s3:GetObject\",\"Resource\":\"arn:aws:s3:::domainnameplaceholder/*\"}]}";
  var s3policy = s3policyplaceholder.replace("domainnameplaceholder", params.domainname);
  return s3.putBucketPolicy({
    Bucket: params.domainname, 
    Policy: s3policy,
  }).promise();
}
 
//function s3BucketPublicAccessBlock
function s3BucketPublicAccessBlock(params){
  console.log("Start function s3BucketPublicAccessBlock. Input: " + params.wwwdomainname);
  return s3.putPublicAccessBlock({
    Bucket: params.wwwdomainname,
    PublicAccessBlockConfiguration: { 
      BlockPublicAcls: true,
      BlockPublicPolicy: true,
      IgnorePublicAcls: true,
      RestrictPublicBuckets: true
    }
  }).promise();
}
 
//function s3BucketWebsiteRedirect
function s3BucketWebsiteRedirect(params){
  console.log("Start function s3BucketWebsiteRedirect. Input: " + params.wwwdomainname + " and " + params.domainname);
  return s3.putBucketWebsite({
    Bucket: params.wwwdomainname,
    WebsiteConfiguration: { 
      RedirectAllRequestsTo: {
        HostName: params.domainname
      }
    }
  }).promise();
}
 
// Error Handling
function errorResponse(message) {
  this.name = "s3-buckets-add";
  this.message = message;
}
errorResponse.prototype = new Error();
 
exports.handler = (event, context, callback) => {
    console.log("Start s3-buckets-add");
    // console.log(event);
    customerData = event;
 
    // Display available information
    console.log(customerData);
 
    // create params variable to pass to s3 functions
    var params = {
      domainname: customerData.domainname,
      wwwdomainname: customerData.wwwdomainname
    };
 
    // Invoke function s3BucketCreate
    s3BucketCreate(params).then(() => {
      customerData.status = customerData.status + "S3 Bucket creation success. ";
      s3BucketWebsite(params).then(() => {
          customerData.status = customerData.status + "S3 Bucket configured as website. ";
          s3BucketPolicy(params).then(() => {
            customerData.status = customerData.status + "S3 Bucket Policy set. ";
            s3wwwBucketCreate(params).then(() => {
              customerData.status = customerData.status + "S3 wwwBucket creation success. ";
              s3BucketPublicAccessBlock(params).then(() => {
                customerData.status = customerData.status + "S3 wwwBucket public access blocked. ";
                s3BucketWebsiteRedirect(params).then(() => {
                  customerData.status = customerData.status + "S3 wwwBucket redirected to S3 Bucket. ";
                  callback(null, customerData); 
                }).catch((err) => {
                  // Error to CloudWatch
                  console.error(err); 
                  // Add Error to Task Output
                  const error = new errorResponse(err.message);
                  callback(error);
                });
              }).catch((err) => {
                // Error to CloudWatch
                console.error(err); 
                // Add Error to Task Output
                const error = new errorResponse(err.message);
                callback(error);
              });
            }).catch((err) => {
              // Error to CloudWatch
              console.error(err); 
              // Add Error to Task Output
              const error = new errorResponse(err.message);
              callback(error);
            });
          }).catch((err) => {
            // Error to CloudWatch
            console.error(err); 
            // Add Error to Task Output
            const error = new errorResponse(err.message);
            callback(error); 
          });
      }).catch((err) => {
        // Error to CloudWatch
        console.error(err); 
        // Add Error to Task Output
        const error = new errorResponse(err.message);
        callback(error); 
      });
    }).catch((err) => {
      // Error to CloudWatch
      console.error(err); 
      // Add Error to Task Output
      const error = new errorResponse(err.message);
      callback(error);
    });
};

• Comment / uncomment: ctrl + /
• Word wrap: alt + z
• toggle from editor to terminal: ctrl + `
• toggle from terminal to editor: ctrl + 1

• Tabs: ALT 009
• Regex:
  • Search for title in string <h1>(.+?)<\/h1>
  • and replace to by using the $1: #### $1