I’m working on a project that requires direct interaction with the vRO API from a third party system. Maybe I’m getting less effective at using Google in my old age, but I had a heck of a time finding good solid code for authenticating to the API using anything, including PowerShell.

Typically I’d use Invoke-vRARestMethod from PowerVRA for this, but unfortunately I had some additional requirements that mandate me using a bearer token.

As a point of reference, you can view all of of the available APIs for vRealize Automation by navigating to this url: https://{your-vra-url}/automation-ui/api-docs/. We’ll be working with the “Orchestrator” API

The function below will return a bearer token, which you can use in future API calls to vRA.

Param’s:
Line 3: vra_server – the fqdn of your vRA 8 instance
Line 4: password – password associated with your vRA 8 account used to authenticate. As mentioned, please be smart here. Plantext passwords are bad.
Line 5: username – Unlike vRA 7, vRA 8 needs you to split your username and domain apart. For example, in vRA 7 you’d authenticate using the username jon.smith@rainpole.com, where in vRA 8 you authenticate using just the username jon.smith
Line 6: domain – This is where the domain goes (Example: rainpole.com)

Lines 14-24: We’re setting the body up to be included in the rest request. There’s some special formatting required so we’re using the replace method to modify the string.

Line 27: Build a simple variable with the URI used for authentication. More can be found on this in the API documentation mentioned above.

Lines 29-35: Make the API call, build the bearer token (essentially add the word “Bearer ” to the beginning of the authentication token), and return it.

Short and sweet. I’ll be adding some more snippets and problems in the future.

The post Authenticate to vRealize Orchestrator API using PowerShell appeared first on VirtJunkie.

I recently took a position at a new company and had the opportunity to stand up a greenfield environment. Part of this environment was obviously configuring monitoring. Since we’re pretty heavily tied to VMware here, we’re using vROps for our monitoring. For escalation of alerts, we’re using OpsGenie, and for normal collaboration we’re using Teams. Unfortunately, vROps versions older than 8.4 do not support sending alerts to OpsGenie or Teams out of the box. Fortunately, we can still get this working with a bit of work.

Overview

Approach

We’ll be running this whole project using Photon OS 4.

While we could simply run the python script in a screen, tmux, or similar session, but there are serious limitations in doing that. Admittedly, my python isn’t as strong as some of my other scripting languages, but after a bit of digging, it became clear that things like security, performance, and monitoring are either limited or unavailable if we do this. We’ll use a WSGI server called Gunicorn to run the python script, and use systemd to create a socket and service, and finally, use Nginx to connect directly to the socket. Whew! Even just reading that after completing this entire project feels like a lot, but don’t worry. As always, I’ll explain each step in detail and give you code to create this project.

Create Photon Packer Template

If you don’t know this about me by now, I’m a huge fan of packer. As such, we’ll be using it to create a template. You can find the source code for this packer template on my website’s github repository

To get a local copy, run the following command:

git clone https://github.com/jonhowe/Virtjunkie.com.git

You can find all source code in the directory Virtjunkie.com/Packer/PhotonOS4.

Create Custom ISO

Unfortunately, Photon OS, even in v4, doesn’t support providing the kickstart file via CD-ROM or floppy (see Issue#1113, Issue#798). Unfortunately, this means that if we want (or need) to use the vsphere-iso builder, we’ll need to create an ISO with the kickstart file embedded. I’m not going to duplicate documentation, so here’s the official documentation from VMware on how to add a kickstart file.

Instead of using the sample my_ks.cfg file provided in the ISO, use the version below.

IMPORTANT: Ensure that you replace [yourpassword] with your actual password

Use Packer To Create vSphere Template

IMPORTANT: Ensure that the password assigned to “root_password” in Virtjunkie.com/Packer/PhotonOS4/packer-photon.json.pkr.hcl matches what is set in the kickstart file above

packer build -var-file=nable.auto.pkrvars.hcl -var-file=vars iso-local-4.0GA.json -only=vsphere-iso.vmware-template packer-photon.json.pkr.hcl

Deploy Photon 4 VM

For production systems, I typically use Terraform to deploy my VMs, and that’s what I did. I’m not going to go over that, as I’ve done it before. If nothing else, just deploy the VM from a template, assign it a static IP in the guest customization specification, and then log in using credentials set in the kickstart file.

Configure Server

Install Packages

Execute the commands below to install required packages, and to clone the repository

Configure Shims

There’s a bit of config we’ll need to do in order to make this production ready.

Configure Teams

A fellow VMware vExpert, Shane Moore has an excellent article. So that I don’t recreate the wheel, please see his instructions for how to prep your environment to use Microsoft Teams.

In short, you’ll create a webhook connector in the Microsoft Teams channel, and plug the URL that you get from Teams into the teams shim file.

Configure OpsGenie

OpsGenie is pretty simple. All we’ll need to do is create an integration by selecting your team in Opsgenie, navigating to Integrations, then selecting Add Integration.

In the resulting list of available integrations, select API.

Then note the API key (you can grab it later, it doesn’t disappear when you navigate away). We’ll use that when we configure outbound settings in vROps to use OpsGenie.

Disable Unused Shims

In my environment, I’m only going to be using Teams and OpsGenie alerts. To reduce bloat and access points that could cause me security issues, I’m disabling all shims not in use.

Edit the file loginsightwebhookdemo/webhook-shims/loginsightwebhookdemo/__init__.py

Disable Welcome Page

This could probably be optional, but I appreciate not having a welcome/hello world webpage open for a production service. We’ll disable it.

Edit the file loginsightwebhookdemo/webhook-shims/loginsightwebhookdemo/__init__.py

Configure Systemd

Create two files: /etc/systemd/system/gunicorn.service and /etc/systemd/system/gunicorn.socket and populate with the text below.

/etc/systemd/system/gunicorn.service

This service file ensures that Gunicorn is started the same way, and is managed by systemd. You’ll note that it requires gunicorn.socket, which we will create later.

/etc/systemd/system/gunicorn.socket

This socket file sets permissions for the socket assigned to gunicorn

Configure Nginx

Replace the file /etc/nginx/nginx.conf with the contents below

This is telling Nginx to connect directly to the socket we’re creating with the systemd socket gunicorn.socket we created earlier.

Enable and Start All Services

Finally, enable, and start both services, and restart Nginx

systemctl enable gunicorn.socket
systemctl enable gunicorn.service
systemctl restart gunicorn.socket 
systemctl restart gunicorn.service
systemctl restart nginx.service

Configure vRealize Operations Manager

At this point, all we need to do is configure an outbound instance for Microsoft Teams and OpsGenie.

Add a New Rest Notification Plugin

OpsGenie

To do this, sign into vROps, and navigate to Administration ->Management->Outbound Settings, or https://[Your_vROps_URL]/ui/index.action#/administration/outbound-alert-settings

After you select the Rest Notification Plugin, there are 3 fields required in the Add/Edit Outbound Instance box

Finally, navigate to Alerts -> Configuration -> Notification (https://[Your_vROps_URL]/ui/index.action#/alerts/notifications) and add a new rule that targets the adapter you just created.

Teams

Again, I’m not in the business of recreating the wheel, so check out Shane’s blog for info on how to do this.

References

The post Configure vROps To Send Alerts to OpsGenie and Microsoft Teams appeared first on VirtJunkie.

Jitsi is an open source video conferencing platform that I’ve been hearing about a lot lately, and finally had a chance to look into. In this post I’ll explain how to use Terraform to provision a Jitsi instance when you need a conference and tear it down when you are done. We’ll be using Vultr and their Jitsi “application” and AWS Route 53 for DNS.

Why Am I Writing This Article, and What Does It Accomplish?

Why am I writing this article?

1. We always want to have our apps and infrastructure defined in code
2. We pay for traditional web conferencing software 24 hours a day, 7 days a week, regardless of if we are are using running a conference or not. Why don’t we spin up conference infrastructure when we need it, and tear it down when we don’t?

At a high level, this project will accomplish the following:

1. Provision a Vultr VPS that is pre-configured with Jitsi
2. Take the IP Address that Vultr assigns the VPS and use it to create an A Record in Route 53
3. Copy a script to your VPS that will be used to finish the Jitsi configuration
4. Run the script that we copied and pass a few command line arguments that are specific to our environment

Prerequisites

In addition to having Terraform downloaded and installed, we’ll need the following items:

Vultr Account + API Access

Vultr is definitely my go-to for VPS’s these days. Not only because of their price/performance/feature availability ratio, but because they provide a number of pre-configured applications that are ready, or near ready for use. Jitsi is one of these applications. If you do use Vultr, please do me a favor and use this link to sign up. I’ll get a little kickback, but you’ll get $100 USD to use on the site in your first month.

Once you have a Vultr account, you’ll need to generate and record an API key to use with Terraform. Use the steps below to generate it.

1. Log into Vultr
2. Navigate to Settings, and then API
3. Generate an API key, and copy it somewhere safe, we’ll be using it later

AWS Account + API Access

We’ll be using AWS’s Route 53 service, which is really just a fancy DNS service that’s hooked into AWS. In order to automate Route 53 with Terraform, we’ll need to enable API access.

Use this link to access the IAM Management page

1. Expand the “Access Keys” blade
2. Select “Create New Access Key
3. Save the resulting file, as we’ll use the contents later

Domain Registrar Using Custom Nameservers

In addition to the above, the domain you want to use will need to be configured to use the Route 53 Name Servers. Route 53 will provide you the nameservers when you create a zone, and you’ll simply plug those into your registrar DNS settings page. I’m not going to explain how to create a zone in Route 53, or how to configure your registrar, but if you have questions, throw them in the comments and I’ll do my best to help.

Getting Started

Run This Project

1. Grab the files below, or copy them from my Github Repository
2. Enter the directory that contains the files
3. At a minimum, modify the fields in the auto.tfvars file
   1. vultr_api_key
   1. aws_access_key
   2. aws_secret_key
   3. domain
   4. email
4. Initialize Terraform by running terraform init
5. Create a terraform plan by running terraform plan
6. Apply the configuration by running terraform apply
7. Voila! In less than 5 minutes, you’ve got a functional, secure Jitsi instance, running on a server and domain you control. Upon successful creation, you’ll see text like what we see below giving you the URL and credentials. When you are done with your conference, just run terraform destroy to stop from receiving charges on a server/service you aren’t using.

null_resource.jitsi_config (remote-exec): ------------------------------
null_resource.jitsi_config (remote-exec): |                            |
null_resource.jitsi_config (remote-exec): |   JITSI SETUP COMPLETED!   |
null_resource.jitsi_config (remote-exec): |                            |
null_resource.jitsi_config (remote-exec): ------------------------------
null_resource.jitsi_config (remote-exec): JITSI URL: https://conference.yourdomain.com/

null_resource.jitsi_config (remote-exec): USERNAME: admin
null_resource.jitsi_config (remote-exec): PASSWORD: @#$asdfahgsd34579--23%4asdf

Code

Main.tf

This file does all of the work.

#main.tf
#https://www.virtjunkie.com/jitsi-jit-conferencing-tf-vultr-route53/
#https://github.com/jonhowe/Virtjunkie.com/tree/master/Jitsi-JIT-Conferencing-TF-Vultr-Route53

#Conifugre the Vultr provider
provider "vultr" {
  api_key = var.vultr_api_key
  rate_limit = 700
  retry_limit = 3
}

#Configure the AWS Provider
provider "aws" {
  #profile    = "default"
  #shared_credentials_file = "/home/jhowe/storage/btsync/folders/Sync/awscredentials/credentials"
  region     = var.aws_region
  access_key = var.aws_access_key
  secret_key = var.aws_secret_key
}

#https://www.terraform.io/docs/providers/aws/d/route53_zone.html
data "aws_route53_zone" "selected" {
  name         = "${var.domain}."
  private_zone = false
}

#Provision Vultr Server
resource "vultr_server" "my_server" {
    plan_id = var.vultr_plan_id
    region_id = var.vultr_region
    app_id = var.vultr_app_id
    label = "${var.hostname}.${var.domain}"
    tag = var.vultr_tag
    hostname = "${var.hostname}.${var.domain}"
    enable_ipv6 = false
    auto_backup = false
    ddos_protection = false
    notify_activate = false

    connection {
        type     = "ssh"
        user     = "root"
        
        #https://www.terraform.io/docs/providers/vultr/r/server.html#default_password
        password = self.default_password

        #https://www.terraform.io/docs/provisioners/connection.html#the-self-object
        host     = self.main_ip
    }

    provisioner "local-exec" {
      command = "echo SSH to this server with the command: ssh root@${vultr_server.my_server.main_ip} with the password '${vultr_server.my_server.default_password}'"
    }
}

#Create the Route 53 A Record
#https://www.terraform.io/docs/providers/aws/r/route53_record.html
resource "aws_route53_record" "conference" {
  zone_id = data.aws_route53_zone.selected.zone_id
  name    = "${var.hostname}.${data.aws_route53_zone.selected.name}"
  type    = "A"
  ttl     = "300"
  records = ["${vultr_server.my_server.main_ip}"]
}

#This null resource exists to handle configuration of the Vultr VPS after Route 53
resource "null_resource" "jitsi_config" {
    
    connection {
        type     = "ssh"
        user     = "root"
        
        #https://www.terraform.io/docs/providers/vultr/r/server.html#default_password
        password = vultr_server.my_server.default_password

        #https://www.terraform.io/docs/provisioners/connection.html#the-self-object
        host     = vultr_server.my_server.main_ip
    }

    provisioner "file" {
        source      = "./configure_jitsi_param.sh"
        destination = "/root/configure_jitsi_param.sh"
    }

    provisioner "remote-exec" {
        inline = [
            "chmod +x /root/configure_jitsi_param.sh",
            "/root/configure_jitsi_param.sh ${var.hostname}.${var.domain} ${var.email} y"
        ]
    }
}

Variables.tf

This file defines the variables that we will use in main.tf

#variables.tf
#https://www.virtjunkie.com/jitsi-jit-conferencing-tf-vultr-route53/
#https://github.com/jonhowe/Virtjunkie.com/tree/master/Jitsi-JIT-Conferencing-TF-Vultr-Route53

variable "vultr_api_key" {
    description = "API Key Used by Vultr (https://my.vultr.com/settings/#settingsapi)"
}

variable "vultr_region" {
    description = "Vultr Region Selection (curl https://api.vultr.com/v1/regions/availability?DCID=1)"
    default = 1
}

variable "vultr_plan_id" {
    description = "Vultr Plan for the VPS to use (curl https://api.vultr.com/v1/plans/list)"
    default = 202
}

variable "vultr_tag" {
    description = "Vultr Tag to apply to the new VPS"
    default = "jitsi-conference"
}

variable "vultr_app_id" {
    description = "Vultr App to pre-install. This should always be '47', if jitsi is being provisioned (curl https://api.vultr.com/v1/app/list)"
    default = 47
}

variable "hostname" {
    description = "Hostname to be used"
    default = "conferences"
}

variable "email" {
    description = "email to be used for let's encrypt acme config"
    default = "john.doe@email.com"
}

variable "domain" {
    description = "domain to be used"
    default = "aremyj.am"
}

variable "aws_access_key" {
    description = "AWS Access Key - get it here: (https://console.aws.amazon.com/iam/home?#security_credential)"
}

variable "aws_secret_key" {
    description = "AWS Secret Key - get it here: (https://console.aws.amazon.com/iam/home?#security_credential)"
}

variable "aws_region" {
    description = "AWS Region"
    default = "us-east-1"
}

[yourdomain].auto.tfvars

The auto.tfvars file provides values to the variables defined in the variables.tf file. You’ll have to create this file from scratch, and terraform best practices dictate that you exclude this file from source control. Here’s an example you can use. Modify this for your environment. The name doesn’t matter, as long as it ends with auto.tfvars.

vultr_api_key = "[fill this in]"
vultr_region = 1
vultr_plan_id = 202
vultr_app_id = 47
vultr_tag = "jitsi-conference"
hostname = "conference"
email = "your.email@address.org"
domain = "your-domain.com"
aws_region = "us-east-1"
aws_access_key = "[fill this in]"
aws_secret_key = "[fill this in]"

configure_jitsi_param.sh

Full disclosure, I did not create this script. Vultr created it, and provides it on your Jitsi VPS when you request it. Unfortunately, the version they provide is intended to be executed interactively, so I made a few very minor modifications to allow for us to run it with parameters.

#!/bin/bash
#This script was copied from /opt/vultr/configure_jitsi.sh on a Vultr VPS that has the one-click Jitsi App
#I added lines 7-9 to allow for adding parameters on the CLI and commented lines 11-13 to force the variables to be provided on the CLI
#https://www.vultr.com/docs/one-click-jitsi
#https://www.virtjunkie.com/jitsi-jit-conferencing-tf-vultr-route53/
#https://github.com/jonhowe/Virtjunkie.com/tree/master/Jitsi-JIT-Conferencing-TF-Vultr-Route53
HOSTNAME=$1
EMAIL=$2
response=$3
# User choices
#read -ep "Please specify which domain you would like to use: " HOSTNAME
#read -ep "Please enter your email address for Let's Encrypt Registration: " EMAIL
#read -r -p "Would you like to enable password authorization? [y/N] " response
case "$response" in
    [yY][eE][sS]|[yY])
        AUTH=1
        ;;
    *)
        AUTH=0
        ;;
esac


PROSODYPATH=/etc/prosody/conf.avail/${HOSTNAME}.cfg.lua
JITSIPATH=/etc/jitsi/meet/${HOSTNAME}-config.js
JICOFOPATH=/etc/jitsi/jicofo/sip-communicator.properties

# Remove and purge (Stop first and wait to avoid race condition)
purgeold() {        
        /opt/vultr/stopjitsi.sh
        sleep 5
        apt -y purge jigasi jitsi-meet jitsi-meet-web-config jitsi-meet-prosody jitsi-meet-turnserver jitsi-meet-web jicofo jitsi-videobridge2 jitsi*
}

# Reinstall
reinstalljitsi() {
        echo "jitsi-videobridge2 jitsi-videobridge/jvb-hostname string ${HOSTNAME}" | debconf-set-selections
        echo "jitsi-meet-web-config jitsi-meet/cert-choice string Generate a new self-signed certificate (You will later get a chance to obtain a Let's encrypt certificate)" | debconf-set-selections
        apt-get -y install jitsi-meet
}

# Remove nginx defaults
wipenginx() {
        rm -f /etc/nginx/sites-enabled/default
}

# Configure Lets Encrypt
configssl(){
    systemctl restart nginx
    sed -i -e 's/echo.*Enter your email and press.*/EMAIL=$1/' /usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh
    sed -i -e 's/read EMAIL//'  /usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh
    /usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh ${EMAIL}
}

configprosody() {
  AUHTLINE='authentication = "internal_plain"'
  sed -i "s/authentication\ \=\ \"anonymous\"/${AUTHLINE}/g" ${PROSODYPATH}
  cat << EOT >> ${PROSODYPATH}

VirtualHost "guest.${HOSTNAME}"
    authentication = "anonymous"
    c2s_require_encryption = false

EOT
}

configjitsi() {
        sed -i "s/\/\/\ anonymousdomain\:\ 'guest.example.com',/anonymousdomain\:\ 'guest.${HOSTNAME}',/g" ${JITSIPATH}
}

configjicofo() {
        echo "org.jitsi.jicofo.auth.URL=XMPP:${HOSTNAME}" >> ${JICOFOPATH}
}

registeruser(){
        PASSWORD=$(< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c${1:-16};echo;)
        prosodyctl register admin ${HOSTNAME} ${PASSWORD}
}

restartjitsi() {
        /opt/vultr/stopjitsi.sh
        /opt/vultr/startjitsi.sh
}

completedsetup(){
    echo ""
    echo "------------------------------"
    echo "|                            |"
    echo "|   JITSI SETUP COMPLETED!   |"
    echo "|                            |"
    echo "------------------------------"
    echo "JITSI URL: https://${HOSTNAME}"/
    echo ""
}

outputUser(){
    echo "USERNAME: admin"
    echo "PASSWORD: ${PASSWORD}"
    echo ""
}

# Script start

purgeold
reinstalljitsi
wipenginx
configssl
if [ "$AUTH" == "1" ]; then
    configprosody
    configjitsi
    configjicofo
    registeruser
    restartjitsi    
fi
completedsetup
if [ "$AUTH" == "1" ]; then
    outputUser
fi

Summary + More Reading

There you have it! With this project you can have a fully functional Jitsi instance on your own domain with end to end encryption in less than 5 minutes. When you are done, there’s no harm in deleting it so you aren’t charged.

Here are some references I used while creating this:

• https://github.com/jonhowe/Virtjunkie.com/tree/master/Jitsi-JIT-Conferencing-TF-Vultr-Route53
• https://www.vultr.com/docs/one-click-jitsi
• https://www.terraform.io/docs/providers/aws/d/route53_zone.html
• https://www.terraform.io/docs/providers/aws/r/route53_record.html
• https://www.terraform.io/docs/providers/vultr/r/server.html#default_password
• https://www.terraform.io/docs/provisioners/connection.html#the-self-object
• Create Vultr API Key: https://my.vultr.com/settings/#settingsapi
• Create AWS Access/Secret Key: https://console.aws.amazon.com/iam/home?#security_credential
• Vultr API Reference – has examples that will get you plan, region, and app IDs. https://www.vultr.com/api/

The post Jitsi for Just in Time Conferencing using Terraform on Vultr with Route 53 appeared first on VirtJunkie.

This solution will automatically provision ESXi hosts and “get them on the network” so you can add them to vCenter just by using tools that VMware gives us, and a little bit of creativity.

While I designed this solution to work with hundreds of production ESXi hosts, there’s nothing from preventing you from using this method in smaller production environments, or even your own home lab.

Overview

At a very high level, the process works as follows.. The physical server boots from an ISO, USB Image, or even CD/DVD, if you’re into that sort of thing. The ESXi image on this media will contain a slightly customized ESXi image that looks for a kickstart file pulls the rest of the install instructions from a kickstart file on a web server. This kickstart fill will handle the silent install of ESXi, reboot into the newly installed OS, set a few settings, then download a CSV that contains ESXi host configuration info (such as host name, IP, and Default Gateway), and a python script that will send esxcli commands to the host in order to configure networking.

Diagram

bootstate=0
title=Loading ESXi installer
timeout=5
prefix=
kernel=/b.b00
#kernelopt=cdromBoot runweasel
kernelopt=ks=http://vmware-ks.home.lab:8123/files/VMware/ks/ks.cfg nameserver=192.168.1.60 ip=192.168.1.5 netmask=255.255.255.0 gateway=192.168.1.1
modules=/jumpstrt.gz --- /useropts.gz --- /features.gz --- /k.b00 --- /chardevs.b00 --- /user.b00 --- /procfs.b00 --- /uc_intel.b00 --- /uc_amd.b00 --- /uc_hygon.b00 --- /vmx.v00 --- /vim.v00 --- /sb.v00 --- /s.v00 --- /ata_liba.v00 --- /ata_pata.v00 --- /ata_pata.v01 --- /ata_pata.v02 --- /ata_pata.v03 --- /ata_pata.v04 --- /ata_pata.v05 --- /ata_pata.v06 --- /ata_pata.v07 --- /block_cc.v00 --- /bnxtnet.v00 --- /bnxtroce.v00 --- /brcmfcoe.v00 --- /char_ran.v00 --- /ehci_ehc.v00 --- /elxiscsi.v00 --- /elxnet.v00 --- /hid_hid.v00 --- /i40en.v00 --- /iavmd.v00 --- /igbn.v00 --- /ima_qla4.v00 --- /ipmi_ipm.v00 --- /ipmi_ipm.v01 --- /ipmi_ipm.v02 --- /iser.v00 --- /ixgben.v00 --- /lpfc.v00 --- /lpnic.v00 --- /lsi_mr3.v00 --- /lsi_msgp.v00 --- /lsi_msgp.v01 --- /lsi_msgp.v02 --- /misc_cni.v00 --- /misc_dri.v00 --- /mtip32xx.v00 --- /ne1000.v00 --- /nenic.v00 --- /net_bnx2.v00 --- /net_bnx2.v01 --- /net_cdc_.v00 --- /net_cnic.v00 --- /net_e100.v00 --- /net_e100.v01 --- /net_enic.v00 --- /net_fcoe.v00 --- /net_forc.v00 --- /net_igb.v00 --- /net_ixgb.v00 --- /net_libf.v00 --- /net_mlx4.v00 --- /net_mlx4.v01 --- /net_nx_n.v00 --- /net_tg3.v00 --- /net_usbn.v00 --- /net_vmxn.v00 --- /nfnic.v00 --- /nhpsa.v00 --- /nmlx4_co.v00 --- /nmlx4_en.v00 --- /nmlx4_rd.v00 --- /nmlx5_co.v00 --- /nmlx5_rd.v00 --- /ntg3.v00 --- /nvme.v00 --- /nvmxnet3.v00 --- /nvmxnet3.v01 --- /ohci_usb.v00 --- /pvscsi.v00 --- /qcnic.v00 --- /qedentv.v00 --- /qfle3.v00 --- /qfle3f.v00 --- /qfle3i.v00 --- /qflge.v00 --- /sata_ahc.v00 --- /sata_ata.v00 --- /sata_sat.v00 --- /sata_sat.v01 --- /sata_sat.v02 --- /sata_sat.v03 --- /sata_sat.v04 --- /scsi_aac.v00 --- /scsi_adp.v00 --- /scsi_aic.v00 --- /scsi_bnx.v00 --- /scsi_bnx.v01 --- /scsi_fni.v00 --- /scsi_hps.v00 --- /scsi_ips.v00 --- /scsi_isc.v00 --- /scsi_lib.v00 --- /scsi_meg.v00 --- /scsi_meg.v01 --- /scsi_meg.v02 --- /scsi_mpt.v00 --- /scsi_mpt.v01 --- /scsi_mpt.v02 --- /scsi_qla.v00 --- /sfvmk.v00 --- /shim_isc.v00 --- /shim_isc.v01 --- /shim_lib.v00 --- /shim_lib.v01 --- /shim_lib.v02 --- /shim_lib.v03 --- /shim_lib.v04 --- /shim_lib.v05 --- /shim_vmk.v00 --- /shim_vmk.v01 --- /shim_vmk.v02 --- /smartpqi.v00 --- /uhci_usb.v00 --- /usb_stor.v00 --- /usbcore_.v00 --- /vmkata.v00 --- /vmkfcoe.v00 --- /vmkplexe.v00 --- /vmkusb.v00 --- /vmw_ahci.v00 --- /xhci_xhc.v00 --- /elx_esx_.v00 --- /btldr.t00 --- /esx_dvfi.v00 --- /esx_ui.v00 --- /esxupdt.v00 --- /weaselin.t00 --- /lsu_hp_h.v00 --- /lsu_inte.v00 --- /lsu_lsi_.v00 --- /lsu_lsi_.v01 --- /lsu_lsi_.v02 --- /lsu_lsi_.v03 --- /lsu_lsi_.v04 --- /lsu_smar.v00 --- /native_m.v00 --- /qlnative.v00 --- /rste.v00 --- /vmware_e.v00 --- /vsan.v00 --- /vsanheal.v00 --- /vsanmgmt.v00 --- /tools.t00 --- /xorg.v00 --- /imgdb.tgz --- /imgpayld.tgz
build=
updated=0

#__      ___      _       _             _    _      
#\ \    / (_)    | |     | |           | |  (_)     
# \ \  / / _ _ __| |_    | |_   _ _ __ | | ___  ___ 
#  \ \/ / | | '__| __|   | | | | | '_ \| |/ / |/ _ \
#   \  /  | | |  | || |__| | |_| | | | |   <| |  __/
#    \/   |_|_|   \__\____/ \__,_|_| |_|_|\_\_|\___|

#Jon Howe
#https://www.virtjunkie.com/Poor-Mans-AutoDeploy
#https://github.com/jonhowe/Virtjunkie.com/tree/master/PoorMansAutoDeploy

#  _____ _                     __            _____           _        _ _       _   _             
# / ____| |                   /_ |          |_   _|         | |      | | |     | | (_)            
#| (___ | |_ __ _  __ _  ___   | |  ______    | |  _ __  ___| |_ __ _| | | __ _| |_ _  ___  _ __  
# \___ \| __/ _` |/ _` |/ _ \  | | |______|   | | | '_ \/ __| __/ _` | | |/ _` | __| |/ _ \| '_ \ 
# ____) | || (_| | (_| |  __/  | |           _| |_| | | \__ \ || (_| | | | (_| | |_| | (_) | | | |
#|_____/ \__\__,_|\__, |\___|  |_|          |_____|_| |_|___/\__\__,_|_|_|\__,_|\__|_|\___/|_| |_|
#                  __/ |                                                                          
#                 |___/                                                                           

# Accept the VMware End User License Agreement
vmaccepteula

# The install media is in the CD-ROM drive
#   This was the hardest part for some reason. I was installing from a USB drive TO a separate USB drive.
#   https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.esxi.upgrade.doc/GUID-61A14EBB-5CF3-43EE-87EF-DB8EC6D83698.html
#   https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.esxi.upgrade.doc/GUID-E7274FBA-CABC-43E8-BF74-2924FD3EFE1E.html
#   If you find the wrong device is being selected, you can hit CTRL+F1 during the install, log in (root/[no password]) and enter
#   the following command to get a list of the available devices:
#   localcli storage core device list
install --disk=/vmfs/devices/disks/mpx.vmhba33:C0:T0:L0 --overwritevmfs --novmfsondisk

# Set the root password for the DCUI and Tech Support Mode
rootpw VMware1!

### Temporary Network Configuration
#   Keep in mind, this is temporary. The actual IP address will be set in Stage 2
#   We don't even technically need this, but I wanted to include it in case we need network connectivity in Stage 1

# Set the network to DHCP on the first network adapter
network --bootproto=dhcp --device=vmnic0
# ... or if you want, you can set this to static
# network --bootproto=static --ip=192.168.30.13 --netmask=255.255.255.0 --gateway=192.168.30.1 --hostname=esxi1-mgmt.home.lab --nameserver=192.168.0.1 --addvmportgroup=1

reboot

#  _____ _                     ___              _____ _                 _         _____             __ _       
# / ____| |                   |__ \            / ____(_)               | |       / ____|           / _(_)      
#| (___ | |_ __ _  __ _  ___     ) |  ______  | (___  _ _ __ ___  _ __ | | ___  | |     ___  _ __ | |_ _  __ _ 
# \___ \| __/ _` |/ _` |/ _ \   / /  |______|  \___ \| | '_ ` _ \| '_ \| |/ _ \ | |    / _ \| '_ \|  _| |/ _` |
# ____) | || (_| | (_| |  __/  / /_            ____) | | | | | | | |_) | |  __/ | |___| (_) | | | | | | | (_| |
#|_____/ \__\__,_|\__, |\___| |____|          |_____/|_|_| |_| |_| .__/|_|\___|  \_____\___/|_| |_|_| |_|\__, |
#                  __/ |                                         | |                                      __/ |
#                 |___/                                          |_|                                     |___/ 

%firstboot --interpreter=busybox

# enable & start SSH
vim-cmd hostsvc/enable_ssh
vim-cmd hostsvc/start_ssh

# enable & start ESXi Shell
vim-cmd hostsvc/enable_esx_shell
vim-cmd hostsvc/start_esx_shell

# Suppress ESXi Shell warning
esxcli system settings advanced set -o /UserVars/SuppressShellWarning -i 1

#Temporarily disable the firewall so we can grab a couple files using wget
esxcli network firewall set --enabled false

# Download Host Config CSV, host configuration script, and execute the host configuration script
wget -O ListOfHosts.csv http://jondesktop.home.lan:8123/files/VMware/ks/ListOfHosts.csv
wget -O configurehost.py http://jondesktop.home.lan:8123/files/VMware/ks/configurehost.py
chmod u+x configurehost.py
/bin/python configurehost.py

#Re-enable the firewall
esxcli network firewall set --enabled true

configureHost.py

Here’s where the magic comes in… Since ESXi is deployed with python, we can use it to parse a CSV file that has a list of all details for an ESXi host, and set the relevant details for the host.
Here’s a link to the code

In this very basic example, we are doing the following based on input in the CSV file:

1. Set the hostname
2. Set the IP Address and Netmask of vmk0
3. Set the default gateway for the host
4. Set the VLAN for the management port group on vSwitch0

configureHost.py code

#__      ___      _       _             _    _      
#\ \    / (_)    | |     | |           | |  (_)     
# \ \  / / _ _ __| |_    | |_   _ _ __ | | ___  ___ 
#  \ \/ / | | '__| __|   | | | | | '_ \| |/ / |/ _ \
#   \  /  | | |  | || |__| | |_| | | | |   <| |  __/
#    \/   |_|_|   \__\____/ \__,_|_| |_|_|\_\_|\___|
                                                  
#Jon Howe
#https://www.virtjunkie.com/Poor-Mans-AutoDeploy
#https://github.com/jonhowe/Virtjunkie.com/tree/master/PoorMansAutoDeploy

import os, csv, subprocess

CSVPATH = '/ListOfHosts.csv'

#Gets all mac addresses of interfaces
MAC=subprocess.check_output("esxcli network ip interface list |grep MAC | cut -d ' ' -f 6",shell=True)
                                                                                                                                                                                                                                                                                    
MACADDR = MAC.split()
print("MAC Addresses:")
print(MACADDR)

#Open the CSV file
with open(CSVPATH, 'rt') as csvFile:
        csvReader = csv.reader(csvFile)
        #Process the CSV file, row by row
        for csvRow in csvReader:
                #Check to see if the Mac Address of vmnic0 (IE: MACADDR[0]) matches the row in the CSV we are looking at
                if ((MACADDR[0]).decode('utf-8')) == csvRow[1]:
                        #print("esxcli system hostname set --fqdn=" + csvRow[0])
                        #print("esxcli network ip interface ipv4 set --interface-name=vmk0 --ipv4=" + csvRow[2] + " --netmask=" + csvRow[3] + " --type=static")
                        
                        #Set the hostname to the value in the CSV
                        os.system("esxcli system hostname set --fqdn=" + csvRow[0])
                        
                        #Set the IP Address and Netmask of vmk0
                        os.system("esxcli network ip interface ipv4 set --interface-name=vmk0 --ipv4=" + csvRow[2] + " --netmask=" + csvRow[3] + " --type=static")
                        
                        #Add the default gateway
                        os.system("esxcfg-route " + csvRow[4])
                        
                        #Add the desired VLAN for the Management Network on the vSwitch that gets created by default (vSwitch0)
                        os.system("esxcfg-vswitch -p 'Management Network' -v " + csvRow[5] + " vSwitch0")

docker run -p 8123:80 -v /storage/poormansautodeploy/ks/:/opt/www/files --name=filebrowser -d --restart unless-stopped --log-opt max-size=15m --log-opt max-file=2 mohamnag/nginx-file-browser

ks
├── configurehost.py
├── ks.cfg
└── ListOfHosts.csv

Web Server In Action

genisoimage -relaxed-filenames -J -R -o [yourisoname].iso -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e efiboot.img -no-emul-boot [path to extracted iso]

The post Poor Man’s AutoDeploy Using Custom Kickstart, and Python appeared first on VirtJunkie.

"iso_paths": [
    "[esxi1-nvme] ISO/Windows/14393.0.161119-1705.RS1_REFRESH_SERVER_EVAL_X64FRE_EN-US.ISO",
    "[esxi1-nvme] ISO/VMTools/VMware-tools-windows-11.0.1-14773994.iso"
]

"floppy_files": [
    "./autounattend.xml",
    "Scripts/vmtools.cmd",
    "Scripts/level0-setup.ps1"
]

Linux Templates

Linux templates are pretty straight forward. Being born in open source, they are much easier to automate than Windows. Debian based distributions use a preseed file to silently execute the build. We will use packer to do the following:

1. Create a blank VM
2. Attach Linux installer ISO
3. Create and attach media that has preseed.cfg file to the newly created image
4. Start VM
5. Send commands to the running VM via USB emulation to tell the Ubuntu installer to use the preseed.cfg file
6. Update all packages on the template
7. Shut Down VM
8. Mark as Template

Here’s the directory structure:

Ubuntu18/
 ├── preseed.cfg
 ├── ubuntu-18.04.json
 └── ubuntu-vars.json
 0 directories, 3 files

WS19
 ├── autounattend.xml
 ├── Scripts
 │   ├── level0-setup.ps1
 │   ├── level1-setup.ps1
 │   └── vmtools.cmd
 ├── vars-ws19.json
 └── ws19.json
 1 directory, 6 files

Autounattend.xml

Microsoft allows us to provide an answer file to the Windows installer to achieve a silent install of Windows Server. The installer will look in all attached media to find a file named Autounattend.xml. This file contains a number of settings that are explained in a decent amount of detail on this Microsoft documentation page. Below is a high level overview of the passes and how we are using them.

I found this page helpful in creating the skeleton of the XML file.

windowsPE

• Select disk format
• Select the OS Image (Core, GUI, Datacenter, Standard)
• Add the product key (we’ll be using the 6 month trial)
• Select Locale, Name, Organization

offlineServicing

• This configuration pass is used to apply updates, drivers, or language packs to a Windows image.

generalize

• We are setting the property SkipRearm
• Resets the SID

specialize

• This configuration pass is used to create and configure information in the Windows image, and is specific to the hardware that the Windows image is installing to.
• Install VMware Tools by executing (vmtools.cmd) in this step to ensure it’s available ASAP for Packer to perform the rest of template configuration steps
• Computer name is set
• Reboots the machine

oobeSystem

• Add user, set password
• Execute powershell script (level0-setup.ps1) to enable WinRM, which is required for Packer provisioning

Examples

Here are a few examples of how the script works

Windows

The command below will create a windows template with the following details.

If you want to change the password, you will need to specify it both in the vars file, as well as the autounattend.xml. The section you will change looks like this:

<UserAccounts>
    <AdministratorPassword>
        <Value>VMware1!</Value>
        <PlainText>true</PlainText>
    </AdministratorPassword>
    <LocalAccounts>
        <LocalAccount wcm:action="add">
            <Description>Lab Administrator</Description>
            <DisplayName>labadmin</DisplayName>
            <Group>Administrators</Group>
            <Name>labadmin</Name>
        </LocalAccount>
    </LocalAccounts>
</UserAccounts>
<AutoLogon>
    <Password>
        <Value>VMware1!</Value>
        <PlainText>true</PlainText>
    </Password>
    <Enabled>true</Enabled>
    <Username>Administrator</Username>
</AutoLogon>

In Action

packer build -var-file vars-ws19.json ws19.json
vsphere-iso: output will be in this color.

==> vsphere-iso: Creating VM...
==> vsphere-iso: Customizing hardware...
==> vsphere-iso: Mounting ISO images...
==> vsphere-iso: Creating floppy disk...
    vsphere-iso: Copying files flatly from floppy_files
    vsphere-iso: Copying file: ./autounattend.xml
    vsphere-iso: Copying file: Scripts/vmtools.cmd
    vsphere-iso: Copying file: Scripts/level0-setup.ps1
    vsphere-iso: Done copying files from floppy_files
    vsphere-iso: Collecting paths from floppy_dirs
    vsphere-iso: Resulting paths from floppy_dirs : []
    vsphere-iso: Done copying paths from floppy_dirs
==> vsphere-iso: Uploading created floppy image
==> vsphere-iso: Adding generated Floppy...
==> vsphere-iso: Set boot order...
==> vsphere-iso: Power on VM...
==> vsphere-iso: Waiting for IP...
==> vsphere-iso: IP address: 172.16.1.73
==> vsphere-iso: Using winrm communicator to connect: 172.16.1.73
==> vsphere-iso: Waiting for WinRM to become available...
    vsphere-iso: WinRM connected.
==> vsphere-iso: #< CLIXML
==> vsphere-iso: <Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"><Obj S="progress" RefId="0"><TN RefId="0"><T>System.Management.Automation.PSCustomObject</T><T>System.Object</T></TN><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj><Obj S="progress" RefId="1"><TNRef RefId="0" /><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj></Objs>
==> vsphere-iso: Connected to WinRM!
==> vsphere-iso: Provisioning with Powershell...
==> vsphere-iso: Provisioning with powershell script: Scripts/level1-setup.ps1
    vsphere-iso:
    vsphere-iso:
    vsphere-iso:     Directory: C:\Users\Administrator\Documents\WindowsPowerShell
    vsphere-iso:
    vsphere-iso:
    vsphere-iso: Mode                LastWriteTime         Length Name
    vsphere-iso: ----                -------------         ------ ----
    vsphere-iso: -a----        3/22/2020   3:02 PM              0 Microsoft.PowerShell_profile.ps1
==> vsphere-iso: Restarting Machine
==> vsphere-iso: Waiting for machine to restart...
==> vsphere-iso: A system shutdown is in progress.(1115)
==> vsphere-iso: #< CLIXML
    vsphere-iso: WS19-TPL restarted.
==> vsphere-iso: <Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"><Obj S="progress" RefId="0"><TN RefId="0"><T>System.Management.Automation.PSCustomObject</T><T>System.Object</T></TN><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj><Obj S="progress" RefId="1"><TNRef RefId="0" /><MS><I64 N="SourceId">2</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj></Objs>
==> vsphere-iso: Machine successfully restarted, moving on
==> vsphere-iso: Uploading the Windows update elevated script...
==> vsphere-iso: Uploading the Windows update check for reboot required elevated script...
==> vsphere-iso: Uploading the Windows update script...
==> vsphere-iso: Running Windows update...
    vsphere-iso: Searching for Windows updates...
    vsphere-iso: Found Windows update (2018-10-15; 20.72 MB): Update for Adobe Flash Player for Windows Server 2019 (1809) for x64-based Systems (KB4462930)
    vsphere-iso: Found Windows update (2020-02-11; 20.75 MB): 2020-02 Security Update for Adobe Flash Player for Windows Server 2019 for x64-based Systems (KB4537759)
    vsphere-iso: Found Windows update (2020-02-25; 71.37 MB): 2020-02 Cumulative Update for .NET Framework 3.5, 4.7.2 and 4.8 for Windows Server 2019 for x64 (KB4538156)
    vsphere-iso: Found Windows update (2020-02-25; 2.34 MB): 2020-01 Update for Windows Server 2019 for x64-based Systems (KB4494174)
    vsphere-iso: Found Windows update (2020-03-10; 30.62 MB): Windows Malicious Software Removal Tool x64 - March 2020 (KB890830)
    vsphere-iso: Found Windows update (2020-03-22; 114.85 MB): Security Intelligence Update for Windows Defender Antivirus - KB2267602 (Version 1.311.1752.0)
    vsphere-iso: Found Windows update (2020-03-10; 15349.68 MB): 2020-03 Cumulative Update for Windows Server 2019 (1809) for x64-based Systems (KB4538461)
    vsphere-iso: Downloading Windows updates (7 updates; 15610.33 MB)...
    vsphere-iso: Waiting for operation to complete (system performance: 44% cpu; 31% memory)...
    vsphere-iso: Installing Windows updates...
    vsphere-iso: Waiting for operation to complete (system performance: 20% cpu; 41% memory)...
    vsphere-iso: Waiting for operation to complete (system performance: 51% cpu; 51% memory)...
    vsphere-iso: Waiting for operation to complete (system performance: 51% cpu; 50% memory)...
    vsphere-iso: Waiting for operation to complete (system performance: 50% cpu; 54% memory)...
    vsphere-iso: Waiting for operation to complete (system performance: 43% cpu; 55% memory)...
    vsphere-iso: Waiting for operation to complete (system performance: 0% cpu; 61% memory)...
    vsphere-iso: Waiting for operation to complete (system performance: 38% cpu; 52% memory)...
    vsphere-iso: Waiting for operation to complete (system performance: 63% cpu; 48% memory)...
    vsphere-iso: Waiting for operation to complete (system performance: 27% cpu; 46% memory)...
    vsphere-iso: Waiting for operation to complete (system performance: 45% cpu; 47% memory)...
    vsphere-iso: Waiting for operation to complete (system performance: 55% cpu; 46% memory)...
    vsphere-iso: Waiting for operation to complete (system performance: 53% cpu; 44% memory)...
    vsphere-iso: Waiting for the Windows Modules Installer to exit...
    vsphere-iso: Waiting for operation to complete (system performance: 81% cpu; 44% memory)...
==> vsphere-iso: Restarting the machine...
==> vsphere-iso: Waiting for machine to become available...
==> vsphere-iso: A system shutdown is in progress.(1115)
==> vsphere-iso: A system shutdown is in progress.(1115)
    vsphere-iso: Waiting for the Windows Modules Installer to exit...
    vsphere-iso: WS19-TPL restarted.
==> vsphere-iso: Running Windows update...
    vsphere-iso: Searching for Windows updates...
    vsphere-iso: Found Windows update (2020-03-19; 4.91 MB): Update for Windows Defender Antivirus antimalware platform - KB4052623 (Version 4.18.2003.6)
    vsphere-iso: Downloading Windows updates (1 updates; 4.91 MB)...
    vsphere-iso: Installing Windows updates...
==> vsphere-iso: Shut down VM...
==> vsphere-iso: Deleting Floppy drives...
==> vsphere-iso: Deleting Floppy image...
==> vsphere-iso: Eject CD-ROM drives...
==> vsphere-iso: Convert VM into template...
Build 'vsphere-iso' finished.

==> Builds finished. The artifacts of successful builds are:
--> vsphere-iso: WS19-TPL

Linux

The command below will create a Ubuntu template with the following details:

All of the above can be changed by modifying values in the preseed.cfg file.

In Action

packer build -var-file ubuntu-vars.json ubuntu-18.04.json

jhowe@jonlaptopalt:~/git/Personal/HomeLab/packer/Ubuntu18$ packer build -var-file ubuntu-vars.json ubuntu-18.04.json 
vsphere-iso: output will be in this color.

==> vsphere-iso: Creating VM...
==> vsphere-iso: Customizing hardware...
==> vsphere-iso: Mounting ISO images...
==> vsphere-iso: Creating floppy disk...
    vsphere-iso: Copying files flatly from floppy_files
    vsphere-iso: Copying file: ./preseed.cfg
    vsphere-iso: Done copying files from floppy_files
    vsphere-iso: Collecting paths from floppy_dirs
    vsphere-iso: Resulting paths from floppy_dirs : []
    vsphere-iso: Done copying paths from floppy_dirs
==> vsphere-iso: Uploading created floppy image
==> vsphere-iso: Adding generated Floppy...
==> vsphere-iso: Set boot order...
==> vsphere-iso: Power on VM...
==> vsphere-iso: Waiting 10s for boot...
==> vsphere-iso: Typing boot command...
==> vsphere-iso: Waiting for IP...
==> vsphere-iso: IP address: 172.16.1.72
==> vsphere-iso: Using ssh communicator to connect: 172.16.1.72
==> vsphere-iso: Waiting for SSH to become available...
==> vsphere-iso: Connected to SSH!
==> vsphere-iso: Provisioning with shell script: /tmp/packer-shell338854810
    vsphere-iso: Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease
    vsphere-iso: Hit:2 http://us.archive.ubuntu.com/ubuntu bionic InRelease
    vsphere-iso: Hit:3 http://us.archive.ubuntu.com/ubuntu bionic-updates InRelease
    vsphere-iso: Hit:4 http://us.archive.ubuntu.com/ubuntu bionic-backports InRelease
    vsphere-iso: Reading package lists...
    vsphere-iso: Reading package lists...
    vsphere-iso: Building dependency tree...
    vsphere-iso: Reading state information...
    vsphere-iso: Calculating upgrade...
    vsphere-iso: The following packages have been kept back:
    vsphere-iso:   linux-generic linux-headers-generic linux-image-generic
    vsphere-iso: The following packages will be upgraded:
    vsphere-iso:   dmidecode gcc-8-base libdrm-common libdrm2 libgcc1 libicu60 libnss-systemd
    vsphere-iso:   libpam-systemd libsqlite3-0 libstdc++6 libsystemd0 libudev1 libxml2
    vsphere-iso:   linux-firmware rsync systemd systemd-sysv udev
    vsphere-iso: 18 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
    vsphere-iso: Need to get 89.7 MB of archives.
    vsphere-iso: After this operation, 572 kB of additional disk space will be used.
    vsphere-iso: Get:1 http://us.archive.ubuntu.com/ubuntu bionic-updates/main amd64 perl-modules-5.26 all 5.26.1-6ubuntu0.3 [2,763 kB]

[Omitted some output in order to save space. Apt is downloading new packages]

[Omitted some output in order to save space. Apt is unpacking packages]

[Omitted some output in order to save space. Apt is "Setting up... all upgraded packages]

    vsphere-iso: Processing triggers for systemd (237-3ubuntu10.39) ...
    vsphere-iso: Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
    vsphere-iso: Processing triggers for dbus (1.12.2-1ubuntu1.1) ...
    vsphere-iso: Processing triggers for mime-support (3.60ubuntu1) ...
    vsphere-iso: Processing triggers for ureadahead (0.100.0-21) ...
    vsphere-iso: Processing triggers for install-info (6.5.0.dfsg.1-2) ...
    vsphere-iso: Processing triggers for libc-bin (2.27-3ubuntu1) ...
    vsphere-iso: Template Build Complete
==> vsphere-iso: Shut down VM...
==> vsphere-iso: Deleting Floppy drives...
==> vsphere-iso: Deleting Floppy image...
==> vsphere-iso: Eject CD-ROM drives...
==> vsphere-iso: Convert VM into template...
Build 'vsphere-iso' finished.

==> Builds finished. The artifacts of successful builds are:
--> vsphere-iso: Ubuntu18

The post VMware Template Management using Hashicorp Packer appeared first on VirtJunkie.

I’ve been spending a lot of time lately thinking about how organizations can provide services can operationalize the principles behind Infrastructure as Code in a way that’s accessible. In my experience, if a tool isn’t easy to use, it won’t be used. This is one of the biggest barriers to organizations (and companies that provide services, like mine) benefiting from IaC. It’s my intention that this script will achieve a high level of usability and will also generate and consume reusable code for future use.

Background

In order for this to fit my definition of infrastructure as code, we should be able to define all details of the vCenter in a structured format. Additionally, we should not assume that the vCenter installer is local, and should pull it from an external location if we need it to. While the vcsa-deploy utility can handle the installation of the vCenter, I’ve found most people don’t like it.

With this script, I allow a VI admin to fill out all relevant details of the vCenter in a spreadsheet. The script will then access the spreadsheet directly, and generate the JSON file used by vcsa-deploy. As a bonus, it will call the vcsa-deploy utility and kick off the actual deployment.

Demo

#Github: https://github.com/jonhowe/Virtjunkie.com/tree/master/VCSA-AutoDeploy
#Blog Post: http://www.virtjunkie.com/deploy-vcsa-via-powershell/

<#
.SYNOPSIS
Install the VMware vCenter Server Appliance based on input from a Microsoft Excel File

.DESCRIPTION
This script will install the VMware vCenter Server Appliance based on input from a Microsoft Excel File.

.PARAMETER ForceDownload
Switch parameter that will force a (re) download and extract of the ISO

.PARAMETER URI
This is the URL of the vCenter Installer ISO

.PARAMETER Workspace
Absolute path to the directory that will be used as a workspace. The vCenter ISO will be downloaded there, and will be
extracted there as well.

.PARAMETER DeploymentTarget
Location that the new VCSA will be deployed to.
Valid options are: ESXi, vCenter

.PARAMETER ExcelFile
Absolute path to the Excel file that contains the parameters required for vCenter to be built

.PARAMETER 7z
Absolute path path to the 7z binary
#>
#Requires -modules Microsoft.powershell.archive,importexcel
[CmdletBinding()]
param (
    [switch]$ForceDownload
    ,
    $URI = "http://jondesktop.home.lan:8123/files/VMware/VSMRepo/dlg_VC67U3B/VMware-VCSA-all-6.7.0-15132721.iso"
    ,
    [Parameter(Mandatory=$true)]
    [ValidateScript( {
            if ( -Not ($_ | Test-Path) ) {
                throw "File or folder does not exist"
            }
            return $true
        })]
    $Workspace
    ,
    [validateset("vCenter", "ESXi")]
    $deploymentTarget
    ,
    [ValidateScript( { Test-Path $_ })]
    [string]
    $ExcelFile = "/home/jhowe/git/Personal/AutomatedLab/JonAutoLab/HostDetails.xlsx"
    ,
    [Parameter(Mandatory=$true)]
    [ValidateScript( { Test-Path $_ })]
    [string]
    $7z = '/usr/bin/7z'
)
$ErrorActionPreference = 'Stop'

#region Download and Extract the ISO
$VCSA_Installer_Archive = $workspace + ($uri.Split('/')[-1])
$VCSA_Extracted_Directory = ($VCSA_Installer_Archive.Split(".iso")[0])
$VCSA_CLI_Installer_Path = ($VCSA_Extracted_Directory + "/vcsa-cli-installer/")

#Download the VCSA only if it doesn't already exist
if ((Test-Path $VCSA_Installer_Archive) -eq $false -or $forcedownload) {
    $filesize = (Invoke-WebRequest $uri -Method Head).Headers.'Content-Length'
    [int]$WebServerFileSize = ([convert]::ToInt64($filesize, 10)) / 1024

    Write-Output "Downloading VCSA Installer"
    $wc = New-Object System.Net.WebClient
    $wc.DownloadFileAsync($uri, $VCSA_Installer_Archive)

    #The following loop will print the progressof the download every 15 seconds
    $incomplete = $true
    while ($incomplete) {
        $CurrentSize = ((Get-Item $VCSA_Installer_Archive).length) / 1024
    
        if ($CurrentSize -eq $WebServerFileSize) {
            $incomplete = $false
        }
        else {
            Write-Output "File Download Size is $([math]::round($CurrentSize,0)) / $($WebServerFileSize) ($(($CurrentSize / $WebServerFileSize).ToString("P") )%)"
            Start-Sleep -Seconds 15
        }
    }
}
else {
    Write-Output "VCSA Installer already exists"    
}

#Extract the VCSA only if the directory it should be extracted to doesn't already exist
if ((Test-Path ($VCSA_Extracted_Directory)) -eq $false -or $forcedownload) {
    Write-Output "Unzipping VCSA"
    #Expand-Archive -LiteralPath $VCSA_Installer_Archive -destinationpath $workspace

    Write-Output "Extracting the ISO now..."
    $arguments = "x -bb0 -bd -y -o$($VCSA_Extracted_Directory) $($VCSA_Installer_Archive)"
    Write-Verbose -Message ($7z + " " + $arguments)

    $pinfo = New-Object System.Diagnostics.ProcessStartInfo
    $pinfo.FileName = $7z
    $pinfo.RedirectStandardError = $true
    $pinfo.RedirectStandardOutput = $true
    $pinfo.UseShellExecute = $false
    $pinfo.Arguments = $arguments
    $p = New-Object System.Diagnostics.Process
    $p.StartInfo = $pinfo
    $p.Start() | Out-Null
    $p.WaitForExit()
    $7z_stdout = $p.StandardOutput.ReadToEnd()
    $7z_stderr = $p.StandardError.ReadToEnd()

    $7z_stdout | out-file -Path ($workspace + "7z-stdout.txt")
    $7z_stderr | out-file -Path ($workspace + "7z-stderr.txt")

    if ($p.ExitCode -ne 0) {
        Write-Output "There was an error..."
        Write-Output "Process finished with exit code $($p.ExitCode)"
        Write-Output "Log data written to: $($workspace)7z-stdout.txt and $($workspace)7z-stderr.txt"
        $7z_stderr
    }
    else {
        Write-Output "Extraction completed successfully."
    }
}
else {
    Write-Output "VCSA has already been unzipped"
}
#endregion Download and Extract the ISO

#region import excel file into sections and validate input
$targettype = Import-Excel $excelfile -WorksheetName vCenter -StartRow 1 -EndRow 2 -StartColumn 1 -EndColumn 1
$targetinfo = Import-Excel $excelfile -WorksheetName vCenter -StartRow 1 -EndRow 2 -StartColumn 2 -EndColumn 8
$applianceinfo = Import-Excel $excelfile  -WorksheetName vCenter -StartRow 4 -EndRow 5 -StartColumn 1 -EndColumn 3
$networkinfo = Import-Excel $excelfile  -WorksheetName vCenter -StartRow 7 -EndRow 8 -StartColumn 1 -EndColumn 7
$os = Import-Excel $excelfile  -WorksheetName vCenter -StartRow 10 -EndRow 11 -StartColumn 1 -EndColumn 3 
$sso_ciep = Import-Excel $excelfile  -WorksheetName vCenter -StartRow 13 -EndRow 14 -StartColumn 1 -EndColumn 3

#Ensure the password specified has the appropriate complexity based on vCenter's requirements
$input = ($os."vCenter Root Password")
if (($input -cmatch '[a-z]') `
        -and ($input -match '\d') `
        -and ($input.length -ge 8) `
        -and ($input -match '!|@|#|%|^|&|$')) {
    Write-Output "Password complexity is sufficient. Continuing."
}
else {
    Write-Output "$input does not meet complexity requirements. Password must be at least 8 characters, include one numeric, and one special character."
    exit
}

$deploymentTarget = $targettype.'vCenter or ESXi'

#endregion import excel file into sections and validate input

#region Detect the OS (Windows, Linux, OSX)
if ($IsWindows -or $ENV:OS) {
    $installer_dir = ($VCSA_CLI_Installer_Path + "win32/")
    $installer = ($installer_dir + "vcsa-deploy.exe")
} 
elseif ($IsLinux) {
    $installer_dir = ($VCSA_CLI_Installer_Path + "lin64/")
    $installer = ($installer_dir + "vcsa-deploy")
    chmod -R +x $workspace
}
elseif ($IsMacOS) {
    $installer_dir = ($VCSA_CLI_Installer_Path + "mac/")
    $installer = ($installer_dir + "vcsa-deploy")
}
#endregion Detect the OS (Windows, Linux, OSX)

#region Create and format the JSON

#Detect whether we are deploying to vCenter or ESXi
switch ($deploymentTarget) {
    'vCenter' {
        Write-Output "Deploying to vCenter"
        $target = [ordered] @{
            vc = [ordered]@{
                __comments         = @("'datacenter' must end with a datacenter name, and only with a datacenter name. ",
                    "'target' must end with an ESXi hostname, a cluster name, or a resource pool name. ",
                    "The item 'Resources' must precede the resource pool name. ",
                    "All names are case-sensitive. ",
                    "For details and examples, refer to template help, i.e. vcsa-deploy {install|upgrade|migrate} --template-help")
                hostname           = $targetinfo."Target Host or vCenter"
                username           = $targetinfo."Username"
                password           = $targetinfo."Password"
                deployment_network = $targetinfo."PortGroup"

                <#
                datacenter         = @("Folder 1 (parent of Folder 2)",
                                        "Folder 2 (parent of Your Datacenter)",
                                        "Your Datacenter")
                #>
                datacenter         = $targetinfo."Datacenter (vCenter Only)"
                datastore          = $targetinfo."Datastore"
                <#
                target             = @( "Folder A (parent of Folder B)",
                                        "Folder B (parent of Your ESXi Host, or Cluster)",
                                        "Your ESXi Host, or Cluster")
                #>
                target             = $targetinfo."Cluster Name (vCenter Only)"
            }
        }
    }
    'ESXi' {
        Write-Output "Deploying to ESXi"
        $target = [ordered] @{
            esxi = [ordered]@{
                hostname           = $targetinfo."Target Host or vCenter"
                username           = $targetinfo."Username"
                password           = $targetinfo."Password"
                deployment_network = $targetinfo."PortGroup"
                datastore          = $targetinfo."Datastore"
            }
        }
    }
    Default {
        Write-Error "Invalid Deployment Target Specified. Exiting."
        exit
    }
}

<#
I'm aware that others have imported the json file provided by the installer...
I don't have a great reason that I didn't do that.. maybe I'm just different :-)
Either way - we are simply generating a hashtable here, and then writing it to a file.
#>
$common_properties = @{
    appliance = [ordered]@{
        __comments        = @("You must provide the 'deployment_option' key with a value, which will affect the VCSA's configuration parameters, such as the VCSA's number of vCPUs, the memory size, the storage size, and the maximum numbers of ESXi hosts and VMs which can be managed. For a list of acceptable values, run the supported deployment sizes help, i.e. vcsa-deploy --supported-deployment-sizes")
        thin_disk_mode    = $applianceinfo."Thin Disk?"
        deployment_option = $applianceinfo."vCenter Size"
        name              = $applianceinfo."VM Name"
    }
    network   = [ordered]@{
        ip_family   = $networkinfo."IP v4 or v6"
        mode        = $networkinfo."IP Allocation Mode"
        ip          = $networkinfo."IP Address"
        dns_servers = @($networkinfo.'DNS Server List').split(",")
        prefix      = ($networkinfo."CIDR Block").tostring()
        gateway     = $networkinfo."Default Gateway"
        system_name = $networkinfo."vCenter Hostname"
    }
    os        = [ordered]@{
        password    = $os."vCenter Root Password"
        ntp_servers = $os."NTP Servers"
        ssh_enable  = $os."Enable SSH By Default?"
    }
    sso       = [ordered]@{
        password    = $sso_ciep."SSO Password"
        domain_name = $sso_ciep."SSO Domain Name"
    }
}

$combined = [ordered]@{
    __version  = "2.13.0"
    __comments = "Sample template to deploy a vCenter Server Appliance with an embedded Platform Services Controller on an ESXi host."
    new_vcsa   = $target + $common_properties
    ceip       = [ordered]@{
        description = @{
            __comments = @("++++VMware Customer Experience Improvement Program (CEIP)++++",
                "VMware's Customer Experience Improvement Program (CEIP) ",
                "provides VMware with information that enables VMware to ",
                "improve its products and services, to fix problems, ",
                "and to advise you on how best to deploy and use our ",
                "products. As part of CEIP, VMware collects technical ",
                "information about your organization's use of VMware ",
                "products and services on a regular basis in association ",
                "with your organization's VMware license key(s). This ",
                "information does not personally identify any individual. ",
                "",
                "Additional information regarding the data collected ",
                "through CEIP and the purposes for which it is used by ",
                "VMware is set forth in the Trust & Assurance Center at ",
                "http://www.vmware.com/trustvmware/ceip.html . If you ",
                "prefer not to participate in VMware's CEIP for this ",
                "product, you should disable CEIP by setting ",
                "'ceip_enabled': false. You may join or leave VMware's ",
                "CEIP for this product at any time. Please confirm your ",
                "acknowledgement by passing in the parameter ",
                "--acknowledge-ceip in the command line.",
                "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++")       
        }
        
        settings    = @{ 
            #TODO I believe an extra member is needed if this is enabled
            ceip_enabled = $sso_ciep."Enable CIEP?"
        }
    }
}

$json = $combined | ConvertTo-Json -Depth 99
$fixtrue = $json.replace("`"true`"", "true")
$FinalJSON = $fixtrue.replace("`"false`"", "false")
#endregion Create and format the JSON

#region Write the json file to disk
switch ($deploymentTarget) {
    'vCenter' {  
        $FinalConfigFile = $workspace + "$(($FinalJSON | convertfrom-json).new_vcsa.vc.hostname).json"
    }
    'ESXi' {  
        $FinalConfigFile = $workspace + "$(($FinalJSON | convertfrom-json).new_vcsa.esxi.hostname).json"
    }
    Default { }
}
$FinalJSON | out-file -FilePath $FinalConfigFile
Write-Output ("JSON file written to: " + $FinalConfigFile)
#endregion Write the json file to disk

#region Start the install
Write-Output "Beginning install now.. this will take a few minutes. Please wait."
$arguments = "install --accept-eula --no-ssl-certificate-verification --log-dir $($VCSA_CLI_Installer_Path) $($FinalConfigFile)"

$pinfo = New-Object System.Diagnostics.ProcessStartInfo
$pinfo.FileName = $installer
$pinfo.RedirectStandardError = $true
$pinfo.RedirectStandardOutput = $true
$pinfo.UseShellExecute = $false
$pinfo.Arguments = $arguments
$p = New-Object System.Diagnostics.Process
$p.StartInfo = $pinfo
$p.Start() | Out-Null
$p.WaitForExit()
$stdout = $p.StandardOutput.ReadToEnd()
$stderr = $p.StandardError.ReadToEnd()

$stdout | out-file -Path ($VCSA_CLI_Installer_Path + "stdout.txt")
$stderr | out-file -Path ($VCSA_CLI_Installer_Path + "stderr.txt")
#endregion Start the install

if ($p.ExitCode -ne 0) {
    Write-Output "There was an error..."
    Write-Output "Process finished with exit code $($p.ExitCode)"
    Write-Output "Log data written to: $($VCSA_CLI_Installer_Path)stdout.txt and $($VCSA_CLI_Installer_Path)stderr.txt"
    $stderr
}
else {
    Write-Output "Install completed successfully."
}

Thanks for viewing,
Jon

The post Deploy VCSA via PowerShell appeared first on VirtJunkie.

It’s the network team’s fault…

The Scenario

Imagine you have an environment that has hundreds of VLANs. When you build a new cluster, there’s a real possibility that one host out of many is mis-configured on just a single port group or VLAN. It’s a pretty simple issue, but it’s like finding a needle in a haystack. Because of that, it’s incredibly difficult to troubleshoot, so I wrote a script to run through every ESXi host, and test connectivity on each port group.

How do we achieve this?

To recap, the goal here is to test network connectivity on all VLANs on all ESXi hosts in a cluster. The best way to do this is by creating a VMKernel port on each Port Group, and pinging a separate IP on the network from that VMKernel port. Once the ping results are recorded, the VMKernel port is deleted. Rinse and repeat for every VLAN on every ESXi host. The script will pull all info required for creating the VMKernel ports from a CSV file.

Example Output

CSV File Details

The CSV file will contain the following fields: IP, GW, PG, NetMask, VLAN. Each row in the CSV will contain deails for a single VMKernel port. Here’s an example:

In the example above, three VMKernel ports will be created on every ESXi host in the cluster, and a ping attempt will occur to the IP listed in the “gw” column from the entry in the “IP” column.

The Script

Now to the fun part, and why you’re actually here. First and foremost, you can see a copy of GitHub Repository.

To run this script, you’ll need to pass variables as parameters or modify the default values. For example, you can run the script like this:

PS> Test-ClusterNetworkConnectivity -vcenterserver "myvcenterserver.local" -vswitch "distributed switch name" -clustername "my special cluster" -csvpath c:\path\to\my.csv -verbose

• vCenter Server (-vcenterserver)
• The VDS in use (-vswitch)
• The desired cluster you want to focus on (-clustername)
• The CSV that contains info for the VMKernel ports (-csvpath)
• If you want verbose output printed to the screen (-verbose:$true/$false)

Code Overview (and code)

Lines 1-17
Documentation and parameters

Lines 18-30
Set up the environment, connect to vCenter, import the CSV, get the list of ESXi hosts to work with

Lines 31–33
Begin loop that goes through all ESXi hosts

Lines 34-36
Begin loop for each VMKernel port

Lines 37-50
Create splatting for variables
Instantiate object

Lines 51-70
Create the VMKernel adapter, test the ping, optionally print verbose logging

Lines 71-80
Add results to an array

Lines 81-82
Remove the VMKernel port

Line 85-97
This block exists to handle an error with creating the VMKernel port.

Line 100
Add the results of the ping test to an array

Line 105-107
Output the array of results and disconnect from the vcenter server

<#
.LINK http://www.virtjunkie.com/vsphere-test-cluster-connectivity/
.LINK https://github.com/jonhowe/Virtjunkie.com/blob/master/Test-ClusterNetworkConnectivity.ps1
#>
param(
	$vcenterserver = "vcenter67.pcm.net",
	$vswitch = "UCS-vDS001",
	$clustername = "UCS Cluster",
	$verbose = $false,
	$csvpath = "C:\powershell\PCMlab_test_csv.csv"
)

<#
***Fields in the CSV***
IP, GW, PG, netmask, vlan
#>

#1.) connect to the vcenter
$credential = get-credential
$conn = connect-viserver -server $vcenterserver -credential $credential | out-null

#2.) Get a list of all VMhosts attached to the specified Switch and cluster
$vmhosts = get-vdswitch -name $vswitch | get-vmhost | Where-Object { $_.parent.name -eq $clustername }

#3.) Import the CSV file that has the IP, Gateway, Port Group, NetMask, and vLAN
$ALL_VMK_CSV_Rows = import-csv -path $csvpath

#Create array that stores results
$RS = @()

#3a.) Loop through each VMhost one at a time
foreach ($vmhost in $vmhosts)
{
	#Loop through each VMkernel in the CSV one at a time
	foreach ($ONE_VMK_CSV_ROW in $ALL_VMK_CSV_Rows)
	{
		#Splatting for parameters for new-vmhostnetworkadapter
		$NewVMKParams = @{
			VMHost = $vmhost
			PortGroup = $ONE_VMK_CSV_ROW.pg
			VirtualSwitch = $vswitch
			IP = $ONE_VMK_CSV_ROW.ip
			SubnetMask = $ONE_VMK_CSV_ROW.netmask
		}
		#Optional logging - disabled by default
		Write-Verbose -vb:$true -Message ($NewVMKParams | Out-String)
		
		#create new object to contain results
		$PingResults = New-Object System.Object

		#Test to see if the vmkernel port is created successfully.. if so, continue to test connectivity
		if ($vmkernel = new-vmhostnetworkadapter @NewVMKParams -ErrorAction SilentlyContinue)
		{
			#Get the newly created VMKernel adapter
			$vmkobj = Get-VMHostNetworkAdapter -VMHost $vmhost -VirtualSwitch (Get-VDSwitch -Name $vswitch) -VMKernel | 
				Where-Object { $_.devicename -eq $vmkernel.devicename }
		
			#Use ESXCli to emulate this command: vmkping -I [newly created vmkernel IP] TO: [Gateway IP Specified in the CSV]
			$esxcli = Get-EsxCli -VMHost $vmhost -V2
			$params = $esxcli.network.diag.ping.CreateArgs()
			$params.host = $ONE_VMK_CSV_ROW.GW
			$params.interface = $vmkobj.devicename
			#the $res variable contains the results of the ping.. if you want, you can see all of the result info by printing $res.summary
			$res = $esxcli.network.diag.ping.Invoke($params)
			
			#optional logging - disabled by default
			write-verbose -vb:$verbose -Message ($res.summary | Out-String)
			$output = ("[$($vmhost.name)]: $($res.summary.Recieved / $res.summary.transmitted * 100)% success. Source: $($ONE_VMK_CSV_ROW.ip) Target: $($res.summary.hostaddr) || PG $($ONE_VMK_CSV_ROW.pg) (VLAN: $($ONE_VMK_CSV_ROW.vlan))")
			Write-Verbose -vb:$verbose -Message $output

			#Add properties to the object
			$PingResults | Add-Member -type NoteProperty -name Cluster -Value $clustername        
			$PingResults | Add-Member -type NoteProperty -name VMHost -Value $vmhost.name
			$PingResults | Add-Member -type NoteProperty -name SourceIP -Value $ONE_VMK_CSV_ROW.ip
			$PingResults | Add-Member -type NoteProperty -name TargetIP -Value $res.summary.hostaddr
			$PingResults | Add-Member -type NoteProperty -name PortGroup -Value $ONE_VMK_CSV_ROW.pg
			$PingResults | Add-Member -type NoteProperty -name VLAN -Value $ONE_VMK_CSV_ROW.vlan
			$PingResults | Add-Member -type NoteProperty -name PercentSuccess -Value ($res.summary.Recieved / $res.summary.transmitted * 100)
			$PingResults | Add-Member -type NoteProperty -name vSwitch -Value $vswitch
			
			#3d.) Remove vmkernel port
			remove-vmhostnetworkadapter -nic $vmkobj -confirm:$false
		}
		#if the vmkernel adapter is not created successfully, report the failure
		Else
		{
			#Optional Logging - disabled by default
			$output = ("[$($vmhost.name)]: ERROR || PG $($ONE_VMK_CSV_ROW.pg) (VLAN: $($ONE_VMK_CSV_ROW.vlan)) does not exist on $($vswitch)")
			write-verbose -verbose:$verbose $output

			#Add properties to the object
			$PingResults | Add-Member -type NoteProperty -name Cluster -Value $clustername
			$PingResults | Add-Member -type NoteProperty -name VMHost -Value $vmhost.name
			$PingResults | Add-Member -type NoteProperty -name SourceIP -Value "ERROR"
			$PingResults | Add-Member -type NoteProperty -name TargetIP -Value "ERROR"
			$PingResults | Add-Member -type NoteProperty -name PortGroup -Value $ONE_VMK_CSV_ROW.pg
			$PingResults | Add-Member -type NoteProperty -name VLAN -Value $ONE_VMK_CSV_ROW.vlan
			$PingResults | Add-Member -type NoteProperty -name vSwitch -Value $vswitch
			$PingResults | Add-Member -type NoteProperty -name PercentSuccess -Value "ERROR"
		}

		#add object to array
		$RS += $PingResults

	} # end foreach ($ONE_VMK_CSV_ROW in $ALL_VMK_CSV_Rows)
} # end foreach ($vmhost in $vmhosts)

Write-Output $RS
#3e.) Disconnect from all vCenters
disconnect-viserver -server * -confirm:$false

The post Validate Network Connectivity Across A VMware Cluster appeared first on VirtJunkie.

I created a PowerShell script to help with this. The end result is a PowerShell object that can be consumed in any way you like. In a future post, I’ll show how to take this output and create a new Excel spreadsheet.

The image below is an example of what the script below generates.

As an FYI, we’ll need to install the ImportExcel module in order to parse the excel file. If you want more information, check out the GitHub repository for ImportExcel.

To run this script, at a minimum, you’ll need to modify the $spreadsheetdir variable to point to a directory that contains your RVTools outputs. This is a parameter, so it can be done by adding the -spreadsheetdir "[path to my RVTools directory]" if you like, or just feel free to modify the script.

Lines 12-45
Define column headers to be pulled for each spreadsheet

Line 50
Get the contents of the spreadsheet directory

Line 54
Loop through each spreadsheet in the spreadsheet directory

Lines 58
Get a distinct list of all clusters in the particular RVTools file. This will allow us to present the results one cluster at a time

Lines 68-82
Import the properties defined in lines 10-44 from the vInfo, vHost, and vDisk worksheet for later consumption

Lines 84-101
Do some math/formatting/voodoo to convert the values from RVTools into something useful

Lines 63,64 and 103-110
Take values from RVTools and calculations (from lines 84-101) and put them into a custom PowerShell object.

You can find the script in the public GitHub repository I keep all of the scripts for this site in:
https://github.com/jonhowe/Virtjunkie.com

<#
.LINK http://www.virtjunkie.com/powershell-parse-rvtools/
.LINK https://github.com/jonhowe/Virtjunkie.com/blob/master/Parse-RVTools-Generic.ps1
#>
param(
    # Migrate Powered Off VMs?
    [switch]$IncludePoweredOffVMs,

    # Directory that contains RVTools Exports
    $spreadsheetdir = "/path/to/directory/",
    
    # Array that contains column headers to be gathered from the vHost worksheet
    $vHostProperties = @(
        'Host', 
        "CPU usage %",
        "Memory usage %",
        'model',
        "Service Tag",
        "Current EVC",
        "Max EVC",
        "# VMs",
        "# Memory",
        "# Cores",
        "HT Active",
        "vCPUs per Core"
    ),

    # Array that contains column headers to be gathered from the vInfo worksheet
    $vInfoProperties = @(
        'VM',
        'Template',
        "HW version",
        "Provisioned MB",
        "In Use MB",
        "OS according to the VMware Tools",
        'memory'
    ),

    # Array that contains column headers to be gathered from the vDisk worksheet
    $vDiskProperties = @(
        "vm",
        "Raw LUN ID",
        "Raw Comp. Mode",
        "Shared Bus"
    )

)

# Get all spreadsheets in the spreadsheet directory
$spreadsheets = $spreadsheetdir | Get-ChildItem

$rs = @()
# Loop through all spreadsheets in the spreadsheet dir
foreach ($sheet in $SpreadSheets)
{
    # Use the importexcel module to import the vCluster Worksheet
    # We do this as an easy way to get the 
    $vCluster = Import-Excel -Path $sheet -Sheet vCluster
    
    foreach ($cluster in $vCluster)
    {
        $ClusterDetails = New-Object System.Object
        $ClusterDetails | Add-Member -type NoteProperty -name vCenter -Value ($cluster."VI SDK Server")
        $ClusterDetails | Add-Member -type NoteProperty -name Cluster -Value $cluster.name 
       
        # Parse and store the properties defined in the $vInfoProperties array
        #   from the vInfo tab
        $vInfo = Import-Excel -Path $sheet -WorksheetName vInfo | 
            Where-Object { $_.Cluster -eq $cluster.name } | 
            Select-Object $vInfoProperties

        # Parse and store the properties defined in the $vHostProperties array
        #   from the vHost tab
        $vHost = Import-Excel -Path $sheet -WorksheetName vHost | 
            Where-Object { $_.Cluster -eq $cluster.name } | 
            Select-Object $vHostProperties

        # Parse and store the properties defined in the $vDiskProperties array
        #   from the vDisk tab
        $vDisk = Import-Excel -Path $sheet -WorksheetName vDisk | 
            Where-Object {$_.Cluster -eq $cluster.name } | 
            Select-Object $vDiskProperties

        # Calculate the sum of the column Provisioned MB from all rows in the vInfo tab, convert to GB, and round
        $TotalProvisionedGB = [math]::round(($vInfo | Measure-Object -Sum "Provisioned MB").Sum / 1024, 2)
        $ClusterDetails | Add-Member -type NoteProperty -name TotalProvisionedGB -Value $TotalProvisionedGB        

        # Calculate the sum of the column In Use MB from all rows in the vInfo tab, convert to GB, and round
        $TotalInUseGB = [math]::round(($vInfo | Measure-Object -Sum "In Use MB").Sum / 1024, 2)
        $ClusterDetails | Add-Member -type NoteProperty -name TotalInUseGB -Value $TotalInUseGB
        
        # Calculate average vCPU to PCPU, the total RAM in the cluster, and the average RAM Usage
        $vCPUtoPCPU = [math]::round(($vHost | Measure-Object -Average "vCPUs per Core").average, 2)
        $clusterRAM = [math]::round(($vHost | Measure-Object -Sum "# Memory").Sum / 1024, 2)
        $AvgClusterRamUsage = [math]::round(($vHost | Measure-Object -Average "Memory Usage %").average, 2)

        #Detect RDMs and VMs with Shared Bus based on info in the vDisk sheet
        $RDMs = ($vDisk | ? { $_."Raw LUN ID" -ne $null})
        $SharedBusVMs = ($vDisk | ? { $_."Shared Bus" -ne $null -and $_."Shared Bus" -ne "noSharing" -and $_."Shared Bus" -ne 0})

        $clusterCoreCount = [math]::round(($cluster | Measure-Object -Sum "NumCpuCores").sum,0)

        $ClusterDetails | Add-Member -type NoteProperty -name AvgClusterRamUsage -Value $AvgClusterRamUsage
        $ClusterDetails | Add-Member -type NoteProperty -name ClusterRamGB -Value $clusterRAM
        $ClusterDetails | Add-Member -type NoteProperty -name ClusterVMCount -Value $vInfo.count
        $ClusterDetails | Add-Member -type NoteProperty -name ClusterHostCount -Value ($vHost.count)
        $ClusterDetails | Add-Member -type NoteProperty -name TotalClusterCores -Value $clusterCoreCount
        $ClusterDetails | Add-Member -type NoteProperty -name vCPUtoPCPU -Value ($vCPUtoPCPU)
        $ClusterDetails | Add-Member -type NoteProperty -name RDMCount -Value ($RDMs.count)
        $ClusterDetails | Add-Member -type NoteProperty -name SharedBusVMs -Value ($SharedBusVMs.count)
        
        $rs += $ClusterDetails
    }
}

$rs

The post RVTools: Parse Multiple Outputs Using Powershell appeared first on VirtJunkie.

It should be pretty self explanatory.. just edit the variables in the parameter section and call the function.

Cheers!

Github: https://github.com/jonhowe/Virtjunkie.com/blob/master/Deploy-VM.ps1

function New-VirtJunkieLinkedClone {
    [CmdletBinding()]
    <#
    Github: https://github.com/jonhowe/Virtjunkie.com/blob/master/Deploy-VM.ps1
    Link: https://www.virtjunkie.com/?p=774
    #>
    param (
        #vCenter Info
        $vCenter            = 'vcsa.home.lab',
        $username           = 'administrator@vsphere.local',
        $password           = 'VMWare1!',
        $TemplateCustomSpec = 'WS16to19',

        #Parent VM Info
        $ParentVMName       = 'WS16-GUI',
        $SnapshotName       = "Base",

        #New VM Info
        $VMName             = "vra-iaas-2",
        $VMIP               = "192.168.86.214",
        $VMNetmask          = "255.255.255.0",
        $VMGateway          = "192.168.86.1",
        $VMDNS              = "192.168.86.232"
    )
    
    begin {
        $vc_conn = Connect-VIServer -Server $vCenter -User $username -Password $password
    }
    
    process {
        # Get the OS CustomizationSpec and clone
        $OSCusSpec = Get-OSCustomizationSpec -Name $TemplateCustomSpec | 
            New-OSCustomizationSpec -Name 'tempcustomspec' -Type NonPersistent

        #Update Spec with IP information
        Get-OSCustomizationNicMapping -OSCustomizationSpec $OSCusSpec |
            Set-OSCustomizationNicMapping -IPMode UseStaticIP `
                -IPAddress $VMIP `
                -SubnetMask $VMNetmask  `
                -DefaultGateway $VMGateway `
                -Dns $VMDNS

        $mySourceVM = Get-VM -Name $ParentVMName
        $myReferenceSnapshot = Get-Snapshot -VM $mySourceVM -Name $SnapshotName 
        $Cluster = Get-Cluster 'Cluster'
        $myDatastore = Get-Datastore -Name 'Shared'

        New-VM -Name $VMName -VM $mySourceVM -LinkedClone -ReferenceSnapshot $myReferenceSnapshot -ResourcePool $Cluster `
        -Datastore $myDatastore -OSCustomizationSpec $OSCusSpec
    }
    
    end {
        Get-OSCustomizationSpec -Name $OSCusSpec | Remove-OSCustomizationSpec -Confirm:$false
    }
}

The post PowerCli – Deploy Linked Clone with Static IP appeared first on VirtJunkie.

I ran into a challenge on this project where I needed to validate some details from the job that was submitted asynchronously. It was more difficult and annoying than I thought it would be, but I managed to get a function together that can get a bunch of information.

The script assumes a pre-existing connection to vCenter and accepts the parameters below.

1. Status_to_check – with a default value of “Initiate vMotion Receive operation”
2. vCenter – the FQDN of the vCenter server. I added this in case you are connected to multiple vCenter servers.

function Test-vCenterTasks
{
    param(
        $status_to_check = "Initiate vMotion receive operation",
        $vcenter
    )
    
    $tasks = get-task -Server $vcenter
    $vmotions=$tasks | ? { $_.name -eq $status_to_check}
    $rs = @()
    foreach ($vmotion in $vmotions)
    {
        $vminflight = $vmotion.ExtensionData.Info.EntityName
        $vmobj = get-vm $vminflight -Server $vcenter
        $cluster = $vmobj | Get-Cluster -Server $vcenter

        $obj = New-Object –typename PSObject
        $obj | Add-Member –membertype NoteProperty –name VM_Name –value $vmobj.name
        $obj | Add-Member –membertype NoteProperty –name VM_Cluster –value $cluster.name
        $obj | Add-Member –membertype NoteProperty –name StartTime –value $vmotion.StartTime
        $obj | Add-Member –membertype NoteProperty –name Progress –value $vmotion.PercentComplete
        $obj | Add-Member –membertype NoteProperty –name Description –value $vmotion.Description
        $rs += $obj
    }

    return $rs
}

Test-vCenterTasks -vcenter "vcenter-server-fqdn"

The post PowerCLI – Get All VM Details from get-task appeared first on VirtJunkie.