Virtual Machines on Digi Hunch

Azure Deets

Mon, 25 Oct 2021 23:13:52 -0400

Both Azure and AWS are leading players in public cloud. AWS developed a lot of SMB customer in technology. Azure attracted many enterprises from their on-prem customers. In addition to clienteles, their models to manage resources in the cloud are also different in several aspects. While this post is by no means a comprehensive comparison, it serves as a refresher on how Azure is different from AWS in cloud engineering.

Subscription and Resource Group

Both subscription and resource group are means to manage resources in group. Subscription is associated with a credit card and groups resource financially. Resource group groups resources logically. A subscription can have multiple resource groups, as illustrated here in the hierarchy map.

Azure CLI

There are two CLI tools: Azure PowerShell for PowerShell users, and Azure CLI for Linux users. The distinction is not clear, because PowerShell can also run on multiple platforms such as MacOS and Linux. On the other hand, Linux Bash can run on Windows (e.g. using WSL2). The different lies more in the command nomenclature. For example, to list subscriptions, the PowerShell Cmdlets reads:

Get-AzSubscription

The Azure CLI comes much cleaner:

az account list

The command and argument naming in Azure CLI align with Linux commands. I prefer Azure CLI for its succinctness. When looking up VM skus, we can use this command:

az vm list-skus -l eastus2

The return is a JSON document. In advanced use cases, we can filter the result by using jq utility. Alternatively, we can filter the result and groom the output with jmespath query, just like AWS CLI. For example, in eastus2 region, we look for instances that:

Has virtualMachines as resource Type
Has AcceleratedNetworking enabled
Has EncryptionAtHostSupported enabled
Has PremiumIO enabled

For each of the result, we print out:

number of vCPUs
Memory size

For this use case we will have to provide a JMESPath query as below, then output the result as a table:

az vm list-skus -l eastus2 --query "[?resourceType=='virtualMachines' && capabilities[?name=='AcceleratedNetworkingEnabled' && value=='True'] && capabilities[?name=='EncryptionAtHostSupported' && value=='True'] && capabilities[?name=='PremiumIO' && value=='True']].{Name:name,vCPUs:capabilities[?name=='vCPUs'].value|[0],MemoryGB:capabilities[?name=='MemoryGB'].value|[0]}" --output table

The result looks like this:

I use variations of the command above very often to find out the best instance for AKS nodes.

Infrastructure as Code

The native infrastructure as code option is ARM (Azure Resource Manager) template in JSON format. It is extremely wordy and perhaps why Azure later developed Bicep as the second generation of IaC tool. Terraform has a provider for Azure as well. For comparison among ARM, Terraform and Bicep, I have written a blog post for Slalom build covering more details.

Networking

Here is a great post comparing Azure Network with AWS.

At a high level, Azure Virtual Network (or VNet) is the equivalent of VPC in Amazon. Likewise, peering can be configured between VNets. As to subnet, Azure is different because there is no conceptual distinction between public subnet and private subnet. In AWS, public subnet is subnet attached with an Internet Gateway via a network route. So “private” or “public” are in terms of outbound traffic. On the other side, Azure does not distinguish between private or public subnet. Resources connected to a VNet have access out to the Internet by default. As to inbound traffic, you can make a VM available on Internet by giving it a public IP (same as AWS). You can make it available to other VNet, by configuring a service endpoint. Customers typically need custom routes to redirect outbound traffic (e.g. through firewall). In VPC, subnets are mapped to availability zones one-to-one, whereas in Azure, a subnet may traverse multiple availability zones.

With Azure, it is also important to understand difference between Azure service endpoint and Azure private endpoint:

Azure service endpoint: provides connectivity to Azure services over n optimized route over the Azure backbone network. Traffic will leave your VNet.
Azure private endpoint: a NIC that uses private IP from your VNet. This NIC connects you privately and securely to a service powered by Azure Private Link. By enabling a private endpoint, you’re bringing the service into your VNet.

On security group, we can associated a network security group with a network interface, or with a subnet. In contrast, in AWS, a security group can only be associated with an instance’s network interface.

Azure Bastion and Jump Box

There is a managed service called Azure Bastion. It is a SSH/RDP proxy fully managed as PaaS. However, its use case is virtual machines. It cannot be used to access other services. For example, if you create a private AKS cluster, then you need a command terminal to access the API server. This is not what Azure Bastion can do. Instead, you either need a virtual machine in the AKS network as jump box.

The alternative is an Azure Cloud Shell, which will require storage but can be configured to be placed inside of a V-Net. However, Azure Cloud Shell is not running inside of a full-fledged Linux operating system. You cannot install commands.

To create a bastion host, e.g. without public IP address, use the following CLI command:

az vm create -n MyBastion -g AutomationTest --image UbuntuLTS --subnet suitable-porpoise-node-subnet --vnet-name suitable-porpoise-vnet --ssh-key-values ~/.ssh/id_rsa.pub --authentication-type ssh

If the bastion host is needed with a public Ip, configure the network security group accordingly. The AZ CLI command above will create a VM, with an OS user named after the command line terminal user.

IAM

Azure AD is a managed identity service. Here is the difference between Active Directory and Azure AD.

Azure RBAC is a mechanism for authorization. Just like IAM policies, Azure RBAC enforces permissions using role assignment, which consists of:

security principal ( user, group, service principal, or managed idenity)
role definition: defines what actions is allowed and what is not allowed
scope: the object of the action

To enforce RBAC, one needs to create role assignment objects, each specifying principal, role, and scope.

Service Principle and Managed Identity

The two concepts may appear confusing. I find this article a great reference to demystify them. The takeaway is: service principle is the equivalent of service account in old Active Directory. Managed identity is a service principle automatically managed by a resource. Managed identity can be user assigned or system assigned.

Managed Identity is a “wrapper” around a service principal. It is automatically created and automatically rotated.

Azure DevOps

I think of Azure DevOps (ADO) of a managed pipeline implementation, with a repository (just like BitBucket), a board to manage tickets (similar to JIRA), Wiki (just like Confluence), Artifactory. The Pipelines is the part that’s similar to Jenkins. ADO calls a build pipeline a Pipeline, and a release/deployment pipeline a Release. A pipeline and a release are fundamentally the same but they are used in different ways. A pipeline’s input is usually the code repository, and the output is artifact. A release’s input is usually an artifact, and it connects to infrastructure in different environments.

ADO has its own ecosystem for plugins, managed under Visual Studio marketplace. Many extensions are open-source. If you are not happy with an extension, you can publish your own extension to market place.

Logging and Monitoring

Azure Monitor manages metrics, logs and alerts. To further analyze logs, create a log analytics workspaces, where you can run Kusto queries. You can create a workbook and embed Kusto queries into visual objects on the workbook.

Storage

Azure manages storage resources under storage account. The resource classes include Blob (object), File, Queue, Table and Disk (block). They have a few acronyms on redundancy levels.

Redundancy Option	Data Copy	Access level
LRS – Locally redundant	synchronously copy your data three times within the AZ in the primary region.	Write is acknowledged after three synchronous writes.
ZRS – Zone-redundant	synchronously copy your data across three AZs in the primary region. your data is still accessible for both read and write even if one AZ becomes unavailable.	Write is acknowledged after three synchronous writes. If an AZ becomes unavailable, Azure undertakes networking updates (e.g. DNS re-pointing). Application may perceive a blip where re-try policies may help.
GRS – Geo-redundant	LRS in primary region + asynchronously copy your data to a single AZ in the secondary region + LRS in secondary region	your data in the secondary region isn’t available for read or write access unless there is a failover to the secondary region. for read access to the secondary region, configure your storage account to use RA-GRS (read-access geo-redundant storage) If the primary region becomes unavailable, you can choose to fail over to the secondary region. After the failover has completed, the secondary region becomes the primary region, and you can again read and write data.
GZRS – Geo-zone-redundant	ZRS in primary region + asynchronously copy your data to a single AZ in the secondary region + LRS in secondary region	your data in the secondary region isn’t available for read or write access unless there is a failover to the secondary region. for read access to the secondary region, configure your storage account to use RA-GZRS (read-access geo-zone-redundant storage) If the primary region becomes unavailable, you can choose to fail over to the secondary region. After the failover has completed, the secondary region becomes the primary region, and you can again read and write data.

The disaster recovery and failover happens at storage account level.

Infrastructure deployment in Terraform 1/2

Wed, 11 Aug 2021 21:44:00 -0400

Terraform is an excellent Infrastructure-as-Code (IaC) tool based on Hashicorp Configuration Language (HCL). Compared to JSON or YAML based declarative templates (e.g. CloudFormation and ARM), HCL is more concise, thanks to the flexibility of HCL. On the other hand, HCL is not as flexible as general purpose languages. For that sake, I see HCL as semi-declarative IaC. This post is my notes about best practices with Terraform development, from the context of AWS, but also applies to other cloud platforms.

Complex Types

There are three primitive types (string, number and bool) that forms collection types and structural types. Here are some common ones:

list: element may repeat, and order is maintained:

["orange", "banana", "orange", "apple"]

set: elements are unique and unordered

["apple", "banana", "orange"]

tuple: each element has its own type

["a", 15, true]

object: defined by a schema with named attributes each with its own type

{
 name = "John"
 age = 52
}

list of object

[
 {
 alloc_id = "0b7271a3219bc1fc2"
 subnet_id = "0c02af76c2c3e46fa"
 },
 {
 alloc_id = "0440c334c48d4247f"
 subnet_id = "02652e69fa2a71de8"
 }
]

map of string

{
 property = "foo"
 attribute = "bar"
}

map of object

{
 objkey1 = {
 alloc_id = "0b7271a3219bc1fc2"
 subnet_id = "0c02af76c2c3e46fa"
 }
 objkey2 = {
 alloc_id = "0440c334c48d4247f"
 subnet_id = "02652e69fa2a71de8"
 }
}

Whenever applicable, Terraform converts types implicitly or explicitly. For example, when a list or tuple is converted to set, all elements are converted to string and duplicates are removed. Object and map are very similar. Map of string can be converted to object if the attributes comply with the schema. Additional attributes not in the schema are discarded.

HCL Types is similar to Python

Although being totally different beasts, the complex types between HCL and Python are similar, to the point I suspect the HCL design is influenced by Python. I summarize the similarities as such:

Python Types	list []	tuple ()	set {}	dict {}
Terraform Types	list []	tuple []	set []	map {}
ordered	Y You can access item by index	Y	N you cannot access an item by index or key; however you can loop over all itmes	N key-value pair that allows you to access item by key
changeable (mutable)	Y	N You cannot update, add or remove items	Y Add or remove only. no change to existing elements	Keys must remain unique or the values get overwritten; Values are mutable
allow duplicate elements	Y	Y	N	Keys must be unique; values don’t have to

In Python, list and tuple allow elements of mixed types but in IaC like Terraform we mostly don’t need mixed types. In Terraform, an object is a map without a defined type. In most situations, lists and tuples behave identically, as do maps and objects.

Another area of similarity is with the comprehension of list and dict/maps. In Python for example,

## Supposed you need to create a list:
lst=[]
for x in range(10):
 lst.append(x**2)

print(lst)

### that can be simplified as the following to create the list:
lst = [x**2 for x in range(10)]

## You can even add contidion
even_numbers=[num for num in range(10) if num%2==0]

## You can introduce function calls:
words = ["hello", "world", "python", "list", "comprehension"]
lengths = [len(word) for word in words]

## You can even combine two lists
lst1=[1,2,3,4]
lst2=['a','b','c','d']

pair=[[i,j] for i in lst1 for j in lst2]

print(pair)

## With dict, it's similar
evens={x:x**2 for x in range(10) if x%2==0}
print(evens)

In Terraform, we use similar techniques:

[for s in var.list : upper(s)] # build a tuple/list from a list

[for k, v in var.map : length(k) + length(v)] # build a list from a map

{for s in var.list : s => upper(s)} # build a map from a list

[for s in var.list : upper(s) if s != ""] # build a tuple/list from a list with condition

Note that the documentation of Terraform doesn’t explicitly call them out as comprehensions. However, it’s exactly the same idea as comprehensions in Python. Even the range() function exists both in Python and Terraform.

Modularization

Modules allows you to group related resources together. They can also be re-used and called by other modules. It is fairly straightforward to create a module:

put the resource declarations into a sub-directory
define input and output in the directory

However, the introduction of module complicates the directory structure and variable referencing, which is important to take into account before starting creating modules. This guideline has further discussion about when to create a module. I re-wrote the terraform templates in Orthweb project to leverage modularization wherever possible, but there is still some stand-alone resource (e.g. random_id) not belonging to any module.

To reference resources across modules, you need to import those resources (using data source) from within the module. There are a couple of ways. You may pass the argument of data source as input variable, or you can leverage the filter capability of data source. Let’s look at one example of each mechanism.

In the example below, we import a subnet by subnet id:

data "aws_subnet" "private_subnet" {
 id = var.private_subnet_id
}

In the example below, we import a subnet by filtering from all subnets in the VPC by tag:

data "aws_subnet" "private_subnet" {
 vpc_id = var.vpc_id
 filter {
 name = "tag:Name"
 values = ["Private"]
 }
}

There are pros and cons of each approach. A module with mechanism 1 is more transferrable across different environment, because the ID of subnet is explicitly provided. However, authors needs to manage those explicit variables with code. Mechanism 2 fetches target resources with filter. It depends on a well-implemented tagging policy in the resource farm.

The Terraform Registry (since 2017) contains a lot of pre-built modules for each backend platform (e.g. AWS). If you find any module that can be used in your project, the module repo can be referenced directly by Git repository URL. You should be aware of the risk of this practice though. Many platforms are keen to publish modules for their platform. Anyone can publish their own modules to the community as well.

You can quickly generate module documentation with terraform-docs.

Local Execution

Local execution is the basic workflow mode which is mostly seen with very small collaboration team. In this mode, the developer executes terraform binary (Terraform CLI) from their workstation (e.g. Laptop). The Terraform CLI converts code into API calls to interface cloud provider. The most frequently used commands (from terraform directory) are:

terraform init
terraform plan
terraform apply

The init command initializes the working directory. The plan command figures out the delta between code and infrastructure. It outlines the changes it is about to make. The apply command commits the change. The documentation of Terraform CLI commands is here.

Terraform keeps track of the infrastructure it manages in state file. This article explains the purpose of state. State management collaboration difficult with local execution because the state file by default is created in the working directory on user’s workstation. Although the state file can be configured to be stored in a shared location such as S3, it still requires a mechanism to lock the state in a multi-developer collaboration.

In large operations, the same code base in Terraform, is usually used to created several different sets of infrastructures, for example, in different geographic regions. So it is a 1-to-many relationship between the code repo and the infrastructure state. To further complicate things, each state might have been deployed using different revisions of the code. To overcome that challenge, Terraform introduced the concept of workspace, which is essentially an instance of state describing a particular group of infrastructure being managed by the same source code. When there are many workspaces, it becomes tricky to manage them with CLI commands.

State management is a major challenge that needs to be solved for team collaboration in local execution workflow. Each state must use the same revision of Terraform code. You can use Git in combination as a workaround to that limitation but the point is you cannot tie a workspace to a commit with the workspace commands.

In some enterprise environment, the execution is from a VM (e.g. ADO agent on-premise) without Internet access, which poses another challenge. First, we need to pre-load required providers manually. The enterprise needs a proxy solution to safely download packages from Hashicorp website. One good option is Nexus Repository, with both open-source and pro supports. It is a full-function artifactory repo that can host helm repo, apt repo, yum repo, etc. Second, we also need to configure Terraform so it picks up providers locally. Managing plugins without Internet access requires understanding of the order in which Terraform tries to load plugins during initialization.

Remote Execution

In remote execution, the code is executed in Terraform Enterprise or Terraform Cloud. Both are remote web servers. The difference is that Terraform Enterprise is self-hosted service, requiring IT specialist to install and maintain Terraform Enterprise. Terraform Cloud on the other hand, is a managed SaaS service. The pricing model includes a free plan for small number of users.

Terraform workspace configuration

In Terraform Enterprise or Cloud, the remote execution is organized in workspaces. You need to create an organization, and then create workspace under the organization in order to execute code. With each workflow, you can specify version control system (VCS) and subdirectory, to tell the workspace where to fetch Terraform code from. The workspace also allows you to define secrets and variables specific to the workspace. When you execute a workspace plan, the secrets and variables are passed from workspace to the execution logic.

Terraform Workspace Variable configuration

You will also need to design the Terraform code in a way to work seamlessly with the secrets and variables loaded from the workspace. The variable declaration in code should match the definition in workspace. There are already a number of variables that came in handy. Check out this guide.

Each execution is referred to as a “run”, with its own run id. A workspace involves may runs, which may succeed or fail. Each run pulls a specific commit of the source repository, and goes through stages such as plan, and apply. The UI from each run result list out the status of each result, in a very easy to read format.

The state data is persisted in the web server as they were generated. Therefore the collaborator do not need to worry about managing state with CLI tools. If there are files that you do not want picked up by the execution engine, their locations can be added to a file .terraformignore. Refer to this guide.

AWS profile

Local execution still has a lot of use cases in enterprises such as testing with temporary resources. A common challenge is authentication. As discussed, Terraform CLI picks up identity information from AWS CLI and authenticates its way into the backend to run API calls against. So AWS CLI must be configured correctly with the sufficient permission to provision resources. On the other hand, enterprises usually offload IAM to an identity store, such as AzureAD, Okta, etc. Putting those together, the pattern of authentication and authorization usually looks like this:

User logs on via SSO (e.g. SAML). The validation response gives a name of an IAM role.
Upon successful authentication, user takes the IAM role. The role does not have any capability, except for assuming a second IAM role.
The second IAM role (the functional role) grants user the permission to do its business.

The steps above, can be carried out in AWS console, or with AWS cli using assume-role command. However, when we put Terraform in the picture, it becomes a little involving because the credential information is updated whenever the functional role is assumed, and the assume-role command takes a pretty long argument.

To skip typing the long command every time, there are some handy tools, such as aws-azure-login. An even better tool that works with a variety of identity stores is saml2aws. The tool allows you to configure identity backend, assume the functional role, and update credential information in aws credential file, all with a single command. The AWS CLI configuration reads:

[default]
region = us-east-1
output = json
cli_history = enabled
cli_pager =
role_session_name = functional_operation

[profile function_user]
source_profile = default
role_session_name = functional_operation
role_arn = arn:aws:iam::9998887766:role/admin-access
region = us-east-1

In Terraform provider, we need to tell it to assume that role as well:

provider "aws" {
 region = "us-east-1"
 assume_role {
 role_arn = "arn:aws:iam::9998887766:role/admin-access"
 session_name = "terraform"
 }
}

This will ensure Terraform assumes appropriate role before doing its job.

AWS EC2 SSH Key Pair

RSA key authentication for SSH should be used for Linux Instances. When creating an EC2 instance, we give it our public key so we can then later authenticate through SSH. If the key is already stored in AWS, we just need to tell EC2 the name of the key, in the key_name property. If the code is likely to be executed from several different places by different users, then we can write the code so it picks up public key from user’s workstation (~/.ssh/id_rsa.pub). Here is an example:

variable "local_pubkey_file" {
 type = string
 default = "~/.ssh/id_rsa.pub"
}
data "local_file" "pubkey" {
 filename = pathexpand(var.local_pubkey_file)
}
resource "aws_key_pair" "user-pubkey" {
 key_name = "runner-pubkey"
 public_key = data.local_file.pubkey.content
}
resource "aws_instance" "bastion" {
 instance_type = "t2.micro"
 key_name = aws_key_pair.user-pubkey.key_name
 ......
}

For remote execution, we can even add an option to pass public key in as variable, to override the key file variable. For an example, check out my orthweb project.

To upload files to EC2 instance from Terraform execution environment, we can use the file provisioner with ssh as connection type.

Virtualization 2 of 4 – Graphics Computing

Sat, 01 Aug 2020 18:24:00 -0400

We covered hypervisor in previous post. In this article we focus on the virtualization of graphics computing resource.

GPU vs CPU

GPU is a specialized type of microprocessor primarily designed for quick image rendering. GPU appeared as a response to graphically intense applications that put a burden on the CPU and degrated computer performance. They became a way to offload those tasks from CPUs, but modern graphics processors are powerful enough to perform rapid mathematical calculations for many other purposes apart from rendering.

CPU consists of a few cores (up to 23) optimized for sequential serial processing, which is designed to maximize the performance of a single task within a job. GPU uses thousands of smaller and more efficient cores for massively parallel architecture aimed at handling multiple functions at the same time. Typical uses cases for GPUs, in addition to graphics display, includes Games, 3D visualization, Image processing, big data and deep machine learning.

Moving to virtualization world, the most primitive mechanism for graphics acceleration is Soft 3D, which is commonly used in virtual desktops, or DaaS (desktop as a service). The Software 3D renderer (Soft 3D) uses the Soft 3D graphics driver to provide support for software-accelerated 3D graphics without any physical GPUs being installed in the ESXi host. With respect to GPU in virtualized environment, VMware developed a few technologies.

vSGA (Virtual Shared Graphics Acceleration)

The physical GPUs in the server are virtualized and shared across multiple guest VMs. This option involves installing an Nvidia driver into the hypervisor itself, and each guest VM uses a proprietary VMware SVGA 3D driver that communicates with the Nvidia driver in ESX. The biggest limitation here is that these drivers only work with DirectX up to 9.0c, and OpenGL up to 2.1. This technology was introduced in early 2013 and is used in light workload for knowledge worker, such as PowerPoint, Visio and web browsing.

vSGA

vDGA (Virtual Dedicated Graphics Acceleration)

vDGA, also known as “GPU passthrough”. It provides each VM with unrestricted, fully dedicated access to one of the host’s GPUs. The hypervisor is drilling a direct hole in itself between the GPU and the guest. This technology allows you to present an internal PCI GPU directly to a VM guest. The device acts as if it were directly driven by the VM guest, and the guest detects the PCI device as if it were physically connected, using the “real” driver. There is no special drivers in the hypervisor. vDGA offers the highest level of performance for users with the most intensive graphics computing needs.

GPU passthrough

The main advantage to vDGA is that since the GPU is passed through to the guest and the guest uses regular Nvidia drivers, it fully supports everything the Nvidia driver can do natively. This enables all versions of DirectX, OpenGL and even CUDA. The downside is that vDGA is expensive, since you need one GPU per user. There is also a lack of vMotion support.

VMware added support for vDGA in late 2013. The target market is high-end users with intensive graphical applications (oil&gas, scientific simulations, CAD/CAM, etc

vGPU (Virtual GPU)

vGPU is also known as Virtual Shared Pass-Through Graphics Acceleration. This technology sites somewhere in between the two previously introduced, as an option to strike a balance between cost-effectiveness and resource-sharing. It is essentially vDGA but with multiple users per GPU, instead of one-to-one mapping. Like vDGA, with vGPU you install the real Nvidia driver in guest VMs, and the hypervisor passes the graphics commands directly to the hypervisor without any translation.

vGPU gives you all that plus the ability to share a GPU across up to 8 VMs. The idea of vGPU is that you get better performance than vSGA option, with a portion of cost when compared to vDGA. The use case for vGPU will be the higher-end knowledge workers who need real “GPU” access but don’t need full-on multi-thousand dollar graphics workstations.

VMware partners with Nvidia on vGPU development. Below is the use-case chart from previous VMware white paper:

The diagram below illustrates the architecture of virtual GPU (NVIDIA Grid):

high-level architecture of GRID vGPU

The best white paper about the three technologies and their use cases is on VMware website.

Identify Graphics driver

On Linux VM, we can simply use lspci to identify graphics driver.

[root@ghrender ~]# lspci | grep VGA
03:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. Integrated Matrox G200eW3 Graphics Controller (rev 04)
3b:00.0 VGA compatible controller: NVIDIA Corporation GP104GL [Quadro P5000] (rev a1)In the result, the far left column is specified domain, e.g. 3b:00.0

To display details on graphics card by specified domain (3b:00.0 for example) with memory information:

[root@ghrender ~]# lspci -v -s 3b:00.0
3b:00.0 VGA compatible controller: NVIDIA Corporation GP104GL [Quadro P5000] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: NVIDIA Corporation Device 11b2
	Flags: bus master, fast devsel, latency 0, IRQ 190, NUMA node 0
	Memory at ab000000 (32-bit, non-prefetchable) [size=16M]
	Memory at 382fe0000000 (64-bit, prefetchable) [size=256M]
	Memory at 382ff0000000 (64-bit, prefetchable) [size=32M]
	I/O ports at 6000 [size=128]
	[virtual] Expansion ROM at ac080000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Legacy Endpoint, MSI 00
	Capabilities: [100] Virtual Channel
	Capabilities: [250] Latency Tolerance Reporting
	Capabilities: [128] Power Budgeting <?>
	Capabilities: [420] Advanced Error Reporting
	Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Capabilities: [900] #19
	Kernel driver in use: nvidia
	Kernel modules: nouveau, nvidia_drm, nvidia

The lshw command can also identify onboard Intel/AMD or Nvidia dedicated GPU:

[root@ghrender ~]# lshw -C display
 *-display
 description: VGA compatible controller
 product: Integrated Matrox G200eW3 Graphics Controller
 vendor: Matrox Electronics Systems Ltd.
 physical id: 0
 bus info: pci@0000:03:00.0
 version: 04
 width: 32 bits
 clock: 66MHz
 capabilities: pm vga_controller bus_master cap_list rom
 configuration: driver=mgag200 latency=64 maxlatency=32 mingnt=16
 resources: irq:16 memory:91000000-91ffffff memory:92808000-9280bfff memory:92000000-927fffff
 *-display
 description: VGA compatible controller
 product: GP104GL [Quadro P5000]
 vendor: NVIDIA Corporation
 physical id: 0
 bus info: pci@0000:3b:00.0
 version: a1
 width: 64 bits
 clock: 33MHz
 capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
 configuration: driver=nvidia latency=0
 resources: iomemory:382f0-382ef iomemory:382f0-382ef irq:190 memory:ab000000-abffffff memory:382fe0000000-382fefffffff memory:382ff0000000-382ff1ffffff ioport:6000(size=128) memory:ac080000-ac0fffff

Virtualization 1 of 4 – Hypervisor

Mon, 27 Jul 2020 22:52:00 -0400

In broad terms, virtualization of computing resource is about isolation of resources, at different levels. There are five levels of virtualization:

Application level, such as JVM, .NET CLR
Library (user-level API) level
Operating system level, such as LXC, Docker, OpenVZ
Hardware abstraction layer (HAL) level, such as VMware, Xen, etc
Instruction set architecture (ISA) level

In my context I deal mostly with OS level and HAL (hardware abstraction layer) level of virtualization. In loose terms, the word containerization refers to OS level virtualization, while the word virtualization is exclusively reserved for HAL level virtualization, also referred to as hypervisor-based virtualization. This post will just focus on this family of technology and loosely refers to it as virtualization.

Virtualization technology evolved from on-premise data centre environment and now is the backbone of cloud computing. The challenges of IT operation in the era of virtualization involves managing VM sprawling, investigating performance issues, planning capacity and addressing storage I/O block. The idea of virtualization is sharing (thus isolating) resources for better utilization, leading to better return on investment. This posting is to cover only the very basics of virtualization.

Hypervisor

Hypervisor is the software layer which provides the capability to run multiple virtual machines on the same physical host. It is broken down into two categories:

Type I hypervisor (aka bare metal hypervisor): directly run on physical hardware. They control the hardware as well as manage the virtual machines. For example, Linux KVM, VMware ESXi, Xen and Microsoft Hyper-V
Type II hypervisor: runs as an application or service on top of the host operating system, which is installed on the bare metal. Guest operating system calls need to traverse via the host operating system stack to reach hardware resource. For example, Oracle Virtual Box, VMware Fusion and Linux Containers (LXC)

Hypervisor Types

Virtualization Techniques

The most primitive form of technology that can be arguably categorized under virtualization is hardware emulation, where a piece of (more accessible) hardware imitates another (less accessible). The architecture limits itself in functional testing only, and is not built for performance or production at all.

The original virtualization technology deals with CPU and memory virtualization. In this well-written whitepaper fromVMware, there are three CPU virtualization techniques introduced for x86 architecture.

The x86 architecture offers four levels of privilege known as Ring 0,1,2 and 3 to operating system and applications to manage access to the computer hardware. User-level applications typically run in Ring 3, the OS must execute its privileged instructions in Ring 0 since it needs to have direct access to memory and hardware. The two main challenges with virtualizing x86 architecture are:

A virtualization layer between hardware operating system who expects Ring 0 privilege;
Some instructions with different semantics when not executed in Ring 0 cannot be virtualized effectively. They need to be translated at runtime.

These challenges makes true virtualization of x86 architecture impossible and thus VMware developed three alternative technologies.

Full virtualization (using binary translation): virtual machine presents a complete simulation of the actual hardware environment so that an unmodified guest OS can run in isolation. The Guest OS is not aware that the underlying environment it is running on is virtualized, and issues hardware calls to communicate with (what it thinks as) hardware. The virtual processors have to understand guest CPU instruction, and reproduce the equivalent CPU instructions of the host machine. VMware’s technology to address this is called Binary Translation. This overhead makes true full virtualization difficult to achieve. In real life, a virtual environment that provides “enough representation of the underlying hardware” can be considered to provide full virtualization as long as it allows guest OS to run without modification. Full virtualization comes with a performance penalty.

Paravirtualization (aka OS assisted virtualization): refers to communication between the guest OS and the hypervisor to improve performance and efficiency. In this technology, guest OS is modified with an interface to host hardware to be able to communicate and operate seamlessly. Since the guest OS is modified, the VM does not need to be a complete simulation of the hardware. The modified guest OS knows it is running on a virtualized environment, and (vm driver) makes API calls (known as ‘hyper calls’) to the hypervisor. This allows para-virtualization technology to achieve performance closer to non-virtualized environment. However, since paravirtualization cannot support unmodified operating systems, its compatibility and portability is poor.

Hardware-Assisted Virtualization: hardware vendors such as Intel and AMD both have developed extensions (new features) to simplify virtualization techniques, for example, the introduction of privileged instructions with new CPU execution mode feature to allow hypervisor to run in a new root mode below ring 0. This removed the need for full virtualization and paravirtualization. With VMware originally as a promoter of full virtualization and Xen for paravirtualization, most virtualization technologies today utilizes hardware-assisted virtualization feature, for example, Linux KVM, VMware workstation, VMware fusion, Xen, VirtualBox, etc. Intel’s virtualization extension is VT-x. AMD’s counterpart is AMD-V technology.

To virtualize memory, another level of memory virtualization is required (similar to the virtual memory support in Linux). Hypervisor is responsible for mapping guest physical memory to the actual machine memory, and it uses shadow page tables to accelerate the mappings, usually at a performance cost.

Popular hypervisors

On the market there are a few popular hypervisor technologies. They are all type 1 hypervisors:

Xen is an open-source hypervisor project originally developed in Cambridge University, licensed under GPLv2. . Based on that, Citrix developed its commercial product XenServer, a bare-metal virtualization platform with enterprise-grade features for x86 and AMD environments. Oracle VM is another commercial implementation of Xen. The Xen project also supports many cloud platforms such as Openstack, Cloudstac, etc. Xen project supports paravirtualization (Xen-PV) as well as hardware-assisted virtualization (Xen-HVM) for virtualization of X86, IA64, ARM and other CPU architectures. The earlier versions does not support memory overcommit (aka “dynamic memory optimization”, “memory ballooning“, or as Citrix calls it “dynamic memory control, DMC”). This delivers better performance but also has higher budgetary requirement on hardware since there isn’t room for over-subscription.

Hyper-V is a Microsoft product. It executes in high CPU privilege (Microsoft calls it ring -1 which is equivalent to root mode as Intel calls it). On the guest VM, OS kernel and drivers run in ring 0, application rin in ring 3. This eliminates the need for binary translation. Hyper-V does not support memory overcommit either. Hyper-V is well integrated with Windows platform. It supports Linux as well although with some performance penalty.

Linux KVM (Kernel-based Virtual Machine) is a full open-source virtualization solution for GNU/Linux. What makes KVM a special hypervisor is that it uses a loadable kernel module kvm.ko that turns itself into a hypervisor and provides VMs with direct access to the hardware. So it is a type 1 hypervisor despite of the presence of Linux OS. KVM also contains a processor specific module, kvm-intel.ko or kvm-amd.ko. KVM leverages qemu to access devices. Because KVM runs as a process inside of Linux OS, KVM can use many existing feature in Linux kernel. Redhat has an enterprise solution based on KVM.

Xen vs KVM

VMware ESXi is VMware’s premium hypervisor product (not open-source) and is available for ~~free download~~, although the advanced features are not free. (Update no free download link anymore.) VMware vSphere is virtualization platform built on top of ESXi, including a whole family of virtualization products.

Market segments and players

Virtualization involves many market segments such as virtual desktop infrastructure (VDI, for desktop virtualization), server virtualization is the predominant domain in the virtualization of data centre environment. This effort led to Hyper-Converged Infrastructure (HCI) where almost all the traditional hardware resources are software-defined through the virtualization layer. The management of infrastructure is abstracted away from the physical hardware management. The three most fundamental areas in HCI are:

Server (compute) virtualization: the previous section covers the virtualization of memory and x86 CPU, which are the main focus on computing resource virtualization. Additionally, graphics computing resources can be virtualized today. Example products include: VMware vShpere (compute virtualization based on ESXi hypervisor).

Storage Virtualization: the technology to abstract physical data storage resource to make them appear as if they were a centralized resource. Storage virtualization takes place at three levels depending on the use case: block-level, file-level and object level. Example products include: VMWare vSAN (vSphere-native storage), HPE 3PAR (Tier-1 storage), EMC VxRail, PureStorage Flash Array (Tier 1), etc. Storage Virtualization enables Software-Defined Storage (SDS), the provisioning and management of data storage independent of the underlying hardware.

Network Virtualization: the technology to abstract network resources that were traditionally delivered in hardware to software. Network virtualization decouples network services from the underlying hardware management and allows virtual provisioning of an entire network. VLAN is a classic example of network virtualization. There are also various overlay technologies such as VXLAN, which provides an industry framework for overlaying virtualized layer 2 network over layer 3 network (used in Docker network) using an encapsulation mechanism and a control plane. Example products include: VMware NSX Data Center (L2-L7 network and security virtualization platform), Cisco ACI, Palo Alto Panorama. Network Virtualization enables Software-Defined Network (SDN), an approach to network management that enables dynamic, programmatically efficient network configuration in order to improve network performance and monitoring, making it more like cloud computing than traditional network management.

Delivery model

Virtualization allows managed service providers (MSPs) to deliver IT service in the following three models:

Iaas (Infrastructure as a Service): MSP delivers VM to customers.
PaaS (Platform as a Service): MSP delivers environments to customers (e.g. Database as a Service, managed RabbitMQ service, etc).
SaaS (Software as a Service): MSP delivers entire application for the customer.

IT service delivery models enabled by virtualization technology

Since virtualization is the backbone of cloud computing. This model is also referred to as cloud computing delivery model.

Virtualization and Containerization

These two concepts are similar and could be confusing to beginners. Both provide a mechanism to isolate computing resource for different applications, for the purpose of higher utilization of resource. The difference lies in how and where the isolation is made. Virtualization requires a guest operating system per VM (OS level isolation), whereas the container technology isolates application processes along with its runtime into a container (dependency level isolation), using some new Linux kernel features such as namespaces and cgroups. All containers make their system calls to the container engine on the host operating system. So they share a kernel on the same host. In this sense, container engine running on OS could be considered as type 2 hypervisor.

From VMs to containers

VMware is a major player in enterprise data centre virtualization, which is facing fierce competition from public and private cloud vendors. VMware also has its own private cloud services. Docker is the most popular container technology that conforms to the specifications of Open Container Initiative (OCI), a governance structure for industry standards around container formats and runtimes.

Virtualization and Cloud

Among public cloud vendors, AWS EC2 used Xen PV and Xen HVM in its earlier implementations. It has transitioned to AWS bare metal. The history is well summarized here. Microsoft Azure runs Azure Hypervisor as the native hypervisor in Azure Cloud Services platform. It is a customized version of Microsoft Hyper-V specifically for Azure platform. With GCP, Google Compute Engine (GCE) instance runs VMs on KVM as hypervisor. It can also enable nested virtualization.

The scope of cloud computing is evolving overtime. It originally only refers to a business model of offering IT services (in one of the three delivery models outlined above) based on virtualization technology. Therefore I cannot make comparison between a technology and a business model. Today, with public cloud vendor extending their offerings (with various managed services and platforms) and people’s misuse of the terms, the buzz-word “cloud” seems to suggest anything that is offered in public cloud service. The essence still remain the same where managed services and managed platforms are built on top of virtualized compute unit under the hood, which are driven by virtualization technologies.