Load Balancers on Digi Hunch

Kubernetes Platform as a Service and Red Hat OpenShift

Sun, 25 Jun 2023 11:10:15 -0400

The Three-layer model

Kubernetes is so complex that it becomes a buzz word itself. I categorize the related work into three layers: a cluster layer, a platform layer and an application layer, by their purposes. The three layers are illustrated as below:

Let’s examine each layer in this model and where the Kubernetes Platform as a Service fits in.

The Kubernetes Cluster Layer

At the bottom, the Kubernetes Cluster layer is the foundational layer. It focus on using self-hosted VMs or cloud resources to build a functional Kubernetes cluster and worker node groups. A functional cluster includes a highly available control plane, as well as scalable node groups that all communicate with the control plane. Cloud Service Providers like AWS and Azure provides managed Kubernetes service, which takes away the complexity (and flexibility as well) of managing control plane components such as etcd store and API server. The managed services also automatically provisions computing nodes and join them into the cluster. The cluster layer may also involve integration with of CNI and CSI, to ensure Pod-to-Pod communication and available storage classes.

Professionals working at this layer are infrastructure experts who understand networking, storage, as well as how to manage cloud resources or VMs, infrastructure as code. On a daily basis, they deal with VPCs/V-Nets, subnets, EBS/Azure Disk, File storage, EC2/Azure VMs, etc. When the team is doing a bad job at this layer, you might see symptoms like unresponsive cluster API, orphaned worker nodes, or kubectl failing to connect to cluster endpoint.

The tenants (applications) of the Kubernetes platform does not directly interact with this layer. If you decide to switch CSP vendor, this layer requires 100% re-engineering because the managed Kubernetes service by each CSP is different.

The Kubernetes Platform Layer

The Platform layer sits in the middle. When organization decides to adopt Kubernetes, they often underestimate the efforts required in this layer. This layer works on a functional cluster, without directly interacting with the underlying cloud resources. This layer involves any Kubernetes abstractions that do not creates tangible business value. Rather, this layer is an enabler. It allows the applications to deploy smoothly, evolve quickly, and more importantly, focus on the business.

Teams working on this layer needs to be Kubernetes experts. On a daily basis, they play with common CNCF toolings, such as Prometheus, ArgoCD, Istio, Cilium, Tekton, Open Policy Agent, etc. They are comfortable with Operators, Helm Charts, Ingress, etc. Inside of the Kubernetes cluster, they also manage the foundational services such as Event streaming (e.g. Kafka), PostgreSQL database (e.g. PostgreSQL), software-defined storage (e.g. Ceph), service mesh (e.g. Istio), Authentication (e.g. Keykloak) , etc. These services act as the infrastructure layer to the business workload. If the team is doing a bad job, you would see data loss with database, observability service not populating data, ingress does not process request, etc.

The tenants (application) share services in this layer. If you decide to switch CSP vendor. I estimate 80% of the work at this layer is portable, and 20% requires re-engineering. That is because each CSP offers different external resources, therefor the low level Kubernetes objects in this layer, such as storage classes, load balancers, supported CNIs are different. High level objects such as Kafka remains portable across platforms.

The Application Layer

The next layer at the top is application layer. Workloads in this layer are directly linked to the business value. The applications are very diverse. Most of the time, the release team is the main player at this layer. If the organization develops its own application, the software development team also work at this layer.

In terms of knowledge, the members of development team are experts in software engineering, and Software Development Life Cycle (SDLC), etc. On a daily basis, they deal with programming languages, product development, build and release. If they screw up their work, expect business errors, such as orders sent to wrong client, incorrect balance sheet, etc. This team has high visibility in the organization due to its direct link to business value.

This layer of work involves multiple tenants. Each tenant is isolated within their own namespace. When you switch CSP vendor, this layer should be readily portable with minimal effort.

It is also worth noting that, with solid platform and cluster layers, the team working at this layer do not write bespoke code for networking, observability, authentication and authorization, encryption and many other aspects not relevant to the core business. Once deployed, the application services are resilient, scale to demands, and cost efficient. This layer reaps the benefits of Kubernetes.

Kubernetes Platform as a Service

As the Kubernetes dust is still settling, a builder’s title may not always reflect which layer she or he focuses on. Today it is pretty common for infrastructure engineers to expand their role into the platform layer, or likewise, a software engineer to drill down to the platform layer.

The boundary between platform layer and cluster layer is clear. The cluster layer deals with underlying infrastructure, either in the cloud or on premise. They abstract away the complex infrastructure world from those working with the platform layer.

The boundary between platform layer and application layer is a little tricky to articulate. The application layer focuses on implementing the business logics. The platform layer takes care of the functions that are not part of business logic but essential to the business application. Take an HTTP request for example, application developer should not have to write code to terminate TLS (not part of business logic). They should only write the code to process the HTTP request (business logic). TLS termination is delegated to an Ingress, to be configured by platform builders.

The folks working at the Platform layer needs to interface with both sides. They provide Platform as a Service to the Application teams. However, their work appears mostly invisible in an organization. Their effort is oftentimes underestimated. There are several reasons for that. First, the platform layer does not directly create tangible business value. They are just someone else’s enabler. Second, their building blocks involve a lot of abstractions by Kubernetes API. Third, the idea of platform engineering is newly emerged. There hasn’t been a populous recognition of its value.

Red Hat OpenShift Container Platform

The platform team builds the platform with their choice of open-source tools. For clusters using OpenShift Kubernetes Engine, Red Hat introduces Open Shift container platform consisting of Red Hat’s opinionated (but validated) choice of toolings, for example:

OpenShift Service Mesh: Istio
OpenShift Streams: Apache Kafka
OpenShift GitOps: ArgoCD
OpenShift Container Platform Pipelines: Tekton
OpenShift Serverless: Knative
OpenShift Data Foundation: Ceph

Clients building their clusters with OpenShift Kubernetes Engine may build their own platform with the toolings in the OpenShift enterprise Kubernetes container platform. For more services, check out the documentation for OpenShift Container Platform. For customers with OpenShift Kubernetes Engine, their options to DIY platform are:

Entry-Level: Red Hat OpenShift Kubernetes Engine: Enterprise Kubernetes distribution on RHEL CoreOS
Mid-Level: Red Hat OpenShift Container Platform (RHOCP):
Plus-Level: Red Hat OpenShift Platform Plus: RHOCP + advanced cluster management, security, data management essentials, enterprise container registry

OpenShift runs the business model of Kubernetes PaaS.This is a unique business model that I do not find a matching competitor. Even if you choose to DIY your own platform, the Red Hat’s choices are still a great reference. The OpenShift enterprise Kubernetes container platform maps perfectly to the platform layer of the three-layer model, aiming to simplify the work in the platform layer.

Managed RedHat OpenShift

At first, the OpenShift container platform started as a value add-on to the Kubernetes Engine. Now it’s a separate product line in their business model. In the mean time, OpenShift partners with major CSPs, to develop the cloud service editions, including:

Red Hat OpenShift on AWS (ROSA)
Microsoft Azure Red Hat OpenShift (ARO)
Red Hat OpenShift Dedicated – on AWS and GCP
Red Hat OpenShift on IBM Cloud

These offerings are managed Kubernetes Platform as a Service in the cloud. Since RedHat is the only player in this model, we can refer to them as managed OpenShift services. In addition to an already-confusing world of Kubernetes platform portfolios, these offerings gives consumers even more options. On AWS for example, users have the following options:

Managed Platform: OpenShift Dedicated, managed by Red Hat
Managed Platform: Red Hat OpenShift Service on AWS (ROSA), managed by Red Hat and AWS
Self-built cluster: OpenShift Container Platform

This video discussed more details about these options, such as support model. It is also worth noting that these options tend to be much pricier than managed clusters such as EKS and AKS.

Since a Managed RedHat Platform makes it easy to deploy, let’s take ROSA as an example and create a cluster. To enable ROSA in AWS console, click on “Getting Started”. The next page ensures ROSA is enabled and checks other prerequisite such as meeting service quotas and creating ELB service-linked role, as show below:

Now, with an AWS account (and ROSA enabled), a RedHat account, and the rosa-cli utility, we can create a cluster with just a few commands. As a note, be wary of the cost and do not forget to delete the cluster afterwards.

Create a ROSA cluster

With the following set of commands, we can kick off cluster creation, using STS. We can bring our own VPC, so long as it meets certain prerequisites. I use the Terraform template in the vpc-base project, to create the underlying VPC. We’ll need the followings from this template:

The CIDR range of the VPC: as input with a default
The subnet Ids of the private subnet to place, printed in the output

The subnets are private subnets, because we want to provision the cluster with private node and private endpoint. When we use rosa CLI, we provide the CIDR and subnet IDs.

# start with AWS cli configured to the correct profile
rosa login # with redhat account and past token
rosa create account-roles --mode auto -y # this command creates the IAM roles ManagedOpenShift-*-Role, with RedHat account as trust entity
rosa verify permissions # optional
rosa verify quota # optional

export ROSA_CLUSTER_NAME="dhc" \
OPENSHIFT_VERSION=4.13.4 \
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) \
AWS_DEFAULT_REGION=us-east-1

rosa create cluster --sts --private \
 --cluster-name $ROSA_CLUSTER_NAME \
 --multi-az \
 --private-link \
 --region $AWS_DEFAULT_REGION \
 --version $OPENSHIFT_VERSION \
 --enable-autoscaling \
 --min-replicas 3 \
 --max-replicas 3 \
 --compute-machine-type m5.xlarge \
 --machine-cidr 147.206.0.0/16 \
 --subnet-ids subnet-052852a1fb4d7d2ad,subnet-06d8d40ae39d55c47,subnet-0f67ce08bc588012c

The CLI will pick up the correct VPC by CIDR, and prompt you to confirm creation of private cluster. After the command kicks off, it will wait for OIDC provider creation, and role creation. Then it uses a Terraform template to create the related resources including VPC. Use this command to check status:

rosa list clusters
rosa describe cluster -c dhc

When the second command displays the state of waiting (Waiting for OIDC configuration), we can create OIDC provider:

rosa create operator-roles -c $ROSA_CLUSTER_NAME --mode auto --yes
rosa create oidc-provider -c $ROSA_CLUSTER_NAME --mode auto --yes

Throughout the process, we can monitor the install log (terraform output) with:

rosa logs install -c dhc --watch

In the log, you might see errors with terminals connecting to the terraform backend, which doesn’t necessarily indicate a cluster creation error. Always check the cluster state until it reports success.

Kick the tires

Eventually the describe cluster command will show ready state. We can now create an admin user:

rosa create admin -c $ROSA_CLUSTER_NAME

The command above prints an oc command (OpenShift CLI, equivalent to kubectl) with password to log in. Let’s examine the cluster with oc. Because it is a private cluster, the endpoint is not available publicly. However, it is accessible from the Bastion host. Use the SSM Session Manager technique from my previous post to SSH to the Bastion Host, which should have oc installed. To install oc yourself, use HomeBrew on Mac. On Linux or Windows, log on to OpenShift console, go to Downloads on the left pannel and find it out under CLI tools.

The oc command may report insecure TLS on the login URL. Wait for a few minutes for the certificate to come off as safe. Once you run the oc command with password, it should return “Login successful” and then we can connect to the cluster:

$ oc get node # or kubectl get node
NAME STATUS ROLES AGE VERSION
ip-147-206-135-41.ec2.internal Ready,SchedulingDisabled infra,worker 3m5s v1.26.5+7d22122
ip-147-206-155-141.ec2.internal Ready control-plane,master 25m v1.26.5+7d22122
ip-147-206-156-81.ec2.internal Ready worker 19m v1.26.5+7d22122
ip-147-206-164-21.ec2.internal Ready infra,worker 3m3s v1.26.5+7d22122
ip-147-206-179-90.ec2.internal Ready worker 19m v1.26.5+7d22122
ip-147-206-191-118.ec2.internal Ready,SchedulingDisabled control-plane,master 26m v1.26.5+7d22122
ip-147-206-192-232.ec2.internal Ready worker 19m v1.26.5+7d22122
ip-147-206-193-198.ec2.internal Ready infra,worker 3m20s v1.26.5+7d22122
ip-147-206-218-114.ec2.internal Ready control-plane,master 26m v1.26.5+7d22122

You can use oc the same way you’d use kubectl. Both works through SOCK5 proxy. In the meantime, log in to the RedHat console with your Red Hat credential, you can see the cluster in Ready state as well:

The rosa create admin command creates a htpasswd type (username-password) of identity provider (IdP) with a user named cluster-admin and a preset password. In real life however, we often configure third party IdP with OIDC integration. I’ll have to leave this to the next blog post.

We shall see the nodes as EC2 instances from AWS console as well. Note that there are three roles: control-plane, worker and infra. The infra nodes are for infrastructure services. These services (Ingress Controller, GitOps, Pipeliens) are the ones in the platform player as we discussed above.

There are many customizations you can make in this installation process and I’d have to defer to the ROSA documentation. To clean up, use the following ROSA command:

rosa remove cluster -c dhc

The output also gives you the command to delete operator roles and OIDC provider, for example:

rosa delete operator-roles -c 23o4u3j98tqmlbtjo612opb7a4bbim5f --mode auto --yes
rosa delete oidc-provider -c 23o4u3j98tqmlbtjo612opb7a4bbim5f --mode auto --yes

Then we can destroy the VPCs using terraform.

ROSA with HCP

Update Oct 2023:

The deployment above provisioned a few nodes for control plane, which add to the overall time to provision a cluster. In Aug 2023, there is a new option Hosted Control Plane (HCP) that came to allow users to provision a hosted control plane. This results in cost savings and shorter time to provision a cluster. Here is a table of comparison between the ROSA with HCP and ROSA classic.

Final words

In this post, I discussed the three-layer model and pointed out that platform layer isn’t as visible as the other two. I also experimented ROSA as a turn-key Kubernetes platform with its opinionated stack of services.

Some misinformed organizations even skip the entire platform layer in their estimate of effort. They build a cluster, ran a hello-world service and assumes they can start putting applications on the Kubernetes cluster. There are also customers who purchased the entire Managed OpenShift platform but only use it as a cluster. Yikes!

The concept of Kubernetes platform, or generally platform engineering is still spreading. The consulting team that I worked in full-time last year re-branded itself as platform engineering. Marketings are pushing it. Builders are doing it. We’ll keep an eye, on whether customers are buying it.

Infrastructure deployment in Terraform 1/2

Wed, 11 Aug 2021 21:44:00 -0400

Terraform is an excellent Infrastructure-as-Code (IaC) tool based on Hashicorp Configuration Language (HCL). Compared to JSON or YAML based declarative templates (e.g. CloudFormation and ARM), HCL is more concise, thanks to the flexibility of HCL. On the other hand, HCL is not as flexible as general purpose languages. For that sake, I see HCL as semi-declarative IaC. This post is my notes about best practices with Terraform development, from the context of AWS, but also applies to other cloud platforms.

Complex Types

There are three primitive types (string, number and bool) that forms collection types and structural types. Here are some common ones:

list: element may repeat, and order is maintained:

["orange", "banana", "orange", "apple"]

set: elements are unique and unordered

["apple", "banana", "orange"]

tuple: each element has its own type

["a", 15, true]

object: defined by a schema with named attributes each with its own type

{
 name = "John"
 age = 52
}

list of object

[
 {
 alloc_id = "0b7271a3219bc1fc2"
 subnet_id = "0c02af76c2c3e46fa"
 },
 {
 alloc_id = "0440c334c48d4247f"
 subnet_id = "02652e69fa2a71de8"
 }
]

map of string

{
 property = "foo"
 attribute = "bar"
}

map of object

{
 objkey1 = {
 alloc_id = "0b7271a3219bc1fc2"
 subnet_id = "0c02af76c2c3e46fa"
 }
 objkey2 = {
 alloc_id = "0440c334c48d4247f"
 subnet_id = "02652e69fa2a71de8"
 }
}

Whenever applicable, Terraform converts types implicitly or explicitly. For example, when a list or tuple is converted to set, all elements are converted to string and duplicates are removed. Object and map are very similar. Map of string can be converted to object if the attributes comply with the schema. Additional attributes not in the schema are discarded.

HCL Types is similar to Python

Although being totally different beasts, the complex types between HCL and Python are similar, to the point I suspect the HCL design is influenced by Python. I summarize the similarities as such:

Python Types	list []	tuple ()	set {}	dict {}
Terraform Types	list []	tuple []	set []	map {}
ordered	Y You can access item by index	Y	N you cannot access an item by index or key; however you can loop over all itmes	N key-value pair that allows you to access item by key
changeable (mutable)	Y	N You cannot update, add or remove items	Y Add or remove only. no change to existing elements	Keys must remain unique or the values get overwritten; Values are mutable
allow duplicate elements	Y	Y	N	Keys must be unique; values don’t have to

In Python, list and tuple allow elements of mixed types but in IaC like Terraform we mostly don’t need mixed types. In Terraform, an object is a map without a defined type. In most situations, lists and tuples behave identically, as do maps and objects.

Another area of similarity is with the comprehension of list and dict/maps. In Python for example,

## Supposed you need to create a list:
lst=[]
for x in range(10):
 lst.append(x**2)

print(lst)

### that can be simplified as the following to create the list:
lst = [x**2 for x in range(10)]

## You can even add contidion
even_numbers=[num for num in range(10) if num%2==0]

## You can introduce function calls:
words = ["hello", "world", "python", "list", "comprehension"]
lengths = [len(word) for word in words]

## You can even combine two lists
lst1=[1,2,3,4]
lst2=['a','b','c','d']

pair=[[i,j] for i in lst1 for j in lst2]

print(pair)

## With dict, it's similar
evens={x:x**2 for x in range(10) if x%2==0}
print(evens)

In Terraform, we use similar techniques:

[for s in var.list : upper(s)] # build a tuple/list from a list

[for k, v in var.map : length(k) + length(v)] # build a list from a map

{for s in var.list : s => upper(s)} # build a map from a list

[for s in var.list : upper(s) if s != ""] # build a tuple/list from a list with condition

Note that the documentation of Terraform doesn’t explicitly call them out as comprehensions. However, it’s exactly the same idea as comprehensions in Python. Even the range() function exists both in Python and Terraform.

Modularization

Modules allows you to group related resources together. They can also be re-used and called by other modules. It is fairly straightforward to create a module:

put the resource declarations into a sub-directory
define input and output in the directory

However, the introduction of module complicates the directory structure and variable referencing, which is important to take into account before starting creating modules. This guideline has further discussion about when to create a module. I re-wrote the terraform templates in Orthweb project to leverage modularization wherever possible, but there is still some stand-alone resource (e.g. random_id) not belonging to any module.

To reference resources across modules, you need to import those resources (using data source) from within the module. There are a couple of ways. You may pass the argument of data source as input variable, or you can leverage the filter capability of data source. Let’s look at one example of each mechanism.

In the example below, we import a subnet by subnet id:

data "aws_subnet" "private_subnet" {
 id = var.private_subnet_id
}

In the example below, we import a subnet by filtering from all subnets in the VPC by tag:

data "aws_subnet" "private_subnet" {
 vpc_id = var.vpc_id
 filter {
 name = "tag:Name"
 values = ["Private"]
 }
}

There are pros and cons of each approach. A module with mechanism 1 is more transferrable across different environment, because the ID of subnet is explicitly provided. However, authors needs to manage those explicit variables with code. Mechanism 2 fetches target resources with filter. It depends on a well-implemented tagging policy in the resource farm.

The Terraform Registry (since 2017) contains a lot of pre-built modules for each backend platform (e.g. AWS). If you find any module that can be used in your project, the module repo can be referenced directly by Git repository URL. You should be aware of the risk of this practice though. Many platforms are keen to publish modules for their platform. Anyone can publish their own modules to the community as well.

You can quickly generate module documentation with terraform-docs.

Local Execution

Local execution is the basic workflow mode which is mostly seen with very small collaboration team. In this mode, the developer executes terraform binary (Terraform CLI) from their workstation (e.g. Laptop). The Terraform CLI converts code into API calls to interface cloud provider. The most frequently used commands (from terraform directory) are:

terraform init
terraform plan
terraform apply

The init command initializes the working directory. The plan command figures out the delta between code and infrastructure. It outlines the changes it is about to make. The apply command commits the change. The documentation of Terraform CLI commands is here.

Terraform keeps track of the infrastructure it manages in state file. This article explains the purpose of state. State management collaboration difficult with local execution because the state file by default is created in the working directory on user’s workstation. Although the state file can be configured to be stored in a shared location such as S3, it still requires a mechanism to lock the state in a multi-developer collaboration.

In large operations, the same code base in Terraform, is usually used to created several different sets of infrastructures, for example, in different geographic regions. So it is a 1-to-many relationship between the code repo and the infrastructure state. To further complicate things, each state might have been deployed using different revisions of the code. To overcome that challenge, Terraform introduced the concept of workspace, which is essentially an instance of state describing a particular group of infrastructure being managed by the same source code. When there are many workspaces, it becomes tricky to manage them with CLI commands.

State management is a major challenge that needs to be solved for team collaboration in local execution workflow. Each state must use the same revision of Terraform code. You can use Git in combination as a workaround to that limitation but the point is you cannot tie a workspace to a commit with the workspace commands.

In some enterprise environment, the execution is from a VM (e.g. ADO agent on-premise) without Internet access, which poses another challenge. First, we need to pre-load required providers manually. The enterprise needs a proxy solution to safely download packages from Hashicorp website. One good option is Nexus Repository, with both open-source and pro supports. It is a full-function artifactory repo that can host helm repo, apt repo, yum repo, etc. Second, we also need to configure Terraform so it picks up providers locally. Managing plugins without Internet access requires understanding of the order in which Terraform tries to load plugins during initialization.

Remote Execution

In remote execution, the code is executed in Terraform Enterprise or Terraform Cloud. Both are remote web servers. The difference is that Terraform Enterprise is self-hosted service, requiring IT specialist to install and maintain Terraform Enterprise. Terraform Cloud on the other hand, is a managed SaaS service. The pricing model includes a free plan for small number of users.

Terraform workspace configuration

In Terraform Enterprise or Cloud, the remote execution is organized in workspaces. You need to create an organization, and then create workspace under the organization in order to execute code. With each workflow, you can specify version control system (VCS) and subdirectory, to tell the workspace where to fetch Terraform code from. The workspace also allows you to define secrets and variables specific to the workspace. When you execute a workspace plan, the secrets and variables are passed from workspace to the execution logic.

Terraform Workspace Variable configuration

You will also need to design the Terraform code in a way to work seamlessly with the secrets and variables loaded from the workspace. The variable declaration in code should match the definition in workspace. There are already a number of variables that came in handy. Check out this guide.

Each execution is referred to as a “run”, with its own run id. A workspace involves may runs, which may succeed or fail. Each run pulls a specific commit of the source repository, and goes through stages such as plan, and apply. The UI from each run result list out the status of each result, in a very easy to read format.

The state data is persisted in the web server as they were generated. Therefore the collaborator do not need to worry about managing state with CLI tools. If there are files that you do not want picked up by the execution engine, their locations can be added to a file .terraformignore. Refer to this guide.

AWS profile

Local execution still has a lot of use cases in enterprises such as testing with temporary resources. A common challenge is authentication. As discussed, Terraform CLI picks up identity information from AWS CLI and authenticates its way into the backend to run API calls against. So AWS CLI must be configured correctly with the sufficient permission to provision resources. On the other hand, enterprises usually offload IAM to an identity store, such as AzureAD, Okta, etc. Putting those together, the pattern of authentication and authorization usually looks like this:

User logs on via SSO (e.g. SAML). The validation response gives a name of an IAM role.
Upon successful authentication, user takes the IAM role. The role does not have any capability, except for assuming a second IAM role.
The second IAM role (the functional role) grants user the permission to do its business.

The steps above, can be carried out in AWS console, or with AWS cli using assume-role command. However, when we put Terraform in the picture, it becomes a little involving because the credential information is updated whenever the functional role is assumed, and the assume-role command takes a pretty long argument.

To skip typing the long command every time, there are some handy tools, such as aws-azure-login. An even better tool that works with a variety of identity stores is saml2aws. The tool allows you to configure identity backend, assume the functional role, and update credential information in aws credential file, all with a single command. The AWS CLI configuration reads:

[default]
region = us-east-1
output = json
cli_history = enabled
cli_pager =
role_session_name = functional_operation

[profile function_user]
source_profile = default
role_session_name = functional_operation
role_arn = arn:aws:iam::9998887766:role/admin-access
region = us-east-1

In Terraform provider, we need to tell it to assume that role as well:

provider "aws" {
 region = "us-east-1"
 assume_role {
 role_arn = "arn:aws:iam::9998887766:role/admin-access"
 session_name = "terraform"
 }
}

This will ensure Terraform assumes appropriate role before doing its job.

AWS EC2 SSH Key Pair

RSA key authentication for SSH should be used for Linux Instances. When creating an EC2 instance, we give it our public key so we can then later authenticate through SSH. If the key is already stored in AWS, we just need to tell EC2 the name of the key, in the key_name property. If the code is likely to be executed from several different places by different users, then we can write the code so it picks up public key from user’s workstation (~/.ssh/id_rsa.pub). Here is an example:

variable "local_pubkey_file" {
 type = string
 default = "~/.ssh/id_rsa.pub"
}
data "local_file" "pubkey" {
 filename = pathexpand(var.local_pubkey_file)
}
resource "aws_key_pair" "user-pubkey" {
 key_name = "runner-pubkey"
 public_key = data.local_file.pubkey.content
}
resource "aws_instance" "bastion" {
 instance_type = "t2.micro"
 key_name = aws_key_pair.user-pubkey.key_name
 ......
}

For remote execution, we can even add an option to pass public key in as variable, to override the key file variable. For an example, check out my orthweb project.

To upload files to EC2 instance from Terraform execution environment, we can use the file provisioner with ssh as connection type.