knative serving on Digi Hunch

Knative Eventing Introduction

Fri, 29 Apr 2022 10:16:00 -0400

In the previous post, I mentioned that Knative Serving and Knative Eventing should be seen as two different projects. The former is supposed to be widely used as a serving layer for microservices, whereas the latter has a narrower customer base. There are a dozen companies who need to build Platform as a Service, and will benefit from event-driven architecture. They are the niche customer for Knative Eventing. In this post, we go through two demos for Eventing.

Event Driven Architecture

Before jumping into Knative eventing, let’s first understand the value of event-driven architecture. As covered in Knative presentation in 2020, a growing micro-service architecture involves lots of service-to-service communications that looks like a spider web in a diagram. Two services communicating with each other are tightly coupled: changing one requires a change of the other. Event-driven architecture evolves to decouple those interdependent services. Knative eventing tries to standardize the event data with Cloud Event and also proposes a framework to drive this architecture.

The documentation uses different words to express the same concepts. So let’s take a note first: source = producer which emits events, and sink = subscriber = consumer which is the destination of events. Knative Eventing supports two models of in even-driven architecture.

Brokers and Triggers provides an “event mesh”. Event producers delivers events to a Broker. Triggers then distribute them uniformly by consumers.

Broker is a “hub” for events. It is a central location to receive and send events for delivery. We can choose from the following broker types:

MT (multi-tenant) channel-based broker (which a channel implementation, such as Kaka channel or an in-memory channel)
GCP broker
Apache Kafka broker
RabbitMQ broker

Triggers represents a desire to subscribe to events from a specific broker. It can act as a filter

channel-subscriber model

Channels and Subscriptions provide a “event pipe” model which transforms and routes events between Channels using Subscriptions. This model makes more sense when events out of one system needs to be transformed and then routed to another process.

A channel can be either a generic channel object, or a custom channel implementation (such as Kafka channel or in-memory channel)

The subscription consists of a Subscription object, which specifies the Channel and the Sink (aka the Subscriber) to deliver events to. You can also specify some Sink-specific options, such as how to handle failures.

The key difference is Broker-Trigger model allows you to filter events, where as Channel-Subscription model allows you to transform events.

Note that a Knative Service can act as both a Source and a Sink for events, and for good reason. You may want to consume events from the Broker and send modified events back to the Broker, as you would in any pipeline use-case. Between these components, Knative Eventing uses CloudEvents to send information back and forth.

Knative Eventing Lab

We can create a lab cluster using Kind as per the instruction in real-quicK-cluster. In docker-desktop settings, I configured 6 CPU and 8GB memory as the lab involves a number of nodes. The creation takes a couple minutes. This Knative eventing lab has four paths one can take, based on different model and different implementations:

Lab A demonstrates the configuration of broker-trigger model using Kafka based broker.
Lab B demonstrates the configuration of channel-subscription model, using in-memory channel.

The two labs share some common steps. We first install Knative Eventing using the provided manifests:

$ kubectl apply -f https://github.com/knative/eventing/releases/download/knative-v1.3.0/eventing-crds.yaml
$ kubectl apply -f https://github.com/knative/eventing/releases/download/knative-v1.3.0/eventing-core.yaml
$ kubectl get pods -n knative-eventing

The Pods should come to READY fairly quickly. In the mean time, we can configure the consumer as well using the manifests below:

apiVersion: v1
kind: Namespace
metadata:
 name: event-example
---
apiVersion: apps/v1
kind: Deployment
metadata:
 name: hello-display
 namespace: event-example
spec:
 replicas: 1
 selector:
 matchLabels: &labels
 app: hello-display
 template:
 metadata:
 labels: *labels
 spec:
 containers:
 - name: event-display
 image: gcr.io/knative-releases/knative.dev/eventing/cmd/event_display
---
kind: Service
apiVersion: v1
metadata:
 name: hello-display
 namespace: event-example
spec:
 selector:
 app: hello-display
 ports:
 - protocol: TCP
 port: 80
 targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
 name: goodbye-display
 namespace: event-example
spec:
 replicas: 1
 selector:
 matchLabels: &labels
 app: goodbye-display
 template:
 metadata:
 labels: *labels
 spec:
 containers:
 - name: event-display
 image: gcr.io/knative-releases/knative.dev/eventing/cmd/event_display
---
kind: Service
apiVersion: v1
metadata:
 name: goodbye-display
 namespace: event-example
spec:
 selector:
 app: goodbye-display
 ports:
 - protocol: TCP
 port: 80
 targetPort: 8080

The consumers (aka sinks) will be able to process events at the validation step at the end.

Lab A – Install Kafka and configure Kafka-based Broker

This lab is for broker-trigger model. We first configure a Kafka Broker, which requires a Kafka cluster. The document uses Strimzi operator to quickly install Kafka cluster. Instead, here we use Helm to install the Kafka cluster:

helm install my-cluster-kafka bitnami/kafka -n kafka --create-namespace --set volumePermissions.enabled=true --set replicaCount=3

At the end of installation, the Helm notes indicate that the bootstrap server is my-cluster-kafka.kafka.svc.cluster.local:9092. All Kafka pods should come ready in about 5 minutes. After cluster install, we need to install Kafka controller and the Broker-layer data plane which allow you to map a Kafka instance to Broker CRD later:

kubectl apply -f https://github.com/knative-sandbox/eventing-kafka-broker/releases/download/knative-v1.3.0/eventing-kafka-controller.yaml
kubectl apply -f https://github.com/knative-sandbox/eventing-kafka-broker/releases/download/knative-v1.3.0/eventing-kafka-broker.yaml
kubectl get pods -n knative-eventing
kubectl -n knative-eventing edit cm kafka-broker-config

As the command above suggests, once data plane is installed, we edit the default broker config by updating the bootstrap.servers with the bootstrap server address noted above. We also need to make sure the replication factor is no greater than number of partitions, which should not be a problem in our lab as the replica count was set to 3.

Now we can create a Broker in the same namespace with consumer:

apiVersion: eventing.knative.dev/v1
kind: Broker
metadata:
 name: default
 namespace: event-example
 annotations:
 eventing.knative.dev/broker.class: Kafka
spec:
 config:
 apiVersion: v1
 kind: ConfigMap
 name: kafka-broker-config
 namespace: knative-eventing

Once the manifest above is applied, we should see the status of broker (named default) to be READY. We can also validate broker creation with kn CLI command:

 $ kubectl -n event-example get broker default
NAME URL AGE READY REASON
default http://kafka-broker-ingress.knative-eventing.svc.cluster.local/event-example/default 19m True
$ kn broker list -n event-example
NAME URL AGE CONDITIONS READY REASON
default http://kafka-broker-ingress.knative-eventing.svc.cluster.local/event-example/default 19m 7 OK / 7 True

Then we create Triggers that connects both to the broker with filters, and to the consumers by referencing service name. The manifest to create triggers is provided below:

apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
 name: hello-display
 namespace: event-example
spec:
 broker: default
 filter:
 attributes:
 type: greeting
 subscriber:
 ref:
 apiVersion: v1
 kind: Service
 name: hello-display
---
apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
 name: goodbye-display
 namespace: event-example
spec:
 broker: default
 filter:
 attributes:
 source: sendoff
 subscriber:
 ref:
 apiVersion: v1
 kind: Service
 name: goodbye-display

We can confirm the setups above by monitoring the resources (like we did for broker), or use knative CLI:

$ kn trigger list -n event-example
NAME BROKER SINK AGE CONDITIONS READY REASON
goodbye-display default service:goodbye-display 3m24s 6 OK / 6 True
hello-display default service:hello-display 3m24s 6 OK / 6 True

Now we can go to the validation step to send event and validate result.

Lab B – Install in-memory channel and configure Channel

This lab is for channel-subscription model. we first install an in-memory channel:

kubectl apply -f https://github.com/knative/eventing/releases/download/knative-v1.3.0/in-memory-channel.yaml

Then we can map this channel to a Channel resource in Eventing. We can do so by applying the manifest below:

apiVersion: messaging.knative.dev/v1
kind: Channel
metadata:
 name: demo-channel-1
 namespace: event-example
---
apiVersion: messaging.knative.dev/v1
kind: Channel
metadata:
 name: demo-channel-2
 namespace: event-example

The creation of Channel resources can be verified as below

$ kn -n event-example channel list
NAME TYPE URL AGE READY REASON
demo-channel-1 InMemoryChannel http://demo-channel-1-kn-channel.event-example.svc.cluster.local 6m34s True
demo-channel-2 InMemoryChannel http://demo-channel-2-kn-channel.event-example.svc.cluster.local 6m34s True

Now we need to create Subscription which connects Channels to Consumers. Subscriptions can be created with the manifest below:

apiVersion: messaging.knative.dev/v1
kind: Subscription
metadata:
 name: demo-subscription-1
 namespace: event-example
spec:
 channel:
 apiVersion: messaging.knative.dev/v1
 kind: Channel
 name: demo-channel-1
 subscriber:
 ref:
 apiVersion: v1
 kind: Service
 name: hello-display
---
apiVersion: messaging.knative.dev/v1
kind: Subscription
metadata:
 name: demo-subscription-2
 namespace: event-example
spec:
 channel:
 apiVersion: messaging.knative.dev/v1
 kind: Channel
 name: demo-channel-2
 subscriber:
 ref:
 apiVersion: v1
 kind: Service
 name: goodbye-display

We can confirm subscription:

$ kn -n event-example subscription list
NAME CHANNEL SUBSCRIBER REPLY DEAD LETTER SINK READY REASON
demo-subscription-1 Channel:demo-channel-1 service:hello-display True
demo-subscription-2 Channel:demo-channel-2 service:goodbye-display True

Now we can go to validation step.

Validation

For validation, we need to first configure an event source. The source in our lab is a throw-away Pod with curl utility. I use my favourite nicolaka netshoot to fire off the events using. To launch the Pod and get to command shell:

$ kubectl run tmp-shell -n event-example --rm -i --tty --image nicolaka/netshoot -- /bin/bash
If you don't see a command prompt, try pressing enter.
bash-5.1#

We can POST message to the corresponding endpoint URL to fire events. The endpoint is different for each lab:

Lab	URL
A (Kafka-based broker)	http://kafka-broker-ingress.knative-eventing.svc.cluster.local/event-example/default
B (channel)	http://demo-channel-1-kn-channel.event-example.svc.cluster.local http://demo-channel-2-kn-channel.event-example.svc.cluster.local

With the correct URL, we can build a payload for POST method. Below is three curl commands with different payload POST to the same endpoint (taking lab A as an example):

# curl -v "http://broker-ingress.knative-eventing.svc.cluster.local/event-example/default" \
-X POST \
-H "Ce-Id: say-hello" \
-H "Ce-Specversion: 1.0" \
-H "Ce-Type: greeting" \
-H "Ce-Source: not-sendoff" \
-H "Content-Type: application/json" \
-d '{"msg":"Hello Knative!"}'

# curl -v "http://broker-ingress.knative-eventing.svc.cluster.local/event-example/default" \
-X POST \
-H "Ce-Id: say-goodbye" \
-H "Ce-Specversion: 1.0" \
-H "Ce-Type: not-greeting" \
-H "Ce-Source: sendoff" \
-H "Content-Type: application/json" \
-d '{"msg":"Goodbye Knative!"}'

# curl -v "http://broker-ingress.knative-eventing.svc.cluster.local/event-example/default" \
-X POST \
-H "Ce-Id: say-hello-goodbye" \
-H "Ce-Specversion: 1.0" \
-H "Ce-Type: greeting" \
-H "Ce-Source: sendoff" \
-H "Content-Type: application/json" \
-d '{"msg":"Hello Knative! Goodbye Knative!"}'

All the requests should receive 202 return code. As highlighted above, we specified type and source for each event fired to the broker. On the consumer side, we can simply check the pod log to verify the events received. Below is an example of events received as a result of the events sent above (take broker-trigger model as example):

$ kubectl -n event-example logs -l app=hello-display --tail=100
2022/03/09 17:05:59 Failed to read tracing config, using the no-op default: empty json tracing config
☁️ cloudevents.Event
Context Attributes,
 specversion: 1.0
 type: greeting
 source: not-sendoff
 id: say-hello
 datacontenttype: application/json
Extensions,
 knativearrivaltime: 2022-03-09T17:14:19.196589801Z
Data,
 {
 "msg": "Hello Knative!"
 }
☁️ cloudevents.Event
Context Attributes,
 specversion: 1.0
 type: greeting
 source: sendoff
 id: say-hello-goodbye
 datacontenttype: application/json
Extensions,
 knativearrivaltime: 2022-03-09T17:14:42.977803608Z
Data,
 {
 "msg": "Hello Knative! Goodbye Knative!"
 }
$ kubectl -n event-example logs -l app=goodbye-display --tail=100
2022/03/09 17:05:58 Failed to read tracing config, using the no-op default: empty json tracing config
☁️ cloudevents.Event
Context Attributes,
 specversion: 1.0
 type: not-greeting
 source: sendoff
 id: say-goodbye
 datacontenttype: application/json
Extensions,
 knativearrivaltime: 2022-03-09T17:14:27.430702588Z
Data,
 {
 "msg": "Goodbye Knative!"
 }
☁️ cloudevents.Event
Context Attributes,
 specversion: 1.0
 type: greeting
 source: sendoff
 id: say-hello-goodbye
 datacontenttype: application/json
Extensions,
 knativearrivaltime: 2022-03-09T17:14:42.977803608Z
Data,
 {
 "msg": "Hello Knative! Goodbye Knative!"
 }

This is the gist of Knative evening. Note that the format of HTTP payload when we fire off events needs to conform to CloudEvents format. The container image (event_display) used in the container is also created for the purpose of processing events received and display them in stdout.

Conclusion

Persistent storage allows for stateful applications on Kubernetes. It also enables architectural patterns that requires persistent storage. Knative eventing is a framework that enables event driven architecture. It supports many messaging channels (including NATs, a cloud-native messaging system) and broker types (MT channel based, GCP, RabbitMQ).

Knative Serving Introduction

Sun, 17 Apr 2022 11:33:00 -0400

Background

As per IBM‘s definition, Knative enables serverless workloads to run on Kubernetes clusters, and makes building and orchestrating containers with Kubernetes faster and easier. It has drawn a lot of attention recently. It released version 1.0 in November 2021, and was accepted as a CNCF incubating project in March 2022.

Glories aside, the value it delivers is the ability to go serverless on a Kubernetes platform. Originally, Knative was built with three components: build, serving and eventing. The build component was deprecated in favour of the Tekton project. Tekton is cloud native CI/CD pipeline. It connects to source code repo and build artifacts. It can also deploy applications with pipeline as code. Tekton’s documentation includes a quality tutorial here linked to interactive terminal. On the other hand, we’ll focus on Knative Serving in this post, and Knative Eventing in the next.

Knative can be seen as a serverless framework with a number of open-source technologies as building blocks, such as Istio for ingress gateway, Kafka or Google Pub/Sub as event-streaming engine, and Prometheus for observability to name a few. Despite of using the same CLI tool (named kn), Knative Serving and Eventing are considered separate capabilities using different groups of CRDs. They are installed separately, using their respective YAML files, or operator.

Serving vs Eventing ?

After reading the article “Did we market Knative wrong” from Ahmet Balkan, my impression is the two are not related. They could have been two separate projects without sharing a common name. As the author puts, these two shared some core logics. But beyond that, they don’t have anything in common.

Serving is lightweight and requires Istio. It provides the capability to scale from N to 0 when the system is idle, and from 0 to N when requests come in. It is the missing serving layer for running microservices on Kubernetes and Ahmet position it as the first thing that people installed after creating a cluster.

The target user of Eventing is much smaller. Eventing is for event-driven architecture and is more complex than Serving. Ahmet admits that they over-estimated how many people on the planet want to build a Heroku-like PaaS layer on top of Knative. There are a couple of dozen companies who would work through the complexity and build their own Kubernetes-based internal PaaS or even public-facing FaaS using Knative. They are the niche audience of Knative Eventing.

For most platform builders who just need to run micro-service, Knative Serving is all they need.

A Demo on Serving

The definition of term “serverless” is very loose. Instead, Knative documentation promotes the ability to scale to zero and refer to it as “some people call this Serverless”. This suggests that “the ability to scale to zero” should be the better term to describe this capability. To deploy it, we need the Service object with apiVersion serving.knative.dev/v1. Below is a guide for a quick hands-on, with some steps modified from the Knative Serving tutorial and installation guide, with optional steps (extensions, DNS) skipped.

Follow the first part of this post or this guide in real-quicK-cluster project, to install minikube with metal LB and configure istio (using istioctl as per instruction). Then Knative serving can be installed with two YAML manifests:

$ kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.2.0/serving-crds.yaml
$ kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.2.0/serving-core.yaml

Then we can install Knative istio controller to integrate them so we can see the knative gateways:

$ kubectl apply -f https://github.com/knative/net-istio/releases/download/knative-v1.2.0/net-istio.yaml
$ kubectl -n knative-serving get gateway

Now we can install the dummy service, either using kn CLI tool or using YAML manifest. The manifest is given below:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
 name: hello
 namespace: default
spec:
 template:
 metadata:
 name: hello-world
 annotations:
 autoscaling.knative.dev/class: "kpa.autoscaling.knative.dev"
 autoscaling.knative.dev/metric: "rps"
 autoscaling.knative.dev/target: "50"
 autoscaling.knative.dev/min-scale: "0"
 autoscaling.knative.dev/max-scale: "10"
 spec:
 containerConcurrency: 0
 containers:
 - env:
 - name: TARGET
 value: World
 image: gcr.io/knative-samples/helloworld-go
 name: user-container
 ports:
 - containerPort: 8080
 protocol: TCP

We can notice that in the annotations there are some settings to overwrite the default configuration. They are explained in the autoscaling section of the Knative serving documentation. Here we use request per second (rps) as metric, and have 50 as target. The scale range is between 3 and 10 replicas. Once the resource is created, we can validate it with one of the two commands:

$ kubectl get services.serving.knative.dev
$ kn service describe hello

The creation of knative service also created a number of Kubernetes resources, such as deployment and services:

$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello ExternalName <none> knative-local-gateway.istio-system.svc.cluster.local 80/TCP 52s
hello-world ClusterIP 10.110.127.231 <none> 80/TCP 108s
hello-world-private ClusterIP 10.111.27.73 <none> 80/TCP,9090/TCP,9091/TCP,8022/TCP,8012/TCP 108s
kubernetes ClusterIP 10.96.0.1 <none>
$ kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
hello-world-deployment 1/1 1 1 71s

Before testing, let’s mock the DNS entry to resolve to the Istio ingress service IP like below in /etc/hosts:

192.168.64.16 hello.default.example.com

Now we can confirm that the service returns the expected result (after a brief pause):

$ curl hello.default.example.com
Hello World!

Wait for a short period without curl or connection from browser, the service should scale down to zero:

$ kubectl get deploy -l serving.knative.dev/service=hello -w
NAME READY UP-TO-DATE AVAILABLE AGE
hello-world-deployment 0/0 0 0 78s

From a separate terminal, emulate HTTP request with hey command, so the service receives more than 50 rps per pod:

$ hey -n 200 -c 20 -z 20s -m GET http://hello.default.example.com

We can watch the deployment get scaled up to 10. Once the hey command completes its task, the deployment will scale back down:

$ kubectl get deploy -l serving.knative.dev/service=hello -w
NAME READY UP-TO-DATE AVAILABLE AGE
hello-world-deployment 0/0 0 0 78s
hello-world-deployment 0/1 0 0 3m54s
hello-world-deployment 0/1 0 0 3m54s
hello-world-deployment 0/1 0 0 3m54s
hello-world-deployment 0/1 1 0 3m54s
hello-world-deployment 1/1 1 1 4m2s
hello-world-deployment 1/10 1 1 4m4s
hello-world-deployment 1/10 1 1 4m4s
hello-world-deployment 1/10 1 1 4m4s
hello-world-deployment 1/10 10 1 4m5s
hello-world-deployment 2/10 10 2 4m9s
hello-world-deployment 3/10 10 3 4m10s
hello-world-deployment 4/10 10 4 4m10s
hello-world-deployment 7/10 10 7 4m10s
hello-world-deployment 9/10 10 9 4m10s
hello-world-deployment 10/10 10 10 4m11s
hello-world-deployment 10/5 10 10 5m18s
hello-world-deployment 10/5 10 10 5m18s
hello-world-deployment 5/5 5 5 5m18s
hello-world-deployment 5/2 5 5 5m20s
hello-world-deployment 5/2 5 5 5m20s
hello-world-deployment 2/2 2 2 5m20s
hello-world-deployment 2/1 2 2 5m22s
hello-world-deployment 2/1 2 2 5m22s
hello-world-deployment 1/1 1 1 5m22s
hello-world-deployment 1/0 1 1 5m52s
hello-world-deployment 1/0 1 1 5m52s
hello-world-deployment 0/0 0 0 5m52s

Autoscaling in Knative serving is backed by KPA (Knative Pod Autoscaler, used in the example above) or HPA. Also note that the trigger to scale up and down is client connection (RPS). This is different than KEDA. KEDA also supports the capability to scale to zero. However, with KEDA, it is an event that scales a deployment up from zero. In Knative serving, it is the connection to the Service itself that wakes up the service.

Cost of scale-to-zero

As discussed, both KEDA and Knative serving supports the ability to scale to zero and wake up from zero. The trigger to wake up from zero is different. With KEDA, workload wakes up by an event. With Knative Serving, workload has to wake up upon receiving a connection to the service. In the design to solve this problem, Knative has four sub-components :

Activator – When a service is scaled to zero, its request are routed to the activator which waits for a Pod to become ready (wake up) and proxies the traffic while editing the underlying Endpoints to route the traffic directly to the Pod(s)
Autoscaler – KPA
Controller – The main component responsible for watching API objects for Knative CRDs (KService, Configuration, Route, Revision) and manage their lifecycle, create the underlying Kubernetes resources and garbage-collect old objects.
Webhook – a Kubernetes Admission Webhook acting both as validating admission controller and mutating admission controller.

The key player is activator, which receives requests when a service is scaled to zero. The wake-up process is summarized as follows:

After receiving request, activator will buffer (hold onto) the request
Then, it will look at request’s hostname to find which KService it is for.
Then, it will scale up the Kubernetes Deployment and wait for a Pod to become ready.
In the meanwhile, it updates Kubernetes Service to point to Pod IP addresses (so that activator gets out of the network path if the KService is awake).
Finally, activator proxies the request to the started pod

So the “plumbing” of the service connection managed by the Activator pod is the cost of scale to zero. This is also called Load Balancing, and the behaviour can be tweaked using two parameters: activator capacity and target burst capacity:

Target burst capacity: once the target deploy has one more more Pod, service requests may still route via activator or bypassing the activator. Target burst capacity determines at what point service request should start to bypass activator.
Activator capacity: determines how many requests can activator hold on to. Considering the service-to-service connection also routes via Service CRD, there can be a lot of connections via Activator so its capacity needs to be adjusted.

The additional overhead of managing and optimizing the “Load Balancer” is also part of operational cost for the ability to scale to zero.

In comparison, KEDA now has an HTTP add-on still at beta but allows connection-based wake-up. The design is a little different. As discussed in this previous post.

Conclusion

Knative consists of two disparate components: Serving and Eventing. Serving uses KPA to provide scaling based on service request, and the ability to scale to zero. It is often compared with KEDA, a single-purpose lightweight tool for autoscaling. Here is a blurb on their difference by KEDA. The takeaways is that KEDA is more focused on scalability, whereas Knative serving covers more aspects. For example, the Service object of Knative also supports Traffic Splitting. Knative serving can integrate with istio.