Links - November 2023

A list of articles and projects from November that I found interesting.

First of all, a big Thank You if you’re reading this as a subscriber. If you are not, you can always subscribe to receive posts directly in your inbox.

This post is a list of articles and projects I found interesting this month.

Cloud

Upcoming Changes in Kubernetes 1.29

Among other things, this has “Removal of in-tree integrations with cloud providers”. It means that if you are a cloud vendor providing a managed Kubernetes service, and using the in-tree (in the Kubernetes source code repository) cloud controller managers, you have to switch to external ones.

A cloud controller manager is a Kubernetes component that has cloud specific logic. It bridges Kubernetes resource abstractions with cloud vendor specific ones. E.g. It allows Kubernetes to query the underlying cloud provider about the state of actual virtual machines that form Kubernetes nodes.

The aim of this change is to ensure that

“development of each cloud provider should be done in their respective external repos”

This should not affect managed Kubernetes users but it’s interesting to see these efforts behind removing cloud vendor specific code from the main Kubernetes codebase. The KEP (Kubernetes Enhancement Proposal) is almost 3 years old as of this writing.

Kubernetes Enhancement Proposal for Node System Swap Support

Kubernetes does not support Linux swap memory. This KEP is a proposal to enable it at the node level and for workloads via a kubelet configuration.

Convert Google Cloud Architecture Diagrams into Terraform code

The Google Cloud Architecture Diagramming tool lets you visually design your Google Cloud stack. This new feature lets you download the stack as a set of Terraform files that you can run to boot up your infra. 

Security

New SSH Vulnerability

“For the first time, researchers have demonstrated that a large portion of cryptographic keys used to protect data in computer-to-server SSH traffic are vulnerable to complete compromise when naturally occurring computational errors occur while the connection is being established. 

It affects only RSA based keys.

“Of the roughly 1 billion RSA signatures, about one in a million exposed the private key of the host.”

And most importantly:

“The countermeasure to the attacks we describe in this paper is well known: implementations should validate signatures before sending them. OpenSSH, the most common SSH implementation we observed in this data, implements this countermeasure because it uses OpenSSL to generate signatures, and OpenSSL has included countermeasures against RSA fault attacks since 2001.”

Monitoring

Getting started with Tetragon on GKE

Tetragon is a real-time eBPF-based policy enforcement tool. You can install it as a Helm chart in your Kubernetes cluster.

Perses - Dashboard-as-Code

The idea behind the Perses project is “dashboard-as-code”. It supports Prometheus as a data source.

The core idea behind it:

“It aims to become a standard dashboard visualization tool for Prometheus and other datasources. It will focus on being GitOps-compatible and thus enabling a smooth "dashboards as code" workflow via a new and well-defined dashboard definition model.”

Time for some rambling. In one of my past teams, one of our principles was everything-is-a-metric when it comes to monitoring and alerting. The advantage was being able to apply consistent alerting abstractions across all microservices and all infrastructure metrics. Common metrics like e.g. JVM heap size, CPU usage, web server response time were auto-collected from all deployed services, and devs were free to add custom metrics if they wanted. We used a customized version of Grafana for dashboards. 

To take this idea to its logical conclusion, we wanted to have a way where developers could commit their metrics related artifacts alongside their code. These would include Prometheus alerting rules and custom dashboards, if any. Many services needed custom dashboards - e.g. a Kafka-based pipeline’s dashboard needed Kafka specific metrics. If devs had the ability to define their dashboards in a DSL so that they got deployed as part of app deployment, it would mean them not having to worry about Grafana version issues or how the dashboards got rendered. 

The dashboarding part did not happen at that time for various reasons - which is why I find this project exciting. 

Tools

Terraform Wrapper for Local Stack

A Terraform wrapper that lets you test your AWS Terraform resources against LocalStack. LocalStack is a cloud emulator that runs locally on your machine. 

Opinions

Patrick Debois on the Past, Present, And Future of DevOps

Patrick Debois founded the DevOps Days conference in 2009. He is widely known as one of the founders, or popularizers, behind the idea of DevOps. In this interview Debois asserts his definition of DevOps where collaboration and friction removal is the bottom line, and not just between dev and ops. He also talks about what the emerging field of platform enginering grew out of. Insightful thoughts from somebody who has seen it from the beginning.

It’s a bit weird that after all this one of the questions asked to Debois is “Do we need a separate platform team and a separate DevOps team?” A “DevOps team” is a misnomer, considering it’s a philosophy.

Towards Modern Development of Cloud Applications

Although not from November, this paper was discussed a lot this month. It proposes a new programming and deployment model whose goal is to mitigate the various “issues” that plague a microservices model.