Links - December 2023
A list of articles and projects from December that I found interesting.
I’m sending out this Links edition a bit earlier than the usual end of the month.
Wishing you all a Happy New Year in advance.
Talks
Writing Maintainable Infrastructure as Code
Best practices talk on IaC using examples in OpenTofu/Terraform. The talk’s slides are also available.
Interview with Solomon Hykes where he discusses Dagger, DevOps, and Docker
“CI/CD as code”
Projects and Releases
Kubernetes v1.29 released
Yet another Kubernetes release, the last for 2023.
The AWS SDK for Rust is now GA
Rust is a language to watch (and better still, learn) if you work in distributed systems/systems engineering/operations.
Keynote: Linus Torvalds, Creator of Linux & Git, in Conversation with Dirk Hohndel
More on Rust as Linus Torvalds talks about Rust in the kernel
Ubuntu’s chiselled containers
I linked to the InfoQ coverage here as it has more information about existing solutions and issues. You can read Canonical’s post here.
I got to know that the website highscalability.com is for sale. This site curates web scalability related news and stories. I’ve learnt a lot from this website over the last 15 years. The archive remains a goldmine. Check out the All Time Favourites if you have not.
I hope it finds a new home soon.
Engineering Stories
How We’re Making Roblox’s Infrastructure More Efficient and Resilient
Roblox’s well-written story of how their 2021 outage led to the adoption of a failover datacenter, followed by a “cellular” model which creates redundancy and isolation within data centers. Each “cell” has ~1400 machines. I particularly like their philosophy of adopting infrastructure-as-code for the entire cell architecture and then migrating existing services to it.
Not all services currently meet these requirements, so we’ve worked to help service owners meet them where possible, and we’ve built new tools to make it easy to migrate services into cells when ready
They expect the migration to go on till 2025 🙇🏽
Container Network Packet Drop in AKS
A network debugging story.
DHH’s write up on 37signals’ move out of the cloud
The savings look amazing, but don’t assume that this strategy will work for everyone.
Monitoring
Frontend Observability Explained
An introduction to observability for the frontend. It would have been nice to see a few real examples.
Security
Firmware attack based on pre-OS Logo images
Another interesting attack:
LogoFAIL involves logos, specifically those of the hardware seller that are displayed on the device screen early in the boot process, while the UEFI is still running. Image parsers in UEFIs from all three major IBVs are riddled with roughly a dozen critical vulnerabilities that have gone unnoticed until now. By replacing the legitimate logo images with identical-looking ones that have been specially crafted to exploit these bugs, LogoFAIL makes it possible to execute malicious code at the most sensitive stage of the boot process, which is known as DXE, short for Driver Execution Environment.
Tools
KubeStateWatch
“KubeStateWatch is a state monitor for k8s used to send notification with when and what exactly was changed to multiple channels.”
Sysaidmin
A “GPT-powered sysadmin” that can run commands on your system to figure out what’s wrong.
I tried this:
sysaidmin "My USB keyboard keeps disconnecting"
and it walked through a few commands (asking for permission before executing each one), ending with changing the power control setting for all USB buses to “on”.
A better, future version would probably ask me to change the setting only for the keyboard and not all devices.
Khaos
“A lightweight kubernetes operator to test cluster and application resilience via chaos engineering”
Netchecks
“A set of tools for testing network conditions and asserting that they are as expected.”
This is available as a Kubernetes operator and a CLI tool.
How To Deploy an Application Using K8Studio
A GUI to manage Kubernetes clusters.
A Service Level Calculator to play around with SLI/SLO/Error Budgets.
If you want to know more about these concepts, the Google SRE book is a great starting point.
KubeMarine
“An open source, lightweight and powerful management tool built for end-to-end Kubernetes cluster deployment and maintenance.”
This looks like it can be a powerful tool if you are self-managing Kubernetes.
Sleepcycles
"Automatically schedule to shutdown Deployments, CronJobs, StatefulSets and HorizontalPodAutoscalers that occupy resources in your cluster and wake them up only when you need them, reducing that way the overall power consumption”
Handy when you have predictable workload patterns.
Kamaji
A Kubernetes Control Plane Manager that “makes running multiple Control Planes cheaper and easier to deploy and operate”.
Thank you for reading. If you have not subscribed to this newsletter yet, you can do so by clicking on the button below.
Until next year 👋
Photo by Kalle Kortelainen on Unsplash.