The Unspoken Challenges of Deploying to Customer Clouds

Adventures in DevOps

This episode we are joined by Andrew Moreland, co-founder of Chalk. Andrew explains how their company’s core business model is to deploy their software directly into their customers’ cloud environments. This decision was driven by the need to handle highly sensitive data, like PII and financial records, that customers don't want to hand over to a third-party startup.
The conversation delves into the surprising and complex challenges of this approach, which include managing granular IAM permissions and dealing with hidden global policies that can block their application. Andrew and Warren also discuss the real-world network congestion issues that affect cross-cloud traffic, a problem they've encountered multiple times. Andrew shares Chalk's mature philosophy on software releases, where they prioritize backwards compatibility to prevent customer churn, which is a key learning from a competitor.
Finally, the episode explores the advanced technical solutions Chalk has built, such as their unique approach to "bitemporal modeling" to prevent training bias in machine learning datasets. As well as, the decision to move from Python to C++ and Rust for performance, using a symbolic interpreter to execute customer code written in Python without a Python runtime. The episode concludes with picks, including a surprisingly popular hobby and a unique take on high-quality chocolate.
Notable Facts

• Fact - The $1M hidden Kubernetes spend
• Giraffe and Medical Ruler training data bias
• SOLID principles don't produce better code?
• Veritasium - The Hole at the Bottom of Math
• Episode: Auth Showdown on backwards compatible changes

Picks:

• Warren - Switzerland Grocery Store Chocolate
• Andrew - Trek E-Bikes