<!DOCTYPE html><html lang="en"><head><meta http-equiv="Content-Type" content="text/html charset=UTF-8"><meta charset="UTF-8"><meta name="viewport" content="width=device-width"><meta name="x-apple-disable-message-reformatting"><title>TLDR Data</title><meta name="color-scheme" content="light dark"><meta name="supported-color-schemes" content="light dark"><style type="text/css">
:root {
color-scheme: light dark; supported-color-schemes: light dark;
}
*,
*:after,
*:before {
-webkit-box-sizing: border-box; -moz-box-sizing: border-box; box-sizing: border-box;
}
* {
-ms-text-size-adjust: 100%; -webkit-text-size-adjust: 100%;
}
html,
body,
.document {
width: 100% !important; height: 100% !important; margin: 0; padding: 0;
}
body {
-webkit-font-smoothing: antialiased; -moz-osx-font-smoothing: grayscale; text-rendering: optimizeLegibility;
}
div[style*="margin: 16px 0"] {
margin: 0 !important;
}
table,
td {
mso-table-lspace: 0pt; mso-table-rspace: 0pt;
}
table {
border-spacing: 0; border-collapse: collapse; table-layout: fixed; margin: 0 auto;
}
img {
-ms-interpolation-mode: bicubic; max-width: 100%; border: 0;
}
*[x-apple-data-detectors] {
color: inherit !important; text-decoration: none !important;
}
.x-gmail-data-detectors,
.x-gmail-data-detectors *,
.aBn {
border-bottom: 0 !important; cursor: default !important;
}
.btn {
-webkit-transition: all 200ms ease; transition: all 200ms ease;
}
.btn:hover {
background-color: #f67575; border-color: #f67575;
}
* {
font-family: Arial, Helvetica, sans-serif; font-size: 18px;
}
@media screen and (max-width: 600px) {
.container {
width: 100%; margin: auto;
}
.stack {
display: block!important; width: 100%!important; max-width: 100%!important;
}
.btn {
display: block; width: 100%; text-align: center;
}
}
body,
p,
td,
tr,
.body,
table,
h1,
h2,
h3,
h4,
h5,
h6,
div,
span {
background-color: #FEFEFE !important; color: #010101 !important;
}
@media (prefers-color-scheme: dark) {
body,
p,
td,
tr,
.body,
table,
h1,
h2,
h3,
h4,
h5,
h6,
div,
span {
background-color: #27292D !important; color: #FEFEFE !important;
}
}
a {
color: inherit !important; text-decoration: underline !important;
}
</style><!--[if mso | ie]>
<style type="text/css">
a {
background-color: #FEFEFE !important; color: #010101 !important;
}
@media (prefers-color-scheme: dark) {
a {
background-color: #27292D !important; color: #FEFEFE !important;
}
}
</style>
<![endif]--></head><body class="">
<div style="display: none; max-height: 0px; overflow: hidden;">Cursor turned Faire’s analytics work into an AI-native workflow by combining SQL generation, codebase search, and context from multiple systems </div>
<div style="display: none; max-height: 0px; overflow: hidden;">
<br>
</div>
<table align="center" class="document"><tbody><tr><td valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" class="container" width="600"><tbody><tr class="inner-body"><td>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr class="header"><td bgcolor="" class="container">
<table width="100%"><tbody><tr><td class="container">
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" style="margin-top: 0px;" width="100%"><tbody><tr><td style="padding: 0px;">
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div style="text-align: center;">
<span style="margin-right: 0px;"><a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Ftldr.tech%2Fdata%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/JYwO9ZwYmvYODRpBbg9OpULTax64y-MarmVgq4TBmXI=430" rel="noopener noreferrer" target="_blank"><span>Sign Up</span></a>
|<span style="margin-right: 2px; margin-left: 2px;"><a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fadvertise.tldr.tech%3Futm_source=tldrdata%26utm_medium=newsletter%26utm_campaign=advertisetopnav/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/j2CUqvEIwqLOdwdhDAucxbKAvWAOSS4-VmS5JMzeZpM=430" rel="noopener noreferrer" target="_blank"><span>Advertise</span></a></span>|<span style="margin-left: 2px;"><a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fa.tldrnewsletter.com%2Fweb-version%3Fep=1%26lc=1670a604-84b7-11f0-bcf5-55fc1d40139c%26p=88107e1c-bac5-11f0-94b8-41a317feaf72%26pt=campaign%26t=1762427197%26s=44773a0da9c7ea417a879b9fe237f8affadf9972eaed805beb6bc712619b3412/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/3I8mMkNNBQKbOwKhHVxhwLp973oxTaHOgIMJdvdr70U=430"><span>View Online</span></a></span>
<br>
</span></div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="text-align: center;"><span data-darkreader-inline-color="" style="--darkreader-inline-color:#3db3ff; color: rgb(51, 175, 255) !important; font-size: 30px;">T</span><span style="font-size: 30px;"><span data-darkreader-inline-color="" style="color: rgb(232, 192, 96) !important; --darkreader-inline-color:#e8c163; font-size:30px;">L</span><span data-darkreader-inline-color="" style="color: rgb(101, 195, 173) !important; --darkreader-inline-color:#6ec7b2; font-size:30px;">D</span></span><span data-darkreader-inline-color="" style="--darkreader-inline-color:#dd6e6e; color: rgb(220, 107, 107) !important; font-size: 30px;">R</span>
<br>
</td></tr></tbody></table>
<br>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr id="together-with"><td align="center" height="20" style="vertical-align:middle !important;" valign="middle" width="100%"><strong style="vertical-align:middle !important; height: 100%;">Together With </strong>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fpages.awscloud.com%2Fawsmp-mss-databricks-data-intelligence-platform-aws-marketplace.html%3Ftrk=f08c48d0-faa3-455f-bc1c-00bc12700599%26sc_channel=el/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/9TktEVSAyGg3npqxKtHPQFepPmJzlNdXtYHTGU7cK3Q=430"><img src="https://images.tldr.tech/awsroidna.png" valign="middle" style="vertical-align: middle !important; height: 100%;" alt="AWS"></a></td></tr></tbody></table>
<table style="table-layout: fixed; width:100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;">
<div style="text-align: center;">
<h1><strong>TLDR Data <span id="date">2025-11-06</span></strong></h1>
</div>
</td></tr></tbody></table>
<table style="table-layout: fixed; width:100%;" width="100%"><tbody><tr id="sponsy-copy"><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fpages.awscloud.com%2Fawsmp-mss-databricks-data-intelligence-platform-aws-marketplace.html%3Ftrk=f08c48d0-faa3-455f-bc1c-00bc12700599%26sc_channel=el/2/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/qWG7oeFe0j_NWrhG5pNHXVrdy_lnY-y2MbiSZZBaF-I=430">
<span>
<strong>Accelerate data and AI workflows with Databricks (Sponsor)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Disconnected tools slow down development. <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fpages.awscloud.com%2Fawsmp-mss-databricks-data-intelligence-platform-aws-marketplace.html%3Ftrk=f08c48d0-faa3-455f-bc1c-00bc12700599%26sc_channel=el/3/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/OxVieuc15oVlLje3xGZoufFFbUHl4tAxI0q31JF8fIM=430" rel="noopener noreferrer nofollow" target="_blank"><span>Databricks Data Intelligence Platform</span></a> unifies data engineering, analytics, and ML on a single lakehouse built on Amazon S3.
<p></p>
<p>Explore how you can use <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fpages.awscloud.com%2Fawsmp-mss-databricks-data-intelligence-platform-aws-marketplace.html%3Ftrk=f08c48d0-faa3-455f-bc1c-00bc12700599%26sc_channel=el/4/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/ZPAyXNoh__q4AZDHF7WCPV-l3NDdVjSR5-PPs6aMICo=430" rel="noopener noreferrer nofollow" target="_blank"><span>Databricks</span></a> for vector indexing, embedding generation, and RAG deployment—all within your AWS environment. Technical resources walk you through integrations with Amazon Redshift, Amazon Bedrock, and SageMaker to simplify pipelines, optimize security, and move AI apps into production faster.</p>
<p><a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Faws.amazon.com%2Fmarketplace%2Fpp%2Fprodview-wtyi5lgtce6n6%3Ftrk=d2bfe029-6e4c-4327-aca8-8a73df98ef4a%26sc_channel=el/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/NmXJcjBIUakXjc3CjmiZvWgbXRsNbExbrVLAFCkV6QM=430" rel="noopener noreferrer nofollow" target="_blank"><span>Start building today with a 14-day free trial in AWS Marketplace.</span></a>
</p>
</span></span></div>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr>
<tr bgcolor=""><td class="container">
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td style="padding: 0px;">
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">📱</span></div></div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;">
<h1><strong>Deep Dives</strong></h1>
</div>
</div>
</td></tr></tbody></table>
<table style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fmedium.com%2Fpinterest-engineering%2Fa-decade-of-ai-platform-at-pinterest-4e3b37c0f758%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/TElTz_KSG3Px19r57pfsV9p5RAgXvQ4sF1b6nSaFuIU=430">
<span>
<strong>A Decade of AI Platform at Pinterest (15 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Pinterest evolved its AI infrastructure from fragmented, team-specific ML stacks to a unified platform supporting hundreds of millions of inferences per second, enabled by tiered abstractions for features definition, storage, representation, and training frameworks. Key advances included transitioning to GPU-based inference (achieving 100x larger model capacity without increasing cost or latency), scaling architecture with Ray and Model Farm, and standardizing on PyTorch. Success stemmed from balancing local innovation and unification timing with efficiency, iteration velocity, and enablement.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fengineering.prod.whoop.com%2Fglacierbase%2F%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/Ew1Yp_-RzIqKGK88t5x7xgvaU0efiBINgqMSvbWsGX8=430">
<span>
<strong>Glacierbase: Managing Iceberg Schema Migrations at Scale (7 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
WHOOP's data platform manages large Iceberg tables, where schema inconsistencies can cause inefficiencies such as excessive compute or wasted reads. Glacierbase, inspired by Liquibase but tailored for open formats, standardizes migrations for high-value "silver" and "gold" layer tables (e.g., feature tables and ML datasets), excluding raw ingestion or stable CDC tables.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.datadoghq.com%2Fblog%2Fengineering%2Fcdc-replication-search%2F%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/m6Zn3zlWMJPH7epyZ2lwmgykzPma1WjoCXdbGDUdbhk=430">
<span>
<strong>Replication Redefined: How We Built a Low-latency, Multi-tenant Data Replication Platform (7 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Datadog encountered scaling challenges with a shared PostgreSQL database managing both OLTP and complex search queries. To address this, it implemented asynchronous Change Data Capture using Debezium to detect changes in PostgreSQL, streaming via Kafka, and leveraging Kafka Connect for sinks to a dedicated search platform, while denormalizing data during replication to optimize faceted searches and aggregations.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">🚀</span></div>
</div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;">
<h1><strong>Opinions & Advice</strong></h1>
</div>
</div>
</td></tr></tbody></table>
<table style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fcraft.faire.com%2Fwe-moved-analytics-into-an-ide-and-havent-looked-back-f6e0c249cc42%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/5M6jnSu_K4-gR1vR8-N92KDeP9yQ6ePKucCmlYX1mLM=430">
<span>
<strong>We Moved Analytics into an IDE — and Haven't Looked Back (8 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Cursor turned Faire's analytics work into an AI-native workflow by combining SQL generation, codebase search, and context from multiple systems in a single IDE, which cut analysis time from days to hours. Adoption required onboarding, QA habits, and visible wins, but once analysts saw that they could query data, inspect ETL logic, and generate runnable SQL in one place, usage grew rapidly. Key insight for data professionals: the future of analytics is not just AI-assisted, it is AI-native, where analysts interrogate code and data directly and spend more time on decisions, not searching for context.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fbigdata.2minutestreaming.com%2Fp%2Fevent-streaming-is-topping-out%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/BHcmVMU-xd6roiEgnG-mPVXPT2IDUMO5dBYrOhgbum8=430">
<span>
<strong>Event Streaming is Topping Out (14 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
The event streaming market (led by Kafka/Confluent) is saturated, growth is slowing, prices are collapsing, and too many vendors are chasing a small demand. The author predicts heavy consolidation and pivots because Apache Kafka will remain, but many streaming companies and business models around it will not survive.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.confessionsofadataguy.com%2Fcluster-fatigue-polars-and-pyarrow-to-postgres-and-apache-iceberg-streaming-mode%2F%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/S4LUWXZ1E_aR0ZDg8bkzntQlIIvFiKrvI0M3V6n8-qg=430">
<span>
<strong>Cluster Fatigue. Polars and PyArrow to Postgres and Apache Iceberg (streaming mode) (5 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
"Cluster fatigue" refers to the exhaustion and costs associated with managing distributed compute clusters for data processing. To combat this, the strategy involves migrating from resource-intensive distributed workflows to leaner, local or small-scale compute options leveraging tools like Polars and PyArrow for managing massive datasets, facilitating streaming ingestion into Postgres and writes to Apache Iceberg tables in streaming mode.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fmotherduck.com%2Fblog%2Fdata-engineers-answer-10-top-reddit-questions%2F%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/KpNG-2yjEI_kpwYS8JEU1szBZvptlLxMXPJvIX4qr8M=430">
<span>
<strong>4 Senior Data Engineers Answer 10 Top Reddit Questions (27 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Insights from four veteran data engineers emphasize selecting data warehouses versus lakes or lakehouses according to practical constraints like budget, deadlines, and team expertise, challenging initial demands for "real-time" processing to sidestep undue complexity, and initiating data quality efforts with straightforward tests, refining them via failure-driven iterations while prioritizing business understanding, using techniques such as write-audit-publish (WAP).
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">💻</span></div>
</div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;">
<h1><strong>Launches & Tools</strong></h1>
</div>
</div>
</td></tr></tbody></table>
<table style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fgithub.com%2Fcoginiti-dev%2FQuackStore%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/2QZkrSV1HWuxt_hSzRLTh_Kbdcp0mX9NVeirfS-3ru4=430">
<span>
<strong>QuackStore (GitHub Repo)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
The QuackStore extension automatically stores frequently accessed file portions in a local, block-based cache. This dramatically reduces load times for repeated queries on the same data.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fthenewstack.io%2Fapache-arrows-final-frontier-replacing-outdated-database-drivers%2F%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/2nosPmz3cssfFiRE61iUHjjct7bbaIoDtUYIp8j56gU=430">
<span>
<strong>Apache Arrow's Final Frontier: Replacing Outdated Database Drivers (3 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Columnar, founded by core Apache Arrow contributors, has launched with $4 million in seed funding to address performance bottlenecks in analytical data transfers caused by legacy row-based protocols like ODBC and JDBC. Its new ADBC drivers, built on the Arrow columnar format, deliver over 90% query time reductions in some cases and eliminate costly serialization overhead. Supported targets include Redshift, MySQL, SQL Server, and Trino.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fcloud.google.com%2Fblog%2Fproducts%2Fdata-analytics%2Fexploring-the-data-engineering-agent-in-bigquery%2F%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/HreGWx0T5qNkzojNbkbnnBvrZkJ4wHF_5LtSW88w8hs=430">
<span>
<strong>The Data Engineering Agent is Now in Preview (5 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Google has launched the Data Engineering Agent in BigQuery, a Gemini-powered tool that automates complex data engineering tasks, including pipeline creation, transformation, modeling, troubleshooting, and migration, using natural language prompts and best-practice automation.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">🎁</span></div></div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><strong><h1>Miscellaneous</h1></strong></div>
</div>
</td></tr></tbody></table>
<table bgcolor="" style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Ftimodechau.com%2Fpragmatic-orthodoxy-data-signals-1-03-11-25%2F%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/Ljhl88X9NeGb2aRVXrvr9QwI-2xF7lwRIs2WrqgCpiY=430">
<span>
<strong>Pragmatic Orthodoxy - Data Signals #1 - 03.11.25 (6 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Some pragmatic shifts are underway in data architecture: teams prioritize data replication over sophisticated history techniques, or virtualized, view-based models, materializing only proven hot paths (resources are cheap - your time is not). Semantic clarity and domain-driven modeling enable projecting various schema forms from a single semantic foundation, accelerating adaptation for new use cases. These approaches value measured investment in complexity, optimizing for human and operational efficiency rather than hypothetical future needs or technology limitations.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fmartinfowler.com%2Farticles%2Fpatterns-of-distributed-systems%2F%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/1VzrXMR9q8pA0Sb1Rdcs2FJTbn-rorrxcn5M7-Cscmc=430">
<span>
<strong>Catalog of Patterns of Distributed Systems (Website)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
Patterns of Distributed Systems catalogs over 25 foundational architectural patterns addressing consistency, synchronization, partitioning, replication, and coordination challenges in distributed environments. Key techniques explored include clock-based ordering, leader election, majority quorum, idempotency, log replication, partition management, and hybrid clocks. These proven methods enable reliable data storage, efficient messaging, and robust state management in distributed data platforms.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;"><span style="font-size: 36px;">⚡</span></div></div>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding-top: 0px; padding-bottom: 0px;">
<div class="text-block">
<div style="text-align: center;">
<h1><strong>Quick Links</strong></h1>
</div>
</div>
</td></tr></tbody></table>
<table bgcolor="" style="table-layout: fixed; width: 100%;" width="100%"><tbody><tr><td style="padding:0;border-collapse:collapse;border-spacing:0;margin:0;" valign="top">
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fgithub.com%2Fthedatagata%2Fboring-ducklake-semantic-fishing-demo%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/to1Pl9Q5ruiNVZdc97EYsMxdkv4eX_eZF_Nk4VL51SM=430">
<span>
<strong>Boring Ducklake Semantic Fishing Demo (GitHub Repo)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
A multi-stage pipeline ingests and transforms the Google Analytics dataset into a structured fact table and semantic layer, enabling fast querying and analysis in DuckDB/MotherDuck.
</span>
</span>
</div>
</td></tr></tbody></table>
<table align="center" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block">
<span>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fclickhouse.com%2Fblog%2Flibrechat-open-source-agentic-data-stack%3Futm_source=tldrdata/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/wLvi6NOC8-yq3HMcE7ReRWOXw2gcEwB8JzmO1vJTLoI=430">
<span>
<strong>ClickHouse Welcomes LibreChat: Introducing the Open-Source Agentic Data Stack (7 minute read)</strong>
</span>
</a>
<br>
<br>
<span style="font-family: "Helvetica Neue", Helvetica, Arial, Verdana, sans-serif;">
ClickHouse has acquired LibreChat, integrating its open-source, multi-LLM chat platform as a core component of a unified Agentic Data Stack for agent-facing analytics.
</span>
</span>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td align="left" style="word-break: break-word; vertical-align: top; padding: 5px 10px;">
<p style="padding: 0; margin: 0; font-size: 22px; color: #000000; line-height: 1.6; font-weight: bold;">
Want to advertise in TLDR? 📰
</p>
<div class="text-block" style="margin-top: 10px;">
If your company is interested in reaching an audience of data engineering professionals and decision makers, you may want to <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fadvertise.tldr.tech%2F%3Futm_source=tldrdata%26utm_medium=newsletter%26utm_campaign=advertisecta/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/eTqqUZn_SGifyJGy26eXVSpXnPc24XQ_DJMBdXd6vkA=430"><strong><span>advertise with us</span></strong></a>.
</div>
<br>
<!-- New "Want to work at TLDR?" section -->
<p style="padding: 0; margin: 0; font-size: 22px; color: #000000; line-height: 1.6; font-weight: bold;">
Want to work at TLDR? 💼
</p>
<div class="text-block" style="margin-top: 10px;">
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fjobs.ashbyhq.com%2Ftldr.tech/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/9QBwh5HcIvgHdxPZkfSUb93CiDUxS-keVlwU-3CksWk=430" rel="noopener noreferrer" style="color: #0000EE; text-decoration: underline;" target="_blank"><strong>Apply here</strong></a> or send a friend's resume to <a href="mailto:jobs@tldr.tech" style="color: #0000EE; text-decoration: underline;">jobs@tldr.tech</a> and get $1k if we hire them!
</div>
<br>
<div class="text-block">
If you have any comments or feedback, just respond to this email!
<br>
<br> Thanks for reading,
<br>
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.linkedin.com%2Fin%2Fjoelvanveluwen%2F/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/YhZJO9l1UFuJy4v_Jkc0UJYAIlO2Wnz3XvLP0o_4vG0=430"><span>Joel Van Veluwen</span></a>, <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.linkedin.com%2Fin%2Fjennytzurueyching%2F/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/y7Lqe6Og0WLquT-rDNRPFahef2JLYwAIIUHcUo5nL8M=430"><span>Tzu-Ruey Ching</span></a> & <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fwww.linkedin.com%2Fin%2Fremi-turpaud%2F/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/kswgNMaZtoTxLAUhNHu_v4iyHThlh5H6e4gBehRuVp0=430"><span>Remi Turpaud</span></a>
<br>
<br>
</div>
<br>
</td></tr></tbody></table>
<table align="center" bgcolor="" border="0" cellpadding="0" cellspacing="0" width="100%"><tbody><tr><td class="container" style="padding: 15px 15px;">
<div class="text-block" id="testing-id">
<a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Ftldr.tech%2Fdata%2Fmanage%3Femail=silk.theater.56%2540fwdnl.com/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/ECaO_gJWIWNu2yRAyuYAJzdoWO09lL4wni2pFEssAhY=430">Manage your subscriptions</a> to our other newsletters on tech, startups, and programming. Or if TLDR Data isn't for you, please <a href="https://tracking.tldrnewsletter.com/CL0/https:%2F%2Fa.tldrnewsletter.com%2Funsubscribe%3Fep=1%26l=037ede50-92cc-11ee-b0f2-b761aa2217ad%26lc=1670a604-84b7-11f0-bcf5-55fc1d40139c%26p=88107e1c-bac5-11f0-94b8-41a317feaf72%26pt=campaign%26pv=4%26spa=1762426873%26t=1762427197%26s=6edc5652a8bb03720c7e2b896eef99ffe2ea259ae21d4693d93ce58e5732d4f3/1/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/D1tcIPoDz9Yj52KNK8Fb6GprCPZmTtobwtjoIzZhLTk=430">unsubscribe</a>.
<br>
</div>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr></tbody></table>
</td></tr></tbody></table>
<img alt="" src="http://tracking.tldrnewsletter.com/CI0/0100019a58d8d967-94455944-2aa1-4cea-a566-18c222d07d15-000000/1UsbmQE54_cTbzLpWdqBl9Ji2lIxeN8YH5gxxkf1mv8=430" style="display: none; width: 1px; height: 1px;">
</body></html>