Engineering News Feed
Curated daily updates from Netflix, Meta, LinkedIn, Uber, Airbnb, Discord, Cloudflare, and GitHub. Stories are grouped into Web Engineering, Cloud/DevOps, AI/ML, Security, Data Systems, and Infrastructure.
FFmpeg is truly a multi-tool for media processing. As an industry-standard tool it supports a wide variety of audio and video codecs and container formats. It can also orchestrate complex chains of filters for media editing and manipulation. For the people who use our apps, FFmpeg plays an important role in enabling ne
Read OriginalMeta recognizes the long-term benefits of jemalloc, a high-performance memory allocator, in its software infrastructure. We are renewing focus on jemalloc, aiming to reduce maintenance needs and modernize the codebase while continuing to evolve the allocator to adapt to the latest hardware and workloads. We are committ
Read OriginalAuthors: Harshad Sane , Andrew Halaney Imagine this — you click play on Netflix on a Friday night and behind the scenes hundreds of containers spring to action in a few seconds to answer your call. At Netflix, scaling containers efficiently is critical to delivering a seamless streaming experience to millions of member
Read OriginalWe are open-sourcing the initial version of RCCLX – an enhanced version of RCCL that we developed and tested on Meta’s internal workloads. RCCLX is fully integrated with Torchcomms and aims to empower researchers and developers to accelerate innovation, regardless of their chosen backend. Communication patterns for AI
Read Original2025 was a big year for research at Airbnb, as we made significant progress toward our mission to use AI, data science, and machine learning to become the best travel and living platform. Specifically, we doubled down on our presence at long-standing venues like KDD and CIKM — two of the most selective conferences in m
Read OriginalAvneesh Saluja , Santiago Castro , Bowei Yan , Ashish Rastogi Introduction Netflix’s core mission is to connect millions of members around the world with stories they’ll love. This requires not just an incredible catalog, but also a deep, machine-level understanding of every piece of content in that catalog, from the b
Read OriginalHow Airbnb ships dynamic config changes safely and reliably By Cosmo Qiu , Bo Teng , Siyuan Zhou , Ankur Soni , Willis Harvey Dynamic configuration is a core infrastructure capability in modern systems. It allows developers to change runtime behavior without restarting or redeploying services, even as the number of ser
Read OriginalBaolin Li , Lingyi Liu , Binh Tang , Shaojing Li Introduction Pre-training gives Large Language Models (LLMs) broad linguistic ability and general world knowledge, but post-training is the phase that actually aligns them to concrete intents, domain constraints, and the reliability requirements of production environment
Read OriginalRam Srivasta Kannan, Wale Akintayo , Jay Bharadwaj , John Crimmins , Shengwei Wang, Zhitao Zhu Introduction In 2024, the Online Data Stores team at Netflix conducted a comprehensive review of the relational database technologies used across the company. This evaluation examined functionality, performance, and total cos
Read OriginalAnna Sulkina has always been a traveler, and we’re lucky her travels have brought her to Airbnb. Anna is a Senior Director of Engineering, and she’s responsible for Application & Cloud infrastructure. She brings over two decades of industry experience to Airbnb, including work spanning the stack from the frontend t
Read OriginalWe’re sharing details of the role backend aggregation (BAG) plays in building Meta’s gigawatt-scale AI clusters like Prometheus. BAG allows us to seamlessly connect thousands of GPUs across multiple data centers and regions. Our BAG implementation is connecting two different network fabrics – Disaggregated Schedule Fab
Read OriginalPublic school to PhD The story of Airbnb’s Head Economist for Policy and Director of Data Science involves geology, co-teaching with a Nobel Prize winner, and CSI. (No, not the hit TV franchise.) Peter Coles was born and raised in Milwaukee, Wisconsin. He studied math at Princeton, earned his PhD in economics at Stanfo
Read OriginalThe AI Evolution of Graph Search at Netflix: From Structured Queries to Natural Language By Alex Hutter and Bartosz Balukiewicz Our previous blog posts ( part 1 , part 2 , part 3 ) detailed how Netflix’s Graph Search platform addresses the challenges of searching across federated data sets within Netflix’s enterprise e
Read OriginalHow Airbnb rolled out 20+ locally relevant payment methods worldwide in just 14 months By: Gerum Haile , Bo Shi, Yujia Liu , Yanwei Bai , Bo Yuan , Rory MacQueen , Yixia Mao Across the more than 220 global markets that Airbnb operates in, cards are the primary way that guests pay for stays, experiences, and services. H
Read OriginalBy Jacob Meyers and Rob Zienert Temporal is a Durable Execution platform which allows you to write code “as if failures don’t exist”. It’s become increasingly critical to Netflix since its initial adoption in 2021, with users ranging from the operators of our Open Connect global CDN to our Live reliability teams now de
Read OriginalXiaomei Liu , Joseph Lynch , Chris Newton Introduction Behind the Streams: Building a Reliable Cloud Live Streaming Pipeline for Netflix introduced the architecture of the streaming pipeline. This blog post looks at the custom Origin Server we built for Live — the Netflix Live Origin. It sits at the demarcation point b
Read OriginalAV1 — Now Powering 30% of Netflix Streaming Liwei Guo , Zhi Li , Sheldon Radford , Jeff Watts Streaming video has become an integral part of our daily lives. At Netflix, our top priority is delivering the best possible entertainment experience to our members, regardless of their devices or network conditions. One of th
Read OriginalHow Airbnb combines GraphQL infra, product context, and LLMs to generate and maintain convincing, type-safe mock data using a new directive. Introduction Producing valid and realistic mock data for testing and prototyping with GraphQL has been a persistent challenge across the industry for years. Mock data is tedious t
Read OriginalHow Airbnb hardened Mussel, our key-value store, with smarter traffic controls to stay fast and reliable during traffic spikes. By Shravan Gaonkar , Casey Getz , Wonhee Cho Introduction Every request lookup on Airbnb, from stays, experiences, and services search to customer support inquiries ultimately hits Mussel , ou
Read OriginalInternal Editorial