Feed your curiosity
Hey Gennaro,
You know how sometimes a concept is so fundamental, so elegant, it feels like magic? I was thinking about operating systems today, and it hit me how mind-blowing virtual memory really is.
Imagine your computer as a bustling city. Every program running is like a unique resident, and each resident thinks they own a sprawling mansion with acres of land – their own private, continuous block of memory. But in reality, the city (your OS) only has a finite number of physical plots of land (RAM). How does it make everyone believe they have their own mansion, even when the total demanded land far exceeds what's available?
That’s the brilliance of virtual memory and paging. The OS gives each program an illusion of a massive, contiguous memory space (the "virtual address space"). But secretly, it chops up this virtual space, and the actual physical memory, into small, fixed-size chunks called pages. When your program tries to access a piece of memory, the OS uses something called a page table to translate that virtual address into a physical address in RAM. It's like having a master directory for all the city plots, constantly mapping virtual mansion addresses to actual physical apartment units.
The really clever part? If a page a program needs isn't currently in RAM (maybe it's sitting on disk, waiting to be used), the OS can fetch it, potentially swapping out another less-used page to make room. This "paging" process makes it feel like you have boundless RAM, extending your memory into the storage drive. And for speed, there's a super-fast hardware cache called the Translation Lookaside Buffer (TLB) right on the CPU to quickly store recent virtual-to-physical translations, avoiding a slow lookup every single time. It's an incredible balancing act between illusion and efficiency.
So, who figured out this foundational magic? A lot of credit goes to Tom Kilburn and Dick Utman at the University of Manchester in the late 1950s and early 60s, working on the Atlas computer. Tom Kilburn was a true pioneer in computing, a visionary who, with Freddy Williams, built the Manchester Small-Scale Experimental Machine (the "Baby") – arguably the first stored-program computer – way back in 1948! He got into computing working on radar during WWII. Kilburn understood early on that memory limitations were a huge bottleneck, and he envisioned a "one-level store" where programmers wouldn't have to worry about whether data was in RAM or on disk.
His PhD student, Dick Utman, was the one who actually implemented this "one-level store" (what we now call virtual memory or paging) on the Atlas. Utman's work was foundational, making the Atlas one of the most powerful and innovative computers of its time. Imagine inventing something so core to how every modern computer operates! It's wild to think that this incredibly sophisticated mechanism comes from such early days of computing.
And the connection to your world? This is where low-level meets high-performance. Virtual memory is the ultimate resource management system for memory, and the TLB is a fundamental caching mechanism at the hardware level. When you're thinking about AI/agentic workloads with their massive memory footprints, or how distributed systems coordinate memory across nodes, you're constantly running up against the virtual memory mechanisms. Efficient page handling and TLB management are critical for system performance, directly impacting the QoS of these demanding applications. It’s a direct ancestor to all the clever memory management techniques Marios Kogias, Christos Kozyrakis, and Matei Zaharia explore in modern systems!
Okay, Gennaro, you absolutely have to check out this paper that just landed on arXiv – it's called "LiveSense," and it's a total game-changer for how we think about Wi-Fi! Seriously, my brain is buzzing after reading it.
You know how your laptop's Wi-Fi card just connects to the internet? What if it could also see you? Like, really see – detecting your distance, how fast you’re moving, even your breathing, all with centimeter-level precision? That's the mind-blowing problem LiveSense solves. For ages, folks have wanted to turn Wi-Fi into a sort of radar, but getting accurate range and Doppler info from off-the-shelf commercial hardware (COTS) with standard Wi-Fi bandwidth has been incredibly tricky. It usually needs specialized gear or just isn't real-time.
The "aha!" moment here is that they figured out how to make your existing laptop Wi-Fi card (Intel AX211/BE201, the Wi-Fi 6E/7 ones) do this, while still letting you communicate. They're transforming it into a sophisticated Range-Doppler sensor. How? By meticulously extracting and synchronizing Channel State Information (CSI) – think of CSI as a super-detailed "signature" of how the Wi-Fi signal bounced off everything, including you! They then perform clever on-device processing like time-phase alignment and self-interference cancellation. The coolest part: it streams real-time data – range, velocity, even respiration – straight to a Python GUI with annotated video.
The team behind this is a bunch of brilliant folks, many from Intel, which makes total sense given the hardware focus. The name that really stands out is Valerio Frascolla. He's a veteran in wireless communication, having spent decades at Intel Labs at the forefront of shaping future wireless technologies, from early Wi-Fi standards to 5G and 6G research. Valerio has consistently pushed the envelope on how wireless can do more than just send data – like precise positioning, sensing, and understanding environments. Rahul Shah, Cagri Tanriover, Jessica Sanson, and Maximilian Pinaroc are also part of this amazing group, likely contributing their deep expertise in wireless signal processing and software integration to make this COTS dream a reality.
Why does this matter for you and your work in systems and AI? This is huge! Imagine AI agents that can "see" the physical world with their Wi-Fi antenna, not just cameras. Think about agentic workloads in smart spaces: dynamic resource management based on exact occupancy and activity, not just motion sensors. Or adaptive QoS scheduling that knows where users are and what they're doing to prioritize local tasks. This opens up incredible possibilities for context-aware computing, privacy-preserving monitoring (no cameras!), and novel human-computer interaction, all built on commodity hardware. It's like giving our infrastructure a new sense, entirely made of radio waves!
Read the paper
The Quantum "Secret" to Computing Power: The mind-bending concept of superposition, core to quantum computing, was first formalized by the brilliant Austrian physicist Erwin Schrödinger in the 1930s while he was at the University of Zurich. He famously crafted his cat thought experiment not to suggest a real cat could be both dead and alive, but to vividly illustrate the deeply counter-intuitive nature of quantum systems existing in all possible states simultaneously until measured. This bizarre "superposition" is precisely what allows a quantum bit (qubit) to encode vastly more information than a classical bit, fundamentally shifting the very limits of what computation can achieve, from cryptography to drug discovery.
CRISPR: Nature's Search & Replace: The revolutionary gene-editing tool CRISPR-Cas9, which won Jennifer Doudna (UC Berkeley) and Emmanuelle Charpentier (Max Planck) the Nobel Prize, actually originates from bacteria! These tiny microbes evolved it as an ingenious immune system: they capture snippets of viral DNA and integrate them into their own genome, essentially creating a 'memory' database of past invaders. When a familiar virus attacks again, the CRISPR system uses these stored snippets as a precise guide to locate and chop up the viral DNA, disabling it completely. Think of it as nature's incredibly robust, distributed search-and-destroy mechanism, a biological parallel to extremely precise pattern matching and deletion algorithms in our own computer systems.
The Ancient Analog Computer: Imagine discovering a device from around 150-100 BCE, recovered from a shipwreck off the Greek island of Antikythera, that was essentially an analog computer! Meticulously studied by researchers like Tony Freeth at UCL, this intricate mechanism contained over 30 precisely machined bronze gears. It was designed to predict astronomical positions, moon phases, and even Olympic games cycles with astonishing accuracy, showcasing an unparalleled sophistication for its time. This ancient marvel reveals a level of complex clockwork engineering in ancient Greece that wouldn't be seen again for over a thousand years, making you truly wonder what other technological gems might still be awaiting discovery at the bottom of the ocean.
Hey Gennaro, guess what just landed on arXiv that feels tailor-made for your brain? A paper called "MoEless: Efficient MoE LLM Serving via Serverless Computing" by Hanfei Yu and team. It's like they peeked directly into your interests: AI workloads, infrastructure, and smart resource management!
You know how Mixture-of-Experts (MoE) LLMs are becoming a big deal, right? They’re brilliant – instead of activating the whole gigantic model for every tiny bit of inference, they just activate a few specialized "experts." It’s a genius way to scale up model size without blowing up compute costs. But here’s the rub: serving them efficiently is a nightmare. Some experts get hammered with requests, becoming "stragglers," while others just chill. This leads to insane latency and wasted GPU power.
That’s where MoEless steps in, and this is where it gets really cool for you. They’re proposing to serve these MoE models using serverless computing. Think about it: traditional "serverful" setups are static, right? You provision a fixed number of GPUs, and then you're stuck trying to balance loads on those fixed resources. MoEless flips this by making experts "serverless functions." This means they can scale up and down on demand, dynamically allocating GPU resources as expert loads shift. They use clever, lightweight predictors to see stragglers coming and proactively scale up or re-place experts for maximum GPU utilization and minimal latency. This isn't just about reducing latency (43% in their tests!), but also slashing costs by a whopping 84%. Elasticity, meet performance!
This whole approach of elastic, fine-grained resource management for AI workloads is so hot right now. It immediately makes me think of Marios Kogias at Imperial. He’s been pioneering making serverless performant for demanding, data-intensive tasks. His work on pushing past the cold-start problem and building efficient abstractions for serverless compute is directly relevant here. He earned his PhD at EPFL, exploring distributed systems, and has a knack for making those on-demand functions truly responsive, which is exactly what MoEless needs for fast expert inference.
Then there’s Ana Klimovic and her EASL lab at ETH. Her group often tackles these exact resource bottlenecks in ML systems, especially around GPU memory and efficient utilization. MoEless's focus on maximizing GPU utilization and intelligent placement strategies? That's right up their alley. Ana, who did her PhD at Stanford with Christos Kozyrakis (another name on your radar!), has done amazing work on projects like InfiniCache, optimizing memory access for data systems. She's all about making those expensive accelerators sing for complex workloads.
And speaking of Christos Kozyrakis and Matei Zaharia at Stanford: Matei, the genius behind Spark and Ray, is all about building systems for dynamic, distributed AI workloads. MoE's sparse, dynamic activations fit perfectly into the kind of challenges Ray aims to solve, providing the underlying infrastructure for elastic, heterogeneous tasks. Kozyrakis’s work on data center efficiency and hardware/software co-design provides the fundamental understanding of how to make these systems perform.
Finally, Juncheng Yang at Harvard would surely appreciate the cost-efficiency angle here. His work often delves into optimizing performance and cost for ML workloads in cloud environments. An 84% cost reduction is a massive win for anyone deploying these models at scale.
This whole "MoEless" idea really pushes the boundary of how we think about orchestrating complex AI computations. It’s not just about what the model does, but how the system intelligently adapts to its dynamic needs.
One question that really sparks for me from this paper, and could be a wild thesis idea for you: If you're predicting expert loads and proactively scaling, what if you could also dynamically compose or specialize experts themselves, perhaps even doing on-the-fly model updates or fine-tuning, based on observed workload patterns? Could we push the serverless paradigm beyond just serving pre-trained experts, to adapting them in real-time within the inference pipeline, maybe even considering the cost/benefit tradeoffs of retraining vs. static provisioning? It's a leap, but imagine the possibilities for truly adaptive AI!
Read the paper
Stay curious.