Blogs

Research behind the flywheel.

Major engineering and investment research behind the flywheel effect for enterprise developers.

The flywheel

Why owning the learning loop compounds.

6 pieces
  1. Software 2.0

    Andrej Karpathy · Medium · 2017

    Argues that more and more software will be written as neural network weights learned from data, not as hand-written code. Whoever owns the data and the training loop owns the product.

  2. The Reverse Information Paradox

    Satya Nadella · SN Scratchpad · 2026

    To get value from AI, a firm has to hand its know-how to the model provider, who ends up learning more about the customer than the other way round. His answer: keep evals, memory, usage traces and fine-tuned models inside a boundary you govern, and join them into a learning loop that compounds.

  3. Enhance Your AI Agent with Data Flywheels

    Shashank Verma, Nancy Agarwal · NVIDIA Technical Blog · 2025

    Agents lose accuracy as data, documents and APIs change, so NVIDIA builds a loop that keeps learning from production traffic. In its example, a 1B model fine-tuned with LoRA approaches the tool-calling accuracy of a model about 70 times larger.

  4. AI Is a 5-Layer Cake

    Jensen Huang · NVIDIA Blog · 2026

    Frames AI as infrastructure in five stacked layers (energy, chips, infrastructure, models and applications), each resting on the one below. He credits open-source models with driving demand across the whole stack.

  5. The Shift from Models to Compound AI Systems

    Matei Zaharia et al. · Berkeley AI Research · 2024

    The best results increasingly come from systems of models, retrieval, tools and checks working together, not from one model alone. Designing and tuning the whole system becomes the engineering problem.

  6. Building A Generative AI Platform

    Chip Huyen · 2024

    Walks through a production AI platform piece by piece: context, guardrails, model routing, caching and observability. A clear map of everything that has to sit around the model.

Sovereign AI: the investment case

Why investors and builders argue for owning intelligence.

5 pieces
  1. Own Your Intelligence: A How-to Guide for Sovereign AI

    Sonya Huang · Sequoia Capital · 2026

    Companies should own the models behind parts of their product when inference costs grow with usage, when speed matters, or when proprietary data drives improvement. The path: a dedicated team, a benchmark first, then the lightest post-training that moves the evals, with production feeding back in.

  2. Deep Dive: The Open vs. Closed AI Race

    Chamath Palihapitiya · Social Capital · 2026

    Finds open-weight models trailing the best closed models by roughly four months, which makes open versus closed a pressing business decision. The full report, for subscribers, looks at who captures the profits as AI gets cheaper.

  3. What It Means for Nations to Have AI Sovereignty

    Vinod Khosla, interviewed · Marketplace Tech · 2024

    Khosla argues that countries need AI tailored to their own languages, cultures and security interests, rather than depending on one foreign model that could be cut off. He expects nations to use models from many sources alongside their own.

  4. The Rise of Sovereign AI

    Andrew Ng · The Batch · 2026

    Allied nations are building their own AI capability so that no foreign government can switch it off. Full self-sufficiency is unrealistic, Ng notes, so many are building on open-source models instead.

  5. How Nations Are Deploying AI for Strategic Priorities

    Calista Redmond · NVIDIA Blog · 2026

    Governments are building AI on domestic computing, regional data and local talent, so that it reflects their languages, culture and laws. Sets out a five-part national strategy, with examples from France, India and Brazil.

01 · Hyper-Personalization

Foundation models trained on customer events.

4 pieces
  1. PRAGMA: Revolut Foundation Model

    Maxim Ostroukhov et al., Revolut · arXiv · 2026

    Revolut pre-trains a family of transformers on a large, varied corpus of banking events, with a masked-modelling objective built for discrete, variable-length financial records. One model is then adapted to banking tasks such as credit scoring and fraud detection. Built with NVIDIA.

  2. Your Spending Needs Attention: Modeling Financial Habits with Transformers

    D. T. Braithwaite et al., Nubank · arXiv · 2025

    Nubank's team trains transformers on customers' transaction histories, reading each transaction the way a language model reads a word. The resulting foundation model is fine-tuned for tasks such as credit, instead of relying on hand-built features.

  3. Stripe Unveils AI Foundation Model for Payments

    TechCrunch · 2025

    Stripe trained a payments foundation model on tens of billions of transactions, turning each payment into an embedding that carries many subtle signals. Stripe reported a large jump in catching card-testing attacks.

  4. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations

    Jiaqi Zhai et al., Meta · ICML 2024

    Recasts recommendation as sequence generation over user actions and scales it to a trillion parameters. Shows that the scaling behaviour seen in language models also appears in models of customer behaviour.

02 · Industrialized Language Models

Small, specialised models trained on your data.

5 pieces
  1. Distilling the Knowledge in a Neural Network

    Geoffrey Hinton, Oriol Vinyals, Jeff Dean · arXiv · 2015

    A small model can learn much of what a large one knows by training on the large model's full output probabilities, not just the right answers. The founding idea behind compact, specialised models.

  2. LoRA: Low-Rank Adaptation of Large Language Models

    Edward Hu et al., Microsoft · arXiv · 2021

    Freezes a large model and trains small low-rank adapters instead, cutting the trainable parameters by orders of magnitude. It is why one base model can carry a separate, inexpensive adapter for each workflow.

  3. LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4

    Justin Zhao et al., Predibase · arXiv · 2024

    Fine-tunes 310 models across 31 tasks with low-rank adapters. Most of the small fine-tuned models beat GPT-4 on the task they were trained for, while GPT-4 stays ahead on broad tasks.

  4. Small Language Models are the Future of Agentic AI

    Peter Belcak et al., NVIDIA Research · arXiv · 2025

    Most agent work is narrow and repetitive, so small models are capable enough and much cheaper to run. Large models become the exception, kept for open-ended tasks.

  5. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    DeepSeek-AI · arXiv · 2025

    Trains reasoning with reinforcement learning, then distils it into open models from 1.5B to 70B parameters. Showed that strong reasoning can move into models small enough to run on your own hardware.

03 · Evaluation and governance

Proving a model before it ships.

4 pieces
  1. Who Validates the Validators? Aligning LLM-Assisted Evaluation with Human Preferences

    Shreya Shankar et al., UC Berkeley · UIST 2024

    Studies how to check an automated LLM judge against human grades, and finds that people's criteria shift as they see more outputs. The case for calibrating any judge against your own reviewers.

  2. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

    Lianmin Zheng et al., UC Berkeley · NeurIPS 2023

    Tests whether a strong model can grade other models' answers, and finds it agrees with human preferences about as often as people agree with each other. Also catalogues judge biases, such as favouring the first or the longer answer.

  3. Your AI Product Needs Evals

    Hamel Husain · 2024

    A practitioner's guide drawn from client work: unit tests, human and model review, and logging every trace. Teams that invest in evaluation, it argues, improve their products faster.

  4. What We've Learned From A Year of Building with LLMs

    Eugene Yan, Bryan Bischof, Charles Frye, Hamel Husain, Jason Liu, Shreya Shankar · 2024

    Six practitioners on prompting, retrieval, evaluation and operations. A recurring theme: build evals early and look closely at your own data.

04 · Hardware-aware optimisation

Smaller, faster models for the chips you choose.

3 pieces
  1. Compact Language Models via Pruning and Knowledge Distillation

    Saurav Muralidharan et al., NVIDIA · arXiv · 2024

    Prunes a 15B model's depth and width, then retrains it with distillation, deriving 8B and 4B models with up to 40 times fewer training tokens than training from scratch. The basis of NVIDIA's Minitron models.

  2. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

    Ji Lin et al., MIT · MLSys 2024

    Protects the small share of weights that matter most for accuracy while quantising the rest to 4 bits. Lets large models run on smaller GPUs and edge devices with little loss.

  3. QLoRA: Efficient Finetuning of Quantized LLMs

    Tim Dettmers et al., University of Washington · NeurIPS 2023

    Fine-tunes a 4-bit quantised model through low-rank adapters, making it possible to fine-tune a 65B model on a single 48GB GPU.

05 · Inference serving

Running models efficiently, where you decide.

2 pieces
  1. Efficient Memory Management for Large Language Model Serving with PagedAttention

    Woosuk Kwon et al., UC Berkeley · SOSP 2023

    Manages a model's attention memory in pages, the way an operating system manages memory, raising throughput 2 to 4 times over earlier systems. The basis of vLLM, a widely used open-source server.

  2. S-LoRA: Serving Thousands of Concurrent LoRA Adapters

    Ying Sheng et al., UC Berkeley and Stanford · MLSys 2024

    Serves thousands of fine-tuned adapters on top of one base model at the same time. It makes a model per workflow, or per customer, affordable to run.

06 · Model observability

Watching models in production, and learning from it.

3 pieces
  1. Hidden Technical Debt in Machine Learning Systems

    D. Sculley et al., Google · NeurIPS 2015

    The model code is a small part of a real ML system; most of the cost and risk lives in the data dependencies, feedback loops and monitoring around it. Still the standard reference on why production ML is hard.

  2. Operationalizing Machine Learning: An Interview Study

    Shreya Shankar et al., UC Berkeley · 2022

    Interviews with 18 machine learning engineers about running models in production. Velocity, validation and versioning come out as the three things that make or break a deployment.

  3. Data Distribution Shifts and Monitoring

    Chip Huyen · 2022

    Why models degrade after deployment as the data changes, the kinds of shift to expect, and what to monitor to catch them early.