S4-How AI Research Works

Section 1

What Counts as an AI Research Organization

Most large tech companies maintain a research arm that sits apart from (but feeds into) product engineering. Some are dedicated labs; some AI-native companies are essentially research organizations with a product wrapped around them.

Company	Research unit	Style
Google / Alphabet	Google Research, Google DeepMind	Merged research + applied lab
Microsoft	Microsoft Research (MSR)	Long-running academic-style lab, est. 1991
Meta	FAIR (Fundamental AI Research)	Publication-heavy, semi-independent
Amazon	Amazon Science, AWS AI Labs	Product-adjacent applied research
IBM	IBM Research	Oldest corporate lab (1945), multiple global sites
OpenAI / Anthropic / Moonshot AI	The company itself	Research-first; product is downstream of the research

Note: Naming convention: externally, companies almost always brand these units as a "Lab" or "Research" (e.g. Google Research, MSR, IBM Research). "Department" is an internal org-chart term you'd see in a job posting, not on a website or building.

Section 2

The Core Research Loop

Unlike a chemistry or physics lab, AI research has no physical experiments in the traditional sense. Almost all of it happens on a laptop and a compute cluster. The work follows a repeating cycle:

Figure 1. The AI Research Loop

Iterate — most runs underperform, so this cycle repeats often hundreds of times before something ships.

Literature review

Reading papers on arXiv and at conferences (NeurIPS, ICML, ICLR, ACL) to see what's been tried and where gaps are.

Hypothesis

Forming a specific, testable idea (e.g. "a linear-attention variant will cut memory cost without hurting long-context accuracy").

Implementation

Writing the model/training code, usually in Python with PyTorch or JAX.

Large-scale training

Running the idea on a GPU/TPU cluster; big runs can take days to weeks and cost thousands to millions in compute.

Evaluation

Testing the result against benchmarks (coding, reasoning, math, safety, etc.).

Ablation studies

Removing or swapping one component at a time to isolate what's actually responsible for a result.

Real-world Example

Transformer-XL / XLNet

Yang Zhilin's PhD-era papers at Carnegie Mellon addressed a specific hypothesis — that transformers were losing track of context beyond a fixed window. The fix (a recurrence mechanism) was implemented, trained, benchmarked against existing language models, and published — then years later became foundational to Moonshot AI's long-context products.

Section 3

What You Were Missing: Hidden Realities

3.1 Compute is the real laboratory equipment

Instead of beakers or particle accelerators, physical infrastructure is racks of GPUs or TPUs — often tens of thousands of them. This is the single biggest cost driver in modern AI research.

Example — Compute scale: Training a 2-3 trillion-parameter model like Kimi K3 requires thousands of GPUs running in parallel for an extended period, plus cluster stability engineering ("ML systems" or "infrastructure research").

3.2 Data curation is its own research problem

What a model is trained on matters as much as architecture. Labs spend significant research effort on cleaning, filtering, deduplicating, and weighting training data (code, textbooks, dialogue, synthetic data) to shape final behavior.

3.3 Evaluation is harder than it sounds

Benchmarks can be gamed, saturated, or simply fail to measure what matters. A large amount of research goes into designing new evaluations rather than just passing existing ones.

Example — In-house benchmarks: Moonshot AI built its own "PerceptionBench" to test visual perception specifically, rather than relying only on public benchmarks.

3.4 Post-training is a distinct research phase

Pretraining is only the first stage. Supervised fine-tuning, reinforcement learning from human feedback (RLHF), and safety/alignment tuning shape how the model actually behaves once deployed.

3.5 Safety and alignment research is a dedicated track

At frontier labs, a portion of research is specifically about making models more truthful, controllable, and less prone to harmful outputs through red-teaming and interpretability research.

3.6 "Open" vs "closed" research is a strategic choice

Publishing full papers used to be the default. Increasingly, companies choose whether to open-source model weights, publish papers, or keep methods proprietary as a competitive and geopolitical decision.

Example — Open weights as strategy: Moonshot AI released Kimi K3's full weights publicly rather than keeping them proprietary, offering "sovereign AI" infrastructure.

3.7 Talent and geography shape where research happens

Top AI researchers are a scarce resource, and where they work has become geopolitically significant, with labs competing on compute access, publication freedom, and immigration policy.

Example — Talent flow: Yang Zhilin trained in the US (Tsinghua, CMU, Google Brain, Meta) but returned to Beijing to found Moonshot AI.

Section 4

Research Lab vs. Product Engineering

Dimension	Research team	Product / applied engineering team
Goal	Push capability forward; publish or validate new methods	Ship reliable features on a roadmap
Timeline	Can span months to years per project	Weeks to a quarter, tighter deadlines
Output	Papers, model checkpoints, benchmarks	Shipped product, APIs, user-facing features
Risk tolerance	High — most experiments are expected to fail	Low — stability and reliability prioritized
Typical company	OpenAI, Anthropic, DeepMind, Moonshot AI, FAIR	Startups building on top of existing model APIs

Note: Many startups skip a formal research division entirely, building applied products on top of existing foundation models via API to avoid massive compute costs.

Section 5

Quick Glossary

Mixture-of-Experts (MoE)

An architecture where only a subset of the model's parameters activate per input, making very large models cheaper to run.

Context window

How much text/data a model can consider at once when generating a response, measured in tokens.

Ablation study

An experiment where one component is removed or altered to measure its individual contribution to performance.

RLHF

Reinforcement Learning from Human Feedback: a post-training technique using human preference data to steer model behavior.

Benchmark

A standardized test used to compare model performance on a specific skill (coding, math, reasoning, etc.).

Open weights

Making a trained model's parameters publicly downloadable, as opposed to only offering access via a paid API.

From Hypothesis to Published Model