I. “Clumsy” Transparency: Musk-style Reverse Marketing
“We know this algorithm is stupid and needs massive improvement, but at least you can watch us struggle to make it better in real-time.”
This quote came from Musk himself on January 20, 2026, the day the X algorithm was officially open-sourced. To put it plainly, this is a bit like a chef live-streaming the kitchen surveillance feed to the diners—burnt pans, dropped food, and frantic scrambling are all captured on camera. You could call this “sincerity,” or you could call it a brilliant form of expectation management.
But one thing cannot be denied: this is the first time in social media history that a mainstream platform has made a complete recommendation algorithm running in a production environment public. The “open source” move in 2023? That was just a stale code specimen; it sat on GitHub for three years untouched while the actual system had long since evolved beyond recognition. This time is different—X promises to update the code every four weeks, accompanied by developer notes explaining what changed and why.
This architecture diagram looks complex, but it tells one story: every like, repost, and even dwell time is being fed to a Transformer model named Phoenix.
Interestingly, Musk chose to announce this open-sourcing right as the European Commission extended its data retention order on X’s algorithm. Regulatory pressure? Brand crisis PR? Or a genuine belief that “transparency is freedom”? The answer is likely a mix of all three. But regardless of the motive, the code itself—is real.
II. Thunder and Phoenix: A Recommendation System “Created from Nothing”
Let’s dive into the code and see what X’s recommendation system is actually doing.
The core of the entire system is called Home Mixer—a “bartender” role responsible for mixing two types of “liquor” to serve you:
- Thunder: In-memory post storage that consumes creation/deletion events from Kafka in real-time, specifically serving content from people you follow. Latency? Sub-millisecond. This means the moment a creator you follow tweets, Thunder already knows.
- Phoenix: This is the true “Recommendation Engine.” It uses a Two-Tower Model to fish out “unfamiliar content” you might be interested in from hundreds of millions of posts across the network—the User Tower encodes your behavioral history, the Candidate Tower encodes all posts, and then Top-K results are retrieved via vector similarity.
Then, Phoenix uses a Grok Transformer to fine-rank these candidates—predicting the probability that you will like, reply, repost, click, watch a video, follow the author, or even block, mute, or report them. The final score is a weighted sum of all predictions: positive behaviors add points, negative behaviors deduct points.
Final Score = Σ (weighti × P(actioni))
There is a detail here worth savoring: Candidate Isolation. During Transformer inference, candidate posts cannot see each other; they can only see the user context. This ensures that the score of each post does not depend on other posts in the same batch, making scores cacheable and reusable. This is a very “clean” engineering design decision.
But even more radical is this statement: “We removed all handcrafted features and most heuristic rules.”
Put simply, those artificial rules in traditional recommendation systems like “add 5 points if the post has an image” or “add 10 points if the author has over 100k followers” are gone. Now, everything is left for Grok to learn itself. This is both a form of technical “minimalism” and a belief: trusting that a sufficiently powerful model can automatically discover “what constitutes good content.”
III. The Cost of Transparency: When Your “Block” Becomes a Float
After the open-sourcing, people used AI to analyze the algorithm and summarized five major factors affecting the “viral spread” of posts. But what’s more intriguing are those negative predictions:
Predictions:
├── P(notinterested)
├── P(blockauthor)
├── P(mute_author)
└── P(report)
This means that when you block a person, mute a topic, or report a piece of content, you aren’t just “cleaning up your timeline”—you are contributing training data to the entire system. Your disgust is being quantified, modeled, and used to predict “what other people like you will also find disgusting.”
Academia has long worried about “recommendation systems reinforcing information cocoons.” An experimental study in PNAS found that short-term exposure to “biased recommendation algorithms” does indeed change users’ content consumption habits, but the impact on political attitudes was “not as great as imagined.” While another
