This architecture diagram, circulating widely in tech circles, has now been streamlined into a more “violent” form: “Thunder” (Friends & Family) on the left, “Phoenix” (The Universe) on the right, and a cold, calculating judge sitting in the middle—Grok.
1. Deep Insight: When Algorithms Stop “Calculating” and Start “Understanding”
This is actually quite frightening.
If you looked through the code X first open-sourced in 2023, you would have found a typical “Showcase of Engineer Patches”—full of simple, brute-force weighted logic like if (is_verified), and massive, clunky clustering projects like SimClusters. The algorithm back then was like a diligent accountant, holding a calculator and converting your every move into points.
But the xai-org/x-algorithm released this time looks more like an artist possessed by intuition.
The core change can be summed up in one sentence, yet every word strikes like thunder: “We have eliminated every single hand-engineered feature.”
What does this mean? It means metrics that product managers and data scientists have obsessed over—”completion rate,” “like weight,” “blue check coefficient”—have all been tossed into the dustbin of history. In their place is a Grok-based Transformer. It no longer mechanically adds points because “you clicked like”; instead, it uses the Transformer’s attention mechanism to understand the semantic sequence behind that like.
The old algorithms were doing Statistics; the current Thunder (handling the ‘Following’ flow) and Phoenix (handling the ‘For You’ flow) architectures are doing Linguistics. They turn your behavior into a sentence and ask the AI to complete it: “Given that you just watched these three videos, you will likely want to see…”
This is the value anchor: The recommendation system has finally evolved from a probability machine that “doesn’t even know what it’s recommending” into an intelligent agent that “attempts to understand content.”
2. Independent Perspective: The “Castrated” Transformer as an Engineering Victory
Amidst the chorus of praise for Grok, I want to discuss a detail few have noticed—“Candidate Isolation.”
This is actually a very counter-intuitive technical decision.
Anyone who understands Transformers knows that the architecture’s greatest strength lies in Self-Attention—letting every element in the input sequence “look” at every other element. Theoretically, for accurate recommendations, you should let the 100 tweets in the candidate pool “fight it out” to see which ones have the best chemical reaction together (e.g., ensuring you don’t recommend three homogeneous cat videos in a row).
But X’s engineering team made a decision here that “breaks tradition”: during the inference phase, they forcibly cut off the connections between candidate tweets.
“Candidates cannot attend to each other—only to the user context.”
Why cripple their own martial arts? For extreme speed and caching capabilities.
If Tweet A’s score depends on whether it sits next to Tweet B, then every request requires a recalculation, and the computational cost explodes exponentially (O(N²)). But once “isolation” is implemented, Tweet A’s attraction score for you is fixed, and once calculated, it can be stored in the Cache.
This is high-level engineering aesthetics. Academia can complicate models infinitely to improve accuracy by 0.1%, but industry must find a balance between “accurate recommendations” and “fast scrolling.” X chose to let Grok dance in shackles, trading the algorithm’s theoretical ceiling for a millisecond-level user experience. I give this move full marks.
3. Industry Comparison: ByteDance’s “Actuarial Science” vs. X’s “Brute Force”
If recommendation systems were home decoration, top domestic tech giants (like Douyin/TikTok, Xiaohongshu) are taking the “Luxury Interior Design” route.
Even in 2024 and 2025, the mainstream architectures we see are still Embedding-based Two-Tower models combined with extremely complex Feature Engineering. Thousands of engineers are excavating the tiny correlations between “the probability of a user swiping away at the 3rd second” and “comment section dwell time.” This is a victory for actuaries; due to the massive volume of data, this method of piling up manpower and features does indeed bring extreme retention.
X’s Phoenix architecture, however, takes the “Brute Force Aesthetics” route.
- Feature Engineering: Others use thousands of features; X says, “I just need Grok to read your Log.”
- Multi-objective Prediction: Others train separate models for Click-Through Rate (CTR) and Conversion Rate (CVR); X lets the Transformer output 12 probability heads (P(like), P(reply), P(block)…) directly, grabbing all possible emotions at once like an octopus.
This approach relies heavily on the IQ of the base model (Grok). If the base model isn’t strong enough, this “laziness” is a disaster; but if the base model is strong enough, it’s a dimensionality reduction attack. It’s like you’re still using physics formulas to calculate artillery trajectories, while they’ve just let Jarvis take over the fire control system.
This is the war of the past decade: on the left is feature engineering built on manpower; on the right is deep learning that trusts neural networks. X’s new code declares the death of the left side.
4. Unfinished Thoughts: When AI Predicts Your Predictions
While technically exciting, I have a lurking worry.
When all recommendation logic converges inside a large model (Grok), how do we Debug this world?
Previously, if echo chambers became too severe, engineers could manually lower the weight of the similarity_score. But now, all logic is melted into the Transformer’s billions of parameters. If Grok decides that “provoking anger” is the best method to improve retention (because it predicts P(reply) will be high), it might subtly and undetectably inject hostility into the entire network.
Moreover, the documentation mentions a component called Author Diversity Scorer, specifically designed to suppress repetitive authors. This shows that X’s engineers also realize: Pure AI might get stuck in a loop.
Are we handing over the last bit of control over the “public square”? When the algorithm is no longer a decomposable formula but a bottomless black box, are we scrolling through Twitter, or are we being farmed by an alien brain?
5. Final Words: An Elegy for the “Parameter Tuners”
Reading through this x-algorithm document, my biggest feeling wasn’t shock, but a faint sense of loss.
The era belonging to the “Parameter Tuners” might really be ending. In the future, there will be no more jargon like “try weighting this feature 0.5”; it will be replaced by “let’s train a new model.”
X has proven in the most hardcore way: The best code is no code; the best feature is data itself.
This is very Musk, very rough, but also very futuristic. For peers still addicted to tweaking feature engineering, this might be a wake-up call that isn’t gentle.
Since the ladder has been kicked down, try learning to fly.
