Hacker GitHub

← All repos

twitter/the-algorithm

View on GitHub ↗ ★ 73,994 Scala

Source code for the X Recommendation Algorithm

116 mentions on Hacker News by 104 people: 20 stories, 96 comments · first 2022-04-25 · latest 2026-10-19

Mentions per year 116 mentions · busiest year 2023 (78)
20082026
AllStoriesComments NewestTopOldest
  1. A feature proposal to put you in control of the algorithm https://github.com/twitter/the-algorithm/issues/1363
    comment by 13years · 2023-04-02 · in Twitter's Recommendation Algorithm
  2. …Speaking about Ukraine, it seems to be literally a Twitter policy violation ... https://github.com/twitter/the-algorithm/blob/main/visibilit...
    comment by WinstonSmith84 · 2023-04-02 · in Twitter's Recommendation Algorithm
  3. I think they have a bug here here: https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92... Code: ( "has_gte_10k_favs", _.getOrElse(EarlybirdFeature, None).exists(_.favCountV2.exists(_ >= 1000))), Should be: ( "has_gte_10k_favs",…
    comment by d_sc · 2023-04-01 · in Twitter's Recommendation Algorithm
  4. …-> HatefulHighRecall, ... s.SpaceSafetyLabelType.UkraineCrisisTopic -> UkraineCrisisTopic, https://github.com/twitter/the-algorithm/blob/ec83d01dcaebf3...
    comment by amq · 2023-04-01 · in Twitter's Recommendation Algorithm
  5. How do you infer that is what is happening? This comment would seem to contradict https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
    comment by verdverm · 2023-04-01 · in Twitter scrubbing Elon, Republican, Democrat code references on GitHub
  6. Maybe; twitter/the-algorithm/issues : https://github.com/twitter/the-algorithm/issues
    comment by News-Dog · 2023-04-01 · in Ask HN: Where are the April Fools fake tech launches?
  7. Seems random. This file has both 2 and 4: https://github.com/twitter/the-algorithm/blob/main/trust_and...
    comment by xdennis · 2023-04-01 · in Twitter's Recommendation Algorithm
  8. …in Rust [1] and this pull request which simplifies the algorithm to a single c file [2]. [1] https://github.com/twitter/the-algorithm/issues?q=is%3Aissue... [2] https://github.com/twitter/the-algorithm/pull/712
    comment by hotpathdev · 2023-04-01 · in Twitter's Recommendation Algorithm
  9. Weights on different metrics. From https://github.com/twitter/the-algorithm/blob/ec83d01dcaebf3... private def getLinearRankingParams: ThriftRankingParams = { ThriftRankingParams( `type` = Some(ThriftScoringFunctionType.Linear), minScore =…
    comment by sudo_navendu · 2023-04-01 · in Twitter's Recommendation Algorithm
  10. …'the claim' has now suddenly turned into a fact. [0] [1] This also didn't age well at all. [0] https://github.com/twitter/the-algorithm [1] https://github.com/twitter/the-algorithm-ml
    comment by rvz · 2023-04-01 · in Twitter will open source all code used to recommend tweets on March 31
  11. So uh... they use BigQuery and here's the dataset https://github.com/twitter/the-algorithm/blob/main/ann/src/m...
    comment by evantahler · 2023-04-01 · in Twitter's Recommendation Algorithm
  12. Oh god... The MR's opened today are the craziest ones ever. https://github.com/twitter/the-algorithm/pulls?q=is%3Apr+is%... They made my morning
    comment by anshumankmr · 2023-04-01 · in Twitter's Recommendation Algorithm
  13. Well that didn't age well... [0][1] [0] https://github.com/twitter/the-algorithm [1] https://github.com/twitter/the-algorithm-ml
    comment by rvz · 2023-04-01 · in Twitter will open source all code used to recommend tweets on March 31
  14. Yup, they removed it. https://github.com/twitter/the-algorithm/commit/ec83d01dcaeb... Here's what Musk said: > But we are deleting this bs. I only learned about it now! Will be gone by tomorrow. https://twitter.com/elonmusk/status/164190813…
    comment by tric · 2023-03-31 · in Twitter's Recommendation Algorithm
  15. Also this: https://github.com/twitter/the-algorithm/pull/160
    comment by adharmad · 2023-03-31 · in Twitter's Recommendation Algorithm
  16. Just removed https://github.com/twitter/the-algorithm/commit/ec83d01dcaeb...
    comment by maccam912 · 2023-03-31 · in Twitter's Recommendation Algorithm
  17. …a separate PR? Other PRs are similarly creative, like the one called “blue tick for all accounts”: https://github.com/twitter/the-algorithm/pull/357
    comment by thih9 · 2023-03-31
  18. `if (sourceUserId.isDefined || sourceUserId.isDefined) Some(true)` https://github.com/twitter/the-algorithm/blob/main/timeliner...
    comment by jml2 · 2023-03-31 · in Twitter's Recommendation Algorithm
  19. Aside from the spam PRs, there is actually one PR that fixes a bug: https://github.com/twitter/the-algorithm/pull/242/files
    comment by hrpnk · 2023-03-31 · in Twitter's Recommendation Algorithm
  20. Well, sounds like this pull request doesn't get merged. https://github.com/twitter/the-algorithm/pull/234
    comment by Calzifer · 2023-03-31 · in Twitter's Recommendation Algorithm
  21. …dSnapshot(SocialgraphUnfollowsScalaDataset, sc), FeatureName.NumUnfollows, endTs) .filter(_.age < 90) https://github.com/twitter/the-algorithm/blob/main/src/scala...
    comment by HellsMaddy · 2023-03-31 · in Twitter's Recommendation Algorithm
  22. …of that infrastructure aren’t really relevant and is a notable portion of their secret sauce. https://github.com/twitter/the-algorithm/tree/main/src/java/...
    comment by cmckn · 2023-03-31 · in Twitter's Recommendation Algorithm
  23. I thought it was an april fools joke when I saw this: https://github.com/twitter/the-algorithm/blob/main/ci/ci.sh Like a dig at the code quality.
    comment by stusmall · 2023-03-31 · in Twitter's Recommendation Algorithm
  24. …perhaps just for this extract. Sorting it now results in a different order than the commit. [1]: https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
    comment by etc_passwd · 2023-03-31 · in Twitter's Recommendation Algorithm
  25. there may be a hint of which elections were of interest: https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
    comment by pledess · 2023-03-31 · in Twitter's Recommendation Algorithm
  26. I think the relevant part of the code is in this other repo: https://github.com/twitter/the-algorithm Not sure if it has what you were looking for (and maybe you already checked this repo, too!), but it's more relevant than the linked repo…
    comment by mardifoufs · 2023-03-31 · in Twitter's Recommendation Algorithm
  27. …platform that we do not ship changes \* that negatively impacts one group over others. \* From: https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
    comment by jaywalk · 2023-03-31 · in Twitter's Recommendation Algorithm
  28. It's reassuring to know that billion dollar tech companies write CI exactly like I do: https://github.com/twitter/the-algorithm/blob/main/ci/ci.sh Permalink: https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
    comment by koolba · 2023-03-31 · in Twitter's Recommendation Algorithm
  29. …"author_is_power_user", "author_is_democrat", and "author_is_republican" labels are for [1]. [1]: https://github.com/twitter/the-algorithm/blob/main/home-mixe...
    comment by tech234a · 2023-03-31 · in Twitter's Recommendation Algorithm
  30. This is one: https://github.com/twitter/the-algorithm/issues/121 Search for Elon gives this: https://github.com/twitter/the-algorithm/search?q=Elon&type=
    comment by ano-ther · 2023-03-31 · in Twitter's Recommendation Algorithm
  31. Perhaps that's related to this line. Though perhaps this is just used for observing metrics. https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
    comment by Chinjut · 2023-03-31 · in Twitter's Recommendation Algorithm
  32. …metrics[0], so Team Red people see more Team Red content and vice versa. Nothing nefarious. [0] https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
    comment by krapp · 2023-03-31 · in Twitter's Recommendation Algorithm
  33. I answered my own question https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92... "This method reduces the page rank of users who have a low number of followers but a high number of followings."
    comment by pram · 2023-03-31 · in Twitter's Recommendation Algorithm
  34. Did you also skim the accompanying (or rather, main) repo, https://github.com/twitter/the-algorithm ? From a quick clone and line-count, it has: 235 kLOC .scala 136 kLOC .java 22 kLOC .py 7 kLOC .rs So I don't think you did, since you…
    comment by tpmx · 2023-03-31 · in Twitter's Recommendation Algorithm
  35. …they seem to explicitly track how often they're serving Elon's tweets for A/B testing experiments: https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
    comment by devrand · 2023-03-31 · in Twitter's Recommendation Algorithm
  36. …platform that we do not ship changes * that negatively impacts one group over others. */ [0]: https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
    comment by CathalMullan · 2023-03-31 · in Twitter's Recommendation Algorithm
  37. Party in the issues: https://github.com/twitter/the-algorithm/issues
    comment by benatkin · 2023-03-31 · in Twitter's Recommendation Algorithm
  38. I wonder what determines 'cred' for this part: https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92...
    comment by pram · 2023-03-31 · in Twitter's Recommendation Algorithm
  39. https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92... ``` def query_keys(self, language, task=2, size="50"): if task == 2: if language == "ar": self.query_settings["adhoc_v2"]["table"] = "..." elif language == "tr":…
    comment by Kpourdeilami · 2023-03-31 · in Twitter's Recommendation Algorithm
  40. I think this PR is modifying the inputs to the methods that do it: https://github.com/twitter/the-algorithm/pull/17
    comment by jmholla · 2023-03-31 · in Twitter's Recommendation Algorithm
  41. Great pull request here which improves the algorithm: https://github.com/twitter/the-algorithm/pull/17
    comment by PenguinRevolver · 2023-03-31 · in Twitter's Recommendation Algorithm
  42. From https://github.com/twitter/the-algorithm/blob/7f90d0ca342b92... ( "author_is_elon", candidate => candidate .getOrElse(AuthorIdFeature, None).contains(candidate.getOrElse(DDGStatsElonFeature, 0L))), ( "author_is_power_user", candidate…
    comment by tric · 2023-03-31 · in Twitter's Recommendation Algorithm
  43. Main repos: - https://github.com/twitter/the-algorithm - https://github.com/twitter/the-algorithm-ml Blogs: - Eng: https://blog.twitter.com/engineering/en_us/topics/open-sourc... - Biz: https://blog.twitter.com/en_us/topics/company/2023/a-n…
    comment by summarity · 2023-03-31 · in Twitter's Recommendation Algorithm
  44. GitHub repo: https://github.com/twitter/the-algorithm/
    comment by tric · 2023-03-31 · in Twitter's Recommendation Algorithm
  45. …confused and reading the comments now, the link was to an empty, but real, twitter repository at https://github.com/twitter/the-algorithm . Now it is a 404.
    comment by Diesel555 · 2022-04-26 · in twitter/the-algorithm
  46. Will Norris who works on OSS at Twitter posted this[0]: "watch this space https://github.com/twitter/the-algorithm " [0]: https://twitter.com/willnorris/status/1518694675909013504
    comment by johns · 2022-04-25 · in twitter/the-algorithm