bigscience-workshop/petals on Hacker News

https://github.com/bigscience-workshop/petals was meant to be this i think

comment by khimaros · 2026-10-20 · in How abliterated models can get you pwned

…HuggingFace ( https://huggingface.co/bigscience ). Doesn't look like it's very active nowadays: https://github.com/bigscience-workshop/petals Article with more details: https://techcrunch.com/2022/12/20/petals-is-creating-a-free-... >…

comment by wxw · 2026-07-23 · in Petals: Run LLMs at home, BitTorrent-style

This is precisely why we need more projects like this https://github.com/bigscience-workshop/petals There's no fundamental reason why models couldn't be developed and trained using community efforts. It might not be as fast and efficient,…

comment by yogthos · 2026-07-20 · in China’s open-weights AI strategy is winning

…year. There are also projects like Petals which allow you to train models in a distributed fashion https://github.com/bigscience-workshop/petals And yeah, I do think China's government is going to continue subsidizing this tech because…

comment by yogthos · 2026-06-15 · in Statement on US government directive to suspend access to Fable 5 and Mythos 5

there are already projects like Petals https://github.com/bigscience-workshop/petals

comment by yogthos · 2026-06-13 · in Open source AI must win

there was a project trying to achieve some of those goals a few years ago using p2p: petals https://github.com/bigscience-workshop/petals their bloom model was also a collaborative effort https://huggingface.co/docs/transformers/en/model_do…

comment by cpdomina · 2026-06-13 · in Open source AI must win

how does it compare to https://github.com/bigscience-workshop/petals ?

comment by juvo · 2024-07-16 · in Exo: Run your own AI cluster at home with everyday devices

Things like petals ( https://github.com/bigscience-workshop/petals ) exist, distributed computing over willing participants. Right now corporate cash is being rammed into the space so why not snap it up while you can, but the moment it…

comment by Grimblewald · 2024-05-22 · in Chameleon: Meta’s New Multi-Modal LLM

Reminded me of Petals offering distributed ML inference anyone tried this? https://github.com/bigscience-workshop/petals

comment by monkeydust · 2024-03-23 · in Emad Mostaque resigned as CEO of Stability AI

Can check out their project at https://github.com/bigscience-workshop/petals

comment by amrb · 2024-01-02 · in Distributed Inference and Fine-Tuning of Large Language Models over the Internet

…supports fine-tuning and inference[3] afaik. [1] https://github.com/learning-at-home/hivemind [2] https://github.com/bigscience-workshop/petals [3] https://chat.petals.dev/

comment by dvmazur · 2023-11-23 · in Exponentially faster language modelling

https://github.com/bigscience-workshop/petals

comment by brucethemoose2 · 2023-11-02 · in Ask HN: Are there any projects currently exploring distributed AI training?

https://github.com/bigscience-workshop/petals Inference only: https://lite.koboldai.net/

comment by brucethemoose2 · 2023-10-31 · in Mistral 7B,The complete Guide of the Best 7B model

…to have a ballpark estimate of the finetunning performance of a "private" petals cluster. [0] https://github.com/bigscience-workshop/petals/wiki/Launch-yo...

comment by timost · 2023-09-18 · in Run LLMs at home, BitTorrent‑style

Would love to share my 3080 Ti, but after running the commands in the getting started guide ( https://github.com/bigscience-workshop/petals/wiki/Run-Petal... ) it looks like there's a dependency versioning issue: ImportError: cannot import…

comment by teaearlgraycold · 2023-09-17 · in Run LLMs at home, BitTorrent‑style

…easier to develop and maintain, so we can focus on developing features useful for ML researchers. https://github.com/bigscience-workshop/petals/wiki/FAQ:-Freq...

comment by esafak · 2023-09-17 · in Run LLMs at home, BitTorrent‑style

…the entire training data needs to be loaded into VRAM and can't be split up across devices. [0] https://github.com/bigscience-workshop/petals

comment by addandsubtract · 2023-09-05 · in HuggingFace Training Cluster as a Service

…out, you can check out https://health.petals.dev/ If you have a spare GPU, consider contributing: https://github.com/bigscience-workshop/petals . I am not associated with them.

comment by fswd · 2023-08-31 · in Degraded performance on ChatGPT

…If unwilling to spend any money at all and just want to play around with llama70b, look into petals https://github.com/bigscience-workshop/petals

comment by spikedoanz · 2023-08-09 · in Ask HN: Cheapest hardware to run Llama 2 70B

Could this work well with distributed solutions like petals? https://github.com/bigscience-workshop/petals I don't understand how petals can work though. I thought LLMs were typically quite monolithic.

comment by worldsayshi · 2023-08-04 · in Non-determinism in GPT-4 is caused by Sparse MoE

…Colab: https://colab.research.google.com/drive/1uCphNY7gfAUkdDrTx21... Project description: https://github.com/bigscience-workshop/petals

comment by borzunov · 2023-07-19 · in Petals runs Llama 2 (70B) from Colab at 5 tokens/sec

https://github.com/bigscience-workshop/petals seems to have some capabilities in that area, at least for fine-tuning.

comment by khimaros · 2023-05-31 · in Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?

That exists: https://github.com/bigscience-workshop/petals

comment by simonw · 2023-05-31 · in Ask HN: What's the best self hosted/local alternative to GPT-4?

https://github.com/bigscience-workshop/petals

comment by hnjst · 2023-05-25 · in QLoRA: Efficient Finetuning of Quantized LLMs

https://github.com/bigscience-workshop/petals might be up your alley.

comment by khimaros · 2023-04-11 · in Anthropic’s $5B, 4-year plan to take on OpenAI

…(1) some time ago here on HN. There are surely others too, but I don't remember the names. 1. https://github.com/bigscience-workshop/petals

comment by alexvoda · 2023-04-07 · in What it feels like to work in AI right now

Yes there is petals/bloom https://github.com/bigscience-workshop/petals but it's not so great. Maybe it will improve or a better one will come.

comment by ftxbro · 2023-03-31 · in Could you train a ChatGPT-beating model for $85k and run it in a browser?

My understanding is that can work for model inference but not for model training. https://github.com/bigscience-workshop/petals is a project that does this kind of thing for running inference - I tried it out in Google Collab and it seemed…

comment by simonw · 2023-03-31 · in Could you train a ChatGPT-beating model for $85k and run it in a browser?

Right, so sort of like https://github.com/bigscience-workshop/petals but for the training phase. I suppose different training runs could be proposed via a RFC type of procedure. Then it’s not only the open source model maintainers that put…

comment by barefeg · 2023-03-28 · in Show HN: A fully open-source (Apache 2.0)implementation of llama

…- https://github.com/oobabooga/text-generation-webui/issues/14... - https://github.com/bigscience-workshop/petals - https://github.com/alpa-projects/alpa

comment by lxe · 2023-03-22 · in Show HN: Finetune LLaMA-7B on commodity GPUs using your own text

…get the speed of hundreds of tokens per sec, see our benchmarks for parallel forward passes: https://github.com/bigscience-workshop/petals#benchmarks If you have another wording in mind that is more up front, please let us know, we'd be…

comment by borzunov · 2023-03-21 · in Run 100B+ language models at home, BitTorrent‑style

…set up your own swarm, you can't go tuning things on random participants. You can read more at: - https://github.com/bigscience-workshop/petals/wiki/Security,... - https://github.com/bigscience-workshop/petals/wiki/Launch-yo...

comment by Etheryte · 2023-03-20 · in Run 100B+ language models at home, BitTorrent‑style

From https://github.com/bigscience-workshop/petals/wiki/Security,... > Q: Does Petals guarantee that model outputs are correct? > Not by default. A faulty or malicious server could give you incorrect outputs. There are two things you can…

comment by rohfle · 2023-03-20 · in Run 100B+ language models at home, BitTorrent‑style

…to bring down the vRAM requirements? I only know about these: - use 8bit precision - https://github.com/bigscience-workshop/petals - https://github.com/FMInference/FlexGen - https://github.com/microsoft/DeepSpeed Anything that could bring…

comment by underlines · 2023-03-03 · in Facebook LLAMA is being openly distributed via torrents

Does this mean that with big enough compute capacity - say, Petals https://github.com/bigscience-workshop/petals which distributes the model over the internet over GPUs - we can run LLAMA?

comment by VadimPR · 2023-03-03 · in Facebook LLAMA is being openly distributed via torrents

…and that's why there is a project that is using volunteered, distributed GPUs to run BLOOM/BLOOMZ: https://github.com/bigscience-workshop/petals , http://chat.petals.ml .

comment by VadimPR · 2023-03-03 · in Facebook LLAMA is being openly distributed via torrents

…that shares some of the same codebase, but is for distributed training . [0] https://petals.ml/ [1] https://github.com/bigscience-workshop/petals [2] https://github.com/yandex-research/swarm [3] https://twitter.com/m_ryabinin/status/1625175…

comment by jessfyi · 2023-02-15 · in Semantic Search with Phoenix, Axon, Bumblebee, and ExFaiss

Might this be what you're looking for: https://github.com/bigscience-workshop/petals ?

comment by sandkoan · 2023-02-11 · in How to train large models on many GPUs? (2021)

…to the project by running a node in my garage with a single RTX 3060ti in it, and you can too: https://github.com/bigscience-workshop/petals It's early days, but the tech is super promising.

comment by px43 · 2023-02-07 · in Bard and new AI features in Search

That already exists - https://github.com/bigscience-workshop/petals

comment by VadimPR · 2023-02-04 · in Open Assistant: Conversational AI for Everyone

you are probably thinking of https://arxiv.org/abs/2207.03481 for inference, there is https://github.com/bigscience-workshop/petals however, both are only in the research phase. start tinkering!

comment by lucidrains · 2023-02-04 · in Open Assistant: Conversational AI for Everyone

…on your own hardware - you need a LOT of powerful GPUs racked up in order to run inference. https://github.com/bigscience-workshop/petals is a really interesting project here: it works a bit like bittorrent, allowing you to join a larger…

comment by simonw · 2023-01-29 · in Build ChatGPT like chatbots on your website

…project that lets you run a bunch of independent machines as a cluster for inference using Bloom: https://github.com/bigscience-workshop/petals/wiki/Launch-yo... (You'll need around 200GB of GPU memory across the machines in the swarm)

comment by wokwokwok · 2023-01-20 · in ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

For a distributed computing/BitTorrent-style method of running these LLMs, see: https://github.com/bigscience-workshop/petals .

comment by sandkoan · 2023-01-19 · in Let's build GPT: from scratch, in code, spelled out by Andrej Karpathy [video]

…the point of privacy/anonymity since everyone in the distributed swarm have access to your data [0] https://github.com/bigscience-workshop/petals

comment by lairv · 2023-01-16 · in GPT-3 is the best journal I’ve used

Yes: https://github.com/bigscience-workshop/petals

comment by speedgoose · 2023-01-12 · in Ask HN: Can you crowdfund the compute for GPT?

…have enough GPUs at home can contribute theirs and get text generation at one word per second: https://github.com/bigscience-workshop/petals#how-does-it-wo...

comment by espadrine · 2023-01-10 · in Microsoft eyes $10B bet on ChatGPT

…with a distributed swarm of machines. iirc it already supports fine-tuning BLOOM, so.. maybe? [0] https://github.com/bigscience-workshop/petals

comment by pizza · 2022-12-30 · in There’s now an open source alternative to ChatGPT, but good luck running it

https://github.com/bigscience-workshop/petals Since my other account is shadow banned for some unexplained reason, I just wanted to mention the petal project. It's an attempt to bittorrent style distribute the load of running these large…

comment by chamwislothe2nd · 2022-12-29 · in Ask HN: Can I download GPT / ChatGPT to my desktop?

https://github.com/bigscience-workshop/petals

comment by bookofjoe · 2022-12-25 · in Petals is creating a free, distributed network for running text-generating AI