apache/datafusion-ballista

View on GitHub ↗ ★ 2,171 Rust

Apache DataFusion Ballista Distributed Query Engine

8 mentions on Hacker News by 8 people: 1 story, 7 comments · first 2022-10-26 · latest 2025-09-16

Mentions per year 8 mentions · busiest year 2022 (3)

20082026

All Stories Comments Newest Top Oldest

I would expect a better contender for Spark to be something that's actually open source, such as https://github.com/apache/datafusion-ballista
comment by yencabulator · 2025-09-08 · in Polars Cloud and Distributed Polars now available
But why. Unless you need to use low-level map/reduce, just ditch Spark and use https://github.com/apache/datafusion-ballista directly. It supports Python too.
comment by OutOfHere · 2024-05-31 · in DataFusion Comet: Apache Spark Accelerator
…implementation called ballista that could maybe? form the backend of a polars scale out approach: https://github.com/apache/arrow-ballista
comment by sanderjd · 2024-01-09 · in Polars
…execution language avoids the overhead of GC pauses and results in deterministic processing times" https://github.com/apache/arrow-ballista/blob/main/README.md
comment by random_ · 2023-08-13 · in Rust vs. Go in 2023
Can you explain how this might differ from something like https://github.com/apache/arrow-ballista I've seen several variants of "next-gen" spark, but nowhere have I really seen the different tradeoffs/advantages/disadvantages between them.
comment by theLiminator · 2022-12-30 · in I wrote a SQL engine in Python
…[3] https://github.com/dimforge/nalgebra [4] https://github.com/apache/arrow-datafusion [5] https://github.com/apache/arrow-ballista
comment by Gwypaas · 2022-12-22 · in Welcome to Comprehensive Rust
…latency, some WASM support, all that fun stuff). [0] https://github.com/apache/arrow-datafusion [1] https://github.com/apache/arrow-ballista [2] https://github.com/splitgraph/seafowl
comment by mildbyte · 2022-10-26 · in IOx: InfluxData’s New Storage Engine