Hacker GitHub

← All repos

apache/datafusion-ballista

View on GitHub ↗ ★ 2,171 Rust

Apache DataFusion Ballista Distributed Query Engine

8 mentions on Hacker News by 8 people: 1 story, 7 comments · first 2022-10-26 · latest 2025-09-16

Mentions per year 8 mentions · busiest year 2022 (3)
20082026
AllStoriesComments NewestTopOldest
  1. ▲ 1
    Show HN: Luna, an in-memory SQL server for object storage data
    story by f14t · 2025-09-16 · 0 comments
  2. I would expect a better contender for Spark to be something that's actually open source, such as https://github.com/apache/datafusion-ballista
    comment by yencabulator · 2025-09-08 · in Polars Cloud and Distributed Polars now available
  3. But why. Unless you need to use low-level map/reduce, just ditch Spark and use https://github.com/apache/datafusion-ballista directly. It supports Python too.
    comment by OutOfHere · 2024-05-31 · in DataFusion Comet: Apache Spark Accelerator
  4. …implementation called ballista that could maybe? form the backend of a polars scale out approach: https://github.com/apache/arrow-ballista
    comment by sanderjd · 2024-01-09 · in Polars
  5. …execution language avoids the overhead of GC pauses and results in deterministic processing times" https://github.com/apache/arrow-ballista/blob/main/README.md
    comment by random_ · 2023-08-13 · in Rust vs. Go in 2023
  6. Can you explain how this might differ from something like https://github.com/apache/arrow-ballista I've seen several variants of "next-gen" spark, but nowhere have I really seen the different tradeoffs/advantages/disadvantages between them.
    comment by theLiminator · 2022-12-30 · in I wrote a SQL engine in Python
  7. …[3] https://github.com/dimforge/nalgebra [4] https://github.com/apache/arrow-datafusion [5] https://github.com/apache/arrow-ballista
    comment by Gwypaas · 2022-12-22 · in Welcome to Comprehensive Rust
  8. …latency, some WASM support, all that fun stuff). [0] https://github.com/apache/arrow-datafusion [1] https://github.com/apache/arrow-ballista [2] https://github.com/splitgraph/seafowl
    comment by mildbyte · 2022-10-26 · in IOx: InfluxData’s New Storage Engine