apache/airflow on Hacker News

…compare to other ETL tool, like Apache Airflow - used by me and many machine learning folks: - link https://github.com/apache/airflow - watch: 777, forks 16.9k!!!!!, Stars: (only!) 45.1k - issues: 1200 (!!!), PR-s (501!!!).

comment by art_mach · 2026-04-20 · in GitHub's fake star economy

…can just use the AGENTS.md file as an index pointing to other doc files. This example does that - https://github.com/apache/airflow/blob/main/AGENTS.md

comment by meander_water · 2025-08-20 · in AGENTS.md – Open format for guiding coding agents

DAG workflow scheduling. https://github.com/apache/airflow

comment by ethbr1 · 2025-06-19 · in MCP Specification – version 2025-06-18 changes

…AI's been a refreshing take on working with LLMs. Would love some feedback from this community! [1] https://github.com/apache/airflow [2] https://ai.pydantic.dev

comment by jlaneve · 2025-03-26 · in Show HN: Airflow AI SDK to build simple LLM workflows

Examples: https://github.com/apache/airflow/issues?q=is%3Aissue%20stat... Other than the content (which indeed makes no sense), these usually can be recognized by subjective adjectives and polish language[1]. [1]…

comment by Lockal · 2025-02-05 · in AI Is Spamming Open Source Repos with Fake Issues

…server. That would also make it easier to support multiple languages. According to their README: https://github.com/apache/airflow#approach-to-dependencies-o... > Airflow has a lot of dependencies - direct and transitive > The important…

comment by annexrichmond · 2024-08-12 · in New Apache Airflow Operators for Google Generative AI

…You can find the fork network and some related bits under repo insights. (The UX is not great.) https://github.com/apache/airflow/forks?include=active&page=...

comment by tedmiston · 2024-07-29 · in Anyone can access deleted and private repository data on GitHub

…gotten especially popular in the ML ops and data engineering world. A common one is AirFlow: * https://github.com/apache/airflow

comment by caprock · 2024-07-21 · in Ask HN: What's the right tool for this job?

…PythonOperator with hooks, that generally integrate external APIs with Airflow connection system. https://github.com/apache/airflow/blob/main/airflow/provider... I think the bigger problem with Airflow's Operator concept is N*M problem of…

comment by KptMarchewa · 2022-05-23 · in Lessons learned from running Apache Airflow at scale

…3rd party providers, and if you have knowledge in any of them it can be a good jumping off point. https://github.com/apache/airflow/

comment by jamestimmins · 2021-08-26 · in Ask HN: What is a good Python project for a mid lv engineer to contribute to?

…still a complete nightmare to deal with and it has many steps for just packaging the application. https://github.com/apache/airflow I have less trouble packaging and running a 3 year unmaintained Maven Java application, than actively…

comment by JAlexoid · 2021-04-19 · in Java Is Underhyped

Cool, thanks. Also on this GitHub issue - https://github.com/apache/airflow/pull/11195

comment by arthurcolle · 2020-12-17 · in Airflow 2.0

…be improved in the UI rewrite. What's your favorite UI for this? 9) That feature is in the queue https://github.com/apache/airflow/issues/7911

comment by rywalker · 2020-05-29 · in Introduction to Apache Airflow

…primitives, and generally OOP / imperative style is just the wrong tool for the job. [0]: https://github.com/apache/airflow/blob/master/airflow/jobs/s...

comment by djtriptych · 2019-07-11 · in Guix Workflow Language

…functionality and pipeline / DAG configuration approaches are similar and it's also done in Python. https://github.com/apache/airflow The Nim language is new to me. What was the experience like implementing a framework in it?

comment by tedmiston · 2019-01-19 · in Show HN: Pipelines – Language for scripting parrallel pipelines with Python

https://github.com/apache/incubator-airflow#who-uses-airflow :)

comment by tedmiston · 2018-08-29 · in Data Engineering Patterns with Apache Airflow [video]

The RBAC UI has since been merged into Airflow and is now released in Airflow 1.10. https://github.com/apache/incubator-airflow/tree/master/airf... https://github.com/apache/incubator-airflow/commit/05e1861e2... To enable it, set `rbac =…

comment by tedmiston · 2018-08-29 · in Data Engineering Patterns with Apache Airflow [video]

…of stuff to do especially if you like working with Python, Docker, JavaScript, web apps, etc. https://github.com/apache/incubator-airflow

comment by tedmiston · 2018-08-24 · in Ask HN: What exciting open-source science projects can one contribute to?

…and are used in production environments as well. [0] https://github.com/spotify/luigi [1] https://github.com/apache/incubator-airflow

comment by danthelion · 2018-08-17 · in Ask HN: Simple, beginner friendly ETL / Data Engineering project ideas?

…like orchestration and dynamic provisioning on a platform actually work all the way down. [1]: https://github.com/apache/incubator-airflow/ [2]: https://www.astronomer.io/

comment by tedmiston · 2018-05-17 · in Ask HN: What are you working on at your company or startup?

…a SaaS as well as an enterprise distribution with monitoring tools to build upon core Airflow. [1]: https://github.com/apache/incubator-airflow [2]: https://www.astronomer.io/

comment by tedmiston · 2018-05-14 · in Ask HN: How do you handle long-running workflows at your company?

This seems similar to Apache Airflow: https://github.com/apache/incubator-airflow/blob/master/READ...

comment by brylie · 2018-03-26 · in Tools for converting Python code to AWS Step Function JSON

…We like to help people get started with our Airflow and have written a bit of content for it. [1]: https://github.com/apache/incubator-airflow/blob/ba126926d7d... [2]: https://airflow.apache.org/code.html?highlight=dockeroperato... [3]:…

comment by tedmiston · 2018-02-23 · in Retiring Wave

…but even they have me worried. Airflow, originating from AirBNB, is now Apache Airflow: https://github.com/apache/incubator-airflow -- And I've been super hesitant to throw weight at it because of that, even though it looks actively…

comment by scrollaway · 2018-02-23 · in Retiring Wave

…In a phrase, I'd say they're like "Make for data". [0]: https://github.com/spotify/luigi [1]: https://github.com/apache/incubator-airflow

comment by abd12 · 2017-09-12 · in Ask HN: Data pipelines and ETLs

They've been listed as contributors on the Apache Airflow repo[0] [0] https://github.com/apache/incubator-airflow/blame/master/REA...

comment by concam · 2017-03-01 · in Airflow and the Future of Data Engineering: A Q&A

…know that hackernews comment is not the most appropriate place for bug report, but pressing DAG in https://github.com/apache/incubator-airflow/blob/master/airf... or set_downstream in https://github.com/apache/incubator-airflow#L37 does…

comment by kozikow · 2016-09-19 · in Building Sourcegraph - A large-scale code search engine in Go (2014)

…based, it should be fairly straightforward to implement something similar for IRC/IMAP/FTP/etc. [1] https://github.com/apache/incubator-airflow [2] http://oozie.apache.org/ [3] https://github.com/spotify/luigi [4]…

comment by reubano · 2016-07-21 · in Show HN: Riko – A Python stream processing engine modeled after Yahoo! Pipes