Snowflake-Labs/pg_lake
View on GitHub ↗ ★ 1,669 Python
pg_lake: Postgres with Iceberg and data lake access
10 mentions on Hacker News by 9 people: 1 story, 9 comments · first 2025-11-04 · latest 2026-08-10
20082026
-
Apologies, I could have introduced that better. The article links https://github.com/Snowflake-Labs/pg_lake but if you go looking for CDC, it's not there, and all you end up with is links to the private/closed repo that I just linked, in…
comment by jauntywundrkind · 2026-08-10 · in How We Pushed CDC into Postgres -
…- you can find the repo here [1] and a blog post with more context on the project here [2] [1] https://github.com/Snowflake-Labs/pg_lake [2] https://www.snowflake.com/en/blog/engineering/pg-lake-postgr...
comment by plaur782 · 2026-08-10 · in How We Pushed CDC into Postgres -
…it external, routing through Postgres and managing Iceberg tables (but not the data itself) there ( https://github.com/Snowflake-Labs/pg_lake ). Both of these were neck and neck with ClickHouse last time I tried them.
comment by FreakLegion · 2026-07-10 · in Postgres rewritten in Rust, now passing 100% of the Postgres regression tests -
…open source implementation on how else you could build this solution, if you're familiar with PG: https://github.com/Snowflake-Labs/pg_lake All up, I feel this explanation is written from a MySQL-first perspective. "We built this valuable…
comment by ozgune · 2026-02-04 · in AliSQL: Alibaba's open-source MySQL with vector and DuckDB engines -
…in S3 and is stored in Parquet files, safely/transactionally managed via the Iceberg format. https://github.com/Snowflake-Labs/pg_lake/blob/main/docs/ice...
comment by ch71r22 · 2025-11-05 · in Pg_lake: Postgres with Iceberg and data lake access -
…and otherwise stores as text representation, there are a few limitations like very large numerics. https://github.com/Snowflake-Labs/pg_lake/blob/main/docs/ice...
comment by mslot · 2025-11-05 · in Pg_lake: Postgres with Iceberg and data lake access -
…ETL job, I probably wouldn't do it in pg_cron. Think "tiered storage." See the example under https://github.com/Snowflake-Labs/pg_lake/blob/main/docs/ice... : select cron.schedule('flush-queue', '* * * * *', $$ with new_rows as ( delete…
comment by derefr · 2025-11-04 · in Pg_lake: Postgres with Iceberg and data lake access -
…external iceberg table read-only support, but it is limited at the moment. See this example/caveat: https://github.com/Snowflake-Labs/pg_lake/blob/main/docs/fil...
comment by pgguru · 2025-11-04 · in Pg_lake: Postgres with Iceberg and data lake access -
Maybe this could help: https://github.com/Snowflake-Labs/pg_lake?tab=readme-ov-file...
comment by onderkalaci · 2025-11-04 · in Pg_lake: Postgres with Iceberg and data lake access