Hacker GitHub

← All repos

que-rb/que

View on GitHub ↗ ★ 2,326 Ruby

A Ruby job queue that uses PostgreSQL's advisory locks for speed and reliability.

26 mentions on Hacker News by 19 people: 4 stories, 22 comments · first 2013-11-19 · latest 2024-08-19

Mentions per year 26 mentions · busiest year 2023 (5)
20082026
AllStoriesComments NewestTopOldest
  1. …- https://github.com/graphile/worker - https://github.com/pgq/pgq - https://github.com/que-rb/que Probably could easily find more by searching, I only spent about 5 minutes looking and grabbing the first ones I found. I'm all
    comment by SOLAR_FIELDS · 2024-08-19 · in Show HN: PgQueuer – Transform PostgreSQL into a Job Queue
  2. …with Oban Web / Pro :) We've also had a lot of experience with with other libraries like Que ( https://github.com/que-rb/que ) and Sidekiq ( https://sidekiq.org/ ) which have certainly influenced us over the years.
    comment by bgentry · 2023-11-20 · in River: A fast, robust job queue for Go and Postgres
  3. …But you'll probably want to do something with lower overhead. FWIW, Que uses advisory locks [1] [1] https://github.com/que-rb/que
    comment by paulddraper · 2023-09-25 · in Choose Postgres queue technology
  4. …(and in postgres you should not use long transactions) basically we use c#, but we looked into https://github.com/que-rb/que which uses advisory_locks, since our jobs take like 1 min to 2 hours it was a no-brainer to use advisory_locks.…
    comment by merb · 2023-09-25 · in Choose Postgres queue technology
  5. …to several languages. [0] https://www.postgresql.org/docs/current/explicit-locking.htm... [1] https://github.com/que-rb/que
    comment by jpgvm · 2023-09-24 · in Choose Postgres queue technology
  6. …than row-level locks and notification channels to great effect, as you can read in the README. [1]: https://github.com/que-rb/que
    comment by zrail · 2023-04-11 · in SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a Postgres Queue
  7. https://github.com/que-rb/que This one seems to be the most performant. By a lot too, from my understanding (haven't ran any benchmark myself, but the readme shows some good postgres knowledge)
    comment by Fire-Dragon-DoL · 2022-02-16 · in Delayed Job vs. Sidekiq
  8. Implementations that use advisory locks like https://github.com/que-rb/que are much more efficient (atleast when I last tested) and will easily reach 10k job/s on even very modest hardware. There is a Go port of Que but you can also easily…
    comment by jpgvm · 2022-01-29 · in Devious SQL: Message Queuing Using Native PostgreSQL
  9. …volume of jobs and for a variety of reasons benefits from this design over other solutions. [1]: https://github.com/que-rb/que [2]: https://brandur.org/postgres-queues
    comment by amarshall · 2021-12-17 · in Postgres is a great pub/sub and job server (2019)
  10. …to not be lost because atomically the pop and push from two different lists are done together. [1] https://github.com/que-rb/que [2] https://gist.github.com/chanks/7585810
    comment by skunkworker · 2021-06-12 · in Do you really need Redis? How to get away with just PostgreSQL
  11. …rails I’ve used que in the past, which is a job queue adapter for Postgres. It supports activejob. https://github.com/que-rb/que
    comment by mosselman · 2021-06-12 · in Do you really need Redis? How to get away with just PostgreSQL
  12. …It can plug into ActiveJob. Those projects are still humming along without trouble today. [0]: https://github.com/que-rb/que
    comment by scruple · 2020-07-23 · in GoodJob – a Postgres-based ActiveJob back end for Ruby on Rails
  13. …all of its features and guarantees) to the receiving service's database where their workers (e.g. https://github.com/que-rb/que , skip locked polling, etc) are waiting to respond by inserting messages into their database to be replicated…
    comment by bonquesha99 · 2020-05-17 · in What every software engineer should know about Apache Kafka
  14. …crashes, the worker supervisor catches this and the job is marked for a retry at a later time. [1]: https://github.com/que-rb/que/blob/master/lib/que/migrations...
    comment by bgentry · 2019-11-14 · in Turning PostgreSQL into a queue serving 10k jobs per second (2013)
  15. Is this what you’re talking about? https://github.com/que-rb/que
    comment by ianai · 2019-11-14 · in Turning PostgreSQL into a queue serving 10k jobs per second (2013)
  16. The author of this post, Chris Hanks, created the Que queueing library for Ruby: https://github.com/que-rb/que It’s changed significantly since this post as the 1.x betas use a very different structure which should actually be more…
    comment by bgentry · 2019-11-14 · in Turning PostgreSQL into a queue serving 10k jobs per second (2013)
  17. …experiences with Que[1], which implements a pattern similar to the OP using advisory locks. [1]: https://github.com/que-rb/que
    comment by zrail · 2019-11-08 · in System design hack: Postgres is a great pub/sub and job server
  18. So how different is this from something like https://github.com/chanks/que
    comment by brightball · 2016-07-17 · in A Wait-Free Queue as Fast as Fetch-And-Add [pdf]
  19. …less dependency to worry about. You probably only need Redis if you're running a tonne of jobs. [1] https://github.com/chanks/que
    comment by whatupdave · 2016-01-15 · in Rescuing Resque again (2015)
  20. …queries don't incur writes or block one another like SELECT FOR UPDATE would). Feedback is welcome: https://github.com/chanks/que
    comment by chanks · 2014-09-27 · in Don't Use an RDBMS for Messaging
  21. https://github.com/chanks/que/blob/master/README.md is a postgresql queue that had been benchmarked at 10k/a on AWS.
    comment by joevandyk · 2014-07-14 · in Queues and databases
  22. Have you considered using something like https://github.com/chanks/que/blob/master/README.md for critical jobs?
    comment by chrislloyd · 2014-06-27 · in Too Many Signals – Resque on Heroku