vlm-run/vlmrun-hub
View on GitHub ↗ ★ 554 Python
A hub for various industry-specific schemas to be used with VLMs.
9 mentions on Hacker News by 2 people: 1 story, 8 comments · first 2025-02-20 · latest 2025-02-27
20082026
-
We’re adding this as we speak. Ollama support is already there, and here’s vLLM inference: https://github.com/vlm-run/vlmrun-hub/pull/120
comment by fzysingularity · 2025-02-27 · in Replace OCR with Vision Language Models -
BTW Check out the Gemini qualitative results here in our hub: https://github.com/vlm-run/vlmrun-hub?tab=readme-ov-file#-qu... . It gives you an idea of where today's models fail (Gemini Flash, OpenAI gpt4o+mini, open-source ones like Llama…
comment by fzysingularity · 2025-02-27 · in Replace OCR with Vision Language Models -
You can try out some of our schemas with Ollama if you want: https://github.com/vlm-run/vlmrun-hub (instructions in Readme)
comment by EarlyOom · 2025-02-26 · in Replace OCR with Vision Language Models -
You can! it works with Ollama https://github.com/vlm-run/vlmrun-hub At the end of the day its just schemas. You can decide for yourself if its work upgrading to a larger, more expensive model.
comment by EarlyOom · 2025-02-26 · in Replace OCR with Vision Language Models -
Just ran them for Qwen2.5-VL: https://github.com/vlm-run/vlmrun-hub/blob/main/tests/benchm...
comment by fzysingularity · 2025-02-22 · in Run structured extraction on documents/images locally with Ollama and Pydantic -
…days if there's enough folks who could benefit from this. A basic invoice schema is already there: https://github.com/vlm-run/vlmrun-hub/blob/main/vlmrun/hub/s... You can see some of the qualitative results on GPT4o, Gemini, Llama 3.2 11B,…
comment by fzysingularity · 2025-02-20 · in Run structured extraction on documents/images locally with Ollama and Pydantic -
…for videos in the next few weeks. A few video schemas are already added to the main catalog: https://github.com/vlm-run/vlmrun-hub/blob/main/vlmrun/hub/c...
comment by fzysingularity · 2025-02-20 · in Run structured extraction on documents/images locally with Ollama and Pydantic -
…a try with a few of the schemas we have. We’ve locally tested with Llama 3.2 11B Vision on Ollama: https://github.com/vlm-run/vlmrun-hub/blob/main/tests/benchm... FWIW I think Ollama structured outputs API is quite buggy compared to the HF…
comment by fzysingularity · 2025-02-20 · in Run structured extraction on documents/images locally with Ollama and Pydantic -
▲ 170
Run structured extraction on documents/images locally with Ollama and Pydanticstory by EarlyOom · 2025-02-20 · 29 comments