Hacker GitHub

← All repos

guidance-ai/guidance

View on GitHub ↗ ★ 21,813 Jupyter Notebook

A guidance language for controlling large language models.

72 mentions on Hacker News by 64 people: 9 stories, 63 comments · first 2023-05-09 · latest 2026-09-15

Mentions per year 72 mentions · busiest year 2023 (52)
20082026
AllStoriesComments NewestTopOldest
  1. Congrats on the launch! What's different between Jev and Microsoft's Guidance package? https://github.com/guidance-ai/guidance Is it a diffusion generator under the hood?
    comment by theredsix · 2026-09-15 · in Introducing System One Models and Jev
  2. trumps voice wrong https://developers.openai.com/api/docs/guides/structured-out... https://github.com/guidance-ai/guidance https://github.com/noamgat/lm-format-enforcer https://github.com/mlc-ai/xgrammar…
    comment by Der_Einzige · 2026-06-15 · in Not everyone is using AI for everything
  3. ▲ 1
    True explanation behind the Guidance project
    story by StrauXX · 2026-01-19 · 1 comment
  4. …- Guidance (already covered by FlyingLawnmower in this thread), another nice library - https://github.com/guidance-ai/guidance - XGrammar, a less-featureful but really well optimized constrained generation library -…
    comment by mcyc · 2026-01-17 · in LLM Structured Outputs Handbook
  5. …grammar (so long as the backend LLM has full support for Guidance). More on this below." --from https://github.com/guidance-ai/guidance/ I didn't find any more on that comment below. Is there a list of supported LLMs?
    comment by ru552 · 2025-09-23 · in Sampling and structured outputs in LLMs
  6. …on how this works: User friendly library that connects to lots of OSS model serving backends: https://github.com/guidance-ai/guidance/ Core Rust library written for high performance mask computation (written mostly by my collaborator…
    comment by FlyingLawnmower · 2025-09-23 · in Sampling and structured outputs in LLMs
  7. …tokens are valid JSON immediately after a `{`, etc. [0] https://github.com/dottxt-ai/outlines [1] https://github.com/guidance-ai/guidance
    comment by evnc · 2025-08-07 · in GPT-5 for Developers
  8. …degradation and reliability issues. My next experiment on that thread is to either use guidance ( https://github.com/guidance-ai/guidance ) or reimplement some of their heuristics to try to get tool calling without 100% reliance on JSON.
    comment by hedgehog · 2025-04-10 · in Smartfunc: Turn Docstrings into LLM-Functions
  9. …much I appreciate your work. I'm curious if you have considered implementing Microsoft's Guidance ( https://github.com/guidance-ai/guidance )? Their approach offers significant speed improvements, which I understand can sometimes be…
    comment by mwieler · 2024-12-08 · in Structured Outputs with Ollama
  10. …Guidance which forces the models to write valid JSON as they generate text solves this trivially ( https://github.com/guidance-ai/guidance ) For json in particular these frameworks have functions that take in json schemas or pydantic…
    comment by FlyingLawnmower · 2024-08-23 · in StructuredRAG: JSON Response Formatting with Large Language Models
  11. …aware token filling. I hope this implies API-based LLMs may be moving in a similar direction. [1] https://github.com/guidance-ai/guidance
    comment by blixt · 2024-08-07 · in Structured Outputs in the API
  12. Via APIs, yes. But if you have direct access to the model you can use libraries like https://github.com/guidance-ai/guidance to manipulate the output structure directly.
    comment by rcarmo · 2024-05-29 · in What We Learned from a Year of Building with LLMs
  13. …Have you tried out instructor [0] or Guidance [1]? [0]: https://github.com/jxnl/instructor/ [1]: https://github.com/guidance-ai/guidance/tree/main
    comment by Joschkabraun · 2024-04-08 · in Anthropic's Haiku Beats GPT-4 Turbo in Tool Use
  14. …model with focus on JSON tokens and a combination of the technique involved in grammar/guidance https://github.com/guidance-ai/guidance Same principle
    comment by AmazingTurtle · 2024-03-19 · in Inversion: Fast, Reliable Structured LLMs
  15. …and others [2] already solved much more elegantly? 0. https://github.com/outlines-dev/outlines 1. https://github.com/guidance-ai/guidance 2. https://github.com/sgl-project/sglang
    comment by PheonixPharts · 2024-03-12 · in Show HN: Prompts as WASM Programs
  16. Isn't that also similar to the (formerly?) Microsoft Guidance project? https://github.com/guidance-ai/guidance
    comment by nighthawk454 · 2024-03-11 · in Show HN: Prompts as WASM Programs
  17. Mentioned packages https://github.com/eth-sri/lmql https://github.com/outlines-dev/outlines https://github.com/guidance-ai/guidance
    comment by nerpderp82 · 2024-03-06 · in Show HN: Fructose – LLM calls as strongly typed functions
  18. Why do you have Guidance in caps? https://github.com/guidance-ai/guidance or ... https://huggingface.co/docs/text-generation-inference/concep... or ... ? A quick glance through these, they don't seem yet to leverage json_object on OpenAI…
    comment by Terretta · 2024-03-06 · in Show HN: Fructose – LLM calls as strongly typed functions
  19. These may be useful https://github.com/ggerganov/llama.cpp/blob/master/grammars https://github.com/guidance-ai/guidance?tab=readme-ov-file#c... https://github.com/eth-sri/lmql/issues/172
    comment by isaacfung · 2024-02-27 · in Show HN: Supermaven, the first code completion tool with 300k token context
  20. …prompting and external tools. As others have mentioned, instruct tuning would help too. [0] https://github.com/guidance-ai/guidance
    comment by batch12 · 2024-01-07 · in LiteLlama-460M-1T has 460M parameters trained with 1T tokens
  21. Microsoft Guidance: https://github.com/guidance-ai/guidance
    comment by treprinum · 2023-11-01 · in New: LangChain templates – fastest way to build a production-ready LLM app
  22. …to converse with each other. The original comparison to LangChain from Microsoft was Guidance ( https://github.com/guidance-ai/guidance ) which appears to have shifted development a bit. I haven't had much experience with it but from the…
    comment by minimaxir · 2023-11-01 · in New: LangChain templates – fastest way to build a production-ready LLM app
  23. …of our models to ensure safety. It is also possible to restrict the output domain of our models ( https://github.com/guidance-ai/guidance ). Instead of allowing our LVMs to output any words, we could restrict it to only being able to…
    comment by baptiste1 · 2023-10-28 · in Is supervised learning dead for computer vision?
  24. …data if it wasn't a simple extraction. Hopefully OpenAI and others will offer something like https://github.com/guidance-ai/guidance at some point to guarantee overall output structure. Failed validations will retry, but from what I've…
    comment by jrhizor · 2023-10-16 · in Show HN: Elelem – TypeScript LLMs with tracing, retries, and type safety
  25. …that implements generating structured output like https://github.com/outlines-dev/outlines or https://github.com/guidance-ai/guidance . If you have ideas how this should be done let me know - on a github issue would be great to make it…
    comment by jackmpcollins · 2023-09-26 · in Show HN: Magentic – Use LLMs as simple Python functions
  26. …comes down to "pausing" template rendering and calling the LLM with all content generated so far. https://github.com/guidance-ai/guidance/blob/main/guidance/l... This is how we implemented it anyhow, with some more parameters to control…
    comment by verdverm · 2023-09-17 · in Guidance: A guidance language for controlling large language models
  27. This IS Microsoft Guidance, they seem to have spun off a separate GitHub organization for it. https://github.com/microsoft/guidance redirects to https://github.com/guidance-ai/guidance now.
    comment by simonw · 2023-09-17 · in Guidance: A guidance language for controlling large language models
  28. ▲ 103
    Guidance: A guidance language for controlling large language models
    story by bx376 · 2023-09-16 · 41 comments
  29. …a grammar, for example to guarantee JSON output 100% of the time. Here's one library to do this https://github.com/guidance-ai/guidance
    comment by omneity · 2023-09-15 · in Optimizing LLMs from a Dataset Perspective
  30. ▲ 1
    Microsoft Guidance for LLMs is not active anymore
    story by behnamoh · 2023-09-08 · 1 comment
  31. ▲ 335
    Show HN: HackYourNews – AI summaries of the top HN stories
    story by ukuina · 2023-09-07 · 164 comments
  32. …likely help improve this problem. [1] A guidance language for controlling large language models. https://github.com/guidance-ai/guidance [2] Knowledge Infused Decoding https://arxiv.org/abs/2204.03084
    comment by vykthur · 2023-08-29 · in Automatic Generation of Visualizations and Infographics with LLMs
  33. …this specifically but logit control/manipulation is often used to conform to a specific schema. https://github.com/guidance-ai/guidance I'm going to butcher this explanation - after you've generated your selection of logits but before you…
    comment by thewataccount · 2023-08-29 · in Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper
  34. Microsoft Guidance integration? https://github.com/guidance-ai/guidance
    comment by adamgordonbell · 2023-08-17 · in DeepEval – Unit Testing for LLMs
  35. …the output of LLMs (so you can, among many other things, output JSON in a deterministic way) [1]: https://github.com/guidance-ai/guidance
    comment by rmonvfer · 2023-08-15 · in Show HN: LLMs can generate valid JSON 100% of the time
  36. …not even guarantee the output will be valid JSON. When this is, really, a solved problem. I've been using github.com/microsoft/guidance for weeks, and it genuinely, truly guarantees correct output, because it simply does not sample from…
    comment by 2bitencryption · 2023-08-14 · in Show HN: LLMs can generate valid JSON 100% of the time
  37. …[0] https://openai.com/blog/function-calling-and-other-api-updat... [1] https://resgen.app [2] https://github.com/guidance-ai/guidance
    comment by Ilasky · 2023-08-14 · in Show HN: LLMs can generate valid JSON 100% of the time
  38. LQML (and guidance https://github.com/guidance-ai/guidance ) are much more inefficient. They loop over the entire vocabulary at each step, we only do it once at initialization.
    comment by remilouf · 2023-08-14 · in Show HN: LLMs can generate valid JSON 100% of the time
  39. LMQL - https://lmql.ai/ Guidance (microsoft) - almost abandoned - https://github.com/microsoft/guidance
    comment by ploppyploppy · 2023-08-14 · in LangChain Agent Simulation – Multi-Player Dungeons and Dragons
  40. …but its fairly rote and I think other prompting chains have a similar repetitiveness to them. https://github.com/microsoft/guidance
    comment by adamgordonbell · 2023-07-29 · in Show HN: LLMFlows – LangChain alternative for explicit and transparent apps
  41. …of being able to upload a bit of code or a context-free grammar to guide the model is super smart. https://github.com/microsoft/guidance/blob/d2c5e3cbb730e337b...
    comment by ttul · 2023-07-22 · in Llama: Add grammar-based sampling
  42. Isn’t this what Microsoft Guidance does? https://github.com/microsoft/guidance
    comment by eightysixfour · 2023-07-21 · in Llama: Add grammar-based sampling
  43. …that conforms to the requested format . This is what Guidance does already, also from Microsoft: https://github.com/microsoft/guidance But OpenAI apparently does not expose the full scores of all tokens, it only exposes the highest-scoring…
    comment by andy_xor_andrew · 2023-07-21 · in TypeChat
  44. …process of building this very thing and good ol' Misrocoft just swung in and ate my lunch.. :/ [0] https://github.com/microsoft/guidance [1] https://github.com/microsoft/TypeChat/blob/main/src/typechat...
    comment by parentheses · 2023-07-21 · in TypeChat
  45. …not super clear how this differs from another recently released library from Microsoft: Guidance ( https://github.com/microsoft/guidance ). They both seem to aim to solve the problem of getting typed, valid responses back from LLMs
    comment by Zaheer · 2023-07-20 · in TypeChat
  46. …similar project that can actually integrate into the token sampling to enforce formats. [1]: https://github.com/microsoft/guidance
    comment by osaariki · 2023-07-20 · in TypeChat
  47. …that seems incredibly useful to local inference [token healing and acceleration especially]. [0] https://github.com/microsoft/guidance
    comment by ricopags · 2023-07-19 · in Accessing Llama 2 from the command-line with the LLM-replicate plugin
  48. …being the models these prompts were run on. So maybe a simpler library like Microsoft's Guidance ( https://github.com/microsoft/guidance )? It does this really well. Also LangChain has a lot of integrations, that just pop up as soon as new…
    comment by reallymental · 2023-07-18 · in AutoChain, lightweight and testable alternative to LangChain
  49. I am unclear on the status of the project but here is the conversation that seem to be tracking it: https://github.com/microsoft/guidance/discussions/201
    comment by nestorD · 2023-07-17 · in Show HN: Structured output from LLMs without reprompting
  50. What are the benefits over https://github.com/microsoft/guidance/ ?
    comment by sunshadow · 2023-07-16 · in Show HN: Structured output from LLMs without reprompting