guidance-ai/guidance on Hacker News

Congrats on the launch! What's different between Jev and Microsoft's Guidance package? https://github.com/guidance-ai/guidance Is it a diffusion generator under the hood?

comment by theredsix · 2026-09-15 · in Introducing System One Models and Jev

trumps voice wrong https://developers.openai.com/api/docs/guides/structured-out... https://github.com/guidance-ai/guidance https://github.com/noamgat/lm-format-enforcer https://github.com/mlc-ai/xgrammar…

comment by Der_Einzige · 2026-06-15 · in Not everyone is using AI for everything

…- Guidance (already covered by FlyingLawnmower in this thread), another nice library - https://github.com/guidance-ai/guidance - XGrammar, a less-featureful but really well optimized constrained generation library -…

comment by mcyc · 2026-01-17 · in LLM Structured Outputs Handbook

…grammar (so long as the backend LLM has full support for Guidance). More on this below." --from https://github.com/guidance-ai/guidance/ I didn't find any more on that comment below. Is there a list of supported LLMs?

comment by ru552 · 2025-09-23 · in Sampling and structured outputs in LLMs

…on how this works: User friendly library that connects to lots of OSS model serving backends: https://github.com/guidance-ai/guidance/ Core Rust library written for high performance mask computation (written mostly by my collaborator…

comment by FlyingLawnmower · 2025-09-23 · in Sampling and structured outputs in LLMs

…tokens are valid JSON immediately after a `{`, etc. [0] https://github.com/dottxt-ai/outlines [1] https://github.com/guidance-ai/guidance

comment by evnc · 2025-08-07 · in GPT-5 for Developers

…degradation and reliability issues. My next experiment on that thread is to either use guidance ( https://github.com/guidance-ai/guidance ) or reimplement some of their heuristics to try to get tool calling without 100% reliance on JSON.

comment by hedgehog · 2025-04-10 · in Smartfunc: Turn Docstrings into LLM-Functions

…much I appreciate your work. I'm curious if you have considered implementing Microsoft's Guidance ( https://github.com/guidance-ai/guidance )? Their approach offers significant speed improvements, which I understand can sometimes be…

comment by mwieler · 2024-12-08 · in Structured Outputs with Ollama

…Guidance which forces the models to write valid JSON as they generate text solves this trivially ( https://github.com/guidance-ai/guidance ) For json in particular these frameworks have functions that take in json schemas or pydantic…

comment by FlyingLawnmower · 2024-08-23 · in StructuredRAG: JSON Response Formatting with Large Language Models

…aware token filling. I hope this implies API-based LLMs may be moving in a similar direction. [1] https://github.com/guidance-ai/guidance

comment by blixt · 2024-08-07 · in Structured Outputs in the API

Via APIs, yes. But if you have direct access to the model you can use libraries like https://github.com/guidance-ai/guidance to manipulate the output structure directly.

comment by rcarmo · 2024-05-29 · in What We Learned from a Year of Building with LLMs

…Have you tried out instructor [0] or Guidance [1]? [0]: https://github.com/jxnl/instructor/ [1]: https://github.com/guidance-ai/guidance/tree/main

comment by Joschkabraun · 2024-04-08 · in Anthropic's Haiku Beats GPT-4 Turbo in Tool Use

…model with focus on JSON tokens and a combination of the technique involved in grammar/guidance https://github.com/guidance-ai/guidance Same principle

comment by AmazingTurtle · 2024-03-19 · in Inversion: Fast, Reliable Structured LLMs

…and others [2] already solved much more elegantly? 0. https://github.com/outlines-dev/outlines 1. https://github.com/guidance-ai/guidance 2. https://github.com/sgl-project/sglang

comment by PheonixPharts · 2024-03-12 · in Show HN: Prompts as WASM Programs

Isn't that also similar to the (formerly?) Microsoft Guidance project? https://github.com/guidance-ai/guidance

comment by nighthawk454 · 2024-03-11 · in Show HN: Prompts as WASM Programs

Mentioned packages https://github.com/eth-sri/lmql https://github.com/outlines-dev/outlines https://github.com/guidance-ai/guidance

comment by nerpderp82 · 2024-03-06 · in Show HN: Fructose – LLM calls as strongly typed functions

Why do you have Guidance in caps? https://github.com/guidance-ai/guidance or ... https://huggingface.co/docs/text-generation-inference/concep... or ... ? A quick glance through these, they don't seem yet to leverage json_object on OpenAI…

comment by Terretta · 2024-03-06 · in Show HN: Fructose – LLM calls as strongly typed functions

These may be useful https://github.com/ggerganov/llama.cpp/blob/master/grammars https://github.com/guidance-ai/guidance?tab=readme-ov-file#c... https://github.com/eth-sri/lmql/issues/172

comment by isaacfung · 2024-02-27 · in Show HN: Supermaven, the first code completion tool with 300k token context

…prompting and external tools. As others have mentioned, instruct tuning would help too. [0] https://github.com/guidance-ai/guidance

comment by batch12 · 2024-01-07 · in LiteLlama-460M-1T has 460M parameters trained with 1T tokens

Microsoft Guidance: https://github.com/guidance-ai/guidance

comment by treprinum · 2023-11-01 · in New: LangChain templates – fastest way to build a production-ready LLM app

…to converse with each other. The original comparison to LangChain from Microsoft was Guidance ( https://github.com/guidance-ai/guidance ) which appears to have shifted development a bit. I haven't had much experience with it but from the…

comment by minimaxir · 2023-11-01 · in New: LangChain templates – fastest way to build a production-ready LLM app

…of our models to ensure safety. It is also possible to restrict the output domain of our models ( https://github.com/guidance-ai/guidance ). Instead of allowing our LVMs to output any words, we could restrict it to only being able to…

comment by baptiste1 · 2023-10-28 · in Is supervised learning dead for computer vision?

…data if it wasn't a simple extraction. Hopefully OpenAI and others will offer something like https://github.com/guidance-ai/guidance at some point to guarantee overall output structure. Failed validations will retry, but from what I've…

comment by jrhizor · 2023-10-16 · in Show HN: Elelem – TypeScript LLMs with tracing, retries, and type safety

…that implements generating structured output like https://github.com/outlines-dev/outlines or https://github.com/guidance-ai/guidance . If you have ideas how this should be done let me know - on a github issue would be great to make it…

comment by jackmpcollins · 2023-09-26 · in Show HN: Magentic – Use LLMs as simple Python functions

…comes down to "pausing" template rendering and calling the LLM with all content generated so far. https://github.com/guidance-ai/guidance/blob/main/guidance/l... This is how we implemented it anyhow, with some more parameters to control…

comment by verdverm · 2023-09-17 · in Guidance: A guidance language for controlling large language models

This IS Microsoft Guidance, they seem to have spun off a separate GitHub organization for it. https://github.com/microsoft/guidance redirects to https://github.com/guidance-ai/guidance now.

comment by simonw · 2023-09-17 · in Guidance: A guidance language for controlling large language models

…a grammar, for example to guarantee JSON output 100% of the time. Here's one library to do this https://github.com/guidance-ai/guidance

comment by omneity · 2023-09-15 · in Optimizing LLMs from a Dataset Perspective

…likely help improve this problem. [1] A guidance language for controlling large language models. https://github.com/guidance-ai/guidance [2] Knowledge Infused Decoding https://arxiv.org/abs/2204.03084

comment by vykthur · 2023-08-29 · in Automatic Generation of Visualizations and Infographics with LLMs

…this specifically but logit control/manipulation is often used to conform to a specific schema. https://github.com/guidance-ai/guidance I'm going to butcher this explanation - after you've generated your selection of logits but before you…

comment by thewataccount · 2023-08-29 · in Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper

Microsoft Guidance integration? https://github.com/guidance-ai/guidance

comment by adamgordonbell · 2023-08-17 · in DeepEval – Unit Testing for LLMs

…the output of LLMs (so you can, among many other things, output JSON in a deterministic way) [1]: https://github.com/guidance-ai/guidance

comment by rmonvfer · 2023-08-15 · in Show HN: LLMs can generate valid JSON 100% of the time

…not even guarantee the output will be valid JSON. When this is, really, a solved problem. I've been using github.com/microsoft/guidance for weeks, and it genuinely, truly guarantees correct output, because it simply does not sample from…

comment by 2bitencryption · 2023-08-14 · in Show HN: LLMs can generate valid JSON 100% of the time

…[0] https://openai.com/blog/function-calling-and-other-api-updat... [1] https://resgen.app [2] https://github.com/guidance-ai/guidance

comment by Ilasky · 2023-08-14 · in Show HN: LLMs can generate valid JSON 100% of the time

LQML (and guidance https://github.com/guidance-ai/guidance ) are much more inefficient. They loop over the entire vocabulary at each step, we only do it once at initialization.

comment by remilouf · 2023-08-14 · in Show HN: LLMs can generate valid JSON 100% of the time

LMQL - https://lmql.ai/ Guidance (microsoft) - almost abandoned - https://github.com/microsoft/guidance

comment by ploppyploppy · 2023-08-14 · in LangChain Agent Simulation – Multi-Player Dungeons and Dragons

…but its fairly rote and I think other prompting chains have a similar repetitiveness to them. https://github.com/microsoft/guidance

comment by adamgordonbell · 2023-07-29 · in Show HN: LLMFlows – LangChain alternative for explicit and transparent apps

…of being able to upload a bit of code or a context-free grammar to guide the model is super smart. https://github.com/microsoft/guidance/blob/d2c5e3cbb730e337b...

comment by ttul · 2023-07-22 · in Llama: Add grammar-based sampling

Isn’t this what Microsoft Guidance does? https://github.com/microsoft/guidance

comment by eightysixfour · 2023-07-21 · in Llama: Add grammar-based sampling

…that conforms to the requested format . This is what Guidance does already, also from Microsoft: https://github.com/microsoft/guidance But OpenAI apparently does not expose the full scores of all tokens, it only exposes the highest-scoring…

comment by andy_xor_andrew · 2023-07-21 · in TypeChat

…process of building this very thing and good ol' Misrocoft just swung in and ate my lunch.. :/ [0] https://github.com/microsoft/guidance [1] https://github.com/microsoft/TypeChat/blob/main/src/typechat...

comment by parentheses · 2023-07-21 · in TypeChat

…not super clear how this differs from another recently released library from Microsoft: Guidance ( https://github.com/microsoft/guidance ). They both seem to aim to solve the problem of getting typed, valid responses back from LLMs

comment by Zaheer · 2023-07-20 · in TypeChat

…similar project that can actually integrate into the token sampling to enforce formats. [1]: https://github.com/microsoft/guidance

comment by osaariki · 2023-07-20 · in TypeChat

…that seems incredibly useful to local inference [token healing and acceleration especially]. [0] https://github.com/microsoft/guidance

comment by ricopags · 2023-07-19 · in Accessing Llama 2 from the command-line with the LLM-replicate plugin

…being the models these prompts were run on. So maybe a simpler library like Microsoft's Guidance ( https://github.com/microsoft/guidance )? It does this really well. Also LangChain has a lot of integrations, that just pop up as soon as new…

comment by reallymental · 2023-07-18 · in AutoChain, lightweight and testable alternative to LangChain

I am unclear on the status of the project but here is the conversation that seem to be tracking it: https://github.com/microsoft/guidance/discussions/201

comment by nestorD · 2023-07-17 · in Show HN: Structured output from LLMs without reprompting

What are the benefits over https://github.com/microsoft/guidance/ ?

comment by sunshadow · 2023-07-16 · in Show HN: Structured output from LLMs without reprompting

Have you tried guidance? https://github.com/microsoft/guidance

comment by vczf · 2023-07-07 · in GPT-4 API General Availability

…And it has its own caching. Set temp to 0 and it caches. no need for LLM specific caching libs :) ) https://github.com/microsoft/guidance

comment by adamgordonbell · 2023-06-21 · in Emerging architectures for LLM applications

…like Guidance where you have a pre structured JSON format ready to go and all you need is text: https://github.com/microsoft/guidance

comment by imranq · 2023-06-14 · in Native JSON Output from GPT-4

…to fine-tuning validator models to new programming/modeling languages. Here's some examples: - https://github.com/microsoft/guidance - https://github.com/NVIDIA/NeMo-Guardrails/ - https://github.com/r2d4/rellm -…

comment by killthebuddha · 2023-06-07 · in Run and create custom ChatGPT-like bots with OpenChat