NVIDIA/TensorRT
View on GitHub ↗ ★ 13,443 C++
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
4 mentions on Hacker News by 4 people: 1 story, 3 comments · first 2019-07-02 · latest 2023-12-18
20082026
-
…recognize that momentum is an advantage of being first mover. And it isn't one to scoff at. [0] https://github.com/NVIDIA/TensorRT
comment by godelski · 2023-12-18 · in AMD MI300X 30% higher performance than Nvidia H100, even with optimized stack -
…many of the larger "production" inference tools use compilers: - https://github.com/openai/triton - https://github.com/NVIDIA/TensorRT TVM and other compiler-based approaches seem to really perform really well and make supporting different…
comment by jmorgan · 2023-09-26 · in Ollama for Linux – Run LLMs on Linux with GPU Acceleration -
https://github.com/NVIDIA/TensorRT/issues/982 Looks like tensorRT does work with 30x0, but I couldn't find much.
comment by nkingsy · 2023-09-08 · in NVIDIA introduces TensorRT-LLM for accelerating LLM inference on H100/A100 GPUs -
▲ 1
TensorRT Open Source Softwarestory by linux_devil · 2019-07-02 · 0 comments