neuralmagic/deepsparse
View on GitHub ↗ ★ 3,152 Python archived
Sparsity-aware deep learning inference runtime for CPUs
7 mentions on Hacker News by 6 people: 1 story, 6 comments · first 2021-09-10 · latest 2023-11-23
20082026
-
…and they went into some depth: https://www.youtube.com/watch?v=0PAiQ1jTN5k DeepSparse is on GitHub: https://github.com/neuralmagic/deepsparse
comment by RossBencina · 2023-11-23 · in Fast Llama 2 on CPUs with Sparse Fine-Tuning and DeepSparse -
…what was done for Ultralytics benchmarking. This code is open sourced for viewing and use here: https://github.com/neuralmagic/deepsparse/blob/main/examples...
comment by markurtz · 2021-09-11 · in YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance -
…we'd love to see what you think of the performance using the DeepSparse engine for CPU inference: https://github.com/neuralmagic/deepsparse Take a look through our getting started pages that walk through performance benchmarking, training,…
comment by markurtz · 2021-09-11 · in YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance -
I was referring to the benchmark script from the article https://github.com/neuralmagic/deepsparse/blob/main/examples... It looks like they used torch.load instead of torch.jit.load so I don’t think it’s scripted here, but I didn’t check…
comment by chessgecko · 2021-09-11 · in YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance -
…"Based on the amount of work for support, it's on our medium to long-term roadmap currently." https://github.com/neuralmagic/deepsparse/issues/183
comment by ddv · 2021-09-10 · in YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance -
…plan to support ARM CPUs in their runtime engine (currently looks like just a few AMD & Intel CPUs: https://github.com/neuralmagic/deepsparse ). Many IoT and embedded devices with application-class processors could benefit from these…
comment by ramzyo · 2021-09-10 · in YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance