http://www.bing.com:80/search?q=Quantization+Machine+LearningSearch resultshttp://www.bing.com:80/s/a/rsslogo.gifhttp://www.bing.com:80/search?q=Quantization+Machine+LearningCopyright © 2026 Microsoft. All rights reserved. These XML results may not be used, reproduced or transmitted in any manner or for any purpose other than rendering Bing results within an RSS aggregator for your personal, non-commercial use. Any other use of these results requires express written permission from Microsoft Corporation. By accessing this web page or using these results in any manner whatsoever, you agree to be bound by the foregoing restrictions.https://github.com/sgl-project/sglang/blob/main/docs/advanced_features/quantization.mdSGLang supports various quantization methods, including offline quantization and online dynamic quantization. Offline quantization loads pre-quantized model weights directly during inference. This is required for quantization methods such as GPTQ and AWQ, which collect and pre-compute various statistics from the original weights using the calibration dataset. Online quantization dynamically ...Tue, 23 Jun 2026 16:19:00 GMThttps://github.com/sgl-project/sglang/issues/4524According to my tests, static fp8 quantization provides almost 10% higher throughput than dynamic quantization, and with tensor parallelism 2, the difference increases to 30% (H100).Sat, 20 Jun 2026 05:16:00 GMThttps://github.com/sgl-project/sglang/issues/21618🚀 Feature Overview Add support for TurboQuant KV cache quantization to enable memory-efficient long-context LLM inference with near-lossless quality. TurboQuant is a novel online vector quantization method from the Google ICLR 2026 paper...Sun, 29 Mar 2026 05:22:00 GMThttps://github.com/sgl-project/sglang/blob/main/python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.pySGLang is a high-performance serving framework for large language models and multimodal models. - sgl-project/sglangMon, 22 Jun 2026 09:10:00 GMThttps://github.com/sgl-project/sglang/blob/main/python/sglang/srt/layers/quantization/base_config.pySGLang is a high-performance serving framework for large language models and multimodal models. - sgl-project/sglangWed, 06 May 2026 20:07:00 GMThttps://github.com/sgl-project/sglang/blob/main/python/sglang/srt/layers/quantization/fp8_utils.pySGLang is a high-performance serving framework for large language models and multimodal models. - sgl-project/sglangThu, 25 Jun 2026 16:52:00 GMThttps://github.com/sgl-project/sglang/blob/main/benchmark/kernels/quantization/tuning_block_wise_kernel.pySGLang is a high-performance serving framework for large language models and multimodal models. - sgl-project/sglangThu, 18 Jun 2026 05:04:00 GMThttps://github.com/sgl-project/sglang/releasesSGLang is a high-performance serving framework for large language models and multimodal models. - sgl-project/sglangSun, 28 Jun 2026 16:20:00 GMThttps://github.com/sgl-project/sglang/issues/21159This work integrates Intel AutoRound, a post-training quantization (PTQ) toolkit that supports large language models (LLMs), vision-language models (VLMs), and diffusion models. which performs gradient-based optimization of weight rounding and clipping, allowing models to be quantized to low bit widths (2–4 bits) while maintaining strong ...Mon, 23 Mar 2026 03:13:00 GMThttps://github.com/sgl-project/sglang/issues/10641ValueError: The output_size of gate's and up's weight = 64 is not divisible by weight quantization block_n = 128. #10641Sun, 21 Jun 2026 23:45:00 GMT