qalloc

For a model and any size you choose, the per-tensor quantization with the lowest predicted mean KL divergence from the original: an exact knapsack over every tensor’s types, solved in your browser, and the GGUF assembled there too.

Open the allocator →

Early version, shared quietly: the tool works, but the Qwen3.8-27B shard store is due a rebuild with a broader imatrix, and its measured numbers below will be redone then. The LFM2.5-2.6B numbers are not affected.

Usage

• Pick any size; set types, freeze regions or weight tensors, and see what each choice costs.
• Download the result: it streams from the shard store, only the tensors it uses, each sha256-checked, straight to disk.

Models

Measurements

Each published file, as you would download it, against qalloc’s build at the same size,¹¹ For unsloth’s files, at their size without the MTP layer, which qalloc leaves to the download. on two texts: WikiText-2, and bartowski’s calibration-v6. Mean KL divergence from the BF16 model, as llama-perplexity measures it, with qalloc’s change against the release; and the change in how often the top token agrees with the original’s.

Caveat: the Qwen3.8-27B store. Its tensors were quantized with a suboptimal imatrix, computed on chat data only, and they underperform on plain text. Every 27B build below inherits that. The store will be rebuilt with a broader imatrix, and these numbers measured again.

At a glance: qalloc’s mean KL against the release’s at the same size. Lower is closer to the original, so negative is better.

LFM2.5-2.6B

Against bartowski’s quants and LiquidAI’s own. The store holds bartowski’s own tensors, from the same BF16 weights and his imatrix, so against his files the difference is the allocation alone. qalloc’s builds are closer to the original at every size, on both texts, and agree with its top token more often.²² LiquidAI’s files are quantized without an imatrix, which is part of their gap. Its KL model is fitted to the assistant turns of calibration-v6, so WikiText-2 is the out-of-sample check.

Qwen3.8-27B

Against unsloth’s dynamic quants and ISTA-DASLab’s GSQ-RCO. On bartowski’s calibration text qalloc is level with them, a little ahead of unsloth and a little behind ISTA. On WikiText-2 the published files stay closer to the original.³³ ISTA’s quantizer computes its own weights, so their files can only be compared as published.

With the same tensors. To take the store out of the comparison, unsloth’s own recipes were also built from it: their choices of type per tensor, with the same underperforming tensors qalloc has to work with. On that equal footing qalloc is ahead on calibration-v6 at three of the four sizes and level at the smallest, and on WikiText-2 the gap narrows from +63 % to +91 % against their published files to +20 % to +35 %.

Limitations

• LFM2.5 below about 2 bits per weight: the prediction is optimistic, and the real KL is worse. The model is barely usable there anyway.