Qwen3.6-27B-V2-abliterated-uncensored-OptiQ-3.7bpw-mlx

Yes — VISION PRESERVED — even at 3.7 BPW. Most OptiQ releases strip the vision tower; we keep it. The LM portion is OptiQ static-mixed (3/4/8-bit per tensor), and the ViT is spliced back in at BF16 from our 8-bit MLX build, giving you a full multimodal model that fits in 14 GB.

OptiQ static mixed ~3.7 BPW MLX quantization of a abliterated Qwen 3.6 27B v2 (the Jackrong Claude-Opus reasoning distill of Qwen 3.6 27B). Refusals reduced from 91/100 → 4/100 with KL drift of just 0.0176. By the Lemura Labs research team.

Smallest variant in our line. The LM uses mlx-optiq Frobenius-stream sensitivity scans to assign 3-bit to layers that tolerate it and 4–8-bit to the layers that don't, averaging 3.697 bits per weight. The vision tower (333 ViT weights, ~880 MB at BF16) is spliced back in unmodified from the abliterated parent — the cleanest combination of language compression and visual fidelity.

---

TL;DR

---

All Qwen3.6-27B variants

The full Qwen3.6-27B family from Lemura Labs — same abliterated weights (refusal 4/100, KL 0.0176), different quant schemes for different runtimes.

All variants share the same abliterated base weights — pick by your runtime (Apple Silicon → MLX; CUDA/CPU/cross-platform → GGUF) and your RAM budget.

---

Lineage

Qwen/Qwen3.6-27B (Qwen Team — base multimodal pretrain)
 │
 ▼
Jackrong/Qwopus3.6-27B-v2 (Jackrong — Claude-Opus reasoning distill)
 │
 ▼
ablation abliteration (TPE-50) (Lemura Labs)
 ├── 25 random startup trials
 ├── 2 community priors (coder3101, wangzhang)
 └── 23 TPE smart-sampling trials → best at trial 45
 │
 ▼
mlx-optiq static-mixed 3.7 BPW (LM only) (Lemura Labs)
 ├── Frobenius weight-space sensitivity (stream mode, fits 27B)
 └── Multi-tier bit allocation: 3/4/8-bit
 │
 ▼
ViT spliced back in at BF16 from 8-bit build (Lemura Labs)
 ├── 333 ViT weights, ~880 MB total
 └── VLM config (Qwen3_5ForConditionalGeneration) restored
 │
 ▼
this repo — OptiQ-3.7bpw multimodal MLX

Direct upstream links:

• Foundation: Qwen/Qwen3.6-27B
• Claude-Opus distill: Jackrong/Qwopus3.6-27B-v2
• Abliteration tool: the ablation toolkit by Lemura Labs
• Quantization tool: mlx-optiq by Thin Signal, on top of mlx-lm

---

Abliteration Results

the ablation toolkit measures refusals on mlabonne/harmful_behaviors (100 hard red-team prompts) and KL divergence on mlabonne/harmless_alpaca. These were measured at BF16 before quantization; the OptiQ-3.7bpw quant inherits them within calibration noise.

→ 96% reduction in refusals with capability preserved (KL ≈ 0.018, well below the 0.3 healing threshold). No SFT / LoRA healing was required.

---

Method (TPE-50 with community priors → OptiQ-3.7 → vision merge)

Step 1. Abliteration (the ablation toolkit TPE-50)

1. Smoke — 3-trial dry run on the ablation toolkit / Python 3.11 / MPS validated batch size 4 on M4 Max 128 GB.
2. Random search — 25 startup trials over the ablation toolkit's parameter space (direction_index, attn.o_proj.*, mlp.down_proj.*). Best random-stage trial: T4 at direction_index=54.49 — capability-clean but still too refusing (41/100).
3. Community-prior enqueueing — published Qwen 27B the ablation toolkit runs (coder3101 on Qwen 3.5, wangzhang on Qwen 3.6) localized the refusal direction near layer 35–38 of 64. We injected both as study priors via study.enqueue_trial(skip_if_exists=True).
4. TPE smart-sampling — 23 Tree-structured Parzen Estimator trials refined around the community neighborhood. Trial 45 (direction_index=41.42) found the Pareto front: 4/100 refusals at KL=0.0176.
5. Auto-save — best Pareto trial merged into base weights via the ablation toolkit's LoRA-adapter merge path; saved as BF16 safetensors with vision tower fully intact.

Total the ablation toolkit wall-clock: ~13 h on M4 Max 128 GB.

Step 2. LM mixed-precision quantization (OptiQ)

6. Sensitivity analysis — optiq convert --target-bpw 3.7 --candidate-bits 2,3,4,8 --sensitivity stream mmaps each tensor and computes per-tensor Frobenius distance between BF16 reference and candidate quantized versions. For a 27B model on 128 GB unified memory this is the only mode that fits.
7. Multi-tier bit allocation — OptiQ's optimizer assigns each of the 582 quantizable tensors a bit-width from {3, 4, 8} such that the average lands at the 3.7 BPW target. Result: 38% × 8-bit (high-sensitivity), 36% × 4-bit (medium), 26% × 3-bit (low-sensitivity SSM projections and certain MLP gates). No tensor required 2-bit.
8. MLX serialization — quantized LM weights written via mlx-lm quantize hooks, with the per-tensor bit-width recorded in config.json["quantization"] for each module path.

Step 3. Vision splice-back (custom)

9. Why — OptiQ defaults to stripping vision weights and rewriting config.json to a text-only Qwen3_5ForCausalLM. We wanted a multimodal release. The --keep-unused-modalities flag preserves the architecture string but not the weights, so we built a manual splice.
10. What — we copied the 333 vision tensors verbatim from our companion 8-bit MLX build (where mlx_vlm.convert leaves the ViT at BF16) into a 4th safetensors shard, restored vision_config, image_token_id, video_token_id, vision_start/end_token_id, and the nested text_config block in config.json, and re-set architectures to Qwen3_5ForConditionalGeneration.
11. Verification — post-load norm weights at expected ~0.79 mean (no double-sanitize), text generation coherent, vision pipeline correctly reads test imagery.

In-place the ablation toolkit patches used

• ABLATION_AUTO_SAVE — bypass interactive menu, auto-save Pareto-best.
• ABLATION_AUTO_CONTINUE — non-interactive checkpoint resume keeping CLI settings.
• ABLATION_ENQUEUE_PRIORS — load JSON priors before study.optimize.

---

Use it

Inference via mlx-vlm (full multimodal)

pip install mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-OptiQ-3.7bpw-mlx")
config = load_config("lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-OptiQ-3.7bpw-mlx")

messages = [{"role": "user", "content": "Explain the difference between SSM and softmax attention in 3 sentences."}]
prompt = apply_chat_template(processor, config, messages)
print(generate(model, processor, prompt, max_tokens=400, verbose=True))

With an image

out = generate(
 model, processor,
 prompt=apply_chat_template(processor, config, [{"role":"user","content":"Describe this image briefly."}], num_images=1),
 image=["./photo.jpg"],
 max_tokens=400, verbose=True,
)
print(out)

OpenAI-compatible HTTP via mlx-omni-server

pip install mlx-omni-server
mlx-omni-server # serves on http://127.0.0.1:10240

curl -s http://127.0.0.1:10240/v1/chat/completions \
 -H "Content-Type: application/json" \
 -d '{
 "model": "lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored-OptiQ-3.7bpw-mlx",
 "messages": [
 {"role": "system", "content": "Be brief and direct."},
 {"role": "user", "content": "Write a 3-line haiku about a heisenbug."}
 ],
 "max_tokens": 200
 }' | jq -r '.choices[0].message.content'

---

Quantization details

• Source weights: BF16 abliterated checkpoint (12 shards, ~50 GB) — the ablation toolkit T45 merged into Jackrong/Qwopus3.6-27B-v2.
• LM quantization tool: mlx-optiq v0.0.11 (optiq convert --target-bpw 3.7 --candidate-bits 2,3,4,8 --sensitivity stream).
• Achieved LM BPW: 3.697 (target 3.7) — see optiq_metadata.json for the full per-tensor allocation table.
• Effective LM bits/weight: 4.147 (with 16+16-bit scale+bias per group of 64 spread over the bit-width-weighted mean).
• Group size: 64 throughout (LM).
• Vision tower: spliced back at BF16 from the 8-bit MLX build (mlx_vlm.convert leaves ViTs unquantized by default; the 8-bit build's "8-bit" only refers to LM weights). 333 vision tensors, ~880 MB.
• Architecture class: Qwen3_5ForConditionalGeneration (full multimodal restored).

Per-tensor LM bit allocation (summary)

The full per-tensor allocation map lives in optiq_metadata.json. The vision tower (333 weights) does not appear in this table — it is unquantized BF16.

Architecture notes

Qwen 3.6 27B uses a hybrid attention stack — 3 linear-attention (GatedDeltaNet / SSM) layers followed by 1 full-softmax-attention layer, repeated 16× for 64 total layers; hidden 5120, vocab 248320, context 262144. The SSM kernels lack a VJP path in MLX, so backward-pass-based quant methods (DWQ, dynamic quant) cannot be applied to this architecture — OptiQ's forward-only Frobenius-stream sensitivity approach is the only calibration-aware option that works.

For inference latency at 3.7 BPW LM + BF16 ViT, expect ~12–15 tok/s on M4 Max 128 GB at batch size 1, or ~5–7 tok/s on M2/M3 16 GB at batch size 1.

---

Behavior caveats

• Uncensored. Refusal directions were surgically removed; this model will answer prompts the parent would refuse. Use responsibly and within applicable law. The release is provided for safety research, red-teaming, and creative/educational use cases.
• Multimodal preserved. Vision works — the 333 ViT weights are unmodified from the abliterated parent. The abliteration acts on the LM hidden states only.
• Identity preserved. The model still self-identifies as Qwen (developed by Alibaba's Tongyi Lab) — abliteration does not rewrite factual self-knowledge.
• Heavy chain-of-thought. Qwen 3.6 inherits Claude-Opus's verbose reasoning style. For terse answers, use a system prompt like "Be brief and direct. Skip your reasoning.".
• Quantization noise at 3-bit. ~26% of LM tensors are at 3-bit. On long-generation tasks (>1000 tokens) you may notice slightly more drift than the 6/8-bit siblings. For best fidelity at this lineage, prefer 6-bit or 8-bit.

---

Credits

Quantization & release

Lemura Labs

Claude-Opus reasoning distill

Jackrong — author of Qwen 3.6 27B-v2

Foundation model

Qwen Team @ Alibaba Tongyi Lab — Qwen3.6-27B

Abliteration toolkit

the ablation toolkit by Lemura Labs — Optuna-driven refusal-direction ablation with KL guardrails.

Community priors that seeded our TPE search

coder3101/Qwen3.5-27B-zerofuse · wangzhang/Qwen3.6-27B-abliterated

Mixed-precision quantization framework

mlx-optiq by Thin Signal — sensitivity-driven multi-tier MLX quantization · built on mlx-lm and mlx (Apple).

---

License

Apache-2.0, inherited from the foundation (Qwen3.6-27B) and the distill (Qwen 3.6 27B-v2) upstream.

---

Need a hosted endpoint, custom quant, or larger-scale inference? Lemura Labs — multi-provider LLM routing for the Indian developer ecosystem.