Models · 46Qwen3.5-122B-A10B-Abliterated-MLX-3
Qwen3.5-122B-A10B-Abliterated-MLX-3
- Downloads
- 2.4k
- This month
- 198
- Base model
- Chompa1422/Qwen3.5-122B-A10B-abliterated
- Updated
- 1mo ago
Qwen3.5-122B-A10B-Abliterated-MLX-3
Qwen3.5-122B-A10B-Abliterated-MLX-3 is an Apple Silicon-oriented MLX quantization of the abliterated Qwen3.5 122B-A10B vision-language model.
Publisher: Lemura Labs Hugging Face organization: Lemura Labs
This release is the 3-bit / group-size-32 build selected from our vision-quantized candidate run. This model quantizes both the language side and eligible multimodal/vision modules.
Source And Credits
This quant was produced from the local full-precision abliterated model derived from:
- Abliterated source: Chompa1422/Qwen3.5-122B-A10B-abliterated
- Base model: Qwen/Qwen3.5-122B-A10B
Thank you to the Qwen team for the base model, to Chompa1422 for publishing the abliterated source model that made this quantization work possible, and to Pliny the Liberator for the broader abliterated-model research culture that inspired this release.
Quantization
| Field | Value |
|---|---|
| Runtime format | MLX / MLX-VLM safetensors |
| Quantization method | MLZTQ 0.2 custom MLX-VLM affine quantization |
| Weight bits | 3 |
| Group size | 32 |
| Mode | affine |
| Quant profile | mlztq-0.2-visionq_w3-g32 |
| Quant predicate | visionq_w3 |
| Language weights | Quantized |
| Vision/multimodal weights | Quantized where MLX shape constraints allow |
| KV cache | TurboQuant runtime contract recorded; model weights are MLX affine quantized |
Tensor spot checks from the selected artifact:
| Tensor | dtype | Shape | Meaning |
|---|---|---|---|
language_model.model.layers.0.linear_attn.in_proj_qkv.weight | uint32 | (12288, 288) | Language side quantized |
language_model.model.layers.0.linear_attn.in_proj_qkv.scales | bfloat16 | (12288, 96) | Language affine scales |
vision_tower.blocks.0.attn.qkv.weight | uint32 | (3456, 108) | Vision side quantized |
vision_tower.blocks.0.attn.qkv.scales | bfloat16 | (3456, 36) | Vision affine scales |
File Details
| Item | Value |
|---|---|
| Safetensor shards | 12 |
| Indexed tensor payload | 61,488,610,272 bytes |
| Local disk footprint | 57 GiB |
| Decimal payload size | 61.49 GB |
| Model index | model.safetensors.index.json |
| Quant manifest | mlztq_manifest.json |
| Processor files | processor_config.json, preprocessor_config.json, video_preprocessor_config.json |
| Tokenizer files | tokenizer.json, tokenizer_config.json, vocab.json, chat_template.jinja |
Benchmarks
Benchmarks were run with the local MLZ deterministic VLM retention gate on 2026-05-02. Raw JSON, CSV, and Markdown benchmark artifacts are included under benchmarks/results/.
| Bucket | Correct | Total | Accuracy |
|---|---|---|---|
| Quant health | 4 | 4 | 100.0% |
| Text / reasoning | 6 | 6 | 100.0% |
| OCR / document | 6 | 6 | 100.0% |
| Vision general | 5 | 10 | 50.0% |
| Overall gate | 21 | 26 | 80.8% |
Benchmark coverage:
| Bucket | Sources |
|---|---|
| Quant health | Safetensor load, vision path availability, text canary, vision canary |
| Text / reasoning | MMLU-Pro, MMLU, GSM8K |
| OCR / document | ChartQA, DocVQA, local PDF pages with images/text |
| Vision general | RealWorldQA, AI2D, MMMU Accounting, MMMU Biology, MMMU Physics |
Candidate comparison from the same run:
| Candidate | Weight bits / group | Payload GB | Quant health | Text / reasoning | Vision general | OCR / document | Overall | Status |
|---|---|---|---|---|---|---|---|---|
visionq-w6g32 | 6 / 32 | 107.40 | 100.0% | 83.3% | 50.0% | 100.0% | 76.9% | kept locally |
visionq-w5g32 | 5 / 32 | deleted | 100.0% | 100.0% | 40.0% | 83.3% | 73.1% | pruned |
visionq-w4g32 | 4 / 32 | 76.79 | 100.0% | 83.3% | 40.0% | 100.0% | 73.1% | kept locally |
visionq-w3g32 | 3 / 32 | 61.49 | 100.0% | 100.0% | 50.0% | 100.0% | 80.8% | selected |
visionq-w2g32 | 2 / 32 | deleted | 100.0% | 16.7% | 20.0% | 50.0% | 38.5% | pruned |
Inference Engines And Apps
This is an MLX / MLX-VLM safetensors repository for Apple Silicon. It is not a GGUF, AWQ, GPTQ, EXL2, or Transformers fp16 repository.
| Engine / app | Status for this repository | Notes |
|---|---|---|
| MLX-VLM | Intended reference path | Best target for image-text inference because this artifact keeps the VLM processor files and MLX-VLM tensor layout. Requires a loader/runtime version with Qwen3.5 MoE VLM support and affine quantized multimodal weights. |
| LM Studio on Apple Silicon | Intended app target, runtime-dependent | LM Studio's unified MLX engine uses mlx-lm for text generation and mlx-vlm for vision embeddings. Use a recent LM Studio build with MLX support; macOS 14+ is required for MLX models according to LM Studio's system requirements. |
| Custom MLX Python runtimes | Supported if they implement this architecture | Works for runtimes that can read MLX safetensors, Qwen3.5 MoE configs, the chat template, and MLX affine quantized language + vision tensors. |
mlx-lm alone | Text-side only / not sufficient for full VLM use | mlx-lm is useful in the MLX ecosystem, but full image input needs the VLM path and processor stack. |
| Hugging Face Transformers / vLLM / TGI | Not directly loadable | These engines do not load this MLX quantized artifact directly. Use the original/full-precision model or produce a separate backend-specific quant. |
| Ollama / llama.cpp / KoboldCpp | Not directly loadable | These generally expect GGUF for local quantized inference. This repo is MLX safetensors, not GGUF. |
Practical expectation: use this model on high-memory Apple Silicon Macs through MLX-VLM-compatible tooling. For image input, use PNG, JPEG, WebP, and PDF/image workflows supported by the serving app or preprocessing pipeline.
Research And Safety Notice
Why is this model Abliterated?
This model is intended for research, local experimentation, red-team evaluation, and authorized security testing. It may produce content that aligned models normally refuse. Users are responsible for applying appropriate safeguards and complying with laws and platform policies.
This is an abliterated model released for research and development, model-behavior analysis, authorized security testing, and experimentation with local Apple Silicon inference. Abliterated models may respond differently from aligned instruction models. Users are responsible for complying with applicable laws, platform policies, and safety requirements. The authors and uploaders are not responsible for misuse, harm, or unlawful deployment.
Reproducibility
The included mlztq_manifest.json records the source path, quantization recipe, weight format, vision quantization policy, and runtime contract used for this artifact. The benchmark files under benchmarks/results/ record the exact gate rows used to choose this model.