{"uuid": "8eb91682-de84-4798-8786-0aef54a38de9", "vulnerability_lookup_origin": "1a89b78e-f703-45f3-bb86-59eb712668bd", "author": "9f56dd64-161d-43a6-b9c3-555944290a09", "vulnerability": "CVE-2026-3490", "type": "seen", "source": "https://gist.github.com/rahulXs/33981c9d91701115bb44ac406396c782", "content": "# The AI Odyssey: When Your Model Is the Trojan Horse\n\n### All resources from my talk at **OOSC 4.0 | IIIT Allahabad | Aug 28\u201330, 2026**\n\n**Rahul Sharma** | Senior Software Engineer, Red Hat | [@rahulXs](https://github.com/rahulXs)\n\n---\n\n## Contents\n\n1. [Quick Start - Sign Your First Model](#-quick-start--sign-your-first-model)\n2. [Demo Code (in this gist)](#-demo-code-in-this-gist)\n3. [Safety Checklist](#-safety-checklist)\n4. [Open Source Projects (contribute!)](#-open-source-projects-contribute)\n5. [CVEs & Security Advisories](#-cves--security-advisories)\n6. [Research Papers](#-research-papers)\n7. [Articles & Deep Dives](#-articles--deep-dives)\n8. [Incident Timeline](#-incident-timeline)\n9. [About Me](#-about-me)\n\n---\n\n## Quick Start - Sign Your First Model\n\nThree commands. No key management. No GPG. No certificate rotation.\n\n```bash\npip install model-signing\n\nmodel_signing sign ./my-model/\n\nmodel_signing verify ./my-model/ \\\n  --signature model.sig \\\n  --identity \"your-email@example.com\" \\\n  --identity_provider \"https://accounts.google.com\"\n```\n\n- Signing authenticates via OIDC in your browser (Google/GitHub login).\n- The signature is a detached Sigstore bundle (`model.sig`) wrapping an in-toto attestation of SHA-256 hashes.\n- Verification works offline once you have the bundle.\n\n---\n\n## Demo Code (in this gist)\n\nThe exact files from Demo 1; everything runs on **localhost only**, nothing leaves your machine:\n\n| File | What it does |\n|---|---|\n| [`create_demo_model.py`](./create_demo_model.py) | Creates a small legitimate PyTorch model (the contrast) |\n| [`create_malicious_model.py`](./create_malicious_model.py) | Creates a `.pt` file with a reverse shell in `__reduce__` (**DEMO ONLY**, connects to `127.0.0.1:9999`) |\n| [`load_model.py`](./load_model.py) | Loads the model with `weights_only=False` - what legacy tutorials still teach |\n\nRun it yourself (in a VM/container if you want to be extra safe):\n\n```bash\nmkdir -p models && cd oosc-demo\npython create_demo_model.py\npython create_malicious_model.py\n\n# Terminal A (attacker):   nc -l 9999\n# Terminal B (victim):     python load_model.py\n```\n\n> **How it works:** `torch.save()` uses pickle internally. During `torch.load()`, pickle calls `__reduce__()` to reconstruct objects, whatever it returns gets executed. Here that's `os.system(reverse_shell)`, backgrounded so the load \"succeeds\" silently. Not a bug - documented Python behavior. Python docs literally say *\"Never unpickle data from an untrusted source.\"*\n\n---\n\n## Safety Checklist\n\n**Do:**\n\n- Use **SafeTensors** for all new models \u2014 no pickle, no code execution on load\n- Use `torch.load(..., weights_only=True)` at minimum (default since PyTorch 2.6)\n- **Sign** models before sharing (`pip install model-signing`)\n- **Verify** signatures before loading any model\n- Pin models by **commit hash**, not branch name\n\n**Don't:**\n\n- Never disable safety flags (`weights_only=False`) on untrusted files\n- Never set `trust_remote_code=True` without reading the code first\n- Never load pickle models outside a sandbox/VM if you can avoid it\n- Never assume a scanner passing = file is safe (see ShadowPickle below)\n\n---\n\n## Open Source Projects (contribute!)\n\nAll are beginner-friendly and accepting contributions:\n\n| Project | Link | Why it matters |\n|---|---|---|\n| **Model Signing (OMS)** | https://github.com/sigstore/model-transparency | Sign/verify model artifacts. Implements the OpenSSF Model Signing spec. `pip install model-signing` |\n| **OMS Specification** | https://github.com/ossf/model-signing-spec | The formal spec (June 2025), backed by Google, NVIDIA, HiddenLayer, Red Hat |\n| **SafeTensors** | https://github.com/huggingface/safetensors | Safe serialization format tensors + JSON only, zero-copy. Joined PyTorch Foundation, Mar 2026 |\n| **Fickling** | https://github.com/trailofbits/fickling | Pickle decompiler & safety analyzer by Trail of Bits. Inspect suspicious pickles without executing them |\n| **Sigstore** | https://www.sigstore.dev/ | Keyless signing infra (\"Let's Encrypt for code signing\") Fulcio CA + Rekor transparency log |\n\n---\n\n## CVEs & Security Advisories\n\n| ID | What happened | Reference |\n|---|---|---|\n| **CVE-2026-24747** | `weights_only=True` bypassed via heap corruption (type confusion in C++ restricted unpickler, `SETITEM` opcode). CVSS 8.8. Fixed in PyTorch 2.10.0 | [GHSA-63CW-57P8-FM3P](https://github.com/advisories/GHSA-63CW-57P8-FM3P) \u00b7 [RAXE-2026-019](https://raxe.ai/labs/advisories/RAXE-2026-019) |\n| **CVE-2026-3490** | PickleScan blocklist universally bypassed via `pkgutil.resolve_name(\"os:system\")` indirection; 11+ RCE chains all reported CLEAN | [GHSA-VVPJ-8CMC-GX39](https://github.com/advisories/ghsa-vvpj-8cmc-gx39) \u00b7 [RAXE-2026-015](https://raxe.ai/labs/advisories/RAXE-2026-015) |\n| **CVE-2025-1716, -1889, -1944, -1945** | Four PickleScan zero-days | See timeline below |\n\n**Key takeaway:** both the framework defense *and* the principal scanner were broken in 2026. You cannot blocklist your way out of a Turing-complete format. Defense = safer format + cryptographic verification.\n\n---\n\n## Research Papers\n\n- **ShadowPickle: Evading ML Model Scanners via Stealthy Pickle Attacks** *(arXiv, Jul 2026)* three novel attacks exploiting pickle's module import mechanism; best variant evades **63% of scanners** including Hugging Face's. Introduces PickleBench.\n  https://arxiv.org/html/2607.17503\n- **Exploiting Python pickling** David Hamann's classic walkthrough of `__reduce__` RCE (the technique behind our Demo 1).\n  https://davidhamann.de/2020/04/05/exploiting-python-pickle/\n- **DEF CON 33: Loading Models, Launching Shells** Cyrus Parzian, 2025.\n  https://media.defcon.org/DEF%20CON%2033/DEF%20CON%2033%20presentations/\n\n## Articles & Deep Dives\n\n**Model signing / Sigstore:**\n- An Introduction to the OpenSSF Model Signing (OMS) Specification - OpenSSF blog, Jun 2025\n  https://openssf.org/blog/2025/06/25/an-introduction-to-the-openssf-model-signing-oms-specification/\n- Case Study: Google Secures ML Models with Sigstore - OpenSSF, Jul 2025\n  https://openssf.org/blog/2025/07/23/case-study-google-secures-machine-learning-models-with-sigstore/\n- Model authenticity and transparency with Sigstore - Red Hat, Apr 2025\n  https://next.redhat.com/2025/04/10/model-authenticity-and-transparency-with-sigstore/\n- Taming the Wild West of ML: Practical Model Signing with Sigstore \u2014 Sigstore blog\n  https://blog.sigstore.dev/model-transparency-v1.0/\n- model-signing on PyPI\n  https://pypi.org/project/model-signing/\n\n**SafeTensors:**\n- SafeTensors joins the PyTorch Foundation - Hugging Face blog, Mar 2026\n  https://huggingface.co/blog/safetensors-joins-pytorch-foundation\n- PyTorch Foundation announcement - PR Newswire, Apr 2026\n  https://www.prnewswire.com/news-releases/pytorch-foundation-announces-safetensors-as-newest-contributed-project-to-secure-ai-model-execution-302736068.html\n- SafeTensors - official PyTorch ecosystem page\n  https://pytorch.org/projects/safetensors/\n\n**Malicious models on Hugging Face:**\n- ReversingLabs identifies malware ML model on Hugging Face (*nullifAI*) - Feb 2025\n  https://www.reversinglabs.com/blog/rl-identifies-malware-ml-model-hosted-on-hugging-face\n- The Poisoned Model Registry: Hugging Face as Malware Distribution Channel - Lyrie Research, 2026\n  https://lyrie.ai/research/research/huggingface-model-supply-chain-nullifai-pickle-rce\n- Malicious AI Model Repositories Attack Surface - Cloud Security Alliance\n  https://labs.cloudsecurityalliance.org/research/csa-research-note-malicious-ai-model-repositories-attack-sur/\n- Poisoned AI: How Hugging Face Became a Malware Distribution Platform - Hive Security, 2026\n  https://hivesecurity.gitlab.io/blog/huggingface-ai-supply-chain-attacks-2026/\n\n\n---\n\n## About Me\n\nSenior Software Engineer at Red Hat. Builds Python and Go systems for RHEL and OpenShift; works on rule-driven diagnostics engines, multi-agent workflows, and more recently harness engineering.\n\n- GitHub: https://github.com/rahulXs\n- Slides PDF: Upcoming\n\nQuestions about the talk? Leave a comment on this gist, I'll try to answer.\n", "creation_timestamp": "2026-08-24T15:42:13.420232Z"}