AVID-2023-V008

Vulnerability from avid – Published: 2023-03-31 – Updated: 2023-03-31 ATLAS Case Study
Summary
OpenAI built GPT-2, a language model capable of generating high quality text samples. Over concerns that GPT-2 could be used for malicious purposes such as impersonating others, or generating misleading news articles, fake social media content, or spam, OpenAI adopted a tiered release schedule. They initially released a smaller, less powerful version of GPT-2 along with a technical description of the approach, but held back the full trained model. Before the full model was released by OpenAI, researchers at Brown University successfully replicated the model using information released by OpenAI and open source ML artifacts. This demonstrates that a bad actor with sufficient technical skill and compute resources could have replicated GPT-2 and used it for harmful goals before the AI Security community is prepared.
Risk domain
Security
SEP view
S0502: Model theft
Lifecycle
L04: Model Development, L06: Deployment
Organisations
OpenAI GPT-2 (deployer)
Affected artifacts
Artifact Type
OpenAI GPT-2 System
References
URL Label
https://atlas.mitre.org/studies/AML.CS0007 GPT-2 Model Replication
https://www.wired.com/story/dangerous-ai-open-source/ Wired Article, "OpenAI Said Its Code Was Risky. Two Grads Re-Created It Anyway"
https://blog.usejournal.com/opengpt-2-we-replicat… Medium BlogPost, "OpenGPT-2: We Replicated GPT-2 Because You Can Too"

{
  "affects": {
    "artifacts": [
      {
        "name": "OpenAI GPT-2",
        "type": "System"
      }
    ],
    "deployer": [
      "OpenAI GPT-2"
    ],
    "developer": []
  },
  "credit": null,
  "data_type": "AVID",
  "data_version": "0.2",
  "description": {
    "lang": "eng",
    "value": "OpenAI built GPT-2, a language model capable of generating high quality text samples. Over concerns that GPT-2 could be used for malicious purposes such as impersonating others, or generating misleading news articles, fake social media content, or spam, OpenAI adopted a tiered release schedule. They initially released a smaller, less powerful version of GPT-2 along with a technical description of the approach, but held back the full trained model.\n\nBefore the full model was released by OpenAI, researchers at Brown University successfully replicated the model using information released by OpenAI and open source ML artifacts. This demonstrates that a bad actor with sufficient technical skill and compute resources could have replicated GPT-2 and used it for harmful goals before the AI Security community is prepared.\n"
  },
  "impact": {
    "avid": {
      "lifecycle_view": [
        "L04: Model Development",
        "L06: Deployment"
      ],
      "risk_domain": [
        "Security"
      ],
      "sep_view": [
        "S0502: Model theft"
      ],
      "taxonomy_version": "0.2"
    }
  },
  "last_modified_date": "2023-03-31",
  "metadata": {
    "vuln_id": "AVID-2023-V008"
  },
  "problemtype": {
    "classof": "ATLAS Case Study",
    "description": {
      "lang": "eng",
      "value": "GPT-2 Model Replication"
    },
    "type": "Advisory"
  },
  "published_date": "2023-03-31",
  "references": [
    {
      "label": "GPT-2 Model Replication",
      "type": "source",
      "url": "https://atlas.mitre.org/studies/AML.CS0007"
    },
    {
      "label": "Wired Article, \"OpenAI Said Its Code Was Risky. Two Grads Re-Created It Anyway\"",
      "type": "source",
      "url": "https://www.wired.com/story/dangerous-ai-open-source/"
    },
    {
      "label": "Medium BlogPost, \"OpenGPT-2: We Replicated GPT-2 Because You Can Too\"",
      "type": "source",
      "url": "https://blog.usejournal.com/opengpt-2-we-replicated-gpt-2-because-you-can-too-45e34e6d36dc"
    }
  ],
  "reports": null
}


Log in or create an account to share your comment.




Tags
Taxonomy of the tags.


Loading…

Loading…

Loading…

Forecast uses a logistic model when the trend is rising, or an exponential decay model when the trend is falling. Fitted via linearized least squares.

Sightings

Author Source Type Date Other

Nomenclature

  • Seen: The vulnerability was mentioned, discussed, or observed by the user.
  • Confirmed: The vulnerability has been validated from an analyst's perspective.
  • Published Proof of Concept: A public proof of concept is available for this vulnerability.
  • Exploited: The vulnerability was observed as exploited by the user who reported the sighting.
  • Patched: The vulnerability was observed as successfully patched by the user who reported the sighting.
  • Not exploited: The vulnerability was not observed as exploited by the user who reported the sighting.
  • Not confirmed: The user expressed doubt about the validity of the vulnerability.
  • Not patched: The vulnerability was not observed as successfully patched by the user who reported the sighting.

Loading…

Detection rules are retrieved from Rulezet.

Loading…

Loading…