Common Weakness Enumeration

CWE-918

Allowed

Server-Side Request Forgery (SSRF)

Abstraction: Base · Status: Incomplete

The web server receives a URL or similar request from an upstream component and retrieves the contents of this URL, but it does not sufficiently ensure that the request is being sent to the expected destination.

4719 vulnerabilities reference this CWE, most recent first.

GHSA-3Q6G-QMPX-RQW4

Vulnerability from github – Published: 2024-03-14 20:37 – Updated: 2024-03-14 20:37
VLAI
Summary
Whoogle Search Server-Side Request Forgery vulnerability
Details

Whoogle Search is a self-hosted metasearch engine. In versions 0.8.3 and prior, the window endpoint does not sanitize user-supplied input from the location variable and passes it to the send method which sends a GET request on lines 339-343 in request.py, which leads to a server-side request forgery. This issue allows for crafting GET requests to internal and external resources on behalf of the server. For example, this issue would allow for accessing resources on the internal network that the server has access to, even though these resources may not be accessible on the internet. This issue is fixed in version 0.8.4.

Show details on source website

{
  "affected": [
    {
      "package": {
        "ecosystem": "PyPI",
        "name": "whoogle-search"
      },
      "ranges": [
        {
          "events": [
            {
              "introduced": "0"
            },
            {
              "fixed": "0.8.4"
            }
          ],
          "type": "ECOSYSTEM"
        }
      ]
    }
  ],
  "aliases": [
    "CVE-2024-22205"
  ],
  "database_specific": {
    "cwe_ids": [
      "CWE-918"
    ],
    "github_reviewed": true,
    "github_reviewed_at": "2024-03-14T20:37:57Z",
    "nvd_published_at": "2024-01-23T18:15:18Z",
    "severity": "CRITICAL"
  },
  "details": "Whoogle Search is a self-hosted metasearch engine. In versions 0.8.3 and prior, the `window` endpoint does not sanitize user-supplied input from the `location` variable and passes it to the `send` method which sends a `GET` request on lines 339-343 in `request.py,` which leads to a server-side request forgery. This issue allows for crafting GET requests to internal and external resources on behalf of the server. For example, this issue would allow for accessing resources on the internal network that the server has access to, even though these resources may not be accessible on the internet. This issue is fixed in version 0.8.4.\n\n",
  "id": "GHSA-3q6g-qmpx-rqw4",
  "modified": "2024-03-14T20:37:57Z",
  "published": "2024-03-14T20:37:57Z",
  "references": [
    {
      "type": "ADVISORY",
      "url": "https://nvd.nist.gov/vuln/detail/CVE-2024-22205"
    },
    {
      "type": "WEB",
      "url": "https://github.com/benbusby/whoogle-search/commit/3a2e0b262e4a076a20416b45e6b6f23fd265aeda"
    },
    {
      "type": "PACKAGE",
      "url": "https://github.com/benbusby/whoogle-search"
    },
    {
      "type": "WEB",
      "url": "https://github.com/benbusby/whoogle-search/blob/92e8ede24e9277a5440d403f75877209f1269884/app/request.py#L339-L343"
    },
    {
      "type": "WEB",
      "url": "https://github.com/benbusby/whoogle-search/blob/92e8ede24e9277a5440d403f75877209f1269884/app/routes.py#L479"
    },
    {
      "type": "WEB",
      "url": "https://github.com/benbusby/whoogle-search/blob/92e8ede24e9277a5440d403f75877209f1269884/app/routes.py#L496-L557"
    },
    {
      "type": "WEB",
      "url": "https://github.com/benbusby/whoogle-search/blob/92e8ede24e9277a5440d403f75877209f1269884/app/routes.py#L497"
    },
    {
      "type": "WEB",
      "url": "https://github.com/pypa/advisory-database/tree/main/vulns/whoogle-search/PYSEC-2024-18.yaml"
    },
    {
      "type": "ADVISORY",
      "url": "https://securitylab.github.com/advisories/GHSL-2023-186_GHSL-2023-189_benbusby_whoogle-search"
    }
  ],
  "schema_version": "1.4.0",
  "severity": [
    {
      "score": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N",
      "type": "CVSS_V3"
    }
  ],
  "summary": "Whoogle Search Server-Side Request Forgery vulnerability"
}

GHSA-3Q8X-VX89-4P24

Vulnerability from github – Published: 2025-08-13 15:30 – Updated: 2025-08-13 21:30
VLAI
Details

Server side request forgery (SSRF) vulnerability in makeplane plane 0.23.1 via the password recovery.

Show details on source website

{
  "affected": [],
  "aliases": [
    "CVE-2025-50251"
  ],
  "database_specific": {
    "cwe_ids": [
      "CWE-918"
    ],
    "github_reviewed": false,
    "github_reviewed_at": null,
    "nvd_published_at": "2025-08-13T15:15:34Z",
    "severity": "CRITICAL"
  },
  "details": "Server side request forgery (SSRF) vulnerability in makeplane plane 0.23.1 via the password recovery.",
  "id": "GHSA-3q8x-vx89-4p24",
  "modified": "2025-08-13T21:30:27Z",
  "published": "2025-08-13T15:30:35Z",
  "references": [
    {
      "type": "ADVISORY",
      "url": "https://nvd.nist.gov/vuln/detail/CVE-2025-50251"
    },
    {
      "type": "WEB",
      "url": "https://packetstorm.news/files/id/190475"
    },
    {
      "type": "WEB",
      "url": "https://www.exploit-db.com/exploits/52211"
    }
  ],
  "schema_version": "1.4.0",
  "severity": [
    {
      "score": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N",
      "type": "CVSS_V3"
    }
  ]
}

GHSA-3QHM-QFJ3-4RRX

Vulnerability from github – Published: 2022-05-13 01:06 – Updated: 2024-02-06 18:00
VLAI
Summary
elFinder Server Side Request Forgery (SSRF)
Details

A Server Side Request Forgery (SSRF) vulnerability in elFinder before 2.1.49 could allow a malicious user to access the content of internal network resources. This occurs in get_remote_contents() in php/elFinder.class.php.

Show details on source website

{
  "affected": [
    {
      "package": {
        "ecosystem": "Packagist",
        "name": "studio-42/elfinder"
      },
      "ranges": [
        {
          "events": [
            {
              "introduced": "0"
            },
            {
              "fixed": "2.1.49"
            }
          ],
          "type": "ECOSYSTEM"
        }
      ]
    }
  ],
  "aliases": [
    "CVE-2019-6257"
  ],
  "database_specific": {
    "cwe_ids": [
      "CWE-918"
    ],
    "github_reviewed": true,
    "github_reviewed_at": "2024-01-10T19:31:04Z",
    "nvd_published_at": "2019-01-14T08:29:00Z",
    "severity": "HIGH"
  },
  "details": "A Server Side Request Forgery (SSRF) vulnerability in elFinder before 2.1.49 could allow a malicious user to access the content of internal network resources. This occurs in `get_remote_contents()` in `php/elFinder.class.php`.",
  "id": "GHSA-3qhm-qfj3-4rrx",
  "modified": "2024-02-06T18:00:03Z",
  "published": "2022-05-13T01:06:16Z",
  "references": [
    {
      "type": "ADVISORY",
      "url": "https://nvd.nist.gov/vuln/detail/CVE-2019-6257"
    },
    {
      "type": "WEB",
      "url": "https://github.com/Studio-42/elFinder/commit/2f522db8f037a66ce9040ee0b216aa4a0359286c"
    },
    {
      "type": "WEB",
      "url": "https://github.com/FriendsOfPHP/security-advisories/blob/master/studio-42/elfinder/CVE-2019-6257.yaml"
    },
    {
      "type": "PACKAGE",
      "url": "https://github.com/Studio-42/elFinder"
    },
    {
      "type": "WEB",
      "url": "https://github.com/Studio-42/elFinder/blob/2.1.49/Changelog"
    },
    {
      "type": "WEB",
      "url": "https://github.com/Studio-42/elFinder/releases/tag/2.1.49"
    }
  ],
  "schema_version": "1.4.0",
  "severity": [
    {
      "score": "CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:N/A:N",
      "type": "CVSS_V3"
    }
  ],
  "summary": "elFinder Server Side Request Forgery (SSRF)"
}

GHSA-3QQC-XVF4-H32Q

Vulnerability from github – Published: 2022-05-24 17:31 – Updated: 2022-05-24 17:31
VLAI
Details

OX App Suite through 7.10.3 allows SSRF via the the /ajax/messaging/message message API.

Show details on source website

{
  "affected": [],
  "aliases": [
    "CVE-2020-15002"
  ],
  "database_specific": {
    "cwe_ids": [
      "CWE-918"
    ],
    "github_reviewed": false,
    "github_reviewed_at": null,
    "nvd_published_at": "2020-10-23T05:15:00Z",
    "severity": "MODERATE"
  },
  "details": "OX App Suite through 7.10.3 allows SSRF via the the /ajax/messaging/message message API.",
  "id": "GHSA-3qqc-xvf4-h32q",
  "modified": "2022-05-24T17:31:56Z",
  "published": "2022-05-24T17:31:56Z",
  "references": [
    {
      "type": "ADVISORY",
      "url": "https://nvd.nist.gov/vuln/detail/CVE-2020-15002"
    },
    {
      "type": "WEB",
      "url": "https://seclists.org/fulldisclosure/2020/Oct/20"
    },
    {
      "type": "WEB",
      "url": "https://www.open-xchange.com"
    }
  ],
  "schema_version": "1.4.0",
  "severity": []
}

GHSA-3QWH-W55V-784G

Vulnerability from github – Published: 2022-05-24 17:43 – Updated: 2022-05-24 17:43
VLAI
Details

Appspace 6.2.4 allows SSRF via the api/v1/core/proxy/jsonprequest url parameter.

Show details on source website

{
  "affected": [],
  "aliases": [
    "CVE-2021-27670"
  ],
  "database_specific": {
    "cwe_ids": [
      "CWE-918"
    ],
    "github_reviewed": false,
    "github_reviewed_at": null,
    "nvd_published_at": "2021-02-25T01:15:00Z",
    "severity": "CRITICAL"
  },
  "details": "Appspace 6.2.4 allows SSRF via the api/v1/core/proxy/jsonprequest url parameter.",
  "id": "GHSA-3qwh-w55v-784g",
  "modified": "2022-05-24T17:43:03Z",
  "published": "2022-05-24T17:43:03Z",
  "references": [
    {
      "type": "ADVISORY",
      "url": "https://nvd.nist.gov/vuln/detail/CVE-2021-27670"
    },
    {
      "type": "WEB",
      "url": "https://github.com/h3110mb/PoCSSrfApp"
    }
  ],
  "schema_version": "1.4.0",
  "severity": []
}

GHSA-3QX3-XHMF-4JCC

Vulnerability from github – Published: 2023-11-29 09:30 – Updated: 2023-11-29 09:30
VLAI
Details

A server-side request forgery vulnerability in ESM prior to version 11.6.8 allows a low privileged authenticated user to upload arbitrary content, potentially altering configuration. This is possible through the certificate validation functionality where the API accepts uploaded content and doesn't parse for invalid data

Show details on source website

{
  "affected": [],
  "aliases": [
    "CVE-2023-6070"
  ],
  "database_specific": {
    "cwe_ids": [
      "CWE-918"
    ],
    "github_reviewed": false,
    "github_reviewed_at": null,
    "nvd_published_at": "2023-11-29T09:15:21Z",
    "severity": "MODERATE"
  },
  "details": "\nA server-side request forgery vulnerability in ESM prior to version 11.6.8 allows a low privileged authenticated user to upload arbitrary content, potentially altering configuration. This is possible through the certificate validation functionality where the API accepts uploaded content and doesn\u0027t parse for invalid data\n\n",
  "id": "GHSA-3qx3-xhmf-4jcc",
  "modified": "2023-11-29T09:30:27Z",
  "published": "2023-11-29T09:30:27Z",
  "references": [
    {
      "type": "ADVISORY",
      "url": "https://nvd.nist.gov/vuln/detail/CVE-2023-6070"
    },
    {
      "type": "WEB",
      "url": "https://kcm.trellix.com/corporate/index?page=content\u0026id=SB10413"
    }
  ],
  "schema_version": "1.4.0",
  "severity": [
    {
      "score": "CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:L/A:N",
      "type": "CVSS_V3"
    }
  ]
}

GHSA-3R49-GCGV-9GW5

Vulnerability from github – Published: 2023-07-21 21:30 – Updated: 2024-04-04 06:18
VLAI
Details

Server Side Request Forgery vulnerability found in Deskpro Support Desk v2021.21.6 allows attackers to execute arbitrary code via a crafted URL.

Show details on source website

{
  "affected": [],
  "aliases": [
    "CVE-2021-35391"
  ],
  "database_specific": {
    "cwe_ids": [
      "CWE-918"
    ],
    "github_reviewed": false,
    "github_reviewed_at": null,
    "nvd_published_at": "2023-07-21T20:15:11Z",
    "severity": "HIGH"
  },
  "details": "Server Side Request Forgery vulnerability found in Deskpro Support Desk v2021.21.6 allows attackers to execute arbitrary code via a crafted URL.",
  "id": "GHSA-3r49-gcgv-9gw5",
  "modified": "2024-04-04T06:18:41Z",
  "published": "2023-07-21T21:30:31Z",
  "references": [
    {
      "type": "ADVISORY",
      "url": "https://nvd.nist.gov/vuln/detail/CVE-2021-35391"
    },
    {
      "type": "WEB",
      "url": "https://sayaanalam.github.io/CVE-2021-35391.html"
    },
    {
      "type": "WEB",
      "url": "http://deskpro.com"
    }
  ],
  "schema_version": "1.4.0",
  "severity": [
    {
      "score": "CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:H/I:H/A:H",
      "type": "CVSS_V3"
    }
  ]
}

GHSA-3R74-V83P-F4F4

Vulnerability from github – Published: 2024-08-19 17:29 – Updated: 2024-08-19 17:29
VLAI
Summary
Trufflehog vulnerable to Blind SSRF in some Detectors
Details

Impact

What kind of vulnerability is it? Who is impacted?

This vulnerability allows a malicious actor to craft data in a way that, when scanned by specific detectors, could trigger the detector to make an unauthorized request to an endpoint chosen by the attacker. For an exploit to be effective, the target endpoint must be an unauthenticated GET endpoint that produces side effects. The victim must scan the maliciously crafted data and have such an endpoint targeted for the exploit to succeed.

Patches

Has the problem been patched? What versions should users upgrade to?

The vulnerability has been resolved in TruffleHog v3.81.9 and later versions. Users should upgrade to this or a more recent version to mitigate the issue.

Special thanks to Karan Bamal, Security Researcher at Sentinel One for this discovery

Show details on source website

{
  "affected": [
    {
      "package": {
        "ecosystem": "Go",
        "name": "github.com/trufflesecurity/trufflehog/v3"
      },
      "ranges": [
        {
          "events": [
            {
              "introduced": "0"
            },
            {
              "fixed": "3.81.9"
            }
          ],
          "type": "ECOSYSTEM"
        }
      ]
    }
  ],
  "aliases": [
    "CVE-2024-43379"
  ],
  "database_specific": {
    "cwe_ids": [
      "CWE-918"
    ],
    "github_reviewed": true,
    "github_reviewed_at": "2024-08-19T17:29:25Z",
    "nvd_published_at": "2024-08-19T15:15:08Z",
    "severity": "LOW"
  },
  "details": "### Impact\n_What kind of vulnerability is it? Who is impacted?_\n\nThis vulnerability allows a malicious actor to craft data in a way that, when scanned by specific detectors, could trigger the detector to make an unauthorized request to an endpoint chosen by the attacker. For an exploit to be effective, the target endpoint must be an unauthenticated GET endpoint that produces side effects. The victim must scan the maliciously crafted data and have such an endpoint targeted for the exploit to succeed.\n\n### Patches\n_Has the problem been patched? What versions should users upgrade to?_\n\nThe vulnerability has been resolved in TruffleHog v3.81.9 and later versions. Users should upgrade to this or a more recent version to mitigate the issue.\n\n_Special thanks to Karan Bamal, Security Researcher at Sentinel One for this discovery_",
  "id": "GHSA-3r74-v83p-f4f4",
  "modified": "2024-08-19T17:29:25Z",
  "published": "2024-08-19T17:29:25Z",
  "references": [
    {
      "type": "WEB",
      "url": "https://github.com/trufflesecurity/trufflehog/security/advisories/GHSA-3r74-v83p-f4f4"
    },
    {
      "type": "ADVISORY",
      "url": "https://nvd.nist.gov/vuln/detail/CVE-2024-43379"
    },
    {
      "type": "WEB",
      "url": "https://github.com/trufflesecurity/trufflehog/commit/fe5624c70923355128868cffd647b6e2cfe11443"
    },
    {
      "type": "PACKAGE",
      "url": "https://github.com/trufflesecurity/trufflehog"
    }
  ],
  "schema_version": "1.4.0",
  "severity": [
    {
      "score": "CVSS:3.1/AV:N/AC:H/PR:N/UI:R/S:C/C:N/I:L/A:N",
      "type": "CVSS_V3"
    },
    {
      "score": "CVSS:4.0/AV:N/AC:L/AT:P/PR:N/UI:A/VC:N/VI:N/VA:N/SC:N/SI:L/SA:N",
      "type": "CVSS_V4"
    }
  ],
  "summary": "Trufflehog vulnerable to Blind SSRF in some Detectors"
}

GHSA-3R75-XC34-5F44

Vulnerability from github – Published: 2026-05-21 19:28 – Updated: 2026-06-10 18:41
VLAI
Summary
Crawlee for Python: SSRF via sitemap-derived URLs
Details

Overview

  • Vulnerability type: Blind SSRF
  • Affected components: src/crawlee/_utils/sitemap.py, src/crawlee/_utils/robots.py, src/crawlee/request_loaders/_sitemap_request_loader.py, and all built-in HTTP clients.
  • Trigger: an attacker-controlled sitemap or robots.txt containing a URL that points to an internal host (layer 1) or uses a non-http scheme (layer 2).

Two-layer SSRF via sitemap-derived URLs:

1) Cross-host HTTP SSRF

Base case, affects every HTTP client.** Sitemap entries and robots.txt Sitemap: directives were accepted regardless of the host they pointed to. A sitemap on example.com could push http://internal.corp/admin into the crawler's queue, and the configured HTTP client would dispatch the request.

2) Non-HTTP scheme SSRF

Escalation, only CurlImpersonateHttpClient.** Nested-sitemap fetching dispatches the URL straight to the HTTP client, bypassing the Request construction step where Pydantic enforces http(s). Combined with the libcurl-backed CurlImpersonateHttpClient, this lets gopher://, file://, dict://, ftp://, etc., through.

Root cause

Crawlee already validates URL schemes through Pydantic's AnyHttpUrl (via validate_http_url in src/crawlee/_utils/urls.py) wherever a crawl target is materialised as a Request: the Request.url field is declared as Annotated[str, BeforeValidator(validate_http_url), Field(frozen=True)]. Anything that becomes a Request is therefore guaranteed to be http(s).

Two parts of the sitemap pipeline sidestepped this property in different ways:

1) Sitemap-derived URLs were enqueued without any host policy

SitemapRequestLoader took every <urlset><url><loc> entry, wrapped it in Request.from_url (which accepts any valid http(s) URL), and pushed the result into the request queue. RobotsTxtFile.get_sitemaps() returned every Sitemap: directive verbatim. Neither imposed any host check against the parent sitemap or robots.txt URL, so an attacker controlling that content could push internal-network HTTP URLs into the queue and have them crawled by whichever HTTP client was configured.

2) Nested sitemap fetching bypassed the Request chokepoint entirely

When _XmlSitemapParser encountered <sitemapindex><sitemap><loc>…</loc></sitemap></sitemapindex>, or when RobotsTxtFile.parse_sitemaps forwarded Sitemap: directives into the same pipeline, _fetch_and_process_sitemap dispatched the URL directly to the HTTP client:

async with http_client.stream(
    sitemap_url, 
    method='GET', 
    headers=SITEMAP_HEADERS, 
    proxy_info=proxy_info, 
    timeout=timeout,
) as response:
    ...

No Request was constructed, so the Pydantic validator never ran. Before the fix, the HTTP clients' own send_request() and stream() methods did not call validate_http_url either, so a non-http(s) scheme could pass straight through to the backend client.

The non-HTTP escalation in layer 2 is specific to CurlImpersonateHttpClient, which is backed by curl-cffi / libcurl and speaks gopher, file, dict, ftp, and other non-HTTP protocols. The other clients shipped with Crawlee (HttpxHttpClient, ImpitHttpClient, PlaywrightHttpClient) reject non-http(s) schemes at their own backend layer, regardless of what Crawlee passes in, so they were only affected by layer 1.

Vulnerable paths

Layer 1 — cross-host HTTP (all HTTP clients)

  • Source: an attacker-controlled sitemap that lists internal URLs under <urlset><url><loc> or <sitemapindex><sitemap><loc>, or an attacker-controlled robots.txt that lists internal URLs under Sitemap:.
  • Sink: the configured HTTP client issues GET requests against those URLs — either via client.request(url=request.url, …) inside crawl() for regular sitemap URLs, or via client.stream(url, …) inside the nested-sitemap fetch.

Layer 2 — non-HTTP schemes (CurlImpersonateHttpClient only)

  • Source: a nested <sitemap><loc> entry or a robots.txt Sitemap: directive pointing to a non-http(s) URL.
  • Sink: CurlImpersonateHttpClient.stream(...) hands the URL string verbatim to client.request(url=…, …), which dispatches via libcurl.

Hardening in 1.7.0 was added at both producer and consumer ends — see Remediation.

Exploitation preconditions

  1. The crawler uses sitemap loading: any of SitemapRequestLoader, Sitemap.load / parse_sitemap, discover_valid_sitemaps, or RobotsTxtFile.parse_sitemaps.
  2. The attacker controls the body of a sitemap or robots.txt that the crawler fetches — typically by being the target site, or by getting a target site to publish a malicious sitemap.
  3. The crawler's network egress can reach the attacker-chosen destination (e.g., internal services on the same network).
  4. The targeted endpoint accepts unauthenticated requests. Crawlee does not supply credentials to the forged destination, so authenticated services (IMDSv2 with token, password-protected Redis, protected admin panels) are not reachable through this path.

For layer 2 (non-HTTP), the configured HTTP client must additionally be CurlImpersonateHttpClient.

Impact

Layer 1 — cross-host HTTP (any client)

The crawler can be coerced into issuing GET requests against internal HTTP services on its own network: admin panels, unauthenticated internal APIs, cloud metadata endpoints, etc. Read-back is blind — Crawlee surfaces fetched content only through its local Dataset / KeyValueStore (push_data() etc.) and does not natively forward scraped bodies anywhere external — so direct impact is mostly existence/timing probing and occasional state changes via side-effecting GET endpoints. Read-side leakage of internal content is only exploitable end-to-end if the deployer's own application separately exposes scraped data (for example, a public summariser or aggregator built on top of Crawlee).

Layer 2 — non-HTTP escalation (only CurlImpersonateHttpClient)

Under the affected client, attackers gain the libcurl scheme set:

  • gopher:// is the canonical RESP-injection vector: pipeline FLUSHALL, CONFIG SET dir, CONFIG SET dbfilename, SAVE to an unauthenticated Redis on the crawler's network — enough to write attacker-controlled bytes to disk and, in the standard escalation, achieve remote code execution on the Redis host.
  • file:// allows the crawler to read local files (application secrets, configuration) on the crawler host.
  • dict:// and ftp:// permit fingerprinting and limited interaction with text-protocol services.

In both layers, the SSRF is blind in the default configuration. Write-side impact (gopher:// → Redis) and timing-based internal probing do not depend on read-back and remain viable regardless of whether the deployer surfaces scraped content.

Remediation

Both layers are fixed in crawlee==1.7.0. The fix is split across two PRs, applied at the two complementary boundaries of the affected pipeline:

  1. Producer-side filtering — sitemap and robots.txt loaders (PR #1864). SitemapRequestLoader and RobotsTxtFile.get_sitemaps() now run every nested-sitemap entry, every regular sitemap URL, and every Sitemap: directive through crawlee._utils.urls.filter_url. This applies to an EnqueueStrategy (default 'same-hostname') against the parent sitemap / robots.txt URL — cross-host entries are dropped — and rejects non-http(s) schemes. The strategy is stamped onto the emitted Requests, so BasicCrawler._check_url_after_redirects continues policing the policy across redirects.
  2. Consumer-side validation — HTTP-client boundary (PR #1862). validate_http_url(url) is now called at the top of send_request() and stream() in ImpitHttpClient, HttpxHttpClient, CurlImpersonateHttpClient, and PlaywrightHttpClient. Non-http(s) schemes raise pydantic.ValidationError before any backend call. crawl() was already covered, because Request.url is validated by Pydantic on construction.

After these changes, validation is enforced both where sitemap-derived HTTP requests are produced (sitemap and robots.txt loaders) and where they are consumed (HTTP clients). A regression at either layer is caught by the other.

Behaviour change for upgraders

SitemapRequestLoader and RobotsTxtFile.get_sitemaps() now default to enqueue_strategy='same-hostname'. Deployers that legitimately relied on cross-host sitemap entries (e.g., a sitemap index on sitemaps.example.com that points to content on www.example.com) must opt in explicitly with enqueue_strategy='same-domain' or enqueue_strategy='all'.

Finder credits

Show details on source website

{
  "affected": [
    {
      "package": {
        "ecosystem": "PyPI",
        "name": "crawlee"
      },
      "ranges": [
        {
          "events": [
            {
              "introduced": "1.0.0"
            },
            {
              "fixed": "1.7.0"
            }
          ],
          "type": "ECOSYSTEM"
        }
      ]
    }
  ],
  "aliases": [
    "CVE-2026-46497"
  ],
  "database_specific": {
    "cwe_ids": [
      "CWE-918"
    ],
    "github_reviewed": true,
    "github_reviewed_at": "2026-05-21T19:28:10Z",
    "nvd_published_at": "2026-06-10T16:17:08Z",
    "severity": "LOW"
  },
  "details": "## Overview\n\n- **Vulnerability type:** Blind SSRF\n- **Affected components:** `src/crawlee/_utils/sitemap.py`, `src/crawlee/_utils/robots.py`, `src/crawlee/request_loaders/_sitemap_request_loader.py`, and all built-in HTTP clients.\n- **Trigger:** an attacker-controlled sitemap or `robots.txt` containing a URL that points to an internal host (layer 1) or uses a non-http scheme (layer 2).\n\nTwo-layer SSRF via sitemap-derived URLs:\n\n### 1) Cross-host HTTP SSRF\n\nBase case, affects every HTTP client.** Sitemap entries and `robots.txt` `Sitemap:` directives were accepted regardless of the host they pointed to. A sitemap on `example.com` could push `http://internal.corp/admin` into the crawler\u0027s queue, and the configured HTTP client would dispatch the request.\n\n### 2) Non-HTTP scheme SSRF\n\nEscalation, only `CurlImpersonateHttpClient`.** Nested-sitemap fetching dispatches the URL straight to the HTTP client, bypassing the `Request` construction step where Pydantic enforces `http(s)`. Combined with the libcurl-backed `CurlImpersonateHttpClient`, this lets `gopher://`, `file://`, `dict://`, `ftp://`, etc., through.\n\n\n\n## Root cause\n\nCrawlee already validates URL schemes through Pydantic\u0027s `AnyHttpUrl` (via `validate_http_url` in `src/crawlee/_utils/urls.py`) wherever a crawl target is materialised as a `Request`: the `Request.url` field is declared as `Annotated[str, BeforeValidator(validate_http_url), Field(frozen=True)]`. Anything that becomes a `Request` is therefore guaranteed to be `http(s)`.\n\nTwo parts of the sitemap pipeline sidestepped this property in different ways:\n\n### 1) Sitemap-derived URLs were enqueued without any host policy\n\n`SitemapRequestLoader` took every `\u003curlset\u003e\u003curl\u003e\u003cloc\u003e` entry, wrapped it in `Request.from_url` (which accepts any valid `http(s)` URL), and pushed the result into the request queue. `RobotsTxtFile.get_sitemaps()` returned every `Sitemap:` directive verbatim. Neither imposed any host check against the parent sitemap or `robots.txt` URL, so an attacker controlling that content could push internal-network HTTP URLs into the queue and have them crawled by whichever HTTP client was configured.\n\n### 2) Nested sitemap fetching bypassed the `Request` chokepoint entirely\n\nWhen `_XmlSitemapParser` encountered `\u003csitemapindex\u003e\u003csitemap\u003e\u003cloc\u003e\u2026\u003c/loc\u003e\u003c/sitemap\u003e\u003c/sitemapindex\u003e`, or when `RobotsTxtFile.parse_sitemaps` forwarded `Sitemap:` directives into the same pipeline, `_fetch_and_process_sitemap` dispatched the URL directly to the HTTP client:\n\n```python\nasync with http_client.stream(\n    sitemap_url, \n    method=\u0027GET\u0027, \n    headers=SITEMAP_HEADERS, \n    proxy_info=proxy_info, \n    timeout=timeout,\n) as response:\n    ...\n```\n\nNo `Request` was constructed, so the Pydantic validator never ran. Before the fix, the HTTP clients\u0027 own `send_request()` and `stream()` methods did not call `validate_http_url` either, so a non-`http(s)` scheme could pass straight through to the backend client.\n\nThe non-HTTP escalation in layer 2 is **specific to** `CurlImpersonateHttpClient`, which is backed by `curl-cffi` / libcurl and speaks `gopher`, `file`, `dict`, `ftp`, and other non-HTTP protocols. The other clients shipped with Crawlee (`HttpxHttpClient`, `ImpitHttpClient`, `PlaywrightHttpClient`) reject non-`http(s)` schemes at their own backend layer, regardless of what Crawlee passes in, so they were only affected by layer 1.\n\n## Vulnerable paths\n\n### Layer 1 \u2014 cross-host HTTP (all HTTP clients)\n\n- *Source:* an attacker-controlled sitemap that lists internal URLs under `\u003curlset\u003e\u003curl\u003e\u003cloc\u003e` or `\u003csitemapindex\u003e\u003csitemap\u003e\u003cloc\u003e`, or an attacker-controlled `robots.txt` that lists internal URLs under `Sitemap:`.\n- *Sink:* the configured HTTP client issues `GET` requests against those URLs \u2014 either via `client.request(url=request.url, \u2026)` inside `crawl()` for regular sitemap URLs, or via `client.stream(url, \u2026)` inside the nested-sitemap fetch.\n\n### Layer 2 \u2014 non-HTTP schemes (`CurlImpersonateHttpClient` only)\n\n- *Source:* a nested `\u003csitemap\u003e\u003cloc\u003e` entry or a `robots.txt` `Sitemap:` directive pointing to a non-`http(s)` URL.\n- *Sink:* `CurlImpersonateHttpClient.stream(...)` hands the URL string verbatim to `client.request(url=\u2026, \u2026)`, which dispatches via libcurl.\n\nHardening in 1.7.0 was added at both producer and consumer ends \u2014 see *Remediation*.\n\n## Exploitation preconditions\n\n1. The crawler uses sitemap loading: any of `SitemapRequestLoader`, `Sitemap.load` / `parse_sitemap`, `discover_valid_sitemaps`, or `RobotsTxtFile.parse_sitemaps`.\n2. The attacker controls the body of a sitemap or `robots.txt` that the crawler fetches \u2014 typically by being the target site, or by getting a target site to publish a malicious sitemap.\n3. The crawler\u0027s network egress can reach the attacker-chosen destination (e.g., internal services on the same network).\n4. The targeted endpoint accepts unauthenticated requests. Crawlee does not supply credentials to the forged destination, so authenticated services (IMDSv2 with token, password-protected Redis, protected admin panels) are not reachable through this path.\n\nFor layer 2 (non-HTTP), the configured HTTP client must additionally be `CurlImpersonateHttpClient`.\n\n## Impact\n\n### Layer 1 \u2014 cross-host HTTP (any client)\n\nThe crawler can be coerced into issuing `GET` requests against internal HTTP services on its own network: admin panels, unauthenticated internal APIs, cloud metadata endpoints, etc. Read-back is blind \u2014 Crawlee surfaces fetched content only through its local `Dataset` / `KeyValueStore` (`push_data()` etc.) and does not natively forward scraped bodies anywhere external \u2014 so direct impact is mostly existence/timing probing and occasional state changes via side-effecting `GET` endpoints. Read-side leakage of internal content is only exploitable end-to-end if the deployer\u0027s own application separately exposes scraped data (for example, a public summariser or aggregator built on top of Crawlee).\n\n### Layer 2 \u2014 non-HTTP escalation (only `CurlImpersonateHttpClient`)\n\nUnder the affected client, attackers gain the libcurl scheme set:\n\n- `gopher://` is the canonical RESP-injection vector: pipeline `FLUSHALL`, `CONFIG SET dir`, `CONFIG SET dbfilename`, `SAVE` to an unauthenticated Redis on the crawler\u0027s network \u2014 enough to write attacker-controlled bytes to disk and, in the standard escalation, achieve remote code execution on the Redis host.\n- `file://` allows the crawler to read local files (application secrets, configuration) on the crawler host.\n- `dict://` and `ftp://` permit fingerprinting and limited interaction with text-protocol services.\n\nIn both layers, the SSRF is blind in the default configuration. Write-side impact (`gopher://` \u2192 Redis) and timing-based internal probing do not depend on read-back and remain viable regardless of whether the deployer surfaces scraped content.\n\n## Remediation\n\nBoth layers are fixed in `crawlee==1.7.0`. The fix is split across two PRs, applied at the two complementary boundaries of the affected pipeline:\n\n1. **Producer-side filtering \u2014 sitemap and robots.txt loaders (PR #1864).** `SitemapRequestLoader` and `RobotsTxtFile.get_sitemaps()` now run every nested-sitemap entry, every regular sitemap URL, and every `Sitemap:` directive through `crawlee._utils.urls.filter_url`. This applies to an `EnqueueStrategy` (default `\u0027same-hostname\u0027`) against the parent sitemap / `robots.txt` URL \u2014 cross-host entries are dropped \u2014 and rejects non-`http(s)` schemes. The strategy is stamped onto the emitted `Request`s, so `BasicCrawler._check_url_after_redirects` continues policing the policy across redirects.\n2. **Consumer-side validation \u2014 HTTP-client boundary (PR #1862).** `validate_http_url(url)` is now called at the top of `send_request()` and `stream()` in `ImpitHttpClient`, `HttpxHttpClient`, `CurlImpersonateHttpClient`, and `PlaywrightHttpClient`. Non-`http(s)` schemes raise `pydantic.ValidationError` before any backend call. `crawl()` was already covered, because `Request.url` is validated by Pydantic on construction.\n\nAfter these changes, validation is enforced both where sitemap-derived HTTP requests are produced (sitemap and robots.txt loaders) and where they are consumed (HTTP clients). A regression at either layer is caught by the other.\n\n### Behaviour change for upgraders\n\n`SitemapRequestLoader` and `RobotsTxtFile.get_sitemaps()` now default to `enqueue_strategy=\u0027same-hostname\u0027`. Deployers that legitimately relied on cross-host sitemap entries (e.g., a sitemap index on `sitemaps.example.com` that points to content on `www.example.com`) must opt in explicitly with `enqueue_strategy=\u0027same-domain\u0027` or `enqueue_strategy=\u0027all\u0027`.\n\n## Finder credits\n\n- [@r0otsu](https://github.com/r0otsu)\n- [@Yuremin](https://github.com/Yuremin) (Zhengmin Yu)\n- [@FORIMOC](https://github.com/FORIMOC)\n- [@invoke1442](https://github.com/invoke1442) (Ethan Carter)\n- [@Arturo0x90](https://github.com/Arturo0x90) (Arturo Melgarejo)",
  "id": "GHSA-3r75-xc34-5f44",
  "modified": "2026-06-10T18:41:20Z",
  "published": "2026-05-21T19:28:10Z",
  "references": [
    {
      "type": "WEB",
      "url": "https://github.com/apify/crawlee-python/security/advisories/GHSA-3r75-xc34-5f44"
    },
    {
      "type": "ADVISORY",
      "url": "https://nvd.nist.gov/vuln/detail/CVE-2026-46497"
    },
    {
      "type": "PACKAGE",
      "url": "https://github.com/apify/crawlee-python"
    },
    {
      "type": "WEB",
      "url": "https://github.com/apify/crawlee-python/releases/tag/v1.7.0"
    }
  ],
  "schema_version": "1.4.0",
  "severity": [
    {
      "score": "CVSS:4.0/AV:N/AC:L/AT:P/PR:N/UI:P/VC:L/VI:N/VA:N/SC:L/SI:N/SA:N",
      "type": "CVSS_V4"
    }
  ],
  "summary": "Crawlee for Python: SSRF via sitemap-derived URLs"
}

GHSA-3R7C-CF5V-4V53

Vulnerability from github – Published: 2022-05-24 16:58 – Updated: 2024-04-04 02:24
VLAI
Details

OX App Suite 7.10.1 and 7.10.2 allows SSRF.

Show details on source website

{
  "affected": [],
  "aliases": [
    "CVE-2019-14225"
  ],
  "database_specific": {
    "cwe_ids": [
      "CWE-918"
    ],
    "github_reviewed": false,
    "github_reviewed_at": null,
    "nvd_published_at": "2019-10-14T17:15:00Z",
    "severity": "MODERATE"
  },
  "details": "OX App Suite 7.10.1 and 7.10.2 allows SSRF.",
  "id": "GHSA-3r7c-cf5v-4v53",
  "modified": "2024-04-04T02:24:24Z",
  "published": "2022-05-24T16:58:42Z",
  "references": [
    {
      "type": "ADVISORY",
      "url": "https://nvd.nist.gov/vuln/detail/CVE-2019-14225"
    },
    {
      "type": "WEB",
      "url": "https://seclists.org/fulldisclosure/2019/Oct/25"
    },
    {
      "type": "WEB",
      "url": "http://packetstormsecurity.com/files/154826/Open-Xchange-OX-App-Suite-SSRF-XSS-Information-Disclosure-Access-Controls.html"
    }
  ],
  "schema_version": "1.4.0",
  "severity": [
    {
      "score": "CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:L/A:N",
      "type": "CVSS_V3"
    }
  ]
}

No mitigation information available for this CWE.

CAPEC-664: Server Side Request Forgery

An adversary exploits improper input validation by submitting maliciously crafted input to a target application running on a server, with the goal of forcing the server to make a request either to itself, to web services running in the server’s internal network, or to external third parties. If successful, the adversary’s request will be made with the server’s privilege level, bypassing its authentication controls. This ultimately allows the adversary to access sensitive data, execute commands on the server’s network, and make external requests with the stolen identity of the server. Server Side Request Forgery attacks differ from Cross Site Request Forgery attacks in that they target the server itself, whereas CSRF attacks exploit an insecure user authentication mechanism to perform unauthorized actions on the user's behalf.