Content Discovery Scanner
SPIDER X is a comprehensive content discovery scanner.
Overview
SPIDER X is a comprehensive content discovery tool designed to uncover hidden or undocumented web resources. It employs a multi-faceted approach, including a sophisticated headless web crawler, various URL gathering techniques, and targeted bruteforcing capabilities.
By systematically exploring web applications, SPIDER X can reveal potential security vulnerabilities, sensitive information exposure, and unintended access points.
This tool is invaluable for thorough security assessments, helping to map out the complete attack surface of web applications and identify areas that may require further security hardening.
Usage Examples
You can specify a list of target URLs for SPIDER X to find hidden links, endpoints, files and application routes. Optionally, you may configure any settings you’d like. Afterward, simply click on Scan to launch your scan.
Shortly after your scan has been launched, you will be redirected to the page to view your pending scan.
You must provide a list of target domains to scan for vulnerabilities, not URLs. A few examples:
Correct:
example.com api.example.comIncorrect:
https://example.com/ https://api.example.com/Scanner settings
This scanner accepts the following optional parameters:
Scan mode
To help you quickly scan targets with a pre-set configuration, we’ve decided to introduce Scan Modes. The following 5 scan modes are available:
Default
The Default Scan Mode is the most used and most preferred. This scan mode performs every supported content discovery method, including targeted bruteforcing, except headless crawling.
Advanced
The Advanced Scan Mode employs just like the default scan mode, every supported content discovery method. This scan mode will perform headless crawling too.
Headless Crawling can significantly increase your scan time. We only recommend you enable headless crawling if your target is a Single-Page Application (SPA) or a JavaScript-heavy web application.
Crawl
The Crawl Scan Mode is configured to crawl your target but will avoid bruteforcing (forced browsing).
Headless Crawl
The Headless Crawl Scan Mode is configured to perform headless crawling on your target. It will avoid performing any bruteforcing (forced browsing).
Headless Crawling can significantly increase your scan time. We only recommend you enable headless crawling if your target is a Single-Page Application (SPA) or a JavaScript-heavy web application.
Fuzzer
The Fuzzer Scan Mode is configured to only perform targeted bruteforcing on your list of targets.
External sources
SPIDER X is capable of leveraging external sources, such as internet archives, to discover more content.
JavaScript parsing
SPIDER X is capable of enumerating and parsing JavaScript files to discover more content via hard-coded references, such as links, URLs and URIs. JavaScript files are goldmines for penetration testers like you, for this sole reason, this option is enabled by default. We recommend you keep this option enabled whenever possible to increase your probability of discovering more content on your target.
Auto-submit forms
SPIDER X can be deployed as a (headless) web crawler. This option instructs the content discovery tool to submit forms.
CAUTION! This option can result in (permanent) data modifications.
Filter targets
The URL filter can help you reduce noise and remove 404 pages.
Skip SSL/TLS certificate validation
Some targets may have incorrectly or invalid issued SSL/TLS certificates. If your target has an invalid SSL/TLS certificate and can be trusted, you can override this option to turn off SSL/TLS certificate validation.
CAUTION! This option is only recommended if the target has an invalid certificate and is trusted.
Capabilities
SPIDER X is a comprehensive content discovery scanner equipped with the following capabilities:
Limitations
There are currently no limitations reported for SPIDER X.
Best Practices
We recommend you to follow the best practices that we’ve outlined in detail.
Was this page helpful?