Arachnide SEO Spider
GitHub

Download and install

Three steps, no account, no installer wizard. Arachnide is a Node.js application you run from a folder; the launcher installs its dependencies and opens your browser.

Current version: 3.21.0 · MIT license · Node.js >=18.17

Download the ZIP (latest) All releases

  1. Install Node.js

    Download the LTS version from nodejs.org and install it like any other program. You need version 18.17 or newer. To check, open a terminal and type node -v. For JavaScript rendering you also need Google Chrome, Microsoft Edge, Chromium or Brave: Arachnide finds the browser automatically.

  2. Download Arachnide

    Easiest: the ZIP button above (or the green Code button on GitHub, then Download ZIP) and unzip the file. With Git: git clone https://github.com/Jacckisback/arachnide-seo-spider.git

  3. Start it

    Option A, double-click. The launcher installs or updates everything, then opens your browser. Option B, terminal: cd arachnide-seo-spider, npm install, npm start, then open http://localhost:4000. Type a website address and click Crawl.

Launcher to double-click
SystemFile to double-click
macOSstart-mac.command. The first time, right-click it and choose Open, because macOS blocks unsigned scripts.
Windowsstart-windows.bat
Linuxstart-linux.sh

Keep the terminal window open while you use the app and press Ctrl + C to stop it. If port 4000 is busy, Arachnide picks the next free port and shows it in the terminal.

Updating

Download the new ZIP (or run git pull in the folder), then run npm install again. The double-click launchers do this for you at every start.

git pull
npm install

Troubleshooting

MessageFix
cd: no such file or directoryYou are not in the right folder. Use cd followed by the path of the unzipped folder, e.g. cd ~/Downloads/arachnide-seo-spider.
npm ERR! enoent … package.jsonSame cause: run npm install inside the Arachnide folder.
--disable-warning= is not allowed in NODE_OPTIONSYour shell sets an option your Node version doesn't support. Run unset NODE_OPTIONS (macOS/Linux), or update Node.js to the latest LTS.
JavaScript rendering is unavailableInstall Google Chrome. If it is installed in an unusual location, start Arachnide with CHROME_PATH="/path/to/chrome" npm start.
Some pages show no H1 or no contentThe site probably builds its content with JavaScript. Enable JavaScript rendering in the options.

Privacy

Arachnide has no account, no telemetry and no cloud. The interface listens only on 127.0.0.1. Crawls are saved in the crawls folder of the application; API keys for the AI providers and CMS credentials are encrypted on your disk and sent only to the service concerned.

Responsible use

Only crawl websites you own or are allowed to audit. On client websites, keep 2 to 5 simultaneous requests and add a pause on shared hosting.

Next: your first crawl in two minutes