Search results

[Improvement] Add configuration to stop parsing PDFs after X pages

Status: Open.

#1901 In apache/stormcrawler;

· jnioche opened on May 8, 2026

[Improvement] Work towards Maven 4 Compatibility

good first issue

help wanted

Status: Open.

#1662 In apache/stormcrawler;

· rzo1 opened on Sep 19, 2025

Fetcher: optionally slow down fetching from hosts with repeated exceptions

core

enhancement

good first issue

help wanted

Status: Open.

#1106 In apache/stormcrawler;

· jnioche opened on Oct 14, 2023

Delete redirected pages

core

good first issue

Status: Open.

#994 In apache/stormcrawler;

· jnioche opened on Aug 8, 2022

Send host info on a specific stream

enhancement

good first issue

help wanted

Status: Open.

#867 In apache/stormcrawler;

· jnioche opened on Mar 26, 2021

support retry-after in FetcherBolt

good first issue

help wanted

Status: Open.

#784 In apache/stormcrawler;

· jnioche opened on Jan 10, 2020

Indexer bolts to use value from metadata as docID

enhancement

indexer

Status: Open.

#671 In apache/stormcrawler;

· jnioche opened on Dec 5, 2018

New XPATH implementation based on JSON file and organised per hostname or domain

core

enhancement

good first issue

Status: Open.

#586 In apache/stormcrawler;

· jnioche opened on Jun 13, 2018

wish: WARCHdfsBolt with CDX index

good first issue

help wanted

warc

wish

Status: Open.

#567 In apache/stormcrawler;

· dportabella opened on May 5, 2018

Capture stats about hosts on status stream post fetching

good first issue

ideas

Status: Open.

#504 In apache/stormcrawler;

· jnioche opened on Oct 6, 2017

Adaptative URL filter to normalize URLs based on canonical tag

good first issue

Status: Open.

#315 In apache/stormcrawler;

· jnioche opened on Jul 11, 2016

SubDocumentsFilterTest should not use a sitemap file

bug

good first issue

Status: Open.

#304 In apache/stormcrawler;

· jnioche opened on Jun 23, 2016