Built-in Exceptions reference

Here’s a list of all exceptions included in Scrapy and their usage.


exception scrapy.exceptions.DropItem[source]

The exception that must be raised by item pipeline stages to stop processing an Item. For more information see Item Pipeline.


exception scrapy.exceptions.CloseSpider(reason='cancelled')[source]

This exception can be raised from a spider callback to request the spider to be closed/stopped. Supported arguments:


reason (str) – the reason for closing

For example:

def parse_page(self, response):
    if 'Bandwidth exceeded' in response.body:
        raise CloseSpider('bandwidth_exceeded')


exception scrapy.exceptions.DontCloseSpider[source]

This exception can be raised in a spider_idle signal handler to prevent the spider from being closed.


exception scrapy.exceptions.IgnoreRequest[source]

This exception can be raised by the Scheduler or any downloader middleware to indicate that the request should be ignored.


exception scrapy.exceptions.NotConfigured[source]

This exception can be raised by some components to indicate that they will remain disabled. Those components include:

  • Extensions

  • Item pipelines

  • Downloader middlewares

  • Spider middlewares

The exception must be raised in the component’s __init__ method.


exception scrapy.exceptions.NotSupported[source]

This exception is raised to indicate an unsupported feature.