The logging service must be explicitly started through the scrapy.log.start() function.
Scrapy provides 5 logging levels:
How to set the log level¶
You can set the log level using the –loglevel/-L command line option, or using the LOG_LEVEL setting.
How to log messages¶
Here’s a quick example of how to log a message using the WARNING level:
from scrapy import log log.msg("This is a warning", level=log.WARNING)
Logging from Spiders¶
The recommended way to log from spiders is by using the Spider log() method, which already populates the spider argument of the scrapy.log.msg() function. The other arguments are passed directly to the msg() function.
- scrapy.log.start(logfile=None, loglevel=None, logstdout=None)¶
Start the logging facility. This must be called before actually logging any messages. Otherwise, messages logged before this call will get lost.
- logfile (str) – the file path to use for logging output. If omitted, the LOG_FILE setting will be used. If both are None, the log will be sent to standard error.
- loglevel – the minimum logging level to log. Available values are: CRITICAL, ERROR, WARNING, INFO and DEBUG.
- logstdout (boolean) – if True, all standard output (and error) of your application will be logged instead. For example if you “print ‘hello’” it will appear in the Scrapy log. If omitted, the LOG_STDOUT setting will be used.
- scrapy.log.msg(message, level=INFO, spider=None)¶
Log a message
Log level for critical errors
Log level for errors
Log level for warnings
Log level for informational messages (recommended level for production deployments)
Log level for debugging messages (recommended level for development)