/
githubmirror
/
scrapy
Обзор
Документация
Войти
/
githubmirror
/
scrapy
Код
Запросы
0
Пакеты
0
Релизы
0
Аналитика
Безопасность
master
tests/AsyncCrawlerProcess/args_settings.py
25 строк
627 B
Andrey Rakhmatullin
Add `AsyncCrawlerProcess` and `Crawler.crawl_async()` (#6817)
28 май 2025, 16:55
Не верифицирован
28 май 2025, 16:55
05b3b20
Код
Авторство
О чём код?
from typing import Any import scrapy from scrapy.crawler import AsyncCrawlerProcess, Crawler class NoRequestsSpider(scrapy.Spider): name = "no_request" @classmethod def from_crawler(cls, crawler: Crawler, *args: Any, **kwargs: Any): spider = super().from_crawler(crawler, *args, **kwargs) spider.settings.set("FOO", kwargs.get("foo")) return spider async def start(self): self.logger.info(f"The value of FOO is {self.settings.getint('FOO')}") return yield process = AsyncCrawlerProcess(settings={}) process.crawl(NoRequestsSpider, foo=42) process.start()