I think some logic from __main__.py could be moved to the SDK. I think new_configure_logging could be a decorator which is just imported from Apify SDK. I think configure_logger and logger names could be imported.

Similarly, main.py contains _get_scrapy_settings, but it could also be just imported, if turned into this:

def apply_apify_settings(settings: Settings, proxy_config: dict | None = None):
    ...
    return settings

Then it wouldn't need to call get_project_settings() and would leave space for custom modifications before or after applying Apify-specific settings.

The main benefit of doing this is that the template contains less boilerplate and it's easier to control and maintain updates. If new Scrapy version is published and the logic of the monkey patching or anything else needs to be changed, I could just upgrade Apify SDK with updates. As of now the only way is to watch updates to the template manually and update my code by copy-pasting.

Scrapy is a library and I can upgrade it carefully or pin it to a certain version, but Apify is a SaaS platform. If something changes in Apify and it won't be compatible with the old template code, it can just break my actors out of nowhere. Only then I will be prompted to go and see if the template looks differently than last time. Not ideal.

Follow up to #132, vaguely related to apify/actor-templates#264