Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature - Add support for scrapy #92

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

luke-falvey
Copy link

Scrapy is a popular web crawling and web scraping framework implemented in python. Like Django middleware, scrapy provides extensions that allows custom functionality to be inserted into the framework. This PR implements an extension that automatically send errors to Raygun when exceptions are raised in the spider and item pipelines.

TODO:

  • Add scrapy dependencies to setup.py
  • Write tests for extension
  • Implement extension
  • Update README

@coveralls
Copy link

Coverage Status

Coverage increased (+0.3%) to 88.71% when pulling 60728e4 on luke-falvey:development into f65727a on MindscapeHQ:master.

Copy link
Contributor

@samuel-holt samuel-holt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good, I just noticed one thing that was a bit off... It would be good to get these tests passing (or ignore the scrapy tests for unsupported Python versions)

f = failure.Failure()
return f

def test__handler_exception_called(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the double underscore here deliberate? Expected test_handler_exception_called

@BenjaminHarding BenjaminHarding removed their request for review July 15, 2020 23:05
@BenjaminHarding
Copy link

@luke-falvey I don't know enough about this provider (or language) to give a good review at this point unfortunately.

Also is this PR still applicable?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants