Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UnicodeEncodeError: 'ascii' codec can't encode character #6

Open
kaustubhd93 opened this issue Jun 25, 2020 · 0 comments
Open

UnicodeEncodeError: 'ascii' codec can't encode character #6

kaustubhd93 opened this issue Jun 25, 2020 · 0 comments
Assignees
Labels
bug Something isn't working master_py3 Label for master_py3 branch which contains all code in Python3.5

Comments

@kaustubhd93
Copy link
Owner

Steps to reproduce this issue:

  • Run the docker image with sample4 as file for testing
2020-06-25 17:58:39 [scrapy.core.scraper] ERROR: Spider error processing <GET https://teleshow.wp.pl/ads.txt> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/app/adstxt/adstxt/spiders/adstxt_spider.py", line 28, in parse
    adstxtcrawler.get_ads_txt(domain, response.body)
  File "/app/adstxt/adstxt/spiders/adstxtparser/parsers/adstxtcrawler.py", line 56, in get_ads_txt
    hlp.write_to_csv(adstxt["adstxt"],fileName=domain,fieldNames=["partner","pubId","relation","tagId"])
  File "/app/adstxt/adstxt/spiders/adstxtparser/parsers/helper.py", line 43, in write_to_csv
    writer.writerow(info)
  File "/usr/lib/python3.5/csv.py", line 153, in writerow
    return self.writer.writerow(self._dict_to_list(rowdict))
UnicodeEncodeError: 'ascii' codec can't encode character '\ufffd' in position 40: ordinal not in range(128)
2020-06-25 17:58:39 [adstxt] ERROR: <twisted.python.failure.Failure scrapy.spidermiddlewares.httperror.HttpError: Ignoring non-200 response>
2020-06-25 17:58:40 [scrapy.core.scraper] ERROR: Spider error processing <GET https://komorkomania.pl/ads.txt> (referer: None)
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/app/adstxt/adstxt/spiders/adstxt_spider.py", line 28, in parse
    adstxtcrawler.get_ads_txt(domain, response.body)
  File "/app/adstxt/adstxt/spiders/adstxtparser/parsers/adstxtcrawler.py", line 56, in get_ads_txt
    hlp.write_to_csv(adstxt["adstxt"],fileName=domain,fieldNames=["partner","pubId","relation","tagId"])
  File "/app/adstxt/adstxt/spiders/adstxtparser/parsers/helper.py", line 43, in write_to_csv
    writer.writerow(info)
  File "/usr/lib/python3.5/csv.py", line 153, in writerow
    return self.writer.writerow(self._dict_to_list(rowdict))
UnicodeEncodeError: 'ascii' codec can't encode character '\ufffd' in position 40: ordinal not in range(128)
@kaustubhd93 kaustubhd93 added bug Something isn't working master_py3 Label for master_py3 branch which contains all code in Python3.5 labels Jun 25, 2020
@kaustubhd93 kaustubhd93 self-assigned this Jun 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working master_py3 Label for master_py3 branch which contains all code in Python3.5
Projects
None yet
Development

No branches or pull requests

1 participant