add support for python 3.3+

[jesg]

what needs to be done to provide support for both python 3.3+ and python 2.7?

[kovidgoyal]

Not something I am interested in personally, but contributions are welcome. The basic principle of porting to py3 is to

1. Convert received data to unicode, preferably as soon as possible, but definitely before returning it to users of the library

2. Convert data provided by users of the library such as headers, request bodies, etc into bytes before handing it off to httplib (at least on python 2, dont know if httplib handles unicode on py3 or not).

The HTML parsing, form filling etc should all mostly work with unicode already. The biggest amount of work will likelky come in migrating the test suite.

[jelly]

Would you mind adding a dependency to six instead of adding if else statements everyhwere?

[kovidgoyal]

In my opinion six is way too heavyweight for what it does. I prefer to just create a compat module in mechanize itself, that contains compatibility code just for the things mechanize actually needs, which I'm fairly sure will be ~ 100 lines of code. I like to restrict external dependencies only to things that are actually hard/complex to implement.

[davidgu]

Hello Kovid,

I have commented, as referenced in the linked launchpad [1] thread.

[1] https://bugs.launchpad.net/calibre/+bug/1756458

[kovidgoyal]

@davidgu I looked at the diff produced by 2to3 and IMO there si no need for a separate branch, the changes it makes can trivially be made to run on both 2 and 3. I'lll update this bug report when that is done. Note that by no means will that mean that mechanize will actually work on py3 since 2to3 really only catches cosmetic issues.

[kovidgoyal]

All the (actual) issues reported by 2to3 are now fixed via the polyglot.py module.

[noahbarnette]

I too was hoping for Python 3 support and since everyone else else seems to want it too, I'd be happy to work on a port if you y'all are accepting new contributors.

@kovidgoyal Who do I need to talk to? You or @jjlee ?

[kovidgoyal]

me

[baerbock]

@nikolas Are you finished with the port?

[nikolas]

No. I've converted a lot of codebases from Python 2 to code that is compatible with both Python 2 and 3, and this one is exceptionally difficult! Reminds me of the difficulty converting lettuce, which is also a testing tool.

Feel free to help out if you'd like to fork my branch.

And sorry for messing up this thread's history with all these force-pushes. GitHub needs to fix that! Seems like a bug, not a feature :P

[kovidgoyal]

I decided to expend three days of my life on this issue. All tests now pass on python3 and python2. I haven't actually used mechanize on py3, so depending on how comprehensive the test suite is, things may or may not work. YMMV.

[nikolas]

Cool, great work @kovidgoyal!!

[eli-schwartz]

Awesome!

What's the schedule for a new release?

[kovidgoyal]

I'd like some real world feedback from someone actually using it with py3 before I make a release.