thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Nelson <jtnels...@gmail.com>
Subject Re: AW: Support for Python3
Date Wed, 04 Nov 2015 12:08:19 GMT
Hi All,

So I've hacked my client to work with Python3. I am NOT a python guru, so
there are probably better things that I could have done (suggestions are
welcome). Also, I am not yet sure that what I've done is generalizable to
get python3 support as a whole, but I wanted to share so there is another
data point to add to the discussion.

1. Run 2to3 tool on the generated sources

2. The biggest challenge was dealing with the difference in Py3's handling
of bytes and string. To get around this, I had to "hack" (e.g. change the
implementation) the classes provided by the thrift python library in a few
places:
A. Change '__wbuf' and '__rbuf' to be instances of BytesIO instead of
StringIO
B. Change the "write" method of the transport to convert the 'buf' to bytes
whenever necessary
C. Change the "readAll" method of the transport to use a bytearray instead
of an implicit string

You can see those changes I've made to my client at
https://github.com/cinchapi/concourse/blob/develop/concourse-driver-python/concourse/concourse.py#L122
(lines 122-169).

3. Finally, I have to go through data that is returned from the server and
inspect it to see if there are still traces of binary that should be
converted to a string because I've changed the readAll method to use a
bytearray. There's probably a way to implement this so that this manual
checking isn't necessary, but that would be a deeper change to the core
that isn't feasible for me to hack at runtime.

Again, if any people who are more familiar with Python have suggestions for
things I should do differently, please let me know. Otherwise, hopefully
some of what I've done is at least minimally helpful in getting true py3
support soon.

Thanks,
Jeff

On Mon, Nov 2, 2015 at 3:07 PM, Jeff Nelson <jtnelson1@gmail.com> wrote:

> Happy to help test things. I'm working on some hacks to the current Python
> version of thrift to get strings and binary to work properly. I'll of
> course share anything useful that I come up with that is generalizable
>
> Sent from my iPhone
>
>
>
>
> On Mon, Nov 2, 2015 at 11:40 AM -0800, "Jens Geyer" <jensgeyer@hotmail.com
> > wrote:
>
> Maybe Jeff wants to test the stuff and/or contribute otherwise to it? Just an idea ...
>> ________________________________
>> Von: Nobuaki Sukegawa
>> Gesendet: 02.11.2015 16:11
>> An: user@thrift.apache.org
>> Betreff: Re: Support for Python3
>>
>> Hi Jeff,
>>
>> Python 3 support has been almost finished for quite a long time, but not
>> yet merged into the master.
>> There are still a few remaining issues to handle (string and binary).
>> I can only say it's "soon" but we're  definitely  working on it in the hope
>> that we can include this in the next release.
>>
>> On Mon, Nov 2, 2015 at 5:02 PM Jens Geyer  wrote:
>>
>> > Hi Jeff,
>> >
>> > not sure about the state of Py3 but could you just look if there is
>> > already a JIRA ticket for your specific problem? Otherwise please file one.
>> >
>> > Thanks!
>> > ________________________________
>> > Von: Jeff Nelson
>> > Gesendet: 01.11.2015 23:24
>> > An: user@thrift.apache.org
>>
>> > Betreff: Support for Python3
>> >
>> > Hi,
>> >
>> > I recently upgraded a project to use Python3 and am now getting errors from
>> > Thrift. The error is TypeError: string argument expected, got 'bytes' and
>> > the stack trace is
>> >
>> > ...
>> >     self._oprot.writeMessageBegin('login', TMessageType.CALL, self._seqid)
>> >   File
>> >
>> > "/Users/jnelson/.pyenv/versions/3.3.6/lib/python3.3/site-packages/thrift/protocol/TBinaryProtocol.py",
>> > line 46, in writeMessageBegin
>> >     self.writeI32(TBinaryProtocol.VERSION_1 | type)
>> >   File
>> >
>> > "/Users/jnelson/.pyenv/versions/3.3.6/lib/python3.3/site-packages/thrift/protocol/TBinaryProtocol.py",
>> > line 111, in writeI32
>> >     self.trans.write(buff)
>> >   File
>> >
>> > "/Users/jnelson/.pyenv/versions/3.3.6/lib/python3.3/site-packages/thrift/transport/TTransport.py",
>> > line 168, in write
>> >     raise e
>> >   File
>> >
>> > "/Users/jnelson/.pyenv/versions/3.3.6/lib/python3.3/site-packages/thrift/
>> transport/TTransport.py",
>> > line 164, in write
>> >     self.__wbuf.write(buf)
>> >
>> > Trying to figure out if Python3 is supported by Thrift is giving me
>> > conflicting answers. The Jira
>> >  for the issue is still
>> > open, but comments on StackOverflow
>> > <
>> > http://stackoverflow.com/questions/32229690/apache-thrift-python-3-support
>> > >
>> > suggest
>> > that support arrived with the 0.9.3 release (but I'm still getting the
>> > error after upgrading the Python Thrift library to 0.9.3).
>> >
>> > In any case. I'm not a Python export, but it seems to me that this issue is
>> > happening because __wbuf is an instance of StringIO, as opposed to BytesIO
>> > <
>> > http://stackoverflow.com/questions/6479100/python-stringio-replacement-that-works-with-bytes-instead-of-strings
>> > >
>> > .
>> >
>> > So, is Python3 supported at this point, and if not, what is the plan for
>> > adding it?
>> >
>> > Thanks,
>> > Jeff
>> >
>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message