thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nobuaki Sukegawa <nsu...@gmail.com>
Subject Re: AW: Support for Python3
Date Wed, 04 Nov 2015 12:39:17 GMT
Hi Jeff,

Yes ! that's the way we'll handle it.
But since you linked the pull request (
https://github.com/apache/thrift/pull/213), I thought you were aware of the
existing work done by people there.
Seeing your message I've put some work too and it's under the final
verification.
It would be great if you can see if it works with python3 on your env too :
https://github.com/apache/thrift/pull/680
Also any review/suggestions are appreciated.


On Wed, Nov 4, 2015 at 9:08 PM Jeff Nelson <jtnelson1@gmail.com> wrote:

> Hi All,
>
> So I've hacked my client to work with Python3. I am NOT a python guru, so
> there are probably better things that I could have done (suggestions are
> welcome). Also, I am not yet sure that what I've done is generalizable to
> get python3 support as a whole, but I wanted to share so there is another
> data point to add to the discussion.
>
> 1. Run 2to3 tool on the generated sources
>
> 2. The biggest challenge was dealing with the difference in Py3's handling
> of bytes and string. To get around this, I had to "hack" (e.g. change the
> implementation) the classes provided by the thrift python library in a few
> places:
> A. Change '__wbuf' and '__rbuf' to be instances of BytesIO instead of
> StringIO
> B. Change the "write" method of the transport to convert the 'buf' to bytes
> whenever necessary
> C. Change the "readAll" method of the transport to use a bytearray instead
> of an implicit string
>
> You can see those changes I've made to my client at
>
> https://github.com/cinchapi/concourse/blob/develop/concourse-driver-python/concourse/concourse.py#L122
> (lines 122-169).
>
> 3. Finally, I have to go through data that is returned from the server and
> inspect it to see if there are still traces of binary that should be
> converted to a string because I've changed the readAll method to use a
> bytearray. There's probably a way to implement this so that this manual
> checking isn't necessary, but that would be a deeper change to the core
> that isn't feasible for me to hack at runtime.
>
> Again, if any people who are more familiar with Python have suggestions for
> things I should do differently, please let me know. Otherwise, hopefully
> some of what I've done is at least minimally helpful in getting true py3
> support soon.
>
> Thanks,
> Jeff
>
> On Mon, Nov 2, 2015 at 3:07 PM, Jeff Nelson <jtnelson1@gmail.com> wrote:
>
> > Happy to help test things. I'm working on some hacks to the current
> Python
> > version of thrift to get strings and binary to work properly. I'll of
> > course share anything useful that I come up with that is generalizable
> >
> > Sent from my iPhone
> >
> >
> >
> >
> > On Mon, Nov 2, 2015 at 11:40 AM -0800, "Jens Geyer" <
> jensgeyer@hotmail.com
> > > wrote:
> >
> > Maybe Jeff wants to test the stuff and/or contribute otherwise to it?
> Just an idea ...
> >> ________________________________
> >> Von: Nobuaki Sukegawa
> >> Gesendet: 02.11.2015 16:11
> >> An: user@thrift.apache.org
> >> Betreff: Re: Support for Python3
> >>
> >> Hi Jeff,
> >>
> >> Python 3 support has been almost finished for quite a long time, but not
> >> yet merged into the master.
> >> There are still a few remaining issues to handle (string and binary).
> >> I can only say it's "soon" but we're  definitely  working on it in the
> hope
> >> that we can include this in the next release.
> >>
> >> On Mon, Nov 2, 2015 at 5:02 PM Jens Geyer  wrote:
> >>
> >> > Hi Jeff,
> >> >
> >> > not sure about the state of Py3 but could you just look if there is
> >> > already a JIRA ticket for your specific problem? Otherwise please
> file one.
> >> >
> >> > Thanks!
> >> > ________________________________
> >> > Von: Jeff Nelson
> >> > Gesendet: 01.11.2015 23:24
> >> > An: user@thrift.apache.org
> >>
> >> > Betreff: Support for Python3
> >> >
> >> > Hi,
> >> >
> >> > I recently upgraded a project to use Python3 and am now getting
> errors from
> >> > Thrift. The error is TypeError: string argument expected, got 'bytes'
> and
> >> > the stack trace is
> >> >
> >> > ...
> >> >     self._oprot.writeMessageBegin('login', TMessageType.CALL,
> self._seqid)
> >> >   File
> >> >
> >> >
> "/Users/jnelson/.pyenv/versions/3.3.6/lib/python3.3/site-packages/thrift/protocol/TBinaryProtocol.py",
> >> > line 46, in writeMessageBegin
> >> >     self.writeI32(TBinaryProtocol.VERSION_1 | type)
> >> >   File
> >> >
> >> >
> "/Users/jnelson/.pyenv/versions/3.3.6/lib/python3.3/site-packages/thrift/protocol/TBinaryProtocol.py",
> >> > line 111, in writeI32
> >> >     self.trans.write(buff)
> >> >   File
> >> >
> >> >
> "/Users/jnelson/.pyenv/versions/3.3.6/lib/python3.3/site-packages/thrift/transport/TTransport.py",
> >> > line 168, in write
> >> >     raise e
> >> >   File
> >> >
> >> >
> "/Users/jnelson/.pyenv/versions/3.3.6/lib/python3.3/site-packages/thrift/
> >> transport/TTransport.py",
> >> > line 164, in write
> >> >     self.__wbuf.write(buf)
> >> >
> >> > Trying to figure out if Python3 is supported by Thrift is giving me
> >> > conflicting answers. The Jira
> >> >  for the issue is still
> >> > open, but comments on StackOverflow
> >> > <
> >> >
> http://stackoverflow.com/questions/32229690/apache-thrift-python-3-support
> >> > >
> >> > suggest
> >> > that support arrived with the 0.9.3 release (but I'm still getting the
> >> > error after upgrading the Python Thrift library to 0.9.3).
> >> >
> >> > In any case. I'm not a Python export, but it seems to me that this
> issue is
> >> > happening because __wbuf is an instance of StringIO, as opposed to
> BytesIO
> >> > <
> >> >
> http://stackoverflow.com/questions/6479100/python-stringio-replacement-that-works-with-bytes-instead-of-strings
> >> > >
> >> > .
> >> >
> >> > So, is Python3 supported at this point, and if not, what is the plan
> for
> >> > adding it?
> >> >
> >> > Thanks,
> >> > Jeff
> >> >
> >>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message