thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konrad Grochowski <hc...@minions.org.pl>
Subject Re: Thrift in an embedded setting
Date Tue, 12 Jan 2016 09:37:07 GMT
Hey All,

Few thoughts from previous discussion about C++ >= 11:
  - we have to keep old C++ lib (at least for a while) as C++11 lib 
would break iface compatibility
  - to avoid confusion at some point we agreed to use cpp_v2 name for 
new lib, so no user would ask "What is C++2?" (it's not perfect 
solution, still at least it's some solution). Current C++ lib should be 
renamed to cpp_v1, and "--gen cpp" should be an alias for cpp_v1, and 
later (1.0? 1.1? 2.0?) changed to cpp_v2 (keeping v1 for backward 
compatibility)

As for my branch:
  - it is probably a little bit outdated (haven't merged or rebased it 
for a while)
  - does not include CMake build files
  - Contains full copy of cpp_v1 lib, but with std::shared_ptr and 
std::thread instead of boost counterparts (boost usage is reduced 
significantly).
  - Optional fields are now using boost::optional (we need to check - 
maybe some compilers already support C++17 std::optional, so we could 
switch to that - C++17 will probably be released, before cpp_v2 become 
standard :) )
  - I've simplified ThreadManager, but it's not tested
  - while writing compiler I stopped at generating Processors and I 
think that's the last thing that is missing.

Best regards,
Konrad

W dniu 2016-01-12 o 00:12, Roger Meier pisze:
> Yes Randy, that's it!
>
> I just found another issue about std::shared_ptr
> https://issues.apache.org/jira/browse/THRIFT-2221
>
> We can easily switch to boost-less code generation in place within the
> compiler by using the new, modular and testable code from Konrad and
> fine tune the C++ library for bare metal systems via CMake.
>
> good afternoon, good evening, and good night!
> -roger
>
> PS: good morning ;-)
>
> Quoting Randy Abernethy <randy.abernethy@gmail.com>:
>
>> Most boost usage in Thrift is header based. The C++ lib "API" today uses
>> boost::shared_ptr liberally and there are lots of other small 
>> incursions.
>> While the elegance of Boost is uncontested, it is another dependency 
>> and as
>> far as the Thrift C++ lib is concerned, it is redundant in the 
>> context of
>> C++ >= 11. There are also boost thread bits which are lib based. 
>> Again you
>> can work around this stuff but it is much easier, and at no cost of
>> functionality, to just use C++11 these days.
>>
>>
>>
>> On Mon, Jan 11, 2016 at 2:27 PM, Stuart Reynolds <stu@stureynolds.com>
>> wrote:
>>
>>> Just out of interest, what's the nature of Thrift's dependency on 
>>> Boost?
>>>
>>> My recollection (also from embedded systems), is that *much* (but not
>>> all) of Boost doesn't cause any real library dependencies because so
>>> much of the Boost code is provided in full as template in header files
>>> (i.e. it does not necessarily generate linker dependencies on boost
>>> libraries).
>>>
>>> Is this the case for Thrift's use of boost? Does it require some
>>> linking against a boost library -- and could that dependency be cut?
>>>
>>> - Stu
>>>
>>>
>>>
>>> On Mon, Jan 11, 2016 at 2:06 PM, Randy Abernethy
>>> <randy.abernethy@gmail.com> wrote:
>>> > Hey Roger,
>>> >
>>> > I agree with the goal. However I think the easiest way to start is 
>>> with a
>>> > new base (--gen cpp2). In this way we can get something in place for
>>> those
>>> > who need it now and then we can excommunicate the old lib whenever 
>>> the
>>> > community is ok with it. The changes contemplated break the API, not
>>> > something we can just do to the old C++ lib without harming a lot of
>>> folks.
>>> >
>>> > -Randy
>>> >
>>> > On Mon, Jan 11, 2016 at 12:04 PM, Roger Meier 
>>> <roger@bufferoverflow.ch>
>>> > wrote:
>>> >
>>> >> Hi all
>>> >>
>>> >> I think we should rework the existing cpp towards modern C++ to 
>>> awoid a
>>> >> second lib and generator to maintain.
>>> >>
>>> >> The cpp_v2 branch from Konrad is a great place to start, the
>>> >> modularization and testability is awesome!
>>> >> As we decided to base on modern C++ such as C++11, I would like to
>>> >> replace the existing ccp lib and generator instead of adding another
>>> one.
>>> >>
>>> >> I guess this might also enable Thrift on bare metal with C++ 
>>> compilers.
>>> >>
>>> >> best!
>>> >> -roger
>>> >>
>>> >>
>>> >> Quoting Konrad Grochowski <hcorg@minions.org.pl>:
>>> >>
>>> >> Hi Randy,
>>> >>>
>>> >>> Do you think we could somehow merge our cpp2 efforts? (
>>> >>> https://github.com/hcorg/thrift/tree/cpp_v2)
>>> >>> It's my old branch, had to put it on little hiatus, yet I think

>>> some
>>> >>> parts of it could be useful.
>>> >>>
>>> >>> Best regards,
>>> >>> Konrad
>>> >>>
>>> >>> W dniu 11.01.2016 o 20:16, Randy Abernethy pisze:
>>> >>>
>>> >>>> Hi Dane and Tomas,
>>> >>>>
>>> >>>> Great to hear we have a few people willing to pull the oars.

>>> I'll get
>>> our
>>> >>>> cpp2 lib sanitized this week and post a jira/patch in a few
days.
>>> Then we
>>> >>>> can take it from there.
>>> >>>>
>>> >>>> Best,
>>> >>>> Randy
>>> >>>>
>>> >>>>
>>> >>>> On Mon, Jan 11, 2016 at 11:14 AM, Dane Mason 
>>> <danem.mason@gmail.com>
>>> >>>> wrote:
>>> >>>>
>>> >>>> Hey Randy,
>>> >>>>>
>>> >>>>> That sounds perfect for our use case. I would love to contribute.
>>> Let me
>>> >>>>> know when you have more details.
>>> >>>>>
>>> >>>>> Dane
>>> >>>>>
>>> >>>>> On Sun, Jan 10, 2016 at 9:44 AM, Randy Abernethy <
>>> >>>>> randy.abernethy@gmail.com>
>>> >>>>> wrote:
>>> >>>>>
>>> >>>>> Hey Dane,
>>> >>>>>>
>>> >>>>>> My shop has a C++11 (no boost) language implementation
for 
>>> thrift.
>>> We
>>> >>>>>>
>>> >>>>> have
>>> >>>>>
>>> >>>>>> only built out the generator (--gen cpp2) TBinaryProtocol,
>>> >>>>>> TCompactProtocol, framing and basic socket transports.
I have 
>>> tried
>>> to
>>> >>>>>>
>>> >>>>> get
>>> >>>>>
>>> >>>>>> some help cleaning this up for public consumption but
so far no
>>> takers.
>>> >>>>>> Also other quarters have threatened to post such a code
base 
>>> which
>>> has
>>> >>>>>> caused us to hold off (don't want multiple versions
of this 
>>> stuff in
>>> >>>>>> the
>>> >>>>>> trunk). If you are willing to contribute I'll try to
package 
>>> things
>>> up
>>> >>>>>>
>>> >>>>> for
>>> >>>>>
>>> >>>>>> a commit this week so that we can get the community
to pitch 
>>> in and
>>> >>>>>>
>>> >>>>> finish
>>> >>>>>
>>> >>>>>> off a basic C++11 lib. Should work great for your purposes

>>> (which is
>>> >>>>>> exactly what we use it for).
>>> >>>>>>
>>> >>>>>> Best,
>>> >>>>>> Randy
>>> >>>>>>
>>> >>>>>> On Sat, Jan 9, 2016 at 6:20 PM, Dane Mason 
>>> <danem.mason@gmail.com>
>>> >>>>>>
>>> >>>>> wrote:
>>> >>>>>
>>> >>>>>> Sorry I should have been more clear. C++ is fine, but
the boost
>>> >>>>>>>
>>> >>>>>> dependency
>>> >>>>>>
>>> >>>>>>> is problematic.
>>> >>>>>>>
>>> >>>>>>> Dane
>>> >>>>>>>
>>> >>>>>>> On Saturday, January 9, 2016, Randy Abernethy <
>>> >>>>>>>
>>> >>>>>> randy.abernethy@gmail.com
>>> >>>>>
>>> >>>>>> wrote:
>>> >>>>>>>
>>> >>>>>>> Do you need a C impel or can you use C++11 on the
embedded sys?
>>> >>>>>>>>
>>> >>>>>>>> On Saturday, January 9, 2016, Dane Mason 
>>> <danem.mason@gmail.com
>>> >>>>>>>> <javascript:;>> wrote:
>>> >>>>>>>>
>>> >>>>>>>> Hi, I'm part of a project that makes heavy use
of Thrift 
>>> and it's
>>> >>>>>>>>>
>>> >>>>>>>> RPC
>>> >>>>>
>>> >>>>>> functionality across Java, C++, and Python. This project
now 
>>> needs
>>> >>>>>>>>>
>>> >>>>>>>> to
>>> >>>>>
>>> >>>>>> incorporate an embedded component with 256kb ram running
on a
>>> >>>>>>>>>
>>> >>>>>>>> CortexM3.
>>> >>>>>>
>>> >>>>>>> After a bit of research, I've decided that there
is no straight
>>> >>>>>>>>>
>>> >>>>>>>> forward
>>> >>>>>>
>>> >>>>>>> way
>>> >>>>>>>>
>>> >>>>>>>>> of using Thrift in this context out of the
box. We are not
>>> willing
>>> >>>>>>>>>
>>> >>>>>>>> to
>>> >>>>>
>>> >>>>>> bring
>>> >>>>>>>>
>>> >>>>>>>>> GLib in as a dependency, nor are we willing
to use C++ and 
>>> Boost.
>>> >>>>>>>>>
>>> >>>>>>>> What
>>> >>>>>>
>>> >>>>>>> we
>>> >>>>>>>
>>> >>>>>>>> simply need is a way to reuse the binary protocol
and most
>>> >>>>>>>>>
>>> >>>>>>>> importantly,
>>> >>>>>>
>>> >>>>>>> the
>>> >>>>>>>>
>>> >>>>>>>>> IDL used throughout the rest of the project.
We are using 
>>> many
>>> >>>>>>>>>
>>> >>>>>>>> large
>>> >>>>>
>>> >>>>>> structs, so maintaining 4 hand written implementations
across 
>>> our 4
>>> >>>>>>>>> languages doesn't make sense.
>>> >>>>>>>>>
>>> >>>>>>>>> Some solutions I've considered:
>>> >>>>>>>>>
>>> >>>>>>>>> 1. Manually write code serialize and deserialize
the thrift
>>> binary
>>> >>>>>>>>> protocol.
>>> >>>>>>>>> - We are dealing with many large structs,
and ensuring
>>> correctness
>>> >>>>>>>>>
>>> >>>>>>>> of
>>> >>>>>
>>> >>>>>> this
>>> >>>>>>>>
>>> >>>>>>>>> handwritten code is tedious and error prone.
>>> >>>>>>>>>
>>> >>>>>>>>> 2. Use a JSON protocol and use existing
JSON parsers on the C
>>> side.
>>> >>>>>>>>> - Again error prone, may not be performant
enough for our use
>>> case.
>>> >>>>>>>>>
>>> >>>>>>>>> 3. *Possibly *create something like a 
>>> "TFlatBufferProtocol"  (or
>>> >>>>>>>>>
>>> >>>>>>>> any
>>> >>>>>
>>> >>>>>> other
>>> >>>>>>>>
>>> >>>>>>>>> Serialization format with an IDL)
>>> >>>>>>>>> - I haven't looked into this in depth, it
seems messy and 
>>> hard to
>>> >>>>>>>>>
>>> >>>>>>>> maintain.
>>> >>>>>>>>
>>> >>>>>>>>> I'd appreciate any advice you all have to
offer here.
>>> >>>>>>>>>
>>> >>>>>>>>> Thanks,
>>> >>>>>>>>>
>>> >>>>>>>>> Dane
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>
>>> >>> ---
>>> >>> Ta wiadomość została sprawdzona na obecność wirusów przez
>>> oprogramowanie
>>> >>> antywirusowe Avast.
>>> >>> https://www.avast.com/antivirus
>>> >>>
>>> >>
>>> >>
>>> >>
>>>
>
>


Mime
View raw message