tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (3980)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: Tez graduation [Was: Request for mentor assessment]
Date Mon, 23 Jun 2014 20:27:39 GMT
Thanks Hitesh, this is great data too. I also found the issues list
to be a gold source of conversation.

I'll now wait to see what Ted thinks, but as far as I'm concerned you
guys are moving in the direction I suspected (graduation) based on my
admittedly limited searching of mail archives and basic metrics done
the other day.

I'll wait on this for a few days and let the conversation develop.
This is the great thing about a DISCUSS thread - discussion happens! :)


Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA

-----Original Message-----
From: Hitesh Shah <hitesh@apache.org>
Reply-To: "general@incubator.apache.org" <general@incubator.apache.org>
Date: Monday, June 23, 2014 1:20 PM
To: "general@incubator.apache.org" <general@incubator.apache.org>
Cc: "dev@tez.incubator.apache.org" <dev@tez.incubator.apache.org>
Subject: Re: Tez graduation [Was: Request for mentor assessment]

>Hi folks, 
>To be very clear, the project's source of truth has always been its
>status file: 
>For the tez website, http://tez.incubator.apache.org/team-list.html, the
>onus was for the committers to test out their commit privileges, update
>the teamlist in the source tree and publish the updated website. I
>understand it has created a lot of confusion for folks outside of the
>project. I will go ahead and update the internal list to match the source
>of truth.
>Even though I am well aware that some of the initial committers on the
>incubator proposal have not been active ( my take on active is as simple
>as sending a single mail to any mailing list ) on the project since
>inception, my understanding had been that all committers and mentors from
>an incubator podling become PMC for the top level project when it
>graduates. Is my understanding incorrect? In any case, to directly
>address the diversity question, I believe the PMC would have members from
>companies such as Hortonworks, Yahoo, LinkedIn, Twitter to name a few.
>As for our community, we have had multiple talks and meetups both on Tez
>alone as well as ones with the folks from Hive and Pig.
>  - http://www.meetup.com/Apache-Tez-User-Group. The only meetup we have
>had till date had a detailed talk on Tez in addition to presentations
>from folks from Hive and Pig on how they were using or planning to use
>  - There was a whole evening devoted to Tez and applications on Tez as
>part of the Bay Area Hadoop User Group -
>  - Talks - there have been quite a few talks on it over the past year.
>For example, one can look at the recently completed Hadoop Summit in San
>Jose and search for Tez related talks of which only one talk was just
>about Tez and the others about applications built on top of it.
>We have been working with the Hive and Pig folks mostly via JIRA to
>address their requirements as they built out their platforms to work on
>Tez. One can look at https://issues.apache.org/jira/browse/HIVE-4660 and
>https://issues.apache.org/jira/browse/PIG-3446 to see the considerable
>amount of work both these communities have put in to get their respective
>applications to work on top of Tez. Most of the interaction has been on
>JIRA and not really on the dev mailing lists. Most design discussions
>also tend to happen there. Subscribing to issues@ might give one a better
>perspective on community interaction.
>To be frank, the Tez community for the past year has been working towards
>helping make Apache Hive and Apache Pig faster. As the platform evolves
>and grows, other applications such as Flink are considering it too.
>Hopefully, at some point down the line, we would like to work with the
>MapReduce community to have them consider using Tez.
>‹ Hitesh
>On Jun 23, 2014, at 12:50 AM, Ted Dunning <ted.dunning@gmail.com> wrote:
>> Let me repeat that my desired outcome here is Tez graduation.
>> It is just that I want to see Tez graduate as a viable project.
>> On Mon, Jun 23, 2014 at 12:43 AM, Mattmann, Chris A (3980) <
>> chris.a.mattmann@jpl.nasa.gov> wrote:
>>> Thanks Ted, these are all important points that require responses. I
>>> CC'ed the dev@tez.i.a.o
>>> list, where I hope the Tez community can respond to your points below
>>> detail.
>>> Guys, please see Ted's comments below and please try to address them in
>>> light of the
>>> recent [DISCUSS] Graduation thread I raised. It's fine if the answer to
>>> some of the
>>> points is (1) we plan on addressing them by X date, with Y action; (2)
>>> don't think
>>> this is a valid point *because* ..*explanation*; (3) you didn't
>>> this factoid
>>> _here_, etc.
>>> I'm hoping the other mentors on the project can step up and have some
>>> insight
>>> into the below. I look forward to the discussion.
>>> Cheers,
>>> Chris
>>> -----Original Message-----
>>> From: Ted Dunning <ted.dunning@gmail.com>
>>> Reply-To: "general@incubator.apache.org" <general@incubator.apache.org>
>>> Date: Monday, June 23, 2014 12:12 AM
>>> To: "general@incubator.apache.org" <general@incubator.apache.org>
>>> Subject: Re: Tez graduation [Was: Request for mentor assessment]
>>>> On Sun, Jun 22, 2014 at 9:33 AM, Mattmann, Chris A (3980) <
>>>> chris.a.mattmann@jpl.nasa.gov> wrote:
>>>>>> Regardless of whether it is an explicit requirement to have some
>>>>> specific
>>>>>> level of diversity, there is a requirement that the group
>>>>>> community building.
>>>>> I'm also sympathetic to this: you can see my thread here where
>>>>> originally
>>>>> I didn't
>>>>> think much had been going on until I explicitly checked the lists and
>>>>> went
>>>>> through
>>>>> the archives and saw ~7 IIRC PPMC members/committers added since
>>>>> entering
>>>>> [1]. That's
>>>>> community building regardless of what affiliation those new members
>>>>> from.
>>>>> The other metrics and statistics (health of the mailing lists/open
>>>>> discussions have
>>>>> been increasing; releases have been made, etc.) are all there and
>>>>> outlined
>>>>> in [1].
>>>> As I read the private list, there has been absolutely no effort at
>>>> committers until last month when the topic of graduation has come up.
>>>> There
>>>> is no substantive discussion on the list of the potential committers,
>>>> simply rather perfunctory +1 votes.
>>>> In reading the thread about whether there should be/is a diversity
>>>> requirement, I come to a bit of a different conclusion as well.  What
>>>> see
>>>> in the discussion is that the participants seem to agree that
>>>> - there should not be a simple count-based diversity measure because
>>>> would dissuade valuable commercial contributions
>>>> - the chance/likelihood of project failure due to a dominant
>>>> contributor pulling out should not be considered since project
>>>>dormancy or
>>>> retirement isn't a bad thing.  Better to try and fail than never try.
>>>> - there is clear recognition of the risk of commercial companies
>>>> a project not so much because of a desire to build community but
>>>>rather as
>>>> a marketing maneuver.
>>>> - there is no mention of the problems that have arisen in the Hadoop
>>>> groups
>>>> relative to use of groups and Apache trademarks for marketing
>>>> I don't want to impute motives to actions since I cannot know what
>>>> hold in their hearts, but I don't see evidence of efforts to really
>>>> the community.  There was one meat-space meetup a year ago.  There
>>>> been a few presentations of what Tez is.
>>>> I do see a fair number of public statements of the form "Tez is
>>>> Hortonworks
>>>> answer to Impala" or "The future of Hadoop runs on Tez", most of which
>>>> seem
>>>> to come out of Hortonworks' marketing department.
>>>> In looking at the dev list back to September of last year, I see a
>>>> number of procedural messages (votes, results, release mechanics), a
>>>> user questions (NPE in such and so, how do I install) and very few
>>>> discussions.  There was one thread last November about combiners,
>>>> in January about map parallelism, and one more in March about reducer
>>>> parallelism.  To me this looks a lot like a project where all design
>>>> discussions are occurring off-list.
>>>> What I would like to see would include
>>>> - a strong effort on the part of Tez to bring all design and
>>>> implementation
>>>> discussions to the list,
>>>> - continuous community engagement efforts such as weekly hangouts in
>>>> different timezones to help new contributors understand what is
>>>> technically in the project in an interactive setting
>>>> - lots of public talks focussed on how outsiders can contribute and
>>>> the
>>>> design works
>>>> - cross posting on related projects offering cross pollination
>>>> opportunities.  Such related groups might be Drill, Optiq, Tajo and
>>>> These postings would say things like "how could you guys help us with
>>>> Tez"
>>>> or "Tez has this and such, would that be helpful to you guys".
>>>> - inviting contributors from related projects to give technical
>>>> presentations in the context of Tez
>>>> Now, my searches have been fairly cursory and could have missed
>>>> activities, but I have a bit of a hard time believing that I have
>>>> major efforts along these lines.
>>>> Based on this evidence, it really does look to me like Tez is a purely
>>>> commercial development masquerading as an Apache community.
>>>> I agree that superficial diversity metrics are counter-productive,
>>>>but I
>>>> also assert that there isn't any obvious evidence of serious community
>>>> building here and there is significant evidence that building
>>>> isn't even the point of the project.  That *is* a substantive issue
>>>> relative to graduation.
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: general-help@incubator.apache.org
>To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>For additional commands, e-mail: general-help@incubator.apache.org

View raw message