phoenix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Heather <james.heat...@mendeley.com>
Subject Re: setting up community repo of Phoenix for CDH5?
Date Wed, 30 Sep 2015 12:22:12 GMT
Hi JM,

Is there a way of getting hold of the parcel you built? I'm keen to try it!

Best wishes,

James

On 21/09/15 14:00, Jean-Marc Spaggiari wrote:
> Hi James,
>
> I have been able to build a parcel and installed it on my own cluster. 
> Build it with 4.5.2 that Andrew modified. I'm in touch with Cloudera 
> to see what's the next steps. I will let you know very shortly...
>
> JM
>
> 2015-09-21 6:25 GMT-04:00 James Heather <james.heather@mendeley.com 
> <mailto:james.heather@mendeley.com>>:
>
>     @JM how did you get on with the parcel building?
>
>     Has anyone managed to get 4.5 working on CDH5 now? I was going to
>     stick with 4.3 on our cluster until we had a parcel, but I'm now
>     needing to use pherf, and that doesn't seem to exist in 4.3.
>
>     James
>
>
>     On 16/09/15 12:46, Jean-Marc Spaggiari wrote:
>>     @James: I'm working on the parcel building ;) If not me I will
>>     try to find someone to do it. Stay tuned.
>>     @Andrewy: It works for me that way, cool! I just have a signature
>>     issue where it says I have no signature. Will that be an issue?
>>
>>     Thanks all,
>>
>>     JM
>>
>>     2015-09-16 3:24 GMT-04:00 James Heather
>>     <james.heather@mendeley.com <mailto:james.heather@mendeley.com>>:
>>
>>         Great! Thank you!
>>
>>         I'd wondered about parcel building. It did look as though a
>>         parcel is just a .tgz, containing the classes and a few bits
>>         of meta, so hopefully it's doable. It would be really nice if
>>         we could provide a working 4.5 parcel.
>>
>>         James
>>
>>         On 16 Sep 2015 01:02, "Andrew Purtell" <apurtell@apache.org
>>         <mailto:apurtell@apache.org>> wrote:
>>
>>             I used dev/make_rc.sh, built with Maven 3.2.2, Java 7u79.
>>             Ubuntu build host.
>>
>>
>>             On Tue, Sep 15, 2015 at 4:58 PM, Jean-Marc Spaggiari
>>             <jean-marc@spaggiari.org
>>             <mailto:jean-marc@spaggiari.org>> wrote:
>>
>>                 No, I don't know why. I will ask and see if I can get
>>                 a respons on that. I have also started the thread for
>>                 the Parcel. I will see if I find enough help to work
>>                 on that.
>>
>>                 Regarding the branch you made, I tried to build it
>>                 but got the error below. what's the command to build it?
>>
>>                 Thanks,
>>
>>                 JM
>>
>>                 [INFO]
>>                 ------------------------------------------------------------------------
>>                 [ERROR] FATAL ERROR
>>                 [INFO]
>>                 ------------------------------------------------------------------------
>>                 [INFO] null
>>                 [INFO]
>>                 ------------------------------------------------------------------------
>>                 [INFO] Trace
>>                 java.lang.NullPointerException
>>                     at
>>                 org.apache.maven.plugin.surefire.report.DefaultReporterFactory.mergeFromOtherFactories(DefaultReporterFactory.java:82)
>>                     at
>>                 org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:182)
>>                     at
>>                 org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1019)
>>                     at
>>                 org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:853)
>>                     at
>>                 org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:751)
>>                     at
>>                 org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
>>                     at
>>                 org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
>>                     at
>>                 org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
>>                     at
>>                 org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
>>                     at
>>                 org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
>>                     at
>>                 org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
>>                     at
>>                 org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
>>                     at
>>                 org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
>>                     at
>>                 org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
>>                     at
>>                 org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
>>                     at
>>                 org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
>>                     at
>>                 sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>                 Method)
>>                     at
>>                 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>                     at
>>                 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>                     at java.lang.reflect.Method.invoke(Method.java:606)
>>                     at
>>                 org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
>>                     at
>>                 org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
>>                     at
>>                 org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
>>                     at
>>                 org.codehaus.classworlds.Launcher.main(Launcher.java:375)
>>                 [INFO]
>>                 ------------------------------------------------------------------------
>>                 [INFO] Total time: 2 minutes 17 seconds
>>                 [INFO] Finished at: Tue Sep 15 19:55:03 EDT 2015
>>                 [INFO] Final Memory: 134M/1648M
>>                 [INFO]
>>                 ------------------------------------------------------------------------
>>
>>
>>                 2015-09-15 19:55 GMT-04:00 Andrew Purtell
>>                 <apurtell@apache.org <mailto:apurtell@apache.org>>:
>>
>>                     Cool, thanks J-M.
>>
>>                     Do you know why support for query tracing was
>>                     removed? If it's just a matter of porting it to
>>                     the HTrace that ships with CDH, I can look at that.
>>
>>
>>                     On Tue, Sep 15, 2015 at 4:49 PM, Jean-Marc
>>                     Spaggiari <jean-marc@spaggiari.org
>>                     <mailto:jean-marc@spaggiari.org>> wrote:
>>
>>                         Nice! I will see if there is a way to build a
>>                         parcel from that the same way there is a
>>                         parcel for Apache Phoenix 4.3 in Cloudera
>>                         Labs... Will clone what you did and try to
>>                         build it locally...
>>
>>                         2015-09-15 19:45 GMT-04:00 Andrew Purtell
>>                         <andrew.purtell@gmail.com
>>                         <mailto:andrew.purtell@gmail.com>>:
>>
>>                             I pushed updates to branch
>>                             4.5-HBase-1.0-cdh5 and the tag
>>                             v4.5.2-cdh5.4.5 (1fcb5cf). This is the
>>                             pending Phoenix 4.5.2 release, currently
>>                             at RC1, likely to pass, that will build
>>                             against CDH 5.4.5. If you want release
>>                             tarballs I built from this, get them here:
>>                                 Binary:
>>                             http://apurtell.s3.amazonaws.com/phoenix/phoenix-4.5.2-cdh5.4.5-bin.tar.gz
>>                                 Source:
>>                             http://apurtell.s3.amazonaws.com/phoenix/phoenix-4.5.2-cdh5.4.5-src.tar.gz
>>
>>
>>                             ​The source and these binaries
>>                             incorporate changes from the Cloudera
>>                             Labs fork of Phoenix
>>                             (https://github.com/cloudera-labs/phoenix),
>>                             licensed under the ASL v2, Neither the
>>                             source or binary artifacts are in any way
>>                             "official" or supported by the Apache
>>                             Phoenix project. The source and artifacts
>>                             are provided by me in a personal capacity
>>                             for the convenience of would-be Phoenix
>>                             users that also use CDH 5.4(.5). Please
>>                             don't contact the Apache Phoenix project
>>                             for any issues regarding this source and
>>                             these binaries.
>>                             ​
>>
>>                             On Mon, Sep 14, 2015 at 10:52 AM, James
>>                             Heather <james.heather@mendeley.com
>>                             <mailto:james.heather@mendeley.com>> wrote:
>>
>>                                 Done! Thanks for helping!
>>
>>                                 The branches in the repo mirror those
>>                                 in vanilla Phoenix. We shouldn't push
>>                                 any changes to the vanilla branches,
>>                                 but only to "*-cdh5" branches (or any
>>                                 temporary side branches we need to
>>                                 create).
>>
>>                                 The issue tracker will be very
>>                                 useful, yes.
>>
>>                                 James
>>
>>
>>                                 On 14/09/15 17:22, Andrew Purtell wrote:
>>>                                 This is great James.
>>>
>>>                                 Since this is conveniently on
>>>                                 Github, maybe we use the issue
>>>                                 tracker there? Interested parties
>>>                                 can set a watch. Would you be
>>>                                 willing to add 'apurtell' as a
>>>                                 collaborator on the repo? I will
>>>                                 fork and send over PRs of course,
>>>                                 but you might want help?
>>>
>>>
>>>                                 On Sep 14, 2015, at 6:21 AM, James
>>>                                 Heather <james.heather@mendeley.com
>>>                                 <mailto:james.heather@mendeley.com>>
>>>                                 wrote:
>>>
>>>>                                 I've set up a repo at
>>>>
>>>>                                 https://github.com/chiastic-security/phoenix-for-cloudera
>>>>
>>>>                                 It is a fork of the vanilla Phoenix
>>>>                                 github mirror. I've created a
>>>>                                 branch called "4.5-HBase-1.0-cdh5",
>>>>                                 which we can use for making a
>>>>                                 CDH5-compatible version. I've not
>>>>                                 made any of the necessary changes
>>>>                                 so far.
>>>>
>>>>                                 I chose that branch, by the way,
>>>>                                 because it's the latest release,
>>>>                                 and is using the same version of
>>>>                                 HBase as CDH5.4. The master branch
>>>>                                 of the Phoenix repo is building a
>>>>                                 snapshot of (the forthcoming)
>>>>                                 Phoenix 4.6, against HBase 1.1...
>>>>                                 presumably there will also be a
>>>>                                 Phoenix 4.6 for HBase 1.0?
>>>>
>>>>                                 I'm not certain of the best way to
>>>>                                 manage this. Perhaps we need a new
>>>>                                 mailing list for those who want to
>>>>                                 help, to avoid cluttering this list up.
>>>>
>>>>                                 James
>>>>
>>>>                                 On 13/09/15 02:54, Jean-Marc
>>>>                                 Spaggiari wrote:
>>>>>                                 Exact. There is some some code
>>>>>                                 change because of what has been
>>>>>                                 back ported into CDH and what has
>>>>>                                 not been. But overall, it should
>>>>>                                 not be rocket science. Mostly
>>>>>                                 method signatures...
>>>>>
>>>>>                                 Let us know when the repo is
>>>>>                                 available so we can help...
>>>>>
>>>>>                                 Thanks,
>>>>>
>>>>>                                 JM
>>>>>
>>>>>                                 2015-09-12 18:38 GMT-04:00 Krishna
>>>>>                                 <research800@gmail.com
>>>>>                                 <mailto:research800@gmail.com>>:
>>>>>
>>>>>                                     As explained here, there are
>>>>>                                     some code changes too in
>>>>>                                     addition to pom related changes.
>>>>>
>>>>>                                     http://stackoverflow.com/a/31934434/165130
>>>>>
>>>>>
>>>>>
>>>>>                                     On Friday, September 11, 2015,
>>>>>                                     Andrew Purtell
>>>>>                                     <andrew.purtell@gmail.com
>>>>>                                     <mailto:andrew.purtell@gmail.com>>
>>>>>                                     wrote:
>>>>>
>>>>>                                         Or once parameterized, add
>>>>>                                         a default off profile that
>>>>>                                         redefines them all in one
>>>>>                                         shot after the builder
>>>>>                                         activates the profile on
>>>>>                                         the maven command line
>>>>>                                         with -P ...
>>>>>
>>>>>
>>>>>
>>>>>                                         On Sep 11, 2015, at 7:05
>>>>>                                         AM, Andrew Purtell
>>>>>                                         <andrew.purtell@gmail.com
>>>>>                                         <mailto:andrew.purtell@gmail.com>>
>>>>>                                         wrote:
>>>>>
>>>>>>                                         The group IDs and
>>>>>>                                         versions can be
>>>>>>                                         parameterized in the POM
>>>>>>                                         so they can be overridden
>>>>>>                                         on the maven command line
>>>>>>                                         with -D. That would be
>>>>>>                                         easy and something I
>>>>>>                                         think we could get
>>>>>>                                         committed without any
>>>>>>                                         controversy.
>>>>>>
>>>>>>
>>>>>>                                         On Sep 11, 2015, at 6:53
>>>>>>                                         AM, James Heather
>>>>>>                                         <james.heather@mendeley.com
>>>>>>                                         <mailto:james.heather@mendeley.com>>
>>>>>>                                         wrote:
>>>>>>
>>>>>>>                                         Yes, my plan is to
>>>>>>>                                         create a fork of the
>>>>>>>                                         main repo, so that we
>>>>>>>                                         can still merge new
>>>>>>>                                         Phoenix code into the
>>>>>>>                                         CDH-compatible version.
>>>>>>>
>>>>>>>                                         Before that, I do wonder
>>>>>>>                                         whether it's possible
to
>>>>>>>                                         suggest a few changes
to
>>>>>>>                                         the main repo that would
>>>>>>>                                         allow for compiling a
>>>>>>>                                         CDH-compatible version,
>>>>>>>                                         without needing to
>>>>>>>                                         maintain a separate
>>>>>>>                                         repo. The bulk of the
>>>>>>>                                         changes are to
>>>>>>>                                         dependencies in the pom,
>>>>>>>                                         which suggests that it
>>>>>>>                                         could be done to accept
>>>>>>>                                         a switch to mvn build.
>>>>>>>
>>>>>>>                                         James
>>>>>>>
>>>>>>>                                         On 11/09/15 14:50,
>>>>>>>                                         Andrew Purtell wrote:
>>>>>>>>                                         The first step I
think
>>>>>>>>                                         is a repo with code
>>>>>>>>                                         that compiles. Please
>>>>>>>>                                         initialize it by
>>>>>>>>                                         forking
>>>>>>>>                                         github.com/apache/phoenix
>>>>>>>>                                         <http://github.com/apache/phoenix>
>>>>>>>>                                         so we have common
>>>>>>>>                                         ancestors. Once we
have
>>>>>>>>                                         a clear idea (by
diff)
>>>>>>>>                                         what is required
we can
>>>>>>>>                                         figure out if we
can
>>>>>>>>                                         support compatibility
>>>>>>>>                                         in some way.
>>>>>>>>
>>>>>>>>
>>>>>>>>                                         On Sep 9, 2015, at
>>>>>>>>                                         11:00 PM, Krishna
>>>>>>>>                                         <research800@gmail.com
>>>>>>>>                                         <mailto:research800@gmail.com>>
>>>>>>>>                                         wrote:
>>>>>>>>
>>>>>>>>>                                         I can volunteer
to
>>>>>>>>>                                         spend some time
on this.
>>>>>>>>>
>>>>>>>>>                                         CDH artifacts
are
>>>>>>>>>                                         available in
Maven
>>>>>>>>>                                         repo but from
reading
>>>>>>>>>                                         other threads
on
>>>>>>>>>                                         CDH-Phoenix
>>>>>>>>>                                         compatibilty,
it looks
>>>>>>>>>                                         like there are
some
>>>>>>>>>                                         code changes
to be
>>>>>>>>>                                         made in Phoenix
to
>>>>>>>>>                                         successfully
compile
>>>>>>>>>                                         against CDH.
>>>>>>>>>
>>>>>>>>>                                         Here are questions
to
>>>>>>>>>                                         address:
>>>>>>>>>                                         1) How to maintain
CDH
>>>>>>>>>                                         compatible Phoenix
>>>>>>>>>                                         code base?
>>>>>>>>>                                         2) Is having
a CDH
>>>>>>>>>                                         compatible branch
even
>>>>>>>>>                                         an option?
>>>>>>>>>
>>>>>>>>>                                         Krishna
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                                         On Friday, August
28,
>>>>>>>>>                                         2015, Andrew
Purtell
>>>>>>>>>                                         <andrew.purtell@gmail.com
>>>>>>>>>                                         <mailto:andrew.purtell@gmail.com>>
>>>>>>>>>                                         wrote:
>>>>>>>>>
>>>>>>>>>                                             Yes I am
>>>>>>>>>                                             interested.
>>>>>>>>>                                             Assuming
CDH
>>>>>>>>>                                             artifacts
are
>>>>>>>>>                                             publicly
available
>>>>>>>>>                                             in a Maven
repo
>>>>>>>>>                                             somewhere,
which I
>>>>>>>>>                                             believe is
the
>>>>>>>>>                                             case, perhaps
we
>>>>>>>>>                                             (the Phoenix
>>>>>>>>>                                             project/community)
>>>>>>>>>                                             could set
up a
>>>>>>>>>                                             Jenkins job
that
>>>>>>>>>                                             builds against
>>>>>>>>>                                             them and
makes the
>>>>>>>>>                                             resulting
build
>>>>>>>>>                                             artifacts
>>>>>>>>>                                             available.
They
>>>>>>>>>                                             would never
be an
>>>>>>>>>                                             official
release,
>>>>>>>>>                                             just a best
effort
>>>>>>>>>                                             convenience.
Would
>>>>>>>>>                                             that work?
I think
>>>>>>>>>                                             little must
be
>>>>>>>>>                                             done besides
>>>>>>>>>                                             compile against
>>>>>>>>>                                             the CDH artifacts
>>>>>>>>>                                             for binary
>>>>>>>>>                                             compatibility.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                                             > On Aug
28, 2015,
>>>>>>>>>                                             at 11:19
AM, James
>>>>>>>>>                                             Heather
>>>>>>>>>                                             <james.heather@mendeley.com
>>>>>>>>>                                             <mailto:james.heather@mendeley.com>>
>>>>>>>>>                                             wrote:
>>>>>>>>>                                             >
>>>>>>>>>                                             > Is anyone
>>>>>>>>>                                             interested
in
>>>>>>>>>                                             helping with
>>>>>>>>>                                             getting an
>>>>>>>>>                                             up-to-date
>>>>>>>>>                                             CDH5-compatible
>>>>>>>>>                                             build of
Phoenix
>>>>>>>>>                                             up and running?
>>>>>>>>>                                             >
>>>>>>>>>                                             > Cloudera
has a
>>>>>>>>>                                             build of
Phoenix
>>>>>>>>>                                             4.3
>>>>>>>>>                                             (https://github.com/cloudera-labs/phoenix),
>>>>>>>>>                                             but this
is now
>>>>>>>>>                                             two versions
>>>>>>>>>                                             behind, and
there
>>>>>>>>>                                             seems little
>>>>>>>>>                                             desire at
Cloudera
>>>>>>>>>                                             to keep it
updated.
>>>>>>>>>                                             >
>>>>>>>>>                                             > I imagine
that
>>>>>>>>>                                             by looking
at the
>>>>>>>>>                                             differences
>>>>>>>>>                                             between vanilla
>>>>>>>>>                                             4.3 and cloudera
>>>>>>>>>                                             labs 4.3,
and with
>>>>>>>>>                                             some guidance
from
>>>>>>>>>                                             this list,
we
>>>>>>>>>                                             could get
a good
>>>>>>>>>                                             idea of what
would
>>>>>>>>>                                             need to be
>>>>>>>>>                                             modified
in 4.5+
>>>>>>>>>                                             and keep
a
>>>>>>>>>                                             CDH5-compatible
>>>>>>>>>                                             build up
to date.
>>>>>>>>>                                             >
>>>>>>>>>                                             > Yes?
>>>>>>>>>                                             >
>>>>>>>>>                                             > James
>>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>
>>
>>
>>
>>
>>
>>                     -- 
>>                     Best regards,
>>
>>                        - Andy
>>
>>                     Problems worthy of attack prove their worth by
>>                     hitting back. - Piet Hein (via Tom White)
>>
>>
>>
>>
>>
>>             -- 
>>             Best regards,
>>
>>                - Andy
>>
>>             Problems worthy of attack prove their worth by hitting
>>             back. - Piet Hein (via Tom White)
>>
>>
>
>


Mime
View raw message