calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@apache.org>
Subject Re: Jenkins CI
Date Fri, 15 May 2015 18:27:14 GMT
Forking the conversation onto dev@kylin (dev@calcite to bcc, some of these
details are useful there too)

builds.apache.org is for unit testing. It's up to the project to provide
unit tests that can run in the environment. It's a pretty standard (maybe a
little old?) Jenkins install. Projects like Phoenix and HBase run UT's
there, some of which spin in mini-HDFS and mini-MR clusters in the process
for support. Phoenix also has a IT suite that can run on either a mini-XX
cluster, or be invoked against a real cluster.

Has Kylin spent much time with surefire/failsafe yet? Or maybe you don't
use maven at all?

Access to builds.apache.org is provided via project chair blessing to a
user's account. After that, they can login with their LDAP (id.apache.org)
creds and administer to jenkins -- set up new jobs and such. Generally
projects have 2 kinds of jenkins jobs: pre-commit and post-commit. There's
a pre-commit per project, and it's tied into a JIRA query that's currently
owned by Jake (on CC). There's generally a post-commit job per active
branch, and it's driven by polling the src repo.

Poke around on builds.apache.org, see what other projects are doing, and
ask more questions. INFRA is there to help us out, but don't abuse it ;)

-n

On Thu, May 14, 2015 at 10:57 PM, hongbin ma <mahongbin@apache.org> wrote:

> hi Nick,
>
> Thank you for you message. I don't know if it's okay to discuss our(Apache
> Kylin) CI requirements at Calcite's mail list. Please do let me know if
> it's inappropriate.
>
> For us Kylin, it's best if we could be provided with hadoop environment
> which enables launching MR jobs, Hive queries and HBase scans. I'm not sure
> if https://builds.apache.org/ could provide us such a isolated/shared
> hadoop env.
>
> If that's not possible, we do have a maven profile that run all the
> integration tests with hbase minicluster. The concern is that minicluster
> generates a lot of logs, and it may take around 30min~60min to finish all
> the test cases. We once tried http://travis-ci.org/, but our job was
> terminated because of too much logs. I don't know if
> https://builds.apache.org/ has similar limitations.
>
> BTW, can you give me a link to the script you mentioned?
>
>
>
> On Fri, May 15, 2015 at 1:38 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
>
> > I just set up the basics for Phoenix. Now that hbase 1.1 is released, I
> can
> > look at these items over here too. Mahone, let me know what you need
> done.
> > On Phoenix we reused HBase's scripts, which were recycled from Hadoop and
> > others. I think Hadoop just rebooted theirs, so maybe have a look and see
> > if you like it.
> >
> > On Thursday, May 14, 2015, hongbin ma <mahongbin@apache.org> wrote:
> >
> > > +1
> > >
> > > this is Mahone from apache kylin, please keep updated on the progress
> so
> > > that we can borrow your experience:)
> > >
> > > On Fri, May 15, 2015 at 1:14 AM, Julian Hyde <julianhyde@gmail.com
> > > <javascript:;>> wrote:
> > >
> > > > +1
> > > >
> > > > There is already an issue:
> > > > https://issues.apache.org/jira/browse/CALCITE-623
> > > >
> > > > On Thu, May 14, 2015 at 8:57 AM, Vladimir Sitnikov
> > > > <sitnikov.vladimir@gmail.com <javascript:;>> wrote:
> > > > > Hi,
> > > > >
> > > > > I think it would be nice to setup a Jenkins CI for the following:
> > > > > 1) Teting JIRA patches. I find it nice to have CI reports on the
> JIRA
> > > > patches.
> > > > > 2) Testing github pull requests
> > > > > (
> https://blogs.apache.org/infra/entry/github_pull_request_builds_now
> > )
> > > > > 3) In the long run -- integration testing (ASF Jenkins supports
> > > > > docker, so we can use it to test against postgresql, mysql, etc)
> > > > >
> > > > > If I understand wiki properly, someone from PMC Chairs can grant
> > admin
> > > > > account to a commiter
> > > > > http://wiki.apache.org/general/Jenkins?action=show&redirect=Hudson
> > > > >
> > > > > Is it the case?
> > > > > Are there any PMC members on the list that could grant me the
> > account?
> > > > >
> > > > > Vladimir
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > >
> > > *Bin Mahone | 马洪宾*
> > > Apache Kylin: http://kylin.io
> > > Github: https://github.com/binmahone
> > >
> >
>
>
>
> --
> Regards,
>
> *Bin Mahone | 马洪宾*
> Apache Kylin: http://kylin.io
> Github: https://github.com/binmahone
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message