lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Thacker <va...@vthacker.in>
Subject Re: Release a 6.6.1
Date Sat, 26 Aug 2017 13:37:20 GMT
I tried building it a few times through today and Solr tests would fail
always:

One of them reproduced twice for me : ant test
 -Dtestcase=TestZkAclsWithHadoopAuth -Dtests.seed=1E9BBBB18DB3E57B
-Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=Kwajalein
-Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1


I'm going to try understanding what's wrong with the tests before moving
forward


On Sat, Aug 26, 2017 at 11:28 AM, Varun Thacker <varun@vthacker.in> wrote:

> Thanks Steve! I changed the exit condition and now trying to built the RC
> again
>
> I had to commit the change on branch_6_6 - https://git-wip-us.apache.org/
> repos/asf?p=lucene-solr.git;h=6dd958a.  If it doesn't work I'll revert
> the change.
>
> On Thu, Aug 24, 2017 at 4:11 AM, Steve Rowe <sarowe@gmail.com> wrote:
>
>> Hi Varun,
>>
>> I’m not sure what’s happening, but I’d guess that “result = p.poll()” in
>> runAndSendGPGPassword() indicates that the process hasn’t completed or has
>> an error condition.  Maybe print out “result"?  The python docs say that
>> None is returned when the process hasn’t completed.
>>
>> If the process hasn’t completed, then it’s just a matter of waiting until
>> it does.  The loop that looks for a password prompt has an exit condition
>> of an empty line returned from p.stdout.readline(), but maybe there should
>> be an additional exit condition: p.poll() returning non-None.
>>
>> --
>> Steve
>> www.lucidworks.com
>>
>> > On Aug 23, 2017, at 5:51 PM, Varun Thacker <varun@vthacker.in> wrote:
>> >
>> > So I tried again and ran into the same issue -
>> >
>> >   File "dev-tools/scripts/buildAndPushRelease.py", line 114, in prepare
>> >     runAndSendGPGPassword(cmd, gpgPassword)
>> >   File "dev-tools/scripts/buildAndPushRelease.py", line 60, in
>> runAndSendGPGPassword
>> >     raise RuntimeError(msg)
>> > RuntimeError:     FAILED: ant -Dversion=6.6.1 -Dgpg.key=7F3DE8DA
>> prepare-release [see log /tmp/release.log]
>> >
>> > At this point I took the the runAndSendGPGPassword method and ran it
>> from a separate python program and it passed. So not sure why it fails when
>> used directly from buildAndPushRelease especially since the output shows
>> build successful for the "prepare-release" target and the "sign-artifacts"
>> target completed as well.
>> >
>> >
>> >
>> >
>> > On Wed, Aug 23, 2017 at 9:11 PM, Ishan Chattopadhyaya <
>> ichattopadhyaya@gmail.com> wrote:
>> > Cool, all the best! In cases when the release process completed without
>> further errors, except for the one above, the generated artifacts
>> eventually failed the smoke test (with some missing signature files error).
>> So, I had to redo them making sure that the sign-artifacts step was not
>> missed (I remember waiting for that step in order to wait for the password
>> prompt).
>> >
>> > On Wed, Aug 23, 2017 at 4:42 PM, Varun Thacker <varun@vthacker.in>
>> wrote:
>> > Hi Ishan,
>> >
>> > That's useful info!
>> >
>> > The failure I posted was from my second attempt with "export
>> GPG_TTY=$(tty)" present. I was prompted the password.
>> >
>> > Once I entered the password the "sign-artifacts:" phase looks to have
>> completed. The attached output doesn't even show a failure.
>> >
>> > I'm going to give it another try in the meanwhile
>> >
>> > On Wed, Aug 23, 2017 at 4:20 PM, Ishan Chattopadhyaya <
>> ichattopadhyaya@gmail.com> wrote:
>> > Varun, I had the same issue. Please see my notes in the end of the
>> ReleaseToDo section.
>> >
>> > On Wed, Aug 23, 2017 at 1:43 PM, Varun Thacker <varun@vthacker.in>
>> wrote:
>> > An update on the RC build : In the first couple of attempts a Solr test
>> would fail so the process would get aborted.
>> >
>> > Then I hit "gpg: signing failed: Inappropriate ioctl for device" in the
>> "prepare-release" phase. I was able to fix this by installing a couple of
>> extra packages and following some instructions online.
>> >
>> > In the last attempt I hit this:
>> >
>> > Prepare release...
>> >   git pull...
>> >   git clone is clean
>> >   git rev: f4fb90886690c829a062f4243a62825f810ad359
>> >   Check DOAP files
>> >   ant clean test validate documentation-lint
>> >   lucene prepare-release
>> >     FAILED: ant -Dversion=6.6.1 -Dgpg.key=7F3DE8DA prepare-release [see
>> log /tmp/release.log]
>> > Traceback (most recent call last):
>> >   File "dev-tools/scripts/buildAndPushRelease.py", line 313, in
>> <module>
>> >     main()
>> >   File "dev-tools/scripts/buildAndPushRelease.py", line 294, in main
>> >     rev = prepare(c.root, c.version, c.key_id, c.key_password)
>> >   File "dev-tools/scripts/buildAndPushRelease.py", line 114, in prepare
>> >     runAndSendGPGPassword(cmd, gpgPassword)
>> >   File "dev-tools/scripts/buildAndPushRelease.py", line 60, in
>> runAndSendGPGPassword
>> >     raise RuntimeError(msg)
>> > RuntimeError:     FAILED: ant -Dversion=6.6.1 -Dgpg.key=7F3DE8DA
>> prepare-release [see log /tmp/release.log]
>> >
>> > Here's the release.log output : https://www.dropbox.com/s/yssv
>> qbwcpjrq7ah/release.log?dl=0 which seems to indicate that it succeeded ?
>> >
>> >
>> > On Tue, Aug 22, 2017 at 8:10 AM, Varun Thacker <varun@vthacker.in>
>> wrote:
>> > Hi Uwe,
>> >
>> > I can backport it and then I'll begin working on the RC
>> >
>> > On Tue, Aug 22, 2017 at 2:17 AM, Uwe Schindler <uwe@thetaphi.de> wrote:
>> > Hi,
>> >
>> >
>> >
>> > The windows startup script fix was committed to 7.0, but not 6.6.1. If
>> Varun has time to test the fixes in https://issues.apache.org/jira
>> /browse/SOLR-8689 we may include them, but I left them out for now.
>> Please reopen the issue if you think it’s worth. It should be a simple
>> cherry pick.
>> >
>> >
>> >
>> > Uwe
>> >
>> >
>> >
>> > -----
>> >
>> > Uwe Schindler
>> >
>> > Achterdiek 19, D-28357 Bremen
>> >
>> > http://www.thetaphi.de
>> >
>> > eMail: uwe@thetaphi.de
>> >
>> >
>> >
>> > From: Uwe Schindler [mailto:uwe@thetaphi.de]
>> > Sent: Monday, August 21, 2017 2:11 PM
>> >
>> >
>> > To: dev@lucene.apache.org
>> > Subject: RE: Release a 6.6.1
>> >
>> >
>> >
>> > Hi,
>> >
>> >
>> >
>> > Is there any interest to maybe get the Windows Shell scripts fixed, so
>> they work with Java 9? Currently you cannot start Solr on Windows systems,
>> as the version number checking and the GC options are not working. It’s
>> https://issues.apache.org/jira/browse/SOLR-8689 - I am just waiting for
>> feedback and wanted to put it into 7.0. But if there is interest, I can
>> backport to 6.6, too (not yet planned).
>> >
>> >
>> >
>> > Yesterday (before your mail), I already backported a Hadoop 2.7.2 ->
>> 2.7.4 update, so it works now with Java 9. This made the ugly workaround
>> obsolete (changing java.version sysprop temporarily). This fix is now in
>> 6.6.1 and 7.0 branch.
>> >
>> >
>> >
>> > Uwe
>> >
>> >
>> >
>> > -----
>> >
>> > Uwe Schindler
>> >
>> > Achterdiek 19, D-28357 Bremen
>> >
>> > http://www.thetaphi.de
>> >
>> > eMail: uwe@thetaphi.de
>> >
>> >
>> >
>> > From: Varun Thacker [mailto:varun@vthacker.in]
>> > Sent: Monday, August 21, 2017 12:49 PM
>> > To: dev@lucene.apache.org
>> > Subject: Re: Release a 6.6.1
>> >
>> >
>> >
>> > Hi Everyone,
>> >
>> >
>> >
>> > Does anyone plan on committing any more fixes to the release branch?
>> >
>> >
>> >
>> > I've disabled SOLR-11247 for the branch so that we get a few Jenkins
>> runs without any failures.
>> >
>> >
>> >
>> > I plan on creating an RC in 12-14 hours time from now if all goes well
>> >
>> >
>> >
>> > On Mon, Aug 21, 2017 at 8:50 AM, Erick Erickson <
>> erickerickson@gmail.com> wrote:
>> >
>> > Uwe:
>> >
>> >
>> >
>> > As far as I'm concerned, please do put both in. Varun is the RM and has
>> final say of course. He may be traveling though and be a little delayed
>> responding.
>> >
>> >
>> >
>> > Erick
>> >
>> >
>> >
>> > On Sun, Aug 20, 2017 at 5:41 AM, Uwe Schindler <uwe@thetaphi.de> wrote:
>> >
>> > Hi,
>> >
>> >
>> >
>> > I just noticed, that our Hadoop friends released Hadoop 2.7.4. This
>> fixes the stupid Java 9 bug in their static initializer
>> (StringIndexOutOfBounds). So I’d like to also get
>> https://issues.apache.org/jira/browse/SOLR-11261 in. If Jenkins is happy
>> on 7.x and master, this should be easy.
>> >
>> >
>> >
>> > If you think it’s too risky (Hadoop 2.7.2 -> 2.7.4), we can live with
>> the workaround in Lucene 6.6.1! But the workaround is really hacky: It
>> changes the “java.version” system property temporarily on Java 9 while
>> initializing Hadoop, which is not something you should ever do!
>> >
>> >
>> >
>> > Uwe
>> >
>> >
>> >
>> > -----
>> >
>> > Uwe Schindler
>> >
>> > Achterdiek 19, D-28357 Bremen
>> >
>> > http://www.thetaphi.de
>> >
>> > eMail: uwe@thetaphi.de
>> >
>> >
>> >
>> > From: Uwe Schindler [mailto:uwe@thetaphi.de]
>> > Sent: Sunday, August 20, 2017 12:53 PM
>> > To: dev@lucene.apache.org
>> > Subject: RE: Release a 6.6.1
>> >
>> >
>> >
>> > Hi,
>> >
>> >
>> >
>> > I need to backport SOLR-10966 to branch 6.6, otherwise Jenkins does not
>> pass with Java 9.
>> >
>> >
>> >
>> > Uwe
>> >
>> >
>> >
>> > -----
>> >
>> > Uwe Schindler
>> >
>> > Achterdiek 19, D-28357 Bremen
>> >
>> > http://www.thetaphi.de
>> >
>> > eMail: uwe@thetaphi.de
>> >
>> >
>> >
>> > From: Uwe Schindler [mailto:uwe@thetaphi.de]
>> > Sent: Saturday, August 19, 2017 12:00 AM
>> > To: dev@lucene.apache.org
>> > Subject: Re: Release a 6.6.1
>> >
>> >
>> >
>> > Hi,
>> >
>> > I enabled Jenkins jobs on 👮. ASF was active already.
>> >
>> > Uwe
>> >
>> > Am 18. August 2017 23:34:23 MESZ schrieb Varun Thacker <
>> varun@vthacker.in>:
>> >
>> > From the bug fixes in lucene 7.0 do we need to backport any of these
>> issues :  LUCENE-7859 / LUCENE-7871 / LUCENE-7914 ?
>> >
>> >
>> >
>> > I plan on backporting these three Solr fixes on Sunday
>> >
>> >
>> >
>> > SOLR-10698
>> >
>> > SOLR-10719
>> >
>> > SOLR-11228
>> >
>> >
>> >
>> > looking through the 7.0 bug fixes these two look important to get in as
>> well :
>> >
>> >
>> >
>> > SOLR-10983
>> >
>> > SOLR-9262
>> >
>> >
>> >
>> > So if no one get's to it I'll try backporting them as well
>> >
>> >
>> >
>> > Can someone please enable Jenkins on the branch again?
>> >
>> >
>> >
>> >
>> >
>> > On Thu, Aug 17, 2017 at 3:18 PM, Erick Erickson <
>> erickerickson@gmail.com> wrote:
>> >
>> > Right, that was the original note before we decided to backport a
>> > bunch of other stuff and I decided it made no sense to omit this one.
>> > All that has to happen is remove the " (note, not in 7.0, is in 7.1)"
>> > bits since it's in 6.6, 6.x, 7.0, 7.1 and master.
>> >
>> > Good catch!
>> >
>> >
>> >
>> >
>> > On Thu, Aug 17, 2017 at 3:10 PM, Varun Thacker <varun@vthacker.in>
>> wrote:
>> > > Should I then go remove the note part from the CHANGES entry in
>> branch_6_6 ?
>> > >
>> > > * SOLR-11177: CoreContainer.load needs to send lazily loaded core
>> > > descriptors to the proper list rather than send
>> > >   them all to the transient lists. (Erick Erickson) (note, not in
>> 7.0, is in
>> > > 7.1)
>> > >
>> > > I see a commit for this in branch_7_0
>> > >
>> > > Commit c73b5429b722b09b9353ec82627a35e2b864b823 in lucene-solr's
>> branch
>> > > refs/heads/branch_7_0 from Erick
>> > > [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c73b542
>> ]
>> > >
>> > >
>> > >
>> > > On Thu, Aug 17, 2017 at 2:48 PM, Erick Erickson <
>> erickerickson@gmail.com>
>> > > wrote:
>> > >>
>> > >> Well, it is in 7.0. Everything I moved to 6.6.1 is also in 7.0, or
>> should
>> > >> be.
>> > >>
>> > >> On Thu, Aug 17, 2017 at 2:31 PM, Varun Thacker <varun@vthacker.in>
>> wrote:
>> > >> > Hi Erick,
>> > >> >
>> > >> > I was going through the CHANGES file from the 6_6 branch and just
>> > >> > curious
>> > >> > why are we not planning on putting SOLR-11177 in 7.0 ?
>> > >> >
>> > >> > On Thu, Aug 17, 2017 at 7:45 AM, Erick Erickson
>> > >> > <erickerickson@gmail.com>
>> > >> > wrote:
>> > >> >>
>> > >> >> OK, I'm done with my changes for 7.0, I think Varun might
have a
>> few
>> > >> >> too.
>> > >> >>
>> > >> >> And things didn't melt down overnight so...
>> > >> >>
>> > >> >> On Wed, Aug 16, 2017 at 12:25 PM, Anshum Gupta <
>> anshum@anshumgupta.net>
>> > >> >> wrote:
>> > >> >> > +1 on getting the fixes into 7.0 if you are confident
with
>> those, and
>> > >> >> > if
>> > >> >> > they are a part of 6.6.1.
>> > >> >> >
>> > >> >> > Thanks for taking care of this Erick.
>> > >> >> >
>> > >> >> > On Wed, Aug 16, 2017 at 12:24 PM Erick Erickson
>> > >> >> > <erickerickson@gmail.com>
>> > >> >> > wrote:
>> > >> >> >>
>> > >> >> >> FYI:
>> > >> >> >>
>> > >> >> >> I'll be backporting the following to SOLR 7.0 today:
>> > >> >> >>
>> > >> >> >> SOLR-11024: ParallelStream should set the StreamContext
when
>> > >> >> >> constructing SolrStreams:
>> > >> >> >> SOLR-11177: CoreContainer.load needs to send lazily
loaded core
>> > >> >> >> descriptors to the proper list rather than send them
all to the
>> > >> >> >> transient lists.
>> > >> >> >> SOLR-11122: Creating a core should write a core.properties
file
>> > >> >> >> first
>> > >> >> >> and clean up on failure
>> > >> >> >>
>> > >> >> >> and those as well as several others to 6.6.1.
>> > >> >> >>
>> > >> >> >> Since some of these depend on others, I need to add
them in a
>> > >> >> >> specific
>> > >> >> >> order. I intend to run minimal tests for each JIRA
before
>> pushing,
>> > >> >> >> then when they are all in place go through the full
test cycle,
>> > >> >> >> precommit and all that. I doubt that other than the
flurry of
>> commit
>> > >> >> >> messages anyone will notice, at least if this stuff
is as safe
>> to
>> > >> >> >> backport as I believe people _better_ not.
>> > >> >> >>
>> > >> >> >> If these cause any serious problems for the 7.0 code
line,
>> feel free
>> > >> >> >> to back them out any time after today.
>> > >> >> >>
>> > >> >> >> Why am I bothering the 7.0 code line? well, it's
awkward to
>> have a
>> > >> >> >> fix
>> > >> >> >> in 6.6.1, skip 7.0 and have it show up again in 7.1.
We can
>> live
>> > >> >> >> with
>> > >> >> >> some thing not being in 7.0 if any of this causes
disruptions
>> > >> >> >> though.
>> > >> >> >>
>> > >> >> >> Erick
>> > >> >> >>
>> > >> >> >> On Wed, Aug 16, 2017 at 11:39 AM, Erik Hatcher
>> > >> >> >> <erik.hatcher@gmail.com>
>> > >> >> >> wrote:
>> > >> >> >> > Yes, I’m confident in that patch and its safety,
thanks
>> Varun!
>> > >> >> >> >
>> > >> >> >> > Erik
>> > >> >> >> >
>> > >> >> >> > On Aug 16, 2017, at 2:22 PM, Varun Thacker <
>> varun@vthacker.in>
>> > >> >> >> > wrote:
>> > >> >> >> >
>> > >> >> >> > @Erik -If you are confident with the patch and
if you think
>> it's
>> > >> >> >> > safe
>> > >> >> >> > then
>> > >> >> >> > please go ahead and commit it. I'll try having
a look at it
>> > >> >> >> > tomorrow
>> > >> >> >> > as
>> > >> >> >> > well.
>> > >> >> >> >
>> > >> >> >> > I will start back-porting issues which I am
comfortable
>> with. If
>> > >> >> >> > others
>> > >> >> >> > can
>> > >> >> >> > chime in as well it will be great.
>> > >> >> >> >
>> > >> >> >> > I'll aim to cut an RC on Monday 21st August
evening PST time
>> so
>> > >> >> >> > everyone
>> > >> >> >> > get's time to get the fixes in.
>> > >> >> >> >
>> > >> >> >> > Any objections in the approach?
>> > >> >> >> >
>> > >> >> >> > On Tue, Aug 15, 2017 at 4:56 AM, Erik Hatcher
>> > >> >> >> > <erik.hatcher@gmail.com>
>> > >> >> >> > wrote:
>> > >> >> >> >>
>> > >> >> >> >> I’d like to get https://issues.apache.org/jira
>> /browse/SOLR-10874
>> > >> >> >> >> in
>> > >> >> >> >> soon
>> > >> >> >> >> as well.   Varun, if you’d like to take
this one over
>> that’d be
>> > >> >> >> >> fine
>> > >> >> >> >> by
>> > >> >> >> >> me
>> > >> >> >> >> too ;) otherwise prod me back channel and
I’ll get to it.
>> > >> >> >> >> Ideally I
>> > >> >> >> >> should
>> > >> >> >> >> have applied this one ages ago for 7.0,
so maybe it can
>> make it
>> > >> >> >> >> in
>> > >> >> >> >> time
>> > >> >> >> >> for
>> > >> >> >> >> 6.6.1 and 7.0.
>> > >> >> >> >>
>> > >> >> >> >> Erik
>> > >> >> >> >>
>> > >> >> >> >> On Aug 14, 2017, at 5:51 PM, Varun Thacker
<
>> varun@vthacker.in>
>> > >> >> >> >> wrote:
>> > >> >> >> >>
>> > >> >> >> >> From the change log of 6.7 / 7.0 and 7.1
the total count of
>> bug
>> > >> >> >> >> fixes
>> > >> >> >> >> under lucene and solr are
>> > >> >> >> >>
>> > >> >> >> >> Lucene : 7
>> > >> >> >> >> Solr : 61
>> > >> >> >> >>
>> > >> >> >> >>
>> > >> >> >> >> I'd be happy to volunteer as a release manager
for a bug fix
>> > >> >> >> >> 6.6.1
>> > >> >> >> >> release
>> > >> >> >> >> if others in the community think it's a
good idea
>> > >> >> >> >>
>> > >> >> >> >>
>> > >> >> >> >
>> > >> >> >> >
>> > >> >> >>
>> > >> >> >>
>> > >> >> >> ------------------------------------------------------------
>> ---------
>> > >> >> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > >> >> >> For additional commands, e-mail: dev-help@lucene.apache.org
>> > >> >> >>
>> > >> >> >
>> > >> >>
>> > >> >> ------------------------------------------------------------
>> ---------
>> > >> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > >> >> For additional commands, e-mail: dev-help@lucene.apache.org
>> > >> >>
>> > >> >
>> > >>
>> > >> ------------------------------------------------------------
>> ---------
>> > >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > >> For additional commands, e-mail: dev-help@lucene.apache.org
>> > >>
>> > >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: dev-help@lucene.apache.org
>> >
>> >
>> >
>> >
>> > --
>> > Uwe Schindler
>> > Achterdiek 19, 28357 Bremen
>> > https://www.thetaphi.de
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>

Mime
View raw message