spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shane knapp <skn...@berkeley.edu>
Subject Re: moving the spark jenkins job builder repo from dbricks --> spark
Date Wed, 17 Oct 2018 17:48:09 GMT
On Wed, Oct 17, 2018 at 10:25 AM Yin Huai <yhuai@databricks.com> wrote:

> Shane, Thank you for initiating this work! Can we do an audit of jenkins
> users and trim down the list?
>
> re pruning external (spark-specific) users w/shell and jenkins login
access:  we can absolutely do this.

limiting logins for EECS students/faculty/staff is possible, but i will
need to do some experiments.  we're using SSSD to manage our LDAP logins,
and it is supposed to handle group filtering but i haven't had much luck
actually getting it working.


> Also, for packaging jobs, those branch snapshot jobs are active (for
> example,
> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-master-maven-snapshots/
> for publishing snapshot builds from master branch). They still need
> credentials. After we remove the encrypted credential file, are we planning
> to use jenkins as the single place to manage those credentials and we just
> refer to them in jenkins job config?
>
> well, since the creds in the repo are actually encrypted, i think that
keeping them in there is actually fine.  since i wasn't the one who set any
of this up, however, i will defer to josh about this.

shane


> On Wed, Oct 10, 2018 at 12:06 PM shane knapp <sknapp@berkeley.edu> wrote:
>
>> Not sure if that's what you meant; but it should be ok for the jenkins
>>> servers to manually sync with master after you (or someone else) have
>>> verified the changes. That should prevent inadvertent breakages since
>>> I don't expect it to be easy to test those scripts without access to
>>> some test jenkins server.
>>>
>>> JJB has some built-in lint and testing, so that'll be the first step in
>> verifying the build configs.
>>
>> i still have a dream where i have a fully functioning jenkins staging
>> deployment...  one day i will make that happen.  :)
>>
>> shane
>>
>> --
>> Shane Knapp
>> UC Berkeley EECS Research / RISELab Staff Technical Lead
>> https://rise.cs.berkeley.edu
>>
>

-- 
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

Mime
View raw message