spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Boesch <java...@gmail.com>
Subject Re: Iterative pyspark / scala codebase development
Date Fri, 27 Mar 2015 17:18:04 GMT
Compile alone did not show the scala code changes AFAICT. I will reverify.

2015-03-27 10:16 GMT-07:00 Davies Liu <davies@databricks.com>:

> I usually just open a terminal to do `build/sbt ~compile`, coding in
> IntelliJ, then run python tests in another terminal once it compiled
> successfully.
>
> On Fri, Mar 27, 2015 at 10:11 AM, Reynold Xin <rxin@databricks.com> wrote:
> > Python is tough if you need to change Scala at the same time.
> >
> > sbt/sbt assembly/assembly
> >
> > can be slightly faster than just assembly.
> >
> >
> > On Fri, Mar 27, 2015 at 10:02 AM, Stephen Boesch <javadba@gmail.com>
> wrote:
> >
> >> I am iteratively making changes to the scala side of some new pyspark
> code
> >> and re-testing from the python/pyspark side.
> >>
> >> Presently my only solution is to rebuild completely
> >>
> >>       sbt assembly
> >>
> >> after any scala side change - no matter how small.
> >>
> >> Any better / expedited way for pyspark to see small scala side updates?
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message