spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <matei.zaha...@gmail.com>
Subject Re: addSparkListener in python
Date Wed, 20 Nov 2013 02:46:18 GMT
Take a look at http://py4j.sourceforge.net/advanced_topics.html#implementing-java-interfaces-from-python-callback.
Not 100% sure this will work, but this would be the way to do it.

Matei

On Nov 19, 2013, at 5:32 PM, Umar Javed <umarj.javed@gmail.com> wrote:

> It takes a SparkListener() as the argument. Any idea how I construct a
> scala object (a SparkListener() in this case) inside python and then pass
> it on to the Java SparkContext member? This is the only API call I need
> right now.
> 
> 
> On Tue, Nov 19, 2013 at 4:25 PM, Matei Zaharia <matei.zaharia@gmail.com>wrote:
> 
>> Sorry, this is currently not available in Python, though you might be able
>> to do it through Py4J by accessing the private Java SparkContext member of
>> the Python SparkContext. I’m curious, which pieces of the API do you want
>> to use?
>> 
>> Matei
>> 
>> On Nov 19, 2013, at 1:44 PM, Umar Javed <umarj.javed@gmail.com> wrote:
>> 
>>> Is there a way to write the addSparkListener() api call for pyspark, or
>>> should I switch to writing my scripts in scala in order to use this api
>>> call?
>>> 
>>> thanks!
>>> Umar
>> 
>> 


Mime
View raw message