spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Darabos <daniel.dara...@lynxanalytics.com>
Subject Re: Better line number hints for logging?
Date Wed, 04 Jun 2014 15:44:35 GMT
Oh, this would be super useful for us too!

Actually wouldn't it be best if you could see the whole call stack on the
UI, rather than just one line? (Of course you would have to click to expand
it.)


On Wed, Jun 4, 2014 at 2:38 AM, John Salvatier <jsalvatier@gmail.com> wrote:

> Ok, I will probably open a Jira.
>
>
> On Tue, Jun 3, 2014 at 5:29 PM, Matei Zaharia <matei.zaharia@gmail.com>
> wrote:
>
>> You can use RDD.setName to give it a name. There’s also a creationSite
>> field that is private[spark] — we may want to add a public setter for that
>> later. If the name isn’t enough and you’d like this, please open a JIRA
>> issue for it.
>>
>> Matei
>>
>> On Jun 3, 2014, at 5:22 PM, John Salvatier <jsalvatier@gmail.com> wrote:
>>
>> I have created some extension methods for RDDs in RichRecordRDD and these
>> are working exceptionally well for me.
>>
>> However, when looking at the logs, its impossible to tell what's going on
>> because all the line number hints point to RichRecordRDD.scala rather than
>> the code that uses it. For example:
>>
>>> INFO scheduler.DAGScheduler: Submitting Stage 122 (MappedRDD[1223] at
>>> map at RichRecordRDD.scala:633), which is now runnable
>>
>> Is there any way set up my extension methods class so that the logs will
>> print a more useful line number?
>>
>>
>>
>

Mime
View raw message