spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ChenJun Zou <stephen....@gmail.com>
Subject Re: sessionState could not be accessed in spark-shell command line
Date Thu, 07 Sep 2017 06:35:26 GMT
I am examined the code and found lazy val is added recently in 2.2.0

2017-09-07 14:34 GMT+08:00 ChenJun Zou <stephen.zcj@gmail.com>:

> thanks,
> my mistake
>
> 2017-09-07 14:21 GMT+08:00 sujith chacko <sujithchacko.2010@gmail.com>:
>
>> If your intention is to just view the logical plan in spark shell  then I
>> think you can follow the query which I mentioned in previous mail.  In
>> spark 2.1.0 sessionState is a private member which you cannot access.
>>
>> Thanks.
>>
>>
>> On Thu, 7 Sep 2017 at 11:39 AM, ChenJun Zou <stephen.zcj@gmail.com>
>> wrote:
>>
>>> spark-2.1.1 I use
>>>
>>>
>>>
>>> 2017-09-07 14:00 GMT+08:00 sujith chacko <sujithchacko.2010@gmail.com>:
>>>
>>>> Hi,
>>>>     may I know which version of spark you are using, in 2.2 I tried
>>>> with below query in spark-shell for viewing the logical plan and it's
>>>> working fine
>>>>
>>>> spark.sql("explain extended select * from table1")
>>>>
>>>> The above query you can use for seeing logical plan.
>>>>
>>>> Thanks,
>>>> Sujith
>>>>
>>>> On Thu, 7 Sep 2017 at 11:03 AM, ChenJun Zou <stephen.zcj@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> when I use spark-shell to get the logical plan of  sql, an error
>>>>> occurs
>>>>>
>>>>> scala> spark.sessionState
>>>>> <console>:30: error: lazy value sessionState in class SparkSession
>>>>> cannot be accessed in org.apache.spark.sql.SparkSession
>>>>>        spark.sessionState
>>>>>              ^
>>>>>
>>>>> But if I use spark-submit to access the "sessionState" variable, It's
>>>>> OK.
>>>>>
>>>>> Is there a way to access it in spark-shell?
>>>>>
>>>>
>>>
>

Mime
View raw message