sessionState could not be accessed in spark-shell command line

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

sessionState could not be accessed in spark-shell command line

ChenJun Zou
Hi, 

when I use spark-shell to get the logical plan of  sql, an error occurs 

scala> spark.sessionState
<console>:30: error: lazy value sessionState in class SparkSession cannot be accessed in org.apache.spark.sql.SparkSession
       spark.sessionState
             ^

But if I use spark-submit to access the "sessionState" variable, It's OK. 

Is there a way to access it in spark-shell?
Reply | Threaded
Open this post in threaded view
|

Re: sessionState could not be accessed in spark-shell command line

sujith71955
Hi, 
    may I know which version of spark you are using, in 2.2 I tried with below query in spark-shell for viewing the logical plan and it's working fine

spark.sql("explain extended select * from table1")

The above query you can use for seeing logical plan.

Thanks,
Sujith 

On Thu, 7 Sep 2017 at 11:03 AM, ChenJun Zou <[hidden email]> wrote:
Hi, 

when I use spark-shell to get the logical plan of  sql, an error occurs 

scala> spark.sessionState
<console>:30: error: lazy value sessionState in class SparkSession cannot be accessed in org.apache.spark.sql.SparkSession
       spark.sessionState
             ^

But if I use spark-submit to access the "sessionState" variable, It's OK. 

Is there a way to access it in spark-shell?
Reply | Threaded
Open this post in threaded view
|

Re: sessionState could not be accessed in spark-shell command line

ChenJun Zou
spark-2.1.1 I use



2017-09-07 14:00 GMT+08:00 sujith chacko <[hidden email]>:
Hi, 
    may I know which version of spark you are using, in 2.2 I tried with below query in spark-shell for viewing the logical plan and it's working fine

spark.sql("explain extended select * from table1")

The above query you can use for seeing logical plan.

Thanks,
Sujith 

On Thu, 7 Sep 2017 at 11:03 AM, ChenJun Zou <[hidden email]> wrote:
Hi, 

when I use spark-shell to get the logical plan of  sql, an error occurs 

scala> spark.sessionState
<console>:30: error: lazy value sessionState in class SparkSession cannot be accessed in org.apache.spark.sql.SparkSession
       spark.sessionState
             ^

But if I use spark-submit to access the "sessionState" variable, It's OK. 

Is there a way to access it in spark-shell?

Reply | Threaded
Open this post in threaded view
|

Re: sessionState could not be accessed in spark-shell command line

sujith71955
If your intention is to just view the logical plan in spark shell  then I think you can follow the query which I mentioned in previous mail.  In spark 2.1.0 sessionState is a private member which you cannot access.

Thanks.


On Thu, 7 Sep 2017 at 11:39 AM, ChenJun Zou <[hidden email]> wrote:
spark-2.1.1 I use



2017-09-07 14:00 GMT+08:00 sujith chacko <[hidden email]>:
Hi, 
    may I know which version of spark you are using, in 2.2 I tried with below query in spark-shell for viewing the logical plan and it's working fine

spark.sql("explain extended select * from table1")

The above query you can use for seeing logical plan.

Thanks,
Sujith 

On Thu, 7 Sep 2017 at 11:03 AM, ChenJun Zou <[hidden email]> wrote:
Hi, 

when I use spark-shell to get the logical plan of  sql, an error occurs 

scala> spark.sessionState
<console>:30: error: lazy value sessionState in class SparkSession cannot be accessed in org.apache.spark.sql.SparkSession
       spark.sessionState
             ^

But if I use spark-submit to access the "sessionState" variable, It's OK. 

Is there a way to access it in spark-shell?

Reply | Threaded
Open this post in threaded view
|

Re: sessionState could not be accessed in spark-shell command line

ChenJun Zou
thanks, 
my mistake 

2017-09-07 14:21 GMT+08:00 sujith chacko <[hidden email]>:
If your intention is to just view the logical plan in spark shell  then I think you can follow the query which I mentioned in previous mail.  In spark 2.1.0 sessionState is a private member which you cannot access.

Thanks.


On Thu, 7 Sep 2017 at 11:39 AM, ChenJun Zou <[hidden email]> wrote:
spark-2.1.1 I use



2017-09-07 14:00 GMT+08:00 sujith chacko <[hidden email]>:
Hi, 
    may I know which version of spark you are using, in 2.2 I tried with below query in spark-shell for viewing the logical plan and it's working fine

spark.sql("explain extended select * from table1")

The above query you can use for seeing logical plan.

Thanks,
Sujith 

On Thu, 7 Sep 2017 at 11:03 AM, ChenJun Zou <[hidden email]> wrote:
Hi, 

when I use spark-shell to get the logical plan of  sql, an error occurs 

scala> spark.sessionState
<console>:30: error: lazy value sessionState in class SparkSession cannot be accessed in org.apache.spark.sql.SparkSession
       spark.sessionState
             ^

But if I use spark-submit to access the "sessionState" variable, It's OK. 

Is there a way to access it in spark-shell?


Reply | Threaded
Open this post in threaded view
|

Re: sessionState could not be accessed in spark-shell command line

ChenJun Zou
I am examined the code and found lazy val is added recently in 2.2.0

2017-09-07 14:34 GMT+08:00 ChenJun Zou <[hidden email]>:
thanks, 
my mistake 

2017-09-07 14:21 GMT+08:00 sujith chacko <[hidden email]>:
If your intention is to just view the logical plan in spark shell  then I think you can follow the query which I mentioned in previous mail.  In spark 2.1.0 sessionState is a private member which you cannot access.

Thanks.


On Thu, 7 Sep 2017 at 11:39 AM, ChenJun Zou <[hidden email]> wrote:
spark-2.1.1 I use



2017-09-07 14:00 GMT+08:00 sujith chacko <[hidden email]>:
Hi, 
    may I know which version of spark you are using, in 2.2 I tried with below query in spark-shell for viewing the logical plan and it's working fine

spark.sql("explain extended select * from table1")

The above query you can use for seeing logical plan.

Thanks,
Sujith 

On Thu, 7 Sep 2017 at 11:03 AM, ChenJun Zou <[hidden email]> wrote:
Hi, 

when I use spark-shell to get the logical plan of  sql, an error occurs 

scala> spark.sessionState
<console>:30: error: lazy value sessionState in class SparkSession cannot be accessed in org.apache.spark.sql.SparkSession
       spark.sessionState
             ^

But if I use spark-submit to access the "sessionState" variable, It's OK. 

Is there a way to access it in spark-shell?