spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject Re: Spark 2.0 - Join statement compile error
Date Tue, 30 Aug 2016 09:52:10 GMT
Hi Mich,

This is the first time I've been told about $ for string interpolation (as
the function not the placeholder). Thanks for letting me know about it!

What is often used is s"whatever you want to reference inside the string
$-prefix unless it is a complex expression" i.e.

scala> s"I'm using $spark in ${spark.version}"
res0: String = I'm using org.apache.spark.sql.SparkSession@1fc1c7e in
2.1.0-SNAPSHOT


Pozdrawiam,
Jacek Laskowski
----
https://medium.com/@jaceklaskowski/
Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski

On Tue, Aug 30, 2016 at 10:27 AM, Mich Talebzadeh <mich.talebzadeh@gmail.com
> wrote:

> Actually I doubled checked this ā€˜sā€™ String Interpolator
>
> In Scala
>
> scala> val chars =  "This is Scala"
> chars: String = This is Scala
>
> scala> println($"$chars")
> This is Scala
> OK so far fine. In shell (ksh) can do
>
> chars="This is Scala"
> print "$chars"
> This is Scala
>
> In Shell
>
> print "$charsand it is interesting"
>  it is interesting
>
> Have a problem with interpretation!
>
> This will work by using {} brackers
>
> print "${chars} and it is interesting"
> This is Scala and it is interesting
>
>
> In Scala
>
> println($"$charsand it is interesting")
> <console>:24: error: not found: value charsand
>        println($"$charsand it is interesting")
>
> Likewise shell this will work by deploying the brackets {}
>
> println($"${chars} and it is interesting")
> This is Scala and it is interesting
> So I presume it is best practice to use ${} in Scala like shell? Although
> in most cases $VALUE should work.
>
>
> Cheers
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
> On 28 August 2016 at 09:25, Mich Talebzadeh <mich.talebzadeh@gmail.com>
> wrote:
>
>> Yes I realised that. Actually I thought it was s not $. it has been
>> around in shell for years say for actual values --> ${LOG_FILE}, for
>> position 's/ etc
>>
>>
>> cat ${LOG_FILE} | egrep -v 'rows affected|return status|&&&' | sed -e
>> 's/^[            ]*//g' -e 's/^        //g' -e '/^$/d' > temp.out
>>
>>
>> Dr Mich Talebzadeh
>>
>>
>>
>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>> On 28 August 2016 at 08:47, Jacek Laskowski <jacek@japila.pl> wrote:
>>
>>> Hi Mich,
>>>
>>> This is Scala's string interpolation which allow for replacing
>>> $-prefixed expressions with their values.
>>>
>>> It's what cool kids use in Scala to do templating and concatenation šŸ˜
>>>
>>> Jacek
>>>
>>> On 23 Aug 2016 9:21 a.m., "Mich Talebzadeh" <mich.talebzadeh@gmail.com>
>>> wrote:
>>>
>>>> What is   --> s below before the text of sql?
>>>>
>>>> *var* sales_order_sql_stmt =* s*"""SELECT ORDER_NUMBER ,
>>>> INVENTORY_ITEM_ID, ORGANIZATION_ID,
>>>>
>>>>           from_unixtime(unix_timestamp(SCHEDULE_SHIP_DATE,'yyyy-MM-dd'),
>>>> 'yyyy-MM-dd') AS schedule_date
>>>>
>>>>       FROM sales_order_demand
>>>>
>>>>       WHERE unix_timestamp(SCHEDULE_SHIP_DATE,'yyyy-MM-dd') >= $
>>>> planning_start_date  limit 10"""
>>>>
>>>> Dr Mich Talebzadeh
>>>>
>>>>
>>>>
>>>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>>>
>>>>
>>>>
>>>> http://talebzadehmich.wordpress.com
>>>>
>>>>
>>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>>> any loss, damage or destruction of data or any other property which may
>>>> arise from relying on this email's technical content is explicitly
>>>> disclaimed. The author will in no case be liable for any monetary damages
>>>> arising from such loss, damage or destruction.
>>>>
>>>>
>>>>
>>>> On 23 August 2016 at 07:31, Deepak Sharma <deepakmca05@gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> On Tue, Aug 23, 2016 at 10:32 AM, Deepak Sharma <deepakmca05@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> *val* *df** = **sales_demand**.**join**(**product_master**,*
>>>>>> *sales_demand**.$"INVENTORY_ITEM_ID" =**== **product_master*
>>>>>> *.$"INVENTORY_ITEM_ID",**"inner"**)*
>>>>>
>>>>>
>>>>> Ignore the last statement.
>>>>> It should look something like this:
>>>>> *val* *df** = **sales_demand**.**join**(**product_master**,$"*
>>>>> *sales_demand**.INVENTORY_ITEM_ID" =**== $"**product_master*
>>>>> *.INVENTORY_ITEM_ID",**"inner"**)*
>>>>>
>>>>>
>>>>> --
>>>>> Thanks
>>>>> Deepak
>>>>> www.bigdatabig.com
>>>>> www.keosha.net
>>>>>
>>>>
>>>>
>>
>

Mime
View raw message