spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Som Lima <somplastic...@gmail.com>
Subject Re: Converting a date to milliseconds with time zone in Scala Eclipse IDE
Date Wed, 29 Apr 2020 08:47:29 GMT
Also you may be surprised  to learn that I started programming in scala
just yesterday. I was really please I had a challenge to solve rather than
copying example programmes which can be boring.

Judging from answers received I think some may find this information useful.

 I used a scala specific IDE  I got from http://scala-ide.org.

If you do use it there is a bug I fixed to make it work.
I added -vm to eclipse.ini file . The BUG  is:-
 then on the  NEXT line you put the path to jdk8.
Other jdk versions  can also causes other errors.

Eclipse.ini

-startup
plugins/org.eclipse.equinox.launcher_1.4.0.v20161219-1356.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.500.v20170531-1133
-Xmx256m
-Xms200m

-XX:MaxPermSize=384m
-vm
/path/to/java/jdk8u242-b08/bin


On Tue, 28 Apr 2020, 22:22 Mich Talebzadeh, <mich.talebzadeh@gmail.com>
wrote:

> Hi,
>
> Thank you all,
>
> I am just thinking of passing that date   06/04/2020 12:03:43  and
> getting the correct format from the module. In effect
>
> This date format  yyyy-MM-dd'T'HH:mm:ss.SSSSSSSSSZ as pattern
>
> in other words rather than new Date()  pass "06/04/2020 12:03:43" as string
>
> REgards,
>
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Tue, 28 Apr 2020 at 21:31, Som Lima <somplasticllc@gmail.com> wrote:
>
>> import java.time._
>> import java.util.Date
>> import java.text.SimpleDateFormat
>> import java.util.Locale
>> import java.util.SimpleTimeZone
>>
>> object CalendarDemo extends App {
>>
>>     println("Calendar Demo")
>>      val pattern = "EEEEE dd MMMMM yyyy HH:mm:ss.SSSZ";
>>     val simpleDateFormat = (new SimpleDateFormat(pattern, new
>> Locale("en", "UK")));
>>     val date = simpleDateFormat.format(new Date());
>>     System.out.println(date);
>>
>>     val pattern2 = "dd yyyy MM HH:mm:ss.SSSSSSSSSZ";
>>     val simpleDateFormat2 = (new SimpleDateFormat(pattern2, new
>> Locale("en", "UK")));
>>     val date2 = simpleDateFormat2.format(new Date());
>>     System.out.println(date2);
>>
>> /* *
>>     Pattern Syntax
>>
>> You can use the following symbols in your formatting pattern:
>> G Era designator (before christ, after christ)
>> y Year (e.g. 12 or 2012). Use either yy or yyyy.
>> M Month in year. Number of M's determine length of format (e.g. MM, MMM
>> or MMMMM)
>> d Day in month. Number of d's determine length of format (e.g. d or dd)
>> h Hour of day, 1-12 (AM / PM) (normally hh)
>> H Hour of day, 0-23 (normally HH)
>> m Minute in hour, 0-59 (normally mm)
>> s Second in minute, 0-59 (normally ss)
>> S Millisecond in second, 0-999 (normally SSS)
>> E Day in week (e.g Monday, Tuesday etc.)
>> D Day in year (1-366)
>> F Day of week in month (e.g. 1st Thursday of December)
>> w Week in year (1-53)
>> W Week in month (0-5)
>> a AM / PM marker
>> k Hour in day (1-24, unlike HH's 0-23)
>> K Hour in day, AM / PM (0-11)
>> z Time Zone
>> ' Escape for text delimiter
>> ' Single quote
>>     **/
>>
>> }
>>
>>
>> On Tue, 28 Apr 2020, 19:18 Edgardo Szrajber, <szrajber@yahoo.com.invalid>
>> wrote:
>>
>>> Hi
>>> please check combining unix_timestamp and from_unixtime,
>>> Something like:
>>> from_unixtime(unix_timestamp( "06-04-2020 12:03:43"),"yyyy-MM-dd'T'HH:mm:ss
>>> Z")
>>>
>>> please note that I just wrote without any validation.
>>>
>>> In any case, you might want to check the documentation of both functions
>>> to check all valid formats. Also note that this functions are universal
>>> (not only in Spark, Hive) so you have a huge amount of documentation
>>> available.
>>>
>>> Bentzi
>>>
>>>
>>> On Tuesday, April 28, 2020, 08:32:18 PM GMT+3, Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>>
>>> Unfortunately that did not work.
>>>
>>> any other suggestions?
>>>
>>> thanks
>>>
>>> Dr Mich Talebzadeh
>>>
>>>
>>>
>>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>>
>>>
>>>
>>> http://talebzadehmich.wordpress.com
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 28 Apr 2020 at 17:41, Mich Talebzadeh <mich.talebzadeh@gmail.com>
>>> wrote:
>>>
>>> Thanks Neeraj, I'll check it out. !
>>>
>>> Dr Mich Talebzadeh
>>>
>>>
>>>
>>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>>
>>>
>>>
>>> http://talebzadehmich.wordpress.com
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>
>>> On Tue, 28 Apr 2020 at 17:26, neeraj bhadani <
>>> bhadani.neeraj.08@gmail.com> wrote:
>>>
>>> Hi Mich,
>>>     You can try Spark DateTime function here and see if that helps.
>>>
>>>
>>> https://medium.com/expedia-group-tech/deep-dive-into-apache-spark-datetime-functions-b66de737950a
>>>
>>> Regards,
>>> Neeraj
>>>
>>> On Tue, Apr 28, 2020 at 5:15 PM Mich Talebzadeh <
>>> mich.talebzadeh@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> I have a date in format like 06/04/2020 12:03:43 and we want it to be
>>> displayed as follows:
>>>
>>> yyyy-MM-dd'T'HH:mm:ss.SSSSSSSSSZ format
>>>
>>> So the input date is  GMT date time just we do not receive the
>>> information with it
>>>
>>> The output should have timezone information
>>>
>>>
>>> Appreciate any ideas.
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Dr Mich Talebzadeh
>>>
>>>
>>>
>>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>>
>>>
>>>
>>> http://talebzadehmich.wordpress.com
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>>

Mime
View raw message