spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shyam P <shyamabigd...@gmail.com>
Subject Re: Is there any spark API function to handle a group of companies at once in this scenario?
Date Mon, 08 Apr 2019 07:18:58 GMT
Hi Mich,
 thanks for your prompt reply.
I get few company financial data like profits and etc results .
I would get this company data through Kafka topics which is fed by an rest
service.
I am thinking of using spark-structured streaming.
Put them back in HIVE/C*.

Regards,
Shyam



On Sun, Apr 7, 2019 at 2:04 PM Mich Talebzadeh <mich.talebzadeh@gmail.com>
wrote:

> Are these ticker prices for these companies like share value etc?
>
> How do you get this company data in Spark? Are you using Spark streaming
> to get the prices, then work out the stats (AVG, STDDEV etc) and put them
> back into DB?
>
> HTH
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Fri, 5 Apr 2019 at 10:51, Shyam P <shyamabigdata@gmail.com> wrote:
>
>> Hi ,
>> In my scenario I have few companies , for which I need to calculate few
>> stats like avg I need to be stored in Cassandra , for next set of records I
>> need to get previously calculated and over it i need to calculate
>> accumulated results ( i.e preset set of data + previously stored stats) and
>> stored it back to Cassandra.
>>
>> what function/API of spark be used while calculating the above for a
>> group of companies?
>>
>>
>> Regards,
>> Shyam
>>
>

Mime
View raw message