spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From swetha kadiyala <swethakadiyal...@gmail.com>
Subject Read text file row by row and apply conditions
Date Mon, 30 Sep 2019 03:26:30 GMT
dear friends,

I am new to spark. can you please help me to read the below text file using
spark and scala.

Sample data

bret|lee|A|12345|ae545|gfddfg|86786786
142343345|aaaa|D|ae342
67567|67777|U|aadfsd|34k4|84304|020|sdnfsdfn|3243|benej|32432|jsfsdf|3423
67564|67747|U|aad434|3435|843454|203|sdn454dfn|3233|gdfg|34325|sdfsddf|7657


I am receiving indicator type with 3 rd column of each row. if my indicator
type=A, then i need to store that particular row data into a table called
Table1.
if indicator type=D then i have to store data into seperate table called
TableB and same as indicator type=U then i have to store all rows data into
a separate table called Table3.

Can anyone help me how to read row by row and split the columns and apply
the condition based on indicator type and store columns data into
respective tables.

Thanks,
Swetha

Mime
View raw message