nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohit" <>
Subject ConvertCSVToAvro taking a lot of time when passing single record as an input.
Date Mon, 02 Apr 2018 12:49:53 GMT


I'm trying to capture bad records from ConvertCSVToAvro processor. For that,
I'm using two SplitText processors in a row to create chunks and then each
record per flow file.


My flow is  - ListFile -> FetchFile -> SplitText(10000 records) ->
SplitText(1 record) -> ConvertCSVToAvro -> *(futher processing)


I have a 10 MB file with 15 columns per row and 64000 records. Normal flow
(without SplitText) completes in few seconds. But when I'm using the above
flow, ConvertCSVToAvro processor works drastically slow(45-50 rec/sec).

I'm not able to conclude where I'm doing wrong in the flow. 


I'm using Nifi 1.5.0 .


Any quick input would be appreciated.






View raw message