nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Payne <>
Subject Re: PutSql Loading is slow with Support Fragmented Transactions as True...
Date Fri, 17 Aug 2018 19:16:07 GMT

That property will basically determine whether pushing the data is transactional or not.
For example, if you want to ensure that all data from your CSV is pushed or none of it is,
then you'll want that to be true. If you're okay with pushing the data in a bit at a time,
you can set it to false.

Now, that said, the flow that you're using to convert CSV to JSON, then JSON to SQL, then
put SQL is fairly laborious and slow. Instead, I would recommend replacing all 3 ConvertRecord,
ConvertJSONToSql, and PutSQL Processor with a single PutDatabaseRecord processor. Then
you can put the CSV data directly to the database without the need to convert to JSON or SQL,
and this should be faster.


> On Aug 17, 2018, at 3:02 PM, rey26 <> wrote:
> I have a flow where i am putting data into Teradata database from csv files.I
> am reading with csv and using convert record processor with source as
> CSVReader and Target as JSON Reader and than JSONtoSql and then Putsql, but
> loading is quite slow.
> Since in the csv each file , each line is independent of each other , i
> though to mark Support Fragmented Transactions as False and it improved  the
> load performance.
> Can someone explains why its default value is True ?
> Does it will have any impact on data?
> Whats the exact use of this parameter? 
> --
> Sent from:

View raw message