nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jorge Machado <>
Subject Re: Stackoverflow question: Moving data from one RDB to another through NiFi
Date Thu, 22 Mar 2018 07:18:03 GMT
Sqoop does not import into a mySql database. Just into Hive if you tell him to do so. 
You could use Nifi but if you have a lot of data may be you should try Spark.  which reads
and writes in Parallel. 
Using Nifi would work to but you have the overhead of pumping the data over “insert” unless
you copy the files into the server and on the server then use some import bulk…..

Jorge Machado

> On 22 Mar 2018, at 08:13, Brett Ryan <> wrote:
> Could Sqoop [1] be an option?
>  [1]:
>> On 22 Mar 2018, at 16:33, Sivaprasanna <> wrote:
>> I had a chance to attempt a question raised on stackoverflow regarding
>> moving data from SQL Server to MySQL using NiFi. The user is using
>> GenerateTableFetch to read data from SQL Server and then try to use LOAD
>> DATA command in ExecuteSQL but this involves writing the read SQL Server
>> data to filesystem and then load it, which is a performance hit, I
>> suggested the user to try PutDatabaseRecord but I have never tried the
>> approach myself and going by the docs, I think it won't show any
>> performance benefit than LOAD DATA because the former reads from file and
>> inserts at a high speed while the latter reads content and parses it
>> according to the configured Record Reader and insert the rows as a single
>> batch. Confused, I wanted to get the community's opinion/thoughts on this.
>> Please attempt the questions, if you have better suggestions.
>> Links:
>>  -
>>  -
>> Thanks,
>> Sivaprasanna

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message