sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gwen Shapira <gshap...@cloudera.com>
Subject Re: Oracle to Hadoop data ingestion in Real-time
Date Fri, 26 Dec 2014 20:48:23 GMT
Just note that while Tungsten / Goldengate read changes off the redo log,
and therefore have minimal impact on performance of the source system,
LinkedIn Databus adds triggers and indexes, and therefore will impact
insert performance on the source tables.


On Fri, Dec 26, 2014 at 12:31 PM, burakkk <burak.isikli@gmail.com> wrote:

> Databus can also be used. Databus is designed to watch a database, view
> and process updates in near real-time. You can find more details about
> databus on this link:
> https://github.com/linkedin/databus
>
> Best regards...
>
>
> On Mon, Dec 15, 2014 at 2:41 AM, Gwen Shapira <gshapira@cloudera.com>
> wrote:
>
>> AFAIK Tungsten is open source and is the only open source CDC solution
>> that I know of.
>>
>> On Sun, Dec 14, 2014 at 4:20 PM, Venkat, Ankam
>> <Ankam.Venkat@centurylink.com> wrote:
>> > Thanks Gwen.
>> >
>> > I am looking for open source solutions only.
>> >
>> > Is Tungsten open source tool?  Any other open source tools?
>> >
>> > Regards,
>> > Venkat Ankam
>> >
>> > -----Original Message-----
>> > From: Gwen Shapira [mailto:gshapira@cloudera.com]
>> > Sent: Sunday, December 14, 2014 11:38 AM
>> > To: user@sqoop.apache.org
>> > Subject: Re: Oracle to Hadoop data ingestion in Real-time
>> >
>> > GoldenGate can be used with flume to ingest data from Oracle to Hadoop
>> in real time. Oracle's MetaLink (i.e. MyOracleSupport) will have code
>> examples on how to do that.
>> >
>> > I heard from booking.com that Tungsten works great for MySQL to Hadoop
>> in real time. So its worth trying for Oracle.
>> >
>> > Another tool to try is DBVisit Replication, since works on Oracle SE.
>> > Their website may not mention Hadoop, but I believe they have the
>> capability, so worth contacting them.
>> >
>> > Gwen
>> >
>> > On Sun, Dec 14, 2014 at 6:34 AM, Venkat, Ankam <
>> Ankam.Venkat@centurylink.com> wrote:
>> >> I have a requirement to ingest the data from oracle database to hadoop
>> >> in real-time.
>> >>
>> >> I know Sqoop is not the answer.  Anybody tried Tungsten replicator to
>> >> ingest data from Oracle to Hadoop in real-time?
>> >>
>> >> Are there any other tools available?
>> >>
>> >> Any help is highly appreciated.
>> >>
>> >> Regards,
>> >>
>> >> Venkat Ankam
>> >>
>> >>
>> >>
>> >> This communication is the property of CenturyLink and may contain
>> >> confidential or privileged information. Unauthorized use of this
>> >> communication is strictly prohibited and may be unlawful. If you have
>> >> received this communication in error, please immediately notify the
>> >> sender by reply e-mail and destroy all copies of the communication and
>> >> any attachments.
>> > This communication is the property of CenturyLink and may contain
>> confidential or privileged information. Unauthorized use of this
>> communication is strictly prohibited and may be unlawful. If you have
>> received this communication in error, please immediately notify the sender
>> by reply e-mail and destroy all copies of the communication and any
>> attachments.
>>
>
>
>
> --
>
> *BURAK ISIKLI* | *http://burakisikli.wordpress.com
> <http://burakisikli.wordpress.com>*
>
>

Mime
View raw message