nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nijandan D <nijandha...@gmail.com>
Subject Re: Custom Processor with Hive database connection pooling service not working
Date Fri, 10 Feb 2017 16:48:15 GMT
Hi Bryan,

Thanks for your update. It's working fine. Thank you so much.


Regards,
Nijandan D

On Fri, Feb 10, 2017 at 7:21 PM, Bryan Bende <bbende@gmail.com> wrote:

> Hello,
>
> In general, to link a processor to a controller service, you need two
> dependencies as shown in the wiki page in the previous email:
>
> - The processor project needs a provided dependency on the JAR where
> the service interface is
> - The NAR project needs a NAR dependency on the NAR where the service
> interface is
>
> In the case of Hive, the interface and implementation are both in
> nifi-hive-processors which get packaged into nifi-hive-nar, so you
> need:
>
> In your processors pom.xml:
>
>          <dependency>
>             <groupId>org.apache.nifi</groupId>
>             <artifactId>nifi-hive-processors</artifactId>
>             <version>replace with the version of NiFi you are
> using</version>
>             <scope>provided</scope>
>         </dependency>
>
> In your NAR pom.xml:
>
>        <dependency>
>             <groupId>org.apache.nifi</groupId>
>             <artifactId>nifi-hive-nar</artifactId>
>             <version>replace with the version of NiFi you are
> using</version>
>             <type>nar</type>
>         </dependency>
>
>
> -Bryan
>
> On Fri, Feb 10, 2017 at 2:25 AM, Nijandan D <nijandhan10@gmail.com> wrote:
> > Hi Bryan,
> >
> > I am new to Nifi, I'm tried to create custom processor with hive
> connection
> > pooling service as per your concern. But
> > same issue is occuring in custom processor. It shows only connection id.
> I
> > have no idea about pom file dependency.
> > Here I mentioned the steps what i have followed to create custom
> processor
> > with hive connection below. If it's wrong,
> > Please provide any link to get workflow for creating custom processor
> with
> > hive database connection pooling service.
> >
> > - Added below line in custom processor java class
> >   public static final PropertyDescriptor HIVE_DBCP_SERVICE = new
> > PropertyDescriptor.Builder()
> >             .name("Hive Database Connection Pooling Service")
> >             .description("The Hive Controller Service that is used to
> obtain
> > connection(s) to the Hive database")
> >             .required(true)
> >             .identifiesControllerService(HiveDBCPService.class)
> >             .build();
> >
> > - After imported HiveDBCPService in same java class
> >   import org.apache.nifi.dbcp.hive.HiveDBCPService;
> >
> > - After searched dependency at maven repositories for HiveDBCPService and
> > added
> >
> > - In nifi-custom-nar pom file, added following lines,
> >   <dependency>
> >             <groupId>org.apache.nifi</groupId>
> >             <artifactId>nifi-hive-processors</artifactId>
> >             <version>1.0.0</version>
> >         </dependency>
> >         <dependency>
> >             <groupId>org.apache.nifi</groupId>
> >             <artifactId>nifi-hadoop-libraries-nar</artifactId>
> >             <type>nar</type>
> >         </dependency>
> >
> > - In nifi-custom-processor pom file, added following lines,
> >   <dependency>
> >             <groupId>commons-dbcp</groupId>
> >             <artifactId>commons-dbcp</artifactId>
> >             <version>1.5-SNAPSHOT</version>
> >             <type>jar</type>
> >         </dependency>
> >         <dependency>
> >             <groupId>org.apache.hadoop</groupId>
> >             <artifactId>hadoop-core</artifactId>
> >             <version>1.2.1</version>
> >             <type>jar</type>
> >         </dependency>
> >         <dependency>
> >             <groupId>org.apache.hadoop</groupId>
> >             <artifactId>hadoop-common</artifactId>
> >             <version>3.0.0-SNAPSHOT</version>
> >             <type>jar</type>
> >         </dependency>
> >         <dependency>
> >             <groupId>org.apache.hive</groupId>
> >             <artifactId>hive-jdbc</artifactId>
> >             <version>1.3.0-SNAPSHOT</version>
> >             <type>jar</type>
> >         </dependency>
> >         <dependency>
> >             <groupId>org.apache.nifi</groupId>
> >             <artifactId>nifi-hadoop-utils</artifactId>
> >             <version>1.0.0</version>
> >             <type>jar</type>
> >         </dependency>
> >         <dependency>
> >             <groupId>org.apache.nifi</groupId>
> >             <artifactId>nifi-hive-processors</artifactId>
> >             <version>1.0.0</version>
> >             <type>jar</type>
> >         </dependency>
> >
> > - After taking build and replaced Nar file in lib folder, restarted Nifi
> > source. Same error connection id only showing in
> > connection dropdown box.
> >
> > Thanks in advance.
> >
> > Regards,
> > Nijandan D
> >
> > On Thu, Feb 9, 2017 at 8:22 PM, Bryan Bende <bbende@gmail.com> wrote:
> >>
> >> Hello,
> >>
> >> This usually means your custom NAR is not correctly linked to the NAR
> >> where the service is.
> >>
> >> Take a look at these resources for examples:
> >>
> >>
> >> https://cwiki.apache.org/confluence/display/NIFI/Maven+
> Projects+for+Extensions#MavenProjectsforExtensions-
> LinkingProcessorsandControllerServices
> >>
> >>
> >> https://github.com/bbende/nifi-dependency-example/blob/
> master/nifi-example-processor-bundle/nifi-example-
> processors-nar/pom.xml#L40
> >>
> >> Thanks,
> >>
> >> Bryan
> >>
> >> On Wed, Feb 8, 2017 at 11:49 PM, cool nijandhan <nijandhan10@gmail.com>
> >> wrote:
> >> > Hi
> >> >
> >> > I am developing custom processor with hive database connection pooling
> >> > service. Added necessary dependency in pom file and added necessary
> >> > folders
> >> > in custom processor folder. I have created class file and able to
> >> > generate
> >> > nar file. Placed nar file in lib directory and restarted nifi server.
> >> > Its
> >> > look like everything working fine. But services are not showing in
> >> > database
> >> > connection dropdown box. Instead it showing connection id, but every
> >> > time
> >> > services are creating when click "create" in controller section. For
> >> > other
> >> > processors, it seems working fine. Please find the attached
> screenshots.
> >> >
> >> > Any help appreciated.
> >> >
> >> > Thanks
> >
> >
>

Mime
View raw message