nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc <phroc...@apache.org>
Subject Re: minifi cpp problem: protocol transmission failed, yielding
Date Mon, 07 May 2018 10:42:48 GMT
Hi Arne,
  Thanks for sending this. What I see in this example is that the file
listing is empty and there is no flow file produced. Per the log
statements below a call to get on the process session was null,
resulting in the RPG yielding. "protocol transmission failed" is a
catch all here. I see toward the end the log GetFile starts to capture
data, so perhaps the log ended a bit early, but in the case where "Get
is null for Minifi" resulting in "protocol transmission failed,
yielding"...this is an  expected and valid warning.


[2018-05-07 08:24:20.675]
[org::apache::nifi::minifi::core::ProcessSession] [debug] Get is null
for Minifi
[2018-05-07 08:24:20.675]
[org::apache::nifi::minifi::core::logging::LoggerConfiguration]
[debug] org::apache::nifi::minifi::c2::C2Agent logger got sinks from
namespace org::apache::nifi::minifi and level debug from namespace
org::apache::nifi::minifi
[2018-05-07 08:24:20.675]
[org::apache::nifi::minifi::RemoteProcessorGroupPort] [warning]
protocol transmission failed, yielding


Though I did notice something when I inspected your config a little closer.

In your connection can you change:

Connections:
    - name: TransferFilesToRPG
      id: 471deef6-2a6e-4a7d-912a-81cc17e3a207
      source name: GetFile
      source relationship name: success
      destination id: 8e7979f9-0161-1000-941e-3be83b4479b0
      max work queue size: 0
      max work queue data size: 1 MB
      flowfile expiration: 60 sec


to

Connections:
    - name: TransferFilesToRPG
      id: 471deef6-2a6e-4a7d-912a-81cc17e3a207
      source name: GetFile
      source relationship name: success
      destination id: 29fd32ba-0163-1000-c751-29aa2e8750c5
      max work queue size: 0
      max work queue data size: 1 MB
      flowfile expiration: 60 sec

I changed the destination of the input port in the RPG. Since the RPG
could be a collection of input and output ports I changed this to
direct the data to the input port.

Let me know if this helps. Thanks,
   Marc

On Mon, May 7, 2018 at 4:17 AM, Arne Oslebo <arne.oslebo@uninett.no> wrote:
> Hello Marc,
>
> here is what I think is the relevant part of the log file.
>
> Thank you for the help,
> Arne
>
> On 04/05/2018 15:48, Marc wrote:
>> Arne,
>>    Great. Would you be able to change your minifi-log.properties to
>> log to debug?  You should be able to change all instances of INFO to
>> DEBUG. It'll be chatty, but feel free to attach the log to the E-mail
>> if you can.  s/INFO/DEBUG/
>>
>> At that level we get more useful information about why the connection
>> failed. My gut reaction is that this likely has something to do with
>> host verification ( I've seen that a number of times myself ) of the
>> server certificate's common name ( or subject alt name ). The logs
>> will help guide us to give you better information.
>>   Thanks!
>>    Marc
>>
>> On Fri, May 4, 2018 at 7:30 AM, Arne Oslebo <arne.oslebo@uninett.no> wrote:
>>>
>>> On 04/05/2018 12:54, Marc Parisi wrote:
>>>
>>> Arne,
>>>   Do you have your certs defined in minifi.properties ? You either need
>>> those properties or an SSL context service.
>>>
>>>
>>> Yes, the certs are defined in minifi.properties:
>>> nifi.remote.input.http.enabled=true
>>> nifi.remote.input.secure=true
>>> nifi.security.need.ClientAuth=yes
>>> nifi.security.client.certificate=/opt/minifi/conf/cert.pem
>>> nifi.security.client.private.key=/opt/minifi/conf/key.pem
>>> nifi.security.client.ca.certificate=/opt/minifi/conf/nifi-cert.pem
>>>
>>> No password on the key.
>>>
>>> I've also tried using an SSL context service but get the same result: minifi
>>> authenticates with the nifi cluster but no data is transmitted. Same warning
>>> in the  minifi-app.log:
>>> [org::apache::nifi::minifi::RemoteProcessorGroupPort] [warning] protocol
>>> transmission failed, yielding
>>>
>>> Best regards,
>>> Arne
>>>
>>>
>>>
>>> On Fri, May 4, 2018, 6:47 AM Arne Oslebo <arne.oslebo@uninett.no> wrote:
>>>> Hello,
>>>>
>>>> I've pulled minifi cpp from master and I'm trying to send data to a
>>>> secure nifi cluster with it. The config.yml is simply a getfile
>>>> connected to a remote processing group:
>>>>
>>>> Flow Controller:
>>>>     name: MiNiFi Flow
>>>>
>>>> Processors:
>>>>     - name: GetFile
>>>>       class: org.apache.nifi.processors.standard.GetFile
>>>>       max concurrent tasks: 1
>>>>       scheduling strategy: TIMER_DRIVEN
>>>>       scheduling period: 1 sec
>>>>       penalization period: 30 sec
>>>>       yield period: 1 sec
>>>>       run duration nanos: 0
>>>>       auto-terminated relationships list:
>>>>       Properties:
>>>>           Input Directory: /tmp/getfile
>>>>           Keep Source File: true
>>>>
>>>> Connections:
>>>>     - name: TransferFilesToRPG
>>>>       id: 471deef6-2a6e-4a7d-912a-81cc17e3a207
>>>>       source name: GetFile
>>>>       source relationship name: success
>>>>       destination id: 8e7979f9-0161-1000-941e-3be83b4479b0
>>>>       max work queue size: 0
>>>>       max work queue data size: 1 MB
>>>>       flowfile expiration: 60 sec
>>>>
>>>> Remote Processing Groups:
>>>>     - name: NiFi Flow
>>>>       id: 8e7979f9-0161-1000-941e-3be83b4479b0
>>>>       url: https://w.x.y.z:8443/nifi
>>>>       timeout: 30 secs
>>>>       yield period: 10 sec
>>>>       Input Ports:
>>>>           - id: 29fd32ba-0163-1000-c751-29aa2e8750c5
>>>>             name: Minifi
>>>>             max concurrent tasks: 1
>>>>             transport protocol: HTTP
>>>>
>>>> Certificate and key is set in minifi.properties and minifi successfully
>>>> authenticates with the cluster:
>>>> 2018-05-04 11:23:12,105 INFO [NiFi Web Server-85]
>>>> o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=minifi,
>>>> OU=NIFI) GET https://w.x.y.z:8443/nifi-api/site-to-site (source ip:
>>>> w.x.y.z)
>>>> 2018-05-04 11:23:12,105 INFO [NiFi Web Server-85]
>>>> o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=minifi,
>>>> OU=NIFI
>>>> 2018-05-04 11:23:12,169 INFO [NiFi Web Server-84]
>>>> o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (CN=minifi,
>>>> OU=NIFI) GET https://w.x.y.z:8443/nifi-api/site-to-site/peers (source
>>>> ip: w.x.y.z)
>>>> 2018-05-04 11:23:12,170 INFO [NiFi Web Server-84]
>>>> o.a.n.w.s.NiFiAuthenticationFilter Authentication success for CN=minifi,
>>>> OU=NIFI
>>>>
>>>> The problem is that no data is sent to the cluster. In the
>>>> minifi-app.log I find a warning that protocol transmission failed:
>>>> [2018-05-04 11:23:21.945]
>>>> [org::apache::nifi::minifi::processors::GetFile] [info] GetFile process
>>>> /tmp/getfile/test
>>>> [2018-05-04 11:23:21.945]
>>>> [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring
>>>> ca111018-4f7c-11e8-88be-000c297ab98c from GetFile to relationship success
>>>> [2018-05-04 11:23:22.196]
>>>> [org::apache::nifi::minifi::RemoteProcessorGroupPort] [warning] protocol
>>>> transmission failed, yielding
>>>>
>>>> I've tried both RAW and HTTP as transport protocol with the same result.
>>>>
>>>> Any idea what might be wrong?
>>>>
>>>> Thanks for any help,
>>>> Arne
>>>
>

Mime
View raw message