nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sivaprasanna <sivaprasanna...@gmail.com>
Subject Re: How do you use a custom region?
Date Fri, 30 Nov 2018 16:58:55 GMT
I understand where the problem is. The reason why we replaced
*this.client.setEndpoint(urlstr);* with *this.client.setEndpoint(urlstr,
this.client.getServiceName(), this.region.getName()); *was that when
working VPC enabled services, the endpoint URL wasn't properly parsed.
Having said that, *this.region.getName()* shouldn't be there. I'll work on
a solution. Thanks for bringing this up.

-
Sivaprasanna

On Fri, Nov 30, 2018 at 9:49 PM Andrew McDonald <amcdonald@ccri.com> wrote:

> Setting line 289 in AbstractAWSProcessor.java to
>
> this.client.setEndpoint(urlstr);
>
> worked for me.
>
> Regards,Andrew
> On 11/30/18 09:12, Andrew McDonald wrote:
>
> This is on an air-gapped system so I'll type in the essential part
>
> Failed to receive messages from Amazon SQS due to
> com.amazonaws.services.sqs.model.AmazonSQSException: Credential should be
> scoped to a valid region, not 'us-east-1'. (Service: AmazonSQS; Status
> Code: 403; Error Code: SignatureDoesNotMatch; Request ID:
> xxxxx-xxxx-xxxx-xxxx-xxxxxxxx).
>
> I believe that line 289 in AbstractAWSProcessor.java is setting the wrong
> regionId
>
> this.client.setEndpoint(urlstr, this.client.getServiceName(), this.region.getName());
>
> The amazon docs for the third parameter (regionId) state:
>
> * @param regionId*            The ID of the region in which this service resides AND
the*            overriding region for signing purposes.
>
> This would explain the SignatureDoesNotMatch error.
>
> The properties for the sqs processor work for nifi 1.4.0.
>
> Regards, Andrew
>
>
> On 11/29/18 23:22, Sivaprasanna wrote:
>
> Andrew, Can you please share the complete error message and also show us
> how the properties look for the GetSQS processor?
>
> Thanks.
>
> On Fri, Nov 30, 2018 at 3:45 AM Andrew McDonald <amcdonald@ccri.com>
> wrote:
>
>> So I've installed nifi 1.8.0 and endpoint override didn't help
>>
>> The GetSQS is throwing and exception
>>
>> com.amazonaws.services.sqs.model.AmazonSQSException: Credential should be
>> scoped to a valid region, not 'us-east-1'.
>>
>> I'm guessing that the override endpoint is not overriding the region b/c
>> 'us-east-1' is not my region.
>>
>> Regards, Andrew
>> On 11/29/18 02:09, Sivaprasanna wrote:
>>
>> Yeah. This was added in 1.8.0 for SQS. However, the reason why a custom
>> enum was added on the NiFi side[1] was to have a proper readable region
>> instead of just the region code i.e., Asia Pacific (Singapore) instead
>> of ap-southeast-1. However, I raised a request later to the AWS Java SDK
>> team to have a readable name. It was added recently, if I remember
>> correctly. So ideally this enum has to be removed and the one on the
>> official AWS SDK has to be leveraged completely. I have created a Jira[2]
>> and started working on it. I'll raise a PR soon.
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-5129
>> [2] https://issues.apache.org/jira/browse/NIFI-5850
>>
>> Thanks,
>>
>> On Thu, Nov 29, 2018 at 2:20 AM Andrew McDonald <amcdonald@ccri.com>
>> wrote:
>>
>>> zenfenan  added this later to 1.8.0, yay!
>>> On 11/28/18 14:23, Andrew McDonald wrote:
>>>
>>>
>>> This workaround doesn't work for sqs because it doesn't have endpoint
>>> override URL property
>>>
>>>
>>> On 11/28/18 12:16, Michael Moser wrote:
>>>
>>> Greetings!  This JIRA ticket [1] describes the recommended work around
>>> for AWS regions that aren't in the list.
>>>
>>> -- Mike
>>>
>>> [1] - https://issues.apache.org/jira/browse/NIFI-4523
>>>
>>>
>>>
>>> On Wed, Nov 28, 2018 at 11:44 AM Jon Logan <jmlogan@buffalo.edu> wrote:
>>>
>>>> Andrew,
>>>>
>>>> I know there's a few regions not in the list. I'm not sure which region
>>>> you're targeting, but at least for the case of one of the new regions, I
>>>> submitted a PR for this. I haven't dug into it deeply, but it seems like
a
>>>> better way to do this might be to remove the enum entirely and get the
>>>> region list via the AWS API, or allow a free-form entry.
>>>>
>>>> https://github.com/apache/nifi/pull/3187
>>>>
>>>>
>>>> Jon
>>>>
>>>> On Wed, Nov 28, 2018 at 10:35 AM Andrew McDonald <amcdonald@ccri.com>
>>>> wrote:
>>>>
>>>>> I'm trying to upgrade from 1.4.0 to 1.7.1 but the s3 processors can
>>>>> not
>>>>> be initialized.
>>>>>
>>>>> Nifi 1.7.1 uses 1.11.319 and is throwing an IllegalArgumentException:
>>>>> no
>>>>> region provided
>>>>>
>>>>> The region I'm using isn't in the enum, so is it possible to use a
>>>>> custom region?
>>>>>
>>>>> Regards,Andrew
>>>>>
>>>>>
>>>>>

Mime
View raw message