sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Cecho" <jar...@apache.org>
Subject Re: Review Request 40625: SQOOP-2709 Sqoop2: HDFS: Impersonation on secured cluster doesn't work
Date Mon, 30 Nov 2015 20:38:09 GMT


> On Nov. 25, 2015, 3:33 a.m., Dian Fu wrote:
> >
> 
> Dian Fu wrote:
>     For the integration test, I'm currently working on SQOOP-2704 to enable kerberos
in the integration tests. I will add tests for this scenario under that JIRA.

You're awesome, thank you Dian!


> On Nov. 25, 2015, 3:33 a.m., Dian Fu wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java,
line 82
> > <https://reviews.apache.org/r/40625/diff/1/?file=1137847#file1137847line82>
> >
> >     The implementation of "SecurityUtils.generateDelegationTokens()" overrides the
field "HdfsConstants.DELEGATION_TOKENS" each time it's called. I think it should not override
the old ones as both "HdfsFromInitializer" and "HdfsToInitializer" call "SecurityUtils.generateDelegationTokens()"
to generate delegation tokens for HDFS paths. These paths may be different.

Good thinking.

We're strictly separating Contexts for each From/To portion, so even in case when someone
will be doing HDFS to HDFS transfer, each Initializator (From/To) will be working with it's
own Context instance.


> On Nov. 25, 2015, 3:33 a.m., Dian Fu wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/security/SecurityUtils.java,
line 121
> > <https://reviews.apache.org/r/40625/diff/1/?file=1137848#file1137848line121>
> >
> >     baos.flush() is not needed.

Good point.


- Jarek


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40625/#review107910
-----------------------------------------------------------


On Nov. 24, 2015, 12:58 a.m., Jarek Cecho wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/40625/
> -----------------------------------------------------------
> 
> (Updated Nov. 24, 2015, 12:58 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-2709
>     https://issues.apache.org/jira/browse/SQOOP-2709
> 
> 
> Repository: sqoop-sqoop2
> 
> 
> Description
> -------
> 
> I've provided util class that can retrieve delegation token for "current" user and store
it in our Context that is passed to execution engine.
> 
> 
> Diffs
> -----
> 
>   connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConstants.java
39ee4a3 
>   connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
583acdd 
>   connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
be837ca 
>   connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
04acd18 
>   connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
998b903 
>   connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
2bad23a 
>   connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
5856371 
>   connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/security/SecurityUtils.java
PRE-CREATION 
>   connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/security/TestSecurityUtils.java
PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/40625/diff/
> 
> 
> Testing
> -------
> 
> I've tested the patch on secured real cluster to make sure that it's working. Sadly I
did not included any integration test as our suite currently doesn't have any support for
MiniKDC (this is something that we will add later).
> 
> 
> Thanks,
> 
> Jarek Cecho
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message