sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahmed El Baz" <ahel...@microsoft.com>
Subject Re: Review Request: Fix Sqoop unit test failures on Windows
Date Tue, 09 Apr 2013 21:38:48 GMT


> On April 5, 2013, 4:21 p.m., Venkat Ranganathan wrote:
> > src/java/org/apache/sqoop/util/ClassLoaderStack.java, line 78
> > <https://reviews.apache.org/r/10229/diff/1/?file=276974#file276974line78>
> >
> >     Why is the JarURL format getting changed.   File(jarFile).toURI().toURL() will
still have the scheme as file right?
> 
> Ahmed El Baz wrote:
>     The objective is to construct the URL path using URL class which resolves the failures
on Windows due to the preceding "jar:" and ending "!/" parts. I believe this is also favorable
than handcrafting the URL string.
>     
>     Verified this is working on both Windows and Linux
> 
> Venkat Ranganathan wrote:
>     Are you saying this does not work on Windows?
>     
>     String urlPath = "jar:file:" + new File(jarFile).getAbsolutePath() + "!/";
> 
> Ahmed El Baz wrote:
>     Correct. Using the crafted String urlPath is failing on Windows, thus the need to
use the URL class.
> 
> Venkat Ranganathan wrote:
>     That is interesting. remember using it in my earlier work.  I understand that you
want to get it consistent across Unix and windows.   Can you please see the content below
and try it on Windows and Linux? I don't think the issue is because of jar: and !/ suffixes.
  The issue is because file urls crafted with file:// syntax expect the file part to start
with / and not a drive letter.   That is why using file: (without // after file) helps.  In
any case, URLClassLoader can work with Jar URLs without the prefix if we want the full jarfile
contents.   I think there is a windows issue with dynamically replacing jarfiles because of
a Java bug.   That is a different issue.
>     
>     ============
>     import java.io.File;
>     import java.net.JarURLConnection;
>     import java.net.URL;
>     import java.util.Map;
>     import java.util.jar.Attributes;
>     import java.util.jar.Manifest;
>     
>     
>     
>     public class JarURLConn {
>     	
>     	public static void main(String args[]) {
>     		if (args.length < 1) {
>     			System.out.println("Usage JavaURLConn jar1 ...");
>     			System.exit(1);
>     		}
>     		for (int i = 0; i < args.length; ++i) {
>     			System.out.println("Information on jar file " + args[i]);
>     			System.out.println("\n\n");
>     			try {
>     				printJarInfo(args[i]);
>     			}
>     			catch (Exception e) {
>     				System.out.println("Caught exception : " + e.toString());
>     			}
>     		}
>     	}
>     	
>     
>     	private static void printJarInfo(String jarFile) throws Exception {
>     		File f = new File(jarFile);
>     		if (f.canRead()) {
>     			StringBuilder sb = new StringBuilder();
>     			sb.append("jar:file:");
>     			sb.append(f.getAbsolutePath());
>     			sb.append("!/");
>     			String jarURLStr = sb.toString();
>                             System.out.println("Jar URL is " + jarURLStr);
>     			URL url = new URL(jarURLStr);
>     			JarURLConnection conn = (JarURLConnection) url.openConnection();
>     			Manifest manifest = conn.getManifest();
>     			Map<String, Attributes> attrMap = manifest.getEntries();
>     			
>     			for (String key : attrMap.keySet()) {
>     				Attributes attrs = attrMap.get(key);
>     				for (Object attrKey : attrs.keySet()) {
>     					System.out.println("Attribute " + attrKey + " = "
>     							+ attrs.get(attrKey));
>     				}
>     			}
>     			 
>     		}
>     		else {
>     			System.out.println("Can't access jar file " + jarFile);
>     		}
>     		
>     	}
>     }
>     =======

Thanks Venkat,
I have tried the sample above, and yes the error happens when we add the extra "://". Are
you suggesting to change the fix to remove the slashes from the file:// part. I believe using
URL APIs would abstract all those issues. Please let me know what you think


> On April 5, 2013, 4:21 p.m., Venkat Ranganathan wrote:
> > src/test/com/cloudera/sqoop/orm/TestClassWriter.java, line 177
> > <https://reviews.apache.org/r/10229/diff/1/?file=276979#file276979line177>
> >
> >     Nit - tabs instead of spaces
> 
> Venkat Ranganathan wrote:
>     Please check as mentioned above

Will do
Thanks


> On April 5, 2013, 4:21 p.m., Venkat Ranganathan wrote:
> > src/test/com/cloudera/sqoop/hive/TestHiveImport.java, line 192
> > <https://reviews.apache.org/r/10229/diff/1/?file=276975#file276975line192>
> >
> >     Nit: Trailing spaces
> 
> Ahmed El Baz wrote:
>     Did not see a trailing space in this line, but fixed a trailing space in the next
one.
>     Thanks
> 
> Venkat Ranganathan wrote:
>     Can you please make sure after a upload your diff to check if there are no red bars
in the beginning or end.   That will signify the space issues.

Will doThanks


- Ahmed


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10229/#review18705
-----------------------------------------------------------


On April 1, 2013, 8:48 p.m., Ahmed El Baz wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10229/
> -----------------------------------------------------------
> 
> (Updated April 1, 2013, 8:48 p.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Description
> -------
> 
> Please see SQOOP-955 for details
> 
> 
> This addresses bug SQOOP-955.
>     https://issues.apache.org/jira/browse/SQOOP-955
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/hive/HiveImport.java ce34286 
>   src/java/org/apache/sqoop/orm/CompilationManager.java 92effb5 
>   src/java/org/apache/sqoop/util/ClassLoaderStack.java 8e41cb3 
>   src/test/com/cloudera/sqoop/hive/TestHiveImport.java 170bc66 
>   src/test/com/cloudera/sqoop/io/TestNamedFifo.java 9f0a585 
>   src/test/com/cloudera/sqoop/lib/TestBlobRef.java 2054a9b 
>   src/test/com/cloudera/sqoop/lib/TestClobRef.java 8cb05e9 
>   src/test/com/cloudera/sqoop/orm/TestClassWriter.java 3b77571 
>   testdata/hive/bin/hive.cmd PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/10229/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Ahmed El Baz
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message