hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8545) Filesystem Implementation for OpenStack Swift
Date Tue, 02 Jul 2013 12:36:25 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13697728#comment-13697728
] 

Steve Loughran commented on HADOOP-8545:
----------------------------------------

The auth package classes are all designed for Jackson binding and MUST NOT be final, else
you see a stack trace. Test to replicate this is {{TestSwiftRestClient}}
{code}
	at org.apache.hadoop.fs.swift.util.JSONUtil.toObject(JSONUtil.java:81)
	at org.apache.hadoop.fs.swift.http.SwiftRestClient$AuthenticationPost.extractResult(SwiftRestClient.java:1127)
	at org.apache.hadoop.fs.swift.http.SwiftRestClient$AuthenticationPost.extractResult(SwiftRestClient.java:1085)
	at org.apache.hadoop.fs.swift.http.SwiftRestClient.perform(SwiftRestClient.java:1407)
	at org.apache.hadoop.fs.swift.http.SwiftRestClient.authenticate(SwiftRestClient.java:1080)
	at org.apache.hadoop.fs.swift.http.TestSwiftRestClient.testPutAndDelete(TestSwiftRestClient.java:84)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
Caused by: org.codehaus.jackson.map.JsonMappingException: No suitable constructor found for
type [simple type, class org.apache.hadoop.fs.swift.auth.AuthenticationWrapper]: can not instantiate
from JSON object (need to add/enable type information?)
 at [Source: java.io.StringReader@7d0c3a08; line: 1, column: 2]
	at org.codehaus.jackson.map.JsonMappingException.from(JsonMappingException.java:163)
	at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:483)
	at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350)
	at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2402)
	at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1602)
	at org.apache.hadoop.fs.swift.util.JSONUtil.toObject(JSONUtil.java:75)
	... 14 more
{code}

Same for {{SwiftObjectFileStatus}}

{code}
Running org.apache.hadoop.fs.swift.TestSwiftFileSystemBasicOps
Tests run: 16, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 130.317 sec <<<
FAILURE!
testLsRoot(org.apache.hadoop.fs.swift.TestSwiftFileSystemBasicOps)  Time elapsed: 3267 sec
 <<< ERROR!
org.apache.hadoop.fs.swift.exceptions.SwiftJsonMarshallingException: org.codehaus.jackson.map.JsonMappingException:
No suitable constructor found for type [simple type, class org.apache.hadoop.fs.swift.snative.SwiftObjectFileStatus]:
can not instantiate from JSON object (need to add/enable type information?)
 at [Source: java.io.StringReader@6179d854; line: 1, column: 3] source: [{"hash": "d41d8cd98f00b204e9800998ecf8427e",
"last_modified": "2013-05-13T21:05:26.066260", "bytes": 0, "name": "user", "content_type":
"application/octet-stream"}, {"subdir": "user/"}]
	at org.apache.hadoop.fs.swift.util.JSONUtil.toObject(JSONUtil.java:121)
	at org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.listDirectory(SwiftNativeFileSystemStore.java:354)
	at org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystemStore.listSubPaths(SwiftNativeFileSystemStore.java:399)
	at org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem.listStatus(SwiftNativeFileSystem.java:436)
	at org.apache.hadoop.fs.swift.TestSwiftFileSystemBasicOps.testLsRoot(TestSwiftFileSystemBasicOps.java:51)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:62)
Caused by: org.codehaus.jackson.map.JsonMappingException: No suitable constructor found for
type [simple type, class org.apache.hadoop.fs.swift.snative.SwiftObjectFileStatus]: can not
instantiate from JSON object (need to add/enable type information?)
 at [Source: java.io.StringReader@6179d854; line: 1, column: 3]
	at org.codehaus.jackson.map.JsonMappingException.from(JsonMappingException.java:163)
	at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:483)
	at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350)
	at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:120)
	at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:97)
	at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:26)
	at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2402)
	at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1616)
	at org.apache.hadoop.fs.swift.util.JSONUtil.toObject(JSONUtil.java:115)
	... 13 more
{code}

The variable names are fixed to match those in the Keystone and Swift protocols, hence the
use of underscores.

I've reverted those files and pulled in taken the other changes: the switch to {{isDirectory()}},
the javadocs on the exceptions and the various minor formatting patches; added some more javadocs
on the tests. Tested against public OpenStack implementations
                
> Filesystem Implementation for OpenStack Swift
> ---------------------------------------------
>
>                 Key: HADOOP-8545
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8545
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs
>    Affects Versions: 1.2.0, 2.0.3-alpha
>            Reporter: Tim Miller
>            Assignee: Dmitry Mezhensky
>              Labels: hadoop, patch
>         Attachments: HADOOP-8545-026.patch, HADOOP-8545-027.patch, HADOOP-8545-028.patch,
HADOOP-8545-029.patch, HADOOP-8545-030.patch, HADOOP-8545-10.patch, HADOOP-8545-11.patch,
HADOOP-8545-12.patch, HADOOP-8545-13.patch, HADOOP-8545-14.patch, HADOOP-8545-15.patch, HADOOP-8545-16.patch,
HADOOP-8545-17.patch, HADOOP-8545-18.patch, HADOOP-8545-19.patch, HADOOP-8545-1.patch, HADOOP-8545-20.patch,
HADOOP-8545-21.patch, HADOOP-8545-22.patch, HADOOP-8545-23.patch, HADOOP-8545-24.patch, HADOOP-8545-25.patch,
HADOOP-8545-2.patch, HADOOP-8545-3.patch, HADOOP-8545-4.patch, HADOOP-8545-5.patch, HADOOP-8545-6.patch,
HADOOP-8545-7.patch, HADOOP-8545-8.patch, HADOOP-8545-9.patch, HADOOP-8545-javaclouds-2.patch,
HADOOP-8545.patch, HADOOP-8545.patch, HADOOP-8545.suresh.patch
>
>
> ,Add a filesystem implementation for OpenStack Swift object store, similar to the one
which exists today for S3.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message