spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Haoyuan Li <haoyuan...@gmail.com>
Subject Re: StorageLevel: OFF_HEAP
Date Wed, 18 Mar 2015 19:59:01 GMT
Did you recompile it with Tachyon 0.6.0?

Also, Tachyon 0.6.1 has been released this morning:
http://tachyon-project.org/ ; https://github.com/amplab/tachyon/releases

Best regards,

Haoyuan

On Wed, Mar 18, 2015 at 11:48 AM, Ranga <sranga@gmail.com> wrote:

> I just tested with Spark-1.3.0 + Tachyon-0.6.0 and still see the same
> issue. Here are the logs:
> 15/03/18 11:44:11 ERROR : Invalid method name: 'getDataFolder'
> 15/03/18 11:44:11 ERROR : Invalid method name: 'user_getFileId'
> 15/03/18 11:44:11 ERROR storage.TachyonBlockManager: Failed 10 attempts to
> create tachyon dir in
> /tmp_spark_tachyon/spark-e3538a20-5e42-48a4-ad67-4b97aded90e4/<driver>
>
> Thanks for any other pointers.
>
>
> - Ranga
>
>
>
> On Wed, Mar 18, 2015 at 9:53 AM, Ranga <sranga@gmail.com> wrote:
>
>> Thanks for the information. Will rebuild with 0.6.0 till the patch is
>> merged.
>>
>> On Tue, Mar 17, 2015 at 7:24 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>>> Ranga:
>>> Take a look at https://github.com/apache/spark/pull/4867
>>>
>>> Cheers
>>>
>>> On Tue, Mar 17, 2015 at 6:08 PM, fightfate@163.com <fightfate@163.com>
>>> wrote:
>>>
>>>> Hi, Ranga
>>>>
>>>> That's true. Typically a version mis-match issue. Note that spark 1.2.1
>>>> has tachyon built in with version 0.5.0 , I think you may need to rebuild
>>>> spark
>>>> with your current tachyon release.
>>>> We had used tachyon for several of our spark projects in a production
>>>> environment.
>>>>
>>>> Thanks,
>>>> Sun.
>>>>
>>>> ------------------------------
>>>> fightfate@163.com
>>>>
>>>>
>>>> *From:* Ranga <sranga@gmail.com>
>>>> *Date:* 2015-03-18 06:45
>>>> *To:* user@spark.apache.org
>>>> *Subject:* StorageLevel: OFF_HEAP
>>>> Hi
>>>>
>>>> I am trying to use the OFF_HEAP storage level in my Spark (1.2.1)
>>>> cluster. The Tachyon (0.6.0-SNAPSHOT) nodes seem to be up and running.
>>>> However, when I try to persist the RDD, I get the following error:
>>>>
>>>> ERROR [2015-03-16 22:22:54,017] ({Executor task launch worker-0}
>>>> TachyonFS.java[connect]:364)  - Invalid method name:
>>>> 'getUserUnderfsTempFolder'
>>>> ERROR [2015-03-16 22:22:54,050] ({Executor task launch worker-0}
>>>> TachyonFS.java[getFileId]:1020)  - Invalid method name: 'user_getFileId'
>>>>
>>>> Is this because of a version mis-match?
>>>>
>>>> On a different note, I was wondering if Tachyon has been used in a
>>>> production environment by anybody in this group?
>>>>
>>>> Appreciate your help with this.
>>>>
>>>>
>>>> - Ranga
>>>>
>>>>
>>>
>>
>


-- 
Haoyuan Li
AMPLab, EECS, UC Berkeley
http://www.cs.berkeley.edu/~haoyuan/

Mime
View raw message