spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gourav Sengupta <gourav.sengu...@gmail.com>
Subject Re: Announcing Delta Lake 0.2.0
Date Thu, 20 Jun 2019 06:08:33 GMT
Hi Ayan,

Delta is obviously well thought through, its been available in Databricks
since last year and a half now I think and besides that it is from some of
the best minds at work :)

But what may not be well tested in Delta is its availability as a storage
class for HIVE.

How about your testing? Are you doing it in S3? What is the kind of volume
you are testing it with if I may ask.


Regards,
Gourav Sengupta

On Thu, Jun 20, 2019 at 12:58 AM ayan guha <guha.ayan@gmail.com> wrote:

> Hi
>
> We are using Delta features. The only problem we faced till now is Hive
> can not read DELTA outputs by itself (even if the Hive metastore is
> shared). However, if we create hive external table pointing to the folder
> (and with Vacuum), it can read the data.
>
> Other than that, the feature looks good and well thought out. We are doing
> a volume testing now....
>
> Best
> Ayan
>
> On Thu, Jun 20, 2019 at 9:52 AM Liwen Sun <liwen.sun@databricks.com>
> wrote:
>
>> Hi Gourav,
>>
>> Thanks for the suggestion. Please open a Github issue at
>> https://github.com/delta-io/delta/issues to describe your use case and
>> requirements for "external tables" so we can better track this feature and
>> also get feedback from the community.
>>
>> Regards,
>> Liwen
>>
>> On Wed, Jun 19, 2019 at 12:11 PM Gourav Sengupta <
>> gourav.sengupta@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> does Delta support external tables? I think that most users will be
>>> needing this.
>>>
>>>
>>> Regards,
>>> Gourav
>>>
>>> On Wed, Jun 19, 2019 at 8:04 PM Liwen Sun <liwen.sun@databricks.com>
>>> wrote:
>>>
>>>> We are delighted to announce the availability of Delta Lake 0.2.0!
>>>>
>>>> To try out Delta Lake 0.2.0, please follow the Delta Lake Quickstart:
>>>> https://docs.delta.io/0.2.0/quick-start.html
>>>>
>>>> To view the release notes:
>>>> https://github.com/delta-io/delta/releases/tag/v0.2.0
>>>>
>>>> This release introduces two main features:
>>>>
>>>> *Cloud storage support*
>>>> In addition to HDFS, you can now configure Delta Lake to read and write
>>>> data on cloud storage services such as Amazon S3 and Azure Blob Storage.
>>>> For configuration instructions, please see:
>>>> https://docs.delta.io/0.2.0/delta-storage.html
>>>>
>>>> *Improved concurrency*
>>>> Delta Lake now allows concurrent append-only writes while still
>>>> ensuring serializability. For concurrency control in Delta Lake, please
>>>> see: https://docs.delta.io/0.2.0/delta-concurrency.html
>>>>
>>>> We have also greatly expanded the test coverage as part of this release.
>>>>
>>>> We would like to acknowledge all community members for contributing to
>>>> this release.
>>>>
>>>> Best regards,
>>>> Liwen Sun
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Delta Lake Users and Developers" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to delta-users+unsubscribe@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/delta-users/CAE4dWq9g90NkUr_SLs2J6kFPbOpxx4wy6MEgb%3DQ5pBxkUcK%2B-A%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/delta-users/CAE4dWq9g90NkUr_SLs2J6kFPbOpxx4wy6MEgb%3DQ5pBxkUcK%2B-A%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>
>
> --
> Best Regards,
> Ayan Guha
>

Mime
View raw message