hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chao Sun <sunc...@apache.org>
Subject Re: Wire compatibility between Hadoop 3.x client and 2.x server
Date Tue, 20 Oct 2020 16:44:00 GMT
Steve, yes that is my understanding as well, although from my experience
ppl usually prefer RPC in prod for better performance.

On Tue, Oct 20, 2020 at 8:51 AM Steve Loughran <stevel@cloudera.com> wrote:

> I have this belief that webhdfs:// is better for cross-version
> compatibility. That right?
>
> On Fri, 16 Oct 2020 at 19:59, Chao Sun <sunchao@apache.org> wrote:
>
>> Thanks for the replies all!
>>
>> > But in the opposite case, it might have problem, because 2.x server may
>> not understand new client calls added in 3.x
>>
>> Yes not expecting this to work. I'm thinking about the case where one
>> upgrades existing 2.x clients to 3.x and expects it to still work against
>> 2.x server, which should not involve those new APIs.
>>
>> > Another backcompat issue was HDFS-15191, in 3.2.1.
>>
>> Thanks for pointing this out! In fact this looks like a serious bug in
>> 3.2.1. Glad to see it is fixed in 3.2.2.
>>
>> > But aside from that, we've been using 3x client libraries against both
>> 2x
>> and 3x clusters without issue.
>>
>> Great! thanks.
>>
>> IMO it will be great if the community maintains official compatibility doc
>> w.r.t 2.x/3.x, which can help the migration easier.
>>
>> Chao
>>
>> On Tue, Oct 13, 2020 at 11:57 PM Wu,Jianliang(vip.com) <
>> jianliang.wu@vipshop.com> wrote:
>>
>> > Ok, I will file a HDFS jira to report this issue.
>> >
>> > > 2020年10月13日 20:43,Wei-Chiu Chuang <weichiu@cloudera.com.INVALID>
写道:
>> > >
>> > > Thanks Jialiang for reporting the issue.
>> > > That sounds bad and should've not happened. Could you file a HDFS jira
>> > and
>> > > fill in more details?
>> > >
>> > > On Mon, Oct 12, 2020 at 8:59 PM Wu,Jianliang(vip.com) <
>> > > jianliang.wu@vipshop.com> wrote:
>> > >
>> > >> In our case, when nn has upgraded to 3.1.3 and dn’s version was still
>> > >> 2.6,  we found hive to call getContentSummary method , the client and
>> > >> server was not compatible  because of hadoop3 added new PROVIDED
>> storage
>> > >> type.
>> > >>
>> > >> 2020年10月13日 06:41,Chao Sun <sunchao@apache.org<mailto:
>> > sunchao@apache.org>>
>> > >> 写道:
>> > >>
>> > >>
>> > >>
>> > >>
>> >
>> 本电子邮件可能为保密文件。如果阁下非电子邮件所指定之收件人,谨请立即通知本人。敬请阁下不要使用、保存、复印、打印、散布本电子邮件及其内容,或将其用于其他任何目的或向任何人披露。谢谢您的合作!
>> > >> This communication is intended only for the addressee(s) and may
>> contain
>> > >> information that is privileged and confidential. You are hereby
>> notified
>> > >> that, if you are not an intended recipient listed above, or an
>> > authorized
>> > >> employee or agent of an addressee of this communication responsible
>> for
>> > >> delivering e-mail messages to an intended recipient, any
>> dissemination,
>> > >> distribution or reproduction of this communication (including any
>> > >> attachments hereto) is strictly prohibited. If you have received this
>> > >> communication in error, please notify us immediately by a reply
>> e-mail
>> > >> addressed to the sender and permanently delete the original e-mail
>> > >> communication and any attachments from all storage devices without
>> > making
>> > >> or otherwise retaining a copy.
>> > >>
>> >
>> >
>> 本电子邮件可能为保密文件。如果阁下非电子邮件所指定之收件人,谨请立即通知本人。敬请阁下不要使用、保存、复印、打印、散布本电子邮件及其内容,或将其用于其他任何目的或向任何人披露。谢谢您的合作!
>> > This communication is intended only for the addressee(s) and may contain
>> > information that is privileged and confidential. You are hereby notified
>> > that, if you are not an intended recipient listed above, or an
>> authorized
>> > employee or agent of an addressee of this communication responsible for
>> > delivering e-mail messages to an intended recipient, any dissemination,
>> > distribution or reproduction of this communication (including any
>> > attachments hereto) is strictly prohibited. If you have received this
>> > communication in error, please notify us immediately by a reply e-mail
>> > addressed to the sender and permanently delete the original e-mail
>> > communication and any attachments from all storage devices without
>> making
>> > or otherwise retaining a copy.
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message