hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@cloudera.com.INVALID>
Subject Re: Wire compatibility between Hadoop 3.x client and 2.x server
Date Tue, 20 Oct 2020 15:51:12 GMT
I have this belief that webhdfs:// is better for cross-version
compatibility. That right?

On Fri, 16 Oct 2020 at 19:59, Chao Sun <sunchao@apache.org> wrote:

> Thanks for the replies all!
>
> > But in the opposite case, it might have problem, because 2.x server may
> not understand new client calls added in 3.x
>
> Yes not expecting this to work. I'm thinking about the case where one
> upgrades existing 2.x clients to 3.x and expects it to still work against
> 2.x server, which should not involve those new APIs.
>
> > Another backcompat issue was HDFS-15191, in 3.2.1.
>
> Thanks for pointing this out! In fact this looks like a serious bug in
> 3.2.1. Glad to see it is fixed in 3.2.2.
>
> > But aside from that, we've been using 3x client libraries against both 2x
> and 3x clusters without issue.
>
> Great! thanks.
>
> IMO it will be great if the community maintains official compatibility doc
> w.r.t 2.x/3.x, which can help the migration easier.
>
> Chao
>
> On Tue, Oct 13, 2020 at 11:57 PM Wu,Jianliang(vip.com) <
> jianliang.wu@vipshop.com> wrote:
>
> > Ok, I will file a HDFS jira to report this issue.
> >
> > > 2020年10月13日 20:43,Wei-Chiu Chuang <weichiu@cloudera.com.INVALID>
写道:
> > >
> > > Thanks Jialiang for reporting the issue.
> > > That sounds bad and should've not happened. Could you file a HDFS jira
> > and
> > > fill in more details?
> > >
> > > On Mon, Oct 12, 2020 at 8:59 PM Wu,Jianliang(vip.com) <
> > > jianliang.wu@vipshop.com> wrote:
> > >
> > >> In our case, when nn has upgraded to 3.1.3 and dn’s version was still
> > >> 2.6,  we found hive to call getContentSummary method , the client and
> > >> server was not compatible  because of hadoop3 added new PROVIDED
> storage
> > >> type.
> > >>
> > >> 2020年10月13日 06:41,Chao Sun <sunchao@apache.org<mailto:
> > sunchao@apache.org>>
> > >> 写道:
> > >>
> > >>
> > >>
> > >>
> >
> 本电子邮件可能为保密文件。如果阁下非电子邮件所指定之收件人,谨请立即通知本人。敬请阁下不要使用、保存、复印、打印、散布本电子邮件及其内容,或将其用于其他任何目的或向任何人披露。谢谢您的合作!
> > >> This communication is intended only for the addressee(s) and may
> contain
> > >> information that is privileged and confidential. You are hereby
> notified
> > >> that, if you are not an intended recipient listed above, or an
> > authorized
> > >> employee or agent of an addressee of this communication responsible
> for
> > >> delivering e-mail messages to an intended recipient, any
> dissemination,
> > >> distribution or reproduction of this communication (including any
> > >> attachments hereto) is strictly prohibited. If you have received this
> > >> communication in error, please notify us immediately by a reply e-mail
> > >> addressed to the sender and permanently delete the original e-mail
> > >> communication and any attachments from all storage devices without
> > making
> > >> or otherwise retaining a copy.
> > >>
> >
> >
> 本电子邮件可能为保密文件。如果阁下非电子邮件所指定之收件人,谨请立即通知本人。敬请阁下不要使用、保存、复印、打印、散布本电子邮件及其内容,或将其用于其他任何目的或向任何人披露。谢谢您的合作!
> > This communication is intended only for the addressee(s) and may contain
> > information that is privileged and confidential. You are hereby notified
> > that, if you are not an intended recipient listed above, or an authorized
> > employee or agent of an addressee of this communication responsible for
> > delivering e-mail messages to an intended recipient, any dissemination,
> > distribution or reproduction of this communication (including any
> > attachments hereto) is strictly prohibited. If you have received this
> > communication in error, please notify us immediately by a reply e-mail
> > addressed to the sender and permanently delete the original e-mail
> > communication and any attachments from all storage devices without making
> > or otherwise retaining a copy.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message