kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sachin Mittal <sjmit...@gmail.com>
Subject Re: Deleting topics in windows - fix estimate or workaround
Date Mon, 02 Dec 2019 05:17:35 GMT
Hi Edward,
I hear you.
So basically we are saying that unless major sponsors of Kafka project get
actual requirement for support on windows this issue will linger on.

Anyway my couple of takes from this issue is:

1. One should not overlook the potential opportunity for windows based
systems.
There may not be much incentive now, but if the software runs seamlessly on
windows, chances for such an opportunity increases.

2. There have been pull requests issued for fix of these issues, question
is how can we prioritize to get them to be merged in order to have a stable
windows release.

Thanks
Sachin




On Mon, Dec 2, 2019 at 5:34 AM Edward Capriolo <edlinuxguru@gmail.com>
wrote:

> My development environment is windows. I am actually able to run
> integration tests launching Kafka and Zookeeper. However I have noticed
> that the test warn likely about deleting a file that is locked and I tried
> deleting a topic at one point and saw some of  the issues you mentioned. I
> personally think it is a blessing that it works at all, (because my
> environment does not support docker either so I would really be up the
> creek), but because it seems spotty in terms of features I do not count on
> it 100%.
>
> Because Java is supposedly 100% pure it seems like everything SHOULD work.
>
> I pessimistic hot take: Generally what (windows support) takes is some
> large organization (say the us government, or some bank) pays the largest
> consulting group/support company for a contract and demand windows support.
> Then magically it happens :)
>
> On Sun, Dec 1, 2019 at 12:47 PM Sachin Mittal <sjmittal@gmail.com> wrote:
>
> > Hi,
> > Firstly development environment is windows and I have been able to run
> > almost any big data framework on windows (or cygwin) be it HBase,
> Cassandra
> > or anything else.
> > Now just for running Kafka switching to linux is bit difficult.
> >
> > Further Kafka site does mention that windows is supported and I feel this
> > critical level bug should be addressed before making a release.
> > I clearly remember lower versions of Kafka did run on windows without any
> > problem. This issue started from 0.11 version I guess.
> >
> > Mostly in production we do shift to linux based OS or typically deploy on
> > such a cloud server.
> > However in this present case the client has a windows operating system
> and
> > we need to get this working without burdening them for linux based
> server.
> >
> > IT really moves slow in India and the time it will take them to provision
> > for a linux machines, the project may very well be lost.
> >
> > Would like to know if there is anything that can be done to make Kafka
> work
> > on windows.
> > I do feel that Kafka should continue to support windows because there are
> > still lot of businesses that use windows for some purpose where you do
> need
> > some kind of big data analysis.
> >
> > Like in this case we are trying to deploy a streams application in store
> > for collecting data from POS terminals which all run on windows.
> > All data analysis is store specific so we need a central server which can
> > process data from multiple terminals. Having them provision a windows
> > machine is very easy as that's all they have.
> >
> > Now to force them to install linux and that to over 1000's of stores
> nation
> > wide really needs some heavy lifting.
> >
> > Further moving the processing process to a central linux cloud is not
> > possible for now, because of lack of internet connectivity or a bad
> network
> > backbone, plus there are many more restrictions on moving data to cloud.
> >
> > So to conclude having a good windows support is essential for a thriving
> > kafka community and also to businesses that use and provide support on
> > kafka.
> >
> >
> > Please let me know how hard it would be to have a deployable kafka
> release
> > to windows. If there is some patch that can be released I am all good for
> > that too.
> >
> > Thanks
> > Sachin
> >
> >
> >
> > On Sun, Dec 1, 2019 at 8:54 PM Israel Ekpo <israelekpo@gmail.com> wrote:
> >
> > > Sachin
> > >
> > > Are you running production Kafka on Windows?
> > >
> > > Is it possible to migrate to an alternate UNIX based environment?
> > >
> > > It would be great to learn why you chose to run it on Windows
> > >
> > >
> > > On Sun, Dec 1, 2019 at 2:22 AM Sachin Mittal <sjmittal@gmail.com>
> wrote:
> > >
> > > > Hi All,
> > > > I hope we are well aware of the critical bug in windows where kafka
> > > crashes
> > > > when we delete a topic. This affects other areas too like in stream
> > > > processing when trying to reset a stream using StreamsResetter.
> > > >
> > > > Right now only workaround I have found is to stop zookeeper and kafka
> > > > server and manually delete directories and files containing streams,
> > > topics
> > > > and offset information.
> > > >
> > > > However doing this every time is kind of unproductive.
> > > >
> > > > I see that there are multiple critical bugs logged in JIRA
> > > > https://issues.apache.org/jira/browse/KAFKA-6203
> > > > https://issues.apache.org/jira/browse/KAFKA-1194
> > > > around the same issue.
> > > >
> > > > I would like to know by when would the fix be available?
> > > > I see that there have been multiple pull requests issued around fixes
> > of
> > > > these issues.
> > > >
> > > > I wanted to know if one or more pull requests need to be merged to
> get
> > > the
> > > > fix out or if there is something I can try config wise to have some
> > > > workaround for this issue.
> > > >
> > > > Please note that there can be few of us who might be using windows in
> > > > production too, so this fix is highly important for us.
> > > >
> > > > Please let me know what can be done to address this issue.
> > > >
> > > > Thanks
> > > > Sachin
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message