kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sachin Mittal <sjmit...@gmail.com>
Subject Re: Deleting topics in windows - fix estimate or workaround
Date Sun, 01 Dec 2019 17:46:41 GMT
Hi,
Firstly development environment is windows and I have been able to run
almost any big data framework on windows (or cygwin) be it HBase, Cassandra
or anything else.
Now just for running Kafka switching to linux is bit difficult.

Further Kafka site does mention that windows is supported and I feel this
critical level bug should be addressed before making a release.
I clearly remember lower versions of Kafka did run on windows without any
problem. This issue started from 0.11 version I guess.

Mostly in production we do shift to linux based OS or typically deploy on
such a cloud server.
However in this present case the client has a windows operating system and
we need to get this working without burdening them for linux based server.

IT really moves slow in India and the time it will take them to provision
for a linux machines, the project may very well be lost.

Would like to know if there is anything that can be done to make Kafka work
on windows.
I do feel that Kafka should continue to support windows because there are
still lot of businesses that use windows for some purpose where you do need
some kind of big data analysis.

Like in this case we are trying to deploy a streams application in store
for collecting data from POS terminals which all run on windows.
All data analysis is store specific so we need a central server which can
process data from multiple terminals. Having them provision a windows
machine is very easy as that's all they have.

Now to force them to install linux and that to over 1000's of stores nation
wide really needs some heavy lifting.

Further moving the processing process to a central linux cloud is not
possible for now, because of lack of internet connectivity or a bad network
backbone, plus there are many more restrictions on moving data to cloud.

So to conclude having a good windows support is essential for a thriving
kafka community and also to businesses that use and provide support on
kafka.


Please let me know how hard it would be to have a deployable kafka release
to windows. If there is some patch that can be released I am all good for
that too.

Thanks
Sachin



On Sun, Dec 1, 2019 at 8:54 PM Israel Ekpo <israelekpo@gmail.com> wrote:

> Sachin
>
> Are you running production Kafka on Windows?
>
> Is it possible to migrate to an alternate UNIX based environment?
>
> It would be great to learn why you chose to run it on Windows
>
>
> On Sun, Dec 1, 2019 at 2:22 AM Sachin Mittal <sjmittal@gmail.com> wrote:
>
> > Hi All,
> > I hope we are well aware of the critical bug in windows where kafka
> crashes
> > when we delete a topic. This affects other areas too like in stream
> > processing when trying to reset a stream using StreamsResetter.
> >
> > Right now only workaround I have found is to stop zookeeper and kafka
> > server and manually delete directories and files containing streams,
> topics
> > and offset information.
> >
> > However doing this every time is kind of unproductive.
> >
> > I see that there are multiple critical bugs logged in JIRA
> > https://issues.apache.org/jira/browse/KAFKA-6203
> > https://issues.apache.org/jira/browse/KAFKA-1194
> > around the same issue.
> >
> > I would like to know by when would the fix be available?
> > I see that there have been multiple pull requests issued around fixes of
> > these issues.
> >
> > I wanted to know if one or more pull requests need to be merged to get
> the
> > fix out or if there is something I can try config wise to have some
> > workaround for this issue.
> >
> > Please note that there can be few of us who might be using windows in
> > production too, so this fix is highly important for us.
> >
> > Please let me know what can be done to address this issue.
> >
> > Thanks
> > Sachin
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message