kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: Deleting topics in windows - fix estimate or workaround
Date Mon, 02 Dec 2019 00:03:37 GMT
My development environment is windows. I am actually able to run
integration tests launching Kafka and Zookeeper. However I have noticed
that the test warn likely about deleting a file that is locked and I tried
deleting a topic at one point and saw some of  the issues you mentioned. I
personally think it is a blessing that it works at all, (because my
environment does not support docker either so I would really be up the
creek), but because it seems spotty in terms of features I do not count on
it 100%.

Because Java is supposedly 100% pure it seems like everything SHOULD work.

I pessimistic hot take: Generally what (windows support) takes is some
large organization (say the us government, or some bank) pays the largest
consulting group/support company for a contract and demand windows support.
Then magically it happens :)

On Sun, Dec 1, 2019 at 12:47 PM Sachin Mittal <sjmittal@gmail.com> wrote:

> Hi,
> Firstly development environment is windows and I have been able to run
> almost any big data framework on windows (or cygwin) be it HBase, Cassandra
> or anything else.
> Now just for running Kafka switching to linux is bit difficult.
>
> Further Kafka site does mention that windows is supported and I feel this
> critical level bug should be addressed before making a release.
> I clearly remember lower versions of Kafka did run on windows without any
> problem. This issue started from 0.11 version I guess.
>
> Mostly in production we do shift to linux based OS or typically deploy on
> such a cloud server.
> However in this present case the client has a windows operating system and
> we need to get this working without burdening them for linux based server.
>
> IT really moves slow in India and the time it will take them to provision
> for a linux machines, the project may very well be lost.
>
> Would like to know if there is anything that can be done to make Kafka work
> on windows.
> I do feel that Kafka should continue to support windows because there are
> still lot of businesses that use windows for some purpose where you do need
> some kind of big data analysis.
>
> Like in this case we are trying to deploy a streams application in store
> for collecting data from POS terminals which all run on windows.
> All data analysis is store specific so we need a central server which can
> process data from multiple terminals. Having them provision a windows
> machine is very easy as that's all they have.
>
> Now to force them to install linux and that to over 1000's of stores nation
> wide really needs some heavy lifting.
>
> Further moving the processing process to a central linux cloud is not
> possible for now, because of lack of internet connectivity or a bad network
> backbone, plus there are many more restrictions on moving data to cloud.
>
> So to conclude having a good windows support is essential for a thriving
> kafka community and also to businesses that use and provide support on
> kafka.
>
>
> Please let me know how hard it would be to have a deployable kafka release
> to windows. If there is some patch that can be released I am all good for
> that too.
>
> Thanks
> Sachin
>
>
>
> On Sun, Dec 1, 2019 at 8:54 PM Israel Ekpo <israelekpo@gmail.com> wrote:
>
> > Sachin
> >
> > Are you running production Kafka on Windows?
> >
> > Is it possible to migrate to an alternate UNIX based environment?
> >
> > It would be great to learn why you chose to run it on Windows
> >
> >
> > On Sun, Dec 1, 2019 at 2:22 AM Sachin Mittal <sjmittal@gmail.com> wrote:
> >
> > > Hi All,
> > > I hope we are well aware of the critical bug in windows where kafka
> > crashes
> > > when we delete a topic. This affects other areas too like in stream
> > > processing when trying to reset a stream using StreamsResetter.
> > >
> > > Right now only workaround I have found is to stop zookeeper and kafka
> > > server and manually delete directories and files containing streams,
> > topics
> > > and offset information.
> > >
> > > However doing this every time is kind of unproductive.
> > >
> > > I see that there are multiple critical bugs logged in JIRA
> > > https://issues.apache.org/jira/browse/KAFKA-6203
> > > https://issues.apache.org/jira/browse/KAFKA-1194
> > > around the same issue.
> > >
> > > I would like to know by when would the fix be available?
> > > I see that there have been multiple pull requests issued around fixes
> of
> > > these issues.
> > >
> > > I wanted to know if one or more pull requests need to be merged to get
> > the
> > > fix out or if there is something I can try config wise to have some
> > > workaround for this issue.
> > >
> > > Please note that there can be few of us who might be using windows in
> > > production too, so this fix is highly important for us.
> > >
> > > Please let me know what can be done to address this issue.
> > >
> > > Thanks
> > > Sachin
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message