metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin Leet <justinjl...@gmail.com>
Subject Re: [DISCUSS] Metron debug info tool
Date Wed, 11 Apr 2018 16:59:04 GMT
First off, this is super nice, and a great way to let us be able to debug
and help others debug quickly, easily, and hopefully more consistently.

I super briefly glanced at at it, so these might already be there, but I'd
like to be able to filter what I get back, e.g. if I give the options for
Storm and Metron, I'd like to limit to just those. Nothing complicated, but
something quick and simple.

Hand in hand with that, I'd like the option to print to screen (maybe just
for the non-config stuff or just print out the relevant filenames?).  At
that point, it'd be really easy to grep or otherwise search through
things.  Tarball is nice, especially when passing things off to someone
else, or when you need to dig through a lot of larger config files, but I
suspect a lot of use cases will be "Hey, real quick what's going on?"

Other than that, does anyone have any thoughts on putting something like
this into the management UI (for the non-Ambari managed stuff)?  That seems
like it would be the natural place to get that stuff, keep it up to date,
and even build in an export if we wanted to.  Would make it a lot easier
for end users to be able to get a quick view into what's going on, and
could let us build in some slightly better filtering and search
capabilities.



On Wed, Apr 11, 2018 at 12:10 PM, Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:

> Hey guys,
>
> I wanted to bring attention to a tool I created for gathering cluster
> details for debugging purposes. There are a number of locations that
> properties get materialized, e.g. from Ambari -> properties file -> flux ->
> Storm, which means a lot of hunting to guarantee that the changes you've
> made are percolating correctly. Furthermore, it's generally useful to get a
> sense of how your cluster is configured by gathering all of that info in
> one place. I created a Python tool that does just that, and bundles up the
> results in a tarball. Here is an overview of the artifacts I'm gathering -
> you can see what commands are being used by looking at the script.
>
> Ambari
>     full cluster config detail
>
> Storm
>     cluster summary
>     cluster configuration
>     topology summary (enrichments and indexing)
>     topology status summary (enrichments and indexing)
>
> Kafka
>     broker info
>     topics list
>     topic details (enrichments and indexing)
>
> Metron
>     local file system configuration files
>     zookeeper configuration
>     flux files
>     lib directory file listing
>     rpm listing
>
> Hadoop
>     version info
>
> *** Are there any features/details you'd like to see added to this? Any
> concerns or suggestions? ***
>
> I am  also planning to add log file support along with md5sum of the jar
> files deployed in Metron's lib directory.
>
> https://github.com/apache/metron/pull/988/files#diff-
> 0eddfa8f1dd67247e0803e405497b6e2
>
> Cheers,
> Mike Miklavcic
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message