metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Foley <mfo...@hortonworks.com>
Subject Re: Elasticsearch masters_also_are_datanodes doesn't work.
Date Wed, 13 Sep 2017 21:18:18 GMT
Interesting, I had previously found that many (but not all) ES params are extensively processed
by code fairly deep in the guts of ES itself (unrelated to Metron’s use of it), to allow
this kind of flexibility.
Thanks very much for sharing your results, Mike.  Looks like we’ll continue with the release.

Laurens, I hope you can find what’s going on, on your system.  I would suggest that, if
it isn’t being written through to the yml file, it seems to be an Ambari or install issue
rather than an ES issue per se.

Thanks,
--Matt

From: Michael Miklavcic <michael.miklavcic@gmail.com>
Date: Wednesday, September 13, 2017 at 2:08 PM
To: Matt Foley <mfoley@hortonworks.com>
Cc: Laurens Vets <laurens@daemon.be>, "dev@metron.apache.org" <dev@metron.apache.org>
Subject: Re: Elasticsearch masters_also_are_datanodes doesn't work.

Here's what I found. Data is flowing into ES indexes. I can change the "masters_also_are_datanodes"
property and it is reflected in the node.data property in elasticsearch.yml.

Here's my search for this property in the source code. It is properly being used int the ES
mpack scripts afaik.

Targets
    Occurrences of 'masters_also_are_datanodes' in Directory devprojects/metron/metron-deployment
Found Occurrences  (6 usages found)
    Unclassified occurrence  (5 usages found)
        metron-deployment  (2 usages found)
            metron-deployment/other-examples/manual-install  (1 usage found)
                Manual_Install_CentOS6.md  (1 usage found)
                    810 "masters_also_are_datanodes" to "true"
            metron-deployment/roles/ambari_config/vars  (1 usage found)
                single_node_vm.yml  (1 usage found)
                    104 masters_also_are_datanodes: "1"
        metron_mpack  (3 usages found)
            common-services.ELASTICSEARCH.2.3.3.configuration  (1 usage found)
                elastic-site.xml  (1 usage found)
                    30 <name>masters_also_are_datanodes</name>
            common-services.ELASTICSEARCH.2.3.3.package.scripts  (1 usage found)
                params.py  (1 usage found)
                    26 masters_also_are_datanodes = config['configurations']['elastic-site']['masters_also_are_datanodes']
            common-services.ELASTICSEARCH.2.3.3.package.templates  (1 usage found)
                elasticsearch.master.yaml.j2  (1 usage found)
                    36   data: {{ masters_also_are_datanodes }}
    Usage in string constants  (1 usage found)
        metron_mpack  (1 usage found)
            common-services.ELASTICSEARCH.2.3.3.package.scripts  (1 usage found)
                params.py  (1 usage found)
                    26 masters_also_are_datanodes = config['configurations']['elastic-site']['masters_also_are_datanodes']

It should be noted that regardless of the ES documentation, this seems to work with a variety
of values including "true" (with quotes) and 1 (without quotes). When I set the value to garbage
through Ambari, I see this in /var/log/elasticsearch/metron.log
java.lang.IllegalArgumentException: value cannot be parsed to boolean [ true/1/on/yes OR false/0/off/no
]
        at org.elasticsearch.common.Booleans.parseBooleanExact(Booleans.java:97)
        at org.elasticsearch.cluster.node.DiscoveryNode.dataNode(DiscoveryNode.java:256)
        at org.elasticsearch.cluster.node.DiscoveryNodes$Builder.build(DiscoveryNodes.java:659)
        at org.elasticsearch.cluster.ClusterState$Builder.nodes(ClusterState.java:571)
        at org.elasticsearch.cluster.service.InternalClusterService.doStart(InternalClusterService.java:205)
        at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:68)
        at org.elasticsearch.node.Node.start(Node.java:279)
        at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:206)
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:272)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)

Apparently, even yes/no will work! I tried it, and can confirm they work with quotes. But
surprisingly, this does NOT work if you pass true/false or yes/no without quotes. In summary,
here are the supported values:

  *   1
  *   0
  *   "true"
  *   "false"
  *   "yes"
  *   "no"
Mike


On Wed, Sep 13, 2017 at 2:17 PM, Matt Foley <mfoley@hortonworks.com<mailto:mfoley@hortonworks.com>>
wrote:
Sorry, you’ve beat me.

Mike, what have you found in full-dev?  Does ES work correctly there?


From: Laurens Vets <laurens@daemon.be<mailto:laurens@daemon.be>>
Date: Wednesday, September 13, 2017 at 1:14 PM
To: Matt Foley <mfoley@hortonworks.com<mailto:mfoley@hortonworks.com>>
Cc: "dev@metron.apache.org<mailto:dev@metron.apache.org>" <dev@metron.apache.org<mailto:dev@metron.apache.org>>,
Michael Miklavcic <michael.miklavcic@gmail.com<mailto:michael.miklavcic@gmail.com>>

Subject: Re: Elasticsearch masters_also_are_datanodes doesn't work.


Yes, I'm saving the changes. The orange bar pops up saying I have to restart elasticsearch,
which I do. I'm logged in with the admin user, it has privileges to do everything else. I'm
using a bare metal install via Ambari. Everything else seems to work through Ambari except
this :)

On 2017-09-13 12:04, Matt Foley wrote:
That it doesn't work in Ambari, at least to the point of writing thru to the config file (whether
or not it actually works with ES), is quite mystifying.  Silly question, but, are you pressing
the "Save" button after changing the field?  Are you perhaps not logged into Ambari with an
ambari user id that has privs to change this config?  Did you use non-default user configuration
with Ambari, and if so are you running with an admin-priv ambari account?

From: Laurens Vets <laurens@daemon.be<mailto:laurens@daemon.be>>
Date: Tuesday, September 12, 2017 at 8:32 PM
To: "dev@metron.apache.org<mailto:dev@metron.apache.org>" <dev@metron.apache.org<mailto:dev@metron.apache.org>>
Cc: Michael Miklavcic <michael.miklavcic@gmail.com<mailto:michael.miklavcic@gmail.com>>,
Matt Foley <mfoley@hortonworks.com<mailto:mfoley@hortonworks.com>>
Subject: Re: Elasticsearch masters_also_are_datanodes doesn't work.

See inline.
Now, Laurens, two question for you:   First, please confirm: Are you setting the value of
masters_also_are_datanodes via the masters_also_are_datanodes field in the Ambari configuration
GUI?  Or directly editing files?
I'm using Ambari to edit the configuration. However, I have to manually edit the file locally
as the Ambari configuration doesn't work.
Second, do you set it at the initial install time, or afterward?  I'm pretty sure it needs
to be set at initial install time, before you ever launch the Elasticsearch nodes.  I'm dubious
about whether an Elasticsearch master can learn to be a datanode if it first wakes up and
configures itself to not be one.  But I'm not an Elasticsearch expert, so if someone else
knows different, please say so.
In this case, I forgot to set it initially, so I tried to change it afterwards to no avail.
Hope this helps more than obfuscates.
--Matt



From: Michael Miklavcic <michael.miklavcic@gmail.com<mailto:michael.miklavcic@gmail.com>>
Date: Tuesday, September 12, 2017 at 4:54 PM
To: "dev@metron.apache.org<mailto:dev@metron.apache.org>" <dev@metron.apache.org<mailto:dev@metron.apache.org>>
Cc: Matt Foley <mfoley@hortonworks.com<mailto:mfoley@hortonworks.com>>
Subject: Re: Elasticsearch masters_also_are_datanodes doesn't work.
At the very least, the value provided by default seems to have changed to a "1" instead of
"true" without the tooltip having been updated to match.
Error! Filename not specified.
On Tue, Sep 12, 2017 at 4:00 PM, Michael Miklavcic <michael.miklavcic@gmail.com<mailto:michael.miklavcic@gmail.com>>
wrote:
I think this is our default setup for full dev. It's only a 1-node VM, so I'm pretty sure
that it would not work otherwise. I'm spinning up full dev now and will look into it also.
On Tue, Sep 12, 2017 at 3:04 PM, Laurens Vets <laurens@daemon.be<mailto:laurens@daemon.be>>
wrote:
https://issues.apache.org/jira/browse/METRON-1181

I'll also spin up a full-dev environment and see what happens there.


On 2017-09-10 21:50, Matt Foley wrote:
Laurens, please open a jira.  Altho it may seem obvious, please
include full repro.
This may be a showstopper, as it presumably (?) prevents ES from
working on a single-node deployment?

On 9/10/17, 4:01 PM, "Laurens Vets" <laurens@daemon.be<mailto:laurens@daemon.be>>
wrote:

    Another issue I noticed. Setting "masters_also_are_datanodes" in Ambari
    to "true" does not work.
    The settings in /etc/elasticsearch/elasticsearch.yml will always be
    false when restarting elasticsearch...





Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message