metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Shlyakhtin <Vladimir.Shlyakh...@sstech.us>
Subject threatintel_taxii_load.sh throws exception
Date Mon, 10 Jul 2017 12:11:59 GMT
Hello,

After upgrading from 0.3.0 version we noticed that taxii loader does not work.

Here is details:

# /usr/metron/0.3.1/bin/threatintel_taxii_load.sh -c /usr/local/opentaxii/etc/connections.conf/guest.phishtank_com.json
-e /usr/local/opentaxii/etc/connections.conf/extractor.json -p 10000
17/07/10 07:35:42 WARN extractor.TransformFilterExtractorDecorator: Unable to setup zookeeper
client - zk_quorum url not provided. **This will limit some Stellar functionality**
Exception in thread "main" java.lang.IllegalStateException: Extractor must be a STIX Extractor
        at org.apache.metron.dataloads.nonbulk.taxii.TaxiiLoader.main(TaxiiLoader.java:202)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:233)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:148)


# cat /usr/local/opentaxii/etc/connections.conf/guest.phishtank_com.json
{
  "endpoint" : "http://10.10.110.23:9000/services/discovery"
  ,"port" : "9000"
  ,"type" : "DISCOVER"
  ,"collection" : "guest.phishtank_com"
  ,"table" : "threatintel"
  ,"columnFamily" : "t"
  ,"allowedIndicatorTypes" : [ "domainname:FQDN", "address:IPV_4_ADDR" ]
}

# cat /usr/local/opentaxii/etc/connections.conf/extractor.json
{
  "config": {
    "columns": {
      "domain": 0
    },
    "indicator_column": "domain",
    "type" : "malicious_domain",
    "separator" : ","
  },
  "extractor" : "STIX"
}


"zk_quorum"  parameter we specified as well and got exception (without "Unable to setup zookeeper
client" warning).

Exception is thrown due to false in condition:
```
if(e instanceof StixExtractor) {
```

in file TaxiiLoader.java.

I changed exception message to:
```
throw new IllegalStateException("Extractor must be a STIX Extractor" + " - " + e.getClass().getName());
```

and got:
```
Extractor must be a STIX Extractor - org.apache.metron.dataloads.extractor.TransformFilterExtractorDecorator
```

Seems like it is related to commit c5bbf5acef05de16a18db9397745a6795427f6b8.

Let me know if this is bug and I should open jira issue.

Thank you


- Vladimir

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message