metron-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bbende <...@git.apache.org>
Subject [GitHub] incubator-metron pull request: Metron 86 - SolrWriter
Date Sat, 02 Apr 2016 00:21:55 GMT
Github user bbende commented on the pull request:

    https://github.com/apache/incubator-metron/pull/67#issuecomment-204616131
  
    @merrimanr I think using the auto commits would be the way to go and gives you the flexibility
to tune the values to your needs. You could still leave the optional commit in the writer
in case it is needed for testing or anything else. 
    
    In terms of the appropriate commit intervals, a lot of it is trial and error depending
on how much latency the application can tolerate to see updates, vs. speed & amount of
incoming data. Soft commits are lightweight operations (some overhead) that will make updates
visible, but the updates aren't flushed to disk until a hard commit. 
    
    This post has a good description of the tread-offs:
    https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
    
    Metron might be the "Index-Heavy, Query-light" scenario? but not sure. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message