lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell Taylor <Russell.Tay...@interactivedata.com>
Subject RE: Replacing a group of documents (Delete/Insert) without a query on the index ever showing an empty list (Docs)
Date Fri, 27 Mar 2015 13:07:37 GMT
Hi Shawn, thanks for the quick reply.

I've looked at both methods and I think that they won't work for a number of reasons:

1)
uniqueKey:
 I could use the uniqueKey and overwrite the original document but I need to remove the documents
which 
are not on my new input list and the issue with the uniqueKey method is I don't know what
to delete.

Documents on the index:
"docs": [
{
"id":"1"
"keyField":"A"
},{
"id":"2"
"keyField":"A"
},{
"id":"3"
"keyField":"B"
}
]
New Documents to go on index
"docs": [
{
"id":"1"
"keyField":"A"
},{
"id":"3"
"keyField":"B"
}
]
I would never know that id:2 should be deleted. (on some new document lists the delete list
could be in the millions).

2)
openSearcher:
My openSearcher is set to false and I've also commented out autoSoftCommit so I don't get
a partial list being returned on a query.
<!--
<autoSoftCommit>
       <maxTime>${solr.autoSoftCommit.maxTime:1000}</maxTime>
</autoSoftCommit>
-->


So is there another way to keep the original set of documents until the new set has been added
to the index?


Thanks


Russ.




-----Original Message-----
From: Shawn Heisey [mailto:apache@elyograg.org] 
Sent: 26 March 2015 16:06
To: solr-user@lucene.apache.org
Subject: Re: Replacing a group of documents (Delete/Insert) without a query on the index ever
showing an empty list (Docs)

On 3/26/2015 9:53 AM, Russell Taylor wrote:
> I have an index which is made up of groups of documents, each group is defined by a field
called keyField (keyField:A).
> I need to delete all the keyField:A documents and replace them with a 
> brand new set without the index ever returning zero documents on a query.
>
> At the moment I deleteByQuery:keyField:A and then insert a 
> SolrInputDocument list via SolrJ into my index. I have a small time 
> period where somebody doing a q=fieldKey:A can be returned an empty list.
>
> FYI: The keyField group might be just 100 documents or up to 10 million.

As long as you don't have any commits with openSearcher=true happening between the delete
and the insert, that would work ... but why go through the manual delete if you don't have
to?

If you define a suitable uniqueKey field in your schema, simply indexing a new document with
the same value in the uniqueKeyfield as an existing document will delete the old document.

https://wiki.apache.org/solr/UniqueKey

Thanks,
Shawn



*******************************************************
This message (including any files transmitted with it) may contain confidential and/or proprietary
information, is the property of Interactive Data Corporation and/or its subsidiaries, and
is directed only to the addressee(s). If you are not the designated recipient or have reason
to believe you received this message in error, please delete this message from your system
and notify the sender immediately. An unintended recipient's disclosure, copying, distribution,
or use of this message or any attachments is prohibited and may be unlawful. 
*******************************************************
Mime
View raw message