lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jürgen Wagner (DVT)" <juergen.wag...@devoteam.com>
Subject Re: PDF search functionality using Solr
Date Tue, 06 Jan 2015 17:55:53 GMT
Hello,
  no matter which search platform you will use, this will pose two
challenges:

- The size of the documents will render search less and less useful as
the likelihood of matches increases with document size. So, without a
proper semantic extraction (e.g., using decent NER or relationship
extraction with a commercial text mining product), I doubt you will get
the required precision to make this overly usefiul.

- PDFs can have their own character sets based on the characters
actually used. Such file-specific character sets are almost impossible
to parse, i.e., if your PDFs happen to use this "feature" of the PDF
format, you won't be lucky getting any meaningful text out of them.

My suggestion is to use the Jira REST API to collect all necessary
documents and index the resulting XML or attachment formats. As the REST
API provides filtering capabilities, you could easily create incremental
feeds to avoid humongous indexing every time there's new information in
Jira. Dumping Jira stuff as PDF seems to me to be the least suitable way
of handling this.

Best regards,
--Jürgen


On 06.01.2015 18:30, Ganesh.Yadav@sungard.com wrote:
> Hello Solr-users and developers,
> Can you please suggest,
>
> 1.       What I should do to index PDF content information column wise?
>
> 2.       Do I need to extract the contents using one of the Analyzer, Tokenize and Filter
combination and then add it to Index? How can test the results on command prompt? I do not
know the selection of specific Analyzer, Tokenizer and Filter for this purpose
>
> 3.       How can I verify that the needed column info is extracted out of PDF and is
indexed?
>
> 4.       So for example How to verify Ticket number is extracted in Ticket_number tag
and is indexed?
>
> 5.       Is it ok to post 4 GB worth of PDF to be imported and indexed by Solr? I think
I saw some posts complaining on how large size that can be posted ?
>
> 6.       What will enable Solr to search in any PDF out of many, with different words
such as "Runtime" "Error" "XXXX" and result will provide the link to the PDF
>
> My PDFs are nothing but Jira ticket system.
> PDF has info on
> Ticket Number:
> Desc:
> Client:
> Status:
> Submitter:
> And so on:
>
>
> 1.       I imported PDF document in Solr and it does the necessary searching and I can
test some of it using the browse client interface provided.
>
> 2.       I have 80 GB worth of PDFs.
>
> 3.       Total number of PDFs are about 200
>
> 4.       Many PDFs are of size 4 GB
>
> 5.       What do you suggest me to import such a large PDFs? What tools can you suggest
to extract PDF contents first in some XML format and later Post that XML to be indexed by
Solr.?
>
>
>
>
>
>
>
> Your early response is much appreciated.
>
>
>
> Thanks
>
> G
>
>


-- 

Mit freundlichen Grüßen/Kind regards/Cordialement vôtre/Atentamente/С
уважением
*i.A. Jürgen Wagner*
Head of Competence Center "Intelligence"
& Senior Cloud Consultant

Devoteam GmbH, Industriestr. 3, 70565 Stuttgart, Germany
Phone: +49 6151 868-8725, Fax: +49 711 13353-53, Mobile: +49 171 864 1543
E-Mail: juergen.wagner@devoteam.com
<mailto:juergen.wagner@devoteam.com>, URL: www.devoteam.de
<http://www.devoteam.de/>

------------------------------------------------------------------------
Managing Board: Jürgen Hatzipantelis (CEO)
Address of Record: 64331 Weiterstadt, Germany; Commercial Register:
Amtsgericht Darmstadt HRB 6450; Tax Number: DE 172 993 071



Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message