manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Has anyone successfully populated Elastic Search properties from custom SharePoint Meta data fields?
Date Tue, 30 Aug 2016 15:00:58 GMT
Hi Mike,

Download the plugin distribution zip, and unpack it.  Inside you should
find what you need to actually install into ES.  There is also a README in
there.

Thanks,
Karl


On Tue, Aug 30, 2016 at 10:52 AM, Furkan KAMACI <furkankamaci@gmail.com>
wrote:

> Hi Mike,
>
> You can check here for plugins: https://manifoldcf.
> apache.org/en_US/download.html#Latest+release+%28Apache+
> ManifoldCF+Plugin+for+Elastic+Search%2C+version+2.1%2C+2015+Jul+25%29
>
> Kind Regards,
> Furkan KAMACI
>
> On Tue, Aug 30, 2016 at 5:48 PM, Holtz, Michael (IT Consultant) <
> MHoltz@consultantemail.com> wrote:
>
>> Karl,
>>
>>
>>
>> I got what your’re saying, I should use the 1.5 plugin. Got it.
>>
>>
>>
>> I will try that
>>
>>
>>
>> Thanks,
>>
>> Mike H.
>>
>>
>>
>> *From:* Holtz, Michael (IT Consultant)
>> *Sent:* Tuesday, August 30, 2016 10:47 AM
>> *To:* user@manifoldcf.apache.org
>> *Subject:* RE: Has anyone successfully populated Elastic Search
>> properties from custom SharePoint Meta data fields?
>>
>>
>>
>> Hmm that’s confusing, here is the documentation:
>>
>>
>>
>> *Apache ManifoldCF ElasticSearch plugin compatibility*
>>
>> *ManifoldCF versions*
>>
>> *Plugin version*
>>
>> 0.1.x-1.4.x
>>
>> 0.x
>>
>> 1.5.x
>>
>> 1.x
>>
>> >=1.6.x
>>
>> 2.x
>>
>>
>>
>>
>>
>>
>>
>> I’m running version 2.4 of manifold so based on this I went with the 2.1
>> plugin.
>>
>>
>>
>>
>>
>> Where can I get the 1.0 plugin? I don’t see it on the site or in the 2.4
>> build.
>>
>>
>>
>> Thanks,
>>
>> Mike H.
>>
>>
>>
>> *From:* Karl Wright [mailto:daddywri@gmail.com <daddywri@gmail.com>]
>> *Sent:* Tuesday, August 30, 2016 10:38 AM
>> *To:* user@manifoldcf.apache.org
>> *Subject:* Re: Has anyone successfully populated Elastic Search
>> properties from custom SharePoint Meta data fields?
>>
>>
>>
>> You need the ES-1.5 plugin.  That's the older (deprecated) 1.0 plugin.
>>
>>
>>
>> Karl
>>
>>
>>
>>
>>
>> On Tue, Aug 30, 2016 at 10:36 AM, Holtz, Michael (IT Consultant) <
>> MHoltz@consultantemail.com> wrote:
>>
>> Elasticsearch 2.3.3
>>
>> Plugin 2.1
>>
>>
>>
>> Here is the current error
>>
>>
>>
>> C:\APPS\elasticsearch-2.3.3>bin\plugin install
>> file:///C:/Users/holtzm/workspace/apache-manifoldcf-2.4/
>> dist/plugins/elasticsearch/elasticsearch/elasticsearch-plugin-mcf-2.1.jar
>>
>> -> Installing from file:/C:/Users/holtzm/workspac
>> e/apache-manifoldcf-2.4/dist/plugins/elasticsearch/elastics
>> earch/elasticsearch-plugin-mcf-2.1.jar...
>>
>> Trying file:/C:/Users/holtzm/workspace/apache-manifoldcf-2.4/dist/
>> plugins/elasticsearch/elasticsearch/elasticsearch-plugin-mcf-2.1.jar ...
>>
>> Downloading .DONE
>>
>> Verifying file:/C:/Users/holtzm/workspace/apache-manifoldcf-2.4/dist/
>> plugins/elasticsearch/elasticsearch/elasticsearch-plugin-mcf-2.1.jar
>> checksums if available ...
>>
>> NOTE: Unable to verify checksum for downloaded plugin (unable to find
>> .sha1 or .md5 file to verify)
>>
>> ERROR: Could not find plugin descriptor 'plugin-descriptor.properties' in
>> plugin zip
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *From:* Karl Wright [mailto:daddywri@gmail.com]
>> *Sent:* Tuesday, August 30, 2016 10:22 AM
>>
>>
>> *To:* user@manifoldcf.apache.org
>> *Subject:* Re: Has anyone successfully populated Elastic Search
>> properties from custom SharePoint Meta data fields?
>>
>>
>>
>> Hi Mike,
>>
>>
>>
>> If you are just trying things out you can use the single-process example
>> and it will make your life easier for experimentation.
>>
>>
>>
>> As far as the ES plugin, there are two: one for ES up to 1.5, and one for
>> 1.5 onward.  Which did you try to install?
>>
>>
>>
>> Karl
>>
>>
>>
>>
>>
>> On Tue, Aug 30, 2016 at 9:56 AM, Holtz, Michael (IT Consultant) <
>> MHoltz@consultantemail.com> wrote:
>>
>> Karl,
>>
>>
>>
>> You are correct.
>>
>>
>>
>> Before I setup zookeeper I would like to prove out a few use cases.
>>
>>
>>
>> 1.       Index custom meta data fields from SharePoint into the
>> ElasticSearch index
>>
>> 2.       Full text indexing of SharePoint content in ElasticSearch
>>
>> 3.       SharePoint security information indexed in ElasticSearch
>>
>> 4.       ElasticSearch Plugin to enforce SharePoint security settings
>> for search
>>
>>
>>
>> At this point I’ve only managed to see #2 and #3 work
>>
>>
>>
>> For #4 I was getting an error when installing the ElasticSearch plugin.
>>
>> I would like to see #1 work before I do anything else. Are there any
>> example configurations that show how to do this?
>>
>>
>>
>>
>>
>> Thanks,
>>
>> Mike H.
>>
>>
>>
>> *From:* Karl Wright [mailto:daddywri@gmail.com]
>> *Sent:* Tuesday, August 30, 2016 9:49 AM
>>
>>
>> *To:* user@manifoldcf.apache.org
>> *Subject:* Re: Has anyone successfully populated Elastic Search
>> properties from custom SharePoint Meta data fields?
>>
>>
>>
>> Hi Mike,
>>
>>
>>
>> This sounds like you're having a lock problem.
>>
>>
>>
>> Let me guess: you are using the multiprocess example with file-based
>> locking?  If so, you need to stop everything and do the "lock clean"
>> procedure.  I would also strongly recommend that you switch to the
>> zookeeper multiprocess example.
>>
>>
>>
>> Thanks,
>>
>> Karl
>>
>>
>>
>>
>>
>> On Tue, Aug 30, 2016 at 9:44 AM, Holtz, Michael (IT Consultant) <
>> MHoltz@consultantemail.com> wrote:
>>
>> I currently cannot view the job. The UI is hanging when I click the
>> “view” link for the job. I’m wondering if that has anything to do with the
>> “Metadata Adjuster” I added?
>>
>>
>>
>> I did not schedule the job.
>>
>>
>>
>> At this point the job has finished running and the Document Status and
>> Queue status reports. I do not see any errors reported there.
>>
>>
>>
>> Also I still do not see my custom meta data fields from SharePoint in the
>> ElasticSearch index.
>>
>>
>>
>> We are hoping to use this in production, but at this point it is not
>> looking too good.
>>
>>
>>
>> Thanks,
>>
>> Mike H.
>>
>>
>>
>> *From:* Karl Wright [mailto:daddywri@gmail.com]
>> *Sent:* Tuesday, August 30, 2016 9:09 AM
>>
>>
>> *To:* user@manifoldcf.apache.org
>> *Subject:* Re: Has anyone successfully populated Elastic Search
>> properties from custom SharePoint Meta data fields?
>>
>>
>>
>> It is also important to know whether you have your job set up as
>> "continuous".  If so, it will run forever.
>>
>>
>>
>> Karl
>>
>>
>>
>>
>>
>> On Tue, Aug 30, 2016 at 8:48 AM, Furkan KAMACI <furkankamaci@gmail.com>
>> wrote:
>>
>> Hi Mike,
>>
>>
>>
>> Could you tell us what "Document Status" and "Queue Status" says as Karl
>> mentioned?
>>
>>
>>
>> Kind Regards,
>>
>> Furkan KAMACI
>>
>>
>>
>> On Tue, Aug 30, 2016 at 3:39 PM, Holtz, Michael (IT Consultant) <
>> MHoltz@consultantemail.com> wrote:
>>
>> I’m looking at manifoldcf.log and I see no errors.
>>
>>
>>
>> I have only 13 files and they are all very tiny < 15kb .docx files.
>>
>>
>>
>> Mike H.
>>
>>
>>
>> *From:* Karl Wright [mailto:daddywri@gmail.com]
>> *Sent:* Monday, August 29, 2016 5:26 PM
>>
>>
>> *To:* user@manifoldcf.apache.org
>> *Subject:* Re: Has anyone successfully populated Elastic Search
>> properties from custom SharePoint Meta data fields?
>>
>>
>>
>> You should be looking at the ManifoldCF.log file, and the examples have
>> logging for errors at least as being the default.
>>
>>
>>
>> The only other possibility is that you have a very large document that is
>> taking a long time to transfer/index.
>>
>>
>>
>> Also, you can generally see what MCF is up to using the "document status"
>> and "queue status" reports.
>>
>>
>>
>> Karl
>>
>>
>>
>>
>>
>> On Mon, Aug 29, 2016 at 5:22 PM, Holtz, Michael (IT Consultant) <
>> MHoltz@consultantemail.com> wrote:
>>
>> Karl,
>>
>>
>>
>> I see no errors in the log.
>>
>>
>>
>> I probably need to configure the logging I think.
>>
>>
>>
>> Thanks,
>>
>> Mike H.
>>
>>
>>
>> *From:* Karl Wright [mailto:daddywri@gmail.com]
>> *Sent:* Monday, August 29, 2016 5:20 PM
>>
>>
>> *To:* user@manifoldcf.apache.org
>> *Subject:* Re: Has anyone successfully populated Elastic Search
>> properties from custom SharePoint Meta data fields?
>>
>>
>>
>> Hi Mike,
>>
>>
>>
>> Have a look in the log.  If there are errors there please send them to
>> us.  Generally, "hangs" are due to MCF retrying documents that have errors.
>>
>>
>>
>> Karl
>>
>>
>>
>>
>>
>> On Mon, Aug 29, 2016 at 5:18 PM, Holtz, Michael (IT Consultant) <
>> MHoltz@consultantemail.com> wrote:
>>
>> Karl,
>>
>>
>>
>> Yes, it shows up. I include all meta-data.
>>
>>
>>
>> I’m no trying to use a MetaData adjuster like this
>>
>>
>>
>> Parameter
>>
>> field1                     “${Field 1}”
>>
>>
>>
>> And now I see “field1”=”” in Elastic Search.
>>
>>
>>
>> Also it seems each time I make a change the job will hang and I need to
>> restart everything including ElasticSearch to get it to complete.
>>
>>
>>
>> Thanks,
>>
>> Mike H.
>>
>>
>>
>> *From:* Karl Wright [mailto:daddywri@gmail.com]
>> *Sent:* Monday, August 29, 2016 5:13 PM
>> *To:* user@manifoldcf.apache.org
>> *Subject:* Re: Has anyone successfully populated Elastic Search
>> properties from custom SharePoint Meta data fields?
>>
>>
>>
>> Hi Mike,
>>
>>
>>
>> If I were you I'd try to independently verify that the custom field shows
>> up.  Do you see it appear in the pulldown for selecting metadata?  If not,
>> it probably is not accessible through the inspection methods that give us
>> access into SharePoint fields.
>>
>>
>>
>> If it appears there then we should look at what ES gets sent.  It's
>> possible it appears but under a different field name.
>>
>>
>>
>> Karl
>>
>>
>>
>>
>>
>> On Mon, Aug 29, 2016 at 5:01 PM, Holtz, Michael (IT Consultant) <
>> MHoltz@consultantemail.com> wrote:
>>
>> Hi,
>>
>>
>>
>> I have created a custom content type with a custom field “Field 1”.
>>
>>
>>
>> I have configured the job to include all meta data.
>>
>>
>>
>> I do not see “Field 1” being populated in the Elastic Search index.
>>
>>
>>
>>
>>
>> Does anyone have any experience with this?
>>
>>
>>
>> Thanks in advance,
>>
>> Mike H
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>

Mime
View raw message