manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Issue Extracting Authorities.
Date Thu, 21 Dec 2017 10:12:47 GMT
Right, we cannot distribute jcifs.jar for licensing reasons.  You can also
build ManifoldCF yourself from the distribution sources and libs and then
run "ant make-deps" to download the missing jars.  All of this is described
in the "how-to-build-and-deploy" page.

Thanks,
Karl


On Wed, Dec 20, 2017 at 11:25 PM, Shinichiro Abe <shinichiro.abe.1@gmail.com
> wrote:

> Hi,
>
> > 6. Created Repository connection of Type: “File System” (There was no
> windows share connector available in the drop down as stated in the
> documentation)
> LocalFileConnector does not get access tokens of windows shared files.
> To use SharedDriveConnector, you want to put the following for
> o.a.manifoldcf.connectorsconfigurationfile(i.e. connectors.xml) :
>   <repositoryconnector name="Windows shares" class="org.apache.manifoldcf.
> crawler.connectors.sharedrive.SharedDriveConnector"/>
> Then you need to download jcifs.jar and put into libdir(i.e.
> connector-lib).
>
> Regards,
> Shinichiro Abe
>
>
> 2017-12-21 11:40 GMT+09:00 Damien Collis <Damien.Collis@aas.com.au>:
>
>> Hi User Group,
>>
>>
>>
>> I am attempting to use Manifoldcf 2.8.1 and Solr 7.1.0 to index windows
>> file system documents.
>>
>> I am currently experiencing issues extracting the authority tokens,
>> essentially no security tokens are being propagated to Solr
>>
>>
>>
>> I have implemented the following to no success.
>>
>>
>>
>> 1. Added new Authority Group “LinkGroup”
>>
>> 2. Created an authority connection to  my AD domain controller associated
>> to the “LinkGroup” Authority Group – *Connection status: Connection
>> Working*
>>
>> 3. Tested the http://haystack:8345/mcf-combined-service-2.8.1/UserACLs?
>> username=user@domain and received:
>>
>> AUTHORIZED:LinkGroup
>>
>> TOKEN:LinkGroup:S-1-5-21-1537756157-1994918190-4060197294-17387
>>
>> TOKEN:LinkGroup:S-1-5-21-1537756157-1994918190-4060197294-1198
>>
>> TOKEN:LinkGroup:S-1-5-21-1537756157-1994918190-4060197294-1190
>>
>> ….
>>
>>
>>
>> 4. Added fields to the Solr Schema xml file.
>>
>> <field name="allow_token_document" type="string" indexed="true"
>> stored="true" multiValued="true" required="false" default="__nosecurity__"/>
>>
>> <field name="allow_token_parent" type="string" indexed="true"
>> stored="true" multiValued="true" required="false" default="__nosecurity__"/>
>>
>> <field name="allow_token_share" type="string" indexed="true"
>> stored="true" multiValued="true" required="false" default="__nosecurity__"/>
>>
>> <field name="deny_token_document" type="string" indexed="true"
>> stored="true" multiValued="true" required="false" default="__nosecurity__"/>
>>
>> <field name="deny_token_parent" type="string" indexed="true"
>> stored="true" multiValued="true" required="false" default="__nosecurity__"/>
>>
>> <field name="deny_token_share" type="string" indexed="true" stored="true"
>> multiValued="true" required="false" default="__nosecurity__"/>
>>
>>
>>
>> 5. Copied  apache-manifoldcf-solr-7.x-plugin-2.2.jar to
>>  D:\ProgramFiles\solr-7.1.0a\solr-7.1.0-bin\contrib\extraction\lib  (I
>> wasn’t sure of the exact location to copy this lib)
>>
>>
>>
>> 6. Created Repository connection of Type: “File System” (There was no
>> windows share connector available in the drop down as stated in the
>> documentation)
>>
>>
>>
>> 7. Created job to crawl LinkGroup file system.
>>
>>
>>
>>
>>
>> I can see the following in my Solr logs, I was expecting to see the
>> access tokens, but I’m not sure how that information is passed to Solr or
>> if it is presented in the logs:
>>
>>
>>
>> 2017-12-20 21:14:07.086 INFO  (qtp466002798-20) [   x:LinkGroup]
>> o.a.s.u.p.LogUpdateProcessorFactory [LinkGroup]  webapp=/solr
>> path=/update/extract params={literal.uri=\\serverna
>> me\HaystackTest\All.txt&resource.name=All.txt&literal.id=
>> file:////servername/HaystackTest/All.txt&wt=xml&version=2.2}
>> {add=[file:////servername/HaystackTest/All.txt (1587339011890872320)]} 0
>> 33
>>
>> 2017-12-20 21:14:07.102 INFO  (qtp466002798-19) [   x:LinkGroup]
>> o.a.s.u.p.LogUpdateProcessorFactory [LinkGroup]  webapp=/solr
>> path=/update/extract params={literal.uri=\\serverna
>> me\HaystackTest\secured.txt&resource.name=secured.txt&literal.id
>> =file:////servername/HaystackTest/secured.txt&wt=
>> xml&version=2.2}{add=[file:////servername/HaystackTest/secured.txt
>> (1587339011907649536)]} 0 46
>>
>> 2017-12-20 21:14:20.055 INFO  (qtp466002798-15) [   x:LinkGroup]
>> o.a.s.u.DirectUpdateHandler2 start commit{_version_=1587339025506
>> 631680,optimize=false,openSearcher=true,waitSearcher
>> =true,expungeDeletes=false,softCommit=false,prepareCommit=false}
>>
>> 2017-12-20 21:14:20.055 INFO  (qtp466002798-15) [   x:LinkGroup]
>> o.a.s.u.SolrIndexWriter Calling setCommitData with
>> IW:org.apache.solr.update.SolrIndexWriter@68f515e5
>> commitCommandVersion:1587339025506631680
>>
>> 2017-12-20 21:14:20.070 INFO  (qtp466002798-15) [   x:LinkGroup]
>> o.a.s.s.SolrIndexSearcher Opening [Searcher@30e03581[LinkGroup] main]
>>
>> 2017-12-20 21:14:20.070 INFO  (searcherExecutor-7-thread-1-processing-x:LinkGroup)
>> [   x:LinkGroup] o.a.s.c.QuerySenderListener QuerySenderListener sending
>> requests to Searcher@30e03581[LinkGroup] main{ExitableDirectoryReader(U
>> ninvertingDirectoryReader(Uninverting(_26(7.1.0):C2)
>> Uninverting(_27(7.1.0):C2)))}
>>
>> 2017-12-20 21:14:20.070 INFO  (qtp466002798-15) [   x:LinkGroup]
>> o.a.s.u.DirectUpdateHandler2 end_commit_flush
>>
>> 2017-12-20 21:14:20.070 INFO  (searcherExecutor-7-thread-1-processing-x:LinkGroup)
>> [   x:LinkGroup] o.a.s.c.QuerySenderListener QuerySenderListener done.
>>
>> 2017-12-20 21:14:20.070 INFO  (searcherExecutor-7-thread-1-processing-x:LinkGroup)
>> [   x:LinkGroup] o.a.s.c.SolrCore [LinkGroup] Registered new searcher
>> Searcher@30e03581[LinkGroup] main{ExitableDirectoryReader(U
>> ninvertingDirectoryReader(Uninverting(_26(7.1.0):C2)
>> Uninverting(_27(7.1.0):C2)))}
>>
>> 2017-12-20 21:14:20.070 INFO  (qtp466002798-15) [   x:LinkGroup]
>> o.a.s.u.p.LogUpdateProcessorFactory [LinkGroup]  webapp=/solr
>> path=/update/extract params={commit=true&wt=xml&version=2.2}{commit=} 0
>> 25
>>
>>
>>
>>
>>
>> Any assistance would be highly appreciated.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Regards
>>
>> *Damien Collis*
>> Team Leader – Systems Integration
>> Link Group
>>
>>
>> *   Level 4, 1A Homebush Bay Drive, Rhodes NSW 2138
>> <https://maps.google.com/?q=1A+Homebush+Bay+Drive,+Rhodes+NSW+2138&entry=gmail&source=g>
>> : Email: damien.collis@linkgroup.com
>> ( Ph: +61 2 8571 5616 <+61%202%208571%205616>
>>
>>
>>
>
>

Mime
View raw message