manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shinichiro Abe <shinichiro.ab...@gmail.com>
Subject Re: Issue Extracting Authorities.
Date Thu, 21 Dec 2017 04:25:13 GMT
Hi,

> 6. Created Repository connection of Type: “File System” (There was no
windows share connector available in the drop down as stated in the
documentation)
LocalFileConnector does not get access tokens of windows shared files.
To use SharedDriveConnector, you want to put the following for
o.a.manifoldcf.connectorsconfigurationfile(i.e. connectors.xml) :
  <repositoryconnector name="Windows shares"
class="org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector"/>
Then you need to download jcifs.jar and put into libdir(i.e. connector-lib).

Regards,
Shinichiro Abe


2017-12-21 11:40 GMT+09:00 Damien Collis <Damien.Collis@aas.com.au>:

> Hi User Group,
>
>
>
> I am attempting to use Manifoldcf 2.8.1 and Solr 7.1.0 to index windows
> file system documents.
>
> I am currently experiencing issues extracting the authority tokens,
> essentially no security tokens are being propagated to Solr
>
>
>
> I have implemented the following to no success.
>
>
>
> 1. Added new Authority Group “LinkGroup”
>
> 2. Created an authority connection to  my AD domain controller associated
> to the “LinkGroup” Authority Group – *Connection status: Connection
> Working*
>
> 3. Tested the http://haystack:8345/mcf-combined-service-2.8.1/UserACLs?
> username=user@domain and received:
>
> AUTHORIZED:LinkGroup
>
> TOKEN:LinkGroup:S-1-5-21-1537756157-1994918190-4060197294-17387
>
> TOKEN:LinkGroup:S-1-5-21-1537756157-1994918190-4060197294-1198
>
> TOKEN:LinkGroup:S-1-5-21-1537756157-1994918190-4060197294-1190
>
> ….
>
>
>
> 4. Added fields to the Solr Schema xml file.
>
> <field name="allow_token_document" type="string" indexed="true"
> stored="true" multiValued="true" required="false" default="__nosecurity__"/>
>
> <field name="allow_token_parent" type="string" indexed="true"
> stored="true" multiValued="true" required="false" default="__nosecurity__"/>
>
> <field name="allow_token_share" type="string" indexed="true" stored="true"
> multiValued="true" required="false" default="__nosecurity__"/>
>
> <field name="deny_token_document" type="string" indexed="true"
> stored="true" multiValued="true" required="false" default="__nosecurity__"/>
>
> <field name="deny_token_parent" type="string" indexed="true" stored="true"
> multiValued="true" required="false" default="__nosecurity__"/>
>
> <field name="deny_token_share" type="string" indexed="true" stored="true"
> multiValued="true" required="false" default="__nosecurity__"/>
>
>
>
> 5. Copied  apache-manifoldcf-solr-7.x-plugin-2.2.jar to
>  D:\ProgramFiles\solr-7.1.0a\solr-7.1.0-bin\contrib\extraction\lib  (I
> wasn’t sure of the exact location to copy this lib)
>
>
>
> 6. Created Repository connection of Type: “File System” (There was no
> windows share connector available in the drop down as stated in the
> documentation)
>
>
>
> 7. Created job to crawl LinkGroup file system.
>
>
>
>
>
> I can see the following in my Solr logs, I was expecting to see the access
> tokens, but I’m not sure how that information is passed to Solr or if it is
> presented in the logs:
>
>
>
> 2017-12-20 21:14:07.086 INFO  (qtp466002798-20) [   x:LinkGroup] o.a.s.u.p.LogUpdateProcessorFactory
> [LinkGroup]  webapp=/solr path=/update/extract params={literal.uri=\\
> servername\HaystackTest\All.txt&resource.name=All.txt&literal.id
> =file:////servername/HaystackTest/All.txt&wt=xml&
> version=2.2}{add=[file:////servername/HaystackTest/All.txt
> (1587339011890872320)]} 0 33
>
> 2017-12-20 21:14:07.102 INFO  (qtp466002798-19) [   x:LinkGroup] o.a.s.u.p.LogUpdateProcessorFactory
> [LinkGroup]  webapp=/solr path=/update/extract params={literal.uri=\\
> servername\HaystackTest\secured.txt&resource.name=secured.txt&literal.id
> =file:////servername/HaystackTest/secured.txt&wt=xml&version=2.
> 2}{add=[file:////servername/HaystackTest/secured.txt
> (1587339011907649536)]} 0 46
>
> 2017-12-20 21:14:20.055 INFO  (qtp466002798-15) [   x:LinkGroup]
> o.a.s.u.DirectUpdateHandler2 start commit{_version_=
> 1587339025506631680,optimize=false,openSearcher=true,waitSearcher=true,
> expungeDeletes=false,softCommit=false,prepareCommit=false}
>
> 2017-12-20 21:14:20.055 INFO  (qtp466002798-15) [   x:LinkGroup]
> o.a.s.u.SolrIndexWriter Calling setCommitData with
> IW:org.apache.solr.update.SolrIndexWriter@68f515e5 commitCommandVersion:
> 1587339025506631680
>
> 2017-12-20 21:14:20.070 INFO  (qtp466002798-15) [   x:LinkGroup]
> o.a.s.s.SolrIndexSearcher Opening [Searcher@30e03581[LinkGroup] main]
>
> 2017-12-20 21:14:20.070 INFO  (searcherExecutor-7-thread-1-processing-x:LinkGroup)
> [   x:LinkGroup] o.a.s.c.QuerySenderListener QuerySenderListener sending
> requests to Searcher@30e03581[LinkGroup] main{ExitableDirectoryReader(
> UninvertingDirectoryReader(Uninverting(_26(7.1.0):C2)
> Uninverting(_27(7.1.0):C2)))}
>
> 2017-12-20 21:14:20.070 INFO  (qtp466002798-15) [   x:LinkGroup]
> o.a.s.u.DirectUpdateHandler2 end_commit_flush
>
> 2017-12-20 21:14:20.070 INFO  (searcherExecutor-7-thread-1-processing-x:LinkGroup)
> [   x:LinkGroup] o.a.s.c.QuerySenderListener QuerySenderListener done.
>
> 2017-12-20 21:14:20.070 INFO  (searcherExecutor-7-thread-1-processing-x:LinkGroup)
> [   x:LinkGroup] o.a.s.c.SolrCore [LinkGroup] Registered new searcher
> Searcher@30e03581[LinkGroup] main{ExitableDirectoryReader(
> UninvertingDirectoryReader(Uninverting(_26(7.1.0):C2)
> Uninverting(_27(7.1.0):C2)))}
>
> 2017-12-20 21:14:20.070 INFO  (qtp466002798-15) [   x:LinkGroup] o.a.s.u.p.LogUpdateProcessorFactory
> [LinkGroup]  webapp=/solr path=/update/extract params={commit=true&wt=xml&version=2.2}{commit=}
> 0 25
>
>
>
>
>
> Any assistance would be highly appreciated.
>
>
>
>
>
>
>
>
>
> Regards
>
> *Damien Collis*
> Team Leader – Systems Integration
> Link Group
>
>
> *   Level 4, 1A Homebush Bay Drive, Rhodes NSW 2138
> <https://maps.google.com/?q=1A+Homebush+Bay+Drive,+Rhodes+NSW+2138&entry=gmail&source=g>
> : Email: damien.collis@linkgroup.com
> ( Ph: +61 2 8571 5616 <+61%202%208571%205616>
>
>
>

Mime
View raw message