manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Query in Sharepoint connector
Date Thu, 10 Jul 2014 15:58:49 GMT
Hi Ameya,

Tika is running on Solr in this case.  I don't know how you'd explore what
it was doing in that context.  That query would be best directed at one of
the Solr lists.

Thanks,
Karl



On Thu, Jul 10, 2014 at 11:35 AM, Ameya Aware <ameya.aware@gmail.com> wrote:

> Hi Karl,
>
> In which file i can see how Tika is is extracting metadata for file on
> SharePoint server?
>
> Thanks,
> Ameya
>
>
> On Wed, Jul 9, 2014 at 4:26 PM, Karl Wright <daddywri@gmail.com> wrote:
>
>> This is a result of there being metadata in a docx file that Tika can
>> extract, in addition to the metadata from SharePoint.
>>
>> Karl
>>
>>
>>
>> On Wed, Jul 9, 2014 at 4:23 PM, Ameya Aware <ameya.aware@gmail.com>
>> wrote:
>>
>>> Ok sure. Thanks Karl.
>>>
>>> I have one more query regarding SP crawling.
>>>
>>> If you see attached file, you will see that .html files gives very fewer
>>> metadata than .docx/.pptx files.
>>>
>>> Has the coding being done in this way?
>>>
>>> Thanks,
>>> Ameya
>>>
>>>
>>>
>>> On Wed, Jul 9, 2014 at 4:13 PM, Karl Wright <daddywri@gmail.com> wrote:
>>>
>>>> That's up to you and the Solr Cell people.  It's not an MCF problem.  I
>>>> suggest posting a query to the Solr mailing list.
>>>>
>>>> Karl
>>>>
>>>>
>>>>
>>>> On Wed, Jul 9, 2014 at 4:04 PM, Ameya Aware <ameya.aware@gmail.com>
>>>> wrote:
>>>>
>>>>> So what could be the solution on this now?
>>>>>
>>>>> Thanks,
>>>>> Ameya
>>>>>
>>>>>
>>>>> On Wed, Jul 9, 2014 at 4:00 PM, Karl Wright <daddywri@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ameya,
>>>>>>
>>>>>> Have a look at this one line:
>>>>>>
>>>>>> INFO  - 2014-07-09 15:30:56.243;
>>>>>> org.apache.solr.update.processor.LogUpdateProcessor; [collection1]
>>>>>> webapp=/solr path=/update/extract
>>>>>> params={literal.deny_token_document=DEAD_AUTHORITY&literal.DocIcon=docx&
>>>>>> resource.name
>>>>>> =Anarchism-201310091123505625.docx&literal.allow_token_document=S-1-5-21-1482846375-2276666860-3536682573-500&literal.allow_token_document=S-1-5-21-1482846375-2276666860-3536682573-68651&literal.FolderChildCount=0&version=2.2&literal.ItemChildCount=0&literal.GUID=Ameya&literal.ParentVersionString=&literal._CopySource=&
>>>>>> literal.cat
>>>>>> =&literal.FileSizeDisplay=1264155&literal._CheckinComment=&literal.Edit=0&
>>>>>> literal.id=
>>>>>> http://sharepointten:10800/sites/siteecho/Shared%2520Documents/Anarchism-201310091123505625.docx&literal.LinkFilenameNoMenu=Anarchism-201310091123505625.docx&literal.Created=2014-06-03+11:21:53&literal._UIVersionString=1.0&wt=xml&literal.Title=Anarchism&literal.Modified=2014-06-03+11:21:53&literal.Author=Sharepoint+Backup&literal.FileLeafRef=Anarchism-201310091123505625.docx&literal.LinkFilename=Anarchism-201310091123505625.docx&literal.lcf_metadata_id=81&literal.Editor=Administrator&literal.ParentLeafName=&literal.CheckoutUser=&literal.ContentType=Document}
>>>>>> {add=[
>>>>>> http://sharepointten:10800/sites/siteecho/Shared%20Documents/Anarchism-201310091123505625.docx
>>>>>> (1473180342671114240)]} 0 66
>>>>>>
>>>>>>
>>>>>> Note that literal.Created and literal.Modified are both reasonable
>>>>>> dates.
>>>>>>
>>>>>> So MCF is not supplying the last_modified field; probably that is
>>>>>> being extracted by Solr Cell.
>>>>>>
>>>>>> Karl
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jul 9, 2014 at 3:33 PM, Ameya Aware <ameya.aware@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Karl,
>>>>>>>
>>>>>>> Please find attached solr log file.
>>>>>>>
>>>>>>> Please let me know if you need anything else.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ameya
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jul 9, 2014 at 3:07 PM, Karl Wright <daddywri@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Ameya,
>>>>>>>>
>>>>>>>> I need to see the solr log output.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Karl
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jul 9, 2014 at 3:04 PM, Ameya Aware <ameya.aware@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Please see if you needed below information.
>>>>>>>>>
>>>>>>>>> [image: Inline image 1]
>>>>>>>>>
>>>>>>>>> Or please let me know where would i get information from.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Ameya
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jul 9, 2014 at 2:51 PM, Karl Wright <daddywri@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Ameya,
>>>>>>>>>>
>>>>>>>>>> Can you include the Solr [INFO] log entry for one of these
>>>>>>>>>> indexing actions?  I want to see if last_modified is set incorrectly
>>>>>>>>>> there.  If you are running these documents through Solr Cell, it may well
>>>>>>>>>> be Tika that is providing the last_modified date, not ManifoldCF.
>>>>>>>>>>
>>>>>>>>>> Karl
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Jul 9, 2014 at 2:37 PM, Ameya Aware <
>>>>>>>>>> ameya.aware@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Karl,
>>>>>>>>>>>
>>>>>>>>>>> Please find below screenshot.
>>>>>>>>>>>
>>>>>>>>>>> [image: Inline image 1]
>>>>>>>>>>>
>>>>>>>>>>> looks like indexing date comes out to be good.
>>>>>>>>>>>
>>>>>>>>>>> Also, just for your reference pasting my Solr screenshot as well.
>>>>>>>>>>>
>>>>>>>>>>> [image: Inline image 2]
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Ameya
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jul 9, 2014 at 2:20 PM, Karl Wright <daddywri@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> On second look, it all looks fine.
>>>>>>>>>>>>
>>>>>>>>>>>> The last thing to check is to look at what is getting set as
>>>>>>>>>>>> data.  Around line 1996 in SharePointRepository, there is this code:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> >>>>>>
>>>>>>>>>>>>                 if (modifiedDate != null)
>>>>>>>>>>>>                   data.setModifiedDate(modifiedDate);
>>>>>>>>>>>>                 if (createdDate != null)
>>>>>>>>>>>>                   data.setCreatedDate(createdDate);
>>>>>>>>>>>> <<<<<<
>>>>>>>>>>>>
>>>>>>>>>>>> Can you add this line:
>>>>>>>>>>>>
>>>>>>>>>>>> >>>>>>
>>>>>>>>>>>>                System.out.println("Indexing modified date:
>>>>>>>>>>>> "+modifiedDate);
>>>>>>>>>>>> <<<<<<
>>>>>>>>>>>>
>>>>>>>>>>>> ... and recrawl?
>>>>>>>>>>>>
>>>>>>>>>>>> If that works, we'll have to start looking at Solr.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Karl
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jul 9, 2014 at 2:13 PM, Karl Wright <daddywri@gmail.com
>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Actually, looking at your screen shot, it is harder to see for
>>>>>>>>>>>>> sure since there are multiple threads active.  So it may well be that there
>>>>>>>>>>>>> is no issue with the parsing.  Let me see if I can confirm that.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 2:05 PM, Karl Wright <
>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://issues.apache.org/jira
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 2:04 PM, Ameya Aware <
>>>>>>>>>>>>>> ameya.aware@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> how do i open the ticket?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Ameya
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 2:03 PM, Karl Wright <
>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> bq. How is that coming good?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> No idea.  It may be a bug in the SimpleDateFormat class
>>>>>>>>>>>>>>>> pertaining to only specific dates.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 2:01 PM, Ameya Aware <
>>>>>>>>>>>>>>>> ameya.aware@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ok.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> But then same thing should happen in created date also ,
>>>>>>>>>>>>>>>>> isnt it?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> How is that coming good?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Ameya
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 1:56 PM, Karl Wright <
>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> This shows clearly that the parsing is not doing the
>>>>>>>>>>>>>>>>>> expected thing.  It's not clear why, since it's a pretty straight usage of
>>>>>>>>>>>>>>>>>> SimpleDateFormat, but that is what is going wrong.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please open a ticket for us to look at this.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 1:53 PM, Ameya Aware <
>>>>>>>>>>>>>>>>>> ameya.aware@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Karl,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Please find screenshot below to show modified date
>>>>>>>>>>>>>>>>>>> values as date object and string as well.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [image: Inline image 1]
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>> Ameya
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 12:32 PM, Karl Wright <
>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi Ameya,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> The ability to parse Microsoft's special 8601 dates is
>>>>>>>>>>>>>>>>>>>> in fact already there.  So what might be happening is a timezone issue,
>>>>>>>>>>>>>>>>>>>> since the timezone is not being explicitly set during parsing.  Printing
>>>>>>>>>>>>>>>>>>>> the value of modifiedDateValue will show us if that is indeed the problem.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 12:19 PM, Karl Wright <
>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi Ameya,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Try printing "modifiedDateValue", in addition to
>>>>>>>>>>>>>>>>>>>>> printing "modifiedDate".  The parsed form is a date object, not a string.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 11:20 AM, Ameya Aware <
>>>>>>>>>>>>>>>>>>>>> ameya.aware@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Ok.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> But created date for all files is coming good.
>>>>>>>>>>>>>>>>>>>>>> Also,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> if (modifyDate != null)
>>>>>>>>>>>>>>>>>>>>>>                 {
>>>>>>>>>>>>>>>>>>>>>>                   // Item has a modified date, so we
>>>>>>>>>>>>>>>>>>>>>> presume it exists
>>>>>>>>>>>>>>>>>>>>>>                   Date modifiedDateValue =
>>>>>>>>>>>>>>>>>>>>>> DateParser.parseISO8601Date(modifiedDate);
>>>>>>>>>>>>>>>>>>>>>>                   Date createdDateValue =
>>>>>>>>>>>>>>>>>>>>>> DateParser.parseISO8601Date(createdDate);
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>                   System.out.println("Modified date
>>>>>>>>>>>>>>>>>>>>>> string is: '"+modifiedDate+"'");
>>>>>>>>>>>>>>>>>>>>>>                   System.out.println("Modify Date:" +
>>>>>>>>>>>>>>>>>>>>>> modifyDate);
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> <<<<<<<<
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Above code prints out correct date even after parsing.
>>>>>>>>>>>>>>>>>>>>>> So is the issue coming after this step??
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I am using Sharepoint 2010.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>> Ameya
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 11:10 AM, Karl Wright <
>>>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Ok, well SharePoint claims these dates are ISO8601
>>>>>>>>>>>>>>>>>>>>>>> dates, but they are clearly not in this case.  Here are the tests for 8601
>>>>>>>>>>>>>>>>>>>>>>> dates in the MCF core code:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> >>>>>>
>>>>>>>>>>>>>>>>>>>>>>>     Date d =
>>>>>>>>>>>>>>>>>>>>>>> DateParser.parseISO8601Date("96-11-15T01:32:33.344GMT");
>>>>>>>>>>>>>>>>>>>>>>>     assertNotNull(d);
>>>>>>>>>>>>>>>>>>>>>>>     d =
>>>>>>>>>>>>>>>>>>>>>>> DateParser.parseISO8601Date("2012-11-15T01:32:33.344Z");
>>>>>>>>>>>>>>>>>>>>>>>     assertNotNull(d);
>>>>>>>>>>>>>>>>>>>>>>>     d =
>>>>>>>>>>>>>>>>>>>>>>> DateParser.parseISO8601Date("2012-11-15T01:32:33Z");
>>>>>>>>>>>>>>>>>>>>>>>     assertNotNull(d);
>>>>>>>>>>>>>>>>>>>>>>>     d =
>>>>>>>>>>>>>>>>>>>>>>> DateParser.parseISO8601Date("2012-11-15T01:32:33+0100");
>>>>>>>>>>>>>>>>>>>>>>>     assertNotNull(d);
>>>>>>>>>>>>>>>>>>>>>>>     d =
>>>>>>>>>>>>>>>>>>>>>>> DateParser.parseISO8601Date("2012-11-15T01:32:33-03:00");
>>>>>>>>>>>>>>>>>>>>>>>     assertNotNull(d);
>>>>>>>>>>>>>>>>>>>>>>>     d =
>>>>>>>>>>>>>>>>>>>>>>> DateParser.parseISO8601Date("2012-11-15T01:32:33GMT-03:00");
>>>>>>>>>>>>>>>>>>>>>>>     assertNotNull(d);
>>>>>>>>>>>>>>>>>>>>>>>     d =
>>>>>>>>>>>>>>>>>>>>>>> DateParser.parseISO8601Date("2012-11-15T01:32:33.001-04:00");
>>>>>>>>>>>>>>>>>>>>>>>     assertNotNull(d);
>>>>>>>>>>>>>>>>>>>>>>> <<<<<<
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> You will note that there is supposed to be a "T" and
>>>>>>>>>>>>>>>>>>>>>>> a timezone in an ISO-8601 date.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> What version of SharePoint are you using, and what
>>>>>>>>>>>>>>>>>>>>>>> is the locale settings for the server that your SharePoint is running on?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 11:06 AM, Ameya Aware <
>>>>>>>>>>>>>>>>>>>>>>> ameya.aware@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Please find below screenshot for dates.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Modify Date i added on my own.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> [image: Inline image 1]
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>> Ameya
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 11:03 AM, Karl Wright <
>>>>>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Hi Ameya,
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> The SharePoint connector parses the date.  Can you
>>>>>>>>>>>>>>>>>>>>>>>>> send me some EXAMPLES of the dates coming back so that I can be sure they
>>>>>>>>>>>>>>>>>>>>>>>>> will parse correctly?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:59 AM, Ameya Aware <
>>>>>>>>>>>>>>>>>>>>>>>>> ameya.aware@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Karl,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I ran the job and at this point values coming for
>>>>>>>>>>>>>>>>>>>>>>>>>> modified date are correct.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Ameya
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:25 AM, Karl Wright <
>>>>>>>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Ameya,
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> I will provide instructions for how I'd like you
>>>>>>>>>>>>>>>>>>>>>>>>>>> to research this.  I don't suggest running under eclipse for this research.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Around line 1047 in SharePointRepository.java,
>>>>>>>>>>>>>>>>>>>>>>>>>>> there is this code:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>                 String modifiedDate =
>>>>>>>>>>>>>>>>>>>>>>>>>>> values.get("Modified");
>>>>>>>>>>>>>>>>>>>>>>>>>>>                 String createdDate =
>>>>>>>>>>>>>>>>>>>>>>>>>>> values.get("Created");
>>>>>>>>>>>>>>>>>>>>>>>>>>>                 String guid = values.get("GUID");
>>>>>>>>>>>>>>>>>>>>>>>>>>>                 String modifyDate =
>>>>>>>>>>>>>>>>>>>>>>>>>>> values.get("Last_x0020_Modified");
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> <<<<<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Please add this line:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>                System.out.println("Modified date
>>>>>>>>>>>>>>>>>>>>>>>>>>> string is: '"+modifiedDate+"'");
>>>>>>>>>>>>>>>>>>>>>>>>>>> <<<<<
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Please run the job and send me some examples of
>>>>>>>>>>>>>>>>>>>>>>>>>>> the modified date string.
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:13 AM, Ameya Aware <
>>>>>>>>>>>>>>>>>>>>>>>>>>> ameya.aware@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> i am finding it difficult debug the code..
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Whatever changes i do,i just build from scratch and check if changes are
>>>>>>>>>>>>>>>>>>>>>>>>>>>> done or not.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you help me with how can i debug this code?
>>>>>>>>>>>>>>>>>>>>>>>>>>>> (i am using eclipse IDE)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Ameya
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 10:08 AM, Karl Wright <
>>>>>>>>>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Ameya,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Other users have had no trouble with this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> attribute in the past.  SharePoint, though, has been known to use
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> non-ISO-8601-format dates in some cases.  I wonder if this is one of those
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cases?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In order to determine this, you will need to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> edit the code for the SharePoint connector and add debugging output.  Are
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you in a position to do that?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jul 9, 2014 at 9:51 AM, Ameya Aware <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ameya.aware@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Karl,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am seeing Shared documents from Sharepoint.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Ameya
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jul 8, 2014 at 5:52 PM, Karl Wright <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Ameya,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> There is no difference in treatment between
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> created date and modified date that I can find in the connector.  Can you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tell me what kind of SharePoint entity you are seeing this on?  Eg
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> documents, list items, attachments?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent from my Windows Phone
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ------------------------------
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: Ameya Aware
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: 7/8/2014 3:41 PM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: Karl Wright
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: Query in Sharepoint connector
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>  I did not get you properly. Please see
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> below if it satisfies your query.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Last modified is date field in Sharepoint.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> When i run job and send metadata to Solr, the date which is being sent to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Solr is far different than that of it is in Sharepoint.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please let me know if you need any more
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> details.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Ameya
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Jul 8, 2014 at 3:35 PM, Karl Wright
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <daddywri@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What does this field look like in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> SharePoint?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Karl
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent from my Windows Phone
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: Ameya Aware
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sent: 7/8/2014 1:50 PM
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: dev@manifoldcf.apache.org
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Query in Sharepoint connector
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Last_modified metadata sent from Sharepoint
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to Solr is not giving correct
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> values.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Even last_modified showing lesser value
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> than created_by date. (Created_by
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> date is coming good).
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is this bug?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Ameya
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message