From dev-return-25684-apmail-tika-dev-archive=tika.apache.org@tika.apache.org Fri Jun 9 19:42:38 2017 Return-Path: X-Original-To: apmail-tika-dev-archive@www.apache.org Delivered-To: apmail-tika-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CA1AA193B4 for ; Fri, 9 Jun 2017 19:42:37 +0000 (UTC) Received: (qmail 90055 invoked by uid 500); 9 Jun 2017 19:42:37 -0000 Delivered-To: apmail-tika-dev-archive@tika.apache.org Received: (qmail 89955 invoked by uid 500); 9 Jun 2017 19:42:37 -0000 Mailing-List: contact dev-help@tika.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tika.apache.org Delivered-To: mailing list dev@tika.apache.org Received: (qmail 89922 invoked by uid 99); 9 Jun 2017 19:42:37 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Jun 2017 19:42:37 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id D9EF0C09D8 for ; Fri, 9 Jun 2017 19:42:36 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.211 X-Spam-Level: X-Spam-Status: No, score=-99.211 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id cdxWZsoofmLm for ; Fri, 9 Jun 2017 19:42:36 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 674FD61161 for ; Fri, 9 Jun 2017 19:42:27 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id D5DFCE0DE8 for ; Fri, 9 Jun 2017 19:42:26 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id B9D7921E24 for ; Fri, 9 Jun 2017 19:42:24 +0000 (UTC) Date: Fri, 9 Jun 2017 19:42:24 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: dev@tika.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (TIKA-2262) Supporting Image-to-Text (Image Captioning) in Tika for Image MIME Types MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/TIKA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16044895#comment-16044895 ] ASF GitHub Bot commented on TIKA-2262: -------------------------------------- thammegowda commented on a change in pull request #180: Fix for TIKA-2262: Supporting Image-to-Text (Image Captioning) in Tika URL: https://github.com/apache/tika/pull/180#discussion_r121201948 ########## File path: tika-parsers/src/main/java/org/apache/tika/parser/recognition/ObjectRecogniser.java ########## @@ -69,7 +69,7 @@ * @throws SAXException when an issue with XML occurs * @throws TikaException any generic error */ - List recognise(InputStream stream, ContentHandler handler, + List recognise(InputStream stream, ContentHandler handler, Review comment: well done :+1: ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org > Supporting Image-to-Text (Image Captioning) in Tika for Image MIME Types > ------------------------------------------------------------------------ > > Key: TIKA-2262 > URL: https://issues.apache.org/jira/browse/TIKA-2262 > Project: Tika > Issue Type: Improvement > Components: parser > Reporter: Thamme Gowda > Labels: deeplearning, gsoc2017, machine_learning > > h2. Background: > Image captions are a small piece of text, usually of one line, added to the metadata of images to provide a brief summary of the scenery in the image. > It is a challenging and interesting problem in the domain of computer vision. Tika already has a support for image recognition via [Object Recognition Parser, TIKA-1993| https://issues.apache.org/jira/browse/TIKA-1993] which uses an InceptionV3 model pre-trained on ImageNet dataset using tensorflow. > Captioning an image is a very useful feature since it helps text based Information Retrieval(IR) systems to "understand" the scenery in images. > h2. Technical details and references: > * Google has long back open sourced their 'show and tell' neural network and its model for autogenerating captions. [Source Code| https://github.com/tensorflow/models/tree/master/im2txt], [Research blog| https://research.googleblog.com/2016/09/show-and-tell-image-captioning-open.html] > * Integrate it the same way as the ObjectRecognitionParser > ** Create a RESTful API Service [similar to this| https://wiki.apache.org/tika/TikaAndVision#A2._Tensorflow_Using_REST_Server] > ** Extend or enhance ObjectRecognitionParser or one of its implementation > h2. {skills, learning, homework} for GSoC students > * Knowledge of languages: java AND python, and maven build system > * RESTful APIs > * tensorflow/keras, > * deeplearning > ---- > Alternatively, a little more harder path for experienced: > [Import keras/tensorflow model to deeplearning4j|https://deeplearning4j.org/model-import-keras ] and run them natively inside JVM. > h4. Benefits > * no RESTful integration required. thus no external dependencies > * easy to distribute on hadoop/spark clusters > h4. Hurdles: > * This is a work in progress feature on deeplearning4j and hence expected to have lots of troubles on the way! -- This message was sent by Atlassian JIRA (v6.3.15#6346)