tika-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TIKA-2298) To improve object recognition parser so that it may work without external RESTful service setup
Date Wed, 21 Jun 2017 23:34:00 GMT

    [ https://issues.apache.org/jira/browse/TIKA-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16058435#comment-16058435
] 

ASF GitHub Bot commented on TIKA-2298:
--------------------------------------

thammegowda commented on a change in pull request #182: Creation of TIKA-2298 contributed
by asmehra95- Import of vgg16 via Deeplearning4j into tika-dl
URL: https://github.com/apache/tika/pull/182#discussion_r123379864
 
 

 ##########
 File path: tika-dl/src/main/java/org/apache/tika/dl/imagerec/DL4JVGG16Net.java
 ##########
 @@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.tika.dl.imagerec;
+
+import org.apache.tika.config.Field;
+import org.apache.tika.config.Param;
+import org.apache.tika.exception.TikaConfigException;
+import org.apache.tika.exception.TikaException;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.external.ExternalParser;
+import org.apache.tika.parser.recognition.ObjectRecogniser;
+import org.apache.tika.parser.recognition.RecognisedObject;
+import org.datavec.image.loader.NativeImageLoader;
+import org.deeplearning4j.nn.graph.ComputationGraph;
+import org.deeplearning4j.nn.modelimport.keras.trainedmodels.TrainedModelHelper;
+import org.deeplearning4j.nn.modelimport.keras.trainedmodels.TrainedModels;
+import org.deeplearning4j.util.ModelSerializer;
+import org.nd4j.linalg.api.ndarray.INDArray;
+import org.nd4j.linalg.dataset.api.preprocessor.DataNormalization;
+import org.nd4j.linalg.dataset.api.preprocessor.VGG16ImagePreProcessor;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.xml.sax.ContentHandler;
+import org.xml.sax.SAXException;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.*;
+import java.util.regex.Pattern;
+
+public class DL4JVGG16Net extends ExternalParser implements ObjectRecogniser {
+
+    private static final Logger LOG = LoggerFactory.getLogger(DL4JVGG16Net.class);
+    public static final Set<MediaType> SUPPORTED_MIMES = Collections.singleton(MediaType.image("jpeg"));
+    private static final LineConsumer IGNORED_LINE_LOGGER = new LineConsumer() {
+        @Override
+        public void consume(String line) {
+            LOG.debug(line);
+        }
+    };
+    private static final String HOME_DIR = System.getProperty("user.home");
+    private static final String BASE_DIR = ".dl4j/trainedmodels";
+    private static String MODEL_DIR = HOME_DIR + File.separator + BASE_DIR;
+    private static String MODEL_DIR_PREPROCESSED = MODEL_DIR + File.separator + "tikaPreprocessed"
+ File.separator;
+    @Field
+    private String modelType = "VGG16";
+    @Field
+    private File modelFile;
+    @Field
+    private String outPattern = "(.*) \\(score = ([0-9]+\\.[0-9]+)\\)$";
+    @Field
+    private String serialize = "yes";
+    private File locationToSave;
+    private boolean available = false;
+    private ComputationGraph model;
+
+    public Set<MediaType> getSupportedMimes() {
+        return SUPPORTED_MIMES;
+    }
+
+    @Override
+    public boolean isAvailable() {
+        return available;
+    }
+
+    @Override
+    public void initialize(Map<String, Param> params) throws TikaConfigException {
+        try {
+            TrainedModelHelper helper;
+            switch (modelType) {
+                case "VGG16NOTOP":
+                    throw new TikaConfigException("VGG16NOTOP is not supported right now");
+                /*# TODO hookup VGGNOTOP by uncommenting following code once the issue is
resolved by dl4j team
+                modelFile = new File(MODEL_DIR_PREPROCESSED+File.separator+"vgg16_notop.zip");
+				locationToSave= new File(MODEL_DIR+File.separator+"tikaPreprocessed"+File.separator+"vgg16.zip");
+                    helper = new TrainedModelHelper(TrainedModels.VGG16NOTOP);
+                    break;*/
+                case "VGG16":
+                    helper = new TrainedModelHelper(TrainedModels.VGG16);
+                    modelFile = new File(MODEL_DIR_PREPROCESSED + File.separator + "vgg16.zip");
+                    locationToSave = new File(MODEL_DIR + File.separator + "tikaPreprocessed"
+ File.separator + "vgg16.zip");
+                    break;
+                default:
+                    throw new TikaConfigException("Unknown or unsupported model");
+            }
+            if (serialize.trim().toLowerCase(Locale.ROOT).equals("yes")) {
+                if (!modelFile.exists()) {
+                    LOG.warn("Preprocessed Model doesn't exist at {}", modelFile);
+                    modelFile.getParentFile().mkdirs();
+                    model = helper.loadModel();
+                    LOG.info("Saving the Loaded model for future use. Saved models are more
optimised to consume less resources.");
+                    ModelSerializer.writeModel(model, locationToSave, true);
+                    available = true;
+                } else {
+                    model = ModelSerializer.restoreComputationGraph(locationToSave);
+                    LOG.info("Preprocessed Model Loaded from {}", locationToSave);
+                    available = true;
+                }
+
+            } else if (serialize.trim().toLowerCase(Locale.ROOT).equals("no")) {
+                LOG.info("Weight graph model loaded via dl4j Helper functions");
+                model = helper.loadModel();
+                available = true;
+            } else {
+                throw new TikaConfigException("Configuration Error. serialization can be
either yes or no.");
+            }
+
+            if (!available) {
+                return;
+            }
+            HashMap<Pattern, String> patterns = new HashMap<>();
+            patterns.put(Pattern.compile(outPattern), null);
+            setMetadataExtractionPatterns(patterns);
+            setIgnoredLineConsumer(IGNORED_LINE_LOGGER);
+        } catch (Exception e) {
+            LOG.warn("exception occured");
+            throw new TikaConfigException(e.getMessage(), e);
+        }
+    }
+
+    @Override
+    public List<RecognisedObject> recognise(InputStream stream, ContentHandler handler,
+                                            Metadata metadata, ParseContext context)
+            throws IOException, SAXException, TikaException {
+        NativeImageLoader loader = new NativeImageLoader(224, 224, 3);
 
 Review comment:
   Can the `loader` object be declared as a member variable and then reused it here?
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> To improve object recognition parser so that it may work without external RESTful service
setup
> -----------------------------------------------------------------------------------------------
>
>                 Key: TIKA-2298
>                 URL: https://issues.apache.org/jira/browse/TIKA-2298
>             Project: Tika
>          Issue Type: Improvement
>          Components: parser
>    Affects Versions: 1.14
>            Reporter: Avtar Singh
>              Labels: ObjectRecognitionParser
>             Fix For: 1.16
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> When ObjectRecognitionParser was built to do image recognition, there wasn't
> good support for Java frameworks.  All the popular neural networks were in
> C++ or python.  Since there was nothing that runs within JVM, we tried
> several ways to glue them to Tika (like CLI, JNI, gRPC, REST).
> However, this game is changing slowly now. Deeplearning4j, the most famous
> neural network library for JVM, now supports importing models that are
> pre-trained in python/C++ based kits [5].
> *Improvement:*
> It will be nice to have an implementation of ObjectRecogniser that
> doesn't require any external setup(like installation of native libraries or
> starting REST services). Reasons: easy to distribute and also to cut the IO
> time.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message