flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] dianfu commented on a change in pull request #12422: [FLINK-17901][python] Add module interface in PyFlink
Date Wed, 03 Jun 2020 02:41:51 GMT

dianfu commented on a change in pull request #12422:
URL: https://github.com/apache/flink/pull/12422#discussion_r434276134



##########
File path: flink-python/pyflink/table/tests/test_table_environment_api.py
##########
@@ -628,6 +628,15 @@ def test_table_environment_with_blink_planner(self):
 
         self.assert_equals(results, ['2,hi,hello\n', '3,hello,hello\n'])
 
+    def test_unload_and_load_module(self):

Review comment:
       Move it to class TableEnvironmentTest?

##########
File path: flink-python/pyflink/table/module.py
##########
@@ -0,0 +1,77 @@
+################################################################################
+#  Licensed to the Apache Software Foundation (ASF) under one
+#  or more contributor license agreements.  See the NOTICE file
+#  distributed with this work for additional information
+#  regarding copyright ownership.  The ASF licenses this file
+#  to you under the Apache License, Version 2.0 (the
+#  "License"); you may not use this file except in compliance
+#  with the License.  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+from pyflink.java_gateway import get_gateway
+from pyflink.table.function_definition import FunctionDefinition
+
+__all__ = ['Module', 'HiveModule']
+
+
+class Module(object):
+    """
+    Modules define a set of metadata, including functions, user defined types, operators,
rules,
+    etc.
+    Metadata from modules are regarded as built-in or system metadata that users can take
advantages
+    of.
+    """
+
+    def __init__(self, j_module):
+        self._j_module = j_module
+
+    @staticmethod
+    def _get(j_module):
+        if j_module.getClass().getName() == 'org.apache.flink.table.module.hive.HiveModule':
+            return HiveModule(j_hive_module=j_module)
+        else:
+            return Module(j_module)
+
+    def list_functions(self):

Review comment:
       This interfaces `list_functions` and `get_function_definition` are designed to used
by:
   - Internally by the ModuleManager
   - Used to create custom Module
   
   So I think there is no need to expose them to Python users and we should remove them(and
also the Python FunctionDefinition class).

##########
File path: flink-python/pyflink/table/table_environment.py
##########
@@ -148,6 +147,29 @@ def get_catalog(self, catalog_name):
         else:
             return None
 
+    def load_module(self, module_name, module):
+        """
+        Loads a :class:`~pyflink.table.Module` under a unique name. Modules will be kept
+        in the loaded order.
+        ValidationException is thrown when there is already a module with the same name.
+
+        :param module_name: Name of the :class:`~pyflink.table.Module`.
+        :type module_name: str
+        :param module: The module instance.
+        :type module: pyflink.table.Module

Review comment:
       Add the version information
   ```
   .. versionadded:: 1.12.0
   ```

##########
File path: flink-python/pyflink/table/table_environment.py
##########
@@ -148,6 +147,29 @@ def get_catalog(self, catalog_name):
         else:
             return None
 
+    def load_module(self, module_name, module):
+        """
+        Loads a :class:`~pyflink.table.Module` under a unique name. Modules will be kept
+        in the loaded order.
+        ValidationException is thrown when there is already a module with the same name.
+
+        :param module_name: Name of the :class:`~pyflink.table.Module`.
+        :type module_name: str
+        :param module: The module instance.
+        :type module: pyflink.table.Module
+        """
+        self._j_tenv.loadModule(module_name, module._j_module)
+
+    def unload_module(self, module_name):
+        """
+        Unloads a :class:`~pyflink.table.Module` with given name.
+        ValidationException is thrown when there is no module with the given name.
+
+        :param module_name: Name of the :class:`~pyflink.table.Module`.
+        :type module_name: str
+        """
+        self._j_tenv.unloadModule(module_name)

Review comment:
       ditto

##########
File path: flink-python/pyflink/table/module.py
##########
@@ -0,0 +1,77 @@
+################################################################################
+#  Licensed to the Apache Software Foundation (ASF) under one
+#  or more contributor license agreements.  See the NOTICE file
+#  distributed with this work for additional information
+#  regarding copyright ownership.  The ASF licenses this file
+#  to you under the Apache License, Version 2.0 (the
+#  "License"); you may not use this file except in compliance
+#  with the License.  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+from pyflink.java_gateway import get_gateway
+from pyflink.table.function_definition import FunctionDefinition
+
+__all__ = ['Module', 'HiveModule']
+
+
+class Module(object):
+    """
+    Modules define a set of metadata, including functions, user defined types, operators,
rules,
+    etc.
+    Metadata from modules are regarded as built-in or system metadata that users can take
advantages
+    of.
+    """
+
+    def __init__(self, j_module):
+        self._j_module = j_module
+
+    @staticmethod
+    def _get(j_module):

Review comment:
       It seems that this method could be removed. What do you think?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message