flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] sunjincheng121 commented on a change in pull request #8474: [FLINK-12409][python] Adds from_elements in TableEnvironment
Date Mon, 27 May 2019 08:56:01 GMT
sunjincheng121 commented on a change in pull request #8474: [FLINK-12409][python] Adds from_elements
in TableEnvironment
URL: https://github.com/apache/flink/pull/8474#discussion_r287636618
 
 

 ##########
 File path: flink-python/pyflink/serializers.py
 ##########
 @@ -0,0 +1,219 @@
+################################################################################
+#  Licensed to the Apache Software Foundation (ASF) under one
+#  or more contributor license agreements.  See the NOTICE file
+#  distributed with this work for additional information
+#  regarding copyright ownership.  The ASF licenses this file
+#  to you under the Apache License, Version 2.0 (the
+#  "License"); you may not use this file except in compliance
+#  with the License.  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+
+import sys
+import struct
+from abc import ABCMeta, abstractmethod
+
+if sys.version < '3':
+    import cPickle as pickle
+    protocol = 2
+    from itertools import imap as map, chain
+else:
+    import pickle
+    protocol = 3
+    xrange = range
+
+
+class SpecialLengths(object):
+    END_OF_DATA_SECTION = -1
+    NULL = -2
+
+
+class Serializer(object):
+
+    __metaclass__ = ABCMeta
+
+    # Note: our notion of "equality" is that output generated by
+    # equal serializers can be deserialized using the same serializer.
+
+    # This default implementation handles the simple cases;
+    # subclasses should override __eq__ as appropriate.
+
+    def __eq__(self, other):
+        return isinstance(other, self.__class__) and self.__dict__ == other.__dict__
+
+    def __ne__(self, other):
+        return not self.__eq__(other)
+
+    def __repr__(self):
+        return "%s()" % self.__class__.__name__
+
+    def __hash__(self):
+        return hash(str(self))
+
+    @abstractmethod
+    def dump_stream(self, iterator, stream):
+        """
+        Serializes an iterator of objects to the output stream.
+        """
+        pass
+
+    @abstractmethod
+    def load_stream(self, stream):
+        """
+        Returns an iterator of deserialized objects from the input stream.
+        """
+        pass
+
+    def _load_stream_without_unbatching(self, stream):
+        """
+        Returns an iterator of deserialized batches (iterable) of objects from the input
stream.
+        If the serializer does not operate on batches the default implementation returns
an
+        iterator of single element lists.
+        """
+        return map(lambda x: [x], self.load_stream(stream))
+
+
+class FramedSerializer(Serializer):
+    """
+    Serializer that writes objects as a stream of (length, data) pairs,
+    where C{length} is a 32-bit integer and data is C{length} bytes.
+    """
+
+    def __init__(self):
+        # On Python 2.6, we can't write bytearrays to streams, so we need to convert them
+        # to strings first. Check if the version number is that old.
+        self._only_write_strings = sys.version_info[0:2] <= (2, 6)
 
 Review comment:
   I think we do not need this check, due to we are supported the 2.x from 2.7, What do you
think?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message