ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Taras Ledkov (Jira)" <j...@apache.org>
Subject [jira] [Created] (IGNITE-13154) Introduce aility to manage manage binary types by the users
Date Tue, 16 Jun 2020 08:54:00 GMT
Taras Ledkov created IGNITE-13154:

             Summary: Introduce aility to manage manage binary types by the users
                 Key: IGNITE-13154
                 URL: https://issues.apache.org/jira/browse/IGNITE-13154
             Project: Ignite
          Issue Type: Improvement
          Components: binary
            Reporter: Taras Ledkov
            Assignee: Taras Ledkov
             Fix For: 2.9

We need a way to change schema (including unsupported changes, such as column type change)
without cluster restart. This is for the case when all data associated with the binary type
has been removed, so removing the old schema is safe.

Now users must stop the cluster and remove the folder with binary metadata manually.

The proposed way is to introduce internal API to manage binary types and public command line
interface (via control.sh). That way one can remove a cache from the cluster, then unregister
corresponding binary types, then launch a new version of an application that would register
the new schema and reload the data.

*The current implementation has restrictions:*
- all cluster nodes must support remove type feature.
- the cluster must not contains data with type to remove.
- operation of the update type must not be launched on the cluster for the type to remove
(operation examples: put into cache, BinaryObjectBuilder#build).
- client nodes process metadata operation asynchronously so type may be removed at the client
after any delay after the remove type operation is completed.
- if the node that contains the old version of the type joins to the cluster where type was
removed the type is propagated to cluster metadata (because metadata tombstone not supported).
- if the node that contains the old version of the type cannot join to the cluster where type
was removed and then updated to the new version (because metadata versioned tombstone not

So, user scenarios looks like:
# Be sure that all server nodes supports remove type feature.
# Remove caches contains the data with type to remove.
# Stops the client node with older version.
# Stops all operation with type to remove (don't create binary objects, don't run compute
jobs with type to remove).
# Remove the type on the stable topology (and production destination topolog).
# Waits any delay (depends on the cluster size and clients count)
# Produce operations with new version of the type.

*Proposed command line interface*
New commands (all commands are _experimental_ ):
- {{--meta list}} prints info about all available binary types:
{{typeId=<ID>, typeName=<name>, fields=<fields_count>, schemas=<schemas_count>,
- {{\-\-meta details (\-\- typeId <ID>| \-\-typeName <name>)}} prints detailed
info info about specified type. The type may be specified by type name or type ID.
output example:
typeId=0x1FBFBC0C (532659212)
  name=fld3, type=long[], fieldId=0x2FFF95 (3145621)
  name=fld2, type=double, fieldId=0x2FFF94 (3145620)
  name=fld1, type=Date, fieldId=0x2FFF93 (3145619)
  name=fld0, type=int, fieldId=0x2FFF92 (3145618)
  schemaId=0x6C5CC179 (1818018169), fields=[fld0]
  schemaId=0x70E46431 (1894016049), fields=[fld0, fld1, fld2, fld3]
- {{\-\-meta remove (\-\- typeId <ID>| \-\-typeName <name>) [\-\-out <file_name>]}}
removes metadata for specified type form cluster and saves the removed metadata to the specified
file. If the file name isn't specified the output file name is: {{<typeId>.bin}}
The command requires confirmation.
*N.B.*: The all session of thin clients (ODBC, JDBC, thin client) are closed (to cleanup local
cache of the binary metadata).
- {{\-\-meta update \-\-in <file_name>]}} update cluster metadata from specified file
(file name is required)
The command requires confirmation.

This message was sent by Atlassian Jira

View raw message