flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kl0u <...@git.apache.org>
Subject [GitHub] flink pull request #5750: [FLINK-8973] [E2E] HA end-to-end test with StateMa...
Date Tue, 27 Mar 2018 09:38:34 GMT
Github user kl0u commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5750#discussion_r177364376
  
    --- Diff: flink-end-to-end-tests/test-scripts/common.sh ---
    @@ -39,6 +39,109 @@ cd $TEST_ROOT
     export TEST_DATA_DIR=$TEST_INFRA_DIR/temp-test-directory-$(date +%S%N)
     echo "TEST_DATA_DIR: $TEST_DATA_DIR"
     
    +function revert_default_config() {
    +
    +    # first revert the conf/masters file
    +    echo "localhost:8081" > ${FLINK_DIR}/conf/masters
    +
    +    # and then the conf/flink-conf.yaml
    +    sed 's/^    //g' > ${FLINK_DIR}/conf/flink-conf.yaml << EOL
    +    #==============================================================================
    +    # Common
    +    #==============================================================================
    +
    +    jobmanager.rpc.address: localhost
    +    jobmanager.rpc.port: 6123
    +    jobmanager.heap.mb: 1024
    +    taskmanager.heap.mb: 1024
    +    taskmanager.numberOfTaskSlots: 1
    +    parallelism.default: 1
    +
    +    #==============================================================================
    +    # Web Frontend
    +    #==============================================================================
    +
    +    web.port: 8081
    +EOL
    +}
    +
    +function create_ha_config() {
    +
    +    # create the masters file (only one currently).
    +    # This must have all the masters to be used in HA.
    +    echo "localhost:8081" > ${FLINK_DIR}/conf/masters
    +
    +    # clean up the dir that will be used for zookeeper storage
    +    # (see high-availability.zookeeper.storageDir below)
    +    if [ -e $TEST_DATA_DIR/recovery ]; then
    +       echo "File ${TEST_DATA_DIR}/recovery exists. Deleting it..."
    +       rm -rf $TEST_DATA_DIR/recovery
    +    fi
    +
    +    # then move on to create the flink-conf.yaml
    +    sed 's/^    //g' > ${FLINK_DIR}/conf/flink-conf.yaml << EOL
    +    #==============================================================================
    +    # Common
    +    #==============================================================================
    +
    +    jobmanager.rpc.address: localhost
    +    jobmanager.rpc.port: 6123
    +    jobmanager.heap.mb: 1024
    +    taskmanager.heap.mb: 1024
    +    taskmanager.numberOfTaskSlots: 4
    +    parallelism.default: 1
    +
    +    #==============================================================================
    +    # High Availability
    +    #==============================================================================
    +
    +    high-availability: zookeeper
    +    high-availability.zookeeper.storageDir: file://${TEST_DATA_DIR}/recovery/
    +    high-availability.zookeeper.quorum: localhost:2181
    +    high-availability.zookeeper.path.root: /flink
    +    high-availability.cluster-id: /test_cluster_one
    +
    +    #==============================================================================
    +    # Web Frontend
    +    #==============================================================================
    +
    +    web.port: 8081
    +EOL
    +}
    +
    +function start_ha_cluster {
    +    echo "Setting up HA Cluster..."
    +    create_ha_config
    +    start_local_zk
    +    start_cluster
    +}
    +
    +function start_local_zk {
    +    # Parses the zoo.cfg and starts locally zk.
    +
    +    # This is almost the same code as the
    +    # /bin/start-zookeeper-quorum.sh without the SSH part and only running for localhost.
    --- End diff --
    
    The problem is that the `zookeeper` script does not take into account that the fact that
we may want to launch `zookeeper` locally, so it always asks for `SSH` credentials.


---

Mime
View raw message