dolphinscheduler-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From leon...@apache.org
Subject [incubator-dolphinscheduler-website] branch asf-site updated: test (#31)
Date Thu, 21 Nov 2019 03:38:39 GMT
This is an automated email from the ASF dual-hosted git repository.

leonbao pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-dolphinscheduler-website.git


View the commit online:
https://github.com/apache/incubator-dolphinscheduler-website/commit/d785cf9cbcc58ce5e14afd249fdbc2c54e388e8b

The following commit(s) were added to refs/heads/asf-site by this push:
     new d785cf9  test (#31)
d785cf9 is described below

commit d785cf9cbcc58ce5e14afd249fdbc2c54e388e8b
Author: bao liang <29528966+lenboo@users.noreply.github.com>
AuthorDate: Thu Nov 21 11:38:31 2019 +0800

    test (#31)
    
    * update asf-site
    
    * update asf-site
    
    * update asf-site
    
    * merge from upstream
    
    * merge from upstream
    
    * test
    
    * test
---
 en-us/blog/architecture-design.html | 304 ------------------------------------
 en-us/blog/architecture-design.json |   6 -
 en-us/blog/index.html               |  27 ----
 en-us/blog/meetup_2019_10_26.html   |  40 -----
 en-us/blog/meetup_2019_10_26.json   |   6 -
 en-us/index.html                    |   2 +-
 6 files changed, 1 insertion(+), 384 deletions(-)

diff --git a/en-us/blog/architecture-design.html b/en-us/blog/architecture-design.html
deleted file mode 100644
index 66eddfd..0000000
--- a/en-us/blog/architecture-design.html
+++ /dev/null
@@ -1,304 +0,0 @@
-<!DOCTYPE html>
-<html lang="en">
-
-<head>
-	<meta charset="UTF-8">
-	<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0,
user-scalable=no">
-	<meta name="keywords" content="architecture-design" />
-	<meta name="description" content="architecture-design" />
-	<!-- 网页标签标题 -->
-	<title>architecture-design</title>
-	<link rel="shortcut icon" href="/img/docsite.ico"/>
-	<link rel="stylesheet" href="/build/blogDetail.css" />
-</head>
-<body>
-	<div id="root"><div class="blog-detail-page" data-reactroot=""><header class="header-container
header-container-normal"><div class="header-body"><a href="/en-us/index.html"><img
class="logo" src="/img/hlogo_colorful.svg"/></a><div class="search search-normal"><span
class="icon-search"></span></div><span class="language-switch language-switch-normal">中</span><div
class="header-menu"><img class="header-menu-toggle" src="/img/system/menu_gray.png"/><ul><li
class="menu-item menu-item-normal [...]
-<p>Before explaining the architecture of the schedule system, let us first understand
the common nouns of the schedule system.</p>
-<h3>1.Noun Interpretation</h3>
-<p><strong>DAG:</strong> Full name Directed Acyclic Graph,referred
to as DAG。Tasks in the workflow are assembled in the form of directed acyclic graphs, which
are topologically traversed from nodes with zero indegrees of ingress until there are no successor
nodes. For example, the following picture:</p>
-<p align="center">
-  <img src="https://analysys.github.io/easyscheduler_docs_cn/images/dag_examples_cn.jpg"
alt="dag示例"  width="60%" />
-  <p align="center">
-        <em>dag example</em>
-  </p>
-</p>
-<p><strong>Process definition</strong>: Visualization <strong>DAG</strong>
by dragging task nodes and establishing associations of task nodes</p>
-<p><strong>Process instance</strong>: A process instance is an instantiation
of a process definition, which can be generated by manual startup or  scheduling. The process
definition runs once, a new process instance is generated</p>
-<p><strong>Task instance</strong>: A task instance is the instantiation
of a specific task node when a process instance runs, which indicates the specific task execution
status</p>
-<p><strong>Task type</strong>: Currently supports SHELL, SQL, SUB_PROCESS
(sub-process), PROCEDURE, MR, SPARK, PYTHON, DEPENDENT (dependency), and plans to support
dynamic plug-in extension, note: the sub-<strong>SUB_PROCESS</strong> is also
A separate process definition that can be launched separately</p>
-<p><strong>Schedule mode</strong> :  The system supports timing schedule
and manual schedule based on cron expressions. Command type support: start workflow, start
execution from current node, resume fault-tolerant workflow, resume pause process, start execution
from failed node, complement, timer, rerun, pause, stop, resume waiting thread. Where <strong>recovers
the fault-tolerant workflow</strong> and <strong>restores the waiting thread</strong>
The two command types are used by the sc [...]
-<p><strong>Timed schedule</strong>: The system uses <strong>quartz</strong>
distributed scheduler and supports the generation of cron expression visualization</p>
-<p><strong>Dependency</strong>: The system does not only support <strong>DAG</strong>
Simple dependencies between predecessors and successor nodes, but also provides <strong>task
dependencies</strong> nodes, support for custom task dependencies between processes**</p>
-<p><strong>Priority</strong>: Supports the priority of process instances
and task instances. If the process instance and task instance priority are not set, the default
is first in, first out.</p>
-<p><strong>Mail Alert</strong>: Support <strong>SQL Task</strong>
Query Result Email Send, Process Instance Run Result Email Alert and Fault Tolerant Alert
Notification</p>
-<p><strong>Failure policy</strong>: For tasks running in parallel, if there
are tasks that fail, two failure policy processing methods are provided. <strong>Continue</strong>
means that the status of the task is run in parallel until the end of the process failure.
<strong>End</strong> means that once a failed task is found, Kill also drops the
running parallel task and the process ends.</p>
-<p><strong>Complement</strong>: Complement historical data, support **
interval parallel and serial ** two complement methods</p>
-<h3>2.System architecture</h3>
-<h4>2.1 System Architecture Diagram</h4>
-<p align="center">
-  <img src="https://user-images.githubusercontent.com/48329107/62609545-8f973480-b934-11e9-9a58-d8133222f14d.png"
alt="System Architecture Diagram"  />
-  <p align="center">
-        <em>System Architecture Diagram</em>
-  </p>
-</p>
-<h4>2.2 Architectural description</h4>
-<ul>
-<li>
-<p><strong>MasterServer</strong></p>
-<p>MasterServer adopts the distributed non-central design concept. MasterServer is
mainly responsible for DAG task split, task submission monitoring, and monitoring the health
status of other MasterServer and WorkerServer.
-When the MasterServer service starts, it registers a temporary node with Zookeeper, and listens
to the Zookeeper temporary node state change for fault tolerance processing.</p>
-<h5>The service mainly contains:</h5>
-<ul>
-<li>
-<p><strong>Distributed Quartz</strong> distributed scheduling component,
mainly responsible for the start and stop operation of the scheduled task. When the quartz
picks up the task, the master internally has a thread pool to be responsible for the subsequent
operations of the task.</p>
-</li>
-<li>
-<p><strong>MasterSchedulerThread</strong> is a scan thread that periodically
scans the <strong>command</strong> table in the database for different business
operations based on different ** command types**</p>
-</li>
-<li>
-<p><strong>MasterExecThread</strong> is mainly responsible for DAG task
segmentation, task submission monitoring, logic processing of various command types</p>
-</li>
-<li>
-<p><strong>MasterTaskExecThread</strong> is mainly responsible for task
persistence</p>
-</li>
-</ul>
-</li>
-<li>
-<p><strong>WorkerServer</strong></p>
-<ul>
-<li>
-<p>WorkerServer also adopts a distributed, non-central design concept. WorkerServer
is mainly responsible for task execution and providing log services. When the WorkerServer
service starts, it registers the temporary node with Zookeeper and maintains the heartbeat.</p>
-<h5>This service contains:</h5>
-<ul>
-<li><strong>FetchTaskThread</strong> is mainly responsible for continuously
receiving tasks from <strong>Task Queue</strong> and calling <strong>TaskScheduleThread</strong>
corresponding executors according to different task types.</li>
-<li><strong>LoggerServer</strong> is an RPC service that provides functions
such as log fragment viewing, refresh and download.</li>
-</ul>
-</li>
-<li>
-<p><strong>ZooKeeper</strong></p>
-<p>The ZooKeeper service, the MasterServer and the WorkerServer nodes in the system
all use the ZooKeeper for cluster management and fault tolerance. In addition, the system
also performs event monitoring and distributed locking based on ZooKeeper.
-We have also implemented queues based on Redis, but we hope that EasyScheduler relies on
as few components as possible, so we finally removed the Redis implementation.</p>
-</li>
-<li>
-<p><strong>Task Queue</strong></p>
-<p>The task queue operation is provided. Currently, the queue is also implemented based
on Zookeeper. Since there is less information stored in the queue, there is no need to worry
about too much data in the queue. In fact, we have over-measured a million-level data storage
queue, which has no effect on system stability and performance.</p>
-</li>
-<li>
-<p><strong>Alert</strong></p>
-<p>Provides alarm-related interfaces. The interfaces mainly include <strong>Alarms</strong>.
The storage, query, and notification functions of the two types of alarm data. The notification
function has two types: <strong>mail notification</strong> and <strong>SNMP
(not yet implemented)</strong>.</p>
-</li>
-<li>
-<p><strong>API</strong></p>
-<p>The API interface layer is mainly responsible for processing requests from the front-end
UI layer. The service provides a RESTful api to provide request services externally.
-Interfaces include workflow creation, definition, query, modification, release, offline,
manual start, stop, pause, resume, start execution from this node, and more.</p>
-</li>
-<li>
-<p><strong>UI</strong></p>
-<p>The front-end page of the system provides various visual operation interfaces of
the system. For details, see the <strong>[System User Manual] (System User <a href="http://Manual.md">Manual.md</a>)</strong>
section.</p>
-</li>
-</ul>
-</li>
-</ul>
-<h4>2.3 Architectural Design Ideas</h4>
-<h5>I. Decentralized vs centralization</h5>
-<h6>Centralization Thought</h6>
-<p>The centralized design concept is relatively simple. The nodes in the distributed
cluster are divided into two roles according to their roles:</p>
-<p align="center">
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/master_slave.png"
alt="master-slave role" width="50%" />
- </p>
-<ul>
-<li>The role of Master is mainly responsible for task distribution and supervising
the health status of Slave. It can dynamically balance the task to Slave, so that the Slave
node will not be &quot;busy&quot; or &quot;free&quot;.</li>
-<li>The role of the Worker is mainly responsible for the execution of the task and
maintains the heartbeat with the Master so that the Master can assign tasks to the Slave.</li>
-</ul>
-<p>Problems in the design of centralized :</p>
-<ul>
-<li>Once the Master has a problem, the group has no leader and the entire cluster will
crash. In order to solve this problem, most Master/Slave architecture modes adopt the design
scheme of the master and backup masters, which can be hot standby or cold standby, automatic
switching or manual switching, and more and more new systems are available. Automatically
elects the ability to switch masters to improve system availability.</li>
-<li>Another problem is that if the Scheduler is on the Master, although it can support
different tasks in one DAG running on different machines, it will generate overload of the
Master. If the Scheduler is on the Slave, all tasks in a DAG can only be submitted on one
machine. If there are more parallel tasks, the pressure on the Slave may be larger.</li>
-</ul>
-<h6>Decentralization</h6>
- <p align="center"
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/decentralization.png"
alt="decentralized" width="50%" />
- </p>
-<ul>
-<li>
-<p>In the decentralized design, there is usually no Master/Slave concept, all roles
are the same, the status is equal, the global Internet is a typical decentralized distributed
system, networked arbitrary node equipment down machine , all will only affect a small range
of features.</p>
-</li>
-<li>
-<p>The core design of decentralized design is that there is no &quot;manager&quot;
that is different from other nodes in the entire distributed system, so there is no single
point of failure problem. However, since there is no &quot;manager&quot; node, each
node needs to communicate with other nodes to get the necessary machine information, and the
unreliable line of distributed system communication greatly increases the difficulty of implementing
the above functions.</p>
-</li>
-<li>
-<p>In fact, truly decentralized distributed systems are rare. Instead, dynamic centralized
distributed systems are constantly emerging. Under this architecture, the managers in the
cluster are dynamically selected, rather than preset, and when the cluster fails, the nodes
of the cluster will spontaneously hold &quot;meetings&quot; to elect new &quot;managers&quot;.
Go to preside over the work. The most typical case is the Etcd implemented in ZooKeeper and
Go.</p>
-</li>
-<li>
-<p>Decentralization of EasyScheduler is the registration of Master/Worker to ZooKeeper.
The Master Cluster and the Worker Cluster are not centered, and the Zookeeper distributed
lock is used to elect one Master or Worker as the “manager” to perform the task.</p>
-</li>
-</ul>
-<h5>二、Distributed lock practice</h5>
-<p>EasyScheduler uses ZooKeeper distributed locks to implement only one Master to execute
the Scheduler at the same time, or only one Worker to perform task submission.</p>
-<ol>
-<li>The core process algorithm for obtaining distributed locks is as follows</li>
-</ol>
- <p align="center">
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/distributed_lock.png"
alt="Get Distributed Lock Process" width="50%" />
- </p>
-<ol start="2">
-<li>Scheduler thread distributed lock implementation flow chart in EasyScheduler:</li>
-</ol>
- <p align="center">
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/distributed_lock_procss.png"
alt="Get Distributed Lock Process" width="50%" />
- </p>
-<h5>Third, the thread is insufficient loop waiting problem</h5>
-<ul>
-<li>If there is no subprocess in a DAG, if the number of data in the Command is greater
than the threshold set by the thread pool, the direct process waits or fails.</li>
-<li>If a large number of sub-processes are nested in a large DAG, the following figure
will result in a &quot;dead&quot; state:</li>
-</ul>
- <p align="center">
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/lack_thread.png"
alt="Thread is not enough to wait for loop" width="50%" />
- </p>
-<p>In the above figure, MainFlowThread waits for SubFlowThread1 to end, SubFlowThread1
waits for SubFlowThread2 to end, SubFlowThread2 waits for SubFlowThread3 to end, and SubFlowThread3
waits for a new thread in the thread pool, then the entire DAG process cannot end, and thus
the thread cannot be released. This forms the state of the child parent process loop waiting.
At this point, the scheduling cluster will no longer be available unless a new Master is started
to add threads to brea [...]
-<p>It seems a bit unsatisfactory to start a new Master to break the deadlock, so we
proposed the following three options to reduce this risk:</p>
-<ol>
-<li>Calculate the sum of the threads of all Masters, and then calculate the number
of threads required for each DAG, that is, pre-calculate before the DAG process is executed.
Because it is a multi-master thread pool, the total number of threads is unlikely to be obtained
in real time.</li>
-<li>Judge the single master thread pool. If the thread pool is full, let the thread
fail directly.</li>
-<li>Add a Command type with insufficient resources. If the thread pool is insufficient,
the main process will be suspended. This way, the thread pool has a new thread, which can
make the process with insufficient resources hang up and wake up again.</li>
-</ol>
-<p>Note: The Master Scheduler thread is FIFO-enabled when it gets the Command.</p>
-<p>So we chose the third way to solve the problem of insufficient threads.</p>
-<h5>IV. Fault Tolerant Design</h5>
-<p>Fault tolerance is divided into service fault tolerance and task retry. Service
fault tolerance is divided into two types: Master Fault Tolerance and Worker Fault Tolerance.</p>
-<h6>1. Downtime fault tolerance</h6>
-<p>Service fault tolerance design relies on ZooKeeper's Watcher mechanism. The implementation
principle is as follows:</p>
- <p align="center">
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/fault-tolerant.png"
alt="EasyScheduler Fault Tolerant Design" width="40%" />
- </p>
-<p>The Master monitors the directories of other Masters and Workers. If the remove
event is detected, the process instance is fault-tolerant or the task instance is fault-tolerant
according to the specific business logic.</p>
-<ul>
-<li>Master fault tolerance flow chart:</li>
-</ul>
- <p align="center">
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/fault-tolerant_master.png"
alt="Master Fault Tolerance Flowchart" width="40%" />
- </p>
-<p>After the ZooKeeper Master is fault-tolerant, it is rescheduled by the Scheduler
thread in EasyScheduler. It traverses the DAG to find the &quot;Running&quot; and
&quot;Submit Successful&quot; tasks, and monitors the status of its task instance
for the &quot;Running&quot; task. You need to determine whether the Task Queue already
exists. If it exists, monitor the status of the task instance. If it does not exist, resubmit
the task instance.</p>
-<ul>
-<li>Worker fault tolerance flow chart:</li>
-</ul>
- <p align="center">
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/fault-tolerant_worker.png"
alt="Worker Fault Tolerance Flowchart" width="40%" />
- </p>
-<p>Once the Master Scheduler thread finds the task instance as &quot;need to be
fault tolerant&quot;, it takes over the task and resubmits.</p>
-<p>Note: Because the &quot;network jitter&quot; may cause the node to lose
the heartbeat of ZooKeeper in a short time, the node's remove event occurs. In this case,
we use the easiest way, that is, once the node has timeout connection with ZooKeeper, it will
directly stop the Master or Worker service.</p>
-<h6>2. Task failure retry</h6>
-<p>Here we must first distinguish between the concept of task failure retry, process
failure recovery, and process failure rerun:</p>
-<ul>
-<li>Task failure Retry is task level, which is automatically performed by the scheduling
system. For example, if a shell task sets the number of retries to 3 times, then the shell
task will try to run up to 3 times after failing to run.</li>
-<li>Process failure recovery is process level, is done manually, recovery can only
be performed from the failed node ** or ** from the current node **</li>
-<li>Process failure rerun is also process level, is done manually, rerun is from the
start node</li>
-</ul>
-<p>Next, let's talk about the topic, we divided the task nodes in the workflow into
two types.</p>
-<ul>
-<li>One is a business node, which corresponds to an actual script or processing statement,
such as a Shell node, an MR node, a Spark node, a dependent node, and so on.</li>
-<li>There is also a logical node, which does not do the actual script or statement
processing, but the logical processing of the entire process flow, such as sub-flow sections.</li>
-</ul>
-<p>Each ** service node** can configure the number of failed retries. When the task
node fails, it will automatically retry until it succeeds or exceeds the configured number
of retries. <strong>Logical node</strong> does not support failed retry. But the
tasks in the logical nodes support retry.</p>
-<p>If there is a task failure in the workflow that reaches the maximum number of retries,
the workflow will fail to stop, and the failed workflow can be manually rerun or process resumed.</p>
-<h5>V. Task priority design</h5>
-<p>In the early scheduling design, if there is no priority design and fair scheduling
design, it will encounter the situation that the task submitted first may be completed simultaneously
with the task submitted subsequently, but the priority of the process or task cannot be set.
We have redesigned this, and we are currently designing it as follows:</p>
-<ul>
-<li>
-<p>According to ** different process instance priority ** prioritizes ** same process
instance priority ** prioritizes ** task priority within the same process ** takes precedence
over ** same process ** commit order from high Go to low for task processing.</p>
-<ul>
-<li>
-<p>The specific implementation is to resolve the priority according to the json of
the task instance, and then save the ** process instance priority _ process instance id_task
priority _ task id** information in the ZooKeeper task queue, when obtained from the task
queue, Through string comparison, you can get the task that needs to be executed first.</p>
-<ul>
-<li>
-<p>The priority of the process definition is that some processes need to be processed
before other processes. This can be configured at the start of the process or at the time
of scheduled start. There are 5 levels, followed by HIGHEST, HIGH, MEDIUM, LOW, and LOWEST.
As shown below</p>
-<p align="center">
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/process_priority.png"
alt="Process Priority Configuration" width="40%" />
- </p>
-</li>
-<li>
-<p>The priority of the task is also divided into 5 levels, followed by HIGHEST, HIGH,
MEDIUM, LOW, and LOWEST. As shown below</p>
-<p align="center">
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/task_priority.png"
alt="task priority configuration" width="35%" />
- </p>
-</li>
-</ul>
-</li>
-</ul>
-</li>
-</ul>
-<h5>VI. Logback and gRPC implement log access</h5>
-<ul>
-<li>Since the Web (UI) and Worker are not necessarily on the same machine, viewing
the log is not as it is for querying local files. There are two options:
-<ul>
-<li>Put the logs on the ES search engine</li>
-<li>Obtain remote log information through gRPC communication</li>
-</ul>
-</li>
-<li>Considering the lightweightness of EasyScheduler as much as possible, gRPC was
chosen to implement remote access log information.</li>
-</ul>
- <p align="center">
-   <img src="https://analysys.github.io/easyscheduler_docs_cn/images/grpc.png" alt="grpc
remote access" width="50%" />
- </p>
-<ul>
-<li>We use a custom Logback FileAppender and Filter function to generate a log file
for each task instance.</li>
-<li>The main implementation of FileAppender is as follows:</li>
-</ul>
-<pre><code class="language-java"> <span class="hljs-comment">/**
-  * task log appender
-  */</span>
- Public <span class="hljs-class"><span class="hljs-keyword">class</span>
<span class="hljs-title">TaskLogAppender</span> <span class="hljs-keyword">extends</span>
<span class="hljs-title">FileAppender</span>&lt;<span class="hljs-title">ILoggingEvent</span>
</span>{
- 
-     ...
-
-    <span class="hljs-meta">@Override</span>
-    <span class="hljs-function">Protected <span class="hljs-keyword">void</span>
<span class="hljs-title">append</span><span class="hljs-params">(ILoggingEvent
event)</span> </span>{
-
-        If (currentlyActiveFile == <span class="hljs-keyword">null</span>){
-            currentlyActiveFile = getFile();
-        }
-        String activeFile = currentlyActiveFile;
-        <span class="hljs-comment">// thread name: taskThreadName-processDefineId_processInstanceId_taskInstanceId</span>
-        String threadName = event.getThreadName();
-        String[] threadNameArr = threadName.split(<span class="hljs-string">"-"</span>);
-        <span class="hljs-comment">// logId = processDefineId_processInstanceId_taskInstanceId</span>
-        String logId = threadNameArr[<span class="hljs-number">1</span>];
-        ...
-        <span class="hljs-keyword">super</span>.subAppend(event);
-    }
-}
-</code></pre>
-<p>Generate a log in the form of /process definition id/process instance id/task instance
id.log</p>
-<ul>
-<li>Filter matches the thread name starting with TaskLogInfo:</li>
-<li>TaskLogFilter is implemented as follows:</li>
-</ul>
-<pre><code class="language-java"> <span class="hljs-comment">/**
- * task log filter
- */</span>
-Public <span class="hljs-class"><span class="hljs-keyword">class</span>
<span class="hljs-title">TaskLogFilter</span> <span class="hljs-keyword">extends</span>
<span class="hljs-title">Filter</span>&lt;<span class="hljs-title">ILoggingEvent</span>
</span>{
-
-    <span class="hljs-meta">@Override</span>
-    <span class="hljs-function">Public FilterReply <span class="hljs-title">decide</span><span
class="hljs-params">(ILoggingEvent event)</span> </span>{
-        If (event.getThreadName().startsWith(<span class="hljs-string">"TaskLogInfo-"</span>)){
-            Return FilterReply.ACCEPT;
-        }
-        Return FilterReply.DENY;
-    }
-}
-</code></pre>
-<h3>summary</h3>
-<p>Starting from the scheduling, this paper introduces the architecture principle and
implementation ideas of the big data distributed workflow scheduling system-EasyScheduler.
To be continued</p>
-</section><footer class="footer-container"><div class="footer-body"><img
src="/img/ds_gray.svg"/><div class="cols-container"><div class="col col-12"><h3>Disclaimer</h3><p>Apache
DolphinScheduler (incubating) is an effort undergoing incubation at The Apache Software Foundation
(ASF), sponsored by Incubator. 
-Incubation is required of all newly accepted projects until a further review indicates 
-that the infrastructure, communications, and decision making process have stabilized in a
manner consistent with other successful ASF projects. 
-While incubation status is not necessarily a reflection of the completeness or stability
of the code, 
-it does indicate that the project has yet to be fully endorsed by the ASF.</p></div><div
class="col col-6"><dl><dt>Documentation</dt><dd><a href="/en-us/docs/developer_guide/architecture-design.html"
target="_self">Overview</a></dd><dd><a href="/en-us/docs/user_doc/quick-start.html"
target="_self">Quick start</a></dd><dd><a href="/en-us/docs/development/developers.html"
target="_self">Developer guide</a></dd></dl></div><div class="col
col-6"><dl><dt>ASF</dt><dd><a href="http://www.apache [...]
-	<script src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js"></script>
-	<script src="https://f.alicdn.com/react/15.4.1/react-dom.min.js"></script>
-	<script>
-		window.rootPath = '';
-  </script>
-	<script src="/build/blogDetail.js"></script>
-</body>
-</html>
\ No newline at end of file
diff --git a/en-us/blog/architecture-design.json b/en-us/blog/architecture-design.json
deleted file mode 100644
index 3952afd..0000000
--- a/en-us/blog/architecture-design.json
+++ /dev/null
@@ -1,6 +0,0 @@
-{
-  "filename": "architecture-design.md",
-  "__html": "<h2>Architecture Design</h2>\n<p>Before explaining the architecture
of the schedule system, let us first understand the common nouns of the schedule system.</p>\n<h3>1.Noun
Interpretation</h3>\n<p><strong>DAG:</strong> Full name Directed
Acyclic Graph,referred to as DAG。Tasks in the workflow are assembled in the form of directed
acyclic graphs, which are topologically traversed from nodes with zero indegrees of ingress
until there are no successor nodes. For example, the fol [...]
-  "link": "/en-us/blog/architecture-design.html",
-  "meta": {}
-}
\ No newline at end of file
diff --git a/en-us/blog/index.html b/en-us/blog/index.html
deleted file mode 100644
index fc5d2a7..0000000
--- a/en-us/blog/index.html
+++ /dev/null
@@ -1,27 +0,0 @@
-<!DOCTYPE html>
-<html lang="en">
-
-<head>
-	<meta charset="UTF-8">
-	<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0,
user-scalable=no">
-	<meta name="keywords" content="blog,DolphinScheduler blog" />
-	<meta name="description" content="page description" />
-	<!-- 网页标签标题 -->
-	<title>Apache DolphinScheduler | BLOG</title>
-	<link rel="shortcut icon" href="/img/docsite.ico"/>
-	<link rel="stylesheet" href="/build/blog.css" />
-</head>
-<body>
-	<div id="root"><div class="blog-list-page" data-reactroot=""><header class="header-container
header-container-normal"><div class="header-body"><a href="/en-us/index.html"><img
class="logo" src="/img/hlogo_colorful.svg"/></a><div class="search search-normal"><span
class="icon-search"></span></div><span class="language-switch language-switch-normal">中</span><div
class="header-menu"><img class="header-menu-toggle" src="/img/system/menu_gray.png"/><ul><li
class="menu-item menu-item-normal"> [...]
-Incubation is required of all newly accepted projects until a further review indicates 
-that the infrastructure, communications, and decision making process have stabilized in a
manner consistent with other successful ASF projects. 
-While incubation status is not necessarily a reflection of the completeness or stability
of the code, 
-it does indicate that the project has yet to be fully endorsed by the ASF.</p></div><div
class="col col-6"><dl><dt>Documentation</dt><dd><a href="/en-us/docs/developer_guide/architecture-design.html"
target="_self">Overview</a></dd><dd><a href="/en-us/docs/user_doc/quick-start.html"
target="_self">Quick start</a></dd><dd><a href="/en-us/docs/development/developers.html"
target="_self">Developer guide</a></dd></dl></div><div class="col
col-6"><dl><dt>ASF</dt><dd><a href="http://www.apache [...]
-	<script src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js"></script>
-	<script src="https://f.alicdn.com/react/15.4.1/react-dom.min.js"></script>
-	<script>
-		window.rootPath = '';
-  </script>
-	<script src="/build/blog.js"></script>
-</body>
-</html>
\ No newline at end of file
diff --git a/en-us/blog/meetup_2019_10_26.html b/en-us/blog/meetup_2019_10_26.html
deleted file mode 100644
index af7a35a..0000000
--- a/en-us/blog/meetup_2019_10_26.html
+++ /dev/null
@@ -1,40 +0,0 @@
-<!DOCTYPE html>
-<html lang="en">
-
-<head>
-	<meta charset="UTF-8">
-	<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0,
user-scalable=no">
-	<meta name="keywords" content="meetup_2019_10_26" />
-	<meta name="description" content="meetup_2019_10_26" />
-	<!-- 网页标签标题 -->
-	<title>meetup_2019_10_26</title>
-	<link rel="shortcut icon" href="/img/docsite.ico"/>
-	<link rel="stylesheet" href="/build/blogDetail.css" />
-</head>
-<body>
-	<div id="root"><div class="blog-detail-page" data-reactroot=""><header class="header-container
header-container-normal"><div class="header-body"><a href="/en-us/index.html"><img
class="logo" src="/img/hlogo_colorful.svg"/></a><div class="search search-normal"><span
class="icon-search"></span></div><span class="language-switch language-switch-normal">中</span><div
class="header-menu"><img class="header-menu-toggle" src="/img/system/menu_gray.png"/><ul><li
class="menu-item menu-item-normal [...]
-Apache Dolphin Scheduler(Incubating) Meetup has been held successfully in Shanghai 2019.10.26.</p>
-<p>Address: Shanghai Changning Yuyuan Road 1107 Chuangyi space (Hongji) 3r20</p>
-<p>The meetup was begin at 2:00 pm, and close at about 5:00 pm.</p>
-<p>The Agenda is as following:</p>
-<ul>
-<li>Introduction/overview of DolphinScheduler (William-GuoWei 14:00-14:20). <a href="/download/2019-10-26/DolphinScheduler_guowei.pptx">PPT</a></li>
-<li>DolphinScheduler internals, fairly technical: how DolphinScheduler works and so
on (Zhanwei Qiao 14:20-15:00) <a href="/download/2019-10-26/DolphinScheduler_qiaozhanwei.pptx">PPT</a></li>
-<li>From open source users to PPMC --- Talking about Me and DolphinScheduler(Baoqi
Wu from guandata 15:10-15:50)<a href="/download/2019-10-26/Dolphinescheduler_baoqiwu.pptx">PPT</a></li>
-<li>Application and Practice of Dolphin Scheduler in cathay-ins (Zongyao Zhang from
cathay-ins 15:50-16:30)<a href="/download/2019-10-26/DolphinScheduler_zhangzongyao.pptx">PPT</a></li>
-<li>Recently released features and Roadmap (Lidong Dai from analysys 16:30-17:00) <a
href="/download/2019-10-26/DolphinScheduler_dailidong.pptx">PPT</a></li>
-<li>Free discussion</li>
-</ul>
-</section><footer class="footer-container"><div class="footer-body"><img
src="/img/ds_gray.svg"/><div class="cols-container"><div class="col col-12"><h3>Disclaimer</h3><p>Apache
DolphinScheduler (incubating) is an effort undergoing incubation at The Apache Software Foundation
(ASF), sponsored by Incubator. 
-Incubation is required of all newly accepted projects until a further review indicates 
-that the infrastructure, communications, and decision making process have stabilized in a
manner consistent with other successful ASF projects. 
-While incubation status is not necessarily a reflection of the completeness or stability
of the code, 
-it does indicate that the project has yet to be fully endorsed by the ASF.</p></div><div
class="col col-6"><dl><dt>Documentation</dt><dd><a href="/en-us/docs/developer_guide/architecture-design.html"
target="_self">Overview</a></dd><dd><a href="/en-us/docs/user_doc/quick-start.html"
target="_self">Quick start</a></dd><dd><a href="/en-us/docs/development/developers.html"
target="_self">Developer guide</a></dd></dl></div><div class="col
col-6"><dl><dt>ASF</dt><dd><a href="http://www.apache [...]
-	<script src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js"></script>
-	<script src="https://f.alicdn.com/react/15.4.1/react-dom.min.js"></script>
-	<script>
-		window.rootPath = '';
-  </script>
-	<script src="/build/blogDetail.js"></script>
-</body>
-</html>
\ No newline at end of file
diff --git a/en-us/blog/meetup_2019_10_26.json b/en-us/blog/meetup_2019_10_26.json
deleted file mode 100644
index 37dbf45..0000000
--- a/en-us/blog/meetup_2019_10_26.json
+++ /dev/null
@@ -1,6 +0,0 @@
-{
-  "filename": "meetup_2019_10_26.md",
-  "__html": "<p><img src=\"/img/2019-10-26-user.jpg\" alt=\"avatar\">\nApache
Dolphin Scheduler(Incubating) Meetup has been held successfully in Shanghai 2019.10.26.</p>\n<p>Address:
Shanghai Changning Yuyuan Road 1107 Chuangyi space (Hongji) 3r20</p>\n<p>The meetup
was begin at 2:00 pm, and close at about 5:00 pm.</p>\n<p>The Agenda is as following:</p>\n<ul>\n<li>Introduction/overview
of DolphinScheduler (William-GuoWei 14:00-14:20). <a href=\"/download/2019-10-26/DolphinScheduler_guow
[...]
-  "link": "/en-us/blog/meetup_2019_10_26.html",
-  "meta": {}
-}
\ No newline at end of file
diff --git a/en-us/index.html b/en-us/index.html
index 4fb2dba..069b410 100644
--- a/en-us/index.html
+++ b/en-us/index.html
@@ -14,7 +14,7 @@
 <body>
 	<div id="root"><div class="home-page" data-reactroot=""><section class="top-section"><header
class="header-container header-container-primary"><div class="header-body"><a
href="/en-us/index.html"><img class="logo" src="/img/hlogo_white.svg"/></a><div
class="search search-primary"><span class="icon-search"></span></div><span
class="language-switch language-switch-primary">中</span><div class="header-menu"><img
class="header-menu-toggle" src="/img/system/menu_white.png"/><ul><li class="men
[...]
 Incubation is required of all newly accepted projects until a further review indicates 
-that the infrastructure, communications, and decision making process have stabilized in a
manner consistent with other successful ASF projects. 
+that the infrastructure, communications, decision making process have stabilized in a manner
consistent with other successful ASF projects.
 While incubation status is not necessarily a reflection of the completeness or stability
of the code, 
 it does indicate that the project has yet to be fully endorsed by the ASF.</p></div><div
class="col col-6"><dl><dt>Documentation</dt><dd><a href="/en-us/docs/developer_guide/architecture-design.html"
target="_self">Overview</a></dd><dd><a href="/en-us/docs/user_doc/quick-start.html"
target="_self">Quick start</a></dd><dd><a href="/en-us/docs/development/developers.html"
target="_self">Developer guide</a></dd></dl></div><div class="col
col-6"><dl><dt>ASF</dt><dd><a href="http://www.apache [...]
 	<script src="https://f.alicdn.com/react/15.4.1/react-with-addons.min.js"></script>


Mime
View raw message