spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (SPARK-15469) Datanode and nodemanger appear in jps but arent start correctly.
Date Sun, 22 May 2016 13:13:12 GMT

     [ https://issues.apache.org/jira/browse/SPARK-15469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved SPARK-15469.
-------------------------------
    Resolution: Invalid

HDFS/YARN question, not Spark

> Datanode and nodemanger appear in jps but arent start correctly.
> ----------------------------------------------------------------
>
>                 Key: SPARK-15469
>                 URL: https://issues.apache.org/jira/browse/SPARK-15469
>             Project: Spark
>          Issue Type: Bug
>            Reporter: Jon
>
> when I execute jps command datanode and nodemanagger appears, but it seems that is not starting correctly, because if I check logs it seems that they arent running correctly.
> Namenode log:
> 2016-05-22 11:40:37,725 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
> 2016-05-22 11:40:37,731 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: createNameNode []
> 2016-05-22 11:40:38,109 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
> 2016-05-22 11:40:38,217 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
> 2016-05-22 11:40:38,217 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system started
> 2016-05-22 11:40:38,220 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: fs.defaultFS is hdfs://masternode:9000
> 2016-05-22 11:40:38,221 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Clients are to use masternode:9000 to access this namenode/service.
> 2016-05-22 11:40:38,412 INFO org.apache.hadoop.hdfs.DFSUtil: Starting Web-server for hdfs at: http://0.0.0.0:50070
> 2016-05-22 11:40:38,486 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
> 2016-05-22 11:40:38,496 INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
> 2016-05-22 11:40:38,503 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.namenode is not defined
> 2016-05-22 11:40:38,509 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
> 2016-05-22 11:40:38,513 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context hdfs
> 2016-05-22 11:40:38,514 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
> 2016-05-22 11:40:38,514 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
> 2016-05-22 11:40:38,543 INFO org.apache.hadoop.http.HttpServer2: Added filter 'org.apache.hadoop.hdfs.web.AuthFilter' (class=org.apache.hadoop.hdfs.web.AuthFilter)
> 2016-05-22 11:40:38,544 INFO org.apache.hadoop.http.HttpServer2: addJerseyResourcePackage: packageName=org.apache.hadoop.hdfs.server.namenode.web.resources;org.apache.hadoop.hdfs.web.resources, pathSpec=/webhdfs/v1/*
> 2016-05-22 11:40:38,564 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 50070
> 2016-05-22 11:40:38,564 INFO org.mortbay.log: jetty-6.1.26
> 2016-05-22 11:40:38,751 INFO org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:50070
> 2016-05-22 11:40:38,785 WARN org.apache.hadoop.hdfs.server.common.Util: Path /home/hadoopadmin/namenodeeeeeeeee should be specified as a URI in configuration files. Please update hdfs configuration.
> 2016-05-22 11:40:38,785 WARN org.apache.hadoop.hdfs.server.common.Util: Path /home/hadoopadmin/namenodeeeeeeeee should be specified as a URI in configuration files. Please update hdfs configuration.
> 2016-05-22 11:40:38,785 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of data loss due to lack of redundant storage directories!
> 2016-05-22 11:40:38,785 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one namespace edits storage directory (dfs.namenode.edits.dir) configured. Beware of data loss due to lack of redundant storage directories!
> 2016-05-22 11:40:38,791 WARN org.apache.hadoop.hdfs.server.common.Util: Path /home/hadoopadmin/namenodeeeeeeeee should be specified as a URI in configuration files. Please update hdfs configuration.
> 2016-05-22 11:40:38,791 WARN org.apache.hadoop.hdfs.server.common.Util: Path /home/hadoopadmin/namenodeeeeeeeee should be specified as a URI in configuration files. Please update hdfs configuration.
> 2016-05-22 11:40:38,823 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: No KeyProvider found.
> 2016-05-22 11:40:38,824 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsLock is fair:true
> 2016-05-22 11:40:38,866 INFO org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000
> 2016-05-22 11:40:38,867 INFO org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
> 2016-05-22 11:40:38,868 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
> 2016-05-22 11:40:38,870 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: The block deletion will start around 2016 May 22 11:40:38
> 2016-05-22 11:40:38,872 INFO org.apache.hadoop.util.GSet: Computing capacity for map BlocksMap
> 2016-05-22 11:40:38,872 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
> 2016-05-22 11:40:38,873 INFO org.apache.hadoop.util.GSet: 2.0% max memory 889 MB = 17.8 MB
> 2016-05-22 11:40:38,873 INFO org.apache.hadoop.util.GSet: capacity      = 2^21 = 2097152 entries
> 2016-05-22 11:40:38,880 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: dfs.block.access.token.enable=false
> 2016-05-22 11:40:38,881 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: defaultReplication         = 1
> 2016-05-22 11:40:38,881 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplication             = 512
> 2016-05-22 11:40:38,881 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: minReplication             = 1
> 2016-05-22 11:40:38,881 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplicationStreams      = 2
> 2016-05-22 11:40:38,881 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: shouldCheckForEnoughRacks  = false
> 2016-05-22 11:40:38,881 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: replicationRecheckInterval = 3000
> 2016-05-22 11:40:38,881 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: encryptDataTransfer        = false
> 2016-05-22 11:40:38,881 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
> 2016-05-22 11:40:38,888 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner             = hadoopadmin (auth:SIMPLE)
> 2016-05-22 11:40:38,889 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup          = supergroup
> 2016-05-22 11:40:38,889 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled = true
> 2016-05-22 11:40:38,889 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: HA Enabled: false
> 2016-05-22 11:40:38,890 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Append Enabled: true
> 2016-05-22 11:40:39,102 INFO org.apache.hadoop.util.GSet: Computing capacity for map INodeMap
> 2016-05-22 11:40:39,102 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
> 2016-05-22 11:40:39,102 INFO org.apache.hadoop.util.GSet: 1.0% max memory 889 MB = 8.9 MB
> 2016-05-22 11:40:39,102 INFO org.apache.hadoop.util.GSet: capacity      = 2^20 = 1048576 entries
> 2016-05-22 11:40:39,104 INFO org.apache.hadoop.hdfs.server.namenode.FSDirectory: ACLs enabled? false
> 2016-05-22 11:40:39,104 INFO org.apache.hadoop.hdfs.server.namenode.FSDirectory: XAttrs enabled? true
> 2016-05-22 11:40:39,104 INFO org.apache.hadoop.hdfs.server.namenode.FSDirectory: Maximum size of an xattr: 16384
> 2016-05-22 11:40:39,104 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names occuring more than 10 times
> 2016-05-22 11:40:39,113 INFO org.apache.hadoop.util.GSet: Computing capacity for map cachedBlocks
> 2016-05-22 11:40:39,113 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
> 2016-05-22 11:40:39,113 INFO org.apache.hadoop.util.GSet: 0.25% max memory 889 MB = 2.2 MB
> 2016-05-22 11:40:39,113 INFO org.apache.hadoop.util.GSet: capacity      = 2^18 = 262144 entries
> 2016-05-22 11:40:39,115 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
> 2016-05-22 11:40:39,115 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
> 2016-05-22 11:40:39,115 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.extension     = 30000
> 2016-05-22 11:40:39,118 INFO org.apache.hadoop.hdfs.server.namenode.top.metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
> 2016-05-22 11:40:39,118 INFO org.apache.hadoop.hdfs.server.namenode.top.metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
> 2016-05-22 11:40:39,118 INFO org.apache.hadoop.hdfs.server.namenode.top.metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
> 2016-05-22 11:40:39,119 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Retry cache on namenode is enabled
> 2016-05-22 11:40:39,119 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
> 2016-05-22 11:40:39,122 INFO org.apache.hadoop.util.GSet: Computing capacity for map NameNodeRetryCache
> 2016-05-22 11:40:39,122 INFO org.apache.hadoop.util.GSet: VM type       = 64-bit
> 2016-05-22 11:40:39,122 INFO org.apache.hadoop.util.GSet: 0.029999999329447746% max memory 889 MB = 273.1 KB
> 2016-05-22 11:40:39,122 INFO org.apache.hadoop.util.GSet: capacity      = 2^15 = 32768 entries
> 2016-05-22 11:40:39,135 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /home/hadoopadmin/namenodeeeeeeeee/in_use.lock acquired by nodename 30068@masternode
> 2016-05-22 11:40:39,190 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Recovering unfinalized segments in /home/hadoopadmin/namenodeeeeeeeee/current
> 2016-05-22 11:40:39,191 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: No edit log streams selected.
> 2016-05-22 11:40:39,232 INFO org.apache.hadoop.hdfs.server.namenode.FSImageFormatPBINode: Loading 1 INodes.
> 2016-05-22 11:40:39,265 INFO org.apache.hadoop.hdfs.server.namenode.FSImageFormatProtobuf: Loaded FSImage in 0 seconds.
> 2016-05-22 11:40:39,265 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Loaded image for txid 0 from /home/hadoopadmin/namenodeeeeeeeee/current/fsimage_0000000000000000000
> 2016-05-22 11:40:39,275 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Need to save fs image? false (staleImage=false, haEnabled=false, isRollingUpgrade=false)
> 2016-05-22 11:40:39,275 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 1
> 2016-05-22 11:40:39,423 INFO org.apache.hadoop.hdfs.server.namenode.NameCache: initialized with 0 entries 0 lookups
> 2016-05-22 11:40:39,423 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Finished loading FSImage in 297 msecs
> 2016-05-22 11:40:39,686 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: RPC server is binding to masternode:9000
> 2016-05-22 11:40:39,693 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
> 2016-05-22 11:40:39,707 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 9000
> 2016-05-22 11:40:39,735 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered FSNamesystemState MBean
> 2016-05-22 11:40:39,736 WARN org.apache.hadoop.hdfs.server.common.Util: Path /home/hadoopadmin/namenodeeeeeeeee should be specified as a URI in configuration files. Please update hdfs configuration.
> 2016-05-22 11:40:39,748 INFO org.apache.hadoop.hdfs.server.namenode.LeaseManager: Number of blocks under construction: 0
> 2016-05-22 11:40:39,748 INFO org.apache.hadoop.hdfs.server.namenode.LeaseManager: Number of blocks under construction: 0
> 2016-05-22 11:40:39,748 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: initializing replication queues
> 2016-05-22 11:40:39,749 INFO org.apache.hadoop.hdfs.StateChange: STATE* Leaving safe mode after 0 secs
> 2016-05-22 11:40:39,749 INFO org.apache.hadoop.hdfs.StateChange: STATE* Network topology has 0 racks and 0 datanodes
> 2016-05-22 11:40:39,749 INFO org.apache.hadoop.hdfs.StateChange: STATE* UnderReplicatedBlocks has 0 blocks
> 2016-05-22 11:40:39,760 INFO org.apache.hadoop.hdfs.server.blockmanagement.DatanodeDescriptor: Number of failed storage changes from 0 to 0
> 2016-05-22 11:40:39,786 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Total number of blocks            = 0
> 2016-05-22 11:40:39,786 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of invalid blocks          = 0
> 2016-05-22 11:40:39,786 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of under-replicated blocks = 0
> 2016-05-22 11:40:39,786 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of  over-replicated blocks = 0
> 2016-05-22 11:40:39,786 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Number of blocks being written    = 0
> 2016-05-22 11:40:39,786 INFO org.apache.hadoop.hdfs.StateChange: STATE* Replication Queue initialization scan for invalid, over- and under-replicated blocks completed in 36 msec
> 2016-05-22 11:40:39,800 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 9000: starting
> 2016-05-22 11:40:39,801 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
> 2016-05-22 11:40:39,803 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: NameNode RPC up at: masternode/10.18.0.50:9000
> 2016-05-22 11:40:39,803 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Starting services required for active state
> 2016-05-22 11:40:39,807 INFO org.apache.hadoop.hdfs.server.blockmanagement.CacheReplicationMonitor: Starting CacheReplicationMonitor with interval 30000 milliseconds
> 2016-05-22 11:41:49,208 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:41:49,208 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:41:49,208 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 1
> 2016-05-22 11:41:49,208 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 5
> 2016-05-22 11:41:49,209 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 5
> 2016-05-22 11:41:49,212 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000001 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000001-0000000000000000002
> 2016-05-22 11:41:49,214 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 3
> 2016-05-22 11:42:49,280 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:42:49,280 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:42:49,280 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 3
> 2016-05-22 11:42:49,281 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 4
> 2016-05-22 11:42:49,281 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 4
> 2016-05-22 11:42:49,282 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000003 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000003-0000000000000000004
> 2016-05-22 11:42:49,282 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 5
> 2016-05-22 11:43:49,303 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:43:49,303 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:43:49,303 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 5
> 2016-05-22 11:43:49,304 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 4
> 2016-05-22 11:43:49,305 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 5
> 2016-05-22 11:43:49,306 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000005 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000005-0000000000000000006
> 2016-05-22 11:43:49,306 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 7
> 2016-05-22 11:44:49,320 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:44:49,320 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:44:49,320 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 7
> 2016-05-22 11:44:49,320 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 5
> 2016-05-22 11:44:49,322 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 5
> 2016-05-22 11:44:49,326 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000007 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000007-0000000000000000008
> 2016-05-22 11:44:49,326 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 9
> 2016-05-22 11:45:49,340 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:45:49,340 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:45:49,340 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 9
> 2016-05-22 11:45:49,341 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 4
> 2016-05-22 11:45:49,342 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 5
> 2016-05-22 11:45:49,343 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000009 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000009-0000000000000000010
> 2016-05-22 11:45:49,343 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 11
> 2016-05-22 11:46:49,380 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:46:49,380 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:46:49,381 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 11
> 2016-05-22 11:46:49,381 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 12
> 2016-05-22 11:46:49,382 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 13
> 2016-05-22 11:46:49,383 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000011 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000011-0000000000000000012
> 2016-05-22 11:46:49,383 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 13
> 2016-05-22 11:47:49,399 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:47:49,399 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:47:49,399 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 13
> 2016-05-22 11:47:49,399 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 4
> 2016-05-22 11:47:49,400 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 4
> 2016-05-22 11:47:49,401 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000013 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000013-0000000000000000014
> 2016-05-22 11:47:49,401 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 15
> 2016-05-22 11:48:49,419 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:48:49,419 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:48:49,419 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 15
> 2016-05-22 11:48:49,421 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 2 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 4
> 2016-05-22 11:48:49,424 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 2 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 7
> 2016-05-22 11:48:49,425 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000015 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000015-0000000000000000016
> 2016-05-22 11:48:49,425 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 17
> 2016-05-22 11:49:49,446 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:49:49,446 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:49:49,446 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 17
> 2016-05-22 11:49:49,447 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 5
> 2016-05-22 11:49:49,449 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 7
> 2016-05-22 11:49:49,450 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000017 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000017-0000000000000000018
> 2016-05-22 11:49:49,450 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 19
> 2016-05-22 11:50:49,475 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:50:49,475 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:50:49,475 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 19
> 2016-05-22 11:50:49,476 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 13
> 2016-05-22 11:50:49,477 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 13
> 2016-05-22 11:50:49,478 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000019 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000019-0000000000000000020
> 2016-05-22 11:50:49,478 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 21
> 2016-05-22 11:51:49,497 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:51:49,497 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:51:49,497 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 21
> 2016-05-22 11:51:49,499 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 2 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 4
> 2016-05-22 11:51:49,500 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 2 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 5
> 2016-05-22 11:51:49,501 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000021 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000021-0000000000000000022
> 2016-05-22 11:51:49,501 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 23
> 2016-05-22 11:52:49,516 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:52:49,516 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:52:49,516 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 23
> 2016-05-22 11:52:49,517 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 5
> 2016-05-22 11:52:49,518 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 1 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 6
> 2016-05-22 11:52:49,519 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000023 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000023-0000000000000000024
> 2016-05-22 11:52:49,519 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 25
> 2016-05-22 11:53:49,538 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:53:49,538 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:53:49,538 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 25
> 2016-05-22 11:53:49,538 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 5
> 2016-05-22 11:53:49,539 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 6
> 2016-05-22 11:53:49,540 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000025 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000025-0000000000000000026
> 2016-05-22 11:53:49,540 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 27
> 2016-05-22 11:54:49,570 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:54:49,571 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:54:49,571 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 27
> 2016-05-22 11:54:49,571 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 15
> 2016-05-22 11:54:49,572 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 16
> 2016-05-22 11:54:49,573 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000027 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000027-0000000000000000028
> 2016-05-22 11:54:49,573 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 29
> 2016-05-22 11:55:49,586 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:55:49,586 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:55:49,586 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 29
> 2016-05-22 11:55:49,586 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 4
> 2016-05-22 11:55:49,587 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 4
> 2016-05-22 11:55:49,588 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000029 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000029-0000000000000000030
> 2016-05-22 11:55:49,588 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 31
> 2016-05-22 11:56:49,647 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Roll Edit Log from 10.18.0.50
> 2016-05-22 11:56:49,647 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Rolling edit logs
> 2016-05-22 11:56:49,647 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Ending log segment 31
> 2016-05-22 11:56:49,647 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 2 SyncTimes(ms): 5
> 2016-05-22 11:56:49,649 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 2 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 3 SyncTimes(ms): 7
> 2016-05-22 11:56:49,649 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Finalizing edits file /home/hadoopadmin/namenodeeeeeeeee/current/edits_inprogress_0000000000000000031 -> /home/hadoopadmin/namenodeeeeeeeee/current/edits_0000000000000000031-0000000000000000032
> 2016-05-22 11:56:49,650 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 33
> Nodemanager log:
> STARTUP_MSG:   java = 1.8.0_91
> ************************************************************/
> 2016-05-22 11:41:11,219 INFO org.apache.hadoop.yarn.server.nodemanager.NodeManager: registered UNIX signal handlers for [TERM, HUP, INT]
> 2016-05-22 11:41:12,264 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher
> 2016-05-22 11:41:12,265 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher
> 2016-05-22 11:41:12,266 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService
> 2016-05-22 11:41:12,266 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices
> 2016-05-22 11:41:12,266 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl
> 2016-05-22 11:41:12,267 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher
> 2016-05-22 11:41:12,286 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl
> 2016-05-22 11:41:12,286 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.NodeManager
> 2016-05-22 11:41:12,326 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
> 2016-05-22 11:41:12,397 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
> 2016-05-22 11:41:12,398 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NodeManager metrics system started
> 2016-05-22 11:41:12,420 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.event.LogHandlerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler
> 2016-05-22 11:41:12,421 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.sharedcache.SharedCacheUploadService
> 2016-05-22 11:41:12,421 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: per directory file limit = 8192
> 2016-05-22 11:41:12,478 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: usercache path : file:/tmp/hadoop-hadoopadmin/nm-local-dir/usercache_DEL_1463913672424
> 2016-05-22 11:41:12,529 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker
> 2016-05-22 11:41:12,548 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:  Using ResourceCalculatorPlugin : org.apache.hadoop.yarn.util.LinuxResourceCalculatorPlugin@2dfaea86
> 2016-05-22 11:41:12,548 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:  Using ResourceCalculatorProcessTree : null
> 2016-05-22 11:41:12,549 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Physical memory check enabled: true
> 2016-05-22 11:41:12,549 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Virtual memory check enabled: true
> 2016-05-22 11:41:12,552 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: NodeManager configured with 8 G physical memory allocated to containers, which is more than 80% of the total physical memory available (3.9 G). Thrashing might happen.
> 2016-05-22 11:41:12,557 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Initialized nodemanager for null: physical-memory=8192 virtual-memory=17204 virtual-cores=8
> 2016-05-22 11:41:12,596 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
> 2016-05-22 11:41:12,619 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 40484
> 2016-05-22 11:41:12,651 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.api.ContainerManagementProtocolPB to the server
> 2016-05-22 11:41:12,651 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Blocking new container-requests as container manager rpc server is still starting.
> 2016-05-22 11:41:12,651 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
> 2016-05-22 11:41:12,652 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 40484: starting
> 2016-05-22 11:41:12,661 INFO org.apache.hadoop.yarn.server.nodemanager.security.NMContainerTokenSecretManager: Updating node address : ubuntuslave:40484
> 2016-05-22 11:41:12,668 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
> 2016-05-22 11:41:12,669 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 8040
> 2016-05-22 11:41:12,671 INFO org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB to the server
> 2016-05-22 11:41:12,672 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8040: starting
> 2016-05-22 11:41:12,672 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
> 2016-05-22 11:41:12,673 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Localizer started on port 8040
> 2016-05-22 11:41:12,675 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: ContainerManager started at ubuntuslave/10.17.0.89:40484
> 2016-05-22 11:41:12,675 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: ContainerManager bound to 0.0.0.0/0.0.0.0:0
> 2016-05-22 11:41:12,676 INFO org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer: Instantiating NMWebApp at 0.0.0.0:8042
> 2016-05-22 11:41:12,749 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
> 2016-05-22 11:41:12,758 INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
> 2016-05-22 11:41:12,763 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.nodemanager is not defined
> 2016-05-22 11:41:12,771 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
> 2016-05-22 11:41:12,773 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context node
> 2016-05-22 11:41:12,773 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
> 2016-05-22 11:41:12,773 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
> 2016-05-22 11:41:12,776 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /node/*
> 2016-05-22 11:41:12,777 INFO org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
> 2016-05-22 11:41:12,786 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 8042
> 2016-05-22 11:41:12,786 INFO org.mortbay.log: jetty-6.1.26
> 2016-05-22 11:41:12,813 INFO org.mortbay.log: Extract jar:file:/usr/local/hadoop-2.7.1/share/hadoop/yarn/hadoop-yarn-common-2.7.1.jar!/webapps/node to /tmp/Jetty_0_0_0_0_8042_node____19tj0x/webapp
> 2016-05-22 11:41:13,010 INFO org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@0.0.0.0:8042
> 2016-05-22 11:41:13,010 INFO org.apache.hadoop.yarn.webapp.WebApps: Web app /node started at 8042
> 2016-05-22 11:41:13,316 INFO org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
> 2016-05-22 11:41:13,324 INFO org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at masternode/10.18.0.50:8031
> 2016-05-22 11:41:13,417 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out 0 NM container statuses: []
> 2016-05-22 11:41:13,426 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Registering with RM using containers :[]
> 2016-05-22 11:41:33,471 INFO org.apache.hadoop.ipc.Client: Retrying connect to server
> datanode log:
> STARTUP_MSG:   java = 1.8.0_91
> ************************************************************/
> 2016-05-22 11:40:40,852 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT]
> 2016-05-22 11:40:41,523 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
> 2016-05-22 11:40:41,607 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
> 2016-05-22 11:40:41,607 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started
> 2016-05-22 11:40:41,612 INFO org.apache.hadoop.hdfs.server.datanode.BlockScanner: Initialized block scanner with targetBytesPerSec 1048576
> 2016-05-22 11:40:41,614 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is ubuntuslave
> 2016-05-22 11:40:41,620 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting DataNode with maxLockedMemory = 0
> 2016-05-22 11:40:41,644 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened streaming server at /0.0.0.0:50010
> 2016-05-22 11:40:41,646 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is 1048576 bytes/s
> 2016-05-22 11:40:41,646 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Number threads for balancing is 5
> 2016-05-22 11:40:41,739 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
> 2016-05-22 11:40:41,750 INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets.
> 2016-05-22 11:40:41,768 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.datanode is not defined
> 2016-05-22 11:40:41,776 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
> 2016-05-22 11:40:41,779 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context datanode
> 2016-05-22 11:40:41,780 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
> 2016-05-22 11:40:41,780 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
> 2016-05-22 11:40:41,796 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 52013
> 2016-05-22 11:40:41,796 INFO org.mortbay.log: jetty-6.1.26
> 2016-05-22 11:40:41,990 INFO org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52013
> 2016-05-22 11:40:42,109 INFO org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer: Listening HTTP traffic on /0.0.0.0:50075
> 2016-05-22 11:40:42,298 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnUserName = hadoopadmin
> 2016-05-22 11:40:42,298 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: supergroup = supergroup
> 2016-05-22 11:40:42,343 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
> 2016-05-22 11:40:42,361 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 50020
> 2016-05-22 11:40:42,388 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at /0.0.0.0:50020
> 2016-05-22 11:40:42,400 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received for nameservices: null
> 2016-05-22 11:40:42,424 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservices: <default>
> 2016-05-22 11:40:42,436 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool <registering> (Datanode Uuid unassigned) service to masternode/10.18.0.50:9000 starting to offer service
> 2016-05-22 11:40:42,444 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting
> 2016-05-22 11:40:42,445 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
> 2016-05-22 11:41:02,555 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: masternode/10.18.0.50:9000. Already tried 0 time(s); maxRetries=45
> yarn-site.xml:
> <configuration>
> <property>
> <name>yarn.resourcemanager.resource-tracker.address</name>
> <value>masternode:8031</value>
> </property>
> <property>
> <name>yarn.resourcemanager.address</name>
> <value>masternode:8032</value>
> </property>
> <property>
> <name>yarn.resourcemanager.scheduler.address</name>
> <value>masternode:8030</value>
> </property>
> <property>
> <name>yarn.resourcemanager.admin.address</name>
> <value>masternode:8033</value>
> </property>
> <property>
> <name>yarn.resourcemanager.webapp.address</name>
> <value>masternode:8088</value>
> </property>
> </configuration>
> core-site.xml:
> <code><configuration>
> <property>
> <name>fs.defaultFS</name>
> <value>masternode:9000</value>
> </property>
> </configuration>
> hdfs-site.xml:
> <configuration>
> <property>
> <name>dfs.name.dir</name>
> <value>file:///home/hadoopadmin/hadooptmp</value>
> </property>
> <property>
> <name>dfs.data.dir</name>
> <value>file:///home/hadoopadmin/hadooptmp</value>
> </property>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> </configuration>
> masters file:
> masternode
> slaves file:
> ubuntusalve1



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message