UberOS271 系統登入的帳號及密碼均為 bigred, 登入後執行 dkls 命令, 執行結果如下 :
bigred@dk:~$ dkls
Docker Utility 0.2 (2015/07/21)
[Container]
--------------------------------------------------------------------------------------------
myweb(816ddf9479ac) Exited
nma2(c639239d3bf8) 172.17.10.51 Exited ()
nma1(f907aa2662c3) 172.17.10.50 Exited ()
rma(68bc2e91dbf2) 172.17.10.30 Exited ()
dna2(febce027f05e) 172.17.10.21 Exited ()
dna1(016988f1e165) 172.17.10.20 Exited ()
nna(d359071723e9) 172.17.10.10 Exited (user:ds01 ds02)
cla01(ad24d7c10d48) 172.17.10.100 Exited (dk:2211->cla01:22, user:ds01 ds02)
[Images]
--------------------------------------------------------------------------------------------
dafu/bigdata 0.2 3cf2e2291746 9 weeks ago 1.491 GB
dafu/myweb latest 7768c27140cc 4 months ago 367.3 MB
由以上資訊得知, 在 UberOS271 系統中已內建 8 部貨櫃主機, 啟動這些貨櫃主機 (不包含 myweb), 命令如下 :
bigred@dk:~$ dkstart a
cla01 starting
java version "1.7.0_79"
Scala compiler version 2.11.5 -- Copyright 2002-2013, LAMP/EPFL
dk:2211->172.17.10.100:22
nna starting
java version "1.7.0_79"
Scala compiler version 2.11.5 -- Copyright 2002-2013, LAMP/EPFL
dna1 starting
java version "1.7.0_79"
Scala compiler version 2.11.5 -- Copyright 2002-2013, LAMP/EPFL
dna2 starting
java version "1.7.0_79"
Scala compiler version 2.11.5 -- Copyright 2002-2013, LAMP/EPFL
rma starting
java version "1.7.0_79"
Scala compiler version 2.11.5 -- Copyright 2002-2013, LAMP/EPFL
nma1 starting
java version "1.7.0_79"
Scala compiler version 2.11.5 -- Copyright 2002-2013, LAMP/EPFL
nma2 starting
java version "1.7.0_79"
Scala compiler version 2.11.5 -- Copyright 2002-2013, LAMP/EPFL
多部貨櫃主機啟動後, 執行 "starthdfs a" 命令, 將 HDFS 系統啟動
bigred@dk:~$ starthdfs a
starting namenode, logging to /tmp/hadoop-bigred-namenode-nna.out
starting secondarynamenode, logging to /tmp/hadoop-bigred-secondarynamenode-nna.out
starting datanode, logging to /tmp/hadoop-bigred-datanode-dna1.out
starting datanode, logging to /tmp/hadoop-bigred-datanode-dna2.out
執行 "hdfs dfs -ls /" 命令, 檢視 HDFS 檔案系統的根目錄內容
bigred@dk:~$ hdfs dfs -ls /
Found 2 items
drwxrwxrwx - bigred biguser 0 2015-09-28 15:28 /tmp
drwxr-xr-x - bigred biguser 0 2015-09-01 19:18 /user
確認 HDFS 正常執行, 執行 "startyarn a" 命令, 將 YARN 系統啟動
bigred@dk:~$ startyarn a
starting resourcemanager, logging to /tmp/yarn-bigred-resourcemanager-rma.out
starting historyserver, logging to /home/bigred/jhs/mapred-bigred-historyserver-rma.out
starting nodemanager, logging to /tmp/yarn-bigred-nodemanager-nma1.out
starting nodemanager, logging to /tmp/yarn-bigred-nodemanager-nma2.out
執行 "yarn node -list -all" 命令, 確認 nma1 及 nma2 這二部 Node Manager 主機正常運作
bigred@dk:~$ yarn node -list -all
15/10/08 20:04:34 INFO client.RMProxy: Connecting to ResourceManager at rma/172.17.10.30:8032
15/10/08 20:04:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Total Nodes:2
Node-Id Node-State Node-Http-Address Number-of-Running-Containers
nma2:44635 RUNNING nma2:8042 0
nma1:46128 RUNNING nma1:8042 0
登入 資料科學家 主機 (cla01), 帳號及密碼均為 ds01, 命令如下 :
bigred@dk:~$ ssh ds01@cla01
ds01@cla01's password:
Welcome to Ubuntu 14.04.3 LTS (GNU/Linux 3.16.0-50-generic x86_64)
* Documentation: https://help.ubuntu.com/
Last login: Tue Oct 6 11:17:29 2015 from 172.17.42.1
ds01@cla01:~$
執行以下 Mapreduce 程式, 確認 Hadoop 系統能正常運作.
::
File Input Format Counters
Bytes Read=118
File Output Format Counters
Bytes Written=97
Job Finished in 56.222 seconds
Estimated value of Pi is 4.00000000000000000000
離開 資料科學家 主機 (cla01), 命令如下 :
ds01@cla01:~$ exit
logout
Connection to cla01 closed.
關閉 YARN 分散運算系統
bigred@dk:~$ stopyarn a
stopping resourcemanager
stopping historyserver
stopping nodemanager
nodemanager did not stop gracefully after 5 seconds: killing with kill -9
stopping nodemanager
nodemanager did not stop gracefully after 5 seconds: killing with kill -9
關閉 HDFS 分散檔案系統
bigred@dk:~$ stophdfs a
stopping namenode
stopping secondarynamenode
stopping datanode
stopping datanode
關閉所有貨櫃主機
bigred@dk:~$ dkstop a
cla01 Exiting
dk:2211->172.17.10.100:22 deleted
nna Exiting
dna1 Exiting
dna2 Exiting
rma Exiting
nma1 Exiting
nma2 Exiting
關閉 UberOS271 主機
bigred@dk:~$ bye
沒有留言:
張貼留言