<rt id="bn8ez"></rt>
<label id="bn8ez"></label>

  • <span id="bn8ez"></span>

    <label id="bn8ez"><meter id="bn8ez"></meter></label>

    ivaneeo's blog

    自由的力量,自由的生活。

      BlogJava :: 首頁 :: 聯(lián)系 :: 聚合  :: 管理
      669 Posts :: 0 Stories :: 64 Comments :: 0 Trackbacks

    今早一來,突然發(fā)現(xiàn)使用-put命令往HDFS里傳數(shù)據(jù)傳不上去了,抱一大堆錯誤,然后我使用bin/hadoop dfsadmin -report查看系統(tǒng)狀態(tài)

    admin@adw1:/home/admin/joe.wangh/hadoop-0.19.2>bin/hadoop dfsadmin -report
    Configured Capacity: 0 (0 KB)
    Present Capacity: 0 (0 KB)
    DFS Remaining: 0 (0 KB)
    DFS Used: 0 (0 KB)
    DFS Used%: ?%

    -------------------------------------------------
    Datanodes available: 0 (0 total, 0 dead)

    使用bin/stop-all.sh關(guān)閉HADOOP

    admin@adw1:/home/admin/joe.wangh/hadoop-0.19.2>bin/stop-all.sh
    stopping jobtracker
    172.16.197.192: stopping tasktracker
    172.16.197.193: stopping tasktracker
    stopping namenode
    172.16.197.193: no datanode to stop
    172.16.197.192: no datanode to stop

    172.16.197.191: stopping secondarynamenode

    哦,看到了吧,發(fā)現(xiàn)datanode前面并沒有啟動起來。去DATANODE上查看一下日志

    admin@adw2:/home/admin/joe.wangh/hadoop-0.19.2/logs>vi hadoop-admin-datanode-adw2.hst.ali.dw.alidc.net.log

    ************************************************************/
    2010-07-21 10:12:11,987 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in /home/admin/joe.wangh/hadoop/data/dfs.data.dir: namenode namespaceID = 898136669; datanode namespaceID = 2127444065
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:233)
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:148)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:288)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:206)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1239)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1194)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1202)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1324)
    ......

    錯誤提示namespaceIDs不一致。

    下面給出兩種解決辦法,我使用的是第二種。

    Workaround 1: Start from scratch

    I can testify that the following steps solve this error, but the side effects won't make you happy (me neither). The crude workaround I have found is to:

    1.     stop the cluster

    2.     delete the data directory on the problematic datanode: the directory is specified by dfs.data.dir in conf/hdfs-site.xml; if you followed this tutorial, the relevant directory is /usr/local/hadoop-datastore/hadoop-hadoop/dfs/data

    3.     reformat the namenode (NOTE: all HDFS data is lost during this process!)

    4.     restart the cluster

    When deleting all the HDFS data and starting from scratch does not sound like a good idea (it might be ok during the initial setup/testing), you might give the second approach a try.

    Workaround 2: Updating namespaceID of problematic datanodes

    Big thanks to Jared Stehler for the following suggestion. I have not tested it myself yet, but feel free to try it out and send me your feedback. This workaround is "minimally invasive" as you only have to edit one file on the problematic datanodes:

    1.     stop the datanode

    2.     edit the value of namespaceID in <dfs.data.dir>/current/VERSION to match the value of the current namenode

    3.     restart the datanode

    If you followed the instructions in my tutorials, the full path of the relevant file is /usr/local/hadoop-datastore/hadoop-hadoop/dfs/data/current/VERSION (background: dfs.data.dir is by default set to ${hadoop.tmp.dir}/dfs/data, and we set hadoop.tmp.dir to /usr/local/hadoop-datastore/hadoop-hadoop).

    If you wonder how the contents of VERSION look like, here's one of mine:

    #contents of <dfs.data.dir>/current/VERSION

    namespaceID=393514426

    storageID=DS-1706792599-10.10.10.1-50010-1204306713481

    cTime=1215607609074

    storageType=DATA_NODE

    layoutVersion=-13

     

    原因:每次namenode format會重新創(chuàng)建一個namenodeId,而tmp/dfs/data下包含了上次format下的id,namenode format清空了namenode下的數(shù)據(jù),但是沒有晴空datanode下的數(shù)據(jù),導(dǎo)致啟動時失敗,所要做的就是每次fotmat前,清空tmp一下 的所有目錄.

    posted on 2011-06-09 14:20 ivaneeo 閱讀(565) 評論(0)  編輯  收藏 所屬分類:
    主站蜘蛛池模板: 亚洲人成人无码.www石榴| 久热综合在线亚洲精品| 亚洲中文无码永久免费| 100部毛片免费全部播放完整| 亚洲人成色77777| 久久免费视频网站| 亚洲国产精品自在线一区二区 | 亚洲精品无码久久一线| 一级做a爰片久久免费| 亚洲中文字幕视频国产| 亚欧乱色国产精品免费视频| 中文亚洲AV片不卡在线观看| 99久久精品毛片免费播放| 久久久久无码精品亚洲日韩| 久久精品国产这里是免费| 亚洲美女激情视频| 一二三四在线播放免费观看中文版视频 | 免费国产美女爽到喷出水来视频| 爱情岛论坛亚洲品质自拍视频网站 | 激情综合亚洲色婷婷五月APP| 最近最好的中文字幕2019免费 | 特黄aa级毛片免费视频播放| 中文字幕中韩乱码亚洲大片| 日本三级在线观看免费| 亚洲无线一二三四区| 日本成人在线免费观看| sihu国产精品永久免费| 亚洲视频免费观看| 成年女人喷潮毛片免费播放| 黄色免费在线观看网址| 亚洲AV无码久久| 成人黄页网站免费观看大全| 香蕉国产在线观看免费| 亚洲国产综合专区电影在线| 免费电视剧在线观看| 亚州**色毛片免费观看| 亚洲一区二区三区夜色| 国产女高清在线看免费观看| 男人进去女人爽免费视频国产| 亚洲第一永久在线观看| 91嫩草私人成人亚洲影院|