<rt id="bn8ez"></rt>
<label id="bn8ez"></label>

  • <span id="bn8ez"></span>

    <label id="bn8ez"><meter id="bn8ez"></meter></label>

    ivaneeo's blog

    自由的力量,自由的生活。

      BlogJava :: 首頁 :: 聯(lián)系 :: 聚合  :: 管理
      669 Posts :: 0 Stories :: 64 Comments :: 0 Trackbacks

    今早一來,突然發(fā)現(xiàn)使用-put命令往HDFS里傳數(shù)據(jù)傳不上去了,抱一大堆錯誤,然后我使用bin/hadoop dfsadmin -report查看系統(tǒng)狀態(tài)

    admin@adw1:/home/admin/joe.wangh/hadoop-0.19.2>bin/hadoop dfsadmin -report
    Configured Capacity: 0 (0 KB)
    Present Capacity: 0 (0 KB)
    DFS Remaining: 0 (0 KB)
    DFS Used: 0 (0 KB)
    DFS Used%: ?%

    -------------------------------------------------
    Datanodes available: 0 (0 total, 0 dead)

    使用bin/stop-all.sh關(guān)閉HADOOP

    admin@adw1:/home/admin/joe.wangh/hadoop-0.19.2>bin/stop-all.sh
    stopping jobtracker
    172.16.197.192: stopping tasktracker
    172.16.197.193: stopping tasktracker
    stopping namenode
    172.16.197.193: no datanode to stop
    172.16.197.192: no datanode to stop

    172.16.197.191: stopping secondarynamenode

    哦,看到了吧,發(fā)現(xiàn)datanode前面并沒有啟動起來。去DATANODE上查看一下日志

    admin@adw2:/home/admin/joe.wangh/hadoop-0.19.2/logs>vi hadoop-admin-datanode-adw2.hst.ali.dw.alidc.net.log

    ************************************************************/
    2010-07-21 10:12:11,987 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs in /home/admin/joe.wangh/hadoop/data/dfs.data.dir: namenode namespaceID = 898136669; datanode namespaceID = 2127444065
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:233)
            at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:148)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:288)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:206)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1239)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1194)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1202)
            at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1324)
    ......

    錯誤提示namespaceIDs不一致。

    下面給出兩種解決辦法,我使用的是第二種。

    Workaround 1: Start from scratch

    I can testify that the following steps solve this error, but the side effects won't make you happy (me neither). The crude workaround I have found is to:

    1.     stop the cluster

    2.     delete the data directory on the problematic datanode: the directory is specified by dfs.data.dir in conf/hdfs-site.xml; if you followed this tutorial, the relevant directory is /usr/local/hadoop-datastore/hadoop-hadoop/dfs/data

    3.     reformat the namenode (NOTE: all HDFS data is lost during this process!)

    4.     restart the cluster

    When deleting all the HDFS data and starting from scratch does not sound like a good idea (it might be ok during the initial setup/testing), you might give the second approach a try.

    Workaround 2: Updating namespaceID of problematic datanodes

    Big thanks to Jared Stehler for the following suggestion. I have not tested it myself yet, but feel free to try it out and send me your feedback. This workaround is "minimally invasive" as you only have to edit one file on the problematic datanodes:

    1.     stop the datanode

    2.     edit the value of namespaceID in <dfs.data.dir>/current/VERSION to match the value of the current namenode

    3.     restart the datanode

    If you followed the instructions in my tutorials, the full path of the relevant file is /usr/local/hadoop-datastore/hadoop-hadoop/dfs/data/current/VERSION (background: dfs.data.dir is by default set to ${hadoop.tmp.dir}/dfs/data, and we set hadoop.tmp.dir to /usr/local/hadoop-datastore/hadoop-hadoop).

    If you wonder how the contents of VERSION look like, here's one of mine:

    #contents of <dfs.data.dir>/current/VERSION

    namespaceID=393514426

    storageID=DS-1706792599-10.10.10.1-50010-1204306713481

    cTime=1215607609074

    storageType=DATA_NODE

    layoutVersion=-13

     

    原因:每次namenode format會重新創(chuàng)建一個namenodeId,而tmp/dfs/data下包含了上次format下的id,namenode format清空了namenode下的數(shù)據(jù),但是沒有晴空datanode下的數(shù)據(jù),導(dǎo)致啟動時失敗,所要做的就是每次fotmat前,清空tmp一下 的所有目錄.

    posted on 2011-06-09 14:20 ivaneeo 閱讀(565) 評論(0)  編輯  收藏 所屬分類:
    主站蜘蛛池模板: 亚洲av日韩av无码黑人| 99久久99久久免费精品小说| 美女被cao免费看在线看网站| 亚洲人成免费网站| 久久成人a毛片免费观看网站| 国产综合精品久久亚洲| 中中文字幕亚洲无线码| 亚洲国产精品狼友中文久久久| 日本免费污片中国特一级| 亚洲精品无码高潮喷水A片软| 久久乐国产精品亚洲综合| 999国内精品永久免费观看| 免费人成视频在线观看免费| 免费大黄网站在线观| 久久久精品免费视频| 亚洲中文字幕无码mv| 毛片免费全部播放一级| 亚洲欧美成人一区二区三区| 亚洲精品夜夜夜妓女网| 免费看男人j放进女人j免费看| 亚洲狠狠综合久久| 四虎国产精品免费视| 亚洲视频免费在线看| 国产人成亚洲第一网站在线播放| 亚洲一区二区三区影院| 热re99久久6国产精品免费| 亚洲成a人片在线观看天堂无码 | 亚洲精品在线观看视频| 国产一区二区三区在线观看免费| 久久国产乱子伦免费精品| 亚洲av无码电影网| 国产精品免费看香蕉| 日韩人妻一区二区三区免费| 在线播放免费人成视频网站| 国产精品久久亚洲不卡动漫 | 阿v视频免费在线观看| 亚洲AV综合色区无码二区爱AV| 久久久久亚洲Av片无码v | 亚洲Aⅴ在线无码播放毛片一线天 亚洲avav天堂av在线网毛片 | 亚洲欧美日韩自偷自拍| 亚洲国产精品久久久久秋霞影院|