2007年3月12日
我在Run Nutch的時(shí)候出現(xiàn)這樣的錯(cuò)誤 -
08/07/07 04:05:41 INFO conf.Configuration: found resource crawl-urlfilter.txt at file:/home/hut/installfiles/nutch-0.9/out/production/nutch-0.9/crawl-urlfilter.txt
08/07/07 04:05:41 INFO conf.Configuration: found resource parse-plugins.xml at file:/home/hut/installfiles/nutch-0.9/out/production/nutch-0.9/parse-plugins.xml
08/07/07 04:05:41 INFO fetcher.Fetcher: fetching http://www.yale.edu/
08/07/07 04:05:41 INFO fetcher.Fetcher: fetching http://www.harvard.edu/
08/07/07 04:05:41 INFO fetcher.Fetcher: fetch of http://www.harvard.edu/ failed with: org.apache.nutch.protocol.ProtocolNotFound: protocol not found for url=http
08/07/07 04:05:41 INFO fetcher.Fetcher: fetch of http://www.yale.edu/ failed with: org.apache.nutch.protocol.ProtocolNotFound: protocol not found for url=http
解決方法：nutch-site.xml
<property>
<name>plugin.includes</name>
<value>
nutch-extensionpoints|protocol-http|urlfilter-regex|parse-(text|html|js)|index-basic|query-(basic|site|url)|summary-basic|scoring-opic|urlnormalizer-(pass|regex|basic)
</value>
<description>Regular expression naming plugin directory names to
include. Any plugin not matching this expression is excluded.
In any case you need at least include the nutch-extensionpoints plugin. By
default Nutch includes crawling just HTML and plain text via HTTP,
and basic indexing and search plugins. In order to use HTTPS please enable
protocol-httpclient, but be aware of possible intermittent problems with the
underlying commons-httpclient library.
</description>
</property>
nutch-extensionpoints|被我錯(cuò)誤的刪除了,還原以后一切工作正常. 默認(rèn)情況下nutch0.9的目錄結(jié)構(gòu)中并沒(méi)有plugin.includes這個(gè)properties, 它會(huì)載入nutch-default.xml里面的plugin.includes所以定義的所有的plugin. 在nutch-site.xml編輯/加入 plugin.includes properties的目的是為了加入我們自己的plugin而覆蓋nutch-default.xml定義的.
Nutch0.9 Crawl在Run的時(shí)候，有時(shí)候會(huì)出現(xiàn) -
java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.lucene.index.MultiReader.isDeleted(MultiReader.java:113)
at org.apache.nutch.indexer.DeleteDuplicates$InputFormat$DDRecordReader.next(DeleteDuplicates.java:176)
at org.apache.hadoop.mapred.MapTask$1.next(MapTask.java:157)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:46)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:175)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:126)
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:604)
at org.apache.nutch.indexer.DeleteDuplicates.dedup(DeleteDuplicates.java:439)
at org.apache.nutch.crawl.Crawl.main(Crawl.java:135)
問(wèn)題的解決方法:
https://issues.apache.org/jira/browse/NUTCH-525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12515955
那是2007年三月二日,我發(fā)了<
定位Java類(lèi)中的每個(gè)方法在類(lèi)文件中的位置>引起了幾個(gè)大蝦的注意并且給我指點(diǎn)，當(dāng)時(shí)看了以后也沒(méi)有怎么弄，也不知道怎么弄.
這是cape給我的建議：
最直接的辦法應(yīng)該是分析class的bytecode,找到每個(gè)Method_info->Code Attribute->LineNumberTable Attribute今天空閑得慌而是我拿這個(gè)method linenumber幾個(gè)keyword在網(wǎng)上狂搜了一把，
http://www.ibm.com/developerworks/cn/java/j-dyn0302/index.html
得到美文一篇，用Javassist來(lái)做. Javassist是一個(gè)專(zhuān)門(mén)操控java class的bytecode的類(lèi)，可以動(dòng)態(tài)的改變其狀態(tài)，我沒(méi)有深究它到底有多大的能耐，有興趣可以去它的官方網(wǎng)站看看,
http://www.csg.is.titech.ac.jp/~chiba/javassist/但是它確實(shí)可以象cape指點(diǎn)的那樣，讓我實(shí)現(xiàn)對(duì)java類(lèi)中的每個(gè)方法的定位(得到類(lèi)中的某個(gè)方法的lineRange)
而是我就隨便翻了幾個(gè)例子以后根據(jù)在線(xiàn)文檔寫(xiě)了不到100行代碼輕松搞定。:) 我高興啊。
干這個(gè)事情的兩個(gè)java的source code可以從這里得到:
http://m.tkk7.com/Files/xmp123/MethodParser.zip
代碼很簡(jiǎn)單，我就不解釋代碼了。
這是示例結(jié)果：
------------------------------------------------------------------------------------------------------------------------------
Analyzing net.blogjava.xmp123.MyMethodInfo ...
net.blogjava.xmp123.MyMethodInfo.getLineList() locate between 56 and 56
net.blogjava.xmp123.MyMethodInfo.updateLineList() locate between 63 and 80
net.blogjava.xmp123.MyMethodInfo.addLineToLineList(int) locate between 87 and 89
net.blogjava.xmp123.MyMethodInfo.setLineList(java.util.List) locate between 95 and 97
...
------------------------------------------------------------------------------------------------------------------------------
希望這東西有用......
摘要:
閱讀全文