)斷點續(xù)傳的原理?
其實斷點續(xù)傳的原理很簡單,就是在Http的請求上和一般的下載有所不同而已。
打個比方,瀏覽器請求服務器上的一個文時,所發(fā)出的請求如下:
假設服務器域名為wwww.sjtu.edu.cn,文件名為down.zip。
GET?/down.zip?HTTP/1.1
Accept:?image/gif,?image/x-xbitmap,?image/jpeg,?image/pjpeg,?application/vnd.ms-
excel,?application/msword,?application/vnd.ms-powerpoint,?*/*
Accept-Language:?zh-cn
Accept-Encoding:?gzip,?deflate
User-Agent:?Mozilla/4.0?(compatible;?MSIE?5.01;?Windows?NT?5.0)
Connection:?Keep-Alive
服務器收到請求后,按要求尋找請求的文件,提取文件的信息,然后返回給瀏覽器,返回信息如下:
200
Content-Length=106786028
Accept-Ranges=bytes
Date=Mon,?30?Apr?2001?12:56:11?GMT
ETag=W/"02ca57e173c11:95b"
Content-Type=application/octet-stream
Server=Microsoft-IIS/5.0
Last-Modified=Mon,?30?Apr?2001?12:56:11?GMT
所謂斷點續(xù)傳,也就是要從文件已經下載的地方開始繼續(xù)下載。所以在客戶端瀏覽器傳給
Web服務器的時候要多加一條信息--從哪里開始。
下面是用自己編的一個"瀏覽器"來傳遞請求信息給Web服務器,要求從2000070字節(jié)開始。
GET?/down.zip?HTTP/1.0
User-Agent:?NetFox
RANGE:?bytes=2000070-
Accept:?text/html,?image/gif,?image/jpeg,?*;?q=.2,?*/*;?q=.2
仔細看一下就會發(fā)現多了一行RANGE:?bytes=2000070-
這一行的意思就是告訴服務器down.zip這個文件從2000070字節(jié)開始傳,前面的字節(jié)不用傳了。
服務器收到這個請求以后,返回的信息如下:
206
Content-Length=106786028
Content-Range=bytes?2000070-106786027/106786028
Date=Mon,?30?Apr?2001?12:55:20?GMT
ETag=W/"02ca57e173c11:95b"
Content-Type=application/octet-stream
Server=Microsoft-IIS/5.0
Last-Modified=Mon,?30?Apr?2001?12:55:20?GMT
和前面服務器返回的信息比較一下,就會發(fā)現增加了一行:
Content-Range=bytes?2000070-106786027/106786028
返回的代碼也改為206了,而不再是200了。
知道了以上原理,就可以進行斷點續(xù)傳的編程了。
(二)Java實現斷點續(xù)傳的關鍵幾點?
(1)用什么方法實現提交RANGE:?bytes=2000070-。
當然用最原始的Socket是肯定能完成的,不過那樣太費事了,其實Java的net包中提供了這種功能。代碼如下:
URL?url?=?new?URL("http://www.sjtu.edu.cn/down.zip");
HttpURLConnection?httpConnection?=?(HttpURLConnection)url.openConnection
();
//設置User-Agent
httpConnection.setRequestProperty("User-Agent","NetFox");
//設置斷點續(xù)傳的開始位置
httpConnection.setRequestProperty("RANGE","bytes=2000070");
//獲得輸入流
InputStream?input?=?httpConnection.getInputStream();
從輸入流中取出的字節(jié)流就是down.zip文件從2000070開始的字節(jié)流。
大家看,其實斷點續(xù)傳用Java實現起來還是很簡單的吧。
接下來要做的事就是怎么保存獲得的流到文件中去了。
保存文件采用的方法。
我采用的是IO包中的RandAccessFile類。
操作相當簡單,假設從2000070處開始保存文件,代碼如下:
RandomAccess?oSavedFile?=?new?RandomAccessFile("down.zip","rw");
long?nPos?=?2000070;
//定位文件指針到nPos位置
oSavedFile.seek(nPos);
byte[]?b?=?new?byte[1024];
int?nRead;
//從輸入流中讀入字節(jié)流,然后寫到文件中
while((nRead=input.read(b,0,1024))?>?0)
{
oSavedFile.write(b,0,nRead);
}
怎么樣,也很簡單吧。
接下來要做的就是整合成一個完整的程序了。包括一系列的線程控制等等。
(三)斷點續(xù)傳內核的實現
主要用了6個類,包括一個測試類。
SiteFileFetch.java負責整個文件的抓取,控制內部線程(FileSplitterFetch類)。
FileSplitterFetch.java負責部分文件的抓取。
FileAccess.java負責文件的存儲。
SiteInfoBean.java要抓取的文件的信息,如文件保存的目錄,名字,抓取文件的URL等。
Utility.java工具類,放一些簡單的方法。
TestMethod.java測試類。
下面是源程序:?
/*
**SiteFileFetch.java
*/
package?NetFox;
import?java.io.*;
import?java.net.*;
public?class?SiteFileFetch?extends?Thread?{
SiteInfoBean?siteInfoBean?=?null;?//文件信息Bean
long[]?nStartPos;?//開始位置
long[]?nEndPos;?//結束位置
FileSplitterFetch[]?fileSplitterFetch;?//子線程對象
long?nFileLength;?//文件長度
boolean?bFirst?=?true;?//是否第一次取文件
boolean?bStop?=?false;?//停止標志
File?tmpFile;?//文件下載的臨時信息
DataOutputStream?output;?//輸出到文件的輸出流
public?SiteFileFetch(SiteInfoBean?bean)?throws?IOException
{
siteInfoBean?=?bean;
//tmpFile?=?File.createTempFile?("zhong","1111",new?File(bean.getSFilePath()));
tmpFile?=?new?File(bean.getSFilePath()+File.separator?+?bean.getSFileName()+".info");
if(tmpFile.exists?())
{
bFirst?=?false;
read_nPos();
}
else
{
nStartPos?=?new?long[bean.getNSplitter()];
nEndPos?=?new?long[bean.getNSplitter()];
}
}
public?void?run()
{
//獲得文件長度
//分割文件
//實例FileSplitterFetch
//啟動FileSplitterFetch線程
//等待子線程返回
try{
if(bFirst)
{
nFileLength?=?getFileSize();
if(nFileLength?==?-1)
{
System.err.println("File?Length?is?not?known!");
}
else?if(nFileLength?==?-2)
{
System.err.println("File?is?not?access!");
}
else
{
for(int?i=0;i<nStartPos.length;i++)
{
nStartPos[i]?=?(long)(i*(nFileLength/nStartPos.length));
}
for(int?i=0;i<nEndPos.length-1;i++)
{
nEndPos[i]?=?nStartPos[i+1];
}
nEndPos[nEndPos.length-1]?=?nFileLength;
}
}
//啟動子線程
fileSplitterFetch?=?new?FileSplitterFetch[nStartPos.length];
for(int?i=0;i<nStartPos.length;i++)
{
fileSplitterFetch[i]?=?new?FileSplitterFetch(siteInfoBean.getSSiteURL(),
siteInfoBean.getSFilePath()?+?File.separator?+?siteInfoBean.getSFileName(),
nStartPos[i],nEndPos[i],i);
Utility.log("Thread?"?+?i?+?"?,?nStartPos?=?"?+?nStartPos[i]?+?",?nEndPos?=?"?+?nEndPos[i]);
fileSplitterFetch[i].start();
}
//?fileSplitterFetch[nPos.length-1]?=?new?FileSplitterFetch(siteInfoBean.getSSiteURL(),
siteInfoBean.getSFilePath()?+?File.separator?+?siteInfoBean.getSFileName(),nPos[nPos.length-1],nFileLength,nPos.length-1);
//?Utility.log("Thread?"?+?(nPos.length-1)?+?"?,?nStartPos?=?"?+?nPos[nPos.length-1]?+?",
nEndPos?=?"?+?nFileLength);
//?fileSplitterFetch[nPos.length-1].start();
//等待子線程結束
//int?count?=?0;
//是否結束while循環(huán)
boolean?breakWhile?=?false;
while(!bStop)
{
write_nPos();
Utility.sleep(500);
breakWhile?=?true;
for(int?i=0;i<nStartPos.length;i++)
{
if(!fileSplitterFetch[i].bDownOver)
{
breakWhile?=?false;
break;
}
}
if(breakWhile)
break;
//count++;
//if(count>4)
//?siteStop();
}
System.err.println("文件下載結束!");
}
catch(Exception?e){e.printStackTrace?();}
}
//獲得文件長度
public?long?getFileSize()
{
int?nFileLength?=?-1;
try{
URL?url?=?new?URL(siteInfoBean.getSSiteURL());
HttpURLConnection?httpConnection?=?(HttpURLConnection)url.openConnection?();
httpConnection.setRequestProperty("User-Agent","NetFox");
int?responseCode=httpConnection.getResponseCode();
if(responseCode>=400)
{
processErrorCode(responseCode);
return?-2;?//-2?represent?access?is?error
}
String?sHeader;
for(int?i=1;;i++)
{
//DataInputStream?in?=?new?DataInputStream(httpConnection.getInputStream?());
//Utility.log(in.readLine());
sHeader=httpConnection.getHeaderFieldKey(i);
if(sHeader!=null)
{
if(sHeader.equals("Content-Length"))
{
nFileLength?=?Integer.parseInt(httpConnection.getHeaderField(sHeader));
break;
}
}
else
break;
}
}
catch(IOException?e){e.printStackTrace?();}
catch(Exception?e){e.printStackTrace?();}
Utility.log(nFileLength);
return?nFileLength;
}
//保存下載信息(文件指針位置)
private?void?write_nPos()
{
try{
output?=?new?DataOutputStream(new?FileOutputStream(tmpFile));
output.writeInt(nStartPos.length);
for(int?i=0;i<nStartPos.length;i++)
{
//?output.writeLong(nPos[i]);
output.writeLong(fileSplitterFetch[i].nStartPos);
output.writeLong(fileSplitterFetch[i].nEndPos);
}
output.close();
}
catch(IOException?e){e.printStackTrace?();}
catch(Exception?e){e.printStackTrace?();}
}
//讀取保存的下載信息(文件指針位置)
private?void?read_nPos()
{
try{
DataInputStream?input?=?new?DataInputStream(new?FileInputStream(tmpFile));
int?nCount?=?input.readInt();
nStartPos?=?new?long[nCount];
nEndPos?=?new?long[nCount];
for(int?i=0;i<nStartPos.length;i++)
{
nStartPos[i]?=?input.readLong();
nEndPos[i]?=?input.readLong();
}
input.close();
}
catch(IOException?e){e.printStackTrace?();}
catch(Exception?e){e.printStackTrace?();}
}
private?void?processErrorCode(int?nErrorCode)
{
System.err.println("Error?Code?:?"?+?nErrorCode);
}
//停止文件下載
public?void?siteStop()
{
bStop?=?true;
for(int?i=0;i<nStartPos.length;i++)
fileSplitterFetch[i].splitterStop();
}
}
/*
**FileSplitterFetch.java
*/
package?NetFox;
import?java.io.*;
import?java.net.*;
public?class?FileSplitterFetch?extends?Thread?{
String?sURL;?//File?URL
long?nStartPos;?//File?Snippet?Start?Position
long?nEndPos;?//File?Snippet?End?Position
int?nThreadID;?//Thread's?ID
boolean?bDownOver?=?false;?//Downing?is?over
boolean?bStop?=?false;?//Stop?identical
FileAccessI?fileAccessI?=?null;?//File?Access?interface
public?FileSplitterFetch(String?sURL,String?sName,long?nStart,long?nEnd,int?id)?throws?IOException
{
this.sURL?=?sURL;
this.nStartPos?=?nStart;
this.nEndPos?=?nEnd;
nThreadID?=?id;
fileAccessI?=?new?FileAccessI(sName,nStartPos);
}
public?void?run()
{
while(nStartPos?<?nEndPos?&&?!bStop)
{
try{
URL?url?=?new?url(/sURL);
HttpURLConnection?httpConnection?=?(HttpURLConnection)url.openConnection?();
httpConnection.setRequestProperty("User-Agent","NetFox");
String?sProperty?=?"bytes="+nStartPos+"-";
httpConnection.setRequestProperty("RANGE",sProperty);
Utility.log(sProperty);
InputStream?input?=?httpConnection.getInputStream();
//logResponseHead(httpConnection);
byte[]?b?=?new?byte[1024];
int?nRead;
while((nRead=input.read(b,0,1024))?>?0?&&?nStartPos?<?nEndPos?&&?!bStop)
{
nStartPos?+=?fileAccessI.write(b,0,nRead);
//if(nThreadID?==?1)
//?Utility.log("nStartPos?=?"?+?nStartPos?+?",?nEndPos?=?"?+?nEndPos);
}
Utility.log("Thread?"?+?nThreadID?+?"?is?over!");
bDownOver?=?true;
//nPos?=?fileAccessI.write?(b,0,nRead);
}
catch(Exception?e){e.printStackTrace?();}
}
}
//打印回應的頭信息
public?void?logResponseHead(HttpURLConnection?con)
{
for(int?i=1;;i++)
{
String?header=con.getHeaderFieldKey(i);
if(header!=null)
//responseHeaders.put(header,httpConnection.getHeaderField(header));
Utility.log(header+"?:?"+con.getHeaderField(header));
else
break;
}
}
public?void?splitterStop()
{
bStop?=?true;
}
}
/*
**FileAccess.java
*/
package?NetFox;
import?java.io.*;
public?class?FileAccessI?implements?Serializable{
RandomAccessFile?oSavedFile;
long?nPos;
public?FileAccessI()?throws?IOException
{
this("",0);
}
public?FileAccessI(String?sName,long?nPos)?throws?IOException
{
oSavedFile?=?new?RandomAccessFile(sName,"rw");
this.nPos?=?nPos;
oSavedFile.seek(nPos);
}
public?synchronized?int?write(byte[]?b,int?nStart,int?nLen)
{
int?n?=?-1;
try{
oSavedFile.write(b,nStart,nLen);
n?=?nLen;
}
catch(IOException?e)
{
e.printStackTrace?();
}
return?n;
}
}
/*
**SiteInfoBean.java
*/
package?NetFox;
public?class?SiteInfoBean?{
private?String?sSiteURL;?//Site's?URL
private?String?sFilePath;?//Saved?File's?Path
private?String?sFileName;?//Saved?File's?Name
private?int?nSplitter;?//Count?of?Splited?Downloading?File
public?SiteInfoBean()
{
//default?value?of?nSplitter?is?5
this("","","",5);
}
public?SiteInfoBean(String?sURL,String?sPath,String?sName,int?nSpiltter)
{
sSiteURL=?sURL;
sFilePath?=?sPath;
sFileName?=?sName;
this.nSplitter?=?nSpiltter;
}
public?String?getSSiteURL()
{
return?sSiteURL;
}
public?void?setSSiteURL(String?value)
{
sSiteURL?=?value;
}
public?String?getSFilePath()
{
return?sFilePath;
}
public?void?setSFilePath(String?value)
{
sFilePath?=?value;
}
public?String?getSFileName()
{
return?sFileName;
}
public?void?setSFileName(String?value)
{
sFileName?=?value;
}
public?int?getNSplitter()
{
return?nSplitter;
}
public?void?setNSplitter(int?nCount)
{
nSplitter?=?nCount;
}
}
/*
**Utility.java
*/
package?NetFox;
public?class?Utility?{
public?Utility()
{
}
public?static?void?sleep(int?nSecond)
{
try{
Thread.sleep(nSecond);
}
catch(Exception?e)
{
e.printStackTrace?();
}
}
public?static?void?log(String?sMsg)
{
System.err.println(sMsg);
}
public?static?void?log(int?sMsg)
{
System.err.println(sMsg);
}
}
/*
**TestMethod.java
*/
package?NetFox;
public?class?TestMethod?{
public?TestMethod()
{?///xx/weblogic60b2_win.exe
try{
SiteInfoBean?bean?=?new?SiteInfoBean("http://localhost/xx/weblogic60b2_win.exe","L:\\temp","weblogic60b2_win.exe",5);
//SiteInfoBean?bean?=?new?SiteInfoBean("http://localhost:8080/down.zip","L:\\temp","weblogic60b2_win.exe",5);
SiteFileFetch?fileFetch?=?new?SiteFileFetch(bean);
fileFetch.start();
}
catch(Exception?e){e.printStackTrace?();}
}
public?static?void?main(String[]?args)
{
new?TestMethod();
}
}
posted on 2007-01-12 19:19
???MengChuChen 閱讀(441)
評論(0) 編輯 收藏 所屬分類:
JavaBasic