亚洲youwu永久无码精品 ,亚洲色婷婷综合开心网,亚洲欧洲无卡二区视頻

shell txt 分析��结

Thu, 26 Nov 2009 03:27:00 GMT

刘凯�?/a> 2009-11-26 11:27 发表评论

perl 使用��结

Wed, 11 Nov 2009 05:36:00 GMT

环境讄��Q�如果有�pȝ��字符�~�码冲突�Q�在当前
vim ~/.bash_profile 下加�?br /> LANG=zh_CN
LC_ALL=zh_CN.UTF8
export LANG LC_ALL

字符�~�码转化:
  # 由decode解析,默认会��?nbsp;�pȝ��~�码输出
  # �?nbsp;linux 下面其实�{��h encode("UTF-8", decode("GBK",$_));
perl -MEncode -ne 'print decode("GBK",$_);'  file.txt

判断数据是否�W�合输出�Q?br />   echo "121" |perl -ne 'print if /2/;'   #print 123

匚w��正则group输出:
  echo "abc121a" |perl -ne 'print $1 if /(\D+)/;'   #print abc

大小写�{化：
  # 全部大小转小�?/span>
  echo "ABC1C2cGJ" |perl -ne 'tr/[A-Z]/[a-z]/; print ;'  #print
  # "L 中间全部��写 "E   ; "U 中间全部大写 "E    ↓
  echo "ABC1C2cGJ" |perl -ne 's/(.*?1)(.*?)(2.*?)/$1\L$2\E$3/g; print ;'  #print ABC1c2cGJ

源文件替换：
   echo "ABC 123" > te
   sed -i 's/ABC/abc/g' te
       或�?nbsp;: perl -i -pe 's/ABC/abc/g' te
   cat te # print abc 123

外部传参:
tt="cc"
echo "gg" |perl -ne ' print "'$tt'" ;'
输出�Q�cc

  perl -e  'print "$ARGV[0]\t$ARGV[1]\n" '  'par1' 'par2'  #print par1 par2

重复列输�?/span>:
   cat xx.txt | awk -F"    " 'a[$1]++'
   或�?nbsp;:
   cat xx.txt |perl -F"\t" -ane  '$a{$F[1]}++;END{
        while(($k,$v)=each(%a)){ print "$k = $v "n"; }
    }'
    �l�果比如�Q?br />        癑ֺ�手机在线 = 7
       中兴 = 2
       万信恒�?nbsp; = 2
   �q�比如：查看各用�?nbsp;有多��个�q�程
   ps -ef |perl  -ane  '$a{$F[0]}++;END{
      while(($k,$v)=each(%a)){ print "$k = $v \n"; }
   }'

�?nbsp;两项  交集
cat BuyMusic.20090525| perl -ne 'BEGIN{
$p1="600902000005416300";
$p2="600902000006211983";
$p_col=30;
$mob_col=0;

}END{
  my @inter = grep {$a{$_}} keys %b; # 求交�?br />   #print $p1,"=",join(",",keys %a),""n";
  #print $p2,"=",join(",",keys %b),""n";
  print "产品 $p1:",scalar keys %a," "n";
  print "产品 $p2:",scalar keys %b," "n";
  print "交集:",scalar @inter," "n";
}
chomp;
@lis=split /\|<>\|/ ;
if( $lis[ $p_col] eq $p1 ){
   $a{$lis[$mob_col]}++;
}
if( $lis[$p_col] eq $p2 ){
   $b{$lis[$mob_col]}++;
}
'

关键�?nbsp;Top 10 �Q�输出源文本数据 :
perl -e  '
  my $num=10; # top 10
  open(MYFILE, "<$ARGV[0]");
  open(MYFILE2, "<$ARGV[0]");

  # 关键字列�?br />   while(){@lis=split /\|<>\|/;$fck{$lis[1]}++ }

  foreach $k (sort { $fck{$b} <=> $fck{$a} } keys %fck){
     if(++$row>$num){last; }
     $arr[@arr] = $k;
  }

  while(){@lis=split /\|<>\|/;
     if(grep { $arr[$_] eq $lis[1] } 0..$#arr){
       # print "$fck{$lis[1]}:$_"; #�?nbsp;关键字出现次数输�?br />        print ;
     }
  }
' qdSearch.log

刘凯�?/a> 2009-11-11 13:36 发表评论

hadoop streaming( hadoop + perl )��试

Fri, 25 Sep 2009 06:33:00 GMT

参�?
http://hadoop.apache.org/common/docs/r0.15.2/streaming.html

注意
目前 streaming �?linux pipe #也就�?cat |wc -l �q�样的管�?不支持，但不妨碍我们使用perl,python 行式命��o�Q�！
原话�?�Q?br /> Can I use UNIX pipes? For example, will -mapper "cut -f1 | sed s/foo/bar/g" work?
    Currently this does not work and gives an "java.io.IOException: Broken pipe" error.
    This is probably a bug that needs to be investigated.
但如果你是强烈的 linux shell pipe 发烧�?�Q?参考下�?br /> $> perl -e 'open( my $fh, "grep -v null tt |sed -n 1,5p |");while ( <$fh> ) {print;} '
     #不过我没��试通过 �Q�！

环境 �Q�hadoop-0.18.3
$> find . -type f -name "*streaming*.jar"
./contrib/streaming/hadoop-0.18.3-streaming.jar

��试数据�Q?br />

-bash-3.00$ head tt
null    false    3702    208100
6005100    false    70    13220
6005127    false    24    4640
6005160    false    25    4820
6005161    false    20    3620
6005164    false    14    1280
6005165    false    37    7080
6005168    false    104    20140
6005169    false    35    6680
6005240    false    169    32140
......

�q�行�Q?br />

c1=" perl -ne  'if(/.*\t(.*)/){\$sum+=\$1;}END{print \"\$sum\";}' "
# 注意 �q�里 $ 要写�?\$    " 写成 \"
echo $c1; # 打印输出 perl -ne 'if(/.*"t(.*)/){$sum+=$1;}END{print $sum;}'
hadoop jar hadoop-0.18.3-streaming.jar
   -input file:///data/hadoop/lky/jar/tt
   -mapper   "/bin/cat"
   -reducer "$c1"
   -output file:///tmp/lky/streamingx8

�l�果:
cat /tmp/lky/streamingx8/*
1166480

本地�q�行输出:
perl -ne 'if(/.*"t(.*)/){$sum+=$1;}END{print $sum;}' < tt
1166480

�l�果正确!!!!

命��o自带文档�Q?br />

-bash-3.00$ hadoop jar hadoop-0.18.3-streaming.jar -info
09/09/25 14:50:12 ERROR streaming.StreamJob: Missing required option -input
Usage: $HADOOP_HOME/bin/hadoop [--config dir] jar \
          $HADOOP_HOME/hadoop-streaming.jar [options]
Options:
  -input    <path>     DFS input file(s) for the Map step
  -output   <path>     DFS output directory for the Reduce step
  -mapper   <cmd|JavaClassName>      The streaming command to run
  -combiner <JavaClassName> Combiner has to be a Java class
  -reducer  <cmd|JavaClassName>      The streaming command to run
  -file     <file>     File/dir to be shipped in the Job jar file
  -dfs    <h:p>|local  Optional. Override DFS configuration
  -jt     <h:p>|local  Optional. Override JobTracker configuration
  -additionalconfspec specfile  Optional.
  -inputformat TextInputFormat(default)|SequenceFileAsTextInputFormat|JavaClassName Optional.
  -outputformat TextOutputFormat(default)|JavaClassName  Optional.
  -partitioner JavaClassName  Optional.
  -numReduceTasks <num>  Optional.
  -inputreader <spec>  Optional.
  -jobconf  <n>=<v>    Optional. Add or override a JobConf property
  -cmdenv   <n>=<v>    Optional. Pass env.var to streaming commands
  -mapdebug <path>  Optional. To run this script when a map task fails
  -reducedebug <path>  Optional. To run this script when a reduce task fails
  -cacheFile fileNameURI
  -cacheArchive fileNameURI
  -verbose

刘凯�?/a> 2009-09-25 14:33 发表评论

部分高��正则�Ҏ�?使用

Thu, 27 Aug 2009 08:04:00 GMT

一些特�D�正则元字符说明:
1. *? �?nbsp;+? �?nbsp;{n,}?  懒惰匚w��
   1.1 非懒�?nbsp;　　　　　　　　　　　　　　　   ↓
    echo "ab2c121a" |perl -ne 'print $1 if /(.*)"d/;'   #print ab2c12
   1.2 懒惰　　　　　　　　　　　　　　　　      ↓
    echo "ab2c121a" |perl -ne 'print $1 if /(.*?)"d/;'   #print ab
2.  回溯引用和前后查�?/span>:
   2.1 向前查找   (?=..)    　　　　　　　　　　　　   ↓
    echo "ab2c121a" |perl -ne 'print $1 if /(.*?)(?=2)/;'  #print ab
   2.2 向后查找 (?<=..)  　　     　　　　　　↓
    echo "ab2c121a" |perl -ne 'print $1 if /(?<=2)(.*)(?=2)/;' #print c1
   2.3 负向-�?/span>/�?nbsp;查找  (?!..) (?)
    #不能匚w�� ..                               ↓
    echo "ab2c121a" |perl -ne 'print $1 if /(?'    #print �?/span>
    echo "ab2c121a" |perl -ne 'print $1 if /(?'    #print c121a
   2.4 条�g ?()  = if   ?()| = if else
    # ?()  例如

必须同时出现 ↓ ↓
echo "

"|perl -ne 'print $2 if /(

)?("w*)(?(1)<"/p>)/' #print xx
echo "

xx"|perl -ne 'print $2,""n" if /(

)?("w*)(?(1)<"/p>)/' #print �I?/span>
echo "xx"|perl -ne 'print $2 if /(

)?("w*)(?(1)<"/p>)/'    #print xx
    # ?()|  例如 �q�是上面的，
    # �?nbsp;�?lt;p> 可以�?lt;/p> 也可以接数字�l�尾                            ↓
echo "

xx1

"|perl -ne 'print $2 if /(

)?("w*)(?(1)<"/p>|"d)/' #print xx1
echo "

xx1"|perl -ne 'print $2 if /(

)?("w*)(?(1)<"/p>|"d)/' # print xx

刘凯�?/a> 2009-08-27 16:04 发表评论

perl vs php (�?

Fri, 15 May 2009 15:09:00 GMT

Perl/Php Translation

arrays

hashes

data structures

array split/join

case conversion

string comparisons

functions

string matching operations

basename/dirname

environment variables

POST/GET parameters

HTML elements

URL encode

MySQL database access

�?Perl arrays �?/th>	�?Php arrays �?/th>
@a = (); @a = ( 'xx', 11, 33.5, ); @a = 12..33; $a[2] = 'something'; $len = scalar(@a); # or $len = @a; @a3 = ('xx', @a1, @a2); ($x, $y) = @a; $a[@a] = 'new'; # push push pop shift unshift splice foreach $i (@a) { .. }	$a = array(); $a = array( 'xx', 11, 33.5, ); $a = range(12,33); $a[2] = 'something'; $len = count($a); $a3 = array_merge('xx', $a1, $a2); list($x, $y) = $a; $a[] = 'new'; # push array_push array_pop array_shift array_unshift array_splice foreach ($a as $i) { .. }

�?Perl arrays �?/th>

�?Php arrays �?/th>

@a = ();

            

            @a = ( 'xx', 11, 33.5, );

            

            @a = 12..33;

            

            $a[2] = 'something';

            

            $len = scalar(@a);

            # or

            $len = @a;

            

            @a3 = ('xx', @a1, @a2);

            

            ($x, $y) = @a;

            

            $a[@a] = 'new'; # push

            

            push

            pop

            shift

            unshift

            splice

            

            foreach $i (@a) { .. }

$a = array();

            

            $a = array( 'xx', 11, 33.5, );

            

            $a = range(12,33);

            

            $a[2] = 'something';

            

            $len = count($a);

            

            

            

            $a3 = array_merge('xx', $a1, $a2);

            

            list($x, $y) = $a;

            

            $a[] = 'new'; # push

            

            array_push

            array_pop

            array_shift

            array_unshift

            array_splice

            

            foreach ($a as $i) { .. }

�?Perl hashes �?/th>	�?Php hashes �?/th>
%h = (); %h = ( 'x' => 'y', 'z' => 'w', ); $h{'x'} = 7; while (($key,$value) = each(%h)) { .. } $a = keys(%h); $b = values(%h); delete $h{'x'};	$h = array(); $h = array( 'x' => 'y', 'z' => 'w', ); $h['x'] = 7; foreach ($h as $key => $value) { .. } $a = array_keys($h); $b = array_values($h); unset( $h['x'] );

�?Perl hashes �?/th>

�?Php hashes �?/th>

%h = ();

            

            %h = ( 'x' => 'y',

            'z' => 'w',

            );

            

            $h{'x'} = 7;

            

            while (($key,$value) = each(%h))

            { .. }

            

            $a = keys(%h);

            $b = values(%h);

            

            delete $h{'x'};

$h = array();

            

            $h = array( 'x' => 'y',

            'z' => 'w',

            );

            

            $h['x'] = 7;

            

            foreach ($h as $key => $value)

            { .. }

            

            $a = array_keys($h);

            $b = array_values($h);

            

            unset( $h['x'] );

�?Perl data structures �?/th>	�?Php data structures �?/th>
%h = ('a'=>13, 'b'=>25); @x = ('hi', 'there', 'all',); @mix = ( %h, @x, [33..39], { x=>15, yy=>23, }, ); $mix[0]->{'b'} # == 25 $mix[0]{'b'} # == 25 $mix[2]->[2] # == 35 $mix[2][2] # == 35	$h = array('a'=>13, 'b'=>25); $x = array('hi', 'there', 'all',); $mix = array($h, $x, range(33,39), array('x'=>15, 'yy'=>23), ); $mix[0]['b'] # == 25 $mix[2][2] # == 35

�?Perl data structures �?/th>

�?Php data structures �?/th>

%h = ('a'=>13, 'b'=>25);

            @x = ('hi', 'there', 'all',);

            

            @mix = ( %h, @x,

            [33..39],

            { x=>15, yy=>23, },

            );

            

            $mix[0]->{'b'}  # == 25

            $mix[0]{'b'}    # == 25

            $mix[2]->[2]    # == 35

            $mix[2][2]      # == 35

$h = array('a'=>13, 'b'=>25);

            $x = array('hi', 'there', 'all',);

            

            $mix = array($h, $x,

            range(33,39),

            array('x'=>15, 'yy'=>23),

            );

            

            $mix[0]['b']  # == 25

            

            $mix[2][2]    # == 35

�?Perl array split/join �?/th>	�?Php array split/join �?/th>
@a = split( '\|', $s ); @a = split( 's+', $s ); $s = join( '\|', @a );	$a = preg_split( '/\|/', $s, -1, PREG_SPLIT_NO_EMPTY ); $a = preg_split( '/s+/', $s, -1, PREG_SPLIT_NO_EMPTY ); $s = join( '\|', $a );

�?Perl array split/join �?/th>

�?Php array split/join �?/th>

@a = split( '|', $s );

            

            @a = split( 's+', $s );

            

            

            $s = join( '|', @a );

$a = preg_split( '/|/', $s,

            -1, PREG_SPLIT_NO_EMPTY );

            $a = preg_split( '/s+/', $s,

            -1, PREG_SPLIT_NO_EMPTY );

            

            $s = join( '|', $a );

�?Perl case conversion �?/th>	�?Php case conversion �?/th>
$s = lc($s); $s = uc($s); $s =~ tr/a-z/A-Z/;	$s = strtolower($s); $s = strtoupper($s);

�?Perl string comparisons �?/th>	�?Php string comparisons �?/th>
$s1 eq $s2 $s1 lt $s2	strcmp($s1,$s2) == 0 # or $s1 === $s2 strcmp($s1,$s2) < 0

�?Perl functions �?/th>	�?Php functions �?/th>
sub foo { my @args = @_; } sub foo { $x = 5; } foo2( @a, %h );	function foo() { $args = func_get_args(); } function foo() { global $x; $x = 5; } function foo2($x, $y) { } foo2( $a, $h );

�?Perl string matching operations �?/th>	�?Php string matching operations �?/th>
$s =~ m/(w+)/; $substr = $1; @all = ($s =~ m/(w+)/g); $s =~ s/s+/X/; $s =~ s/s+/X/g; $s =~ s/^s+\|s+$//g;	preg_match( "/(w+)/", $s, $match ); $substr = $match[1]; preg_match_all( "/(w+)/", $s, $match ); $all = $match[0]; $s = preg_replace( "/s+/", 'X', $s, 1 ); $s = preg_replace( "/s+/", 'X', $s ); $s = trim($s);

�?Perl string matching operations �?/th>

�?Php string matching operations �?/th>

$s =~ m/(w+)/;

            $substr = $1;

            

            @all = ($s =~ m/(w+)/g);

            

            

            $s =~ s/s+/X/;

            $s =~ s/s+/X/g;

            

            $s =~ s/^s+|s+$//g;

preg_match( "/(w+)/", $s, $match );

            $substr = $match[1];

            

            preg_match_all( "/(w+)/", $s, $match );

            $all = $match[0];

            

            $s = preg_replace( "/s+/", 'X', $s, 1 );

            $s = preg_replace( "/s+/", 'X', $s );

            

            $s = trim($s);

�?Perl basename/dirname �?/th>	�?Php basename/dirname �?/th>
use File::Basename; $b = basename($path); $d = dirname($path);	$b = basename($path); $d = dirname($path);

�?Perl environment variables �?/th>	�?Php environment variables �?/th>
%ENV $ENV{REQUEST_METHOD} $ARGV[$i] $0	$_SERVER $_SERVER[REQUEST_METHOD] $argv[$i+1] $argv[0] # Php/CGI only

�?Perl POST/GET parameters �?/th>	�?Php POST/GET parameters �?/th>
#form/hyperlink parameters: # s : single-valued # m : multi-valued use CGI (:standard); $s = param('s'); @m = param('m'); @param_names = param(); $num_params = param();	#form/hyperlink parameters: # s : single-valued # m[] : multi-valued # (such as multi-selections # and checkbox groups) $PARAM = array_merge($_GET, $_POST); $s = $PARAM['s']; # a scalar $m = $PARAM['m']; # an array $param_names = array_keys($PARAM); $num_params = count($PARAM);

�?Perl POST/GET parameters �?/th>

�?Php POST/GET parameters �?/th>

#form/hyperlink parameters:

            # s : single-valued

            # m : multi-valued

            

            use CGI (:standard);

            

            

            

            

            $s = param('s');

            @m = param('m');

            

            @param_names = param();

            $num_params = param();

#form/hyperlink parameters:

            # s   : single-valued

            # m[] : multi-valued

            #       (such as multi-selections

            #        and checkbox groups)

            

            $PARAM

            = array_merge($_GET, $_POST);

            

            $s = $PARAM['s'];  # a scalar

            $m = $PARAM['m'];  # an array

            

            $param_names = array_keys($PARAM);

            $num_params = count($PARAM);

�?Perl HTML elements �?/th>

�?Php HTML elements �?/th>

use CGI (:standard);

            

            

            

            

            

            

            

            $ref = "x.cgi";

            a({href=>$ref}, "yy")

            

            textfield({name=>"yy", size=>5})

            

            password({name=>"yy", size=>5})

            

            textarea({name=>"yy",

            cols=>5, rows=>2})

            

            submit({value=>"yy"})

            

            button( {name=>"xx",

            value=>"yy",

            onclick=>"submit()",

            }

            )

            

            %labels = (0=>'a',1=>'q',2=>'x');

            popup_menu( { name=>"xx",

            values=>[0..2],

            labels=>%labels,

            size=>4,

            }

            )

            

            

            @a = ('xx','yy','zz');

            radio_group( { name=>'nn',

            values=> @a,

            default=>'_',

            linebreak=>1,

            }

            )

            

            %labels = ('xx'=>'L1','yy'=>'L2');

            @a = keys( %labels );

            checkbox_group( { name=>'nn',

            values=> @a,

            labels=> %labels,

            }

            )

            

            table(

            Tr(

            [

            td(['a','b']),

            td(['x','y']),

            ]

            )

            )

# The Perl/CGI functions have the

            # additional property of "stability"

            # when used in reentrant forms.

            # The values of the HTML elements are

            # set according to the incoming

            # parameter values for those elements.

            # The versions below are not stable.

            

            $ref = "x.php";

            yy

            

            

            

            

            

            <br />
            <br />
            

            

            

            

            
            name="xx" value="yy"

            onclick="submit()">

            

            

            

            

            

            $a = array('xx','yy','zz');

            foreach ($a as $_)

            echo "
            name=nn value='$_'>$_
";

            

            

            

            

            $labels = array('xx'=>'L1','yy'=>'L2');

            foreach (array_keys($labels) as $_)

            echo "
            name=nn value='$_'>",

            $labels[$_];

            

            

            

            








            
            a b
            
            
            
            x y

�?Perl URL encode �?/th>	�?Php URL encode �?/th>
use URI::Escape; uri_escape($val) uri_unescape($val)	urlencode($val) urldecode($val)

�?Perl MySQL database access �?/th>	�?Php MySQL database access �?/th>
use DBI; $dbh = DBI->connect( 'DBI:mysql:test:localhost', $usr,$pwd ); $dbh->do( $sql_op ) $query = $dbh->prepare( $sql_op ); $query->execute(); while( @record = $query->fetchrow() ) { .. } $dbh->quote($val)	$dbh = mysql_connect( 'localhost', $usr, $pwd ); mysql_query('USE test') mysql_query( $sql_op ); $results = mysql_query( $sql_op ); while($record = mysql_fetch_row($results)) { .. } "'" . addslashes($val) . "'"

�?Perl MySQL database access �?/th>

�?Php MySQL database access �?/th>

use DBI;

            $dbh = DBI->connect(

            'DBI:mysql:test:localhost',

            $usr,$pwd

            );

            

            $dbh->do( $sql_op )

            

            $query = $dbh->prepare( $sql_op );

            $query->execute();

            

            while(

            @record = $query->fetchrow() )

            { .. }

            

            

            $dbh->quote($val)

 

            $dbh = mysql_connect(

            'localhost', $usr, $pwd

            );

            mysql_query('USE test')

            

            mysql_query( $sql_op );

            

            $results = mysql_query( $sql_op );

            

            

            while($record =

            mysql_fetch_row($results))

            { .. }

            

            

            "'" . addslashes($val) . "'"

刘凯�?/a> 2009-05-15 23:09 发表评论

perl 使用

Wed, 01 Apr 2009 06:12:00 GMT

对应脚本�q�用�Q?br /> 1. shell �l�筹��理脚本的运行。合理结�?crontab , ps -ef ,kill �{�命令�?br /> 2. perl 处理短小�?�?br /> 3. python 有比较复杂结构和逻辑的�?br />
本文主要介绍 perl 的行�U�命令��?�Q�力�?�?��?�?�Q?br />

#最��单的
$ perl -e 'print "Hello World\n"'

#处理文�g �?/span>
$ perl -n -e 'print $_' file1

#�~�码转换
#如果有需�?nbsp;在��用下 encode("UTF-8", decode("GBK",$_));�?nbsp;linux 下默�?nbsp;utf-8
perl -MEncode -ne 'print decode("GBK",$_);' file.txt

#正则使用
# if($_=~/.*\/(.*)$/){ print $1 ;} �q�是perl 巨方便的地方 $1 xx
# next LINE 跛_��下一个��@�?/span>
$ perl -n -e 'next LINE unless /pattern/; print $_'

#去除换行 chomp
perl -e 'print split(/\n/,"asg\n");'

#�?nbsp;awk 一�?nbsp;Begin End
$ perl -ne 'END { print $t } @w = /(\w+)/g; $t += @w' file.txt

#�?nbsp;awk -F"x" 一�?nbsp;切割�?br /> #-a 打开自动分离 (split)  模式
#@F �?nbsp;切割后的数组
perl -F'\t' -ane '
if($F[1]=~/侃侃/ and $F[2]=~/爱情�?){
      print "$F[3]\t$F[4]\t$F[5]\n"
  }
' all2_data.sort.st

实际处理�Q?br />

perl -F'\|\|' -ane '
my $actor,$music ;
if  ( $F[3] ){
  $music=$F[2];
  $actor=$F[3];
}else{
  $music=$F[0];
  $actor=$F[1];
}
  $music =~ tr/[A-Z]/[a-z]/;
  $music =~ s/\s*(.*)\s*$.*$/\1/g;
  $actor =~ tr/[A-Z]/[a-z]/;
  $actor =~ s/\s*(.*)\s*$.*$/\1/g;
print "$actor-$music\n";
' ring.utf8.txt  |sort -u  > ring.actor_music.sort.utf8.txt &
wc -l ring.actor_music.sort.utf8.txt

#�?nbsp;sed 一��h��?br /> # -i �?sed 一�?perl 输出会替换调 �?file.txt
$ perl -i -pe 's/\bPHP\b/Perl/g' file.txt

#外部传参
perl -ne'print "$ARGV[0]\t$ARGV[1]\n" ' file.txt 'par1' 'par2'
#�l�果 .. par1 par2 ..

# 查询�?nbsp;重复�?nbsp;�ơ数�Q��ƈ 列�D出来
cut -d"     "  -f 2 .collection_mobile.data |perl -ne '
   END{
     while (($key,$value)=each(%a)){print $key,"=",$value,"\n";};
   }BEGIN{ %a =(); }
   chomp;
   $a{$_}+=1;
'

�l�果
Ring=532895
CRBT=68500
RingBoxes=880
Song=96765

#一些实�?/span>使用 :)
find . -name "*.mp3" | perl -pe 's/.\/\w+-(\w+)-.*/$1/' | sort | uniq
perl -F'\t' -ane 'if($F[1]=~/$ARGV[0]/ and $F[2]=~/$ARGV[1]/){print "$F[3]\t$F[4]\t$F[5]\n"}' all2_data.sort.st '侃侃' '爱情�?

#�?nbsp;find 合用 -e $ARGV[0] 扚w�� ?excel 倒成文本格式
find . -maxdepth 1 -name "*xls" -exec perl -e '

require("/home/xj_liukaiyi/src/perl/excel/excelUtil.pl");
my $file=$ARGV[0];
sub myRead{
  my $str = "";
  for $tmp(@_){
    $str="$str$tmp\t";
  }
  $str="$str\n";
  system "echo \"$str\" >> $file.data ";
}
&parse_excel("$file",0,\&myRead);
print "$file\n";

' {} \;

参考：
http://www.ibm.com/developerworks/cn/linux/sdk/perl/l-p101/index.html
http://bbs.chinaunix.net/viewthread.php?tid=499434

刘凯�?/a> 2009-04-01 14:12 发表评论

perl 一些有用的 util

Tue, 17 Mar 2009 05:49:00 GMT

比如日志�Q�和shell�q�行

#!/bin/perl -w
use Time::Local;

my $log_input_file="~/log";
sub set_log_input_file{
  $log_input_file=$_[0];
}

#&perl_log("tt","/home/xj_liukaiyi/log");
sub perl_log{
  my $log_msg=shift @_;
  print $log_msg,"\n";
  system 'ttmp1=$(date +%Y-%k-%d);ttmp2=$(date +%M:%S);echo "$ttmp1 $ttmp2:'."$log_msg".'" >>'.$log_input_file;
}

#&system_util("xls -l","~/log");
sub system_util{
  my $comm=shift @_;
  my @return=system $comm;
  if ( $return[(@return-1)] == 0 ) {
     &perl_log("[success]".$comm,$log_input_file);
  }else{
     &perl_log("[error]".$comm,$log_input_file);
  }
  return $return[(@return-1)];
}

1;

excel ��d��util
require("/home/xj_liukaiyi/src/perl/excel/excelUtil.pl");&parse_excel("2w.xls","歌曲列表","&read_wlh);

#!/bin/perl -w
use Spreadsheet::ParseExcel;
use Spreadsheet::ParseExcel::FmtUnicode;
use Encode;
use Unicode::Map();
my $Map = new Unicode::Map("GB2312");

my $excelCODE='gb2312';
my $coslCODE='utf8';

my %parseCache ;
my %writeCache ;

my $oFmtJ = Spreadsheet::ParseExcel::FmtUnicode->new(Unicode_Map =>$excelCODE);
my $oExcel = new Spreadsheet::ParseExcel;

#
#&parse_excel("all4.xls","乐扑","&read_lp);
# sub read_lp{ $_[1 .. 2.3 ] }
sub parse_excel{
my $sExcelName=shift;
my $sSheetName=shift;
my $action_fun=shift;

my($iR, $iC, $oWkS,$oWkC);
#开�?nbsp;excel
my $oBook = $oExcel->Parse($sExcelName,$oFmtJ);
#�Ҏ�� sheet 名称匚w��
for(my $iSheet=0; $iSheet < $oBook->{SheetCount} ; $iSheet++){
    my $t_oWkS = $oBook->{Worksheet}[$iSheet];
    my $sWkSName=encode($coslCODE,decode($excelCODE,$t_oWkS->{Name} ));
    if($sWkSName=~/$sSheetName/){
      $oWkS=$t_oWkS;
      last;
   }
}

for(my $iR = $oWkS->{MinRow} ;
     defined $oWkS->{MaxRow} && $iR <= $oWkS->{MaxRow} ;
     $iR++){
     my @arr_row;
     for(my $iC = $oWkS->{MinCol} ;
         defined $oWkS->{MaxCol} && $iC <= $oWkS->{MaxCol} ;
         $iC++){
       $oWkC=$oWkS->{Cells}[$iR][$iC];
       my $col_value="";
           $col_value=encode($coslCODE,decode($excelCODE,$oWkC->Value) ) if ($oWkC) ;
           push(@arr_row,$col_value);
         }
      push(@arr_row,$iR);
     $action_fun->(@arr_row);
    }
}

#使用 demo
find . -maxdepth 1 -name "*xls" -exec perl -e '
require("/home/xj_liukaiyi/src/perl/excel/excelUtil.pl");
my $file=$ARGV[0];
sub myRead{
  my $str="";
  foreach$tmp(@_){
    $str+"$tmp\t";
  }
  system "echo $str >> $file.data";
}
&parse_excel("$file",0,\&myRead);
print "$file\n";
' {} \;

刘凯�?/a> 2009-03-17 13:49 发表评论

Mon, 02 Mar 2009 09:51:00 GMT

工具��脚�?br />

#!/bin/perl
use File::Find;
use Log::FileSimple;

#use strict;
#use warnings;

# 说明:
#     遍历�?nbsp;"/cygdrive/c/Documents and Settings/Administrator/"
#       满�� "^.*\.txt$" ��d��文�g
#         �q�运�?nbsp;"ls -l $fileAllName "
#    �q�行日志 ./log
#
# perl this.pl
#     "ls -l $fileAllName "
#     "/cygdrive/c/Documents and Settings/Administrator/"
#     "^.*\.txt$"
#     ./log
#
# run_comm 参数1 替换说明�Q?br /> #     $filePath  ��入参�?�Q�遍历根目录 "/cygdrive/c/Documents and Settings/Administrator/"
#    $fileAllName 为当前遍历，满��正则条�g�?nbsp;全�\�?文�g�?br /> #    $logicFileAllName  为当前遍历，满��正则条�g�?nbsp;逻辑路径+文�g�?br /> #    $logicFilePath 为当前遍历，逻辑路径
#    $logicFileName 文�g名称
#    $logicFileExp  文�g后缀
#        �Q�test.txt 文�g
#            $logicFileName �?nbsp;test
#            $logicFileExp �?nbsp;txt
#            �Q?/span>

my $run_comm=$ARGV[0];
my $file_path=$ARGV[1];
my $file_find_rex=$ARGV[2];
my $logfile=$ARGV[3];

my $log = new Log::FileSimple(name=> 'Log::FileSimple logs',
                              file      => "$logfile",
                              mask      => -1,
                              autoflush => 0,
                              );

sub fileThumb{
eval{
  $_tmp_run_comm=$run_comm ;
  if( /$file_find_rex/ ){
    $fileNum++;
    $_tmp_run_comm=~s/\$filePath/$file_path/g;

    my $_file_all_name=$File::Find::name;
    $_tmp_run_comm=~s/\$fileAllName/$_file_all_name/g;
    my $_logic_file_all_name=$_file_all_name;
          $_logic_file_all_name=~s/$file_path//g;
        $_logic_file_all_name=~s/^\s*\.//g;
    $_tmp_run_comm=~s/\$logicFileAllName/\.\/$_logic_file_all_name/g;
    if($_logic_file_all_name=~/^(.*)\/([^\/]*)\.([^\.]*)$/){
      my $_logic_file_path=$1;
      my $_logic_file_name=$2;
      my $_logic_file_exp=$3;
      $_tmp_run_comm=~s/\$logicFilePath/\.\/$_logic_file_path/g;
      $_tmp_run_comm=~s/\$logicFileName/$_logic_file_name/g;
      $_tmp_run_comm=~s/\$logicFileExp/$_logic_file_exp/g;
      }
    #print $_tmp_run_comm,"\n";
    my @return=eval{system("$_tmp_run_comm 2>&1 \|echo \$\? ")};
    if ( $return[(@return-1)] != 0 ) {
      $log->log(message=>"success($fileNum):$_file_all_name");
       }else{
      $log->log(message=>"error($fileNum):$_file_all_name");
        }
  }
}
}

find(\&fileThumb,$file_path);

刘凯�?/a> 2009-03-02 17:51 发表评论

soap (java,perl,要写代码 �q�不�q?100 char)

Sun, 01 Mar 2009 16:00:00 GMT

可能最��单的 soap 哦！
服务器�ؓ java tomcat
下蝲服务器：http://www.apache.org/dist/ws/
   axis/ 为java
   axis-c/    �?c

1.我这下蝲�?axis 1_4 版本 �Q�解�?/webapps/axis 攑ֈ� tomcat 的webapps
启动tomcat�Q�这��׃��修改��M��Q�如有需�?�?web.xml �{�配�|�了�Q?a style="font-size: 12px;" onclick="NEBlog.gPrevBlog.oBlogList._oUpdownMenu.openBlog('blog/static/307422712007552499407', event);return false;"> 参�?>>
    blog �?�q�有很多介绍 axis 大家可以看看日志 > 其下�?"学习"

直接写个文�g HelloWorld.jws 攑ֈ� %TOMCAT_HOME%"webapps"axis

public class HelloWorld {
     public String sayHello()   {
     return "HELLO soap!";
}
}

�{�不�?可以使用看看
http://127.0.0.1:8080/axis/HelloWorld.jws?method=sayHello
�l�果 �Q?br />

<soapenv:Envelope>
<soapenv:Body>
<sayHelloResponse soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<sayHelloReturn xsi:type="xsd:string">HELLO soap!sayHelloReturn>
sayHelloResponse>
soapenv:Body>
soapenv:Envelope>

客户�D?br /> perl cpan >> install SOAP::Lite

#!perl -w
use SOAP::Lite;
print SOAP::Lite
    -> proxy('http://127.0.0.1:8080/axis/HelloWorld.jws')
    -> uri('http://127.0.0.1:8080/axis/HelloWorld.jws')
    -> sayHello()
    -> result;

�l�果�Q?HELLO soap�Q?/span>

刘凯�?/a> 2009-03-02 00:00 发表评论

Memcached 对话 Google ProtocolBuffers (perl)

Thu, 26 Feb 2009 05:45:00 GMT

看看我这�?web �pȝ��架构图，那就知道使用�l�一存储格式的好处了�?br /> 在这�U�结构中你就单纯使用 java �Q�或 php 某一�U�单一语言描述格式�Q�存储你集群交互的数据吗�Q�？使用ProtocolBuffers 吧，Google 已经�?现成的api来帮你扩展了�?/strong>

参考：http://blog.csdn.net/lcj8/archive/2009/02/17/3900157.aspx

     作�ؓ Memcached ��是为集��用，那么作�ؓ在缓存存储的数据我想也应该适用与各�U�环境，�q�时�?数据�l�构协议 Google ProtocolBuffers
参考：http://code.google.com/apis/protocolbuffers/docs/overview.html
其他语言�Q?/strong>http://code.google.com/p/protobuf/wiki/OtherLanguages
�Q�可适用与多�U�语�a��Q?br />     javascript ��d��~�存中的数据 ,你可以想象下你的 web服务开�?的将来！�Q?br />

目前学习perl �?��先上个�q�的代码�Q?br /> 参考：
http://search.cpan.org/~gariev/Google-ProtocolBuffers-0.08/lib/Google/ProtocolBuffers.pm#___top
#!/bin/perl -w
use Cache::Memcached;
use Google::ProtocolBuffers;
#开�?memd
my $memd = new Cache::Memcached{servers => ['10.0.2.15:11211'] };

#声明 �~�存存储 protocol 格式
my $key='test';
Google::ProtocolBuffers->parse("
  message Person{
    required int32 id           =1;
    required string name        =2;
  }
",{create_accessors=>1});

my $data=Person->encode({
  id=>123,
  name=>'liukaiyi'
});

#�~�存存储
$memd->add($key =>$data,3600);

#######################################################################

my $memd = new Cache::Memcached{servers => ['10.0.2.15:11211'] };
my $person;{
#从缓存中取出
  $person = Person->decode($memd->get($key));
}

print $person->id,"-",$person->name;

�l�果�Q?br /> 123-liukaiyi

刘凯�?/a> 2009-02-26 13:45 发表评论

Wed, 25 Feb 2009 08:23:00 GMT

�q�大体上可以看出�Q�服务器端口�~�存技术�?br />
memcached 官方�Q�http://www.danga.com/memcached/
安装前，先安�?libevent
其上�?linux 软�g一般安装，看他们readme文档
$>memcached -d -u nobody -m 512 127.0.0.1 -p 11211

如果有异常，
ln -s /usr/local/lib/libevent-1.4.so.2 /lib/libevent-1.4.so.2
参�?nbsp; http://blog.chinaunix.net/u2/70049/showart_1665279.html
我这��׃��?perl 语言了，
   其他语言参�?http://code.google.com/p/memcached/wiki/Clients

   perl 使用 cpan> install Cache::Memcached ;#会��?perl 我就不说�?/strong>
代码说明�Q�不停对 key 为test 的��D��行递增

#!/bin/perl -w
use Cache::Memcached;

my $memd = new Cache::Memcached{servers => ['127.0.0.1:11211'] };
my $key = 'test';
$memd->add($key => 1,3600) or warn 'Alread added';
while(1){
  print $memd->get($key),"\n";
  $memd->incr($key) or warn 'FAIL!';
}

使用 telnet 127.0.0.1 11211
   $> get test
   VALUE test 0 2
   97 #�q�就是我们递增的�?当然你��用什么语�a�取都是一��L��
   END

刘凯�?/a> 2009-02-25 16:23 发表评论

perl file find

Mon, 23 Feb 2009 15:55:00 GMT
使用 File::Find
use File::Find;
use Time::Piece;

$timeOneDay = 86400;
$dayMultipler = 1;

$ByteCount = 0;
$fileAll = 0;
$fileJpg = 0;
$fileYes = 0;
$pathToTra = ".";

$gt = localtime;
$t1 = $gt->epoch;
find(\&fileThumb, $pathToTra);
$gt = localtime;
$t2 = $gt->epoch;

print "Total KByte: ".($ByteCount/1024)."\n";
print "Total Scan:  ".$fileAll."\n";
print "Total Jpeg:  ".$fileJpg."\n";
print "Total Match: ".$fileYes."\n";
print "Total TimeCost: ".($t2-$t1)."\n";

sub fileThumb
{
   $fileAll++;
   if ((/.*\.jpg$/) )
   {
      $fileJpg++;
      my $lt = 0;
      $lt = localtime;

      my ($dev, $ino, $mode, $nlink, $uid, $gid, $rdev, $size, $atime, $mtime, $ctime, $blksize, $blocks) = stat($_)  or die "Unable to stat $_\n";
      if ( $mtime < ($lt->epoch - $timeOneDay*$dayMultipler) )
      {
         print "[ $fileYes ]"."$File::Find::name";
         print $size." ".$ctime." ".$lt->epoch."\n";
         $ByteCount += $size;
         $fileYes++;
      }
   }
}

刘凯�?/a> 2009-02-23 23:55 发表评论

log4j perl 错误日志提取

Thu, 19 Feb 2009 05:01:00 GMT

日志

#!/bin/perl
use Encode ;
open(NOTESFILES,$ARGV[0]);

my %errorMeganotes;

print ">>>>>>>>>>>>>>",$ARGV[0],"<<<<<<<<<<\n";

while (<NOTESFILES>) {
  if( $_=~/^(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2})\s+ERROR\s+(.*)\s/g ){
     if($errorMeganotes{$2}){
       push @{$errorMeganotes{$2}},$1;
     }else{
       $errorMeganotes{$2}=[$1];
     }
  }
}

for(keys %errorMeganotes){
  my $tmp_keys=$_;
  my @tmp_array= @{$errorMeganotes{$tmp_keys}};
  print "error number :>>",scalar(@tmp_array),"<<\n",encode("GBK",decode("UTF-8",$tmp_keys)),"\n",join(",\t",@tmp_array),"\n\n\n";
}

sh

#!/bin/sh
nowDate=$(date -d '-1 day' +%Y-%m-%d);
echo $nowDate

perl /data/apps/shell/meganotesErrorLog.pl /data/apps/meganotes/tomcat/logs/app/app.log.$nowDate >  _tmp_$nowDate ;
perl /data/apps/shell/meganotesErrorLog.pl /data/apps/schedule/tomcat/logs/app/app.log.$nowDate  >> _tmp_$nowDate ;

mail -s $nowDate-ERROR-log  my@mail.com  <  _tmp_$nowDate;
rm _tmp_$nowDate ;

�l�果�Q?br /> >>>>>>>/data/apps/meganotes/tomcat/logs/app/app.log.2009-02-18<<<<<<<<<
error number :>>2<<
MusicActionOpLogProxy:142 - music.popularTime=[Invalid field value for field "music.popularTime".]
2009-02-18 15:53:54,   2009-02-18 16:00:12
。。。。。�?br />

刘凯�?/a> 2009-02-19 13:01 发表评论

使用 awk 一�?使用 perl �q?她可��q��正则

Tue, 17 Feb 2009 09:39:00 GMT

参考：
http://my.huhoo.net/archives/2007/08/perl_1.html

��单的
   echo "tt" |perl -ne 'print $_ '
多行
   find . -type f |perl -ne 'print $1 $_ if $_=~ /(.*)flv/'
限定行数
   find . -type f |perl -ne 'print $_ if $_=~ /.*flv/ && $tmp++<10 && $tmp>4'
    #当然shell>> find . -type f |grep ".*flv" |sed -n '5,10p' 和上面是一��L��
    # 不过你看�?nbsp; 其中使用 $1 �{�正则重定向 ,哈哈
�l�果大体�?
./test3/0.flv
./flvplayer.swf
./test.flv
./0.flv

perl -h
-e program        one line of program (several -e's allowed, omit programfile)
-n                assume "while (<>) { ... }" loop around program
-p                assume loop like -n but print line also, like sed

刘凯�?/a> 2009-02-17 17:39 发表评论

perl log4j 分析日志

Tue, 17 Feb 2009 07:50:00 GMT

普通的 log4j 日志 �Q?br /> 当然 �Ҏ��要求 �Ҏ��处理 �?br />
#!/bin/perl
open(FILES,"/data/apps/schedule/tomcat/logs/app/app.log.2009-02-17");
while (<FILES>) {
    next if !/^\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s+ERROR\s+/;
    if($_=~/(.*-){3}(.*?):(.*)$/g){
        $error{$2}++ ;
    }elsif($_=~/(.*-){3}(.*)/g ){
        $error{$2}++ ;
    }
    。。。。�?br />     }
print "$_: $error{$_}\n" for(keys %error);

java.net.SocketTimeoutException: 4
test1 illegal!: 4
cd illegal!: 4
test2 illegal!: 4
MusicAlbum not exists: 5
Not Found Any File: 14
java.net.ConnectException: 3
Album not exists: 2
。。�?br />

刘凯�?/a> 2009-02-17 15:50 发表评论

亚洲youwu永久无码精品 ,亚洲色婷婷综合开心网,亚洲欧洲无卡二区视頻

shell txt 分析���结

perl 使用���结

hadoop streaming( hadoop + perl )���试