阅读815 返回首页    go 阿里云 go 技术社区[云栖]


WordCount2

WordCount2中我们使用Hadoop框架中给我们提供的类来实现,我们的目的是用最少的代码来完成同样的任务

import java.io.IOException;

import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.JobClient;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.FileOutputFormat;
import org.apache.hadoop.mapred.lib.LongSumReducer;
import org.apache.hadoop.mapred.lib.TokenCountMapper;



public class WordCount2 {

	public static void main(String[] args) {
		JobClient job =new JobClient();
		JobConf conf = new JobConf(WordCount2.class);
		
		FileInputFormat.addInputPath(conf, new Path(args[0]));
		FileOutputFormat.setOutputPath(conf, new Path(args[1]));
		
		conf.setOutputKeyClass(Text.class);
		conf.setOutputValueClass(LongWritable.class);
		
		conf.setMapperClass(TokenCountMapper.class);
		conf.setCombinerClass(LongSumReducer.class);
		conf.setReducerClass(LongSumReducer.class);
		
		job.setConf(conf);
		
		try {
			JobClient.runJob(conf);
		} catch (IOException e) {
			e.printStackTrace();
		}
	}
}


最后更新:2017-04-04 07:03:49

  上一篇:go 我问自己代言,甄嬛篇
  下一篇:go java方法执行控制台命令工具类