Problem while running MapReduce code

Question

Hi,

I am getting error for classes which are imported. I have written below code. Kindly provide step by step information on creating classed. Shall I create 3 java programs for driver, mapper and reducer?

package AlphabetCountDemo;

import java.util.*;

import java.io.*;


import org.apache.hadoop.conf.Configuration;

import org.apache.hadoop.fs.*;

import org.apache.hadoop.io.*;

import org.apache.hadoop.mapreduce.*;

import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;

import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;


public class AlphabetCountClass {


public static void main(String[] args) throws ClassNotFoundException,IOException,InterruptedException{

  // TODO Auto-generated method stub

  Configuration conf = new Configuration();

  Job job = Job.getInstance(conf, "WordCount");

  job.setMapperClass(AlphabetWordCountMapper.class);

  job.setReducerClass(AlphabetWordCountReducer.class);

  job.setCombinerClass(AlphabetWordCountReducer.class);

  job.setMapOutputKeyClass(IntWritable.class);

  job.setMapOutputValueClass(IntWritable.class);

  job.setOutputKeyClass(IntWritable.class);

  job.setOutputValueClass(IntWritable.class);

  job.setJarByClass(AlphabetWordCount.class);

  FileInputFormat.addInputPath(job, new Path(args[0]));

  FileOutputFormat.setOutputPath(job, new Path(args[1]));

  System.exit(job.waitForCompletion(true) ? 0 : 1);

 }

}

Answer 1

You will have to create a Mapper class and a Reducer class, and then when you create a Driver class and add these classes you will not get the errors.

You can create three different classes for Mapper, Reducer and the Driver or else you can create a single Driver class and then create two inner classes for Mapper and Reducer.