How to merge files in Hadoop

Question

Hi Guys,

I have some files in a folder in Hadoop. I want to merge these files. How can I do that?

Answer 1

Hi@akhtar,

You can use getmerge command in Hadoop. This is the most important and the most useful command on the HDFS filesystem when trying to read the contents of a MapReduce job or PIG job’s output files. This is used for merging a list of files in a directory on the HDFS filesystem into a single local file on the local filesystem. You can see the syntax below.

$ hadoop fs -getmerge /user/data