How to read Spark elements having multiple lines each

0 votes
I have an element in Spark that has multiple lines each. I want to read the element using Spark. How can I do that?
Dec 12, 2018 in Big Data Hadoop by digger
• 26,740 points
• 2,453 views

1 answer to this question.

0 votes

Try this:

val new_records = sc.newAPIHadoopRDD(hadoopConf,classOf[NLineInputFormat],classOf[LongWritable],classOf[Text])

answered Dec 12, 2018 by Omkar
• 69,180 points

Related Questions In Big Data Hadoop

0 votes
1 answer

How to read multiple files in hdfs?

If you want to read files with ...READ MORE

answered Feb 7, 2019 in Big Data Hadoop by Omkar
• 69,180 points
• 5,543 views
0 votes
1 answer

How to sync Hadoop configuration files to multiple nodes?

For syncing Hadoop configuration files, you have ...READ MORE

answered Jun 21, 2018 in Big Data Hadoop by HackTheCode
• 2,941 views
0 votes
1 answer

How do I connect my Spark based HDInsight cluster to my blob storage?

Go through this blog: https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-hadoop-use-blob-storage#access-blobs I went through this ...READ MORE

answered Apr 15, 2018 in Big Data Hadoop by Shubham
• 13,490 points
• 3,344 views
0 votes
1 answer

How to tune Spark jobs & optimize the performance?

You need to know the cluster properly ...READ MORE

answered Apr 18, 2018 in Big Data Hadoop by coldcode
• 2,090 points
• 3,155 views
+1 vote
2 answers
+1 vote
1 answer

Hadoop Mapreduce word count Program

Firstly you need to understand the concept ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
• 14,940 views
0 votes
1 answer

hadoop.mapred vs hadoop.mapreduce?

org.apache.hadoop.mapred is the Old API  org.apache.hadoop.mapreduce is the ...READ MORE

answered Mar 16, 2018 in Data Analytics by nitinrawat895
• 11,380 points
• 5,666 views
+2 votes
11 answers

hadoop fs -put command?

Hi, You can create one directory in HDFS ...READ MORE

answered Mar 16, 2018 in Big Data Hadoop by nitinrawat895
• 11,380 points
• 123,097 views
0 votes
1 answer

How to read more than one files in Apache Spark?

Try this: val text = sc.wholeTextFiles("student/*") text.collect() ...READ MORE

answered Dec 11, 2018 in Big Data Hadoop by Omkar
• 69,180 points
• 3,614 views
–1 vote
1 answer

How to use multiple spark version?

You can use the  SPARK_MAJOR_VERSION for this. Suppose ...READ MORE

answered Dec 27, 2018 in Big Data Hadoop by Omkar
• 69,180 points
• 3,335 views
webinar REGISTER FOR FREE WEBINAR X
REGISTER NOW
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP