This guide will show you how to read in csv files in Apache Spark. We'll walk through how to use this package in both Python and Scala.
In this easy to follow tutorial, learn the basics of Spark DataFrames, how they're composed of RDDs and what they allow you to do in Scala. They're a similar abstraction to pandas DataFrames or R's DataFrames.
This article will walk you through how to resolve the java.lang.OutOfMemoryError: PermGen space exception that can occur when you're trying to start Spark.