Tags » SPARK

Kick Off Spark

My first Spark section:

scala> import org.apache.spark.sql.SQLContext
import org.apache.spark.sql.SQLContext

scala> val sdf = spark.read.option("header", true).csv("Documents/spark/credit_count.txt")
sdf: org.apache.spark.sql.DataFrame = 

scala> sdf.printSchema()
 |-- CARDHLDR: string (nullable = true)
 |-- DEFAULT: string (nullable = true)
 |-- AGE: string (nullable = true)
 |-- ACADMOS: string (nullable = true)
 |-- ADEPCNT: string (nullable = true)
 |-- MAJORDRG: string (nullable = true)
 |-- MINORDRG: string (nullable = true)
 |-- OWNRENT: string (nullable = true)
 |-- INCOME: string (nullable = true)
 |-- SELFEMPL: string (nullable = true)
 |-- INCPER: string (nullable = true)
 |-- EXP_INC: string (nullable = true)
 |-- SPENDING: string (nullable = true)
 |-- LOGSPEND : string (nullable = true)

scala> sdf.createOrReplaceTempView("tmp1")

scala> spark.sql("select count(*) as obs from tmp1").show()
|  obs|

… 134 more words

Starter script for rsparkling (H2O on Spark with R)

The rsparkling R package is an extension package for sparklyr that creates an R front-end for the Sparkling WaterSpark package from H2O. This provides an interface to H2O’s high performance, distributed machine learning algorithms on Spark, using R. 234 more words

Machine Learning


Wishing eyes could forget a pain so apparent.
Splitting the soul and the city apart.
Stumbling along a lonely corridor, with the lights turned off by god. 126 more words


Featured Content

šŸ’„AdvoCare 7 Day Spark ChallengešŸ’„


Take the AdvoCare 7 Day Spark Challenge today! Order a package of Spark Energy Drink and drink it twice a day for 7 days instead of the normal coffee, soda, or other energy drink. 131 more words

The Way I Want To Be Loved

There’s a difference between in being loved and being loved the way you prefer to be loved; the way you longed to be loved.

It might not be the way you deserve to be loved, but it’s what you are familiar with. 74 more words

Hamish and Bree

His name is Hamish, her name is Bree,
Imprisoned by their minds, they long to be free; 226 more words

Simply Inspiring Uninspiring Poems

Spark announced there new unmanned aircraft from the DJI operated by hand movement

Dji unveiled its new unmanned aircraft, dubbed Spark, a 12 megapixel camera with video capability of up to 1080p resolution.

The new aircraft features a very light weight and a small size slightly larger than the palm of the hand. 208 more words