Tags » SPARK


‘spark’ cites the first sentence of every paragraphs. titles links to the original web page.

278 more words

Analyzing Bike Share Data

In this series, I am going to use Spark to analyze the Bay Area’s Bike Share Data. You can download the dataset from http://www.bayareabikeshare.com/open-data

First let’s find out the top popular start terminals… 287 more words


Mysterious Time Gap in Spark Job Timeline

Sometime ago one of my clients asked me a question when reviewing a Spark job: why there is a time gap in the event timeline, sometimes can be as long as one minute. 6,505 more words


Integrate Spring boot Rest API, Spark, MongoDB and Azure

In this post I have shown how to connect end to end Spark, MongoDB, Rest interface and web client to consume rest service created using Spring boot on Azure. 1,540 more words


Tuning Spark Back Pressure by Simulation

Spark back pressure, which can be enabled by setting spark.streaming.backpressure.enabled=true, will dynamically resize batches so as to avoid queue build up. It is implemented using a Proportional Integral Derivative (PID) algorithm. 1,356 more words


Christmas Lights 

They come out during December

And make it special even more

Lights twinkle and sparkle

And I feel the moment slow down