Spark Streaming Cassandra Transformation

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Spark Streaming Cassandra Transformation

Deepu Raj
Hi Team,


1. Can someone please guide on  how to change the following transformation  
change for the below data in Spark 2.0?

{"name":"Michael", "age":30, "state":AZ, "Job": SoftwareDev}
{"name":"Andy", "age":30,  "state":WA, "Job": Mana}
{"name":"Justin", "age":19, , "state":VIC, "Job": DBAdmin}

KafkaUtils.createStream[String, String, StringDecoder, StringDecoder](
ssc, kafkaParams, topicMap, StorageLevel.MEMORY_ONLY)
.map{ case (_,json) =>  
JsonParser.parse(json).extract[MonthlyCommits]}.saveToCassandra("github_stats","commits_aggr")


2. What is the best way to do this push JSON into Cassandra without using  
DF?

Thanks,
Deepu Raj
--
Using Opera's mail client: http://www.opera.com/mail/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]