Quantcast

Spark Streaming Cassandra Transformation

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Spark Streaming Cassandra Transformation

Deepu Raj
Hi Team,


1. Can someone please guide on  how to change the following transformation  
change for the below data in Spark 2.0?

{"name":"Michael", "age":30, "state":AZ, "Job": SoftwareDev}
{"name":"Andy", "age":30,  "state":WA, "Job": Mana}
{"name":"Justin", "age":19, , "state":VIC, "Job": DBAdmin}

KafkaUtils.createStream[String, String, StringDecoder, StringDecoder](
ssc, kafkaParams, topicMap, StorageLevel.MEMORY_ONLY)
.map{ case (_,json) =>  
JsonParser.parse(json).extract[MonthlyCommits]}.saveToCassandra("github_stats","commits_aggr")


2. What is the best way to do this push JSON into Cassandra without using  
DF?

Thanks,
Deepu Raj
--
Using Opera's mail client: http://www.opera.com/mail/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Loading...