IBM Watson Data Platform - Structured Ideas

Welcome to the idea forum for IBM Watson Data Platform — our team welcomes any feedback, requests, and suggestions you have for improving our products! 

This forum allows us to connect your product improvement ideas with IBM product and engineering teams. 

Remove 'insert to code' messagehub functionality on pyspark notebooks

It's dangerous that we inject connection code into pyspark notebooks for connecting to messagehub. Does spark streaming even support messagehub with python? This post suggests it doesn't: https://issues.apache.org/jira/browse/SPARK-16534. The danger here is that we are going to lead users into spending a lot of time trying to connect to message hub from pyspark when that may never work. A PR was provided to implement the functionality but was rejected because of pyspark streaming issues: https://github.com/apache/spark/pull/14340

See also: https://datascix.uservoice.com/forums/387207-general/suggestions/16481014-extend-insert-to-code

  • chris snow
  • Aug 17 2017
  • Needs review
  • Attach files