IBM Watson Data & AI - Structured Ideas

This Ideas portal is being closed.  Please enter new idea at http://ibm.biz/IBMAnalyticsIdeasPortal

Remove 'insert to code' messagehub functionality on pyspark notebooks

It's dangerous that we inject connection code into pyspark notebooks for connecting to messagehub. Does spark streaming even support messagehub with python? This post suggests it doesn't: https://issues.apache.org/jira/browse/SPARK-16534. The danger here is that we are going to lead users into spending a lot of time trying to connect to message hub from pyspark when that may never work. A PR was provided to implement the functionality but was rejected because of pyspark streaming issues: https://github.com/apache/spark/pull/14340

See also: https://datascix.uservoice.com/forums/387207-general/suggestions/16481014-extend-insert-to-code

  • chris snow
  • Dec 14 2018
  • Needs review
Customer Name
Role Summary
  • Attach files