IBM Watson Data & AI - Structured Ideas

This Ideas portal is being closed.  Please enter new idea at http://ibm.biz/IBMAnalyticsIdeasPortal

Provide much more documentation on the process of debugging spark jobs

Intermediate to advanced users need more documentation on debugging spark jobs to identify where their code needs to be improved.

  • chris snow
  • Dec 14 2018
  • In Plan
Why is it useful?
Who would benefit from this IDEA?
How should it work?
Idea Priority
Priority Justification
Customer Name
Submitting Organization
Submitter Tags
  • Dec 14, 2018

    Admin Response

    Thank you so much for this recommendation. We've begun working on exposing our Spark Monitoring Console (Spark History Server) back in the product UI so users will be able to see an option from their notebooks to go to the Spark Monitoring Console (where successful, failed, etc...jobs and all the activity in your cluster).
  • Attach files
  • chris snow commented
    December 14, 2018 17:58

    Hi Cecelia, the UI will be useful but documentation is more important at this stage. Without documentation, the spark history server just provides some pretty graphics.

  • chris snow commented
    December 14, 2018 17:58
  • chris snow commented
    December 14, 2018 17:58

    + documentation on performance tuning / best practices, e.g. Loading from dashdb using partition ranges

  • Anonymous commented
    December 14, 2018 17:58

    This is 100% required and would save IBM support a lot of time and effort also.