IBM Watson Data & AI - Structured Ideas

Welcome to the idea forum for IBM Watson Data & AI — our team welcomes any feedback, requests, and suggestions you have for improving our products! 

This forum allows us to connect your product improvement ideas with IBM product and engineering teams. 

Provide much more documentation on the process of debugging spark jobs

Intermediate to advanced users need more documentation on debugging spark jobs to identify where their code needs to be improved.

  • chris snow
  • Aug 17 2017
  • In Plan
Role Summary
  • Aug 17, 2017

    Admin Response

    Thank you so much for this recommendation. We've begun working on exposing our Spark Monitoring Console (Spark History Server) back in the product UI so users will be able to see an option from their notebooks to go to the Spark Monitoring Console (where successful, failed, etc...jobs and all the activity in your cluster).
  • Attach files
  • chris snow commented
    August 17, 2017 12:22

    Hi Cecelia, the UI will be useful but documentation is more important at this stage. Without documentation, the spark history server just provides some pretty graphics.

  • chris snow commented
    August 17, 2017 12:22
  • chris snow commented
    August 17, 2017 12:22

    + documentation on performance tuning / best practices, e.g. Loading from dashdb using partition ranges

  • Anonymous commented
    August 17, 2017 12:22

    This is 100% required and would save IBM support a lot of time and effort also.