IBM Watson Data & AI - Structured Ideas

Welcome to the idea forum for IBM Watson Data & AI — our team welcomes any feedback, requests, and suggestions you have for improving our products! 

This forum allows us to connect your product improvement ideas with IBM product and engineering teams. 

Provide much more documentation on the process of debugging spark jobs

Intermediate to advanced users need more documentation on debugging spark jobs to identify where their code needs to be improved.

  • chris snow
  • Dec 14 2018
  • In Plan
Role Summary
  • Dec 14, 2018

    Admin Response

    Thank you so much for this recommendation. We've begun working on exposing our Spark Monitoring Console (Spark History Server) back in the product UI so users will be able to see an option from their notebooks to go to the Spark Monitoring Console (where successful, failed, and all the activity in your cluster).
  • Attach files
  • chris snow commented
    14 Dec 17:58

    Hi Cecelia, the UI will be useful but documentation is more important at this stage. Without documentation, the spark history server just provides some pretty graphics.

  • chris snow commented
    14 Dec 17:58
  • chris snow commented
    14 Dec 17:58

    + documentation on performance tuning / best practices, e.g. Loading from dashdb using partition ranges

  • Anonymous commented
    14 Dec 17:58

    This is 100% required and would save IBM support a lot of time and effort also.