Overview

This proposal suggests making the underlying native JavaScript debugging api (already built into Navigator) securely accessible to user JavaScript code. We will produce a graphical JavaScript debugger for Navigator 5.0 which is written entirely in JavaScript and dynamic html. It will not require the use of Java. We will release the html and JavaScript source to this new debugger to the JavaScript programmer community. Perhaps we will even enlist that community in writing some of the code for the debugger itself. The debugger will be completely -- and dynamically -- extensible. The debugger will be able to debug instances of itself. It will be possible for users (i.e. JavaScript programmers) to modify and reload the debugger while using it. We will fully document the api used by the debugger so that anyone could write their own debugger if they don't like ours.

This new debugger will be a showcase of the power of html and JavaScript in Navigator. It will be an essential tool for the growing community of JavaScript programmers.

Goals

• Provide a tool which will best support the needs of the JavaScript programming community.
• Fully expose the debugger api in a powerful but safe way into the language most useful to the debugger's users -- JavaScript!
• Build and distribute a fully extensible self debugging debugger.
• Shift as much as possible of the activity and responsibility of evolving and maintaining a world class debugger to the community who uses it.
• Promote and display the power of html and JavaScript in our client software.

Features

• 100% html and JavaScript.
• 100% free and open.
• Full support for just-in-time debugging.
• Capable of exploring properties of all objects associated with all windows.
• Support arbitrary inspection/evaluation in any stack frame.
• Breakpoints and data watchpoints.
• Capable of intercepting runtime errors and showing their origins
• Fully event driven like any other html/JavaScript document.
• Capable of debugging instances of itself.
• Fully extensible and reloadable while in use.
• All scarey multi-threaded plumbing hidden under the covers.

Architecture of builtin support

This change will allow JavaScript execution to be blocked for debugging while the user interacts with the debugger window (which runs its JavaScript code on its own thread).

The bulk of the new code to enable the debugger will be in libmocha. The js and jsd modules already fully support the debug api (exposed to native code and reflected into Java). However, some additional work is already being done to enhance that debug api. These enhancements will help for this use of the api.

A file called lm_debug.c will be added to libmocha. It will reflect the debug api into JavaScript via a natively implemented JavaScript object called 'jsdapi'. jsdapi will mostly be a wrapper around the existing js/jsd/jsdebug.h. I have already built a single threaded console JavaScript debugger written in JavaScript using this same approach and can leverage the reflection layer built there.

In addition to reflecting the debug api, jsdapi will provide special debugger startup code and thread gateway code to hide that level of plumbing from the debugger JavaScript code.

Navigator startup sequence

• libmocha initializes
  • calls js/jsd to initialize
  • calls lm_debug to initialize
    • if debugger.html is in specific place on disk
      • put js/jsd in an 'active' state - register callback for debugger keyword and js errors so that if the events occur, then lm_debug will be called and will start a debugger window (loading the afore mentioned debugger.html). js/jsd will track all html and js files loaded in Navigator (being careful to discard copies of pages abandoned by Navigator) to allow for just-in-time access to the source for the debugger.
    • else
      • js/jsd not activated and debugging is not available (except via the legacy Java applet).

• The debugger can be invoked by either hitting a JavaScript error or the 'debugger' keyword (which can either be on a page or by typing javascript:debugger in the location bar or via bookmark)
  • the hook that lm_debug set at startup is called. It creates a new window (with a separate JS thread) and initializes an instance of a nativly implemented JavaScript object called 'jsdapi' which is attached to the new window. This jsdapi object can only be created in this fashion and is not accessible from non-debugger windows. The debugger html document is loaded into the window.
    • the debugger.html file then sets up its UI and sets hooks into jsdapi. It notifies jsdapi that it is done loading.
    • at this point a hook event is posted to the debugger and debugging begins.

• hook is encountered on the target mocha thread (breakpoint, interrupt, error, or debugger keyword) and calls through jsd to hook handler in lm_debug. lm_debug maps this to the function in the debugger which needs to be called to handle the hook. That handler, however, needs to be called on a different thread in order to allow the user to interact with the debugger.
• lm_debug posts a message to the FE using the same mechanism used for events like timeouts. The target mocha thread then goes into an event wait loop ready to be told by the debugger to continue execution, or do an expression evaluation, etc.
• the FE retrieves the hook event on the mozilla thread and routes it to the debugger in the same fashion a timeout or mouse event is routed.
• the debugger responds to the hook by gathering data about the target thread state (ie. stack, source code, etc.) by calling methods of jsdapi. It then can display information to the user in nice fancy interactive windows using the full range of layers, tables, frames, and JavaScript.
• Some calls that the debugger makes into jsdapi can be handled directly by calling functions in js/jsd/jsdebug.h (e.g. gather stack information). Other calls will require thread handoffs (e.g. evaluating an expression in the context of a stack frame in the stopped target thread). jsdapi will hide this complexity:
  • the debugger will call a jsdapi method.
  • that method will post an event into the queue in lm_debug where the target thread has been parked since the hook happened. The debugger's mocha thread will then enter its own event queue awaiting the response from the target mocha thread.
  • the target mocha thread will call into jsdebug.h to do the eval. When the eval returns the target mocha thread will post it back to the debugger's queue and go back to waiting for another event.
  • the debugger mocha thread will receive the answer and return it to the debugger. The debugger will go about its business without knowing that this dance happened.

Architecture of the Debugger UI

I think that it will be possible to make at least as rich and smooth a UI using dhtml as that of the current Java-based debugger. I consider the experience gained by the debugger team in doing this much sophisticated html and JavaScript to be one of the big benefits of the project. By becoming real-world users of the technology the debugger is designed to support we assure that the design of the tool and decisions about low-level engine support fit the needs of its users.

The debugger will be fully event driven. While it will be primarily graphical, it will have a console to allow manual inspection of the target code and arbitrary access to the methods of the objects which comprise the debugger itself. I expect that early development iterations of the debugger UI will be primarily console driven.

There is a lot of good dhtml code available to reuse. Our visual dhtml tool is full of treasures. My inclination is to port much of the logic used in the Java version of the debugger into the dhtml world. This is made significantly simpler by the code in jsapi which hides the cross-thread communication and synchronization between code running on the debugger UI/mocha threads and the target mocha thread.

I am hoping that by doing frequent and open updates of the UI source (and by integrating contributions of code) we can get a small community of JavaScript programmers actively contributing to the tool and supporting each other in extending it. I would very much like to see it take on a life of its own.

How to get there from here

• new file: lm_debug.c in libmocha will reflect jsd, start debugger, and route events.
• js/jsd will need to support JSContext filtering for hooks; i.e. each hook will (optionally) have a list of JSContexts (essentially windows) for which it would like to be notified about debug events. For instance, a given debugger might want to be notified of JavaScript errors in all windows except its own, while a second debugger might only want to catch errors in the other debugger.
• The debugger html and JavaScript code needs to be evolved into existence.

• The code to for multi mocha threads it required. mlm has this working and it is scheduled to be part of Nav 5.0.
• It is critical that no cases exist where code can be stopped in a target mocha thread while layout is locked and unable to process the UI for the debugger. Any problems that may exist in this area are bugs and will need fixing for any client of the multi thread code (though the debugger will be 'testing' this a lot and may be the best 'finder' of potential bugs of this sort).
• We'll have to be very careful with potential new security holes. The main protection is that none of the new code will run unless the user has installed the debugger.html file on his disk. Still, we need to exercise due diligence to assure that pages from the web can not exploit anything here.
• Navigator 4.x really sucks up memory in code that does a lot of document.writes to layers. This is said to be caused by layout's use of arenas - memory is not reclaimed until the Window closes. This is bad for a rich dhtml page. This is likely to be fixed in Navigator 5.
• Time is tight. Approval and support are critical.

• libmocha work to implement app and debugger startup code
• write code for jsdapi object
• write a simple console to debug code in simulated 'other' window
• write some proof of concept dhtml parts - e.g. simple inspector
• Add extensions to JSD for multi-context debugging, fast inspection,...
• integration of multi mocha thread code - work with real tragets
• Improve UI and get it all working smoothly

Despite the utter coolness of this project, I've decided that now is not the right time to pursue it. Rewriting so much UI and working around layers issues does not appear to be the best use of time. Nor is it likely to produce a tool which will most benefit the greatest number of customers. Instead, I'm concentrating on improving the existing Java-based UI; smoothing out the user experience and adding a significantly more powerful data browser. Incorporating Rhino into the debugger to facilitate scripting of the debugger's behavior is a strong possibility.

I still would like to do the JSD-in-JS debugger proposed here. Possibly the Raptor based client will be the right platform to support (and be supported by) this debugger. This will certainly be revisited...