Overview

Javadoc

Contents

• Invoking an Executor
• Creating an Executor
  • Implementing an external executor
  • Implementing an internal executor
  • Implementing Executor directly
• Advanced Usage
  • Formats, Process Descriptors, and Class Paths
  • Managing I/O
  • Description of ThreadExecutor
  • Description of ProcessExecutor

Execution API

Invoking an Executor

Invoking an executor is fairly straightforward. First, an Executor must be obtained appropriate to the class to be executed. There are several ways to do this:

• Look for a registered executors in the system's services registry. The Services API describes several ways to do this generically for all service types; to look specifically for executors, you may use the convenience methods Executor.getDefault(), Executor.executors(), Executor.find(Class), or Executor.find(String).
• Create a new executor from a specific implementation. The API provides two public implementations which may be used, both of which assume a standalone Java class (i.e. containing a public static main(String[]) method): an internal executor, which runs the class in a separate thread(group) in the same VM as the IDE; and an external executor, which runs the class in a separate Java process, and may be configured to use a specific VM, class path, etc.

  The external executor provides a measure of safety against VM crashes, and permits a few Java security-related operations; the internal executor is able to run with less overhead, and to make calls into classes in the IDE itself.

• The preferred approach, though, is to use ExecSupport.getExecutor(...), which permits data loaders to specify a particular executor to use on a class (actually, the data object containing it). For example, a data loader designed to look for servlets could mark the data objects it creates to use a special executor that would run the servlets in a web-server-like container. Users can also configure this selection.

The returned ExecutorTask permits limited control over the resulting execution, namely stopping the task, waiting for it to finish, and examining its exit status.

If the ExecutorTask object is not needed, you may even leave the job of finding an appropriate executor and running it properly to the ExecCookie, invoking it with ExecCookie.start(). This cookie may be obtained by a call to DataObject.getCookie(Class) or Node.getCookie(Class), as appropriate.

Even more conveniently, just given a set of data objects or nodes, ExecuteAction.execute(DataObject[],boolean) and ExecuteAction.execute(Node[],boolean) may be used to find the cookie and execute the objects, compiling first if requested.

Creating an Executor

Implementing an external executor

You probably only need to override one method: ProcessExecutor.createProcess(ExecInfo). This method returns an actual system process (already running) which the executor should control. While you could create this process yourself using Runtime.exec, conventionally external executors just keep a template for the command to be run, as a Bean property accessible via ProcessExecutor.getExternalExecutor().

For some uses it will suffice simply to provide a different default value for this property, i.e. set a different value in the constructor for the executor. You may do this if the possible commands to start the process do not involve any extra substitutable tags beyond what is provided by default: the class name (fully-qualified and dot-separated); a list of user-set application arguments; the path to the JDK; and various standard classpath components.

In this case, just create a default NbProcessDescriptor. Its constructor lets you specify the process name itself; a list of process arguments (separated by spaces and possibly quoted); and optionally a user-visible paragraph explaining what the arguments are for, to be used in the custom property editor. Both the process name and the list of arguments may have substitutable tags in them, which are enclosed by curly braces and whose standard names (sans braces) are given in the TAG_XXX constants in ProcessExecutor.Format. You will not need to override createProcess.

However, in many cases these default tags will not suffice for your use. For example, in the case of an RMI-specific executor, the application might need to be started with an argument giving the registry port number. It is inappropriate to modify the NbProcessDescriptor to include this information on every run because the same executor may be shared by many classes, each of which might use its own port number. Also, complex values should be shown for UI purposes as separate configurable properties on the executor's property sheet (as determined by its BeanInfo). So in such cases, you need to create your own format for the process command.

Doing this need not be very difficult. You should still set a proper default value for the executor command by creating an NbProcessDescriptor with your desired command format. The difference is that you may use new tags according to your needs. Now you will need to create a command format, probably by subclassing ProcessExecutor.Format. Your format class should have a constructor specifying at least as much as the super constructor, and then contain some additional information concerning values for the new tags. In the constructor, use MapFormat.getMap() to retrieve the map from tags to their values (which will already have ProcessExecutor.Format's standard tags for the class name, class path, etc.) and add your own tags and matching string values to the map.

Finally, override createProcess. Your implementation should call NbProcessDescriptor.exec(Format) on the external command template from your Bean property. (Or use one of the overloads of this method if you wish to pass in environment variables to the created process, or as of JDK 1.3 release version an alternate working directory.) You can just create a new instance of your format to pass to this method, which will mean providing it with all requisite data to substitute the tags with.

Some executors will be able to supply such additional data via special means, e.g. a tag representing a registry port might get its value automatically from a list of free registry ports in the system. In other cases, the executor must be configured by the user to provide values for these tags, so you should make Bean properties for such values and call their getters when constructing the format. The advantage of doing this over including such values directly into the template string is that you can provide a pleasant property editor for such properties in your BeanInfo, and have the executor do the work of constructing the correct string to pass from this, rather than the user.

Implementing an internal executor

1. ThreadExecutor.checkClass(Class) should examine the Class object already created for the specified class name and ensure that is satisfies the executor's requirements. Note that this assumes that you are executing an actual Java class and that the class to be executed is in fact the same as the class name produced in the ExecInfo; otherwise you will need to directly extend Executor and implement more customized logic.

   If the class is appropriate, just return; otherwise you may throw an IOException preferably detailing the reasons for the failure, as explained in more detail below.

2. ThreadExecutor.executeClass(...) should simply do whatever you like with the class to execute it. In the case of regular internal execution, this just means finding the main method via Java Reflection, and running it with the supplied arguments. Note that within the scope of this method, all I/O via the standard system streams automatically gets redirected to a tab in the Output Window; so if you receive any exceptions (such as within an InvocationTargetException), you may simply print the stack trace.

   ThreadDeath may be thrown by the execution engine if the process is manually stopped (by the user or via its ExecutorTask), so if you catch this it is best not to print its stack trace. Also, the executed code within this method may invoke System.exit to end the application, without shutting down the IDE.

Implementing Executor directly

1. Executor.needsIO() may be overridden to return false if you are sure that the executed process will not try to use the system input, output, and error streams (or if it should not need to). A class running in an external process whose output you wish to capture should generally have its streams redirected to the Output Window, e.g. by using Process.getInputStream(). The streams on the Output Window may be obtained from the execution engine (see below), using ExecutorTask.getInputOutput() on the task returned from ExecutionEngine.execute(...), and then InputOutput.getOut() and so on. Now you can just copy characters from one stream to the other.

   Some executors, such as for servlets (perhaps), already have alternate input, output, and logging facilities, so you should indicate that I/O is not necessary, to save a little overhead.

   Currently this method is actually only informational in Executor, though if defined on a subclass of ProcessExecutor or ThreadExecutor it will have an effect.

   For "internal" executors that run the process in the same VM, the IDE can take care of redirecting the input and output of the process to the Output Window for you; see Managing I/O for details.

2. Executor.execute(ExecInfo) must be implemented to actually do the work of the executor.

   The ExecInfo it is passed will contain the class name to start, as well as a list of arguments - for normal execution styles these arguments should be passed to the main(String[]) method; for other executors they may have a different interpretation (e.g. a servlet executor could interpret them as "name=value" pairs and pass them to ServletConfig).

   This method should start the process asynchronously and must return a ExecutorTask object as soon as the process has started running. If you are doing something very simple as the task of the execution, i.e. it is not a long-running process and/or you wish to explicitly manage its progress, you may simply implement ExecutorTask yourself. However, generally it is better to let the execution engine create a task for you, so that it can be handled consistently by the system; for details, see Managing I/O which discusses how the system-supplied task will behave.

   You may throw an IOException from this method to indicate that the class is not suitable to be run at all. For example, it may not implement a required interface, not be the right type of object, an external command might not have been found, etc. If such an exception is thrown, the user will be prompted to select an alternate executor for the class, under the assumption that this choice was simply incorrect. Please provide a localized message for the exception distinct from the detail message in order to ensure that the specifics of the failure are suitable for presentation to an inexperienced user. Note that exceptions or errors occurring during the actual execution of the process, once it has been successfully started, should be handled within the scope of the ExecutorTask and normally this just means printing the stack trace to the Output Window via an InputOutput handle.

Advanced Usage

Formats, Process Descriptors, and Class Paths

1. NbProcessDescriptor is an object representing the abstract command template. This class has a standard custom property editor which should be familiar to anyone who has used external execution, compilation, or debugging: it displays a dialog with a command name, some arguments in a panel, and an optional description key.

   Typically both the command name and list of arguments are actually templates for the real thing, with embedded substitution codes; then the description key can explain to the user what the substitution codes mean.

   The process descriptor itself does not include any information about how to substitute these codes, if there are any. Rather, it presents ways to create a running process from itself: NbProcessDescriptor.exec() simply runs the literally supplied string, assuming it does not need any substitutions; but NbProcessDescriptor.exec(Format) applies a (textual) format to the command name and arguments list before invoking Runtime.exec. The format can in principle be any format you like.

2. NbClassPath is an object used to represent a classpath, which is of course frequently used in Java-related applications, though implementors of executors unrelated to Java development will probably ignore it. Its main purpose is that it also has a custom property editor, making it convenient to use as a Bean property on service types such as executors.

   Static methods in this class retrieve the four standard classpath components: the plain system classpath (i.e. IDE startup classpath); the boot class path of the VM (usually not included explicitly in commands); the IDE library path, including any dynamically-loaded components such as modules; and a path corresponding to the current Repository. For the last, you can specify a file system capability (such as compilation, debugging, or execution) which should be used to filter filesystems in the Repository according to whether the user intended them to be used for the purpose at hand (for example, JDK sources should not be used during compilation).

3. ProcessExecutor.Format and along similar lines, ExternalCompilerGroup.Format (as well as analogous classes in the standard debugger implementation) provide standard Format implementations suitable for passing as an argument to NbProcessDescriptor.exec. These implementations are useful for two reasons:
   • They extend MapFormat, a highly flexible Format implementation which permits substitution by named keys (and is also normally used to implement substitution of special tokens in file templates). Use of MapFormat allows templates such as:
     -classpath {repository}{:}{extradir}
     to be substituted correctly if values are supplied for the keys. This is more pleasant for a user than e.g. MessageFormat.
   • The default implementations already include in the substitution map a number of keys, such as for classpath components, the class name to execute and arguments (or list of files to compile), the path to the Java installation, and so on.
   Customizing these formats is easy. Just extend MapFormat or one of the more specialized formats just mentioned, passing in any required arguments to the superclass constructor; then use MapFormat.getMap() to retrieve the key-value mapping and modify it with any additional keys you like (do not include the delimiters of the tokens, by default curly braces, in the keys themselves). MapFormat.processKey(String) and other advanced features of MapFormat may be used to give greater flexibility.

Managing I/O

The IDE provides two forms of support for these considerations, which complement one another in that they cover both dynamic scope (code run within a certain thread group and time window) and quasi-lexical scope (code derived from a certain classloader). Both will accomplish automatic I/O redirection and trapping of exit calls; implementors may use either or both to ensure coverage of executed code.

First it is necessary to describe the InputOutput interface which is common to both forms of support. This interface is essentially an abstract description of the public capabilities of an Output Window tab. Thus, a certain level of control is afforded to users of this interface in terms of managing selection of the tab, getting its various I/O streams and using them, controlling whether the two output streams are mixed together, etc.

There are three ways to get a useful InputOutput implementation:

1. You may use TopManager.getIO(String) to create an Output Window tab with a specified name and return its InputOutput representative. Or, use TopManager.getIO(String,false) to try to reuse an existing tab of the same name, if there is one.
2. If ExecutionEngine.execute(...,InputOutput) is called and its third argument is null, this signifies that the engine itself should choose an appropriate Output Window tab, probably named after the supplied task name. The returned ExecutorTask will then already be using the proper InputOutput, and this may be retrieved if needed using ExecutorTask.getInputOutput().
3. You may use the constant InputOutput.NULL to indicate that no I/O at all is desired for a task - that is, its I/O streams will be trapped, but writes will be discarded and reads will return end-of-file. This constant should be used if it is very clear that I/O will not be used and some overhead should be saved; or if the I/O is specifically not desired.

Given an InputOutput, you can use it in either of the two ways given below, or both at the same time, and I/O streams will be trapped.

1. ExecutionEngine.execute(...) is the primitive means for providing I/O services to a block of code using dynamic scope. (You may obtain the ExecutionEngine via TopManager.getDefault().getExecutionEngine().) The supplied Runnable is run asynchronously in its own thread and thread group, i.e. well-isolated from the rest of the IDE. Any uses of the system I/O streams within that thread group will automatically be trapped and redirected to the InputOutput. Note that code which creates new threads will normally create them in the same thread group as the calling code, so it is fine for there to be complex multithreaded code in the supplied runnable; it will all be handled. Runnable-spawned code which is destined for other thread groups will not be handled, however; for example, RequestProcessor.postRequest(Runnable) will execute code in the IDE's main thread group, not the one created by the execution engine.

   The supplied task name is optional and may be null. If it is supplied, it will be used as a display name that may show up e.g. in the Execution View, so that the user may see that the process is running and stop it if needed. If null, the task will not be visible to the user.

   As mentioned above, you may pass an existing InputOutput to this method so as to ask the execution engine to use that I/O; or you may leave this argument null to request that a suitable I/O tab be created for you. In either case, the I/O tab actually in use will be available via ExecutorTask.getInputOutput().

   The task returned from the engine permits you to find its I/O implementation; let the runnable continue asynchronously; stop it at any time; or wait for it to finish (i.e. block) and get its return status. As far as return status goes, zero means success, nonzero numbers mean failure. The execution engine's default task implementation just considers natural termination of the Runnable to be success, and aborted tasks to have failed (and some unspecified nonzero number returned as the status). For some uses, such as execution of external applications which may return a meaningful exit status, you may need to create a special wrapper ExecutorTask implementation which provides the correct exit status (since the execution engine is not aware of such codes).

   The dynamic scope of execute(...) also covers attempted uses of System.exit(int) (or Runtime.exit(int)). If an exit is attempted within the task's dynamic scope (i.e. thread group), this is caught by the IDE's security manager implementation, and the task is instead stopped (as if by ExecutorTask.stop()). In practice this means that all living threads in the thread group will receive ThreadDeath to stop them. So executor implementations should be prepared to have thread death thrown on them, and anyone catching Throwable should specifically consider whether the throwable is a thread death; in this case, the surrounding code should be stopped promptly, and there is no need to print a stack trace. The thread death may be thrown either because of an attempted exit call, or because of an explicit use of ExecutorTask.stop(). (Note that there is no way to recover the exit status which the attempted exit call used.)

2. new NbClassLoader(InputOutput) creates a special classloader that is aware of an InputOutput obtained as above. Normally NbClassLoader is just used to load classes from the Repository, and is used e.g. by TopManager.currentClassLoader() (which also refreshes the classloader as needed). This constructor, however, is "magic" in that it will still load classes from the Repository (always giving preference to its parent TopManager.systemClassLoader()), but I/O calls quasi-lexically contained in such classes (i.e. made directly by such a class, or by other invoked code when such a class is on the stack) will be redirected to the supplied I/O.

   Note that this works regardless of thread group, so that for example a runnable loaded from this classloader which is posted to the RequestProcessor will still use I/O redirection, unlike with ExecutionEngine.execute(...).

   NbClassLoader does not specially handle System.exit calls since such code need not be in any particular thread group, so it does not make sense to try to stop some task. Rather, any code in the system which is outside the IDE's standard trusted codebase which tries to exit the VM will receive a security exception. Note that this exception specifically does nothing in response to printStackTrace(), which is usually desirable because general-purpose exception catching code such as is common in executors just prints any received stack traces, whereas System.exit should simply end the task without triggering a noisy and confusing SecurityException trace.

   The IDE's trusted codebase is currently set by bin/ide.policy in the installation directory, and specifies that Java platform code, code loaded from modules (including test modules), and code loaded from an NbClassLoader with the InputOutput constructor is to be trustworthy; other Filesystems code will typically fall outside of these codebases and so is subject to the security manager. Such "untrusted" code is probably restricted from security-sensitive calls (but do not count on it). Such code can still call TopManager.exit() to explicitly exit the IDE.

Description of ThreadExecutor

There are several places in ThreadExecutor that exceptions may be thrown, and they must be handled differently. First, exceptions (or errors) may be throws during the main execute method itself. To these may be added anything thrown by the runnable passed to the execution engine during its setup phase, when it is setting up I/O management and attempting to load the specified class. All of these throwables may be divided into four groups:

1. ThreadDeath may be caught, but attention is paid to where it is caught. If the thread is about to exit anyway due to something being caught, then nothing additional is done. If the thread might have continued otherwise after the catch block, it is rethrown to make sure that it causes a prompt termination of the process.
2. Other errors are simply rethrown. If thrown inside the runnable passed to the execution engine, they are automatically transmitted to the execute method instead.
3. IOExceptions are rethrown, as they are declared in execute.
4. Other exceptions are wrapped in new FoldingIOException(Throwable).

ThreadExecutor makes use of both forms of I/O management provided by the execution engine; dynamic-scope handling primarily to trap I/O calls made by IDE code as well as exit handling from user code; and (quasi-)lexical handling for the user code, which might spawn new thread groups or use existing thread groups in the VM (such as the AWT event thread). So, most of the work performed by execute except for some redirection of throwables is handled by a Runnable passed to ExecutionEngine.execute; this provides dynamic handling. The runnable also loads the user class to be executed (and transitively, its dependencies in the Repository which are not loaded in the IDE itself) using a magic NbClassLoader; this provides lexical handling.

The trick is to get the two forms of I/O handling to agree on a single InputOutput instance, since they should both have the same effect. The InputOutput is provided by the execution engine, not the executor, and is only available using ExecutorTask.getInputOutput on the task returned by the execution engine. Clearly this is too late to pass to e.g. a constructor for the runnable. So, a special trick is used which other implementors of executors may find helpful:

1. The runnable is constructed with no assigned I/O. It has a synchronized run method.
2. While synchronizing on the runnable, it is passed to the execution engine, which tries to start to run it but blocks on the monitor.
3. Meanwhile, the task is returned anyway; execute obtains the desired InputOutput and passes it to the runnable using a designated access method.
4. execute waits on the runnable, permitting it to start running.
5. The runnable uses the InputOutput which it now has to load classes using a new NbClassLoader.
6. The runnable continues to load the user class and check it for appropriateness, storing up any exceptions for later. When it has been checked, the runnable notifies execute via its monitor to resume, at which point execute looks for any exceptions and rethrows them, or else it returns the task it got from the execution engine.
7. Now independently of the executor, the runnable actually runs the user class to completion.

Description of ProcessExecutor

Each of the three I/O streams from the generated process (created by createProcess which is described in detail above) are assigned to one of the I/O streams associated with the InputOutput. Each such pairing is implemented by a separate thread; this runs in the thread group created by the execution engine, since it was created by the runnable.

The external executor actually keeps track of the entire ExecutorTask provided by the execution engine, not just its InputOutput, since it cannot rely on the execution engine to kill the external process directly. (The execution engine normally just kills all threads in the thread group it created, in order to stop a task it created.) Rather, the internal runnable creates a new ExecutorTask which provides the additional needed behavior and returns it in proxy, while keeping the original for its own use.

The main trick involves program termination (from the IDE). If the code calling ProcessExecutor.execute pays attention to its returned task and directly calls ExecutorTask.stop on this task, then there is no problem: this method is directly implemented to stop both the external process itself, and all I/O proxy threads. Stopping the process causes the result method to unblock and get an exit status, which also notifies task listeners that the task is finished; and when the last proxy thread stops, then the execution engine sees that the thread group is dead (no live threads remain in it) and so it also knows that its own task is finished, causing (among other things) that entry to disappear from the list of processes in the Execution View.

But if the task is stopped from the Execution View (i.e. stop is called on the execution engine's own task, rather than the synthetic task from ProcessExecutor), then the synthetic task needs to be stopped as well. This is implemented by having the synthetic task attach a task listener to the original one; when the original one finishes, the synthetic one stops itself as well (and consequently shuts down the external process and the copy threads, if they have not been killed already).