Better Tab Completions for GroovyShell

GroovyShell provides a text console based environment for interacting with Groovy code. It allows you to type in and execute groovy statements and see the results immediately.  To make the shell a more confortable environment for users to work in, GroovyShell enlists the help of JLine to provide powerful line editing, command history and tab-completion. 

One area in which I feel GroovyShell could use some improvement is tab-completion.  When you hit [tab], the list of "candidates" that GroovyShell provides to JLine are currently limited to the build in shell commands (such as exit, import, inspect, etc).  This is a good start, but the thing that would make the environment immensely more usable is for the shell to supply candidates for variable and method names within the current shell context. 


For example suppose you define a variable in the shell:


     inputFile = new File("data.csv")


When you type in the shell:

    in[tab]   

the shell could go ahead and complete the variable name "inputFile" for you.  Other modern shells and IDE do an excellent job of providing this style of interaction.  When using GroovyShell I find myself compulsively slamming the [tab] key and wondering why the heck it's not working! 


So, I set out to see what was involved to pimp up GroovyShell in this way.  It turns out the task is quite simple.  All the raw functionality is already present in the the shell and JLine, it's just a matter of wiring it all up.

First I took a look at JLine to find out how to add additional completors.  The relevant extension point is the jline.Completor interface, which contains a single  method to be overridden:

int complete(String buffer, int cursor, List candidates)

When invoked,  buffer contains the current contents of the line the user is editing, cursor is the cursor position within that line, and candidates is what the completor must fill up with completions based on the current buffer and cursor position. Easy!

Next, I took a loot at the GroovyShell source to see how I could come up with the candidates.  There are basically only two kinds of completions I was hoping for.  The first is for global variables.  Once you have a reference to GroovyShell's "Shell" instance, these can be retrieved via:  shell.interp.context.variables, which gets you a map of all the globals  currently defined in the shell context.


The other kind of completion is when you already have one or more dots in the expression, optionally followed by some additional characters. For example, suppose you type in


    inputFile.isF[tab]


First, the shell is used to everything to the left of the final dot to resolve it to an object:


    def instance = shell.interp.evaluate(["inputFile"])


Once you have an instance, you can use reflection to look up all public fields and methods that match the prefix (in this case, "isF" which is a prefix for the "isFile" method.).


The last task is to add the new Completor to the JLine object within GroovyShell, without disrupting the functionality of the existing Completors.  I looked through the GroovyShell code, but unfortunately it appears there is no way to do this without making a code change.  Fortunately though the code change is very minimal.  In he InteractiveShell constructor:


    this.reader = new ConsoleReader(
        shell.io.inputStream, 
        new PrintWriter(shell.io.outputStream, 
        true))
    reader.addCompletor(new ReflectionCompletor(shell))
    this.completor = new CommandsMultiCompletor()

Here is source code for the ReflectionCompletor class:

package org.codehaus.groovy.tools.shell
import java.util.List;

import jline.Completor


/**
 * Implements the Completor interface to provide competions for
 * GroovyShell by using reflection on global variables.
 *
 * @version $Id$
 * @author Marty Saxton
 */
class ReflectionCompletor implements Completor {

    private Shell shell;

    ReflectionCompletor(Shell shell) {
        this.shell = shell
    }

    int complete(String buffer, int cursor, List candidates) {

        int identifierStart = findIdentifierStart(buffer, cursor)
        String identifierPrefix = identifierStart != -1 ? buffer.substring(identifierStart, cursor) : ""
        int lastDot = buffer.lastIndexOf('.')

        // if there are no dots, and there is a valid identifier prefix
        if (lastDot == -1 ) {
            if (identifierStart != -1) {
                List myCandidates = findMatchingVariables(identifierPrefix)
                if (myCandidates.size() > 0) {
                    candidates.addAll(myCandidates)
                    return identifierStart

                }
            }
        }
        else {
            // there are 1 or more dots
            // if ends in a dot, or if there is a valid identifier prefix
            if (lastDot == cursor-1 || identifierStart != -1){
                // evaluate the part before the dot to get an instance
                String instanceRefExpression = buffer.substring(0, lastDot)
                def instance = shell.interp.evaluate([instanceRefExpression])
                if (instance != null) {
                    // look for public methods/fields that match the prefix
                    List myCandidates = getPublicFieldsAndMethods(instance, identifierPrefix)
                    if (myCandidates.size() > 0) {
                        candidates.addAll(myCandidates)
                        return lastDot+1
                    }
                }
            }
        }
        
        // no candidates
        return -1  
    }

    /**
     * Parse a buffer to determine the start index of the groovy identifier
     * @param buffer the buffer to parse
     * @param endingAt the end index with the buffer
     * @return the start index of the identifier, or -1 if the buffer
     * does not contain a valid identifier that ends at endingAt
     */
    int findIdentifierStart(String buffer, int endingAt) {
        // if the string is empty then there is no expression
        if (endingAt == 0)
            return -1
        // if the last character is not valid then there is no expression
        char lastChar = buffer.charAt(endingAt-1)
        if (!Character.isJavaIdentifierPart(lastChar))
            return -1
        // scan backwards until the beginning of the expression is found
        int startIndex = endingAt-1
        while (startIndex > 0 && Character.isJavaIdentifierPart(buffer.charAt(startIndex-1)))
            --startIndex
        return startIndex
    }


    /**
     * Build a list of public fields and methods for an object
     * that match a given prefix.
     * @param instance the object
     * @param prefix the prefix that must be matched
     * @return the list of public methods and fields that begin with the prefix
     */
    List getPublicFieldsAndMethods(Object instance, String prefix) {
        def rv = []
        instance.class.fields.each {
            if (it.name.startsWith(prefix))
                rv << it.name
        }
        instance.class.methods.each {
            if (it.name.startsWith(prefix))
                rv << it.name + (it.parameterTypes.length == 0 ? "()" : "(")
        }
        return rv.sort().unique()
    }

    /**
     * Build a list of variables defined in the shell that
     * match a given prefix.
     * @param prefix the prefix to match
     * @return the list of variables that match the prefix
     */
    List findMatchingVariables(String prefix) {
        def matches = []
        for (String varName in shell.interp.context.variables.keySet())
            if (varName.startsWith(prefix))
                matches << varName
        return matches
    }
}