Home
Active Documentation is designed to transmit information from an IntelliJ IDE to a custom web application. The user runs the plugin in their IDE and then clicks connect on the web client. This will establish a WebSocket connection between the web client and a WebSocket server that is created when the plugin initilializes.
Due to the nature of WebSockets, if the user clicks to a different link on the web client page or reloads the page, the connection will be lost. As a result, the web client needs to be developed as a single page website so that the connection is maintained at all times.
Messages transmitted via WebSockets are stringified JSON objects. The objects have four member variables.
1. "source" - A string that names the source of the message
2. "destination" - A string that names the intended destination of the message
3. "command" - The type of operation to perform on the "data"
4. "data" - The data to be processed (can be any type of value)
Both the plugin (WebSocket server) and web client (WebSocket client) are event-based, meaning they perform various actions when listeners trigger. The key listener here is the one associated with the WebSocket connection.
After the plugin first starts the server, it calculates several important data structures, and puts them into a queue of messages that are to be sent once a WebSocket client connects to the server. The structures of each of these can be viewed by throwing in some System.out.println() statements in the source code and printing out their stringified versions (as they are all JSON based) and throwing them into a JSON visualizer like the one found at jsonviewer.stack.hu. These data structures are:
1. "The Project Directory Hierarchy" - A tree structure (with JSON Objects as nodes) that contains the hierarchy of the user's IntelliJ project on the hard disk. Each node in the tree has two attributes. The first is **properties**, which is a JSONObject with a bunch of member variables including "name", "canonicalPath", "isDirectory", etc. The second is **children**, which is a JSONArray of JSONObjects that are other nodes in the hierarchy. As their parent does, these nodes each have their own "children" and "properties" attributes. If a node is not a directory and IntelliJ can handle the filetype, then we can include a property **ast** representing the file's abstact syntax tree as one of the properties in the "properties" attribute of the node.
2. "Abstract Syntax Trees" - A tree structure (with JSON Objects as nodes) representing the AST of a file whose filetype IntelliJ has support for. Each node in the AST has a JSONArray **children** attribute and a JSONObject **properties** attribute. The "properties" attribute contains all sorts of useful values like the text offset of this node in the containing file, text, etc. When users write rules, they will either directly work with this AST or work with some higher level data structures that operate as an abstraction over the AST, allowing users to more quickly write rules. Building these higher level structures is a key part of the larger end goal for Active Documentation.
3. "Project Class Hierarchy" - The JSON object that acts like a Map, where the key is a String with a canonical class name and the corresponding value being a JSON object with a bunch of properties about that class (like superclasses, interfaces, etc). The classes included here are all the classes used by the user in their project. If they define a Dog java class, for instance, the class table would be updated to include Dog and java.lang.Object.
Once this all happens, the user then clicks "Connect" on the web client, establishing the WebSocket connection and allowing these data structures in the message queue to be sent over to the web client. Inside websocket_response_handler.js, there is the following code:
$("uriForm").observe("submit", function(e) {
e.stop();
ws = new WebSocket($F("uri"));
ws.onopen = function() {
log("[WebSocket#onopen]\n");
}
ws.onmessage = function(e) {
// log("[WebSocket#onmessage] Message: '" + e.data + "'\n"); // CONTROLS WHETHER MESSAGES ARE DISPLAYED OR NOT\
var message = JSON.parse(e.data);
console.log(message.command);
if(message.command === "INITIAL_PROJECT_HIERARCHY"){
projectHierarchy = message.data;
addParentPropertyToNodes(projectHierarchy);
console.log("initial project hierarchy added");
}else if(message.command === "UPDATE_RULE_TABLE_AND_CONTAINER"){
console.log("hi");
console.log(message.data.text);
eval(message.data.text);
}else if(message.command === "UPDATE_CODE_IN_FILE"){ ...
By looking at the "command" attribute of the incoming messages, the web-client can respond in the appropriate way to the data it is recieving. In this case, the web-client stores these initial data structures a global variables for all the JavaScript scripts to access. These variables are named projectClassTable and projectHierarchy. Another way to better understand the organization of these datastructures would be to use a web browser's developer tools and examine these global variables from there.
If you look at the global variables defined at the top of precomputedclasstable.js, you will notice that there are two similarly named variables, "classTable" and "projectClassTable". What is the different between these two? The project class table is essentially a map (implemented as a JSON Object) that hold the relationships between the user defined classes. For instance, if the user made "Animal", "Dog", and "Cat" classes, then Animal, Dog, Cat, and java.lang.Object would be included in this table in additions to all the other classes referenced in these classes. For instance, if a Dog has a property that is of type List, then the user defined Breed would be included in the table and so would java.util.List.
On the other hand, the "classTable" contains these relationships for the IntelliJ PSI classes. PSI stands for "Project Structure Interface" and it's what IntelliJ uses to represent code elements. For instance, there are classes like "PsiClass", "PsiMethod", "PsiVariable", "PsiExpression", etc. If someone wanted to find all the expressions in their code, they could traverse the ASTs they have and look for all PsiExpressions. However, there are more subclasses of PsiExpression. For instance, PsiArrayAccessExpression, PsiArrayInitializerExpression, PsiAssignmentExpression, PsiCallExpression, PsiConditionalExpression, PsiFunctionalExpression, and more are all subclasses of PsiExpression. When the user queries for all PsiExpressions, we want them to get all the expressions of these various types. classTable holds the relationships between these classes. The key to remember is that these relationships are not computed in real time. They are precomputed. The way these were precomputed were using a special project and setting the boolean "recomputePsiClassTable" in the PsiPreCompEngine class to true. To find that special project please look here. The file "t.txt" in that zip file is basically made by copying Psi classes from the intellij-community repo on GitHub like this:
Take the pasted text, put it in t.txt and run the python script, which creates t.out. Then copy the lines from t.out into TWF.java in the special project and run the Active Documentation plugin with the boolean recomputePsiClassTable in the PsiPreCompEngine class to true. Run the plugin with the web client and look for the text "FINDME" in the console output of the Active Docs plugin. Right below, you will find the stringified JSON class table. The code below shows the code (in GrepServerToolWindowFactory.java) that prints out the class table:
public static JsonObject generateJavaASTAsJSON(PsiJavaFile psiJavaFile) {
JsonObject root = new JsonObject();
PsiJavaVisitor pjv = new PsiJavaVisitor(initialClassTable);
pjv.visit(psiJavaFile, root);
/*
* This is really important to the functioning of the Web Client.
* Since the client needs to know the relationship between all the
* Psi- classes in order to run queries like "find all PsiExpressions"
* (PsiExpression is a superclass of PsiArrayAccessExpression,
* PsiArrayInitializerExpression, PsiAssignmentExpression,
* PsiCallExpression, etc.). There are a lot of classes here to handle.
* There is a seperate project to use in order to generate a table of
* these relationships. That table is then put into precomputedclasstable.js
* */
if (PsiPreCompEngine.recomputePsiClassTable) { // this is a boolean that should be set to true only if trying to recompute the precomputed class table for psi elements and if working with a special project which is provided as a .zip file on the Github page.
System.out.println("FINDME");
System.out.println(PsiPreCompEngine.ct); // the class table
}
return root;
}
Coming back to the example before about querying for PsiExpressions, use the function psiInstanceOf to check instance of relationships for Psi- classes in the web-client.
This tool has some miscellaneous operations to traverse ASTs and the Project Hierarchy. Some operations including returning the first node (with a DFS traversal) that returns true when passed through a function, finding all files of a certain type, determining if one of the classes in the user's project is the instance of another, etc.
This tool executes all the rules defined by the global variables RuleContainer and ruleTable. RuleContainer is a class that holds all the rule functions. ruleTable gives headers and descriptions to each of those rules. The websocket_response_handler makes sure that the rules are up to date by looking for messages with the command "UPDATE_RULE_TABLE_AND_CONTAINER". This message is sent when the file ruleJson.txt is modified by the user. The ruleJson.txt can live anywhere in the user's project as long as there is only one instance of ruleJson.txt in the project. The file change listener in IntelliJ works really fast if the changes to ruleJson.txt are made in IntelliJ. They are delayed a lot (20-30 seconds) if done from an external application. ruleJson.txt is basically just two commands which set the values of the global variables RuleContainer and ruleTable that are run with an eval() statemetn in websocket_response_handler.js.