JVM: Revamp prompt structure (#425)

This PR revamps the JVM prompt template structure by grouping and
organising different method properties into subtags to allow the LLM
model to pick up the necessary information in a non-confusion way.

---------

Signed-off-by: Arthur Chan <[email protected]>

llm_toolkit/prompt_builder.py

@@ -662,11 +662,9 @@ def _format_target(self, signature: str) -> str:
        method and format it for the prompts creation.
     """
     if '<init>' in signature:
-      target = self._format_target_constructor(signature)
-    else:
-      target = self._format_target_method(signature)
+      return self._format_target_constructor(signature)
 
-    return target.replace('{EXCEPTIONS}', self._format_exceptions())
+    return self._format_target_method(signature)
 
   def _format_requirement(self, signature: str) -> str:
     """Formats a requirement based on the prompt template."""
@@ -714,7 +712,7 @@ def _format_arguments(self) -> str:
 
       argument_descriptions.append(argument)
 
-    return '\n'.join(argument_descriptions)
+    return '<arguments>' + '\n'.join(argument_descriptions) + '</arguments>'
 
   def _format_source_reference(self, signature: str) -> Tuple[str, str]:
     """Formats the source code reference for this target."""
@@ -740,6 +738,7 @@ def _format_problem(self, signature: str) -> str:
                               self._format_requirement(signature))
     problem = problem.replace('{DATA_MAPPING}', self._format_data_filler())
     problem = problem.replace('{ARGUMENTS}', self._format_arguments())
+    problem = problem.replace('{EXCEPTIONS}', self._format_exceptions())
 
     self_source, cross_source = self._format_source_reference(signature)
     problem = problem.replace('{SELF_SOURCE}', self_source)

prompts/template_xml/jvm_base.txt

@@ -4,4 +4,4 @@ The <target> tag contains information of the target method to invoke.
 The <arguments> tag contains information of each of the target method arguments.
 The <exceptions> tag contains a list of exceptions thrown by the target method that you MUST catch.
 The <constructor> tag contains constructor or method call details you MUST use to create the needed object before calling the target method.
-The <requirement> tag contains additional requirements that you MUST follow for this code generation.
+The <requirements> tag contains additional requirements that you MUST follow for this code generation.

prompts/template_xml/jvm_problem.txt

@@ -1,6 +1,14 @@
 <task>
 Your goal is to write a fuzzing harness for the provided method signature using Jazzer framework from Code Intellengence. It is important that the provided solution compiles and actually calls the function specified by the method signature:
 {TARGET}
+{ARGUMENTS}
+{EXCEPTIONS}
+Here is the source code of the target constructor for reference.
+<code>
+{SELF_SOURCE}
+</code>
+Here is a list of source codes of methods that directly invoke the target consturctor for reference.
+{CROSS_SOURCE}
 {REQUIREMENTS}
 {DATA_MAPPING}
 </task>

prompts/template_xml/jvm_problem_constructor.txt

@@ -7,15 +7,4 @@ The constructor signature follows the format of <code>[Full qualified name of th
 For example, for the constructor of class <code>Test</code> of package <code>org.test</code> which takes in a single integer would have the following signature:
 <code>[org.test.Test].<init>(int)</code>
 The target method is belonging to the Java project {PROJECT_NAME} ({PROJECT_URL}).
-Here is the list of arguments of the target constructor with descriptions.
-<arguments>
-{ARGUMENTS}
-</arguments>
-{EXCEPTIONS}
-Here is the source code of the target constructor for reference.
-<code>
-{SELF_SOURCE}
-</code>
-Here is a list of source codes of methods that directly invoke the target consturctor for reference.
-{CROSS_SOURCE}
 </target>

prompts/template_xml/jvm_problem_method.txt

@@ -6,15 +6,4 @@ The method signature follows the format of <code>[Full qualified name of the cla
 For example, for a method <code>test</code> in class <code>Test</code> of package <code>org.test</code> which takes in a single integer would have the following method signature:
 <code>[org.test.Test].test(int)</code>
 The target method is belonging to the Java project {PROJECT_NAME} ({PROJECT_URL}).
-Here is the list of arguments of the target method with descriptions.
-<arguments>
-{ARGUMENTS}
-</arguments>
-{EXCEPTIONS}
-Here is the source code of the target method for reference.
-<code>
-{SELF_SOURCE}
-</code>
-Here is a list of source codes of methods that directly invoke the target method for reference.
-{CROSS_SOURCE}
 </target>