mgsmt-parsing-examples-CoNLL-2022.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Running the MGSMT Parser on Examples from the CoNLL-2022 paper.\n",
    "\n",
    "***by Sagar Indurkhya (indurks@mit.edu)***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-10-11T21:18:33.370557Z",
     "start_time": "2022-10-11T21:18:33.367984Z"
    }
   },
   "source": [
    "## Prelude: Imports and Setting up the Parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-10-15T05:01:45.411599Z",
     "start_time": "2022-10-15T05:01:45.394476Z"
    }
   },
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "# Enable the following line to make the Jupyter notebook take up the full width of the browser window.\n",
    "#display(HTML(\"<style>.container { width:100% !important; }</style>\"))\n",
    "# Enable the following line if working on a \"retina\" screen (e.g. a MBP).\n",
    "%config InlineBackend.figure_format = 'retina'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-10-15T05:01:46.982418Z",
     "start_time": "2022-10-15T05:01:45.414496Z"
    }
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import pprint as pp\n",
    "\n",
    "import mgsmt\n",
    "import mgsmt.experiments.parserexperiment\n",
    "from mgsmt.experiments.parserexperiment import ParserExperiment\n",
    "from mgsmt.experiments.parserexperiment import construct_parser_experiment_configurations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-10-15T05:01:46.998956Z",
     "start_time": "2022-10-15T05:01:46.988309Z"
    }
   },
   "outputs": [],
   "source": [
    "def run_parser(inferred_lexicon, \n",
    "               experiment_params, \n",
    "               evaluation_corpus, \n",
    "               extra_lexical_items, \n",
    "               include_LF_constraints=True, \n",
    "               include_PF_constraints=True,\n",
    "               extract_all_parses=False,\n",
    "               max_num_empty_lexical_items=2,\n",
    "               max_num_movements=4,\n",
    "               max_num_head_movements=2):\n",
    "    # Preprocessing: Obtain an experiment configuration for running the parser.\n",
    "    pe_configs = construct_parser_experiment_configurations(inferred_lexicon,\n",
    "                                                            experiment_params,\n",
    "                                                            evaluation_corpus,\n",
    "                                                            extra_lexical_items)\n",
    "    config = pe_configs[0]\n",
    "    config['evaluate_intermediate_steps'] = False\n",
    "    \n",
    "    # Initialize the Parser.\n",
    "    pe = ParserExperiment(params=config, \n",
    "                          other_args={\"display.jupyter-widgets\": True,\n",
    "                                      \"max_num_empty_lexical_items\": max_num_empty_lexical_items,\n",
    "                                      \"max_num_movements\": max_num_movements,\n",
    "                                      \"max_num_head_movements\": max_num_head_movements})\n",
    "    \n",
    "    # Run the parser.\n",
    "    pe.run(include_LF_constraints=include_LF_constraints, \n",
    "           include_PF_constraints=include_PF_constraints,\n",
    "           extract_all_parses=extract_all_parses)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Experiments."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-10-14T17:31:53.950059Z",
     "start_time": "2022-10-14T17:31:53.947549Z"
    }
   },
   "source": [
    "### Run the MGSMT parser on each of seven specified pairing of interface conditions.\n",
    "\n",
    "**Notes:** \n",
    "- The first six examples (i.e. A-F, which correspond to ICs $I_1$ to $I_6$) are run using the same parameter configuration, whereas the seventh and eighth examples (i.e. G-H, which corresponds to ICs $I_7$ and $I_8$) requires additional allocation of both empty categories and instances of head movement. \n",
    "- For each example, first the inputs (i.e. the lexicon and the interface conditions) are displayed, and then the outputs (i.e. the MG derivation and the lexical items used to generate that derivation) are shown. \n",
    "- When examining the display panel showing the derivation, you can view the conventional SVO depiction of the derivation by going to \"Grammar > Options\" and then unchecking \"Display Head Movement Arrows\" and \"Display Phrasal Movement Arrows\" (displaying these arrows can lead GraphViz to draw the derivation tree in an odd manner as it tries to avoid drawing overlapping edges).\n",
    "- Although the parser is designed to process a large sequence of pairs of interface conditions, here we are running the parser on one pair of interface conditions at a time. As a consequence, you will see, and can ignore, the input and output display panels display \"Lexicon (A)\" repeatedly, even when being run on other lexicons, and likewise for the interface conditions displaying \"$I_{0}$\" repeatedly, even though we are processing other interface conditions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-10-15T05:01:47.016397Z",
     "start_time": "2022-10-15T05:01:47.004608Z"
    }
   },
   "outputs": [],
   "source": [
    "def process_example(example_idx, \n",
    "                    example_code,\n",
    "                    extract_all_parses=False,\n",
    "                    max_num_empty_lexical_items=2,\n",
    "                    max_num_movements=4,\n",
    "                    max_num_head_movements=2):\n",
    "    assert 0 <= example_idx < 8\n",
    "    assert example_code in 'ABCDEFGH'\n",
    "    print('-'*29 + f' Processing Example {example_code} ' + '-'*29)\n",
    "    \n",
    "    # Load the lexicon\n",
    "    with open(f'experiment-data/lexicon-{example_code}.json', 'r') as f_in:\n",
    "        lexicon = json.load(f_in)\n",
    "    \n",
    "    # Load the input sequence of paired interface conditions.\n",
    "    with open('experiment-data/corpus-of-interface-conditions.json', 'r') as f_in:\n",
    "        experiment_params = json.load(f_in)\n",
    "        evaluation_corpus = experiment_params['input_sequence'][example_idx:example_idx+1]\n",
    "    \n",
    "    # Run the parser on the specified interface conditions using the given lexicon.\n",
    "    run_parser(lexicon, \n",
    "               experiment_params, \n",
    "               evaluation_corpus, \n",
    "               extra_lexical_items=[],\n",
    "               extract_all_parses=extract_all_parses,\n",
    "               max_num_empty_lexical_items=max_num_empty_lexical_items,\n",
    "               max_num_movements=max_num_movements,\n",
    "               max_num_head_movements=max_num_head_movements)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-10-15T05:49:07.930769Z",
     "start_time": "2022-10-15T05:01:47.019867Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Run the parser on the first six examples, A-F.\n",
    "for i, ex_id in enumerate('ABCDEF'):\n",
    "    print()\n",
    "    process_example(i, ex_id)\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-10-15T06:27:49.412738Z",
     "start_time": "2022-10-15T05:49:07.932951Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Run the parser on the seventh example, G. \n",
    "process_example(6, 'G',\n",
    "                max_num_empty_lexical_items=6,\n",
    "                max_num_movements=6,\n",
    "                max_num_head_movements=4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-10-15T07:15:14.218187Z",
     "start_time": "2022-10-15T06:27:49.417691Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Run the parser on the eighth example, H. \n",
    "process_example(7, 'H',\n",
    "                max_num_empty_lexical_items=6,\n",
    "                max_num_movements=6,\n",
    "                max_num_head_movements=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run the MGSMT parser on an input where only PF interface conditions are specified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-10-15T07:20:12.191320Z",
     "start_time": "2022-10-15T07:15:14.221885Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Load the lexicon\n",
    "with open('experiment-data/lexicon-A-extra-complementizer.json', 'r') as f_in:\n",
    "    lexicon = json.load(f_in)\n",
    "\n",
    "# Load the interface conditions.\n",
    "with open('experiment-data/corpus-of-interface-conditions.json', 'r') as f_in:\n",
    "    experiment_params = json.load(f_in)\n",
    "    evaluation_corpus = experiment_params['input_sequence'][0:1]\n",
    "\n",
    "# Run the parser on the specified interface conditions using the given lexicon.\n",
    "run_parser(lexicon, \n",
    "           experiment_params, \n",
    "           evaluation_corpus, \n",
    "           extra_lexical_items=[],\n",
    "           include_LF_constraints=False, \n",
    "           include_PF_constraints=True,\n",
    "           extract_all_parses=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run the MGSMT parser on an input where only LF interface conditions are specified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-10-15T07:22:33.541719Z",
     "start_time": "2022-10-15T07:20:12.194653Z"
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Load the lexicon\n",
    "with open('experiment-data/lexicon-A-extra-complementizer.json', 'r') as f_in:\n",
    "    lexicon = json.load(f_in)\n",
    "\n",
    "# Load the interface conditions.\n",
    "with open('experiment-data/corpus-of-interface-conditions.json', 'r') as f_in:\n",
    "    experiment_params = json.load(f_in)\n",
    "    evaluation_corpus = experiment_params['input_sequence'][0:1]\n",
    "\n",
    "# Run the parser on the specified interface conditions using the given lexicon.\n",
    "run_parser(lexicon, \n",
    "           experiment_params, \n",
    "           evaluation_corpus, \n",
    "           extra_lexical_items=[],\n",
    "           include_LF_constraints=True, \n",
    "           include_PF_constraints=False,\n",
    "           extract_all_parses=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-10-11T21:12:25.835856Z",
     "start_time": "2022-10-11T21:12:25.833252Z"
    }
   },
   "source": [
    "# End of Notebook"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}