how to handle incomplete structure token predictions?

[chadrick-kwag]

I wonder how you guys handled incomplete sequences. Here's an example

<thead><tr><td></td><td></td>**<td>**</tr></thead>...

In the first phase when only training the structure decoder, the model gives predictions that are partially incomplete like this.

In the paper there is no specific mention about this problem and how it was handled.

These imperfections affect when calculating TEDS and also moving on to second phase of the training where training the cell decoder will need to be run at the same time. I expect these imperfections will make cell decoder GT assignment confusing.

Can you share on how these imperfections were handled?

[Sunnycheey]

Also want to know how to deal whit this kind of exception.

[zhxgj]

Currently we did not have any post processing to the model output. HTML is tolerant to some of these errors and we do not think we can do much better than HTML. I would suggest to parse the incomplete sequence with something like lxml, and see what the tree looks like.