Lines appended to QuickFix list are not seen (e.g., from async Grepper)

[drmikehenry]

The QuickFix Reflector relies on BufReadPost quickfix autocommands to signal changes to the QuickFix list. Unfortunately, these autocommands are not generated when appending to an existing non-empty list. This causes no problems for commands like :grep, where the entire output is written into the list at once before generating BufReadPost; but when the output is written piecemeal using :caddexpr, such as by an asynchronous search plugin like Grepper (https://github.com/mhinz/vim-grepper), only the first set of lines to be written will generate BufReadPost, leaving QuickFix Reflector unaware of any subsequent changes. The problem does not occur when the QuickFix list window is not open until after the list is finalized, and the problem can be fixed by closing and reopening the QuickFix window.

Unfortunately, I don't see a good way to correct this in QuickFix Reflector itself. I'm currently experimenting with a patch in the Grepper plugin to re-write the entire QuickFix list upon completion of the search using setqflist(getqflist()) (see patch below). This generates one final BufReadPost autocommand to avoid the above problem. This work-around has a drawback for slow searches, however; if the user has time to interact with the partial results in the QuickFix window, he may find it unsettling when the call to setqflist() resets the list and scrolls the QuickFix window back to the start. Also, a similar fix would be required for every other source of QuickFix data that gets delivered in pieces.

In any event, it might be worth adding something the QuickFix Reflector's documentation describing this problem and some possible work-arounds. I believe this is the underlying problem that @Konfekt was seeing in #22.

The problem can be demonstrated without Grepper as follows:

• Create file f.txt with these two lines:

one
two

• Edit the file via vim f.txt.

• Execute these Vim commands:

  :copen
  :cgetexpr 'f.txt:1: one'
  :caddexpr 'f.txt:2: two'
  :%s/one/three/g
  :w

The :w triggers a long series of exceptions caused by adding
a line to the list via :caddexpr. To demonstrate the
work-around, execute :call setqflist(getqflist()) after the
:caddexpr and the :w will then work.

Here is the grepper patch to work around this issue:

diff --git a/bundle/grepper/plugin/grepper.vim b/bundle/grepper/plugin/grepper.vim
index ed3e20d..a96bfd3 100644
--- a/bundle/grepper/plugin/grepper.vim
+++ b/bundle/grepper/plugin/grepper.vim
@@ -809,8 +809,10 @@ function! s:finish_up(flags)
           \ ? cmdline
           \ : {'title': cmdline, 'context': {'query': @/}}
     if qf
+      call setqflist(list)
       call setqflist(list, a:flags.append ? 'a' : 'r', attrs)
     else
+      call setloclist(0, list)
       call setloclist(0, list, a:flags.append ? 'a' : 'r', attrs)
     endif
   catch /E118/

[stefandtw]

In theory it might be possible to add support for asynchronously added quickfix entries to quickfix-reflector. Some thoughts on that:

• QuickFixCmdPre and QuickFixCmdPost could perhaps be used to watch for new entries in the quickfix list and buffer. To find the new entries, the getqflist() objects and qf buffer differences between the Pre and Post events could be compared. Unfortunately the events are probably not tied to the quickfix or location list buffer.
• Needs to handle some tricky situations, like when quickfix buffer is edited, but not saved, before new entries are added.
• Seems like a lot of work. Personally not a priority for me.

You already suggested a workaround. That should work. The scrolling problem can be avoided like this:

let qfWinView = winsaveview()
call setqflist(getqflist())
call winrestview(qfWinView)

quickfix-reflector does the same thing when you save the qf buffer.

I'll add a note to the documentation.

[drmikehenry]

Thanks for the idea to save and restore the winview; that works well. I've incorporated it into my grepper patch. Grepper uses noautocmd caddexpr, so it would still need a patch for QuickFixCmdPre/QuickFixCmdPost to work. I'm not sure if the noautocmd is used to improve performance or some other reason, though. I'd considered whether QuickFix Reflector might be able to wait until the user tries to write any modified lines from the QuickFix list, then use getqflist() along with some processing to get the list of original lines. I didn't look into that to see if it would be difficult, but it would avoid the need to cache the original QuickFix contents before the user gets a chance to modify them. But the current work-around is sufficient for my current needs; thanks!