sort-by with db/-count breaks queries on empty database before transact

[andersmurphy]

UPDATE: THIS PR DOES NOT FIX THE ISSUE

What this PR does

Reverts the sort (short term fix). Feel free to merge it or close it when you work out the caching issue. I tried to delve into the cache stuff but that is going to take me a fair bit longer to get to grips with.

What the issue was

So I’ve encountered this weird issue where if I run a query on an empty database before running a transaction it makes subsequent queries return nil for that attribute.

This should work, but doesn’t.

(comment
  (def schema
    {:transaction/signature
     {:db/unique      :db.unique/identity
      :db/valueType   :db.type/string
      :db/cardinality :db.cardinality/one}
     :transaction/block-time
     {:db/valueType   :db.type/long
      :db/cardinality :db.cardinality/one}})
  
  (def conn
    (d/get-conn "db1" schema
      {:validate-data?    true
       :closed-schema?    true
       :auto-entity-time? true}))

  (d/q '[:find [?block-time ?signature]
         :where
         [?t :transaction/signature ?signature]
         [?t :transaction/block-time ?block-time]]
    @conn)

  (d/transact! conn [{:transaction/signature  "foo"
                      :transaction/block-time 234324324}])

  (d/q '[:find [(max ?bt)]
         :where
         [?t :transaction/block-time ?bt]]
    @conn)
  ;; => nil
  )

This does work.

(comment
  (def schema
    {:transaction/signature
     {:db/unique      :db.unique/identity
      :db/valueType   :db.type/string
      :db/cardinality :db.cardinality/one}
     :transaction/block-time
     {:db/valueType   :db.type/long
      :db/cardinality :db.cardinality/one}})
  
  (def conn
    (d/get-conn "db2" schema
      {:validate-data?    true
       :closed-schema?    true
       :auto-entity-time? true}))
  
  (d/transact! conn [{:transaction/signature  "foo"
                      :transaction/block-time 234324324}])

  (d/q '[:find [?block-time ?signature]
         :where
         [?t :transaction/signature ?signature]
         [?t :transaction/block-time ?block-time]]
    @conn)
  ;; => [234324324 "foo"]

  (d/q '[:find [(max ?bt)]
         :where
         [?t :transaction/block-time ?bt]]
    @conn)
  ;; => [234324324]

  )

So it’s this commit that introduces the issue:

720a7d7

I've also tried reverting this commit on the latest master and it fixes the issue.

So this seems to be caused by this line:

https://github.com/juji-io/datalevin/blob/master/src/datalevin/query.clj#L1468

sort-by (fn [[_ attr _]] (db/-count db [nil attr nil]))

My guess is db/-count is stateful and is now getting called in the reduce and the sort, where as before it was just being called in the reduce. So caching most likely.

[andersmurphy]

Further testing. This DOES NOT fix the issue.

So I've run into a variation of this issue with db.type/string that the above change does not fix.

Seems to be instroduced in 890dc80

So I wonder if there's something more complicated going on here.

[huahaiy]

Thanks. I will investigate more.