MapQuery error in Seurat reference mapping #5257

ceegilllns · 2021-11-03T08:59:59Z

I found reference mapping very useful in single cell analysis using Azimuth app.
To deal with large data, I tried MapQuery function but failed with the following error message,

Error in check_graph(graph, n_vertices, n_neighbors) : 
  ncol(idx) == expected_cols is not TRUE

When I tried TransferData, IntegrateEmbeddings, and ProjectUMAP (or RunUMAP after FindNeighbors) step by step, the error occured when running ProjectUMAP(or RunUMAP).

I also tried reference mapping using (copy-pasted) the example code provided in "Seurat Multimodal reference mapping example 1 (Mapping human peripheral blood cells)" but same error occurred. I used Seurat version 4.0.5 and downloaded most recent reference data.

library(Seurat)
library(SeuratDisk)
library(ggplot2)
library(patchwork)

reference <- LoadH5Seurat("~/Downloads/pbmc_multimodal.h5seurat")

library(SeuratData)
InstallData('pbmc3k')

pbmc3k <- SCTransform(pbmc3k, verbose = FALSE)

anchors <- FindTransferAnchors(
  reference = reference,
  query = pbmc3k,
  normalization.method = "SCT",
  reference.reduction = "spca",
  dims = 1:50
)

pbmc3k <- MapQuery(
  anchorset = anchors,
  query = pbmc3k,
  reference = reference,
  refdata = list(
    celltype.l1 = "celltype.l1",
    celltype.l2 = "celltype.l2",
    predicted_ADT = "ADT"
  ),
  reference.reduction = "spca", 
  reduction.model = "wnn.umap"
)

Does anyone has any suggestions to fix the problem?

Here is the full message,


Finding integration vectors
Finding integration vector weights
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Predicting cell labels
Warning: Keys should be one or more alphanumeric characters followed by an underscore, setting key from predictionscorecelltype.l1_ to predictionscorecelltypel1_
Predicting cell labels
Warning: Feature names cannot have underscores ('_'), replacing with dashes ('-')
Warning: Keys should be one or more alphanumeric characters followed by an underscore, setting key from predictionscorecelltype.l2_ to predictionscorecelltypel2_
Transfering 228 features onto reference data
Warning: Keys should be one or more alphanumeric characters followed by an underscore, setting key from predicted_ADT_ to predictedADT_
  |                                                  | 0 % ~calculating  Integrating dataset 2 with reference dataset
Finding integration vectors
Integrating data
Warning: Keys should be one or more alphanumeric characters followed by an underscore, setting key from ref.spca_ to refspca_
  |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=01s  
Warning: Keys should be one or more alphanumeric characters followed by an underscore, setting key from ref.spca_ to refspca_
Warning: All keys should be one or more alphanumeric characters followed by an underscore '_', setting key to refspca_
Computing nearest neighbors
Warning: The default method for RunUMAP has changed from calling Python UMAP via reticulate to the R-native UWOT using the cosine metric
To use Python UMAP via reticulate, set umap.method to 'umap-learn' and metric to 'correlation'
This message will be shown once per session
Running UMAP projection
Error in check_graph(graph, n_vertices, n_neighbors) : 
  ncol(idx) == expected_cols is not TRUE

The text was updated successfully, but these errors were encountered:

yuhanH · 2021-11-04T21:37:35Z

Could you post sessionInfo() output?

ceegilllns · 2021-11-05T05:32:14Z

> sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.6 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] pbmc3k.SeuratData_3.1.4 SeuratData_0.2.1        patchwork_1.1.1         ggplot2_3.3.5           SeuratDisk_0.0.0.9019   SeuratObject_4.0.2     
[7] Seurat_4.0.5           

loaded via a namespace (and not attached):
  [1] nlme_3.1-149          matrixStats_0.61.0    spatstat.sparse_2.0-0 bit64_4.0.5           RcppAnnoy_0.0.19      RColorBrewer_1.1-2    httr_1.4.2           
  [8] sctransform_0.3.2     tools_4.0.3           utf8_1.2.2            R6_2.5.1              irlba_2.3.3           rpart_4.1-15          KernSmooth_2.23-17   
 [15] uwot_0.1.10.9000      mgcv_1.8-33           DBI_1.1.1             lazyeval_0.2.2        colorspace_2.0-2      withr_2.4.2           tidyselect_1.1.1     
 [22] gridExtra_2.3         bit_4.0.4             compiler_4.0.3        cli_3.1.0             hdf5r_1.3.4           plotly_4.10.0         scales_1.1.1         
 [29] lmtest_0.9-38         spatstat.data_2.1-0   ggridges_0.5.3        pbapply_1.5-0         rappdirs_0.3.3        goftest_1.2-3         stringr_1.4.0        
 [36] digest_0.6.28         spatstat.utils_2.2-0  pkgconfig_2.0.3       htmltools_0.5.2       parallelly_1.28.1     fastmap_1.1.0         htmlwidgets_1.5.4    
 [43] rlang_0.4.12          rstudioapi_0.13       shiny_1.7.1           generics_0.1.1        zoo_1.8-9             jsonlite_1.7.2        ica_1.0-2            
 [50] dplyr_1.0.7           magrittr_2.0.1        Matrix_1.3-4          Rcpp_1.0.7            munsell_0.5.0         fansi_0.5.0           abind_1.4-5          
 [57] reticulate_1.22       lifecycle_1.0.1       stringi_1.7.5         MASS_7.3-53           Rtsne_0.15            plyr_1.8.6            grid_4.0.3           
 [64] parallel_4.0.3        listenv_0.8.0         promises_1.2.0.1      ggrepel_0.9.1         crayon_1.4.2          deldir_1.0-6          miniUI_0.1.1.1       
 [71] lattice_0.20-41       cowplot_1.1.1         splines_4.0.3         tensor_1.5            pillar_1.6.4          igraph_1.2.7          spatstat.geom_2.3-0  
 [78] future.apply_1.8.1    reshape2_1.4.4        codetools_0.2-16      leiden_0.3.9          glue_1.4.2            data.table_1.14.2     BiocManager_1.30.16  
 [85] png_0.1-7             vctrs_0.3.8           httpuv_1.6.3          gtable_0.3.0          RANN_2.6.1            purrr_0.3.4           spatstat.core_2.3-1  
 [92] polyclip_1.10-0       tidyr_1.1.4           scattermore_0.7       future_1.23.0         assertthat_0.2.1      xfun_0.27             mime_0.12            
 [99] xtable_1.8-4          later_1.3.0           survival_3.2-7        viridisLite_0.4.0     tibble_3.1.5          tinytex_0.34          cluster_2.1.0        
[106] globals_0.14.0        fitdistrplus_1.1-6    ellipsis_0.3.2        ROCR_1.0-11

Thanks in advance!

yuhanH · 2021-11-19T18:11:07Z

HI, @ceegilllns
sorry for the delay. The package version looks correct.
Could you try to run MapQuery without setting reduction.model?

pbmc3k <- MapQuery(
  anchorset = anchors,
  query = pbmc3k,
  reference = reference,
  refdata = list(
    celltype.l1 = "celltype.l1",
    celltype.l2 = "celltype.l2",
    predicted_ADT = "ADT"
  ),
  reference.reduction = "spca"
)

If it works, we can focus on the ProjectUMAP step.

ceegilllns · 2021-11-21T05:07:30Z

Hi yuhanH,

I ran the code without setting reduction.model and the function worked without error.

Finding integration vectors
Finding integration vector weights
0%   10   20   30   40   50   60   70   80   90   100%
[----|----|----|----|----|----|----|----|----|----|
**************************************************|
Predicting cell labels
Warning: Keys should be one or more alphanumeric characters followed by an underscore, setting key from predictionscorecelltype.l1_ to predictionscorecelltypel1_
Predicting cell labels
Warning: Feature names cannot have underscores ('_'), replacing with dashes ('-')
Warning: Keys should be one or more alphanumeric characters followed by an underscore, setting key from predictionscorecelltype.l2_ to predictionscorecelltypel2_
Transfering 228 features onto reference data
Warning: Keys should be one or more alphanumeric characters followed by an underscore, setting key from predicted_ADT_ to predictedADT_
  |                                                  | 0 % ~calculating  Integrating dataset 2 with reference dataset
Finding integration vectors
Integrating data
Warning: Keys should be one or more alphanumeric characters followed by an underscore, setting key from ref.spca_ to refspca_
  |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=01s  
Warning: Keys should be one or more alphanumeric characters followed by an underscore, setting key from ref.spca_ to refspca_
Warning: All keys should be one or more alphanumeric characters followed by an underscore '_', setting key to refspca_

And as you may have expected, MapQuery projected pbmc3k data onto ref.spca.

> pbmc3k@reductions
$ref.spca
A dimensional reduction object with key refspca_ 
 Number of dimensions: 50 
 Projected dimensional reduction calculated:  FALSE 
 Jackstraw run: FALSE 
 Computed using assay: SCT

yuhanH · 2021-11-22T03:50:23Z

Great. That means the error happen in the umap projection step. Could you have a try on the below steps?

query.nn <- Seurat::NNHelper(query = pbmc3k[['ref.spca']]@cell.embeddings , 
                     data =  reference[['spca']]@cell.embeddings , 
                     method = "annoy",
                     k = 20, 
                     metric = "cosine"
)
pbmc3k[['ref.umap']]  <- RunUMAP(object = query.nn,  reduction.model = reference[[ "wnn.umap" ]] )

ceegilllns · 2021-11-25T09:15:11Z

I tried 'Seurat::NNHelper' but failed with following message:

 'NNHelper' is not an exported object from 'namespace:Seurat'

uninstallation and reinstallation of Seurat did not resolve this problem.
So, I tried the codes again after running NNHelper, AnnoyNN, AnnoyBuildIndex, and Annoysearch functions in clustering.R
and loading libraries future and future.apply.

And the results are as follows:

> query.nn <- NNHelper(query = pbmc3k[['ref.spca']]@cell.embeddings , 
+                              data =  reference[['spca']]@cell.embeddings , 
+                              method = "annoy",
+                              k = 20, 
+                              metric = "cosine"
+ )
> pbmc3k[['ref.umap']]  <- RunUMAP(object = query.nn,  reduction.model = reference[[ "wnn.umap" ]] )
Warning: The default method for RunUMAP has changed from calling Python UMAP via reticulate to the R-native UWOT using the cosine metric
To use Python UMAP via reticulate, set umap.method to 'umap-learn' and metric to 'correlation'
This message will be shown once per session
Running UMAP projection
Error in check_graph(graph, n_vertices, n_neighbors) : 
  ncol(idx) == expected_cols is not TRUE

> sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.6 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] future.apply_1.8.1      future_1.23.0           pbmc3k.SeuratData_3.1.4 SeuratData_0.2.1        patchwork_1.1.1        
[6] ggplot2_3.3.5           SeuratDisk_0.0.0.9019   SeuratObject_4.0.2      Seurat_4.0.5           

loaded via a namespace (and not attached):
  [1] nlme_3.1-149          matrixStats_0.61.0    spatstat.sparse_2.0-0 bit64_4.0.5           RcppAnnoy_0.0.19      RColorBrewer_1.1-2   
  [7] httr_1.4.2            sctransform_0.3.2     tools_4.0.3           utf8_1.2.2            R6_2.5.1              irlba_2.3.3          
 [13] rpart_4.1-15          KernSmooth_2.23-17    uwot_0.1.10.9000      mgcv_1.8-33           DBI_1.1.1             lazyeval_0.2.2       
 [19] colorspace_2.0-2      withr_2.4.2           tidyselect_1.1.1      gridExtra_2.3         bit_4.0.4             compiler_4.0.3       
 [25] cli_3.1.0             hdf5r_1.3.4           plotly_4.10.0         scales_1.1.1          lmtest_0.9-38         spatstat.data_2.1-0  
 [31] ggridges_0.5.3        pbapply_1.5-0         rappdirs_0.3.3        goftest_1.2-3         stringr_1.4.0         digest_0.6.28        
 [37] spatstat.utils_2.2-0  pkgconfig_2.0.3       htmltools_0.5.2       parallelly_1.28.1     fastmap_1.1.0         htmlwidgets_1.5.4    
 [43] rlang_0.4.12          rstudioapi_0.13       shiny_1.7.1           generics_0.1.1        zoo_1.8-9             jsonlite_1.7.2       
 [49] ica_1.0-2             dplyr_1.0.7           magrittr_2.0.1        Matrix_1.3-4          Rcpp_1.0.7            munsell_0.5.0        
 [55] fansi_0.5.0           abind_1.4-5           reticulate_1.22       lifecycle_1.0.1       stringi_1.7.5         MASS_7.3-53          
 [61] Rtsne_0.15            plyr_1.8.6            grid_4.0.3            parallel_4.0.3        listenv_0.8.0         promises_1.2.0.1     
 [67] ggrepel_0.9.1         crayon_1.4.2          deldir_1.0-6          miniUI_0.1.1.1        lattice_0.20-41       cowplot_1.1.1        
 [73] splines_4.0.3         tensor_1.5            pillar_1.6.4          igraph_1.2.7          spatstat.geom_2.3-0   reshape2_1.4.4       
 [79] codetools_0.2-16      leiden_0.3.9          glue_1.4.2            data.table_1.14.2     BiocManager_1.30.16   png_0.1-7            
 [85] vctrs_0.3.8           httpuv_1.6.3          gtable_0.3.0          RANN_2.6.1            purrr_0.3.4           spatstat.core_2.3-1  
 [91] polyclip_1.10-0       tidyr_1.1.4           scattermore_0.7       assertthat_0.2.1      xfun_0.27             mime_0.12            
 [97] xtable_1.8-4          later_1.3.0           survival_3.2-7        viridisLite_0.4.0     tibble_3.1.5          tinytex_0.34         
[103] cluster_2.1.0         globals_0.14.0        fitdistrplus_1.1-6    ellipsis_0.3.2        ROCR_1.0-11

pebonte · 2021-12-02T10:44:30Z

Hello, I have the exact same issue that you have, same error message. I also have the same message with NNHelper (to run it I think you needed it to run Seurat:::NNHelper instead of Seurat::NNHelper )

I'm also using Seurat_4.0.5.

Did you manage to find a solution ?

yuhanH · 2021-12-02T12:56:04Z

hi @ceegilllns
sorry for the delay... Could you save query.nn as a rds file and email it to me? Thanks.
My email is [email protected]

Hi, @pebonte
Do you have meet this error when you run MapQuery? Or just the error from NNHelper?
NNHelper is an internal function, so it may not work properly by using Seurat:::NNHelper. We can make NNHelper as an exported function in the next version release.

Error in check_graph(graph, n_vertices, n_neighbors) : 
  ncol(idx) == expected_cols is not TRUE

pascaltimshel · 2021-12-02T13:46:24Z

@yuhanH I have the same problem:

#> Running UMAP projection
#> Error in check_graph(graph, n_vertices, n_neighbors): ncol(idx) == expected_cols is not TRUE

and my code works if I remove reduction.model = "umap" from MapQuery()

Full code (using code from your tutorial: https://satijalab.org/seurat/articles/integration_mapping.html)

library(tidyverse)
library(Seurat)
#> Registered S3 method overwritten by 'spatstat.geom':
#>   method     from
#>   print.boxx cli
#> Attaching SeuratObject
library(SeuratData)
#> Warning in if (is.na(desc)) {: the condition has length > 1 and only the first
#> element will be used

#> Warning in if (is.na(desc)) {: the condition has length > 1 and only the first
#> element will be used

#> Warning in if (is.na(desc)) {: the condition has length > 1 and only the first
#> element will be used

#> Warning in if (is.na(desc)) {: the condition has length > 1 and only the first
#> element will be used

#> Warning in if (is.na(desc)) {: the condition has length > 1 and only the first
#> element will be used

#> Warning in if (is.na(desc)) {: the condition has length > 1 and only the first
#> element will be used

#> Warning in if (is.na(desc)) {: the condition has length > 1 and only the first
#> element will be used

#> Warning in if (is.na(desc)) {: the condition has length > 1 and only the first
#> element will be used

#> Warning in if (is.na(desc)) {: the condition has length > 1 and only the first
#> element will be used

#> Warning in if (is.na(desc)) {: the condition has length > 1 and only the first
#> element will be used

#> Warning in if (is.na(desc)) {: the condition has length > 1 and only the first
#> element will be used

#> Warning in if (is.na(desc)) {: the condition has length > 1 and only the first
#> element will be used
#> ── Installed datasets ───────────────────────────────────── SeuratData v0.2.1 ──
#> ✓ panc8 3.0.2
#> ────────────────────────────────────── Key ─────────────────────────────────────
#> ✓ Dataset loaded successfully
#> > Dataset built with a newer version of Seurat than installed
#> ❓ Unknown version of Seurat installed

library(patchwork)

data("panc8") # InstallData("panc8")
dataset.list <- SplitObject(panc8, split.by = "tech")
dataset.list <- dataset.list[c("fluidigmc1", "smartseq2")]
for (i in 1:length(dataset.list)) {
  dataset.list[[i]] <- NormalizeData(dataset.list[[i]], verbose = FALSE)
  dataset.list[[i]] <- FindVariableFeatures(dataset.list[[i]], selection.method = "vst", nfeatures = 2000,
                                            verbose = FALSE)
}
seurat_obj.query <- dataset.list[["fluidigmc1"]]
seurat_obj.ref <- dataset.list[["smartseq2"]]
seurat_obj.ref <- ScaleData(seurat_obj.ref, verbose = FALSE)
seurat_obj.ref <- RunPCA(seurat_obj.ref, npcs = 30, verbose = FALSE) # must run PCA before FindTransferAnchors()
anchors_obj <- FindTransferAnchors(reference = seurat_obj.ref, 
                                   query = seurat_obj.query,
                                   dims = 1:30, 
                                   reference.reduction = "pca")
#> Projecting cell embeddings
#> Finding neighborhoods
#> Finding anchors
#>  Found 770 anchors
#> Filtering anchors
#>  Retained 732 anchors



seurat_obj.ref <- RunUMAP(seurat_obj.ref, dims = 1:30, reduction = "pca", return.model = TRUE)
#> Warning: The default method for RunUMAP has changed from calling Python UMAP via reticulate to the R-native UWOT using the cosine metric
#> To use Python UMAP via reticulate, set umap.method to 'umap-learn' and metric to 'correlation'
#> This message will be shown once per session
#> UMAP will return its model
#> 14:33:05 UMAP embedding parameters a = 0.9922 b = 1.112
#> 14:33:05 Read 2394 rows and found 30 numeric columns
#> 14:33:05 Using Annoy for neighbor search, n_neighbors = 30
#> 14:33:05 Building Annoy index with metric = cosine, n_trees = 50
#> 0%   10   20   30   40   50   60   70   80   90   100%
#> [----|----|----|----|----|----|----|----|----|----|
#> **************************************************|
#> 14:33:05 Writing NN index file to temp file /tmp/RtmpdyN2gw/file16db777cc3d44
#> 14:33:05 Searching Annoy index using 1 thread, search_k = 3000
#> 14:33:06 Annoy recall = 100%
#> 14:33:07 Commencing smooth kNN distance calibration using 1 thread
#> 14:33:07 Initializing from normalized Laplacian + noise
#> 14:33:07 Commencing optimization for 500 epochs, with 96376 positive edges
#> 14:33:11 Optimization finished
seurat_obj.query <- MapQuery(anchorset = anchors_obj,
                             reference = seurat_obj.ref,
                             query = seurat_obj.query,
                             refdata = list(celltype = "celltype"),
                             reference.reduction = "pca",
                             reduction.model = "umap")
#> Finding integration vectors
#> Finding integration vector weights
#> Predicting cell labels
#> Warning: Feature names cannot have underscores ('_'), replacing with dashes
#> ('-')
#> Integrating dataset 2 with reference dataset
#> Finding integration vectors
#> Integrating data
#> Warning: Keys should be one or more alphanumeric characters followed by an
#> underscore, setting key from ref.pca_ to refpca_
#> Warning: Keys should be one or more alphanumeric characters followed by an
#> underscore, setting key from ref.pca_ to refpca_
#> Warning: All keys should be one or more alphanumeric characters followed by an
#> underscore '_', setting key to refpca_
#> Computing nearest neighbors
#> Running UMAP projection
#> Error in check_graph(graph, n_vertices, n_neighbors): ncol(idx) == expected_cols is not TRUE

^{Created on 2021-12-02 by the reprex package (v2.0.1)}

devtools::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 4.1.0 (2021-05-18)
#>  os       Ubuntu 20.04.3 LTS          
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language (EN)                        
#>  collate  en_US.UTF-8                 
#>  ctype    en_US.UTF-8                 
#>  tz       Europe/Copenhagen           
#>  date     2021-12-02                  
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  ! package     * version date       lib source           
#>  P backports     1.2.1   2020-12-09 [?] standard (@1.2.1)
#>    cachem        1.0.6   2021-08-19 [1] CRAN (R 4.1.0)   
#>  P callr         3.7.0   2021-04-20 [?] CRAN (R 4.1.0)   
#>  P cli           3.0.1   2021-07-17 [?] CRAN (R 4.1.0)   
#>  P crayon        1.4.1   2021-02-08 [?] CRAN (R 4.1.0)   
#>  P desc          1.4.0   2021-09-28 [?] CRAN (R 4.1.0)   
#>  P devtools      2.4.2   2021-06-07 [?] CRAN (R 4.1.0)   
#>  P digest        0.6.28  2021-09-23 [?] CRAN (R 4.1.0)   
#>  P ellipsis      0.3.2   2021-04-29 [?] CRAN (R 4.1.0)   
#>  P evaluate      0.14    2019-05-28 [?] standard (@0.14) 
#>  P fansi         0.5.0   2021-05-25 [?] CRAN (R 4.1.0)   
#>  P fastmap       1.1.0   2021-01-25 [?] CRAN (R 4.1.0)   
#>  P fs            1.5.0   2020-07-31 [?] standard (@1.5.0)
#>  P glue          1.4.2   2020-08-27 [?] standard (@1.4.2)
#>  P highr         0.9     2021-04-16 [?] CRAN (R 4.1.0)   
#>  P htmltools     0.5.2   2021-08-25 [?] CRAN (R 4.1.0)   
#>  P knitr         1.36    2021-09-29 [?] CRAN (R 4.1.0)   
#>  P lifecycle     1.0.1   2021-09-24 [?] CRAN (R 4.1.0)   
#>  P magrittr      2.0.1   2020-11-17 [?] standard (@2.0.1)
#>  P memoise       2.0.0   2021-01-26 [?] CRAN (R 4.1.0)   
#>  P pillar        1.6.4   2021-10-18 [?] CRAN (R 4.1.0)   
#>  P pkgbuild      1.2.0   2020-12-15 [?] standard (@1.2.0)
#>  P pkgconfig     2.0.3   2019-09-22 [?] standard (@2.0.3)
#>  P pkgload       1.2.3   2021-10-13 [?] CRAN (R 4.1.0)   
#>  P prettyunits   1.1.1   2020-01-24 [?] standard (@1.1.1)
#>  P processx      3.5.2   2021-04-30 [?] CRAN (R 4.1.0)   
#>  P ps            1.6.0   2021-02-28 [?] CRAN (R 4.1.0)   
#>  P purrr         0.3.4   2020-04-17 [?] standard (@0.3.4)
#>  P R.cache       0.15.0  2021-04-30 [?] CRAN (R 4.1.0)   
#>  P R.methodsS3   1.8.1   2020-08-26 [?] CRAN (R 4.1.0)   
#>  P R.oo          1.24.0  2020-08-26 [?] CRAN (R 4.1.0)   
#>  P R.utils       2.11.0  2021-09-26 [?] CRAN (R 4.1.0)   
#>  P R6            2.5.1   2021-08-19 [?] CRAN (R 4.1.0)   
#>  P remotes       2.4.1   2021-09-29 [?] CRAN (R 4.1.0)   
#>  P reprex        2.0.1   2021-08-05 [?] CRAN (R 4.1.0)   
#>  P rlang         0.4.12  2021-10-18 [?] CRAN (R 4.1.0)   
#>  P rmarkdown     2.11    2021-09-14 [?] CRAN (R 4.1.0)   
#>  P rprojroot     2.0.2   2020-11-15 [?] standard (@2.0.2)
#>  P rstudioapi    0.13    2020-11-12 [?] standard (@0.13) 
#>  P sessioninfo   1.1.1   2018-11-05 [?] CRAN (R 4.1.0)   
#>  P stringi       1.7.5   2021-10-04 [?] CRAN (R 4.1.0)   
#>  P stringr       1.4.0   2019-02-10 [?] standard (@1.4.0)
#>  P styler        1.6.2   2021-09-23 [?] CRAN (R 4.1.0)   
#>  P testthat      3.1.0   2021-10-04 [?] CRAN (R 4.1.0)   
#>  P tibble        3.1.5   2021-09-30 [?] CRAN (R 4.1.0)   
#>  P usethis       2.1.2   2021-10-25 [?] CRAN (R 4.1.0)   
#>  P utf8          1.2.2   2021-07-24 [?] CRAN (R 4.1.0)   
#>  P vctrs         0.3.8   2021-04-29 [?] CRAN (R 4.1.0)   
#>  P withr         2.4.2   2021-04-18 [?] CRAN (R 4.1.0)   
#>  P xfun          0.27    2021-10-18 [?] CRAN (R 4.1.0)   
#>  P yaml          2.2.1   2020-02-01 [?] standard (@2.2.1)
#> 

#>  P ── Loaded and on-disk path mismatch.

^{Created on 2021-12-02 by the reprex package (v2.0.1)}

pebonte · 2021-12-02T13:51:11Z

Hi @yuhanH ,
Yes with MapQuery I had the message :

Computing nearest neighbors
Running UMAP projection
Error in check_graph(graph, n_vertices, n_neighbors) : 
  ncol(idx) == expected_cols is not TRUE

By removing reduction.model it worked but it was projected onto the pca space.

And with NNHelper from Seurat I had :

Error in index$getNNsByVectorList(query[x, ], k, search.k, include.distance) : 
  fv.size() != vector_size

I was trying to repeat the error with the tutorial dataset and not my own dataset for you to be able to reproduce the error but @pascaltimshel actually did it.

pascaltimshel · 2021-12-02T14:23:56Z

@yuhanH :
From #5257 (comment):

Could you try to run MapQuery without setting reduction.model?

For MapQuery()

Can you explain what the reference.reduction = "pca" and reduction.model = "umap" parameters does? It's not really clear to me from the documentation.
What happens when reduction.model = "umap" is removed?
How do I plot the results when reduction.model = "umap" is removed? This does not work: p1 <- DimPlot(seurat_obj.ref, reduction = "umap"; p2 <- DimPlot(seurat_obj.query, reduction = "ref.umap"; p1 + p2

yuhanH · 2021-12-02T21:28:06Z

HI,
This error is caused by an update in uwot. We will fix this bug soon.
You can install the previous version of uwot and restart the RStudio session to solve this error.

devtools::install_version("uwot", version = "0.1.10", repos = "http://cran.us.r-project.org")

hi @pascaltimshel
reduction.model = "umap" is used to specify the reference umap reduction.
If you don't set reduction.model, projected umap will not be generated.
reference.reduction = "pca" is the reference low dimensional space where new query cells are projected.

SammiLyu · 2021-12-03T02:09:42Z

Hi @yuhanH,

I also ran into the same issue as described above and just tried to install the previous version of uwot as you specified.

session info for Seurat here (and uwot version 0.1.10 is loaded):

> sessionInfo("Seurat")
R version 4.0.4 (2021-02-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.5 LTS

Matrix products: default
BLAS:   /opt/R/4.0.4/lib/R/lib/libRblas.so
LAPACK: /opt/R/4.0.4/lib/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
character(0)

other attached packages:
[1] Seurat_4.0.5.9003

loaded via a namespace (and not attached):
  [1] snow_0.4-4            plyr_1.8.6            igraph_1.2.9          lazyeval_0.2.2        shinydashboard_0.7.2  splines_4.0.4         listenv_0.8.0        
  [8] scattermore_0.7       usethis_2.1.3         ggplot2_3.3.5         digest_0.6.29         htmltools_0.5.2       fansi_0.5.0           memoise_2.0.1        
 [15] magrittr_2.0.1        usedist_0.4.0         tensor_1.5            googlesheets4_1.0.0   cluster_2.1.0         ROCR_1.0-11           remotes_2.4.1        
 [22] globals_0.14.0        matrixStats_0.61.0    R.utils_2.11.0        methods_4.0.4         askpass_1.1           spatstat.sparse_2.0-0 prettyunits_1.1.1    
 [29] colorspace_2.0-2      ggrepel_0.9.1         dplyr_1.0.7           callr_3.7.0           crayon_1.4.2          jsonlite_1.7.2        spatstat.data_2.1-0  
 [36] velocyto.R_0.6        survival_3.2-7        zoo_1.8-9             glue_1.5.1            polyclip_1.10-0       utils_4.0.4           gtable_0.3.0         
 [43] gargle_1.2.0          leiden_0.3.9          liger_2.0.1           pkgbuild_1.2.0        future.apply_1.8.1    swne_0.6.20           BiocGenerics_0.36.1  
 [50] abind_1.4-5           scales_1.1.1          graphics_4.0.4        miniUI_0.1.1.1        Rcpp_1.0.7            viridisLite_0.4.0     xtable_1.8-4         
 [57] reticulate_1.22       spatstat.core_2.3-2   bit_4.0.4             rsvd_1.0.5            proxy_0.4-26          NNLM_0.4.4            umap_0.2.7.0         
 [64] DT_0.20               base_4.0.4            htmlwidgets_1.5.4     httr_1.4.2            FNN_1.1.3             RColorBrewer_1.1-2    ellipsis_0.3.2       
 [71] ica_1.0-2             pkgconfig_2.0.3       R.methodsS3_1.8.1     farver_2.1.0          uwot_0.1.10           deldir_1.0-6          utf8_1.2.2           
 [78] tidyselect_1.1.1      labeling_0.4.2        rlang_0.4.12          reshape2_1.4.4        later_1.3.0           stats_4.0.4           cachem_1.0.6         
 [85] munsell_0.5.0         cellranger_1.1.0      tools_4.0.4           cli_3.1.0             generics_0.1.1        devtools_2.4.2        ggridges_0.5.3       
 [92] shinyBS_0.61          stringr_1.4.0         fastmap_1.1.0         goftest_1.2-3         grDevices_4.0.4       processx_3.5.2        bit64_4.0.5          
 [99] fs_1.5.1              fitdistrplus_1.1-6    purrr_0.3.4           RANN_2.6.1            pbapply_1.5-0         future_1.23.0         nlme_3.1-152         
[106] mime_0.12             R.oo_1.24.0           hdf5r_1.3.5           compiler_4.0.4        plotly_4.10.0         curl_4.3.2            png_0.1-7            
[113] testthat_3.1.0        spatstat.utils_2.2-0  tibble_3.1.6          stringi_1.7.6         ps_1.6.0              desc_1.4.0            RSpectra_0.16-0      
[120] lattice_0.20-41       Matrix_1.3-4          SeuratDisk_0.0.0.9019 shinyjs_2.0.0         vctrs_0.3.8           pillar_1.6.4          lifecycle_1.0.1      
[127] BiocManager_1.30.16   spatstat.geom_2.3-0   lmtest_0.9-39         RcppAnnoy_0.0.19      data.table_1.14.2     cowplot_1.1.1         irlba_2.3.3          
[134] httpuv_1.6.3          patchwork_1.1.1       R6_2.5.1              pcaMethods_1.82.0     promises_1.2.0.1      KernSmooth_2.23-18    gridExtra_2.3        
[141] parallelly_1.29.0     sessioninfo_1.1.1     codetools_0.2-18      pkgload_1.2.3         MASS_7.3-53           rprojroot_2.0.2       openssl_1.4.5        
[148] withr_2.4.3           SeuratObject_4.0.4    presto_1.0.0          sctransform_0.3.2     datasets_4.0.4        Azimuth_0.4.3         mgcv_1.8-33          
[155] parallel_4.0.4        grid_4.0.4            rpart_4.1-15          tidyr_1.1.4           googledrive_2.0.0     Rtsne_0.15            Biobase_2.50.0       
[162] shiny_1.7.1

But I am running into this error instead when trying RunUMAP on the example data set (pancreas)

Running UMAP projection
18:07:29 Read 638 rows and found  numeric columns
18:07:29 Processing block 1 of 1
18:07:29 Commencing smooth kNN distance calibration using 1 thread
18:07:29 Initializing by weighted average of neighbor coordinates using 1 thread
18:07:29 Commencing optimization for 167 epochs, with 12743 positive edges
Error in optimize_layout_umap(head_embedding = embedding, tail_embedding = train_embedding,  : 
  object '_uwot_optimize_layout_umap' not found
Error during wrapup: no slot of name "assay.orig" for this object of class "Assay"

I will wait for the update for fixing the bug but if anyone knows what is going on with this and if there is an easy fix, I would really appreciate it!

yuhanH · 2021-12-03T02:12:17Z

Hi @SammiLyu
Your error looks like a seurat-related error. Could you post your scripts here? Thanks

SammiLyu · 2021-12-03T02:22:06Z

No problem!
And sorry this error is actually from both RunUMAP and ProjectUMAP and I am attaching the scripts with ProjectUMAP below as these commands ran on the reference data.

library(Seurat)
library(SeuratDisk)
library(ggplot2)
library(cowplot)
library(patchwork)
library(SeuratData)

InstallData("panc8")
data("panc8")
pancreas.list <- SplitObject(panc8, split.by = "tech")
pancreas.list <- pancreas.list[c("celseq", "celseq2", "fluidigmc1", "smartseq2")]
for (i in 1:length(pancreas.list)) {
  pancreas.list[[i]] <- NormalizeData(pancreas.list[[i]], verbose = FALSE)
  pancreas.list[[i]] <- FindVariableFeatures(pancreas.list[[i]], selection.method = "vst", nfeatures = 2000,
                                             verbose = FALSE)
}
reference.list <- pancreas.list[c("celseq", "celseq2", "smartseq2")]
pancreas.query <- pancreas.list[["fluidigmc1"]]
pancreas.anchors <- FindIntegrationAnchors(object.list = reference.list, dims = 1:30)
pancreas.integrated <- IntegrateData(anchorset = pancreas.anchors, dims = 1:30)
DefaultAssay(pancreas.integrated) <- "integrated"
# Run the standard workflow for visualization and clustering
pancreas.integrated <- ScaleData(pancreas.integrated, verbose = FALSE) %>% 
  RunPCA(npcs = 30, verbose = FALSE) %>%
  RunUMAP(dims = 1:30, reduction = "pca", return.model = TRUE)
pancreas.anchors <- FindTransferAnchors(reference = pancreas.integrated, query = pancreas.query,
                                        dims = 1:30, reference.reduction = "pca")
pancreas.query <- TransferData(anchorset = pancreas.anchors, reference = pancreas.integrated, query = pancreas.query,
                               refdata = list(celltype = "celltype"))
pancreas.query <- IntegrateEmbeddings(anchorset = pancreas.anchors, reference = pancreas.integrated,
                                      query = pancreas.query, new.reduction.name = "ref.pca")
pancreas.query <- ProjectUMAP(query = pancreas.query, query.reduction = "ref.pca", reference = pancreas.integrated,
                              reference.reduction = "pca", reduction.model = "umap")

SammiLyu · 2021-12-03T02:54:03Z

Hi @SammiLyu Your error looks like a seurat-related error. Could you post your scripts here? Thanks

Hi @yuhanH
After restarting the entire session, both RunUMAP and ProjectUMAP seems to be working now! I may have made a mistake detaching uwot and reloading it. Thank you!

yuhanH · 2021-12-03T02:55:11Z

Hi @SammiLyu
Glad to know it is fixed. Thanks for letting me know.

yuhanH · 2021-12-16T21:06:41Z

Thanks for reporting this issue. This bug is fixed in seurat 4.0.6

yuhanH added the more-information-needed We need more information before this can be addressed label Nov 4, 2021

no-response bot removed the more-information-needed We need more information before this can be addressed label Nov 5, 2021

yuhanH added the more-information-needed We need more information before this can be addressed label Nov 19, 2021

no-response bot removed the more-information-needed We need more information before this can be addressed label Nov 21, 2021

yuhanH added the more-information-needed We need more information before this can be addressed label Nov 22, 2021

no-response bot removed the more-information-needed We need more information before this can be addressed label Nov 25, 2021

yuhanH self-assigned this Dec 2, 2021

yuhanH added the bug Something isn't working label Dec 2, 2021

yuhanH closed this as completed Dec 16, 2021

pcm32 mentioned this issue Jan 8, 2022

Move to seurat 4 ebi-gene-expression-group/r-seurat-scripts#29

Merged

ahdee mentioned this issue Jul 25, 2023

Error in Multimodal_reference_mapping Vignette. #7613

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MapQuery error in Seurat reference mapping #5257

MapQuery error in Seurat reference mapping #5257

ceegilllns commented Nov 3, 2021 •

edited

Loading

yuhanH commented Nov 4, 2021

ceegilllns commented Nov 5, 2021

yuhanH commented Nov 19, 2021

ceegilllns commented Nov 21, 2021

yuhanH commented Nov 22, 2021

ceegilllns commented Nov 25, 2021

pebonte commented Dec 2, 2021

yuhanH commented Dec 2, 2021

pascaltimshel commented Dec 2, 2021 •

edited

Loading

pebonte commented Dec 2, 2021

pascaltimshel commented Dec 2, 2021 •

edited

Loading

yuhanH commented Dec 2, 2021 •

edited

Loading

SammiLyu commented Dec 3, 2021

yuhanH commented Dec 3, 2021 •

edited

Loading

SammiLyu commented Dec 3, 2021

SammiLyu commented Dec 3, 2021

yuhanH commented Dec 3, 2021

yuhanH commented Dec 16, 2021

MapQuery error in Seurat reference mapping #5257

MapQuery error in Seurat reference mapping #5257

Comments

ceegilllns commented Nov 3, 2021 • edited Loading

yuhanH commented Nov 4, 2021

ceegilllns commented Nov 5, 2021

yuhanH commented Nov 19, 2021

ceegilllns commented Nov 21, 2021

yuhanH commented Nov 22, 2021

ceegilllns commented Nov 25, 2021

pebonte commented Dec 2, 2021

yuhanH commented Dec 2, 2021

pascaltimshel commented Dec 2, 2021 • edited Loading

pebonte commented Dec 2, 2021

pascaltimshel commented Dec 2, 2021 • edited Loading

yuhanH commented Dec 2, 2021 • edited Loading

SammiLyu commented Dec 3, 2021

yuhanH commented Dec 3, 2021 • edited Loading

SammiLyu commented Dec 3, 2021

SammiLyu commented Dec 3, 2021

yuhanH commented Dec 3, 2021

yuhanH commented Dec 16, 2021

ceegilllns commented Nov 3, 2021 •

edited

Loading

pascaltimshel commented Dec 2, 2021 •

edited

Loading

pascaltimshel commented Dec 2, 2021 •

edited

Loading

yuhanH commented Dec 2, 2021 •

edited

Loading

yuhanH commented Dec 3, 2021 •

edited

Loading