Importing faiss has side effect causing segfault

[lg8080]

Summary

Importing faiss has some undocumented side effects that are indirectly causing a crash on Mac.

Platform

OS: macOS 11.6.4

Faiss version: 1.7.2

Installed from: pip

Running on:

• CPU
• GPU

Interface:

• C++
• Python

Reproduction instructions

This snippet results in a segmentation fault on the last line on Mac:

import faiss
import implicit, scipy
implicit.als.AlternatingLeastSquares().fit(scipy.sparse.csr_matrix(0.5))

The code runs fine if I remove the first line.

System: macOS 11.6.4, 2.3 GHz Quad-Core Intel Core i7, CPython 3.10.2 (but the crash also happens on 3.8.12). Faiss installed from faiss_cpu-1.7.2-cp310-cp310-macosx_10_9_x86_64.whl with pip install faiss-cpu.

List of packages installed:

faiss-cpu==1.7.2
implicit==0.6.1
numpy==1.23.2
scipy==1.9.1
tqdm==4.64.1

Questions

1. What are the side effects of running import faiss?
2. Is there a way to create a version of faiss-cpu that doesn't have these side effects?

[benfred]

See also benfred/implicit#613 (comment)

[wx257osn2]

@lg8080 Based on benfred/implicit#613 (comment), it seems that it should work correctly what faiss built with libomp.dylib from source, instead of faiss-cpu on PyPI which linked to libomp.a .
(Or, implicit built with libomp.a should also work together with faiss-cpu on PyPI.)
Would you try building faiss (or implicit) from source?

NOTE: faiss-{c,g}pu on PyPI are NOT official packages (that are maintained on kyamagu/faiss-wheels, NOT by faiss maintainers), and this problem is specific to faiss-wheels . So, this issue is difficult to deal with for faiss maintainers.

[mdouze]

pip is not a supported way of installing Faiss.
It causes a lot of problems like this, especially on Macs, so please install via conda.

[lg8080]

Thank you all for the answers! Really appreciate it. I will try using conda.