Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python bindings for Lc0 backend. #1261

Merged
merged 5 commits into from
May 23, 2020
Merged

Conversation

mooskagh
Copy link
Member

@mooskagh mooskagh commented May 2, 2020

Building:

  1. Run meson.py with -Dpython_bindings
  2. After the build the file backends.cpython-38-x86_64-linux-gnu.so will appear in the build directory (in Windows, the extension should be .pyd).
  3. Put this file in the following directory structure:
(root)
|
+-- lczero/
    |
    +-- __init__.py
    +-- backends.cpython-38-x86_64-linux-gnu.so
  1. cd into (root) and start python.
  2. Do import lczero.backends and it should all work!

@mooskagh
Copy link
Member Author

mooskagh commented May 2, 2020

Example session:


In [1]: from lczero.backends import Weights, Backend, GameState

In [2]: w = Weights()
Found pb network file: ./id58612

In [3]: w.filters()
Out[3]: 128

In [4]: Backend.available_backends()
Out[4]:
['cudnn-auto',
 'cudnn',
 'cudnn-fp16',
 'opencl',
 'blas',
 'eigen',
 'random',
 'check',
 'roundrobin',
 'multiplexing',
 'demux']

In [5]: b = Backend(weights=w)
Creating backend [cudnn-auto]...
Switching to [cudnn]...
CUDA Runtime version: 10.2.0
Cudnn version: 7.6.5
Latest version of CUDA supported by the driver: 10.2.0
GPU: GeForce GTX 970
GPU memory: 3.93951 Gb
GPU clock frequency: 1253 MHz
GPU compute capability: 5.2

In [6]: g = GameState(moves=['e2e4, e7e5'])
---------------------------------------------------------------------------
LczeroException                           Traceback (most recent call last)
<ipython-input-6-cb5a9b68a873> in <module>
----> 1 g = GameState(moves=['e2e4, e7e5'])

LczeroException: Bad move: e2e4, e7e5

In [7]: g = GameState(moves=['e2e4', 'e7e5'])

In [8]: print(g.as_string())
rnbqkbnr
pppp.ppp
........
....p...
....P...
........
PPPP.PPP
RNBQKBNR KQkq[ah] (from white's eyes) Hash: 8572518730975593504


In [9]: i1 = g.as_input(b)

In [10]: i2 = GameState(fen='2R5/5kpp/4p3/p4p2/3B4/1K5N/4rNPP/8 b - - 0 29').as_input()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-10-3203972071a3> in <module>
----> 1 i2 = GameState(fen='2R5/5kpp/4p3/p4p2/3B4/1K5N/4rNPP/8 b - - 0 29').as_input()

TypeError: function missing required argument 'backend' (pos 1)

In [11]: i2 = GameState(fen='2R5/5kpp/4p3/p4p2/3B4/1K5N/4rNPP/8 b - - 0 29').as_input(b)

In [12]: bin(i1.mask(0))
Out[12]: '0b10000000000001110111100000000'

In [13]: bin(i2.mask(0))
Out[13]: '0b100001000100001100000000000000'

In [14]: o1, o2 = b.evaluate(i1, i2)

In [15]: o1.q()
Out[15]: 0.10548576712608337

In [16]: o2.q()
Out[16]: -0.9555819034576416

In [17]: zip(p1.moves(), o1.p_softmax(*p1.policy_indices()))
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-17-6c83ce4b9dff> in <module>
----> 1 zip(p1.moves(), o1.p_softmax(*p1.policy_indices()))

NameError: name 'p1' is not defined

In [18]: zip(g.moves(), o1.p_softmax(*g.policy_indices()))
Out[18]: <zip at 0x7efca0e8c340>

In [19]: list(zip(g.moves(), o1.p_softmax(*g.policy_indices())))
Out[19]:
[('b1a3', 0.0020779985934495926),
 ('b1c3', 0.047005292028188705),
 ('d1e2', 0.0033786550629884005),
 ('d1f3', 0.004004619084298611),
 ('d1g4', 0.001967670163139701),
 ('d1h5', 0.004279660061001778),
 ('e1e2', 0.0014249634696170688),
 ('f1e2', 0.007004433311522007),
 ('f1d3', 0.005651619751006365),
 ('f1c4', 0.017071455717086792),
 ('f1b5', 0.0045316447503864765),
 ('f1a6', 0.001234786119312048),
 ('g1e2', 0.007879829034209251),
 ('g1f3', 0.8301827907562256),
 ('g1h3', 0.002097159158438444),
 ('a2a3', 0.00787483248859644),
 ('a2a4', 0.004778245929628611),
 ('b2b3', 0.002658447250723839),
 ('b2b4', 0.0014852079330012202),
 ('c2c3', 0.0037954614963382483),
 ('c2c4', 0.00382616207934916),
 ('d2d3', 0.006489641964435577),
 ('d2d4', 0.006822879426181316),
 ('f2f3', 0.0023090150207281113),
 ('f2f4', 0.003144552232697606),
 ('g2g3', 0.0036341759841889143),
 ('g2g4', 0.0014319035690277815),
 ('h2h3', 0.009356374852359295),
 ('h2h4', 0.0026004489045590162)]

In [20]:

@mooskagh mooskagh requested review from borg323 and Tilps May 2, 2020 15:05
meson.build Show resolved Hide resolved
scripts/gen_py_bindings.py Outdated Show resolved Hide resolved
src/python/setup.py Outdated Show resolved Hide resolved
Copy link
Member

@borg323 borg323 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't build with python 3.7 with some link error, but this may be an issue with my setup. However python 3.8 seems to work fine. Also I get a few warnings when compiling.

cpython = dependency('python3')
python.extension_module('backends',
[py_files + files],
include_directories: [includes],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding cpp_args: cc.get_supported_arguments(['-Wno-c99-extensions', '-Wno-pedantic', '-Wno-missing-field-initializers']), here silences a lot of the warnings, but probably not a good idea.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c99-extension that I use will appear in C++20, so I'd ignore that. Other ones probably worth fixing, but I'll do it later (clang also emits some set of warnings).

@sagebei
Copy link

sagebei commented Oct 4, 2022

How to use run MCTS simulations in the backends?

@donaldlin30
Copy link

Is there a way I can retrieve centipawn scores from the binding?

@XintianHan
Copy link

How do I know the winner of the game?

@Hidancloud
Copy link

Can you guys please work a bit more with documentation and tutorials? I'm a big fan of Lc0 and there are so many great things in this repo, but there is no more or less full tutorial & explanation how and what to use here. I suppose, your solution far exceeded the amount of work that should be documented.

@psktam
Copy link

psktam commented Jan 7, 2024

@Hidancloud not sure if you already saw this but just in case looks like there's a more up-to-date solution here: #1856

If all you want to do is install the python bindings, looks like you just need to cd into the project root for this repository and run pip install --user . which will allow you to import the lczero.backends package in python.

@Iriskinn
Copy link

Iriskinn commented Aug 6, 2024

Hi there, I get the following error while installing bindings with pip install -e . (after downloading the repo and replacing c++17 with c++20 in meson.build):

[3/9] Linking target chessboard_test.exe
  FAILED: chessboard_test.exe
  "link"  /MACHINE:x64 /OUT:chessboard_test.exe chessboard_test.exe.p/src_chess_board_test.cc.obj chessboard_test.exe.p/subprojects_googletest-release-1.10.0_googletest_src_gtest-all.cc.obj "/release" "/nologo" "/OPT:REF" "lc0_lib.lib" "/SUBSYSTEM:CONSOLE" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "comdlg32.lib" "advapi32.lib"
  LINK : fatal error LNK1181: cannot open input file 'lc0_lib.lib'

I have Win64 platform with no CUDA or cuDNN, perform the installation in a virtual env.
What can be a problem in my case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants