Skip to content

Commit

Permalink
fix(jsii-pacmak): different packages sharing a namespace fail in Bazel (
Browse files Browse the repository at this point in the history
#4437)

In Bazel, every package is extracted into a separate directory that are all put into the `$PYTHONPATH` individually. However when searching, Python will latch onto the directory that has an `__init__.py` in it and only import from that directory.

Example:

```
eco_package1/
└── site-packages/
    └── eco/
        └── package1/
              └── __init__.py

eco/
└── site-packages/
    └── eco/
        └── __init__.py
```

In this case, the following command will fail:

```py
import eco.package1
```

Because `eco/package1/__init__.py` is not in the same directory as `eco/__init__.py`.

We can fix this by generating a command into every `__init__.py` that uses `pkgutil` to extend the search path for a package; it will put all directories in `$PYTHONPATH` that contain the same package onto the search path for submodules. The command looks like this:

```py
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
```

Resolves #3881, thanks to @psalvaggio-dl for providing a repro and doing the research for a fix.

---

By submitting this pull request, I confirm that my contribution is made under the terms of the [Apache 2.0 license].

[Apache 2.0 license]: https://www.apache.org/licenses/LICENSE-2.0
  • Loading branch information
rix0rrr committed Mar 4, 2024
1 parent 256b806 commit 5c0d9e4
Show file tree
Hide file tree
Showing 4 changed files with 430 additions and 245 deletions.
14 changes: 14 additions & 0 deletions packages/jsii-pacmak/lib/targets/python.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1700,6 +1700,20 @@ class PythonModule implements PythonType {

// Before we write anything else, we need to write out our module headers, this
// is where we handle stuff like imports, any required initialization, etc.

// If multiple packages use the same namespace (in Python, a directory) it
// depends on how they are laid out on disk if deep imports of multiple packages
// will succeed. `pip` merges all packages into the same directory, and deep
// imports work automatically. `bazel` puts packages into different directories,
// and `import aws_cdk.subpackage` will fail if `aws_cdk/__init__.py` and
// `aws_cdk/subpackage/__init__.py` are not in the same directory.
//
// We can get around this by using `pkgutil` to extend the search path for the
// current module (`__path__`) with all packages found on `sys.path`.
code.line('from pkgutil import extend_path');
code.line('__path__ = extend_path(__path__, __name__)');
code.line();

code.line('import abc');
code.line('import builtins');
code.line('import datetime');
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 5c0d9e4

Please sign in to comment.