Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Impala DVT connection does not allow use-ssl parameter that is allowed in impyla under the hood. #743

Closed
wt1309 opened this issue Feb 19, 2023 · 1 comment
Assignees
Labels
good first issue Good issue for new DVT contributors priority: p1 High priority. Fix may be included in the next release.

Comments

@wt1309
Copy link

wt1309 commented Feb 19, 2023

The use-ssl parameter is not included in the DVT connection parameters, however, the Impala connection I am working with today is kerberized and requires SSL flag to be true. I performed the following to get this to work. Plan to eventually submit a pull request once I get some time, but for now, here is the solution:

  1. Missing python library: sudo pip3 install kerberos
  2. Used port 21050 instead of 21000 (21000 works for CLI impala shell)
  3. Update DVT source code to accomodate new parameter use_ssl
  • Add following lines of code after Line 35, 42,and 48 respectively:

    professional-services-data-validator/third_party/ibis/ibis_impala/api.py

    after line 35: use_ssl=False
    after line 42: use_ssl = (use_ssl, False)[use_ssl is None]
    after line 48: use_ssl=use_ssl,

  • Add the following line of code after Line 131 in:

    professional-services-data-validator/data_validation/cli_tools.py

    after line 131: ["use_ssl", "Use SSL when connecting to HiveServer2 (default is False)"],

  1. cd to professional-services-data-validator folder
  2. Then run command to build new code: sudo python3 -m pip install .

Now this connection should work:
data-validation connections add --connection-name <Conneciton_Name> Impala --host --auth-mechanism GSSAPI --kerberos-service-name impala --port 21050 --use-ssl True

@nehanene15 nehanene15 added good first issue Good issue for new DVT contributors priority: p1 High priority. Fix may be included in the next release. labels Mar 28, 2023
@mloeberc mloeberc self-assigned this Mar 30, 2023
mloeberc added a commit that referenced this issue Apr 3, 2023
* feat: add Impala connection optional parameters

* fix: reformatting file with black and flake8
@mloeberc
Copy link
Contributor

mloeberc commented Apr 3, 2023

closed by PR #790

@mloeberc mloeberc closed this as completed Apr 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good issue for new DVT contributors priority: p1 High priority. Fix may be included in the next release.
Projects
None yet
Development

No branches or pull requests

3 participants