Update the way API KEY is read #2

ketakipai · 2022-11-22T11:10:03Z

taking path of env file from user and using it to load environment.
getting api key from loaded environment.

.pre-commit-hooks.yaml

README.md

pii_check/pii_check_hook.py

setup.py

onurcayci · 2022-11-22T13:59:11Z

@ketakipai I would like to flag this for dev discussion.

pii_check/pii_check_hook.py

guyd

I suggest we do the simple changes only. I really think that we should try the pre-commit in our own development lifecycle sooner than later because this will likely uncover un-expected problems/limitations.
We can polish the project configuration once we have confidence that there are no major blockers when using the hook in a developer setup.

ketakipai · 2022-11-23T09:42:55Z

I suggest we do the simple changes only. I really think that we should try the pre-commit in our own development lifecycle sooner than later because this will likely uncover un-expected problems/limitations. We can polish the project configuration once we have confidence that there are no major blockers when using the hook in a developer setup.

You can add the hook by following the README here https://github.com/ketaki99/test-pre-commit

README.md

pii_check/pii_check_hook.py

setup.py

onurcayci · 2022-11-24T13:02:23Z

README.md

        pass_filenames: false
-        additional_dependencies: ["requests"]
        verbose: true
        language: python


please remove these ones as well. As you can see here https://pre-commit.com/#pre-commit-configyaml---hooks only the verbose option works, the other options do not exist for .pre-commit-config.yaml.

onurcayci · 2022-11-24T13:03:28Z

setup.py

@@ -10,7 +10,11 @@
    author="Private AI",
    description=DESCRIPTION,
    packages=find_packages(),
-    install_requires=["pre-commit==2.20.0", "requests==2.28.1"],
+    install_requires=[
+        "pre-commit==2.20.0",


We shouldn't install pre-commit as a dependency. pre-commit is the tool that installs our tool so we are not dependent on it.

…that we need the rest

onurcayci

@ketakipai can you please update the README to only include the valid .pre-commit-config.yaml fields? Currently, you only removed one of the invalid ones but there are still invalid ones on there which can cause confusion and unknown issues later on. I have already attached the documentation but I am attaching it again: https://pre-commit.com/#pre-commit-configyaml---hooks

onurcayci

I have pointed all the .pre-commit-config.yaml options that we need to change in our documentation in order to provide the correct information to our customers.

onurcayci · 2022-11-24T16:55:35Z

README.md

 repos:
  - repo: https://github.com/ketaki99/test-pre-commit
-    rev: test.1
+    rev: test.5
    hooks:
      - id: pii-check
        name: Check for PII


This option is used to override the name of the hook and it is optional. I would rather not include this in our documentation since we are already setting this name in .pre-commit-hooks.yaml.

onurcayci · 2022-11-24T16:55:51Z

README.md

 repos:
  - repo: https://github.com/ketaki99/test-pre-commit
-    rev: test.1
+    rev: test.5
    hooks:
      - id: pii-check
        name: Check for PII
        description: this hook checks if staged files have PII and marks it.


Not a valid .pre-commit-config.yaml keyword.

onurcayci · 2022-11-24T16:57:00Z

README.md

    hooks:
      - id: pii-check
        name: Check for PII
        description: this hook checks if staged files have PII and marks it.
-        entry: pii_check --url URL --enabled-entities ENABLED_ENTITIES
+        entry: pii_check --url URL --env-file-path ENV_FILE_PATH  --enabled-entities ENABLED_ENTITIES


Not a valid .pre-commit-config.yaml keyword. We should instead use args keyword. This means that we need to update our entry keyword in .pre-commit-hooks.yaml.

onurcayci · 2022-11-24T16:57:15Z

README.md

    hooks:
      - id: pii-check
        name: Check for PII
        description: this hook checks if staged files have PII and marks it.
-        entry: pii_check --url URL --enabled-entities ENABLED_ENTITIES
+        entry: pii_check --url URL --env-file-path ENV_FILE_PATH  --enabled-entities ENABLED_ENTITIES
        pass_filenames: false


Not a valid .pre-commit-config.yaml keyword. Please remove.

onurcayci · 2022-11-24T16:59:10Z

.pre-commit-hooks.yaml

  name: Check for PII
  description: this hook checks if modified files have PII and marks it.
-  entry: pii_check --url URL --enabled-entities ENABLED_ENTITIES
+  entry: pii_check --url URL --env-file-path ENV_FILE_PATH --enabled-entities ENABLED_ENTITIES


Due to changes we are going to do, this entry should be updated to be only the following:

Suggested change

entry: pii_check --url URL --env-file-path ENV_FILE_PATH --enabled-entities ENABLED_ENTITIES

entry: pii_check

yaysummeriscoming · 2022-11-25T13:52:04Z

I'll wait for @guyd and @onurcayci to approve before taking a look

yaysummeriscoming · 2022-11-25T13:52:42Z

@ketakipai could you also please merge master into this branch and fix the conflict please?

onurcayci · 2022-11-25T13:48:34Z

.pre-commit-hooks.yaml

+  args:
+    [
+      --url=URL,
+      --env-file-path=ENV_FILE_PATH,
+      --enabled-entities=ENABLED_ENTITIES,
+    ]


this looks incorrect, I don't think this is the valid way of passing arguments.

Here is an example: https://github.com/python-poetry/poetry/blob/master/.pre-commit-hooks.yaml

Since we are passing enabled_entities as a list of args, the traditional way does not work. I referred to pre-commit/pre-commit#971 and changed the other 2 too for uniformity

Can I ask why we need these args here? Looking at the link you shared, the args we have defined here are always passed to the entry point. We are also defining these args in the .pre-commit-config.yaml, which looks like it can cause runtime issues. If this is only to document the available flags that we can pass to the hook, I would find a different way to document them and remove this.

Yes this is mainly a placeholder for args, I have tested with this in pre-commit-hooks and found no issues. The script takes care of optional and non-optional args and doesn't throw an error if enabled-entities is not passed.

If they are placeholder, then we should remove them. Documentation is where we should be showing these things.

I tested that the hook works without these, removing them

guyd

Did a quick review. Looks good.

onurcayci · 2022-11-28T12:51:55Z

.pre-commit-hooks.yaml

+  args:
+    [
+      --url=URL,
+      --env-file-path=ENV_FILE_PATH,
+      --enabled-entities=ENABLED_ENTITIES,
+    ]


Can I ask why we need these args here? Looking at the link you shared, the args we have defined here are always passed to the entry point. We are also defining these args in the .pre-commit-config.yaml, which looks like it can cause runtime issues. If this is only to document the available flags that we can pass to the hook, I would find a different way to document them and remove this.

onurcayci · 2022-11-28T12:52:09Z

.pre-commit-hooks.yaml

+  args:
+    [
+      --url=URL,
+      --env-file-path=ENV_FILE_PATH,
+      --enabled-entities=ENABLED_ENTITIES,
+    ]


Suggested change

args:

[

--url=URL,

--env-file-path=ENV_FILE_PATH,

--enabled-entities=ENABLED_ENTITIES,

]

onurcayci · 2022-11-29T12:37:04Z

README.md

-        entry: pii_check --url URL --enabled-entities ENABLED_ENTITIES
-        pass_filenames: false
-        additional_dependencies: ["requests"]
+        entry: pii_check


One last thing, this is not a valid .pre-commit-config.yaml field. Please remove it.

Suggested change

entry: pii_check

* added all files * updated README * Deid 1284 improve robustness of pii check on off flags (#3) * fixed bugs with the PII_CHECK flags * removed local hook * added suggestion by Guy * added Guys suggestion and removed local hook * DEID-1311-update-git-pre-commit-hook-documentation (#4) * updated README * did the suggested changes * Update the way API KEY is read (#2) * passing path of env file as variable * remove the unnecessary line from setup.py * bug fix and updated script to get api key from env file * updated README * removed pathlib2 dependency and added stuff to check for relative path * updated README * added validation to check if API_KEY is missing * updated README * added all dependencies in setup.py, removed from config * removing unnecessary dependecies from pre-commit-hooks * updated dependencies * removed unnecessary fields mentioned in README, tested and confirmed that we need the rest * updated README * updated README * updated README * updated pre-commit-hooks * removing args from pre-commit-hooks * removing entry from pre-commit-config * updated README to resolve conflicts * Make compatible with beta2 (#5) * test * updated fields to be compatible with beta 2 * removing line added for debugging * updated README to contain latest rev * update README to have new repo * added link_batch and removed an unused function (#6) * added link_batch and removed an unused function * removed test file

ketakipai added 3 commits November 22, 2022 16:17

passing path of env file as variable

ec893a9

remove the unnecessary line from setup.py

aa454fd

bug fix and updated script to get api key from env file

30f4430

ketakipai requested a review from onurcayci November 22, 2022 11:10

updated README

b9a1713

onurcayci requested changes Nov 22, 2022

View reviewed changes

.pre-commit-hooks.yaml Outdated Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

pii_check/pii_check_hook.py Outdated Show resolved Hide resolved

setup.py Show resolved Hide resolved

ketakipai requested a review from guyd November 22, 2022 16:32

onurcayci reviewed Nov 22, 2022

View reviewed changes

pii_check/pii_check_hook.py Outdated Show resolved Hide resolved

pii_check/pii_check_hook.py Outdated Show resolved Hide resolved

guyd reviewed Nov 22, 2022

View reviewed changes

ketakipai added 2 commits November 22, 2022 23:10

removed pathlib2 dependency and added stuff to check for relative path

82b0f93

updated README

fed758e

added validation to check if API_KEY is missing

565a80c

onurcayci requested changes Nov 23, 2022

View reviewed changes

README.md Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

README.md Outdated Show resolved Hide resolved

pii_check/pii_check_hook.py Outdated Show resolved Hide resolved

setup.py Show resolved Hide resolved

ketakipai added 2 commits November 24, 2022 13:04

updated README

bce0e55

added all dependencies in setup.py, removed from config

26f223f

ketakipai requested a review from onurcayci November 24, 2022 07:45

removing unnecessary dependecies from pre-commit-hooks

a5a5da3

onurcayci requested changes Nov 24, 2022

View reviewed changes

updated dependencies

5077f6c

ketakipai requested a review from onurcayci November 24, 2022 13:23

removed unnecessary fields mentioned in README, tested and confirmed …

9904361

…that we need the rest

onurcayci requested changes Nov 24, 2022

View reviewed changes

ketakipai added 3 commits November 25, 2022 13:23

updated README

87337df

updated README

957eb46

updated README

29d8c86

ketakipai requested a review from onurcayci November 25, 2022 08:05

updated pre-commit-hooks

a2c2e03

ketakipai requested a review from yaysummeriscoming November 25, 2022 11:45

onurcayci requested changes Nov 25, 2022

View reviewed changes

guyd approved these changes Nov 25, 2022

View reviewed changes

resolved merge conflicts

a76d7ac

onurcayci requested changes Nov 28, 2022

View reviewed changes

removing args from pre-commit-hooks

2d39dd1

ketakipai requested a review from onurcayci November 29, 2022 12:16

onurcayci requested changes Nov 29, 2022

View reviewed changes

removing entry from pre-commit-config

151d2fa

ketakipai requested a review from onurcayci November 29, 2022 14:15

onurcayci approved these changes Nov 29, 2022

View reviewed changes

updated README to resolve conflicts

cd87bfa

ketakipai merged commit 74e0707 into dev Nov 29, 2022

ketakipai deleted the Update-the-way-API_KEY-is-read branch December 1, 2022 06:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update the way API KEY is read #2

Update the way API KEY is read #2

ketakipai commented Nov 22, 2022

onurcayci commented Nov 22, 2022

guyd left a comment

ketakipai commented Nov 23, 2022

onurcayci Nov 24, 2022

onurcayci Nov 24, 2022

onurcayci left a comment

onurcayci left a comment

onurcayci Nov 24, 2022

onurcayci Nov 24, 2022

onurcayci Nov 24, 2022

onurcayci Nov 24, 2022

onurcayci Nov 24, 2022

yaysummeriscoming commented Nov 25, 2022

yaysummeriscoming commented Nov 25, 2022

onurcayci Nov 25, 2022

onurcayci Nov 25, 2022

ketakipai Nov 28, 2022

onurcayci Nov 28, 2022

ketakipai Nov 28, 2022

onurcayci Nov 28, 2022

ketakipai Nov 29, 2022

guyd left a comment

onurcayci Nov 28, 2022

onurcayci Nov 28, 2022

onurcayci Nov 29, 2022

ketakipai Nov 29, 2022

	entry: pii_check --url URL --env-file-path ENV_FILE_PATH --enabled-entities ENABLED_ENTITIES
	entry: pii_check

Update the way API KEY is read #2

Update the way API KEY is read #2

Conversation

ketakipai commented Nov 22, 2022

onurcayci commented Nov 22, 2022

guyd left a comment

Choose a reason for hiding this comment

ketakipai commented Nov 23, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

onurcayci left a comment

Choose a reason for hiding this comment

onurcayci left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yaysummeriscoming commented Nov 25, 2022

yaysummeriscoming commented Nov 25, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

guyd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment