Add ADaaS documentation. #64

samod · 2024-07-09T09:05:44Z

This PR provides ADaaS public documentation for 2P/3P developers.

app.devrev.ai/devrev/works/ISS-100391

bc-devrev

I didn't mark every instance, but capitalization of titles and common nouns needs to be fixed throughout. Just because something is in the terminology list or is abbreviated doesn't mean it should be capitalized.

bc-devrev · 2024-07-09T17:41:16Z

fern/versions/public.yml

@@ -143,6 +143,42 @@ navigation:
              - page: Developer keyrings
                slug: developer-keyring
                path: ../docs/pages/references/keyrings/developer_keyring.mdx
+          - section: ADaaS


This is really deeply nested. Is it the right place?

Can we move it one level up? @shashankcube ?

I have moved it up one level.

bc-devrev · 2024-07-09T17:57:35Z

fern/docs/pages/references/adaas/extraction_phases.mdx

+A Worker has the responsibility of maintaining its own state that persists between phases in a Sync Run,
+as well as between Sync Runs.
+
+```mermaid


Fern does not support Mermaid diagrams. You can go to mermaid.live and export a PNG for inclusion.

Can Fern render SVGs?

I believe so.

fern/versions/public.yml

bc-devrev · 2024-07-09T20:51:53Z

fern/docs/pages/references/adaas/creating_a_keyring.mdx

+Keyrings are a DevRev-specific mechanism for managing authentication for External Systems.
+They are called Connections in the DevRev app.
+
+They provide a secure way to store and manage credentials within your DevRev Snap-In.


Suggested change

They provide a secure way to store and manage credentials within your DevRev Snap-In.

They provide a secure way to store and manage credentials within your DevRev snap-In.

Are snap-ins always lowercase? Since it is a sort of a brand name for DevRev?

Yes. It's not a brand name. https://www.notion.so/devrev/Should-a-product-or-feature-name-be-capitalized-20088da3e4074b18b0bec42555522c3a?pvs=4

I have modified cases for snap-ins.

bc-devrev · 2024-07-09T20:52:20Z

fern/docs/pages/references/adaas/extracting_attachments.mdx

@@ -0,0 +1,39 @@
+# Attachment Extraction phase
+
+For the Attachment Extraction phase of the import process,


Suggested change

For the Attachment Extraction phase of the import process,

For the attachment extraction phase of the import process,

We had a lengthy discussion regarding capitalizing terms and well, the Airdrop team argues, that capitalizing names of processes, domain objects, ... and other domain entities would make the documentation more clear.

Can we cross-link everything with the Terminology section?

I have removed the Terminology section.

fern/docs/pages/references/adaas/extracting_metadata.mdx

bc-devrev · 2024-07-09T20:53:43Z

fern/docs/pages/references/adaas/extraction_phases.mdx

+	gw -->> user: Import deleted
+```
+
+An Initial Import consists of the following phases:


Suggested change

An Initial Import consists of the following phases:

An initial import consists of the following phases:

bc-devrev · 2024-07-09T20:54:19Z

fern/docs/pages/references/adaas/terminology.mdx

+
+## Forward Sync
+
+A Forward Sync is a sync from an [External System](#external-system) to DevRev.


Suggested change

A Forward Sync is a sync from an [External System](#external-system) to DevRev.

A forward sync is a sync from an [external system](#external-system) to DevRev.

Move the documentation one level up. Co-authored-by: Ben Colborn <[email protected]>

Use hyphens instead of underscores in directory names. Co-authored-by: Ben Colborn <[email protected]>

Co-authored-by: Ben Colborn <[email protected]>

… page in index.

github-actions · 2024-07-15T15:57:40Z

Published docs preview URL: https://devrev-preview-35c29569-add1-473b-9cd5-10f26624d760.docs.buildwithfern.com

erazemk · 2024-07-15T16:29:27Z

Titles are added twice:

github-actions · 2024-07-18T17:54:19Z

Published docs preview URL: https://devrev-preview-4c9e6766-cade-450d-a2ef-b88169b4c07e.docs.buildwithfern.com

samod · 2024-07-18T17:55:00Z

Dear @bc-devrev , @erazemk , also @radovan-jorgic.

I have managed to come through all the suggestions and comments, and also updated and edited the metadata chapter. Could you please re-review?

samod · 2024-07-18T17:55:26Z

And also, how can I generate a preview?

radovan-jorgic · 2024-07-18T18:02:11Z

fern/docs/pages/adaas/delete-phases.mdx

+
+### Triggering event
+
+Airdrop initiates the attachments delete phase when the Sync is deleted from DevRev.


Didn't know this. Attachments deletion starts when import is deleted (and not visible through the UI)? Or should we say when data (and metadata) is deleted instead of when Sync is deleted.

fern/docs/pages/adaas/extracting-attachments.mdx

radovan-jorgic · 2024-07-18T18:07:24Z

fern/docs/pages/adaas/extracting-attachments.mdx

+
+Once the Attachment Extraction phase is done, the snap-in must respond to Airdrop with a message with eventType `EXTRACTION_ATTACHMENTS_DONE`.
+
+If Attachment Extraction failed, the snap-in must respond to Airdrop with a message with eventType `EXTRACTION_ATTACHMENTS_ERROR`.


Just curious if there is agreement regarding usage of word "eventType". Should we use "human language"? Something like "with event of type `EVENT_TYPE``?

I meant "event of type EXTRACTION_ATTACHMENTS_DONE" and not "event of type EVENT_TYPE EXTRACTION_ATTACHMENTS_DONE".

So replace all "event of type EVENT_TYPE" with just "event of type" now.

radovan-jorgic · 2024-07-18T18:11:51Z

fern/docs/pages/adaas/extracting-data.mdx

+
+Once the data extraction is done, the snap-in must respond to Airdrop with a message with eventType `EXTRACTION_DATA_DONE`.
+
+If data extraction failed in any moment of extraction, the snap-in must respond to Airdrop with a message with eventType `EXTRACTION_DATA_ERROR`.


Not tested but I believe we save the latest successful state and retrieve it in next iteration if import failed and we retried it? If this is true, maybe we can write it down also - can be really helpful?

I do not understand what you mean. Please, explain.

If import fails, the latest state (before we emitted error) should be sent to snap-in in next iteration if retry (re-run) button is cliked on the UI. This is not mentioned anywhere.

fern/docs/pages/adaas/extracting-external-sync-units.mdx

github-actions · 2024-07-22T18:17:28Z

Published docs preview URL: https://devrev-preview-adeb82ec-fa7b-4089-a997-c852d90aa618.docs.buildwithfern.com

github-actions · 2024-07-22T19:05:26Z

Published docs preview URL: https://devrev-preview-b33a5c51-6cf8-4391-9224-5d2c96890c81.docs.buildwithfern.com

github-actions · 2024-07-22T19:08:43Z

Published docs preview URL: https://devrev-preview-aa13bfee-9acd-4986-86be-bca6d8f41f13.docs.buildwithfern.com

github-actions · 2024-07-22T19:32:10Z

Published docs preview URL: https://devrev-preview-58fc78a6-43de-4b06-990a-131394a40eb6.docs.buildwithfern.com

github-actions · 2024-07-22T19:48:15Z

Published docs preview URL: https://devrev-preview-a2d1ac73-6764-44e5-951f-94758bf9b97a.docs.buildwithfern.com

erazemk · 2024-07-23T07:56:34Z

fern/docs/pages/adaas/overview.mdx

+_Airdrop_ is DevRev’s solution to migrate data. It allows our customers to bring in their existing data from
+load data from external systems and, export data back to external systems, and keep data in sync between DevRev and the external systems.


Very weird sentence, especially this part: It allows our customers to bring in their existing data from load data from external systems and

fern/docs/pages/adaas/overview.mdx

erazemk · 2024-07-23T08:00:21Z

fern/docs/pages/adaas/getting-started.mdx

+2. Install required tools and packages:
+   - [devrev-cli](https://developer.devrev.ai/snapin-development/references/cli-install) (version 4.7 or higher)
+   - [jq](https://stedolan.github.io/jq)
+   - [golang](https://go.dev/doc/install)


What is Go needed for?

erazemk · 2024-07-23T08:01:49Z

fern/docs/pages/adaas/getting-started.mdx

+3. Create a new repo from the ADaaS template repo:
+On [https://github.com/devrev/adaas-template](https://github.com/devrev/adaas-template) use the dropdown button `Use this template` and
+then `Create a new repository`
+4. Copy `Makefile.variable.example` to `Makefile.variable` and fill in the required variables.


This is not really a problem with the documentation, but I still don't understand why we can't use the standard .env and .env.example, everybody knows and uses those but us.

fern/docs/pages/adaas/extraction-phases.mdx

erazemk · 2024-07-23T08:06:21Z

fern/docs/pages/adaas/extraction-phases.mdx

+Each sync run is comprised out of phases. Phases follow sequentially, and each can consist of one or more invocations
+of the snap-in.
+
+![ADaaS extraction phases](../../img/adaas/adaas-extraction-phases.svg)


@bc-devrev Is there a way we can dynamically adapt the color scheme of the diagrams, e.g. providing separate diagrams for light and dark mode? Mermaid supports such changes, so if there is no way to do it currently, can we at least push Fern to implement Mermaid markdown support, since we already pay them a lot for their platform?

fern/docs/pages/adaas/extraction-phases.mdx

erazemk · 2024-07-23T08:10:01Z

fern/docs/pages/adaas/external-sync-units-extraction.mdx

+In the external sync unit extraction phase, the extractor is expected to obtain a list of external sync units.
+that it can extract with the provided credentials and send it to Airdrop in its response.


Suggested change

In the external sync unit extraction phase, the extractor is expected to obtain a list of external sync units.

that it can extract with the provided credentials and send it to Airdrop in its response.

In the external sync unit extraction phase the extractor is expected to obtain a list of external sync units

that it can extract with the provided credentials, and send them to Airdrop in its response.

Thx. I am keeping "it" since it is a list we are sending.

fern/docs/pages/adaas/external-sync-units-extraction.mdx

erazemk · 2024-07-23T10:58:09Z

fern/docs/pages/adaas/data-attachments-deletion.mdx

+During the delete phases, the snap-in may clean up its state or other side effects in the third party systems.
+In the most common extraction use cases, there is nothing to do and snap-ins may reply with the completion message.


Suggested change

During the delete phases, the snap-in may clean up its state or other side effects in the third party systems.

In the most common extraction use cases, there is nothing to do and snap-ins may reply with the completion message.

During the deletion phases, the snap-in may clean up its state or other side effects in the third party systems.

In the most common extraction use cases, there is nothing to do and snap-ins may reply with the completion message.

fern/docs/pages/adaas/data-attachments-deletion.mdx

Co-authored-by: Erazem Kokot <[email protected]>

github-actions · 2024-07-23T11:17:11Z

Published docs preview URL: https://devrev-preview-b4b11e74-2540-48db-89a2-e97649334899.docs.buildwithfern.com

github-actions · 2024-07-23T11:17:36Z

Published docs preview URL: https://devrev-preview-c7b460fa-44b2-4ea6-888b-8eadac792a72.docs.buildwithfern.com

github-actions · 2024-07-24T10:45:36Z

Published docs preview URL: https://devrev-preview-3e07f908-6dc3-49df-a00f-bf26c9962e2b.docs.buildwithfern.com

Add ADaaS documentation.

abdb35b

samod requested a review from a team as a code owner July 9, 2024 09:05

bc-devrev requested changes Jul 9, 2024

View reviewed changes

samod and others added 10 commits July 10, 2024 10:27

Update fern/versions/public.yml

b420bd6

Move the documentation one level up. Co-authored-by: Ben Colborn <[email protected]>

Update fern/versions/public.yml

40aeee6

Use hyphens instead of underscores in directory names. Co-authored-by: Ben Colborn <[email protected]>

Update fern/docs/pages/references/adaas/extracting_metadata.mdx

0cb1eeb

Co-authored-by: Ben Colborn <[email protected]>

Update fern/docs/pages/references/adaas/adaas_overview.mdx

626a694

Co-authored-by: Ben Colborn <[email protected]>

Update fern/docs/pages/references/adaas/extracting_attachments.mdx

53f4ee5

Co-authored-by: Ben Colborn <[email protected]>

Update fern/docs/pages/references/adaas/adaas_overview.mdx

698f054

Co-authored-by: Ben Colborn <[email protected]>

Move adaas/ up one level. Use hyphens in file names. Remove duplicate…

9c9069e

… page in index.

Modify titles to sentence case.

ac67db1

Add Keyrings and Import sections to Concepts page.

123a77b

Add Keyrings and Import sections to Concepts page.

21c2dea

samod added 4 commits July 15, 2024 18:44

Migrate entities definitions to separate pages.

902c913

Lowercase snap-ins. Remove terminology.

5073650

Add Extract metadata update. Fix of capitalized terminology.

9db8cd8

Add delete phases. Replace Worker with snap-in.

e89cbfb

samod requested a review from bc-devrev July 18, 2024 17:53

samod requested review from shashankcube and radovan-jorgic July 18, 2024 17:56

radovan-jorgic reviewed Jul 18, 2024

View reviewed changes

fern/docs/pages/adaas/extracting-attachments.mdx Outdated Show resolved Hide resolved

radovan-jorgic reviewed Jul 18, 2024

View reviewed changes

fern/docs/pages/adaas/extracting-external-sync-units.mdx Outdated Show resolved Hide resolved

samod added 4 commits July 22, 2024 20:57

Correct the Extract metadata. Replace with .

686b317

Replace Artifacts with artifacts.

8becb0a

Fix Data extraction section.

5473d59

Fix ATtachments extraction section.

ad09872

samod added 2 commits July 22, 2024 21:06

Remove commas from JSON snippet.

f5959ac

Fix a typo.

61007ef

Image edit.

df9a5eb

First uses of terms in italics, pt. 2.

89b1e33

bc-devrev approved these changes Jul 22, 2024

View reviewed changes

erazemk reviewed Jul 23, 2024

View reviewed changes

fern/docs/pages/adaas/overview.mdx Outdated Show resolved Hide resolved

erazemk reviewed Jul 23, 2024

View reviewed changes

fern/docs/pages/adaas/extraction-phases.mdx Outdated Show resolved Hide resolved

erazemk reviewed Jul 23, 2024

View reviewed changes

fern/docs/pages/adaas/extraction-phases.mdx Outdated Show resolved Hide resolved

erazemk reviewed Jul 23, 2024

View reviewed changes

samod and others added 2 commits July 23, 2024 13:15

Apply suggestions from code review

6c73f00

Co-authored-by: Erazem Kokot <[email protected]>

Code review fixes.

91bdf92

radovan-jorgic approved these changes Jul 24, 2024

View reviewed changes

Add chef-cli public repo.

b67952f

samod merged commit 7a14a70 into main Jul 24, 2024
4 checks passed

samod deleted the ISS-100391-adaas-documentation branch July 24, 2024 13:58

	They provide a secure way to store and manage credentials within your DevRev Snap-In.
	They provide a secure way to store and manage credentials within your DevRev snap-In.

		@@ -0,0 +1,39 @@
		# Attachment Extraction phase

		For the Attachment Extraction phase of the import process,

	An Initial Import consists of the following phases:
	An initial import consists of the following phases:


		## Forward Sync

		A Forward Sync is a sync from an [External System](#external-system) to DevRev.


		### Triggering event

		Airdrop initiates the attachments delete phase when the Sync is deleted from DevRev.


		Once the Attachment Extraction phase is done, the snap-in must respond to Airdrop with a message with eventType `EXTRACTION_ATTACHMENTS_DONE`.

		If Attachment Extraction failed, the snap-in must respond to Airdrop with a message with eventType `EXTRACTION_ATTACHMENTS_ERROR`.


		Once the data extraction is done, the snap-in must respond to Airdrop with a message with eventType `EXTRACTION_DATA_DONE`.

		If data extraction failed in any moment of extraction, the snap-in must respond to Airdrop with a message with eventType `EXTRACTION_DATA_ERROR`.

		_Airdrop_ is DevRev’s solution to migrate data. It allows our customers to bring in their existing data from
		load data from external systems and, export data back to external systems, and keep data in sync between DevRev and the external systems.

		In the external sync unit extraction phase, the extractor is expected to obtain a list of external sync units.
		that it can extract with the provided credentials and send it to Airdrop in its response.

		During the delete phases, the snap-in may clean up its state or other side effects in the third party systems.
		In the most common extraction use cases, there is nothing to do and snap-ins may reply with the completion message.

Add ADaaS documentation. #64

Add ADaaS documentation. #64

Conversation

samod commented Jul 9, 2024

bc-devrev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Jul 15, 2024

erazemk commented Jul 15, 2024

github-actions bot commented Jul 18, 2024

samod commented Jul 18, 2024

samod commented Jul 18, 2024

radovan-jorgic Jul 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Jul 22, 2024

github-actions bot commented Jul 22, 2024

github-actions bot commented Jul 22, 2024

github-actions bot commented Jul 22, 2024

github-actions bot commented Jul 22, 2024

erazemk Jul 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

erazemk Jul 23, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Jul 23, 2024

github-actions bot commented Jul 23, 2024

github-actions bot commented Jul 24, 2024

radovan-jorgic Jul 18, 2024 •

edited

Loading

erazemk Jul 23, 2024 •

edited

Loading

erazemk Jul 23, 2024 •

edited

Loading