Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Diagram: Architecture #5452

Open
wants to merge 62 commits into
base: develop
Choose a base branch
from

Conversation

NicolasNewman
Copy link
Contributor

@NicolasNewman NicolasNewman commented Apr 10, 2024

📑 Summary

Architecture diagrams allows users to show relations between services

Resolves #5367

📏 Design Decisions

Parser

Terminology

  • Service: a node within a diagram. Represents services such as from AWS, Azure, Docker, etc
  • Group: a collection of related services (similar in presentation to subgraphs for other diagrams). This could be a VPC on AWS, different APIs which talk to one another, etc

Syntax

Currently, the syntax is defined as:

  • Services are defined as service {id}({icon})[{title}] (in {group_name})?
  • Groups are defined as group {id}[{title}] (in {group_id})?
  • Edges are defined as {id_1} ( ( )?(L|R|T|B)--(L|R|T|B)( ) )? {id_2}

Once I transition from jison to Langium, I plan to add the following extensions:

  • Groups: group {id}({icon})[{title}] (in {group_id})?
    • Allow icons to be used for group names
  • Edges: {id_1} ( ( )?(L|R|T|B)-([{label}])-(L|R|T|B)( ) )? {id_2}
    • Allow edges to have a label
    • Allow forked edges traversing the XY axis (see Layout for more info). I've yet to decide on the syntax for this
  • Services: service {id}( ({icon}) | ("{text}") )[{title}] (in {group_name})?
    • Allow text to be used for a service instead of an icon

Additionally, the syntax for groups (and potentially forked edges?) will be modified to better follow a declare then organize approach

Layout

  • The positions of services are determined by specifying a required direction for an edge between the two
  • Edges which traverse the X & Y axis (LT, RB, etc) are connected by two perpendicular lines
  • Each service can only have one edge per side
  • Internally, a spatial map of each service is maintained which specifies an (x,y) coordinate of where each service is in relation to one another ([0,1] is above [0,0]). This forces the layout to have a consistent grid like pattern

Comments

As mentioned in the issue, the LRTB syntax goes against the ethos of Mermaid. Additionally, while the spatial map creates very neatly organized layouts, it prevents users from having multiple edges going out of one side. Part of the reason I began designing it this way was to avoid this diagram type from essentially being a flowchart with icons.

Going forward, there's a few options that can fix this problem:

  1. Make the LRTB syntax optional. If they aren't ever specified, don't use the spatial map for generating the constraints for fcose and let it figure things out on its own.
  2. Keep the LRTB syntax required and instead add svg icon support to flowcharts. Ultimately the two diagram types are very similar in how they represent nodes and edges. My main reasoning for adding this new diagram type other then SVG support was for more precise control over node positioning like with block diagrams. I've already been using flowcharts to quickly mock up architecture diagrams before making something nicer in softwares like Lucidcharts.

No matter which option we choose, the LRTB syntax should be extended to:

  1. Allow forked edges. This means one edge out the X/Y axis would split into more edges that would expand outwards in the opposite axis. This would solve the issue of being limited to only a max of 4 edges per service.

SVG Icons

SVG icons are currently created and accessed through helper functions inside of rendering-util/svgRegister.ts. A global object stores the mapping of icon names to IconResolvers defined as

type IconResolver = (
  parent: Selection<SVGGElement, unknown, Element | null, unknown>,
  width?: number
) => Selection<SVGGElement, unknown, Element | null, unknown>;

IconResolvers can easily be created through the helper function createIcon. This function takes the SVG code as a string along with the original resolution. It returns a CB function taking the element to inject the SVG into and optionally a different size to scale it by.

const createIcon = (icon: string, originalSize: number): IconResolver => {
  return (
    parent: Selection<SVGGElement, unknown, Element | null, unknown>,
    size: number = originalSize
  ) => {
    parent.html(`<g style="transform: scale(${size / originalSize})">${icon}</g>`);
    return parent;
  };
};

A few generic icons I've created are located in rendering-utils/svg/. Additional icons can be added through the new iconLibraries config option. The types in svgRegister.ts are now exported from mermaid.ts, allowing developers to create their own icon library packages.

// Example icon

// rendering-util/svg/database.ts
export default createIcon(
  `<g>
      <rect width="80" height="80" style="fill: #087ebf; stroke-width: 0px;"/>
     ...
</g>`,
  80
);

// rendering-util/svg/index.ts
const defaultIconLibrary: IconLibrary = {
  database: database,
  // ...
};

// index.html
mermaid.initialize({
  theme: 'base',
  startOnLoad: true,
  // This is just an example. It is already loaded by default within MermaidAPI.ts
  iconLibraries: [defaultIconLibrary],
  // ...
});

Alternatives

From conversations in the linked issue, ZenUML already handles SVG icons on their end. Upon furthur inspection, they rely on declaring an svg module which essentially resolves the imported SVG as a string. Since the end result of both of these approaches are the same (having the SVG code as a string), personally I think it's debatable if we want to go through with this extra step. The main con I foresee is that it'll add more work for people who want to make their own icon library package as they will need to setup the declarations themselves.

On a side note, it looks like ZenUML directly added the official AWS icons to their repo here. IANAL but the terms Amazon states are:

Customers and partners are permitted by AWS to use the resources below to create architecture diagrams

This may mean we can make official icon packs in seperate packages, or have them be lazy loaded as needed. Ultimately I'll leave that decision up to you as that isn't my area of specialties.

Todo

  • Unit tests
  • Documentation
  • More generic default icons? If anyone has ideas let me know and I can put something together in Illustrator
  • Convert parser from jison to Langium
  • Implemented parser extensions
    • Edge labels
    • Icons as group names
    • Text instead of service icon
  • Forked edges
  • Edges between groups
  • More control over styling

📋 Tasks

Make sure you

Copy link

netlify bot commented Apr 10, 2024

Deploy Preview for mermaid-js ready!

Name Link
🔨 Latest commit 2ae2686
🔍 Latest deploy log https://app.netlify.com/sites/mermaid-js/deploys/669e868edab768000742d67b
😎 Deploy Preview https://deploy-preview-5452--mermaid-js.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

codecov bot commented Apr 10, 2024

Codecov Report

Attention: Patch coverage is 0.41889% with 2615 lines in your changes missing coverage. Please review.

Project coverage is 5.20%. Comparing base (5b86fe3) to head (33da2b4).
Report is 4 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           develop   #5452      +/-   ##
==========================================
- Coverage     5.86%   5.20%   -0.67%     
==========================================
  Files          274     298      +24     
  Lines        41087   46514    +5427     
  Branches       488     537      +49     
==========================================
+ Hits          2408    2419      +11     
- Misses       38679   44095    +5416     
Flag Coverage Δ
unit 5.20% <0.41%> (-0.67%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...ages/mermaid/src/rendering-util/svg/aws/awsFull.ts 0.00% <ø> (ø)
packages/mermaid/src/themes/theme-default.js 94.52% <100.00%> (+0.15%) ⬆️
packages/parser/src/language/pie/valueConverter.ts 0.00% <ø> (ø)
.build/jsonSchema.ts 0.00% <0.00%> (ø)
packages/parser/src/language/architecture/index.ts 0.00% <0.00%> (ø)
packages/mermaid/src/docs/.vitepress/config.ts 0.00% <0.00%> (ø)
...s/mermaid/src/diagram-api/diagram-orchestration.ts 0.00% <0.00%> (ø)
packages/parser/src/language/index.ts 0.00% <0.00%> (ø)
packages/mermaid/src/mermaid.ts 0.00% <0.00%> (ø)
packages/mermaid/src/themes/theme-base.js 3.74% <0.00%> (-0.08%) ⬇️
... and 28 more

... and 1 file with indirect coverage changes

@NicolasNewman
Copy link
Contributor Author

@sidharthv96 Continuing our conversation from the issue, I commented on SVG loading and my thought process behind how the layout is handled. Let me know what your thoughts are.

Additionally, an example of what forked edges would look like once implemented:
image

Regarding multiple edges going into one side of a service, it's possible but I'll need to spend some time figuring out the cleanest way to implement that functionality. Ultimately I was planning on using forked edges to handle that scenario but after looking at your example the arrows can't be as neatly represented the same way through forked edges.
image

@NicolasNewman
Copy link
Contributor Author

Update on diagram syntax changes:

Edges between groups

One limitation of fcose is the 3 constraint parameters don't factor in groups. Since this prevented me from directly having edges between group nodes, I had to implement a work-around by extended the parser to signal that an edge endpoint should connect to the group border and not the node itself. In practice, the implementation looks like:
image

architecture
        group left_group(cloud)[Left]
        group right_group(cloud)[Right]
        group top_group(cloud)[Top]
        group bottom_group(cloud)[Bottom]
        group center_group(cloud)[Center]

        service left_disk(disk)[Disk] in left_group
        service right_disk(disk)[Disk] in right_group
        service top_disk(disk)[Disk] in top_group
        service bottom_disk(disk)[Disk] in bottom_group
        service center_disk(disk)[Disk] in center_group

        left_disk{group} (R--L) center_disk{group}
        right_disk{group} (L--R) center_disk{group}
        top_disk{group} (B--T) center_disk{group}
        bottom_disk{group} (T--B) center_disk{group}

Note that the {group} modifier doesn't have to be applied to both ends.

Forked edges

I decided to handle this problem with the concept of a junction node. Internally, its positioned and rendered almost identically to service nodes with the exception that its entirely transparent with no label associated.

image

architecture
        service left_disk(disk)[Disk]
        service top_disk(disk)[Disk]
        service bottom_disk(disk)[Disk]
        service top_gateway(internet)[Gateway]
        service bottom_gateway(internet)[Gateway]
        junction juncC
        junction juncR

        left_disk R--L juncC
        top_disk B--T juncC
        bottom_disk T--B juncC
        juncC R--L juncR
        top_gateway B--T juncR
        bottom_gateway T--B juncR

Additional example

image

architecture
        group left
        group right
        service left_disk(disk)[Disk] in left
        service top_disk(disk)[Disk] in left
        service bottom_disk(disk)[Disk] in left
        service top_gateway(internet)[Gateway] in right
        service bottom_gateway(internet)[Gateway] in right
        junction juncC in left
        junction juncR in right

        left_disk R--L juncC
        top_disk B--T juncC
        bottom_disk T--B juncC


        top_gateway (B--T juncR
        bottom_gateway (T--B juncR

        juncC{group} R--L) juncR{group}

Remaining Goals

As of now, the primary features that remain are:

  1. Controlling arrow orientation for junction edges
  2. Flexible styling
  3. Unit tests
  4. Finalize the docs

Lingering Questions

@sidharthv96

  1. Should a mode be added which handles the positioning on its own? I've mentioned my opinion on that earlier where I feel that it would then be very similar to flowcharts. I personally think it would make more sense to update flowcharts to have the same svg icon support.
  2. How to handle the inclusion of AWS/Azure/etc icons. Should it be included with Mermaid and dynamically loaded? Should it be split off into its own package? If so, who will host it? (that could also serve as a template for anyone who would like to make their own icon library)

@hemeroc
Copy link

hemeroc commented May 15, 2024

Just to share some opinions

  1. Should a mode be added which handles the positioning on its own?

Imho, yes. Architecture diagrams tend to get very large and sometimes some manual positioning would be extremely helpful

  1. How to handle the inclusion of AWS/Azure/etc icons. Should it be included with Mermaid and dynamically loaded?

I would include them and offer the possibility to reference external images. The reason for this is that otherwise we will run into CORS issues e.g. in the GitHub integration as external SVGs cannot be loaded to prevent XSS.

@NicolasNewman
Copy link
Contributor Author

@sidharthv96 When you have the chance, could we get the review process started for this PR? I'm happy with the results as it stands and getting some feedback would be great!

I've recently added documentation for using icons and once I know the implementation details won't change I'll update the docs with the available icons (ideally in an automated fashion akin to how the contributors are added).

@NicolasNewman NicolasNewman marked this pull request as ready for review July 14, 2024 03:38
sidharthv96 and others added 3 commits July 18, 2024 15:29
* develop:
  [autofix.ci] apply automated fixes
  [autofix.ci] apply automated fixes
  docs: Test autofix.ci
  chore: Remove update step from lint.yml
  Add autofix.ci
  fix: double space in wrapped sequence diagram messages
  chore: update browsers list
  chore(deps): update eslint
  chore: move abs below return check
  fix: Handle negative numbers in `formatBytes`
  chore: Use single quotes
  fix: emphasis => em
  fix: Handle spaces after newline
  test: Change emphasis to em
  chore: Fix emphasis type
  feat: Use marked instead of mdast-util-from-markdown
  Add Madness to integrations-community.md
Copy link

argos-ci bot commented Jul 18, 2024

The latest updates on your projects. Learn more about Argos notifications ↗︎

Waiting for the first build to start…

@sidharthv96
Copy link
Member

sidharthv96 commented Jul 18, 2024

I'll just note down points I find here, will do a full review soon.

architecture-beta should be used as the diagram keyword, this gives us a bit of wiggle room to tweak syntax till we finalize.

The bundle size has a 40% increase, which, I assume is mostly the icons?
So we'd definitely want to move icons to a separate package. We can release it under the mermaid org itself.
image

Issues with icon sizing
image

@andig
Copy link

andig commented Jul 18, 2024

As mentioned in the issue, the LRTB syntax goes against the ethos of Mermaid.

Looking at the examples it's not clear to me how to give the edges arrow ends? Can we do this similar to flow charts?

@NicolasNewman
Copy link
Contributor Author

As mentioned in the issue, the LRTB syntax goes against the ethos of Mermaid.

Looking at the examples it's not clear to me how to give the edges arrow ends? Can we do this similar to flow charts?

You can add ( / ) to the ends of the arrows to add them.

For example:

architecture
        service servC(server)[Server 1]
        service servL(server)[Server 2]
        service servR(server)[Server 3]
        service servT(server)[Server 4]
        service servB(server)[Server 5]

        servC (L--R) servL
        servC (R--L) servR
        servC (T--B) servT
        servC (B--T) servB

        servL (T--L) servT
        servL (B--L) servB
        servR (T--R) servT
        servR (B--R) servB

image

@NicolasNewman
Copy link
Contributor Author

NicolasNewman commented Jul 22, 2024

I'll just note down points I find here, will do a full review soon.

architecture-beta should be used as the diagram keyword, this gives us a bit of wiggle room to tweak syntax till we finalize.

The bundle size has a 40% increase, which, I assume is mostly the icons? So we'd definitely want to move icons to a separate package. We can release it under the mermaid org itself. image

Issues with icon sizing image

How did you test the diagram to get that result? I can't seem to reproduce it.

@andig
Copy link

andig commented Jul 22, 2024

You can add ( / ) to the ends of the arrows to add them.

Not being a mermaid or architecture native, consistency would be a plus for me. Worth noting that the flowchart uses a different notation for array ends.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Proposal: Cloud Architecture Diagram
4 participants