Skip to content

Commit

Permalink
add bundle (#47)
Browse files Browse the repository at this point in the history
* add bundle
* update imports
* update workspace
go get -u
* add fieldalignment
addresses #41
* update generator documentation
manual fieldaligns
bundle fix
add tools
refactor copygen location
* fix fieldalignment: exit code 3 may indicate that the tool was unable to align everything
refactor generator error messages
* add removed fieldaligned comments
* fix potential data race
  • Loading branch information
switchupcb committed Dec 4, 2022
1 parent d3d85ae commit 6ad3efa
Show file tree
Hide file tree
Showing 59 changed files with 18,669 additions and 144 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ Every struct uses [fieldalignment](https://pkg.go.dev/golang.org/x/tools/go/anal

### Storage

Disgo adds **6.5 MB** to a compiled binary.
Disgo adds **5 MB** to a compiled binary.

### Contributing

Expand Down
16 changes: 6 additions & 10 deletions _contribution/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ The repository contains a [README](/README.md), [Examples](/_examples/), [Code G
| shard | Sharding Manager. |
| tools | Utility Tools. |

_A [bundler](https://pkg.go.dev/golang.org/x/tools/cmd/bundle) is used to package the API into a `disgo` package (`disgo.go`)_.
_A **bundler** is used to package the API into a `disgo` package (`disgo.go`)_.

#### Structs

Expand All @@ -52,9 +52,7 @@ Comments follow [Effective Go](https://golang.org/doc/effective_go#commentary) a

#### Static Code Analysis

Disgo uses [golangci-lint](https://github.com/golangci/golangci-lint) in order to statically analyze code. You can install golangci-lint with `go install github.com/golangci/golangci-lint/cmd/[email protected]`. If you receive a `diff` error _(while running)_, you must add a `diff` tool in your PATH. There is one located in the `Git` bin.

If you receive `File is not ... with -...`, use `golangci-lint run --disable-all --no-config -Egofmt --fix`.
Disgo uses [golangci-lint](https://github.com/golangci/golangci-lint) in order to statically analyze code. You can install golangci-lint with `go install github.com/golangci/golangci-lint/cmd/[email protected]`. If you receive a `diff` error _(while running)_, you must add a `diff` tool in your PATH. There is one located in the `Git/bin` directory.

| Directory | Command | Description |
| :-------- | :---------------------------- | :------------------------------------------------- |
Expand All @@ -64,6 +62,8 @@ If you receive `File is not ... with -...`, use `golangci-lint run --disable-all
| `shard` | `golangci-lint run ./shard` | Perform static code analysis on the Shard Manager. |
| `tools` | `golangci-lint run ./tools` | Perform static code analysis on the Tools Module. |

_If you receive `File is not ... with -...`, use `golangci-lint run --disable-all --no-config -Egofmt --fix` or ignore it._

#### Fieldalignment

Disgo [fieldaligns](https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/fieldalignment) bundled code to save memory.
Expand All @@ -80,15 +80,11 @@ Integration tests are used to ensure functionality between the API Wrapper and D

#### Running Tests

Use `go test` to run the tests in the current directory. Use `go test ./<dir>` to run tests in a given directory (from the current directory). Use [Github Action Workflow Files](/.github/workflows/) to find the correct test command for a module.
Use `go test` to run the tests in the current directory. Use `go test ./<dir>` to run tests in a given directory (from the current directory). Use [Github Action Workflow Files](/.github/workflows/) to find the correct test command and environment variables for a module.

# Roadmap

Disgo is **STABLE**. Here are the steps required in order to complete it.

1. Bundle Disgo (with `fieldalignment`) _[v10.0.0]_.

The following additional features are being implemented.
Disgo is **STABLE**. The following additional features are being implemented.

1. Voice Connections ([UDP Decision](/_contribution/libraries/), [Audio Processing using Opus](https://discord.com/developers/docs/topics/voice-connections#encrypting-and-sending-voice))
2. [Sharding](https://github.com/switchupcb/disgo/issues/26)
Expand Down
2 changes: 1 addition & 1 deletion _examples/command/autocomplete/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import (
"os"
"strings"

"github.com/switchupcb/disgo"
tools "github.com/switchupcb/disgo/tools"
disgo "github.com/switchupcb/disgo/wrapper"
)

// Environment Variables
Expand Down
2 changes: 1 addition & 1 deletion _examples/command/followup/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ import (
"os"
"time"

"github.com/switchupcb/disgo"
"github.com/switchupcb/disgo/tools"
disgo "github.com/switchupcb/disgo/wrapper"
)

// Environment Variables
Expand Down
2 changes: 1 addition & 1 deletion _examples/command/localization/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ import (
"log"
"os"

"github.com/switchupcb/disgo"
"github.com/switchupcb/disgo/tools"
disgo "github.com/switchupcb/disgo/wrapper"
)

// Environment Variables
Expand Down
2 changes: 1 addition & 1 deletion _examples/command/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"os/signal"
"syscall"

disgo "github.com/switchupcb/disgo/wrapper"
"github.com/switchupcb/disgo"
)

// Environment Variables
Expand Down
2 changes: 1 addition & 1 deletion _examples/command/subcommand/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ import (
"strconv"
"syscall"

"github.com/switchupcb/disgo"
"github.com/switchupcb/disgo/tools"
disgo "github.com/switchupcb/disgo/wrapper"
)

// Environment Variables
Expand Down
6 changes: 3 additions & 3 deletions _examples/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ require (

require (
github.com/andybalholm/brotli v1.0.4 // indirect
github.com/goccy/go-json v0.9.11 // indirect
github.com/goccy/go-json v0.10.0 // indirect
github.com/gorilla/schema v1.2.0 // indirect
github.com/klauspost/compress v1.15.9 // indirect
github.com/klauspost/compress v1.15.12 // indirect
github.com/switchupcb/websocket v1.8.8 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasthttp v1.41.0 // indirect
github.com/valyala/fasthttp v1.43.0 // indirect
golang.org/x/sync v0.1.0 // indirect
)
8 changes: 3 additions & 5 deletions _examples/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@ github.com/gobwas/pool v0.2.0 h1:QEmUOlnSjWtnpRGHF3SauEiOsy82Cup83Vf2LcMlnc8=
github.com/gobwas/pool v0.2.0/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw=
github.com/gobwas/ws v1.0.2 h1:CoAavW/wd/kulfZmSIBt6p24n4j7tHgNVCjsfHVNUbo=
github.com/gobwas/ws v1.0.2/go.mod h1:szmBTxLgaFppYjEmNtny/v3w89xOydFnnZMcgRRu/EM=
github.com/goccy/go-json v0.9.11 h1:/pAaQDLHEoCq/5FFmSKBswWmK6H0e8g4159Kc/X/nqk=
github.com/goccy/go-json v0.9.11/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/goccy/go-json v0.10.0 h1:mXKd9Qw4NuzShiRlOXKews24ufknHO7gx30lsDyokKA=
github.com/golang/protobuf v1.3.5 h1:F768QJ1E9tib+q5Sc8MkdJi1RxLTbRcTf8LJV56aRls=
github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
github.com/google/go-cmp v0.4.0 h1:xsAVV57WRhGj6kEIi8ReJzQlHHqcBYCElAvkovg3B/4=
Expand All @@ -17,8 +16,8 @@ github.com/gorilla/schema v1.2.0/go.mod h1:kgLaKoK1FELgZqMAVxx/5cbj0kT+57qxUrAlI
github.com/gorilla/websocket v1.4.1 h1:q7AeDBpnBk8AogcD4DSag/Ukw/KV+YhzLj2bP5HvKCM=
github.com/gorilla/websocket v1.4.1/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/klauspost/compress v1.10.3/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.15.9 h1:wKRjX6JRtDdrE9qwa4b/Cip7ACOshUI4smpCQanqjSY=
github.com/klauspost/compress v1.15.9/go.mod h1:PhcZ0MbTNciWF3rruxRgKxI5NkcHHrHUDtV4Yw2GlzU=
github.com/klauspost/compress v1.15.12 h1:YClS/PImqYbn+UILDnqxQCZ3RehC9N318SU3kElDUEM=
github.com/switchupcb/disgo v0.0.0-20220726210903-501dcb40b012 h1:8K0Cs/qWX0p21TH6WYeb9ihYT+nLzabfsGyfm/g/Xv4=
github.com/switchupcb/disgo v0.0.0-20220726210903-501dcb40b012/go.mod h1:Wl43EK7HOIj+qYkKGY7bfmhHcs4kOamgHeMk8ZujROQ=
github.com/switchupcb/disgo/tools v0.0.0-20220727073006-e0f01397d626 h1:M9XQfGcnMXzlAXWFzxTLnwY65Y0K4VD4LlsyPt55MSQ=
Expand All @@ -27,8 +26,7 @@ github.com/switchupcb/websocket v1.8.8 h1:0x7RIs90NJ8YggqcLdKeb/LTofJ1BY79n784pk
github.com/switchupcb/websocket v1.8.8/go.mod h1:HdhyzCLfOFPrBv+QNcnDSbv8L8rfJ7ZCulrBKZJHip0=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasthttp v1.41.0 h1:zeR0Z1my1wDHTRiamBCXVglQdbUwgb9uWG3k1HQz6jY=
github.com/valyala/fasthttp v1.41.0/go.mod h1:f6VbjjoI3z1NDOZOv17o6RvtRSWxC77seBFc2uWtgiY=
github.com/valyala/fasthttp v1.43.0 h1:Gy4sb32C98fbzVWZlTM1oTMdLWGyvxR03VhM6cBIU4g=
github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
golang.org/x/crypto v0.0.0-20220214200702-86341886e292/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
Expand Down
2 changes: 1 addition & 1 deletion _examples/image/avatar/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ import (
"net/http"
"os"

"github.com/switchupcb/disgo"
"github.com/switchupcb/disgo/tools"
disgo "github.com/switchupcb/disgo/wrapper"
)

// Environment Variables
Expand Down
2 changes: 1 addition & 1 deletion _examples/message/send/file.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"os"
"path"

disgo "github.com/switchupcb/disgo/wrapper"
"github.com/switchupcb/disgo"
)

// getFile returns a disgo.File for usage in a message.
Expand Down
2 changes: 1 addition & 1 deletion _examples/message/send/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"log"
"os"

disgo "github.com/switchupcb/disgo/wrapper"
"github.com/switchupcb/disgo"
)

// Environment Variables
Expand Down
37 changes: 30 additions & 7 deletions _gen/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,19 @@ Disgo uses generators to easily update and maintain over 10,000 lines of code.

## Build

Use `go build -o gen` to build the executable file for the generator. This may require you to set the `GOWORK` environment variable to `off`.
Use `go build -o gen` from the [`./_gen`](/_gen) directory to build the executable file for the generator. This may require you to set the `GOWORK` environment variable to `off`.

### Dependencies

_The following dependencies are used to download and unzip the latest version of [`dasgo`](https://github.com/switchupcb/dasgo) to `./gen/input/dasgo-10` (git-ignored)._

- [curl](https://curl.se/): [download](https://curl.se/download.html)
- [`unzip`](https://linux.die.net/man/1/unzip)

_The following dependencies are used during code generation._

- [`xstruct`](/tools/xstruct.exe)
- [copygen](https://github.com/switchupcb/copygen): `go install github.com/switchupcb/copygen@latest`

## Dasgo

Expand All @@ -21,9 +33,20 @@ Disgo sources Discord API objects from [dasgo](https://github.com/switchupcb/das

Disgo generates code for features using [copygen](https://github.com/switchupcb/copygen). **This requires corresponding `setup.go` files to be updated.** Use the `diff` from Git to update those files accordingly.

| Step | Description |
| :----------- | :----------------------------------------------------------------- |
| `send.go` | Uses copygen to generate request `Send()` functions. |
| `handle.go` | Uses copygen to generate request **event handling** functionality. |
| `command.go` | Uses copygen to generate request `Command()` functions. |
| Clean | Cleans the generated code. |
| Step | Description |
| :------------- | :----------------------------------------------------------------- |
| `send.go` | Uses copygen to generate request `Send()` functions. |
| `handle.go` | Uses copygen to generate request **event handling** functionality. |
| `sendevent.go` | Uses copygen to generate request `SendEvent()` functions. |
| Clean | Cleans the generated code. |

# Bundle

## Build

A bundler is used to package the API Wrapper into the `disgo` package (`disgo.go`). Use `go build` from the [`./_gen/bundle`](/_gen/bundle) directory to build the executable file for the bundler. This may require you to set the `GOWORK` environment variable to `off`.

### Dependencies

- [bundle](https://pkg.go.dev/golang.org/x/tools/cmd/bundle): `go install golang.org/x/tools/cmd/bundle@latest`
- [fieldalignment](https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/fieldalignment): `go install golang.org/x/tools/go/analysis/passes/fieldalignment/cmd/fieldalignment@latest`
115 changes: 115 additions & 0 deletions _gen/bundle/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
package main

import (
"flag"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"

"github.com/switchupcb/disgo/_gen/bundle/tools"
)

var (
showFieldaligned = flag.Bool("f", false, "Use -f to showcase the output from the fieldalignment tool.")
)

const (
exeDir = "_gen/bundle"
bundlePath = "disgo.go"
pkg = "package disgo"
filemodewrite = 0644
)

func main() {
if err := check(); err != nil {
fmt.Printf("%v", err)
os.Exit(1)
}

flag.Parse()

// disgo generation
if err := generate(); err != nil {
fmt.Printf("%v", err)
os.Exit(1)
}
}

// check checks that the current working directory is `disgo/_gen/bundle`.
func check() error {
cwd, err := os.Getwd()
if err != nil {
return fmt.Errorf("error getting the current working directory: %w", err)
}

cwdDir := filepath.Dir(cwd)
base := filepath.Base(cwdDir) + "/" + filepath.Base(cwd)
if base != exeDir && filepath.Base(filepath.Dir(cwdDir)) != "disgo" {
return fmt.Errorf("This executable must be run from disgo/" + exeDir)
}

return nil
}

// generate generates the disgo bundle.
func generate() error {
if err := os.Chdir("../../"); err != nil {
return fmt.Errorf("chdir: %w", err)
}

// clear the bundled file.
bundle := `//go:generate bundle -o disgo.go -dst . -pkg disgo -prefix "" ./wrapper`
cleared := strings.Join([]string{bundle, pkg}, "\n")
if err := os.WriteFile(bundlePath, []byte(cleared), filemodewrite); err != nil {
return fmt.Errorf("clear: %w", err)
}

bundlegen := exec.Command("go", "generate")
std, err := bundlegen.CombinedOutput()
if err != nil {
return fmt.Errorf("bundle (go generate): %v", string(std))
}

// fix the imports of the bundle.
if err := tools.Imports(bundlePath); err != nil {
return fmt.Errorf("imports: %w", err)
}

// fieldalign the bundle (until there is nothing else to fieldalign).
var fieldalignmentOutput []byte
for i := 0; i < 5; i++ {
fieldalignment := exec.Command("fieldalignment", "-fix", bundlePath)
std, err = fieldalignment.CombinedOutput()
if err != nil && err.Error() == "exit status 3" {
fieldalignmentOutput = append(fieldalignmentOutput, std...)
fmt.Printf("WARNING (fieldalignment): %v\n", err)
fmt.Println("running fieldalignment again...\n")

continue

} else if err != nil {
return fmt.Errorf("fieldalignment: %v", err)
}

fieldalignmentOutput = append(fieldalignmentOutput, std...)

if i == 4 {
return fmt.Errorf("fieldalignment: more calls to fieldalignment were required.")
}

break
}

if *showFieldaligned {
fmt.Println(string(fieldalignmentOutput))
}

// add removed comments to the bundle.
if err := tools.Replace(bundlePath); err != nil {
return fmt.Errorf("replace: %w", err)
}

return nil
}
50 changes: 50 additions & 0 deletions _gen/bundle/tools/imports.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
package tools

import (
"fmt"
"go/format"
"os"
"strings"
)

const (
filemodewrite = 0644
)

var (
// skip represents a set of imports to remove.
skip = map[string]bool{
`"encoding/json"`: true,
}
)

// Imports fixes the imports of the bundler.
func Imports(filepath string) error {
data, err := os.ReadFile(filepath)
if err != nil {
return fmt.Errorf("error reading generated %v file: %w", filepath, err)
}

var output []byte

content := string(data)
for _, line := range strings.Split(content, "\n") {
if skip[strings.TrimSpace(line)] {
continue
}

output = append(output, []byte(line+"\n")...)
}

// gofmt
fmtdata, err := format.Source(output)
if err != nil {
return fmt.Errorf("error formatting generated code: %w", err)
}

if err = os.WriteFile(filepath, fmtdata, filemodewrite); err != nil {
return fmt.Errorf("error writing file: %w", err)
}

return nil
}
Loading

0 comments on commit 6ad3efa

Please sign in to comment.