Skip to content

Commit

Permalink
Merge pull request #14 from Azure/haitao/remove_v1
Browse files Browse the repository at this point in the history
solving shared state via method receiver
  • Loading branch information
haitch committed Dec 10, 2022
2 parents b1cd10f + 46b6c79 commit f530737
Show file tree
Hide file tree
Showing 26 changed files with 929 additions and 1,781 deletions.
32 changes: 14 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,27 +3,23 @@
AsyncJob aiming to help you organize code in dependencyGraph(DAG), instead of a sequential chain.

# Concepts
**Job** is a graph describe code blocks and their connections.
- all tasks added to a job will be blocked from running, until Start() is triggered
- job can be visualized for human to understand
**JobDefinition** is a graph describe code blocks and their connections.
- jobDefinition can be visualized using graphviz, easier for human to understand.

**Step** is a individual code block which can be executed and have inputs, output.
- a step would be started once all it's dependency is finished.
- output of a step can be feed into next step as input, type is checked by go generics.
- step is wrapped in [AsyncTask](https://github.com/Azure/go-asynctask) with strongType info preserved
- you can feed parameters as a step as well.

# asyncjob v2

It is recommanded to use asyncJob v2, it separate job definition and job execution.
**JobInstance** is an instance of JobDefinition, after calling .Start() method from JobDefinition
- all Steps on the definition will be copied to JobInstance.
- each step will be executed once it's precedent step is done.
- jobInstance can be visualized as well, instance visualize contains detailed info(startTime, duration) on each step.

with v1 you create the job and run the job, job can only run once.

with v2, you can create the jobDefiniton, and start it multiple times, that will create multiple job instance.

another key different is on v2, the function provided shouldn't have a receiever object, as that would introduce shared state between multiple instances. (not a problem for v1, since job is one time use)
**StepDefinition** is a individual code block which can be executed and have inputs, output.
- StepDefinition describe it's preceding steps.
- StepDefinition contains generic Params
- output of a step can be feed into next step as input, type is checked by go generics.

no plan to keep v1, once v2 is mature enough, v1 will be removed.
**StepInstance** is instance of StepDefinition
- step is wrapped in [AsyncTask](https://github.com/Azure/go-asynctask)
- a step would be started once all it's dependency is finished.
- executionPolicy can be applied {Retry, ContextEnrichment}

# Usage

Expand Down
42 changes: 28 additions & 14 deletions error.go
Original file line number Diff line number Diff line change
@@ -1,46 +1,60 @@
package asyncjob

import (
"errors"
"fmt"
)

type JobErrorCode string

const (
ErrPrecedentStepFailure JobErrorCode = "precedent step failed"
ErrStepFailed JobErrorCode = "current step failed"
ErrStepFailed JobErrorCode = "step failed"
ErrStepNotInJob JobErrorCode = "trying to reference to a step not registered in job"
)

func (code JobErrorCode) Error() string {
return string(code)
}

type JobError struct {
Code JobErrorCode
StepError error
StepName string
Message string
Code JobErrorCode
StepError error
StepInstance StepInstanceMeta
Message string
}

func newJobError(code JobErrorCode, message string) *JobError {
return &JobError{Code: code, Message: message}
}

func newStepError(stepName string, stepErr error) *JobError {
return &JobError{Code: ErrStepFailed, StepName: stepName, StepError: stepErr}
func newStepError(code JobErrorCode, step StepInstanceMeta, stepErr error) *JobError {
return &JobError{Code: code, StepInstance: step, StepError: stepErr}
}

func (je *JobError) Error() string {
if je.Code == ErrStepFailed && je.StepError != nil {
return fmt.Sprintf("step %q failed: %s", je.StepName, je.StepError.Error())
return fmt.Sprintf("step %q failed: %s", je.StepInstance.GetName(), je.StepError.Error())
}
return je.Code.Error() + ": " + je.Message
}

func (je *JobError) Unwrap() error {
return je.StepError
}

// RootCause track precendent chain and return the first step raised this error.
func (je *JobError) RootCause() error {
// this step failed, return the error
if je.Code == ErrStepFailed {
return je.StepError
return je
}

// precendent step failure, track to the root
if je.Code == ErrPrecedentStepFailure {
precedentStepErr := &JobError{}
if !errors.As(je.StepError, &precedentStepErr) {
return je.StepError
}
return precedentStepErr.RootCause()
}

return je.Code
// no idea
return je
}
12 changes: 5 additions & 7 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,15 +1,13 @@
module github.com/Azure/go-asyncjob
module github.com/Azure/go-asyncjob/v2

go 1.18

require (
github.com/Azure/go-asyncjob/graph v0.2.0
github.com/Azure/go-asynctask v1.3.1
github.com/stretchr/testify v1.8.1
)

require (
github.com/Azure/go-asyncjob/graph v0.2.0 // indirect
github.com/Azure/go-asynctask v1.3.1 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/stretchr/objx v0.5.0 // indirect
github.com/stretchr/testify v1.8.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
2 changes: 1 addition & 1 deletion go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZb
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0 h1:1zr/of2m5FGMsad5YfcqgdqdWrIhu+EBEJRhR1U7z/c=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
Expand Down
92 changes: 0 additions & 92 deletions graph_node.go

This file was deleted.

Loading

0 comments on commit f530737

Please sign in to comment.