eReefs POJO Shared Library

A shared library for accessing core eReefs POJOs (plain old Java objects) from a repository. A POJO is an Entity (also referred to as a Data Object) within the system which captures the values of a domain-specific object. This library provides both file-based (for development and testing) and MongoDB-based (for Production and Test) implementations.

Table of contents

• Repository overview
• Background
  • POJOs
  • Support Classes
• Development
  • Guidelines
  • Virtual Machine
  • Supporting Services
  • Testing
• Deployment
  • Publish library
  • Using library

Repository overview

.
|-- env
|   |-- dev                             <-- Scripts useful for development/testing.
|   |-- services                        <-- Scripts and configuration for deploying services  
|                                           required for development/testing.
|-- src
|   |-- main
|       |-- java                        <-- Source code for the library.
|       |-- resources                   <-- Resources included in the packaged library.
|   |-- test
|       |-- java                        <-- Unit test cases.
|       |-- resources                   <-- Resources referenced by test cases.
|
|-- vagrant                             <-- Vagrant-specific files.
|-- pom.xml                             <-- The Maven definition file for this project.
|-- README.md                           <-- This file.
|-- Vagrantfile                         <-- The Vagrant VM definition for this project, that
                                            references the Vagrant-specific files in the 'vagrant'
                                            directory.

Background

This library groups a each POJO or Entity (and any closely related specialisations if applicable) into its own package, along with any support classes related to that POJO.

POJOs

This library provides support for a number of POJOs shared across applications in the AIMS eReefs Platform:

• Download Definition
• Product Definition
• Metadata
• Job
• Task

Download Definition

package: au.gov.aims.ereefs.pojo.definition.download

Represents a source for downloadable datasets that are consumed by the AIMS eReefs Platform. Each eReefs model consumed by the platform has a Download Definition entity, such as GBR4_v2, GBR4_BGC_924, and GBR1_2.0.

Product Definition

package: au.gov.aims.ereefs.pojo.definition.product

Represents an identifiable output of the AIMS eReefs Platform. For example, Fresh Water Exposure for GBR1 dataset. A Product will normally consist of one or (normally) more files of varying file type depending on the actual Product.

Metadata

package: au.gov.aims.ereefs.pojo.metadata

Represents the metadata associated with a specific file or group of files in the system.

Examples:

• The GBR4_v2 dataset downloaded from eReefs groups data by month, so each Netcdf file contains a month worth of data. Each file has a single Metadata entity.

Job

package: au.gov.aims.ereefs.pojo.job

Represents a grouping of Tasks that are to be executed. Currently, a Job is created by a System Actor called the Job Planner, which implements a restriction where only one (1) active Job can exist in the system at any time.

Status	Description
created	starting state signifying that the Job has been created (likely by the Job Planner) and is awaiting approval.
approval_requested	resting state signifying that the Job is awaiting approval by a team member, the timing of which is outside the control of the system.
approved	transient state signifying that the Job has been approved by a team member but execution has not yet commenced.
denied	end state signifying a team member chose to reject the approval request.
running	transient state signifying Tasks are being executed.
terminating	transient state signifying that the Tasks are being terminated, but the termination process has not yet completed.
terminated	end state signifying one or more Tasks have completed with a status of terminated.
completed	end state signifying all Tasks have completed with a status of success.
failed	end state signifying one or more Tasks have completed with a status of failed.

Task

package: au.gov.aims.ereefs.pojo.task

Represents a single executable instruction to be performed by a TaskHandler (a System Actor, such as ncAggregate or ncAnimate). A Task will relate to a specific Product.

Significant properties of a Task are:

• dependsOn - a list of other Tasks that must be executed successfully prior to the Task being executed.
• status - see the table below.

Status	Description
created	starting state signifying that the Task was created (likely by the Job Planner) and is available for execution.
assigned	transient state signifying that the Task is available for processing by the next available Task Handler (eg: ncAggregate or ncAnimate), but execution has not yet commenced.
running	transient state signifying that execution has commenced.
success	end state signifying execution has completed without an error being reported by the Handler.
failed	end state signifying execution has stopped and the Handler has reported an error.
terminated	end state signifying an external actor (System or Human) has interrupted execution of the Task, either before or during execution.

Support Classes

Each POJO is accompanied by a number of interfaces and classes for reading, writing and possibly manipulating the POJO. The following focused on the support interfaces/classes for the Metadata POJO for a concrete explanation.

• Pojo - the base implementation of a POJO, containing properties/methods applicable to all POJOs.
• Metadata - specialisation of the base POJO, adding properties/methods applicable to a Metadata POJO.
• PojoDao - the base, generic interface for classes that provide read/write functionality related to the POJO. DAO stands for "Data Access Object". This interface defined methods applicable to all POJOs/DAOs.
• MetadataDao - specialised interface that adds methods applicable to interacting with Metadata POJOs.
• MetadataDaoFileImpl - specialisation of the MetadataDao interface for file-based persistence of MetadataPojos.
• MetadataDaoMongoDbImpl - specialisation of the MetadataDao interface for MongoDB-based persistence of MetadataPojos.

Development

Guidelines

Please follow the documented Standard Github workflow and Standard JIRA workflow when working on this project.

Virtual Machine

This repository includes files for provisioning a virtual machine (Vagrant using VirtualBox) for Windows-based developers.

To provision the virtual machine defined in this project, Vagrant and VirtualBox must already be installed (outside of the scope of this README). Once this pre-requisite is met, the VM is provisioned with the following commands from the project root directory.

vagrant up

Once provisioning is complete, connect to the VM via SSH:

vagrant ssh

The remainder of this documentation will reference <project root>. When using the virtual machine, this refers to the path /vagrant.

Supporting Services

This repository includes scripts for starting and stopping a MongoDB instance for development and testing purposes.

To start the MongoDB instance:

<project root>/env/services/mongodb/start.sh

To stop the MongoDB instance:

<project root>/env/services/mongodb/stop.sh

Testing

The <project root>/src/test folder defines test cases and data for comprehensive testing of the library. Note that these tests require access to a MongoDB instance (see above).

To execute the tests:

<project root>/env/dev/maven-test.sh

This script executes the tests within a Maven Docker container that is connected to the same Docker network as the MongoDB instance started above (that is: ereefs-pojo-network). The script sets the environment variable MONGODB_URL accordingly:

MONGODB_URL="mongodb://pojo:pojo@ereefs-pojo-mongodb"

If the environmental variable is not specified, the default value is:

MONGODB_URL="mongodb://localhost:27017"

To use a different MongoDB instance, or to set up the IDE to access a different MongoDB, set/change the environment variable accordingly.

Deployment

Publish library

This repository uses Github Actions for publishing this library, and Github Packages for hosting the published library.

Git Push

When changes are pushed to Github, the push Github Action is executed. This workflow uses Maven to run the Tests defined for the project, with the results published in the Actions tab in Github.

Git Release

When development is complete, the library can be packaged and stored in Github Packages for use by other applications via Maven.

1. Increment the version in pom.xml based on the numbering convention of the project:

    <version>1.6.5</version>
   

   WARNING: the version in pom.xml file must be unique, or this workflow will fail.

2. Commit and push the changes.

3. Use the Github release mechanism to create and publish a release. This will execute the release Github Action to package and deploy the library to Github Packages. Logs are published in the Actions tab in Github.

Using library in an application

The following instructions apply ONLY to the application that is using this library via Maven, NOT to this library.

1. Add the following snippet to the pom.xml file for the application, telling Maven where to find the packaged library:

    <repository>
        <id>github_openaims</id>
        <name>GitHub Open-AIMS Maven repo</name>
        <url>https://maven.pkg.github.com/open-AIMS/*</url>
    </repository>
   

2. Generate a Github personal access token. This is required by Maven to download the Github Package, even though the repository is Public. By default, Maven uses the file $HOME/.m2/settings.xml to contain credentials for accessing repositories. Add the following text to this file, remembering to replace the values for username and password.

    <server>
        <id>github_openaims</id>
        <username> replace with your username </username>
        <password> replace with your personal access token </password>
    </server>
   

   An example can be found in the ncAggregate project.