GridFormat
is a header-only C++ library for reading/writing data from/to standardized grid file formats that are supported by visualization tools such as e.g. ParaView.
Thus, applications that operate on grid-like data structures such as numerical simulations, GIS or computational geometry applications, can leverage GridFormat
to import/export their data from/into interoperable file formats.
The typical use case for GridFormat
is within codes for numerical simulations, for visualization of results or for importing
them for further processing.
A variety of simulation frameworks exist, such as
Dune, DuMuX, Deal.II, Fenics
or MFEM,
which usually provide mechanisms to export/import data produced with the framework into some file formats. However, there are situations
in which one wants to use a format that the framework does not support, or, use some features of the format specification that are
not implemented in the framework. GridFormat
aims to provide access to a variety of file formats through a unified interface and
without the need to convert any data or to use a specific data structure to represent computational grids. Using generic programming
and traits classes, GridFormat
fully operates on the user-given data structures, thereby minimizing the runtime overhead.
GridFormat
also supports writing files from parallel computations that use MPI.
Ideally, simulation frameworks use GridFormat
under-the-hood to avoid duplicate implementation efforts, and implement support for new
formats into GridFormat
such that they are directly available to all other frameworks that utilize it.
Currently, GridFormat
is focused on VTK file formats. However, the API is
suitable for any grid format describing one of the supported grid concepts. Contributions are welcomed,
see below for information on how to contribute.
Prerequisites:
- C++-compiler with C++-20-support (tests run with
gcc-12/13
,clang++-16
) cmake
(tests run with cmake-3.26)
It is easiest to integrate GridFormat
either as a git submodule
or via the FetchContent
module of cmake
. A minimal example (using FetchContent
) of a project using GridFormat
to
write a VTU file and read it back in may look like this:
cmake_minimum_required(VERSION 3.22)
project(some_app_using_gridformat)
include(FetchContent)
FetchContent_Declare(
gridformat
GIT_REPOSITORY https://github.com/dglaeser/gridformat
GIT_TAG main
GIT_PROGRESS true
GIT_SHALLOW true
GIT_SUBMODULES_RECURSE OFF
)
FetchContent_MakeAvailable(gridformat)
add_executable(my_app my_app.cpp)
target_link_libraries(my_app PRIVATE gridformat::gridformat)
#include <array>
#include <vector>
#include <gridformat/gridformat.hpp>
double f(const std::array<double, 2>& x) { return x[0]*x[1]; }
int main () {
// For this example we have no user-defined grid type. Let's just use a predefined one...
GridFormat::ImageGrid<2, double> grid{
{1.0, 1.0}, // domain size
{10, 12} // number of cells (pixels) in each direction
};
// This shows the `GridFormat` API: Construct a writer for the desired format, and add
// point/cell fields as lambdas. Metadata can be added directly and can be ranges or scalars.
// If the lambdas are not suitable or inefficient for your data structures, you can also pass
// a (custom) implementation of `GridFormat::Field` (see documentation)
GridFormat::Writer writer{GridFormat::vtu, grid};
writer.set_meta_data("some_metadata", "i am metadata");
writer.set_point_field("point_field", [&] (const auto& point) { return f(grid.position(point)); });
writer.set_cell_field("cell_field", [&] (const auto& cell) { return f(grid.center(cell)); });
const auto written_file = writer.write("my_test_file"); // extension is added by the writer
// read the data back in (here we create a generic reader, but you can also select specific ones)
auto reader = GridFormat::Reader::from(written_file);
std::vector<double> cell_field_values(reader.number_of_cells());
std::vector<double> point_field_values(reader.number_of_points());
reader.cell_field("cell_field")->export_to(cell_field_values);
reader.point_field("point_field")->export_to(point_field_values);
return 0;
}
Many more formats, options and functions are available, see the API documentation
or have a look at the examples. Moreover, if your data is cumbersome to access via
lambdas (as done in the example above), you may also provide a custom implementation of the Field
interface.
The recommended way of using GridFormat
is to include it via cmake's FetchContent
module (see quickstart).
However, if you want to install GridFormat
locally into a custom location, clone the repository, enter the
folder and type
cmake -DCMAKE_INSTALL_PREFIX=$(pwd)/install \
-DCMAKE_C_COMPILER=/usr/bin/gcc-12 \
-DCMAKE_CXX_COMPILER=/usr/bin/g++-12 \
-B build
cmake --install build
Note that you can omit the explicit definition of C_COMPILER
and CXX_COMPILER
in case your default compiler is compatible.
Moreover, for a system-wide installation you may omit the definition of CMAKE_INSTALL_PREFIX
. After installation, you can
use cmake
to link against GridFormat
in your own project:
find_package(gridformat)
target_link_libraries(... gridformat::gridformat)
GridFormat
has no required dependencies, however, some features are only available if certain dependencies are present. For
instance, the VTK-HDF file formats are only available if
HighFive
is found, which itself requires libhdf5-dev
. If the latter is found on
your system, including GridFormat
via cmake's FetchContent
(see quickstart) automatically brings in HighFive
, as it is
included in GridFormat
as a git submodule. However, when installing GridFormat
from the cloned sources (as described above),
make sure to use git clone --recursive
in case you want to use the HDF file formats.
The availability of some specific features of a file format may also depend on the availability of certain dependencies. For instance, compression of data (e.g. for the VTK-XML file formats) can only be used if the respective compression libraries are found on the system. Dependencies of those features are stated in the API documentation.
GridFormat
comes with a few command-line apps that you can build and install alongside the library.
To include them in the build, pass the option -DGRIDFORMAT_BUILD_BINARIES=ON
to cmake
when configuring (see above).
For performance reasons, you should set -DCMAKE_BUILD_TYPE=Release
when configuring. If successfully built, you can then
use the command-line apps to print information on a grid file to the terminal, or convert between different file formats.
For instance:
gridformat-info my_vti_file.vti # prints info on the contents of the vti file
gridformat-convert my_vti_file.vti vtu # converts an image grid format (.vti) to vtu format
gridformat-convert my_vti_file.vti vtu encoder=ascii # format options can be set as key-value pairs
gridformat-convert my_vti_file.vti vtu -o some_file # choose an output filename
GridFormat
does not operate on a specific grid data structure, but instead, it can be made compatible with any user-defined
grid types by implementing specializations for a few traits classes. For information on how to do this, please have a look at the
traits classes overview,
the examples,
the predefined image grid implementation
or the predefined traits for several frameworks.
GridFormat
comes with predefined traits for
dune grid views (tested dune version: 2.9),
deal.ii triangulations (tested deal.ii version: 9.4.0),
cgal triangulations in
2d and
3d (tested cgal version: 5.5.2),
dolfinx meshes and function spaces (tested dolfinx version: 0.6.0)
and mfem meshes (tested mfem version: 4.5.2).
Users of these frameworks can include these predefined traits and use GridFormat
directly
(see the examples).
When reading from grid files, GridFormat
provides access to the data as specified by the file format. These specifications may not be
sufficient in all applications. For instance, to fully instantiate a simulator for parallel computations, information on the grid
entities shared by different processes is usually required. Since these requirements are simulator-specific, any further processing has
to be done manually by the user and for their data structures. The recommended way to deal with this issue is to add any information
required for reinstantiation as data fields to the output. This way, it is readily available when reading the file.
Find answered questions, ask questions or start discussions through GitHub Discussions.
Contributions are highly welcome! For bug reports, please file an issue. If you want to contribute with features, improvements or bug fixes please fork this project and open a merge request into the main branch of this repository.
In order to configure your local copy for testing, tell cmake
to include the test suite:
# Note: you may have to set a compiler explicitly (see installation section)
cmake -DGRIDFORMAT_BUILD_TESTS=ON -B build
Afterwards, you can build and run all tests with ctest
:
# Note: use, e.g., ctest -j4 if you want to use 4 processors
cd build
make build_tests
ctest
Note that an internet connection is required for the call to cmake
as it pulls in ut on-the-fly.
Moreover, in the configure step a Python script is invoked that produces some test data using VTK.
If your Python environment does not have VTK
, this step is skipped. Note that some tests in the test suite will be skipped in this case.
To create a release, you may use the utility script util/update_versions.py
, which creates a git tag and adjusts the versions and
release dates specified in the cmake setup and the CITATION.cff
file. For each release we maintain a separate branch for bugfixes
and patch releases. As an example, to create a release version 1.2.3
, you may type the following into the console (assumes a clean
repository):
git switch --create releases/1.2.3
# ... maybe continue development ...
python3 util/update_versions.py -v 1.2.3 # modifies versions&dates and creates a commit + tag
git push origin releases/1.2.3
git push origin v1.2.3
Afterwards, a release workflow will be triggered. If this runs through successfully, a release has been created. If not, the new tag
has to be deleted and the procedure has to be repeated after fixing the errors. After a successful release, the version on main
should be increased (without triggering an actual release). Following the above example, you may run the following commands:
git switch main
git switch --create feature/bump-version
python3 util/update_versions.py -v 1.2.4 --skip-tag # only modifies versions, no commit or tag
git commit -m "bump version to v1.2.4" .
git push origin feature/bump-version
and pose a pull request for the changes to be incorporated in main
.
GridFormat
is licensed under the terms and conditions of the MIT License. It can be read online
or in the LICENSES/MIT.txt file.
See LICENSES/MIT.txt for full copying permissions.