FuzzyCorr#

FuzzyCorr was developed along with novel fuzzy map comparison methods to evaluate the performance of hydro-morphodynamic numerical models. The procedure and math behind the package are described in Negreiros et al. 2021 (open-access paper) .

Sediment transport and hydraulic processes can be reproduced with numerical models such as SSIIMM, Hydro_AS_2D, TELEMAC and many more. The accuracy of numerical models is assessed through comparing the simulated and the observed datasets, which constitutes a model validation. With the purpose of analyzing simulated and observed bed elevation change, two methods of comparison can be applied:

  1. Comparison via statistical methods such as RMSE (Root Mean Squared Error) or visual human comparison. However, local measures of similarity (or a similarity) like the RMSE are very sensible to uncertainty of location and amount, thus indicating low agreement even when overall patterns were adequately simulated.

  2. Visual comparison captures global similarity, which is one of the reasons why modelers often use it for model validation. Humans are capable of finding patterns without deliberately trying, and therefore, this type of comparison provides substantial advantages over local similarity measures. Nevertheless, more research has to be done to implement automated validation tools that emulate human thinking. This is necessary because human comparison is not transparent, prone to subjective interpretations, time consuming, and hardly reproducible.

In this context, the concept of fuzzy set theory has capacities to consider similarity of spatial pattern analogous to human thinking. For instance, fuzziness of location introduces a tolerance regarding spatial uncertainty in the results of hydro-morphodynamic models. To this end, fuzzy logic enables an objective validation of such models by overcoming uncertainties in the model structure, parameters and input data.

The algorithms provided with fuzzycorr address the necessity in evaluating (or validating) model performance through the use of fuzzy map comparison. Future developments aim to go beyond a one-way validation towards a two-way communication between the validation algorithms and the models. The two-way communication represents a feedback loop that will eventually enable an automated calibration of numerical hydro-morphodynamic models.

Usage#

Basics#

The following code block exemplifies the usage of fuzzycorr to explore the fuzzy correlation between two (e.g., observed and modeled) maps (in GeoTIFF format):

from flusstools import fuzzycorr as fc

Example (showcase)#

The best way to learn the usage is by examples. In the directory examples, the usage of the modules are demonstrated in a case study. Inside the folder salzach_case, the results from a hydro-morphodynamic numerical simulation ( i.e., simulated bed elevation change, deltaZ) are located in raw_data. For more details on the hydro-morphodynamic numerical refer to Beckers et al. (2020).

The following showcase scripts live in ROOT/examples/fuzzycorr-showcase/:

  • prepro_salzach.py: example of the usage of the class FuzzyPreProcessor of the module prepro.py, where vector data is interpolated and rasterized.

  • classification_salzach.py: example of the usage of the class PreProCategorization of the module prepro.py.

  • fuzzycomparison_salzach.py: example of the usage of the class FuzzyComparison of the module fuzzycomp.py, which creates a correlation (similarity) measure between simulated and observed datasets.

  • plot_salzach.py, plot_class_rasters.py and performance_salzach: example of the usage of the module plotter.py.

  • random_map: example of generating a raster following a uniform random distribution, which uses the module prepro.py.

Structure#

This package contains the following modules, which were designed in Python 3.8:

  • prepro.py includes functions for reading, normalizing and rasterizing vector data. These are preprocessing steps for fuzzy map comparison (module fuzzycomp).

  • fuzzycomp.py provides routines for fuzzy map comparison in continuous valued rasters. Refer to Hagen (2006) for more details.

  • plotter.py: Visualization routines for output and input rasters.

  • The package documentation is located in the folder docs.

Pre-processing: prepro.py#

Hints:

  • many class methods could be imported from geotools

  • already removed: clip_raster, which is a duplicate of raster_mgmt

class flusstools.fuzzycorr.prepro.CategorizationPreProcessor(raster)[source]#

Structured for … (Description to be implemented by Bea)

Parameters

(str) (raster) – path of the raster to be categorized

categorize_raster(class_bins, map_out, save_ascii=True)[source]#

Classifies the raster according to the classification bins

Parameters

  • map_out – path of the project directory

  • class_bins – list of floats

  • save_ascii – bool

Returns

saves the classified raster in the chosen directory

nb_classes(n_classes)[source]#

Generates class bins based on the Natural Breaks method

Parameters

n_classes – integer, number of classes

Returns

list of optimized bins

class flusstools.fuzzycorr.prepro.FuzzyPreProcessor(df, attribute, crs, nodatavalue, res=None, ulc=(nan, nan), lrc=(nan, nan))[source]#

Parent pre-processing structure for the comparison of numeric maps

Parameters

  • df – pandas.DataFrame, can be obtained by reading the textfile as pandas dataframe

  • (str) (crs) – name of the attribute to burn in the raster (ex.: deltaZ, Z)

  • (str) – coordinate reference system

  • (float) (res) – value to indicate nodata cells

  • (float) – resolution of the cell (cell size), is the same for x and y

  • ulc – tuple of floats, upper left corner coordinate, optional

  • lrc – tuple of floats, lower right corner coordinate, optional

array2raster(array, raster_file, save_ascii=True)[source]#

Saves a raster using interpolation

Parameters

  • (str) (raster_file) – path to save the rasterfile

  • (bool) (save_ascii) – true to save also an ascii raster

Returns None

Saves the raster with the selected filename

Hint

Function will be moved to geotools/raster_mgmt in a future release (operated by Bea)

create_polygon(shape_polygon, alpha=nan)[source]#

Creates a polygon surrounding a cloud of shapepoints

Parameters

  • (str) (shape_polygon) – path to save the shapefile

  • (float) (alpha) – excentricity of the alphashape (polygon) to be created

Returns

saves the polygon (*.shp) with the selected filename

Hint

Function can be moved to geotools/shp_mgmt

norm_array(method='linear')[source]#

Normalizes the raw data in equally distanced points depending on the selected resolution

Returns

interpolated and normalized array with selected resolution

Hint

Read more at rosskush/skspatial

plain_raster(shapefile, raster_file, res)[source]#

Converts a shapefile(.shp) to a GeoTIFF raster without normalizing

Parameters

  • (str) (raster_file) – filename with path of the input shapefile (*.shp)

  • (str) – filename with path of the output raster (*.tif)

  • (float) (res) – resolution of the cell

Returns None

saves the raster in the default directory

points_to_grid()[source]#

Creates a grid of new points in the target resolution

Returns

array of size nrow, ncol

Hints:

Read more at http://chris35wills.github.io/gridding_data/

random_raster(raster_file, save_ascii=True, **kwargs)[source]#

Creates a raster of randomly generated values

Keyword Arguments

minmax – tuple of floats, (zmin, zmax) min and max ranges for random values

Returns numpy.ndarray

array of random values within a range of the same size and shape as the original

Fuzzy map comparison core: fuzzycomp.py#

Head structure for fuzzy map comparisons

Usage: fuzzy_comparison = FuzzyComparison()

Descriptions will be updated by Bea

class flusstools.fuzzycorr.fuzzycomp.FuzzyComparison(raster_a, raster_b, neigh=4, halving_distance=2)[source]#

Performing fuzzy map comparison :param raster_a: string, path of the raster to be compared with rasterB :param raster_b: string, path of the raster to be compared with rasterA :param neigh: integer, neighborhood being considered (number of cells from the central cell), default is 4 :param halving_distance: integer, distance (in cells) to which the membership decays to its half, default is 2

fuzzy_numerical(comparison_name, save_dir, map_of_comparison=True)[source]#

Compares a pair of raster maps using fuzzy numerical spatial comparison

Parameters

  • save_dir – string, directory where to save the results

  • comparison_name – string, name of the comparison

  • map_of_comparison – boolean, create map of comparison in the project directory if True

Returns

Global Fuzzy Similarity and comparison map

fuzzy_rmse(comparison_name, save_dir, map_of_comparison=True)[source]#

Compares a pair of raster maps using fuzzy root mean square error as spatial comparison

Parameters

  • comparison_name – string, name of the comparison

  • save_dir – string, directory where to save the results of the map comparison

  • map_of_comparison – boolean, if True it creates map of of local squared errors (in the project directory)

Returns

global fuzzy RMSE and comparison map

get_neighbours(array, x, y)[source]#

Captures the neighbours and their memberships :param array: array A or B :param x: int, cell in x :param y: int, cell in y :return: np.array (float) membership of the neighbours (without mask), np.array (float) neighbours’ cells (without mask)

save_comparison_raster(array_local_measures, directory, file_name)[source]#

Create map of comparison

save_results(measure, directory, name)[source]#

Saves a results file

flusstools.fuzzycorr.fuzzycomp.f_similarity(centre_cell, neighbours)[source]#

Calculates the similarity function for each pair of values (fuzzy numerical method)

Parameters

  • centre_cell – float, cell under analysis in map A

  • neighbours – np.array of floats, neighbours in map B

Returns

np.array of floats, each similarity between each of two cells

flusstools.fuzzycorr.fuzzycomp.squared_error(centre_cell, neighbours)[source]#

Calculates the error measure fuzzy rmse

Parameters

  • centre_cell – float, cell under analysis in map A

  • neighbours – np.array of floats, neighbours in map B

Returns

np.array of floats, each similarity between each of two cells

Plot routines: plotter.py#

Plotting routines and classes for fuzzy comparison maps

class flusstools.fuzzycorr.plotter.RasterDataPlotter(path)[source]#

Class of raster for plotting

Parameters

(str) (path) – path of the raster to be plotted

make_hist(legendx, legendy, fontsize, output_file, figsize, set_ylim=None, set_xlim=None)[source]#

Creates a histogram of numerical raster

Parameters

  • (str) (output_file) – legend of the x axis of he histogram

  • (str) – legend of the y axis of he histogram

  • (int) (fontsize) – size of the font

  • (str) – path for the output file

  • (tuple) (figsize) – of integers, size of the width x height of the figure

  • (float) (set_ylim) – set the maximum limit of the y axis

  • (float) – set the maximum limit of the x axis

Returns

saves the figure of the histogram

plot_categorical_raster(output_file, labels, cmap, box=True)[source]#

Creates a figure of a categorical raster

Parameters

  • output_file – path, file path of the figure

  • (list) (labels) – of strings, labels (i.e., titles)for the categories

  • (str) (cmap) – colormap to plot the raster

  • box – boolean, if False it sets off the frame of the picture

Returns

saves the figure of the raster

plot_categorical_w_window(output_file, labels, cmap, xy, width, height, box=True)[source]#

Creates a figure of a categorical raster with a zoomed window

Parameters

  • (str) (cmap) – file path of the figure

  • (list) (labels) – of strings, labels (i.e., titles)for the categories

  • (str) – colormap to plot the raster

  • (tuple) (xy) – (x,y), origin of the zoomed window, the upper left corner

  • (int) (height) – width (number of cells) of the zoomed window

  • (int) – height (number of cells) of the zoomed window

Returns

saves the figure of the raster

plot_continuous_raster(output_file, cmap, vmax=nan, vmin=nan, box=True)[source]#

Creates a figure of a continuous valued raster

Parameters

  • output_file – path, file path of the figure

  • (str) (cmap) – colormap to plot the raster

  • (float) (vmin) – optional, value maximum of the scale, this value is used in the normalization of the colormap

  • (float) – optional, value minimum of the scale, this value is used in the normalization of the colormap

  • box – boolean, if False it sets off the frame of the picture

Returns

saves the figure of the raster

plot_continuous_w_window(output_file, xy, width, height, bounds, cmap=None, list_colors=None)[source]#

Create a figure of a raster with a zoomed window :param output_file: path, file path of the figure :param xy (tuple): (x,y) origin of the zoomed window, the upper left corner :param width (int): width (number of cells) of the zoomed window :param height (int): height (number of cells) of the zoomed window :param bounds (list): of float, limits for each color of the colormap :param cmap (str): optional, colormap to plot the raster :param list_colors (list): of colors (str), optional, as alternative to using a colormap :returns None: saves the figure of the raster

flusstools.fuzzycorr.plotter.read_raster(raster_path)[source]#

Opens a raster using rasterio

Parameters

raster_path (str) – directory and name of a raster

Returns

a numpy array of the raster

Return type

ndarray

References#