LidarTools#

The laspy_X modules are universal Python3 scripts, which are completely open-source and can be applied on any platform (Window, Linux). However, laspy may crash with larger las files (> 1 GB), and in particular, when the available system memory is small. For these reasons, preferably use the inter-platform and open source laspy_X modules, but if you need to deal with large las files on a weak system, use the Windows-only lastools. Note that lastools builds on LAStools from rapidlasso. It might be possible to run LAStools on Linux with wine (not yet tested with flusstools).

LasPy#

The laspy_X modules extract geospatial information from las files and convert them to ESRI shapefiles or GeoTIFF rasters. las is the typical file format for storing airborne lidar (Light Detection and Ranging) data. The flusstools laspy_X modules make use of the inter-platform and open source laspy Python package. The currently implemented capacities involve:

  • A point shapefile with user-defined point attributes such as intensity, waveform, or nir.

  • Digital elevation model (DEM) with user-defined resolution (pixel size).

  • GeoTIFF rasters with user-defined resolution (pixel size) for any attribute of a las file (e.g., intensity, waveform, or nir).

Computation Power and Memory Errors

Las files can be very large and the laspy_X modules load entire las files in the system memory. A large las file (> 1 GB) may result in your system shutting down Python because it is eating more memory than available. Therefore, consider using las file subsets or computers with large memory. Read more about memory errors in the Troubleshooting section (see below :ref:`memory_error`_).

Usage#

Basics#

To convert a las file to an ESRI shapefile or GeoTIFF, load flusstools.lidartools.laspy_main in Python:

import flusstools.lidartools.laspy_main as hylas
las_file_name = "path/to/a/las-file.las"
methods = ["las2shp", "las2tif"]
hylas.process_file(las_file_name, epsg=3857, methods=methods)

The above code block defines a las_file_name variable and methods to be used with flusstools.lidartools.laspy_main.process_file (see Las File Main Script). The function accepts many more optional arguments:

Loads a las-file and convert it to another geospatial file format (keyword arguments **opts).

param source_file_name

Full directory of the source file to use with methods * if method=”las2*” > provide a las-file name * if method=”shp2*” > provide a shapefile name

type source_file_name

str

param epsg

Authority code to use (try hylas.lookup_epsg(las_file_name) to look up the epsg online).

type epsg

int

keyword create_dem

default: False - set to True for creating a digital elevation model (DEM)

kwtype create_dem

bool

keyword extract_attributes

Attributes to extract from the las-file available in pattr (config.py)

kwtype extract_attributes

str

keyword methods

Enabled list strings are las2shp, las2tif, shp2tif, las2dem

kwtype methods

list [str]

keyword overwrite

Overwrite existing shapefiles and/or GeoTIFFs (default: True).

kwtype overwrite

bool

keyword pixel_size

Use with *2tif to set the size of pixels relative to base units (pixel_size=5 > 5-m pixels)

kwtype pixel_size

float

keyword shapefile_name

Name of the point shapefile to produce with las2*

kwtype shapefile_name

str

keyword tif_prefix

Prefix include folder path to use for GeoTiFFs (defined extract_attributes are appended to file name)

kwtype tif_prefix

str

keyword interpolate_gap_pixels

Fill empty pixels that are not touched by a shapefile point with interpolated values (default: True)

kwtype interpolate_gap_pixels

bool

keyword radius1

Define the x-radius for interpolating pixels (default: -1, corresponding to infinity). Only applicable with interpolate_gap_pixels.

kwtype radius1

float

keyword radius2

Define the y-radius for interpolating pixels (default: -1, corresponding to infinity). Only applicable with interpolate_gap_pixels.

kwtype radius2

float

keyword power

Power of the function for interpolating pixel values (default: 1.0, corresponding to linear).

kwtype power

float

keyword smoothing

Smoothing parameter for interpolating pixel values (default: 0.0).

kwtype smoothing

float

keyword min_points

Minimum number of points to use for interpolation. If the interpolator cannot find at least min_points for a pixel, it assigns a no_data value to that pixel (default: 0).

kwtype min_points

int

keyword max_points

Maximum number of points to use for interpolation. The interpolator will not use more than max_points closest points to interpolate a pixel value (default: 0).

kwtype max_points

int

returns

True if successful, False otherwise

rtype

bool

Note

The LasPoint class (see Las File Main Script) can also be directly called in any script with laspy_processor.LasPoint. Have a look at the laspy_processor.process_file function (Las File Main Script) to see how an instance of the LasPoint class is used.

Application example#

The following code block converts a file called las-example.las first into a shapefile and then into a GeoTIFF. By using the attributes "aci", the scan_angle (a), the classification_flags (c), and the intensity (i) are extracted from the las file. Find out more about applicable attributes in the flusstools.lidartools.laspy_config.wattr dictionary (see below :ref:`laspy_config`_).

import flusstools.lidartools.laspy_main as hylas
import os

las_file_name = os.path.abspath("") + "/data/las-example.las"
shp_file_name = os.path.abspath("") + "/data/example.shp"
epsg = 25832
methods = ["las2tif"]
attribs = "aci"
px_size = 2
tif_prefix = os.path.abspath("") + "/data/sub"

hylas.process_file(
     las_file_name,
     epsg=epsg,
     methods=methods,
     extract_attributes=attribs,
     pixel_size=px_size,
     shapefile_name=shp_file_name,
     tif_prefix=tif_prefix
)

Note

The method las2tif automatically calls the las2shp (flusstools.lidartools.laspy_processor.LasPoint.export2shp) method because the GeoTIFF pixel values are extracted from the attribute table of the point shapefile. So las2shp is the baseline for any other operation.

Code Documentation#

Las File Main Script#

Main script for las file processing. Use as:

process_file(source_file_name, epsg, **opts) (more about arguments in the function doc below)

flusstools.lidartools.laspy_main.lookup_epsg(file_name)[source]#

Starts a google search to retrieve information from a file name (or other str) with information such as UTM32.

Parameters

file_name (str) – file name or other string with words separated by “-” or “_”

Notes

  • This function opens a google search in the default web browser.

  • More information about projections, spatial reference systems, and coordinate systems

can be obtained with the geo_utils package.

process_file(source_file_name, epsg, **opts)

Loads a las-file and converts it to another geospatial file format (keyword arguments **opts).

Note that this function documentation is currently manually implemented because of Sphinx having troubles to look behind decorators.

Arguments:
  • source_file_name (str): Full directory of the source file to use with methods
    • if method="las2*": provide a las-file name

    • if method="shp2*": provide a shapefile name

  • epsg (int): Authority code to use (try hylas.lookup_epsg(las_file_name) to look up the epsg online).

Keyword Arguments (**opts):
  • create_dem (bool): Set to True for creating a digital elevation model (DEM - default: False)

  • extract_attributes (str): Attributes to extract from the las-file available in pattr (config.py)

  • methods (list [str]): Enabled list strings are las2shp, las2tif, shp2tif, las2dem.

  • overwrite (bool): Overwrite existing shapefiles and/or GeoTIFFs (default: True).

  • pixel_size (float): Use with *2tif to set the size of pixels relative to base units (pixel_size=5 indicates 5x5-m pixels)

  • shapefile_name (str): Name of the point shapefile to produce with las2*

  • tif_prefix (str): Prefix include folder path to use for GeoTiFFs (defined extract_attributes are appended to file name)

  • interpolate_gap_pixels (bool): Fill empty pixels that are not touched by a shapefile point with interpolated values (default: True)

  • radius1 (float): Define the x-radius for interpolating pixels (default: -1, corresponding to infinity). Only applicable with interpolate_gap_pixels.

  • radius2 (float): Define the y-radius for interpolating pixels (default: -1, corresponding to infinity). Only applicable with interpolate_gap_pixels.

  • power (float): Power of the function for interpolating pixel values (default: 1.0, corresponding to linear).

  • smoothing (float): Smoothing parameter for interpolating pixel values (default: 0.0).

  • min_points (int): Minimum number of points to use for interpolation. If the interpolator cannot find at least min_points for a pixel, it assigns a no_data value to that pixel (default: 0).

  • max_points (int): Maximum number of points to use for interpolation. The interpolator will not use more than max_points closest points to interpolate a pixel value (default: 0).

Returns:

bool: True if successful, False otherwise.

More information on pixel value interpolation: * interpolate_gap_pixels=True interpolates values at pixels that are not touched by any las point. * The pixel value interpolation uses gdal_grid (i.e., its Python bindings through gdal.Grid()). * Control the interpolation parameters with the keyword arguments radius1, radius2, power, max_points, min_points, and smoothing.

See also

All variables are illustratively explained on the GDAL website.

Las Processor#

class flusstools.lidartools.laspy_processor.LasPoint(las_file_name, epsg=3857, use_attributes='aciw', overwrite=True)[source]#

Las file container to convert datasets to ESRI point shapefiles and/or GeoTIFFs.

Parameters
  • las_file_name (str) – Directory to and name of a las file.

  • epsg (int) – Authority Code - Geodetic Parameter Dataset ID (default: 3857).

  • overwrite (bool) – Overwrite existing shapefiles and/or GeoTIFFs (default: True).

  • use_attributes (str) – Attributes (properties) to use from the las-file available in pattr (config.py). (default: use_attributes="aciw").

Variables
  • las_file (laspy.file.File) – A laspy file object

  • attributes (str) – Defined with use_attributes

  • epsg (int) – Authority code

  • gdf (geopandas.GeoDataFrame) – geopandas data frame containing all points of the las file with the properties (columns) defined by use_attributes

  • offset (laspy.file.File().header.offset) – Offset of las points (auto-read)

  • overwrite (bool) – Enable or disable overwriting existing files (default: True)

  • scale (laspy.file.File().header.scale) – Scale of las points relative to the offset (auto-read)

  • shapefile_name (str) – The name and dicrectorty of a point shapefile where all las-file data is stored

  • srs (osr.SpatialReference) – The geo-spatial reference imported from epsg

create_dem(target_file_name='', pixel_size=1.0, **kwargs)[source]#

Creates a digital elevation model (DEM) in GeoTIFF format from the las file points.

Parameters
  • target_file_name (str) – A file name including an existing directory where the dem will be created< must end on .tif.

  • pixel_size (float) – The size of one pixel relative to the spatial reference system

Keyword Arguments
  • src_shp_file_name (str) – Name of a shapefile from which elevation information is to be extracted (default: name of the las-point shapefile)

  • elevation_field_name (str) – Name of the field from which elevation data is to be extracted (default: "elevation")

  • interpolate_gap_pixels (bool) – Fill empty pixels that are not touched by a shapefile point with interpolated values (default: False)

  • radius1 (float) – Define the x-radius for interpolating pixels (default: -1, corresponding to infinity). Only applicable with interpolate_gap_pixels.

  • radius2 (float) – Define the y-radius for interpolating pixels (default: -1, corresponding to infinity). Only applicable with interpolate_gap_pixels.

  • power (float) – Power of the function for interpolating pixel values (default: 1.0, corresponding to linear).

  • smoothing (float) – Smoothing parameter for interpolating pixel values (default: 0.0).

  • min_points (int) – Minimum number of points to use for interpolation. If the interpolator cannot find at least min_points for a pixel, it assigns a no_data value to that pixel (default: 0).

  • max_points (int) – Maximum number of points to use for interpolation. The interpolator will not use more than max_points closest points to interpolate a pixel value (default: 0).

Hint

This function works independently and does not require the prior creation of a shapefile.

Returns

0 if successful, otherwise -1

Return type

int

export2shp(**kwargs)[source]#

Converts las file points to a point shapefile.

Keyword Arguments

shapefile_name (str) – Optional shapefile name (must end on .shp). (default: '/this/dir/las_file_name.shp').

Returns

/path/to/shapefile.shp, which is a point shapefile created by the function.

Return type

str

get_file_info()[source]#

Prints las file information to console.

Analysis Configuration#

This is the hylas config file

flusstools.lidartools.laspy_config.pattr = {'C': 'classification', 'G': 'gps_time', 'N': 'num_returns', 'R': 'return_num', 'W': 'waveform_packet_size', 'a': 'scan_angle', 'b': 'blue', 'c': 'classification_flags', 'e': 'edge_flight_line', 'g': 'green', 'i': 'intensity', 'n': 'nir', 'r': 'red', 's': 'scan_dir_flag', 'u': 'user_data', 'w': 'wave_packet_desc_index'}#

dict of attributes to extract data layers (shapefile columns or multiple GeoTIFFs) from a las file.

All attributes defined in pattr.values() must be an attribute of a las_file object. Print all available las file attributes with:

print(dir(LasPoint.las_file))
Type

dict

flusstools.lidartools.laspy_config.wattr = {'C': 'Class', 'G': 'GPStime', 'N': 'NumberRet', 'R': 'ReturnNumber', 'W': 'WaveSize', 'a': 'ScanAngle', 'b': 'Blue', 'c': 'ClassFlag', 'e': 'FlightEdge', 'g': 'Green', 'i': 'Intensity', 'n': 'NIR', 'r': 'Red', 's': 'ScanDir', 'u': 'UserData', 'w': 'WaveformDesc'}#

dict with column headers (shapefile attribute table) and GeoTIFF file names to use for parsing attributes.

Type

dict

Troubleshooting#

Memory Errors#

MemoryError

Cause: las files may have a size of several GiB, which may quickly cause a MemoryError (e.g., MemoryError: Unable to allocate 9.1 GiB for an array with shape ...). In particular, the Linux kernel will not attempt to run actions that exceed the commit-able memory.

Solution: Enable memory over-committing:
  • Check the current over-commit mode in Terminal:

    cat /proc/sys/vm/overcommit_memory

  • If 0 is the answer, the system calculates array dimensions and the required memory (e.g., an array with dimensions (266838515, 12, 49) requires a memory of 266838515 * 12 *49 / 1024.0**3 = 146 GiB, which is unlikely to fit in the memory).

  • To enable over-committing, set the commit mode to 1:

    echo 1 | sudo tee /proc/sys/vm/overcommit_memory

Alternative Solution: Use LasTools (see below), which has better capacity to deal with system memory limitations, but works on Windows only (not yet tested: implementation of LasTools in Linux with wine).

LasTools (Windows only)#

lastools is forked from GCS_scripts by Kenny Larrieu. The original code is designed for Python2 and the commercial arcpy library. The tweaked codes of las4windows run with Python 3.8 and work without arcpy. This repository only uses the GUI for lidar processing with LASTools.

Because LASTools is proprietary, its executables can hardly be run on Linux or other UNIX-based systems (not yet tested: implementation of LasTools in Linux with wine). This is why LasTools is a Windows-only application.

Use the GUI

Launch flusstools.lidartools.lastools_GUI.create_gui() to open a graphical user interface that walks you through the lastools scripts implemented in flusstools, and calls relevant functions by a simple mouse click.

Additional requirements#

LASTools is used for LiDAR Data Processing and can be downloaded here.

Usage#

The main function to start processing a las or laz file with lastools is process_lidar, which can be called as follows:

import flusstools.lidartools.lastools_core as lastools

lastools.process_lidar(
    lastoolsdir="C:/dir/to/LAStools/bin",
    lidardir="C:/LiDAR/file/directory",  # las or laz file
    ground_poly="C:/dir/to/Ground-area-shp-file (optional)",  # limit the analysis region
    cores=4,  # numbers of cores to use
    units_code="Meters",  # alternative: "Feet"
    keep_orig_pts=False,  # Keep original ground/veg points (True or False)
    coarse_step="",  # numeric as string (do not use None)
    coarse_bulge="",  # numeric as string (do not use None)
    coarse_spike="",  # numeric as string (do not use None)
    coarse_down_spike="",  # numeric as string (do not use None)
    coarse_offset="",  # numeric as string (do not use None)
    fine_step="",  # numeric as string (do not use None)
    fine_bulge="",  # numeric as string (do not use None)
    fine_spike="",  # numeric as string (do not use None)
    fine_down_spike="",  # numeric as string (do not use None)
    fine_offset=""  # numeric as string (do not use None)
)

Alternatively, lastools can be started as a graphical user interface as follows (from Windows Prompt):

cd C:\dir\to\flusstools\lidartools
python LiDAR_processing_GUI

Code Documentation#

LiDAR processing#

Things to consider adding:

choice of las or laz output set default values for lasground_new params clip structures step lasclassify params to identify buildings use veg polygon (if given) instead of inverse ground polygon to clip veg points

class flusstools.lidartools.lastools_core.DF(*args: Any, **kwargs: Any)[source]#

Extended pandas DataFrame class with an additional title attribute

flusstools.lidartools.lastools_core.ar1_acorr(series, maxlags='')[source]#

Returns lag, autocorrelation, and confidence interval using geometric autocorrelation for AR1 fit of series

flusstools.lidartools.lastools_core.cmd(command)[source]#

Executes command prompt command

flusstools.lidartools.lastools_core.cox_acorr(series, maxlags='')[source]#
Parameters
  • series – (list)

  • maxlags – (str)

Returns

two lists (lags and autocorrelation), using Cox variant 3 of ACF

flusstools.lidartools.lastools_core.ft(x, y)[source]#

Returns the fourier transform magnitude of the x,y data

flusstools.lidartools.lastools_core.lof_text(pwd, src)[source]#

creates a .txt file in pwd (LAStools bin) containing a list of .las/.laz filenames from src directory

flusstools.lidartools.lastools_core.pd(filename)[source]#

returns point density from lasinfo output .txt file

flusstools.lidartools.lastools_core.pts(filename, lastoolsdir)[source]#

returns number of points in las file

flusstools.lidartools.lastools_core.r_confidence_interval(r, n, alpha=0.05)[source]#

Retrieves the confidence interval at the 1-alpha level for correlation of r with n observations when alpha=0.05, it returns the range of possible population correlations at the 95% confidence level so if 0 is not within the bounds, then the correlation is statistically significant at the 95% level

Parameters
  • r – correlation (float)

  • n – number of observations (int)

  • alpha – confidence level (float)

Returns

Confidence interval (low and high) as sequence (list or tuple) of floats.

flusstools.lidartools.lastools_core.white_noise_acf_ci(series, maxlags='')[source]#

Returns the 95% confidence interval for white noise ACF

File functions#

Description

flusstools.lidartools.lastools_fun.browse(root, entry, select='file', ftypes=[('All files', '*')])[source]#

GUI button command opens browser window and adds selected file/folder to entry

flusstools.lidartools.lastools_fun.get_largest(directory)[source]#

returns name of largest file in directory

flusstools.lidartools.lastools_fun.get_las_files(directory)[source]#

returns list of all .las/.laz files in directory (at top level)

flusstools.lidartools.lastools_fun.split_list(list2split, break_pts)[source]#

returns list l split up into sublists at break point indices

flusstools.lidartools.lastools_fun.split_reaches(list_of_reaches, new_reach_pts)[source]#

splits l into sections where new_reach_pts contains the starting indices for each slice