LidarTools#
The laspy_X modules are universal Python3 scripts, which are completely open-source and can be applied on any platform (Window, Linux). However, laspy may crash with larger las files (> 1 GB), and in particular, when the available system memory is small. For these reasons, preferably use the inter-platform and open source laspy_X modules, but if you need to deal with large las files on a weak system, use the Windows-only lastools. Note that lastools builds on LAStools from rapidlasso. It might be possible to run LAStools on Linux with wine (not yet tested with flusstools).
LasPy#
The laspy_X modules extract geospatial information from las files and convert them to ESRI shapefiles or GeoTIFF rasters. las is the typical file format for storing airborne lidar (Light Detection and Ranging) data. The flusstools laspy_X modules make use of the inter-platform and open source laspy Python package. The currently implemented capacities involve:
A point shapefile with user-defined point attributes such as intensity, waveform, or nir.
Digital elevation model (DEM) with user-defined resolution (pixel size).
GeoTIFF rasters with user-defined resolution (pixel size) for any attribute of a las file (e.g., intensity, waveform, or nir).
Computation Power and Memory Errors
Las files can be very large and the laspy_X modules load entire las files in the system memory. A large las file (> 1 GB) may result in your system shutting down Python because it is eating more memory than available. Therefore, consider using las file subsets or computers with large memory. Read more about memory errors in the Troubleshooting section (see below :ref:`memory_error`_).
Usage#
Basics#
To convert a las file to an ESRI shapefile or GeoTIFF, load flusstools.lidartools.laspy_main in Python:
import flusstools.lidartools.laspy_main as hylas
las_file_name = "path/to/a/las-file.las"
methods = ["las2shp", "las2tif"]
hylas.process_file(las_file_name, epsg=3857, methods=methods)
The above code block defines a las_file_name
variable and methods
to be used with flusstools.lidartools.laspy_main.process_file
(see Las File Main Script). The function accepts many more optional arguments:
Loads a las-file and convert it to another geospatial file format (keyword arguments **opts
).
- param source_file_name
Full directory of the source file to use with methods * if method=”las2*” > provide a las-file name * if method=”shp2*” > provide a shapefile name
- type source_file_name
str
- param epsg
Authority code to use (try
hylas.lookup_epsg(las_file_name)
to look up the epsg online).- type epsg
int
- keyword create_dem
default: False - set to True for creating a digital elevation model (DEM)
- kwtype create_dem
bool
- keyword extract_attributes
Attributes to extract from the las-file available in pattr (config.py)
- kwtype extract_attributes
str
- keyword methods
Enabled list strings are las2shp, las2tif, shp2tif, las2dem
- kwtype methods
list [str]
- keyword overwrite
Overwrite existing shapefiles and/or GeoTIFFs (default:
True
).- kwtype overwrite
bool
- keyword pixel_size
Use with *2tif to set the size of pixels relative to base units (pixel_size=5 > 5-m pixels)
- kwtype pixel_size
float
- keyword shapefile_name
Name of the point shapefile to produce with las2*
- kwtype shapefile_name
str
- keyword tif_prefix
Prefix include folder path to use for GeoTiFFs (defined extract_attributes are appended to file name)
- kwtype tif_prefix
str
- keyword interpolate_gap_pixels
Fill empty pixels that are not touched by a shapefile point with interpolated values (default:
True
)- kwtype interpolate_gap_pixels
bool
- keyword radius1
Define the x-radius for interpolating pixels (default:
-1
, corresponding to infinity). Only applicablewith interpolate_gap_pixels
.- kwtype radius1
float
- keyword radius2
Define the y-radius for interpolating pixels (default:
-1
, corresponding to infinity). Only applicablewith interpolate_gap_pixels
.- kwtype radius2
float
- keyword power
Power of the function for interpolating pixel values (default:
1.0
, corresponding to linear).- kwtype power
float
- keyword smoothing
Smoothing parameter for interpolating pixel values (default:
0.0
).- kwtype smoothing
float
- keyword min_points
Minimum number of points to use for interpolation. If the interpolator cannot find at least
min_points
for a pixel, it assigns ano_data
value to that pixel (default:0
).- kwtype min_points
int
- keyword max_points
Maximum number of points to use for interpolation. The interpolator will not use more than
max_points
closest points to interpolate a pixel value (default:0
).- kwtype max_points
int
- returns
True
if successful,False
otherwise- rtype
bool
Note
The LasPoint
class (see Las File Main Script) can also be directly called in any script with laspy_processor.LasPoint
. Have a look at the laspy_processor.process_file
function (Las File Main Script) to see how an instance of the LasPoint
class is used.
Application example#
The following code block converts a file called las-example.las first into a shapefile and then into a GeoTIFF. By using the attributes "aci"
, the scan_angle
(a
), the classification_flags
(c
), and the intensity
(i
) are extracted from the las file. Find out more about applicable attributes in the flusstools.lidartools.laspy_config.wattr dictionary (see below :ref:`laspy_config`_).
import flusstools.lidartools.laspy_main as hylas
import os
las_file_name = os.path.abspath("") + "/data/las-example.las"
shp_file_name = os.path.abspath("") + "/data/example.shp"
epsg = 25832
methods = ["las2tif"]
attribs = "aci"
px_size = 2
tif_prefix = os.path.abspath("") + "/data/sub"
hylas.process_file(
las_file_name,
epsg=epsg,
methods=methods,
extract_attributes=attribs,
pixel_size=px_size,
shapefile_name=shp_file_name,
tif_prefix=tif_prefix
)
Note
The method las2tif
automatically calls the las2shp
(flusstools.lidartools.laspy_processor.LasPoint.export2shp
) method because the GeoTIFF pixel values are extracted from the attribute table of the point shapefile. So las2shp
is the baseline for any other operation.
Code Documentation#
Las File Main Script#
Main script for las file processing. Use as:
process_file(source_file_name, epsg, **opts)
(more about arguments in the function doc below)
- flusstools.lidartools.laspy_main.lookup_epsg(file_name)[source]#
Starts a google search to retrieve information from a file name (or other
str
) with information such as UTM32.- Parameters
file_name (str) – file name or other string with words separated by “-” or “_”
Notes
This function opens a google search in the default web browser.
More information about projections, spatial reference systems, and coordinate systems
can be obtained with the geo_utils package.
process_file(source_file_name, epsg, **opts)
Loads a las-file and converts it to another geospatial file format (keyword arguments **opts
).
Note that this function documentation is currently manually implemented because of Sphinx having troubles to look behind decorators.
- Arguments:
- source_file_name (
str
): Full directory of the source file to use with methods if
method="las2*"
: provide a las-file nameif
method="shp2*"
: provide a shapefile name
- source_file_name (
epsg (
int
): Authority code to use (tryhylas.lookup_epsg(las_file_name)
to look up the epsg online).
- Keyword Arguments (
**opts
): create_dem (
bool
): Set to True for creating a digital elevation model (DEM - default:False
)extract_attributes (
str
): Attributes to extract from the las-file available inpattr
(config.py
)methods (
list
[str
]): Enabled list strings arelas2shp
,las2tif
,shp2tif
,las2dem
.overwrite (
bool
): Overwrite existing shapefiles and/or GeoTIFFs (default:True
).pixel_size (
float
): Use with *2tif to set the size of pixels relative to base units (pixel_size=5
indicates 5x5-m pixels)shapefile_name (
str
): Name of the point shapefile to produce withlas2*
tif_prefix (
str
): Prefix include folder path to use for GeoTiFFs (defined extract_attributes are appended to file name)interpolate_gap_pixels (
bool
): Fill empty pixels that are not touched by a shapefile point with interpolated values (default:True
)radius1 (
float
): Define the x-radius for interpolating pixels (default:-1
, corresponding to infinity). Only applicablewith interpolate_gap_pixels
.radius2 (
float
): Define the y-radius for interpolating pixels (default:-1
, corresponding to infinity). Only applicablewith interpolate_gap_pixels
.power (
float
): Power of the function for interpolating pixel values (default:1.0
, corresponding to linear).smoothing (
float
): Smoothing parameter for interpolating pixel values (default:0.0
).min_points (
int
): Minimum number of points to use for interpolation. If the interpolator cannot find at leastmin_points
for a pixel, it assigns ano_data
value to that pixel (default:0
).max_points (
int
): Maximum number of points to use for interpolation. The interpolator will not use more thanmax_points
closest points to interpolate a pixel value (default:0
).
- Returns:
bool
:True
if successful,False
otherwise.
More information on pixel value interpolation:
* interpolate_gap_pixels=True
interpolates values at pixels that are not touched by any las point.
* The pixel value interpolation uses gdal_grid
(i.e., its Python bindings through gdal.Grid()
).
* Control the interpolation parameters with the keyword arguments radius1
, radius2
, power
, max_points
, min_points
, and smoothing
.
See also
All variables are illustratively explained on the GDAL website.
Las Processor#
- class flusstools.lidartools.laspy_processor.LasPoint(las_file_name, epsg=3857, use_attributes='aciw', overwrite=True)[source]#
Las file container to convert datasets to ESRI point shapefiles and/or GeoTIFFs.
- Parameters
las_file_name (str) – Directory to and name of a las file.
epsg (int) – Authority Code - Geodetic Parameter Dataset ID (default:
3857
).overwrite (bool) – Overwrite existing shapefiles and/or GeoTIFFs (default:
True
).use_attributes (str) – Attributes (properties) to use from the las-file available in pattr (config.py). (default:
use_attributes="aciw"
).
- Variables
las_file (laspy.file.File) – A laspy file object
attributes (str) – Defined with
use_attributes
epsg (int) – Authority code
gdf (geopandas.GeoDataFrame) – geopandas data frame containing all points of the las file with the properties (columns) defined by
use_attributes
offset (laspy.file.File().header.offset) – Offset of las points (auto-read)
overwrite (bool) – Enable or disable overwriting existing files (default:
True
)scale (laspy.file.File().header.scale) – Scale of las points relative to the offset (auto-read)
shapefile_name (str) – The name and dicrectorty of a point shapefile where all las-file data is stored
srs (osr.SpatialReference) – The geo-spatial reference imported from
epsg
- create_dem(target_file_name='', pixel_size=1.0, **kwargs)[source]#
Creates a digital elevation model (DEM) in GeoTIFF format from the las file points.
- Parameters
- Keyword Arguments
src_shp_file_name (str) – Name of a shapefile from which elevation information is to be extracted (default: name of the las-point shapefile)
elevation_field_name (str) – Name of the field from which elevation data is to be extracted (default:
"elevation"
)interpolate_gap_pixels (bool) – Fill empty pixels that are not touched by a shapefile point with interpolated values (default:
False
)radius1 (float) – Define the x-radius for interpolating pixels (default:
-1
, corresponding to infinity). Only applicablewith interpolate_gap_pixels
.radius2 (float) – Define the y-radius for interpolating pixels (default:
-1
, corresponding to infinity). Only applicablewith interpolate_gap_pixels
.power (float) – Power of the function for interpolating pixel values (default:
1.0
, corresponding to linear).smoothing (float) – Smoothing parameter for interpolating pixel values (default:
0.0
).min_points (int) – Minimum number of points to use for interpolation. If the interpolator cannot find at least
min_points
for a pixel, it assigns ano_data
value to that pixel (default:0
).max_points (int) – Maximum number of points to use for interpolation. The interpolator will not use more than
max_points
closest points to interpolate a pixel value (default:0
).
Hint
This function works independently and does not require the prior creation of a shapefile.
- Returns
0
if successful, otherwise-1
- Return type
Analysis Configuration#
This is the hylas config file
- flusstools.lidartools.laspy_config.pattr = {'C': 'classification', 'G': 'gps_time', 'N': 'num_returns', 'R': 'return_num', 'W': 'waveform_packet_size', 'a': 'scan_angle', 'b': 'blue', 'c': 'classification_flags', 'e': 'edge_flight_line', 'g': 'green', 'i': 'intensity', 'n': 'nir', 'r': 'red', 's': 'scan_dir_flag', 'u': 'user_data', 'w': 'wave_packet_desc_index'}#
dict
of attributes to extract data layers (shapefile columns or multiple GeoTIFFs) from a las file.All attributes defined in
pattr.values()
must be an attribute of a las_file object. Print all available las file attributes with:print(dir(LasPoint.las_file))
- Type
dict
- flusstools.lidartools.laspy_config.wattr = {'C': 'Class', 'G': 'GPStime', 'N': 'NumberRet', 'R': 'ReturnNumber', 'W': 'WaveSize', 'a': 'ScanAngle', 'b': 'Blue', 'c': 'ClassFlag', 'e': 'FlightEdge', 'g': 'Green', 'i': 'Intensity', 'n': 'NIR', 'r': 'Red', 's': 'ScanDir', 'u': 'UserData', 'w': 'WaveformDesc'}#
dict
with column headers (shapefile attribute table) and GeoTIFF file names to use for parsing attributes.- Type
dict
Troubleshooting#
Memory Errors#
MemoryError
Cause: las files may have a size of several GiB, which may quickly cause a MemoryError
(e.g., MemoryError: Unable to allocate 9.1 GiB for an array with shape ...
). In particular, the Linux kernel will not attempt to run actions that exceed the commit-able memory.
- Solution: Enable memory over-committing:
- Check the current over-commit mode in Terminal:
cat /proc/sys/vm/overcommit_memory
If
0
is the answer, the system calculates array dimensions and the required memory (e.g., an array with dimensions(266838515, 12, 49)
requires a memory of266838515 * 12 *49 / 1024.0**3
=146
GiB, which is unlikely to fit in the memory).- To enable over-committing, set the commit mode to
1
: echo 1 | sudo tee /proc/sys/vm/overcommit_memory
- To enable over-committing, set the commit mode to
Alternative Solution: Use LasTools (see below), which has better capacity to deal with system memory limitations, but works on Windows only (not yet tested: implementation of LasTools in Linux with wine).
LasTools (Windows only)#
lastools is forked from GCS_scripts by Kenny Larrieu. The original code is designed for Python2 and the commercial arcpy
library. The tweaked codes of las4windows run with Python 3.8 and work without arcpy
. This repository only uses the GUI for lidar processing with LASTools.
Because LASTools is proprietary, its executables can hardly be run on Linux or other UNIX-based systems (not yet tested: implementation of LasTools in Linux with wine). This is why LasTools is a Windows-only application.
Use the GUI
Launch flusstools.lidartools.lastools_GUI.create_gui()
to open a graphical user interface that walks you through the lastools scripts implemented in flusstools, and calls relevant functions by a simple mouse click.
Additional requirements#
LASTools is used for LiDAR Data Processing and can be downloaded here.
Usage#
The main function to start processing a las or laz file with lastools is process_lidar
, which can be called as follows:
import flusstools.lidartools.lastools_core as lastools
lastools.process_lidar(
lastoolsdir="C:/dir/to/LAStools/bin",
lidardir="C:/LiDAR/file/directory", # las or laz file
ground_poly="C:/dir/to/Ground-area-shp-file (optional)", # limit the analysis region
cores=4, # numbers of cores to use
units_code="Meters", # alternative: "Feet"
keep_orig_pts=False, # Keep original ground/veg points (True or False)
coarse_step="", # numeric as string (do not use None)
coarse_bulge="", # numeric as string (do not use None)
coarse_spike="", # numeric as string (do not use None)
coarse_down_spike="", # numeric as string (do not use None)
coarse_offset="", # numeric as string (do not use None)
fine_step="", # numeric as string (do not use None)
fine_bulge="", # numeric as string (do not use None)
fine_spike="", # numeric as string (do not use None)
fine_down_spike="", # numeric as string (do not use None)
fine_offset="" # numeric as string (do not use None)
)
Alternatively, lastools can be started as a graphical user interface as follows (from Windows Prompt):
cd C:\dir\to\flusstools\lidartools
python LiDAR_processing_GUI
Code Documentation#
LiDAR processing#
- Things to consider adding:
choice of las or laz output set default values for lasground_new params clip structures step lasclassify params to identify buildings use veg polygon (if given) instead of inverse ground polygon to clip veg points
- class flusstools.lidartools.lastools_core.DF(*args: Any, **kwargs: Any)[source]#
Extended pandas DataFrame class with an additional title attribute
- flusstools.lidartools.lastools_core.ar1_acorr(series, maxlags='')[source]#
Returns lag, autocorrelation, and confidence interval using geometric autocorrelation for AR1 fit of series
- flusstools.lidartools.lastools_core.cox_acorr(series, maxlags='')[source]#
- Parameters
series – (list)
maxlags – (str)
- Returns
two lists (lags and autocorrelation), using Cox variant 3 of ACF
- flusstools.lidartools.lastools_core.ft(x, y)[source]#
Returns the fourier transform magnitude of the x,y data
- flusstools.lidartools.lastools_core.lof_text(pwd, src)[source]#
creates a .txt file in pwd (LAStools bin) containing a list of .las/.laz filenames from src directory
- flusstools.lidartools.lastools_core.pd(filename)[source]#
returns point density from lasinfo output .txt file
- flusstools.lidartools.lastools_core.pts(filename, lastoolsdir)[source]#
returns number of points in las file
- flusstools.lidartools.lastools_core.r_confidence_interval(r, n, alpha=0.05)[source]#
Retrieves the confidence interval at the 1-alpha level for correlation of r with n observations when alpha=0.05, it returns the range of possible population correlations at the 95% confidence level so if 0 is not within the bounds, then the correlation is statistically significant at the 95% level
- Parameters
r – correlation (float)
n – number of observations (int)
alpha – confidence level (float)
- Returns
Confidence interval (low and high) as sequence (list or tuple) of floats.
File functions#
Description
- flusstools.lidartools.lastools_fun.browse(root, entry, select='file', ftypes=[('All files', '*')])[source]#
GUI button command opens browser window and adds selected file/folder to entry
- flusstools.lidartools.lastools_fun.get_largest(directory)[source]#
returns name of largest file in directory
- flusstools.lidartools.lastools_fun.get_las_files(directory)[source]#
returns list of all .las/.laz files in directory (at top level)