Profiling PolarRoute and MeshiPhi

Results

Tip: Sort by cumtime when viewing snakeviz profile visualisations. Click a function call (line in the table) to set that as the root call. E.g. for environmental mesh building, select build_environmental_mesh.

Create Environmental Mesh

GRF Example

Code: scripts/meshiphi_build.py

Config: configs/environment/grf_example.config.json

MeshiPhi	Python	build_environmental_mesh time/s	Profile	Mesh	Optimisations
2.1.15	3.13	92.4	snakeviz	geoplot*	None
2.2.3	3.13	88.6	snakeviz	geoplot*	None
2.2.3	3.14	168.0	snakeviz	geoplot*	None
2.3.0	3.13	74.5	snakeviz		Integer-based .isel() (PR)
2.3.0	3.14	55.8	snakeviz		Integer-based .isel() (PR)
2.3.0	3.14 freethreaded**	24.2	snakeviz		Integer-based .isel() (PR) & GIL off

* Note: these all show the same mesh output.

** GIL (global interpreter lock) turned off using python-freethreading build from conda-forge; with PYTHON_GIL=0. Note this raises a warning as pandas is not marked as being safe without the GIL.

trim_datapoints for abstract scalar datasets is the function within MeshiPhi that takes up most time during both the split_to_depth and aggregate stages.

The bulk of the time is in the use of xarray.dataset.sel.

This is mainly due to the number of calls to this function, ~17k in this GRF example, or ~145k times in the real data example below.

Each call only takes 5E-6 seconds, but totals up to a large fraction of the overall runtime.

Optimising the approach here should be the first target for speedups in mesh building.

Thomas had already addressed this in (PR #97), using integer-based .isel over .sel, which in addition to using python 3.14 results in a 40% speed up.

Real Production Data

Code: scripts/meshiphi_build.py

Config: configs/environment/amsr_southern.config.json

MeshiPhi	Python	Total Time (s)	Profile	Mesh	Optimisations
2.2.3	3.13	637	snakeviz	geoplot	None
2.3.0	3.14	403	snakeviz	-	Integer-based .isel() (PR)

Add Vehicle

Code: scripts/polarroute_add_vehicle.py

Config: configs/vessels/sda.json

PolarRoute	MeshiPhi	Python	Total Time (s)	Profile	Mesh	Optimisations
1.1.8	2.1.15	3.14	17.8	snakeviz	geoplot	None
1.1.8	2.3.0	3.14	8.4	snakeviz		None
1.1.8	2.3.0	3.14	2.9	snakeviz		Fast list comprehension in get_cellbox MeshiPhi/dw/optimise_get_cellbox

Optimise Route

Code: scripts/polarroute_optimise_route.py

Config: configs/traveltime.config.json

Waypoints: configs/waypoints_example.csv

PolarRoute	MeshiPhi	Python	Compute Time (s)	Smoothing Time (s)	Profile	Mesh/Route	Optimisations
1.1.8	2.3.0	3.14	5.0	111.6	snakeviz	Unsmoothed / Smoothed (Max 2000)	None
1.1.8	2.3.0	3.14	3.8	7.2	snakeviz	Smoothed (Max 20)	Reduce max smoothing iterations to 20