ArcGIS vs QGIS etc Clipping Contest Rematch revisited
Earlier this Last year, in June, Don Meltz wrote an interesting blog “ArcGIS vs QGIS Clipping Contest Rematch” where he let compete ArcGIS and Quantum GIS in a clipping contest. The benchmark contest data set in question is a 878MB ZIP file (ContourClipTest.zip with the (guessed) EPSG Code 2260 – NAD83 / New York East (ftUS)). The blog page gained a lot of comments, even from ESRI since some ArcGIS versions crashed on this test data set.
Find below the various timings compiled from the blog and the comments:
Proprietary software
| Software | Processing time | Hardware/Software |
| ArcGIS 9.3 | crash after 1h 9min: ERROR 999999: Error executing function. Invalid Topology [4gb file limit.] Failed to execute (Clip) | unknown |
| ArcGIS 10.0 | crash likewise | unknown |
| ArcGIS 10.1 | ESRI promise to calculate it in 34 seconds in this updated version (did anyone test?) | unknown |
| GlobalMapper (version?) | 30 mins | unknown |
| GlobalMapper v11.02 | 49 sec | Windows XP w/ 3.5GB RAM |
| Manifold 8 (64bit) | 31 min | Windows XP64 16 gb. RAM and 2.33 GHz |
Note: The two GlobalMapper results are a bit funny, perhaps always minutes?
Free and Open Source Software
| Software | Processing time | Hardware/Software |
| Quantum GIS (version?; Simple features) | 4-5 min | unknown |
| GRASS GIS 7 (topological GIS) | 5 min | Dell PowerEdge 2950 from 2008, Intel Xeon 2.66GHz, 8GB RAM |
| gvSIG | to be done | |
| PostGIS | to be done |
Notes: Hope volunteers will test this also on gvSIG and PostGIS (and other FOSSGIS)! Please report…
Follow
Just to add to this…Cadcorp SIS MapModeller 8.03 minutes 40.Dell Core2 laptop that at the time was being used as a web server, had chrome open and was being used for email. Will post the result later when this test is carried out on a proper desktop.
You mean earlier last year (2011).
JGRASS benchmark here: GIS clipping benchmark: jgrasstools & uDig (on JTS & geotools):https://jgrasstechtips.blogspot.it/2012/11/gis-clipping-benchmark-jgrasstools-udig.html
PC: Intel core i7, 4 GB RAM; Win 7, 64-bit
gvSIG 1.11 – 3-4 min.
gvSIG 1.11 with SEXTANTE – 2-3 min.
QGIS 1.8 – 4-5 min.
MapWindow GIS 4.8.6 – 12 min.
My two cents.Intel Core2 Quad CPU Q6600 @ 2.40GHz 64 bits; 4 GB RAM; Kubuntu 12.04 Kernel 3.2.0-32-genericQGIS 1.8Loading shapefiles: 3:35 min.Clipping: 3:38 min.gvSIG 1.12Loading shapefiles: 0:20 min.Clipping: 3:27 min. (core geoprocess)OpenJUMP 1.5.2Loading shapefiles: “Out of Memory Error”
SpatiaLite benchmark here:https://www.gaia-gis.it/fossil/libspatialite/wiki?name=benchmark-4.0
Sorry for being carelss with using the word “benchmark”, of course it is a “contest” as stated in the title. It may become a benchmark when executed on the same machine with most software packages in question being installed on it.
Just tested on my new laptop (i7, 8gb ram) with Windows 64 and qgis-dev installed with osgeo4w.It tool 0:30 to open the shape and 2:50 to do the clip. I will try test also using Linux, GRASS and other clipping tools available in the Sextante toolbox.
and the winner is …JTS/GEOS because a number of different applications could successfully use JTS/GEOS to perform the task at hand within a reasonable amount of time.The real winner is IMHO Alessandro Furieri because he described the contest (apples and oranges, even though it was mostly JTS/GEOS vs. JTS/GEOS) most accurately.Posted by the person fiddling around with GRASS vector topology.
and the winner is …JTS/GEOS because a number of different applications could successfully use JTS/GEOS to perform the task at hand within a reasonable amount of time.The real winner is IMHO Alessandro Furieri because he described the contest problems (apples and oranges, even though it was mostly JTS/GEOS vs. JTS/GEOS) most accurately.To do it in a more comparable way, the contest should IMHO be run on a (one single dedicated) virtual machine with 1 GB RAM, a 32-bit OS, and contestants would have to provide some publicly available software version, for free or for money, which would then be installed on that testing system.Posted by the person fiddling around with GRASS vector topology.
I can confirm that using ArcGIS 10.1 SP1 with the 64bit Background Geoprocessing package installed can run the benchmark in 29s on a WS 2008 R2 8 cores machine (Xeon X3450 @2.67 Mhz), 8 Gb RAM.
see https://dl.dropbox.com/u/64549871/clip.png
I ran the whole clip on a 64 Win7 laptop (8 Gb Ram / i7 running at 3.4 Ghz) using ArcGIS 10.1 SP1 and 64 Bit Background processing in 20s.
Dell Latitude, Win7 Professional (SP1) 64Bit, Intel Core i5 2,5 GHz, 4GB RAM;with ArcGIS 10.1 (SP1)Loading shape: 1 minClipping: 57swith QGIS 1.8Loading shape: 3-5 minClipping: 2:55 min
MacBookPro, i7 2.7 GHz, 8 GB RAMOSX 10.8.2 Mountain LionQGIS 1.8.0clip: 1:50 minGRASS 6.4.2 (from William Kyngesburye):clip: process killed after 45 min :(GRASS 7.0.svn (from Michael Barton)clip: process killed after 45 min (progress bar was at about 20%)Note: the GRASS 7.0 package used in the test is dated from 01-Nov-2012, so it is not including the latest vector improvements:https://trac.osgeo.org/grass/log/grass/trunk/lib/vectorGDAL 1.9.2ogr2ogr -clipsrc StudyArea1MileBuffer.shp clip.shp Contours20Ft.shpclip: 2:16 min
Based on the currently latest GRASS 7 (r53935) with fresh speed-ups: using the same old machine from 2008 as used earlier in this original blog (see above):- import: 26 seconds including building the topology- topological overlay: 4:02 min
Here’s my test with PostGIS:
4Gb RAM, Intel Core i3 540 @ 3.07 Ghz
Ubuntu 11.10
PostgreSQL 9.0.5 on i686-pc-linux-gnu, compiled by GCC gcc-4.5.real (Ubuntu/Linaro 4.5.2-8ubuntu4) 4.5.2, 32-bit
POSTGIS=”2.0.1 r9979″ GEOS=”3.3.3-CAPI-1.7.4″ PROJ=”Rel. 4.8.0, 6 March 2012″ LIBXML=”2.7.8″
Done in 3 minutes!
For what is worth… I tested again on the same machine (i7, win 64, 16GB ram), QGIS master, ArcGIS 10 and PostGIS 2 (default configuration):
clipping:
2:10 on QGIS
crash on arcgis (after more or less 1 hour)
2:14 on PostGIS
I’ve just run this benchmark using the clip tool in Whitebox Geospatial Analysis Tools https://jblindsay.github.io/ghrg/Whitebox/), which is based on JTS and runs in parallel. The average of 10 iterations was 13.5178 seconds using a 3 GHz 8-core Xeon E5 Mac Pro with 64 GB of RAM.
Benchmark Results:
Iteration 1 = 12.04 sec.
Iteration 2 = 11.47 sec.
Iteration 3 = 17.305 sec.
Iteration 4 = 11.62 sec.
Iteration 5 = 11.489 sec.
Iteration 6 = 11.543 sec.
Iteration 7 = 18.781 sec.
Iteration 8 = 11.657 sec.
Iteration 9 = 17.611 sec.
Iteration 10 = 11.662 sec.
Average = 13.5178 sec.
Just for the record: the huge vector dataset is still available from here:
wget -c “https://web.archive.org/web/20130801000000/https://www.donmeltz.com/files/ContourClipTest.zip”