, ,

Fourth release candidate of GRASS 6.4.0 now available

A fourth release candidate of GRASS 6.4.0 is now available.

Source code:
https://grass.osgeo.org/grass64/source/
https://grass.osgeo.org/grass64/source/grass-6.4.0RC4.tar.gz

To get the RC4 source code from SVN:
svn checkout https://svn.osgeo.org/grass/grass/tags/release_20090412_grass_6_4_0RC4

An announcement has been drafted at
https://trac.osgeo.org/grass/wiki/Release/6.4.0RC4-News
(all RC news will be merged into the final announcement later)

Key improvements of the GRASS 6.4.0 release include enhanced portability for MS-Windows (native support), hundreds of fixes, the new wxPython based portable graphical interface and much new functionality.

Release candidate management at
https://grass.osgeo.org/wiki/GRASS_6.4_Feature_Plan

We hope to get out binary packages over the next days.

/by
, ,

GRASS Project seeks Google Summer of Code students!

Want to earn some money? Participate in a long-term Open Source GIS project? Contribute your brilliant ideas? Please check out our ideas collection at https://grass.osgeo.org/wiki/GRASS_SoC_Ideas_2009 – and don’t hesitate to contact us! The student application deadline is April 3rd 2009.

/by
, , ,

Four days of the Toronto Code Sprint 2009

The Toronto Code Sprint 2009 – pure productivity:

… with great benefits for Mapserver, PostGIS, GDAL, GeoTools, …

Paul nicely summarizes the Toronto Code Sprint Lessons as well as CampToCamp.

/3 Comments/by
,

Top ten myths for Open Source in Geospatial

An interesting email thread is ongoing about Top ten myths which try to harm the reputation of Open Source GIS efforts. Michael P. Gerlek edits a Wiki page collecting the top ~10 myths and misperceptions:
https://wiki.osgeo.org/wiki/Top_Ten_Myths

Especially striking the Eduardo Kanegae‘s comment:

In 2008 I worked on a project all based on ESRI 9.2 family. At that
point I didn´t know much of ESRI products and had only worked with
foss products. Now I feel more confortable to give an opinion for
that:

# myth monopoly : every only remember the (supposed) 30% esri market
share. Remember there´s also very nice commercial products like Safe
FME, CadCorp, ManiFold, PCI, ERDAS, ENVI and others ( and sometimes
most of these are embed on ESRI packs - eg.: raster support on AG
family )

# sustainability : while every major release of ESRI will force you to
re-develop your customizations, FOSS products keep release more
compatible. Example: a MapServer 3.x developer will use the same
principles and concepts on MapServer 5.x version. But, an ArcIMS
developer had to change its base when upgrading to ArcGIS Server 9.1,
recode for adapting to ArcGIS Server 9.2 API´s and now all this
concepts will change again with 9.3 version.

# maintenance : foss product will run more closer to open standards (
eg.: OGC´s ). So, you change foss parts without re-coding your entire
solution. The cost of training a new human resource on
insert/update/delete geo-feature using ArcObjects/ArcSDE is so much
higher when compared to OGC-SFS, per example.

# support : while on FOSS communities you can have a reply on minutes,
on 'esri forum' you can your topic open for months (
https://forums.esri.com/Thread.asp?c=158&f=2284&t=251001 ,
https://forums.esri.com/Thread.asp?c=158&f=2290&t=253698 ,
https://forums.esri.com/Thread.asp?c=93&f=985&t=270205&g=1 ) and NEVER
get a solution. In my sample, we discover a bug on ArcSDE/ArcMap
9.2sp4 but this will certainly NEVER be fixed because sp6 didn´t fixed
and ESRI will probably only look for 9.3 developments for now and on.
Because non-US customers CAN´T contact ESRI directly, we can only keep
suffering with poor local support.

best regards,
--
Eduardo Kanegae

Not much to add at this point…

/2 Comments/by
, , , ,

winGRASS 6.4.0RC2 now available!

The new winGRASS 6.4.0RC2 is now available! It has been packaged into the OSGeo4W installer which also provides QGIS, Mapserver, GDAL and other goodies.

OSGeo4W has two modes – express and advanced:

  • “Express” gives you a short list of suggested packages to select from which have been widely tested.
  • “Advanced” gives a long detailed list of packages and subpackages including development versions and so forth. For the moment GRASS is available in the “advanced” install. After a period of testing it may be moved to the “express” section.

OSGeo4W Installer download:
https://download.osgeo.org/osgeo4w/osgeo4w-setup.exe
Please download it and test. The wizard will guide you through the installation process.

Find GRASS in
Advanced -> … -> Select Packages
All -> Desktop -> grass & tcltk: ActiveTcl wxpython (just click to select) -> Next …

Feedback is appreciated in two “channels”:

OSGeo4W installer related issues:

GRASS related issues:

Update 24-Jan-2009: The package was fixed to support the new wxpython graphical user interface.

BTW: The OSGeo4W team is seeking a new GRASS package maintainer, we hope to find a candidate soon – please speak up.

/by
, ,

GRASS GIS 6.4.0 RC2 released

A second release candidate of GRASS 6.4.0 is now available:

https://grass.osgeo.org/grass64/source/
https://grass.osgeo.org/grass64/source/grass-6.4.0RC2.tar.gz

To get the RC2 source code from SVN:
svn checkout https://svn.osgeo.org/grass/grass/tags/release_20090112_grass_6_4_0RC2

An announcement has been drafted at
https://trac.osgeo.org/grass/wiki/Release/6.4.0RC2-News

All news will be merged into the final announcement later.

Key improvements of the GRASS 6.4.0 release include enhanced
portability for MS-Windows (native support), hundreds of fixes,
the new wxPython based portable graphical interface and much
new functionality.

Release candidate management at
https://grass.osgeo.org/wiki/GRASS_6.4_Feature_Plan

Please test, test, test…

Thanks to all contributors!

Markus

/by
,

MapServer Graduates OSGeo Incubation

OSGeo is pleased to announce that the MapServer project has graduated from the incubation process and is now a full fledged OSGeo project with Steve Lime appointed as project representative (Vice President, MapServer).

MapServer is a web map server for building spatially enabled web applications. MapServer excels at rendering spatial data (maps, images, and vector data) for the Web, supporting a wide variety of geospatial input formats, and Web output formats. MapServer supports a pure CGI mode, as well as developing server applications in languages such as Python, Perl, Java and C#.

https://mapserver.osgeo.org/

Graduating incubation includes requirements for open community operation, a responsible project governance model, code provenance and license verification and general good project operation. Graduating incubation is the OSGeo seal of approval for a project and gives potential users of the project added confidence in the viability and safety of the project (see https://www.osgeo.org/incubator).

/by
, , , , ,

Building a cluster for GRASS GIS and other software from the OSGeo stack

Nov 2008

Lucky to have (access to) a cluster? Here some notes on how to do geospatial number crunching on it a.k.a. HPC (High Performance Computing).

Preparing the disks
We decided to use the ext3 file system. An initial problem was the formatting of the RAID5 disk set since it exceeded the file system specifications. Then, setting the ext3 block size to 4k instead of 1k we could format it.

Storage: a home for GIS data
The disks are available via NFS to all nodes (blades in our case). All raw/original data sets and the GRASS database are sitting in an NFS exported directory which I even link on my laptop to easily add/access/modify stuff.

Front-end machine and blades configuration
The cluster is a (currently) 56CPU blades system, we’ll expand to 128 CPUs later this year (16 blades with 2 procs a 4 core and 16GB RAM per blade). Additionally, we have a front-end machine to run the job manager and to link in further disks, all via NFS.
The blades are configured diskless, i.e. that once started, they receive their operating system from the front end machine via network (10GB/s ethernet). Like this, we have a single directory on the front end which contains all software, this is then propagated to all blades. Very convenient. We use Scientific Linux (the LiveDVD copied onto the disk, there is a special directory to store your modifications which are then merged in on the fly once you boot the blades, pretty cool concept). The job software is (SUN) Grid Engine, also free software. Job control with GRASS I have described here:

https://grass.osgeo.org/wiki/Parallel_GRASS_jobs
-> Grid Engine

GRASS: Avoiding replicated import of large data files through virtual linking
New in GRASS 6.4 is that you can just register a geodata file on the fly with r.external. Altogether I have 1.4TB of new GIS data from our province, naturally I didn’t want to by a new disk array just for my provincial GRASS location! Here r.external comes handy to minimize the “import” to a few bytes. As expected, it leverages GDAL to get data into GRASS, the overhead is minimal.

Power consumption
Power consumption is measured, too: The entire system consumes around 2000W (each blade less than 200W), so it’s going into the direction of “green” computing (there is no such thing!). If we had a solar panel at least…

Outcome
All in all a very nice solution. I made a stress test and removed all internal switches and shut down the blades while I was processing 8000 MODIS satellite maps. Everything survived and the Grid engine job manager collected the crashed jobs and restarted them without complaining. All resulting maps are collected in the target GRASS mapset and could be even exported to common GIS formats, if needed.
If you want to run Web Processing Services (e.g., pyWPS), you can likewise send each session to a node, giving you enormous possibilities for your customers.

Edit 2014:

“Big data” challenges on a cluster – limits and our solutions to overcome them:

  • 2008: internal 10Gb network connection way too slow…
    • Solution: TCP jumbo frames enabled (MTU > 8000) to speed up the internal NFS transfer
  • 2009: hitting an ext3 filesystem limitation (not more than 32k subdirectories allowed but having more files in cell_misc/ as each GRASS GIS raster map consists of multiple files)
    • Solution: adopting XFS filesystem [yikes! …. all to be reformated, i.e. some terabyes had to be “parked” temporarily]
  • 2012: Free inodes on XFS exceeded
    • Solution: Update XFS version [err, reformat everything again]
  • 2013: I/O saturation in NFS connection between chassis and blades
    • Solution: reduction to one job per blade (queue management), 21 blades * 2.5 billion input pixels + 415 million output pixels
  • 2014: GlusterFS saturation
    • Solution: Buy and use a new 48 port switch, ti implement 8-channel trunking (= 8 Gb/s)
/1 Comment/by

From OSGeo-Announce: New German, Spanish and Québécois OSGeo Local Chapters Approved

The OSGeo ticker reports:

The OSGeo board has formally approved three new local chapters. OSGeo local chapters provide a venue to support local users and developers, as well as a mechanism to further OSGeo’s mission and goals in a linguistic, or geographic area.

FOSSGIS e.V. has been approved as a local chapter for the german speaking countries of Germany, Austria and Switzerland with Dr. Georg Lösel as representative. FOSSGIS e.V. has existed since the year 2000 at which time is was incorporated as GRASS-Anwender-Vereinigung (GAV). It was recently renamed FOSSGIS e.V. FOSSGIS e.V is responsible for maintaining the FreeGIS Portal. Since 2006 it has held the FOSSGIS conference promoting FOSS4G software, with 500 participants in 2008. FOSSGIS e.V. also plays an important role in supporting a FOSS4G role in other tradeshows such as InterGeo and AGIT.

https://www.fossgis.de

The OSGeo Spanish Local Chapter has been approved for spanish speaking areas of all continents with Pedro-Juan Ferrer Matoses as representative. Since it’s formation in 2007 the chapter has established a members list, and elected a board of directors. Key activities include work on the Free GIS Book, Spanish Localization and events including Jornadas de SIG Libre and the Jornadas gvSIG.

https://es.osgeo.org

The Comité de direction du chapitre québécois de l’OSGeo has been approved for the region of Quebec (Canada) with Daniel Morissette as representative. This chapter seeks to provide a channel for communication, exchange and cooperation to facilitate the development of the Quebec community around OSGeo products and Open Source Geospatial topics in general. With a primary focus on local community development, it also plans to cooperate with the Francophone Chapter on French translations, and with other nearby chapters on some events. The chapter has approximately 90 members, and has formed a steering committee.

https://wiki.osgeo.org/wiki/Chapitre_qu%C3%A9b%C3%A9cois

/by
, ,

GDAL/OGR 1.5.2 released

A new bugfix release of GDAL/OGR (now at V1.5.2) was released today. This stable branch bug fix release fixes 37 issues in various drivers.

/by