Increasing the stability of processing algorithms

Processing just got a new testing framework to improve the long-term stability of this important plugin. And you can help to improve it, even if you are not a software developer!

This is yet another piece in our never-stopping crusade to improve the stability and quality of the best desktop GIS on the market.


You probably know processing. If you don’t: processing is the number one plugin to enable after every QGIS installation. It offers a very wide variety of geo-algorithms from generic one to very task-specific tools and allows building models and to completely automate workflows this way.

Processing is being improved consistently and gets better with every release. But like always in software development, there is a risk, that an improvement can have undesired side-effects which break previously working parts of an application.

Unit Testing and Continuous Integration

A bit more over a year ago (when working still as a freelancer) I started a crowdfunding initiative for automated testing, a technique in software engineering also known as continuous integration. With every single change a developer does to an application, a number of functions – so called unit tests – are run and their outcome is compared to a known-good control dataset. This way side-effects can be detected early on and fixed before they get deployed to productive environments.

This has been an amazing story of success. Since it has been put into place a lot of new unit tests have been added to a lot of QGIS functionality. A lot of bugs have been discovered by some servers that consistently test all the cool new stuff that comes in. Meanwhile we arrived at a point where we even test meta-quality of our codebase that makes the life for plugin developers easier: are new functions available from python? And do they come with documentation?


However, until now processing was excluded from these checks. But no longer.

To demonstrate the importance of this, take the intersection algorithm. Some months ago it started to fail in certain scenarios, caused by an update on the geometry engine. A totally unrelated change suddenly made the intersection algorithm behave differently. It just produced faulty results. The only thing was a small warning in the message log, a well-hidden place where you normally don’t check for warnings.

A small numbers of tests are already in place and make sure that the following algorithms run properly:

  • Centroid
  • Delete Holes
  • Intersection
  • Densify
  • Polygons to lines

If these unit tests had already been in place, this problem would have triggered an alarm right away. No longer, as of now, at least this same problem will not happen again!

But as you can see, there is only a small number of all the algorithms in processing being tested right now. Is your favorite one not yet included? That’s where you come in. That’s where the whole community of QGIS can help to make QGIS incredibly rock-freakin’-solid!

Help making it better

The tests are based on a very simple infrastructure.

  • A number of test datasets against which an algorithm can be run (e.g. intersection)
  • A simple description of which algorithm to run with which parameters (e.g. the layers polys.gml and multipolys.gml)
  • An expected result dataset (what you produce)

The first piece is already in place. The test datasets have been carefully developed to contain all kind of different geometries and attributes. This ensures that the tested algorithms are robust against all kind of side-effects.

Creating the other pieces couldn’t be easier

  1. Make sure you have a copy of the QGIS git repository.
  2. Choose an algorithm to test. Check the algorithm_test.yaml file that it’s not yet in place.
  3. Open a test dataset in your local QGIS copy: python/plugins/processing/tests/testdata
  4. Run the algorithm and redirect the result to  python/plugins/processing/tests/testdata/expected . Preferably in gml format.
  5. Manually check, that the result you receive is correct and that there have been no errors. This step is important!
  6. Open the Processing menu entry History.
  7. Right click the entry and click on Create Test.
  8. You will see a yaml specification that describes your algorithm. If it looks good, copy it to the end of python/plugins/processing/tests/testdata/algorithm_test.yaml .
  9. Request that your test is integrated. It will automatically be run on our test infrastructure. If it is good, it should be integrated shortly.


What has been done

As a by-product of this development, a couple of things have been developed and fixed.

  • Implementation of a qgis.testing  python module. This comes with some nice pieces which can be used by every python plugin for creating its own unit tests. It comes with a mock iface and a method to compare vector layers with control over attribute and geometry comparison.
  • Too many copies of a mock iface are already floating around in different plugin projects. Help making this one the best and only one!
  • Several fuzzy comparison options have been implemented (epsilon for geometries coords, epsilon for attributes…).
  • QgsVectorFileWriter is now able to produce geometry collections as output. While we cannot represent them in QGIS, a processing algorithm can now create such an output (a number of file formats support this. Shapefiles don’t.).
  • A crash has been fixed that surfaced when running certain algorithms with null geometries.
  • Some improvements have been brought to gdal, mostly concerning the geojson driver


Thank you Victor Olaya, Michael Kirk, Alex Bruy, Martin Dobias

Tagged with: , , , ,
Posted in C++, Programming, Python, QGIS, QGIS Plugins

Geometry generator symbology

difference(buffer($geometry, 250), buffer($geometry, -250))

December traditionally is an amazing time since the weather is usually quite forgiving to long working hours.

Therefore the first parts of our recent crowdfunding project for 2.5D rendering have been merged into QGIS master and will be shipped with QGIS 2.14.

It’s something of the sort of development that we really, really, really like here at an implementation that adds enormous flexibility and enables the user to use QGIS in ways that we never thought of.

Say hello to geometry generators.

Geometry generators allow to use expression syntax to generate a geometry on the fly during the rendering process. The resulting geometry does not have to match with the original geometry type and you can add several differently modified symbol layers on top of each other.



How to use it

It couldn’t be easier to  use. Open the symbology dialog. Choose geometry generator as symbol layer type. Choose what kind of symbol you would like to generate, write your expression and style it in whatever way you like.

Configuration Dialog


It’s your turn

Show the world what you can do with it.


  • Regional Council of Picardy
  • Ville de Nyon
  • Wetu GIT cc
  • All other crowdfunders

Special thanks to Nicolas Rochard and  for the help to bring this project into shape. And to Nyall Dawson for his invaluable inputs throughout the planning and reviewing process.


Tagged with: , ,
Posted in Featured, QGIS

QField Documentation

After getting QField up and running in Android 5, we felt it was time to start documenting how QField works, we started documenting how to install and use QField. We also added a section on how to handle your data to get them on QField. It is all pretty basic, but it dose give some hints.

You can find the documentation here:

As QField’s source code the docs are opensourced (on github) and we look forward for your help in documenting and translating.

Thanks a lot

Posted in Uncategorized

Passing android Intents to Qt

Working on QField I had the necessity of passing values from the to the Qt cpp world, here how I did it using an Intent that is sent to the QtActivity (the one you should not edit that comes with Qt). For much more information see this post series.
hopefully this will be helpful to someone.

This is the first version of the cpp code that I wrote, just for reference. The above code is much cleaner

Posted in Android, C++

QField for Android 5

QField app on Google Play

QField app

QField Karma edition app on Google Play

QField Karma edition app

It’s done, finally we managed to get rid of Ministro so that we finally can say, QField runs on any android from 4.0.3 (ICS). This makes as of today (according to google) 96% of the android installations worldwide. Eventually we want to settle to 4.3 (JB) as minimum to allow us using certain features and avoiding one known issue, but for now it would mean cutting of another 25% of the users.

So as of today it is: 4.0.3 (Ice cream sandwich API 15) is the required minimal Android version to run QField and Android 4.3 (Jelly Bean API 18) is the suggested minimal version.

We tested with 4.4, 5.0.1 and 5.1 but we haven’t had the chance to get our hands on an Android 6 so if you can, let us know how it works.

But adding support for android 5 isn’t the only great news, during the process we also:

  • Reintroduced WMS support
  • Removed ministro dependency. All libs are now bundled so that the installation is as simple as possible
  • Drastically reduced total download size from 300MB+ to 36MB
  • Updated libs to QGIS 2.13 (

We would like to thank to the City of Vevey and the QGIS hack fest Gran Canaria for supporting the development of this critical feature.

QField is an Open Source project led by LLC, more information, the source code and a possibility to donate to the project can be found on the QField page (preferred) or by buying the QField for QGIS Karma edition app.

Also if you need a specific feature, contact us to sponsor its development.

Tagged with: , , ,
Posted in Featured, QField

QGIS Crowdfunding: 2.5D Rendering


QGIS has a great variety of rendering possibilities from categorization to data defined settings which allows to make awesome cartographic products. It also features some extensions like qgis2threejs and globe that make it possible to explore the third dimension. While these extensions are great tools they have their limitations like they do not fully integrate with the styling system or cannot be used in print composers.

2.5D rendering

Mockup of a possible rendering result with a combination of classify and 2.5D effect

This project aims to improve the internal possibilities of QGIS to give an oblique view 3D effect based on a height attribute and an angle while fully preserving all the possibilities which the QGIS styling offers. But it doesn’t stop there, the whole rendering is built in a modular way so you can use all of its parts for countless other possibilities.

Funds are already available for a major part of this project thanks to ADUGA and the Regional Council of Picardy. For the missing pieces we would like to ask the community to help us to create the required extensions to include these amazing new functionalities into QGIS.

Geometry modifiers

Every building exists of the parts: floor, walls and roof. To get a nice effect can paint a shadow on the floor, then draw the walls for the buildings and finally paint the roof.

The wall as well as the roof geometries are nothing but modified versions of the source geometry. The roof is a translated version of it, the walls a 2 dimensional extrusion (the area between footprint and translated roof outline).

A new possibility to define a geometry modifier will be introduced. For every symbol layer in the layer styling it will be possible to define an expression that transforms the geometry before it is being painted.

Rendering order

Because buildings which are further apart from the camera need to be painted first and buildings closest to the camera last it is required that we can control the order in which features are rendered. It will be possible to define an expression (or simply a field) in which features are rendered. For this use-case this will be based on the geometry but you may use this to control the rendering order of any kind of layer. Or if you are a plugin author you can just send any feature request with an order by to the layer.

It is planned that for best compatibility this will be implemented in a provider independent way. Due to performance considerations we plan to add the possibility to delegate this job to the database in the future and will keep this in mind during the implementation.


A series of expressions will need to be developed

Translation of a geometry

Extrusion of a 2D geometry

Evaluation of a string as expression

Degrees to radians

Extract the individual points of a linestring

Azimuth of a line segment

Order a multipolygon by centroid (to render the walls of a building in the appropriate sequence)

Style variables

To make it easy to configure the style a new possibility to assign style variables will be introduced. This will make it possible to specify an expression which will define the height of individual buildings one single time on the style dialog and then reference and use this variable from various places within the rendering system.

Detailed specification

We have written a document that contains more detailed specifications for the individual parts. If you would like to get access to this, please contact us and we will send you a digital copy of it.

Tagged with: , ,
Posted in 3D, C++, Featured, QGIS


Tagged with: ,
Posted in Uncategorized

SpaceMouse in Ubuntu 15.04

While preparing some 3D scenes for an exibition I discovered the SpaceMouse by 3dconnexion. A neat device we plan on installing in front of a projected globe.

To get it to run in Ubuntu first get the drivers from


sudo apt-get install libmotif3
mkdir -p /tmp/3D3dxware-linux
cd /tmp/3D3dxware-linux
cp ~/Downloads/3dxware-linux-v1-8-0.x86_64.tar.gz /tmp/3D3dxware-linux
tar -xf 3dxware-linux-v1-8-0.x86_64.tar.gz
sudo ./

answer yes, 4, yes.
That’s it, you might get an error saying: “Red Hat EL 7 currently not supported for automatic driver startup.”. This is not a big deal as we can start the daemon by doing:

sudo /etc/3DxWare/daemon/3dxsrv -d usb &

to test if all works, launch


I’ll play around more to see how it goes.

For further, a bit older information, see this post

Posted in 3D, tech toys

QGIS Welcome Page

Screenshot from 2015-08-18 21-01-13

Whenever you start QGIS you basically do it because?

Right, because you need to do GIS work. Ah, how I love rhetorical questions to start a post.

And most of the time one continues to work on a QGIS project which he has prepared before. For me 99% of the time, I start QGIS, move the mouse to the top left over “Project” go to “Recent Projects” and select the one I want. If I am lucky my hand is stable enough to hover “Recent Projects” and not “New From Template” which I actually never use.

No longer!

At we just introduced a nice “Welcome Page” to QGIS which lists the recently used projects. With a screenshot next to it!

That’s how my QGIS looks at start right now:

Screenshot from 2015-08-18 21-01-13

Instead of the filename it will show the project title if one is defined.

And you get some recent information about the QGIS project just next to it.

I would never want to miss this feature again.

… Coming soon to a QGIS near you …

Tagged with:
Posted in QGIS

Syntactic sugar for PyQGIS

If you are a python coder you probably already know the with-statement.

If yes, you can directly jump to the with edit-section.
If not, here’s a short summing up.

If you want to edit a file you can do:

This is a bad idea. The file is not closed and you should always do that. We can easily add that, right?

Now what happens if do_some_changes_to(f) causes an exception? f.close() is never called.

You may remember something called finally. That can be appended to a try-block and will be executed in the end no-matter-what.

That’s a lot of code for something that was originally very slick. That’s where the with-statement comes into play:

It does the very same thing. Plus it is easier to read and faster to write than the whole try- finally-block.

Now QGIS has the same for editing layers since today (read: shipped with version 2.12).

with edit(layer)

Instead of writing

You can do:

If all goes well, the changes will be saved persistently to the datasource without any call to commitChanges() .

If any exception occurs, the changes will be rolled back. So like for the with -statement for files: it is easier to read, easier to write and it is comparable to a database transaction: Commit everything in the end or nothing at all if an error happens. How cool is that!

Plus you will get a python exception if something went wrong in committing (normally you will have to check the return value of layer.commitChanges() )and who does that anyway?

If you are writing your own .py files you will have to

And a final note since I am already on the topic of modifying data in persistent backends:

Avoid using the dataProvider() methods!

  • You cannot undo them easily
  • They generate one request per call what may reduce performance
  • They do not emit internal signals for map redraws and other refreshes of the user interface
  • They do not take uncommitted changes into account so the python changes will get overwritten by the user when he commits the layer changes

There is a theoretical advantage of slightly more control over how the communication with the backend happens (e.g. how many features are sent per request). But this is hardly ever required in real life unless you are working with really, really, really huge datasets.

Posted in Programming, Python, QGIS