QGIS Crowdfunding: 2.5D Rendering


QGIS has a great variety of rendering possibilities from categorization to data defined settings which allows to make awesome cartographic products. It also features some extensions like qgis2threejs and globe that make it possible to explore the third dimension. While these extensions are great tools they have their limitations like they do not fully integrate with the styling system or cannot be used in print composers.

2.5D rendering

Mockup of a possible rendering result with a combination of classify and 2.5D effect

This project aims to improve the internal possibilities of QGIS to give an oblique view 3D effect based on a height attribute and an angle while fully preserving all the possibilities which the QGIS styling offers. But it doesn’t stop there, the whole rendering is built in a modular way so you can use all of its parts for countless other possibilities.

Funds are already available for a major part of this project thanks to ADUGA and the Regional Council of Picardy. For the missing pieces we would like to ask the community to help us to create the required extensions to include these amazing new functionalities into QGIS.

Geometry modifiers

Every building exists of the parts: floor, walls and roof. To get a nice effect can paint a shadow on the floor, then draw the walls for the buildings and finally paint the roof.

The wall as well as the roof geometries are nothing but modified versions of the source geometry. The roof is a translated version of it, the walls a 2 dimensional extrusion (the area between footprint and translated roof outline).

A new possibility to define a geometry modifier will be introduced. For every symbol layer in the layer styling it will be possible to define an expression that transforms the geometry before it is being painted.

Rendering order

Because buildings which are further apart from the camera need to be painted first and buildings closest to the camera last it is required that we can control the order in which features are rendered. It will be possible to define an expression (or simply a field) in which features are rendered. For this use-case this will be based on the geometry but you may use this to control the rendering order of any kind of layer. Or if you are a plugin author you can just send any feature request with an order by to the layer.

It is planned that for best compatibility this will be implemented in a provider independent way. Due to performance considerations we plan to add the possibility to delegate this job to the database in the future and will keep this in mind during the implementation.


A series of expressions will need to be developed

Translation of a geometry

Extrusion of a 2D geometry

Evaluation of a string as expression

Degrees to radians

Extract the individual points of a linestring

Azimuth of a line segment

Order a multipolygon by centroid (to render the walls of a building in the appropriate sequence)

Style variables

To make it easy to configure the style a new possibility to assign style variables will be introduced. This will make it possible to specify an expression which will define the height of individual buildings one single time on the style dialog and then reference and use this variable from various places within the rendering system.

Detailed specification

We have written a document that contains more detailed specifications for the individual parts. If you would like to get access to this, please contact us and we will send you a digital copy of it.

Tagged with: , ,
Posted in 3D, C++, Featured, QGIS


Tagged with: ,
Posted in Uncategorized

SpaceMouse in Ubuntu 15.04

While preparing some 3D scenes for an exibition I discovered the SpaceMouse by 3dconnexion. A neat device we plan on installing in front of a projected globe.

To get it to run in Ubuntu first get the drivers from www.3dconnexion.eu/service/drivers.html


sudo apt-get install libmotif3
mkdir -p /tmp/3D3dxware-linux
cd /tmp/3D3dxware-linux
cp ~/Downloads/3dxware-linux-v1-8-0.x86_64.tar.gz /tmp/3D3dxware-linux
tar -xf 3dxware-linux-v1-8-0.x86_64.tar.gz
sudo ./install-3dxunix.sh

answer yes, 4, yes.
That’s it, you might get an error saying: “Red Hat EL 7 currently not supported for automatic driver startup.”. This is not a big deal as we can start the daemon by doing:

sudo /etc/3DxWare/daemon/3dxsrv -d usb &

to test if all works, launch


I’ll play around more to see how it goes.

For further, a bit older information, see this post

Posted in 3D, tech toys

QGIS Welcome Page

Screenshot from 2015-08-18 21-01-13

Whenever you start QGIS you basically do it because?

Right, because you need to do GIS work. Ah, how I love rhetorical questions to start a post.

And most of the time one continues to work on a QGIS project which he has prepared before. For me 99% of the time, I start QGIS, move the mouse to the top left over “Project” go to “Recent Projects” and select the one I want. If I am lucky my hand is stable enough to hover “Recent Projects” and not “New From Template” which I actually never use.

No longer!

At OPENGIS.ch we just introduced a nice “Welcome Page” to QGIS which lists the recently used projects. With a screenshot next to it!

That’s how my QGIS looks at start right now:

Screenshot from 2015-08-18 21-01-13

Instead of the filename it will show the project title if one is defined.

And you get some recent information about the QGIS project just next to it.

I would never want to miss this feature again.

… Coming soon to a QGIS near you …

Tagged with:
Posted in QGIS

Syntactic sugar for PyQGIS

If you are a python coder you probably already know the with-statement.

If yes, you can directly jump to the with edit-section.
If not, here’s a short summing up.

If you want to edit a file you can do:

This is a bad idea. The file is not closed and you should always do that. We can easily add that, right?

Now what happens if do_some_changes_to(f) causes an exception? f.close() is never called.

You may remember something called finally. That can be appended to a try-block and will be executed in the end no-matter-what.

That’s a lot of code for something that was originally very slick. That’s where the with-statement comes into play:

It does the very same thing. Plus it is easier to read and faster to write than the whole try- finally-block.

Now QGIS has the same for editing layers since today (read: shipped with version 2.12).

with edit(layer)

Instead of writing

You can do:

If all goes well, the changes will be saved persistently to the datasource without any call to commitChanges() .

If any exception occurs, the changes will be rolled back. So like for the with -statement for files: it is easier to read, easier to write and it is comparable to a database transaction: Commit everything in the end or nothing at all if an error happens. How cool is that!

Plus you will get a python exception if something went wrong in committing (normally you will have to check the return value of layer.commitChanges() )and who does that anyway?

If you are writing your own .py files you will have to

And a final note since I am already on the topic of modifying data in persistent backends:

Avoid using the dataProvider() methods!

  • You cannot undo them easily
  • They generate one request per call what may reduce performance
  • They do not emit internal signals for map redraws and other refreshes of the user interface
  • They do not take uncommitted changes into account so the python changes will get overwritten by the user when he commits the layer changes

There is a theoretical advantage of slightly more control over how the communication with the backend happens (e.g. how many features are sent per request). But this is hardly ever required in real life unless you are working with really, really, really huge datasets.

Posted in Programming, Python, QGIS

Postgres Expression Compiler for QGIS


This project is all about performance of QGIS with a postgres/postgis database.

A lot of people have QGIS connected to postgres/postgis (if you don’t: it’s a great combination in the open source geo stack). Databases are really optimized for querying. They keep indexes of geometries to be able to find them faster, they keep indexes of attributes to filter faster – and finally they often run on powerful servers.

QGIS tries to be smart enough to make use of this additional information. So whenever a map canvas is being rendered it asks the database only for features in this area. But if it’s about other attributes, QGIS filters by them locally. This means if you have a list of all the cities in the world and only want the capitals, QGIS will get the list with all the cities from the database, check for every city if it’s a capital. If it’s not, it will discard it, if it is it will use it. That’s terribly slow and postgres itself could do it much faster.

That’s where the postgres expression compiler comes into play. It is able to compile QgsExpressions (an SQL dialect) to postgres filter queries (another SQL dialect) and sends the converted SQL to the server if it is able to create matching SQL. If not it will gracefully fallback to the (slow) local evaluation.

Where it can be used

When building enterprise scaled applications dealing with large amounts of data and only requiring a subset of it is a really common task.

Let’s start with some cartography. Think of a small country-side village with some beautiful old houses and a lovely small river making its way through the nearby forest. This village is in a bigger district which has some other cities (some of them more, some of them less worth a visit) and this district is just one out of several others which together form a whole kingdom. One day the king asks a map for him to be drawn of his kingdom. And the man who has been tasked the job settles out and finds our nice small village. It’s a hard job for him to decide how to visualize it. Should he draw all this beautiful old houses? And the river? And the forest? He tried hard to do it but he couldn’t make the lines thin enough to fit everything on that piece of paper on which he had to paint the other 258 villages. After hours of trying he decided to put a small dot for the village there. He could not even label it with the name.

In cartography the task of going to a smaller map scale is called generalization. There are various possibilities to help with this task, an important one is: leave out things. While QGIS cannot replace the whole generalization task which the king’s cartographer had done, it can still help you as you zoom out and in on the map canvas and show certain features only at certain scales. To configure this, you can use the rule based renderer. This let’s you define combinations of filters, scales and symbology. At small scales we will usually have only a subset of features being rendered. And this is where the expression compiler comes into play and helps you to not even let the server send them through the network but already filter all the small cities out before sending them to QGIS.

On an example real world application (the QGEP project) the rendering speed on small scales has been increased by a factor of 3. This number is highly specific to this particular environment. The database server runs on the same machine (network latency, network throughput, CPU and memory shared between QGIS and postgres…) and can be better or worse, depending on the environment and the number of features which can be filtered on the server.

The current status

It is currently in experimental state so it’s opt-in. Look for it in Options => Data Sources => Data Source Handling.

For developers

If you are a developer and want to make use of it, it’s really simple. All you need to do is to use QgsFeatureRequest::setFilterExpression and make sure the setting is turned on.


I would like to thank the Swiss QGIS usergroup who made this project possible. It is amazing how this group helps to push QGIS forward.

Posted in Uncategorized

QField in the wild

QField app on Google Play

QField app

QField Karma edition app on Google Play

QField Karma edition app

QField Experimental is out, after a couple of months of requirements gathering, private early alpha testing and foremost tons of  emails requesting access to the testes group we decided today to put the current BETA version in the playstore. This means that from now on you can install QField just like any other android app by using the playstore.

It is important to remember that QField is still beta and thus it doesn’t have all the features we want yet. Notably, there is no digitizing, due to a bug in android, Android 5 (Lollipop) is currently not supported (android 4.3 and 4.4 are suggested) and for now you need to manually copy the projects and data from the desktop to your device.

QField is an Open Source project led by OPENGIS.ch LLC, more information, the source code and a possibility to donate to the project can be found on the QField page (preferred) or by buying the QField for QGIS Karma edition app.

Also if you need a specific feature, contact us to sponsor its development.

Posted in Android QGIS, Featured, GIS, QField, QGIS

QGIS Quality and Testing

I promised that I will write a bit about what I’ve been up to at the last QGIS developer meeting – apart from the social part we also got some work done there.

So let me start with something that really matters to me and I think can make a big impact.

Unit Testing

At the start of the developer conference Alessandro Pasotti asked me to do a workshop on unit tests. We quickly squatted a room with a couple of other people where we discussed the general system of unit testing in QGIS and a small unit test has been implemented. The idea definitely got some traction as a couple of hours later a pull request with a similar test was ready for merge. Thank you very much Tobias Reber!

It should turn out, that unit testing should keep me busy at this place for some more time. It was very interesting to see that the unit tests which we have really do their job. And their job is to fail (You can see what’s going on here https://travis-ci.org/qgis/QGIS/builds). More than once tests failed for pull requests and some code could be brought into good shape before being integrated.

Another interesting thing to see was, that sometimes a pull request got merged even though it failed. I assume this is due to the fact that some of our unit tests sometimes fail (mainly the ones which require external services) and therefore condition developers not to look too close at the result of the tests. Bottomline for me: prefer to disable unstable tests. It’s easier to interpret the testing result if the noise is removed.

And finally another work I did on the unit testing was related to the Postgres Expression Compiler (more about it in a future post). This is a new implementation which allows some requests to be filtered on the database server instead of the client itself. I didn’t want to leave it alone from unit tests, so I wrote some tests that check if a bunch of sample requests filter the same set of features on the database as they do locally. This requires a postgres/postgis server which we didn’t have to test before. Fortunately we can run that directly on the travis-ci infrastructure (that’s where QGIS gets tested after every change in the code). But then I thought that if we already have a running postgres/postgis server, why only test some expressions. And I started to add further tests for basic functionality like querying the extent, getting default values, filtering by spatial extent and some others.

While doing that I realized, that it’s not only postgres/postgis that should pass these basic tests, but basically any provider. So the code was refactored out into a more generic base class to be able to implement this for various providers and keep adding more tests at one single point. Now the same code tests also SpatiaLite and Shapefiles. That’s really nice because this way we can check if all providers act the same and discover inconsistencies. For instance, when introducing the Shapefile tests it turned out that when selecting features in a given spatial extent not only the features in there but also features without a geometry have been returned. A rather odd behavior that none of the other two providers showed. It’s fixed now.

I am looking forward to seeing the suite of unit tests grow even more and helping QGIS to become more stable.

Posted in Uncategorized

Tak Nødebo

After a week with QGIS members from all over the world we arrived back home and can say that once again, the QGIS developer meeting #13 was a great event.

It started with the QGIS User Conference where a lot of interesting talks from experiences in day-to-day usage to low-level technology insights could be attended. Tim Sutton in the end took the chance to shortcut users and developers directly, asking for feedback and advice – a great way of reducing the gap between development and usage. Also outside the conference rooms it was great to talk to all the users, developers, geo-geeks, to have interesting discussions and to hear how the software we produce is being seen, where it rocks and where it requires more polishing.

The second day of the User Conference was all about workshops and a everybody could get some insights into new topics or get a more profound understanding of possibilities and techniques hands-on. I haven’t attended any workshop myself but I have heard from several people that they were totally worth the time.

The rest of the week it was all about development again. That’s what I thought at least in the beginning – but it turned out that it was much more socializing than development. Which is not necessarily a bad thing! Finally I still got around to do some development done, which I will try to sum up in a later post here.

One thing that I always enjoy is to be able to practice languages. Thank you Violeta, Mar, Paolo, Victor and so many others for the Latin language part! (I don’t even start with English or this page will be too long)

The usual suspects were present as expected but also many new faces could be seen. It is awesome to see how this event gets more attraction and visitors every year!

One of the best things about it is that there is so much knowledge around and that the decision circles are really short. Regardless if some questions you may have are related to QGIS, other FOSSGIS, general open source, geo-spatial or whatsoever, you can be sure that the right people will be there to talk about it and finally get the ok to do something or a good reason not to do something.

If you have never been to a developer meeting or a user conference, you should definitely consider to go there if it is coming close to you. It will be good for you to talk to everybody and it will be good for everybody to talk to you!

Thank You

Finally I would like to say that I really enjoyed having been hosted at Skovskolen. Lene Fischer did an amazing job and it was more than clear that without the many helping hands that have been around it would never have been possible to organize and hold a conference like this. Many thanks also to the students, the kitchen, the technicians, the cleaning crew and all the people from the QGIS community who helped to prepare and give talks and workshops. You all rock!

Posted in QGIS

Performance for mass updating features

This post discusses how to improve the performance of pyqgis code that updates a lot of features by a factor of more than 10.


Once in a while you want to modify every feature of a layer. Or a bunch of features meeting certain criteria. That’s pretty straightforward. Let’s say you want to shift them all in x direction by 0.1 and in y direction by 0.3 (map units).

Straightforward and easy to do:


This code however may take a considerable amount of time. Python has a nice little module called timeit that helps you to benchmark by running the code a couple of times. [1]
Let’s do it 5 times

On a postgres layer with ~11000 features this gives me 25 seconds the first run and then constantly around 17 seconds (refer to [1]).

I wouldn’t be writing about it if we couldn’t do any better :)

Optimize the feature request

The first step to optimization is, that we actually only need the feature id and nothing else. So we modify the request like this:

Nice, down to ~14 seconds.

Grouping in the undo stack

But we can optimize it even further. Remember that you can undo things in QGIS? That’s a nice feature but it isn’t for free. For every operation that is done an item is put onto the “undo stack”, a quite costly operation. But for our operation we don’t really need a separate undo operation for every feature on the layer, we can just have one “grouped” undo item instead. That’s not only faster but also more user-friendly. For this, QgsVectorLayer offers the methods beginEditCommand( text ) and endEditCommand() .

And the winner is… down to ~1.38 seconds. We have just cut down the time to execute this by a factor of 10!

Remember this whenever you do bulk updates on a layer.

Working with the provider

Another very common approach is not to use the vector layer at all and directly work on the provider. This completely bypasses the undo stack. When doing this, what you need to take care of is that QGIS itself sends update operations to the backend in groups when saving a vector layer. If you bypass the vector layer you will have to do this yourself or you will send a new request to your backend for each and every feature which is especially slow if network latency is involved. Therefore the QgsVectorDataProvider object has methods that take more than a single change at once: changeAttributeValues and changeGeometryValues . Use these wisely.

[1] Benchmarking is hard to get right and influenced by many parameters. The main problem is that it is impossible to completely separate the signal from the noise. In this example, the first time iterating the layer took considerably longer, this could have been caused by a cold cache, the postgres connection pool still being empty, system load… When benchmarking, always treat the results with a good portion of reluctance.

Posted in Python, QGIS, QGIS Plugins