{"id":3624,"date":"2018-06-22T13:30:25","date_gmt":"2018-06-22T11:30:25","guid":{"rendered":"https:\/\/www.opengis.ch\/?p=3624"},"modified":"2020-08-06T20:52:57","modified_gmt":"2020-08-06T18:52:57","slug":"threads-in-pyqgis3","status":"publish","type":"post","link":"https:\/\/www.opengis.ch\/fr\/2018\/06\/22\/threads-in-pyqgis3\/","title":{"rendered":"Using Threads in PyQGIS3"},"content":{"rendered":"\n

While porting a plugin to QGIS3 I decided to also move all it’s threading infrastructure to QgsTasks. Here three possible variants to implement this.
the first uses the static method QgsTask.fromFunction<\/code> and is simpler to use. A great quick solution. If you want need control you can look at the second solution that subclasses QgsTask. In this solution I also show how to create subtasks with interdependencies. The third variant, illustrates how to run a processing algorithm in a separate thread.
One thing to be very careful about is never to create widgets or alter gui in a task. This is a strict Qt guideline – gui must never be altered outside of the main thread. So your progress dialog must operate on the main thread, connecting to the progress report signals from the task which operates in the background thread. This also applies to \u00ab\u00a0print\u00a0\u00bb statements — these aren’t safe to use from a background thread in QGIS and can cause random crashes. Use the thread safe QgsMessageLog.logMessage() approach instead. Actually you should forget print and always use QgsMessageLog.<\/p>\n\n\n\n

using QgsTask.fromFunction<\/h2>\n\n\n\n

this is a quick and simple way of running a function in a separate thread. When calling QgsTask.fromFunction()<\/code> you can pass an on_finished<\/code> argument with a callback to be executed at the end of run<\/code>.<\/p>\n\n\n\n

import random\nfrom time import sleep\n\nfrom qgis.core import (QgsApplication, QgsTask, QgsMessageLog, Qgis)\n\nMESSAGE_CATEGORY = 'My tasks from a function'\n\n\ndef run(task, wait_time):\n    \"\"\"a dumb test function\n    to break the task raise an exception\n    to return a successful result return it. This will be passed together\n    with the exception (None in case of success) to the on_finished method\n    \"\"\"\n    QgsMessageLog.logMessage('Started task {}'.format(task.description()),\n                             MESSAGE_CATEGORY, Qgis.Info)\n    wait_time = wait_time \/ 100\n    total = 0\n    iterations = 0\n    for i in range(101):\n        sleep(wait_time)\n        # use task.setProgress to report progress\n        task.setProgress(i)\n        total += random.randint(0, 100)\n        iterations += 1\n        # check task.isCanceled() to handle cancellation\n        if task.isCanceled():\n            stopped(task)\n            return None\n        # raise exceptions to abort task\n        if random.randint(0, 500) == 42:\n            raise Exception('bad value!')\n    return {\n        'total': total, 'iterations': iterations, 'task': task.description()\n        }\n\n\ndef stopped(task):\n    QgsMessageLog.logMessage(\n        'Task \"{name}\" was cancelled'.format(name=task.description()),\n        MESSAGE_CATEGORY, Qgis.Info)\n\n\ndef completed(exception, result=None):\n    \"\"\"this is called when run is finished. Exception is not None if run\n    raises an exception. Result is the return value of run.\"\"\"\n    if exception is None:\n        if result is None:\n            QgsMessageLog.logMessage(\n                'Completed with no exception and no result ' \\\n                '(probably the task was manually canceled by the user)',\n                MESSAGE_CATEGORY, Qgis.Warning)\n        else:\n            QgsMessageLog.logMessage(\n                'Task {name} completed\\n'\n                'Total: {total} ( with {iterations} '\n                'iterations)'.format(\n                    name=result['task'],\n                    total=result['total'],\n                    iterations=result['iterations']),\n                MESSAGE_CATEGORY, Qgis.Info)\n    else:\n        QgsMessageLog.logMessage(\"Exception: {}\".format(exception),\n                                 MESSAGE_CATEGORY, Qgis.Critical)\n        raise exception\n\n\n# a bunch of tasks\ntask1 = QgsTask.fromFunction(\n    'waste cpu 1', run, on_finished=completed, wait_time=4)\ntask2 = QgsTask.fromFunction(\n    'waste cpu 2', run, on_finished=completed, wait_time=3)\nQgsApplication.taskManager().addTask(task1)\nQgsApplication.taskManager().addTask(task2)\n<\/code><\/pre>\n\n\n\n
Subclassing QgsTask<\/h2>\n\n\n\n
this solution gives you the full control over the task behaviour. In this example I also illustrate how to create subtasks dependencies.<\/p>\n\n\n\n
import random\nfrom time import sleep\n\nfrom qgis.core import (Qgis, QgsApplication, QgsMessageLog, QgsTask)\n\nMESSAGE_CATEGORY = 'My subclass tasks'\n\n\nclass MyTask(QgsTask):\n    \"\"\"This shows how to subclass QgsTask\"\"\"\n\n    def __init__(self, description, duration):\n\n        super().__init__(description, QgsTask.CanCancel)\n        self.duration = duration\n        self.total = 0\n        self.iterations = 0\n        self.exception = None\n\n    def run(self):\n        \"\"\"Here you implement your heavy lifting. This method should\n        periodically test for isCancelled() to gracefully abort.\n        This method MUST return True or False\n        raising exceptions will crash QGIS so we handle them internally and\n        raise them in self.finished\n        \"\"\"\n        QgsMessageLog.logMessage('Started task \"{}\"'.format(\n            self.description()), MESSAGE_CATEGORY, Qgis.Info)\n        wait_time = self.duration \/ 100\n        for i in range(101):\n            sleep(wait_time)\n            # use setProgress to report progress\n            self.setProgress(i)\n            self.total += random.randint(0, 100)\n            self.iterations += 1\n            # check isCanceled() to handle cancellation\n            if self.isCanceled():\n                return False\n            # simulate exceptions to show how to abort task\n            if random.randint(0, 500) == 42:\n                # DO NOT raise Exception('bad value!')\n                # this would crash QGIS\n                self.exception = Exception('bad value!')\n                return False\n        return True\n\n    def finished(self, result):\n        \"\"\"This method is automatically called when self.run returns.\n        result is the return value from self.run.\n        This function is automatically called when the task has completed (\n        successfully or otherwise). You just implement finished() to do \n        whatever\n        follow up stuff should happen after the task is complete. finished is\n        always called from the main thread, so it's safe to do GUI\n        operations and raise Python exceptions here.\n        \"\"\"\n        if result:\n            QgsMessageLog.logMessage(\n                'Task \"{name}\" completed\\n' \\\n                'Total: {total} ( with {iterations} iterations)'.format(\n                    name=self.description(),\n                    total=self.total,\n                    iterations=self.iterations),\n                MESSAGE_CATEGORY, Qgis.Success)\n        else:\n            if self.exception is None:\n                QgsMessageLog.logMessage(\n                    'Task \"{name}\" not successful but without exception ' \\\n                    '(probably the task was manually canceled by the '\n                    'user)'.format(\n                        name=self.description()),\n                    MESSAGE_CATEGORY, Qgis.Warning)\n            else:\n                QgsMessageLog.logMessage(\n                    'Task \"{name}\" Exception: {exception}'.format(\n                        name=self.description(), exception=self.exception),\n                    MESSAGE_CATEGORY, Qgis.Critical)\n                raise self.exception\n\n    def cancel(self):\n        QgsMessageLog.logMessage(\n            'Task \"{name}\" was cancelled'.format(name=self.description()),\n            MESSAGE_CATEGORY, Qgis.Info)\n        super().cancel()\n\n\nt1 = MyTask('waste cpu long', 10)\nt2 = MyTask('waste cpu short', 6)\nt3 = MyTask('waste cpu mini', 4)\nst1 = MyTask('waste cpu Subtask 1', 5)\nst2 = MyTask('waste cpu Subtask 2', 2)\nst3 = MyTask('waste cpu Subtask 3', 4)\nt2.addSubTask(st1, [t3, t1])\nt1.addSubTask(st2)\nt1.addSubTask(st3)\nQgsApplication.taskManager().addTask(t1)\nQgsApplication.taskManager().addTask(t2)\nQgsApplication.taskManager().addTask(t3)\n<\/code><\/pre>\n\n\n\n
NEVER, EVER, EVER use print in the QgsTask outside from finished(). finished() is called on the main event loop<\/h3>\n\n\n\n
from qgis.core import (QgsApplication, QgsMessageLog, QgsTask)\n\n\nclass MyTask(QgsTask):\n\n    def __init__(self, description, flags):\n        super().__init__(description, flags)\n\n    def run(self):\n        QgsMessageLog.logMessage('Started task {}'.format(self.description()))\n\n        # print('crashandburn')\n        return True\n\n\nt1 = MyTask('waste cpu', QgsTask.CanCancel)\nQgsApplication.taskManager().addTask(t1)\n<\/code><\/pre>\n\n\n\n
Call a Processing algorithm in a separate thread<\/h2>\n\n\n\n
You can simply execute a processing algorithm in a separate thread thanks to QgsProcessingAlgRunnerTask<\/code>. This class takes a processing algorithm, its parameters, a context and a feedback objects and execute the algorithm. QgsProcessingAlgRunnerTask<\/code> offers an executed<\/code> signal to which you can connect and execute further code. executed<\/code> sends two arguments bool successful<\/code> and dict results<\/code>. If you want to retrieve a memory layer you can pass the context as well by using partial<\/code> or lambda<\/code><\/a>.
If you’re wondering what parameter values you need to specify for an algorithm, and what values are acceptable, try running processing.algorithmHelp('qgis:randompointsinextent')<\/code> in the python console. In QGIS 3.2 you’ll get a detailed list of all the parameter options for the algorithm and a summary of acceptable value types and formats for each. Another nice possibility is to run the algorithm from the gui and check the history after.<\/p>\n\n\n\n
from functools import partial\n\nfrom qgis.core import (\n    Qgis, QgsApplication, QgsMessageLog, QgsProcessingAlgRunnerTask,\n    QgsProcessingContext, QgsProcessingFeedback, QgsProject,\n    )\n\nMESSAGE_CATEGORY = 'My processing tasks'\n\n\ndef task_finished(context, successful, results):\n    if not successful:\n        QgsMessageLog.logMessage('Task finished unsucessfully',\n                                 MESSAGE_CATEGORY,\n                                 Qgis.Warning)\n    output_layer = context.getMapLayer(results['OUTPUT'])\n    # because getMapLayer doesn't transfer ownership the layer will be\n    # deleted when context goes out of scope and you'll get a crash.\n    # takeResultLayer transfers ownership so it's then safe to add it to the\n    # project and give the project ownership.\n    if output_layer.isValid():\n        QgsProject.instance().addMapLayer(\n            context.takeResultLayer(output_layer.id()))\n\n\nalg = QgsApplication.processingRegistry().algorithmById(\n    'qgis:randompointsinextent')\ncontext = QgsProcessingContext()\nfeedback = QgsProcessingFeedback()\nparams = {\n    'EXTENT': '4.63,11.57,44.41,48.78 [EPSG:4326]',\n    'MIN_DISTANCE': 0.1,\n    'POINTS_NUMBER': 100,\n    'TARGET_CRS': 'EPSG:4326',\n    'OUTPUT': 'memory:My random points'\n    }\ntask = QgsProcessingAlgRunnerTask(alg, params, context, feedback)\ntask.executed.connect(partial(task_finished, context))\nQgsApplication.taskManager().addTask(task)\n<\/code><\/pre>\n\n\n\n
I hope this post can help you porting your plugins to QGIS3 and again if you need professional help for your plugins, don’t hesitate to contact us<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"
While porting a plugin to QGIS3 I decided to also move all it’s threading infrastructure to QgsTasks. Here three possible variants to implement this.the first uses the static method QgsTask.fromFunction and is simpler to use. A great quick solution. If you want need control you can look at the second […]<\/p>\n","protected":false},"author":2,"featured_media":11580,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_themeisle_gutenberg_block_has_review":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[37,14,15],"tags":[125],"class_list":["post-3624","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-pyqt","category-python","category-qgis","tag-qgis-org"],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2018\/06\/pexels-wendy-van-zyl-1212179-scaled.jpg?fit=2560%2C1706&ssl=1","jetpack-related-posts":[{"id":1668,"url":"https:\/\/www.opengis.ch\/fr\/2016\/09\/07\/using-threads-in-qgis-python-plugins\/","url_meta":{"origin":3624,"position":0},"title":"Using threads in QGIS python plugins","author":"Marco Bernasocchi","date":"7 septembre 2016","format":false,"excerpt":"Here an example on how to work with threads in a consistent and clean manner in QGIS python plugins","rel":"","context":"Dans "Python"","block_context":{"text":"Python","link":"https:\/\/www.opengis.ch\/fr\/category\/programming\/python\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":147,"url":"https:\/\/www.opengis.ch\/fr\/2010\/12\/01\/qgis-globe-plugin-installer-script\/","url_meta":{"origin":3624,"position":1},"title":"QGis Globe Plugin installer script","author":"Marco Bernasocchi","date":"1 d\u00e9cembre 2010","format":false,"excerpt":"Lately, thanks to ma Master Thesis, I've been co-working on the Globe Plugin for QGis here my install script for a threaded version of QGis with the Globe Plugin. By now the Globe has stereo 3D support, keyboard navigation (try all the num key), mouse navigation, a gui to control\u2026","rel":"","context":"Dans "C++"","block_context":{"text":"C++","link":"https:\/\/www.opengis.ch\/fr\/category\/programming\/cpp\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":13921,"url":"https:\/\/www.opengis.ch\/fr\/2023\/09\/15\/opengis-ch-and-oslandia-a-strategic-partnership-to-advance-qfield-and-qfieldcloud\/","url_meta":{"origin":3624,"position":2},"title":"OPENGIS.ch and Oslandia: A Strategic Partnership to Advance QField and QFieldCloud","author":"Marco Bernasocchi","date":"15 septembre 2023","format":false,"excerpt":"We are extremely happy to announce that we have partnered strategically with Oslandia to push the leading #fieldwork app #QField even further. In the world of fieldwork, accuracy and efficiency are paramount. As GIS specialists, we understand the importance of reliable tools that streamline data collection and analysis processes. That's\u2026","rel":"","context":"Dans "QField"","block_context":{"text":"QField","link":"https:\/\/www.opengis.ch\/fr\/category\/gis\/qfield\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2023\/09\/Qfield_Banner_1500x500_BOSA_new.png?fit=1200%2C400&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2023\/09\/Qfield_Banner_1500x500_BOSA_new.png?fit=1200%2C400&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2023\/09\/Qfield_Banner_1500x500_BOSA_new.png?fit=1200%2C400&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2023\/09\/Qfield_Banner_1500x500_BOSA_new.png?fit=1200%2C400&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2023\/09\/Qfield_Banner_1500x500_BOSA_new.png?fit=1200%2C400&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":116,"url":"https:\/\/www.opengis.ch\/fr\/2010\/02\/17\/custom-php-5-3-1-with-apc-and-xdebug-on-dreamhost-shared-hosting\/","url_meta":{"origin":3624,"position":3},"title":"Custom PHP 5.3.1 with APC and XDEBUG on (Dreamhost) Shared Host","author":"Marco Bernasocchi","date":"17 f\u00e9vrier 2010","format":false,"excerpt":"I've recently been setting up my new dreamhost for symfony projects deployment and the only thing the default PHP is missing is the support for APC (alternate php cache). So after looking at the dreamhost wiki I cleaned up and added some features to the one of the install scripts.\u2026","rel":"","context":"Dans "Web Development"","block_context":{"text":"Web Development","link":"https:\/\/www.opengis.ch\/fr\/category\/web-development\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":12165,"url":"https:\/\/www.opengis.ch\/fr\/2021\/06\/08\/qfieldcloud-now-opensource-happy-10-years-of-field-mapping-with-qgis\/","url_meta":{"origin":3624,"position":4},"title":"QFieldCloud now opensource – Happy 10 Years of field mapping with QGIS","author":"Marco Bernasocchi","date":"8 juin 2021","format":false,"excerpt":"Today, on QField's 10th anniversary, we're extremely proud to publish the results of over 18 months of development and give you the source code of QFieldCloud to go and make your awesome adaptations, solutions, and hopefully contributions :) If you want to quickly try it out, head to\u00a0https:\/\/qfield.cloud where our\u2026","rel":"","context":"Dans "QField"","block_context":{"text":"QField","link":"https:\/\/www.opengis.ch\/fr\/category\/gis\/qfield\/"},"img":{"alt_text":"QField git history","src":"https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2021\/06\/qfield-git-history.png?fit=660%2C280&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2021\/06\/qfield-git-history.png?fit=660%2C280&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2021\/06\/qfield-git-history.png?fit=660%2C280&ssl=1&resize=525%2C300 1.5x"},"classes":[]},{"id":13036,"url":"https:\/\/www.opengis.ch\/fr\/2022\/08\/30\/writing-a-feature-based-processing-algorithm-at-the-example-of-m-value-interpolation\/","url_meta":{"origin":3624,"position":5},"title":"Writing a feature-based processing algorithm at the example of M-value interpolation","author":"isabel","date":"30 ao\u00fbt 2022","format":false,"excerpt":"Amongst all the processing algorithms already available in QGIS, sometimes the one thing you need is missing.\u00a0 This happened not a long time ago, when we were asked to find a way to continuously visualise traffic on the Swiss motorway network (polylines) using frequently measured traffic volumes from discrete measurement\u2026","rel":"","context":"Dans "Processing"","block_context":{"text":"Processing","link":"https:\/\/www.opengis.ch\/fr\/category\/gis\/qgis\/processing\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2022\/08\/Screenshot-2022-07-26-at-10.25.35.png?fit=1200%2C894&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2022\/08\/Screenshot-2022-07-26-at-10.25.35.png?fit=1200%2C894&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2022\/08\/Screenshot-2022-07-26-at-10.25.35.png?fit=1200%2C894&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2022\/08\/Screenshot-2022-07-26-at-10.25.35.png?fit=1200%2C894&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/www.opengis.ch\/wp-content\/uploads\/2022\/08\/Screenshot-2022-07-26-at-10.25.35.png?fit=1200%2C894&ssl=1&resize=1050%2C600 3x"},"classes":[]}],"jetpack_shortlink":"https:\/\/wp.me\/pbdBtI-Ws","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.opengis.ch\/fr\/wp-json\/wp\/v2\/posts\/3624","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.opengis.ch\/fr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.opengis.ch\/fr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.opengis.ch\/fr\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.opengis.ch\/fr\/wp-json\/wp\/v2\/comments?post=3624"}],"version-history":[{"count":5,"href":"https:\/\/www.opengis.ch\/fr\/wp-json\/wp\/v2\/posts\/3624\/revisions"}],"predecessor-version":[{"id":11579,"href":"https:\/\/www.opengis.ch\/fr\/wp-json\/wp\/v2\/posts\/3624\/revisions\/11579"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.opengis.ch\/fr\/wp-json\/wp\/v2\/media\/11580"}],"wp:attachment":[{"href":"https:\/\/www.opengis.ch\/fr\/wp-json\/wp\/v2\/media?parent=3624"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.opengis.ch\/fr\/wp-json\/wp\/v2\/categories?post=3624"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.opengis.ch\/fr\/wp-json\/wp\/v2\/tags?post=3624"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}