Planet Python

Last update: February 02, 2022 10:41 AM UTC

February 01, 2022

ItsMyCode

TypeError: can only concatenate str (not “int”) to str

In Python, we can concatenate values if they are of the same type. Let’s say if you concatenate a string and an integer you will get TypeError: can only concatenate str (not “int”) to str

In this article, we will take a look at what TypeError: can only concatenate str (not “int”) to str means and how to resolve this error with examples.

What is TypeError: can only concatenate str (not “int”) to str

Unlike other programming languages like JavaScript, Python does not allow concatenating values of different types. For Example, we cannot concatenate a string and an integer, string and a list etc.

Example Scenario

Let us take a simple example to reproduce this issue.

product = {
	"item": "iPhone",
	"price": 1599,
	"qty_available": 40
}

print("We have total " + product["qty_available"] + " quantities of Product " + product["item"])

Output

Traceback (most recent call last):
  File "c:\Personal\IJS\Code\main.py", line 7, in <module>
    print("We have total" + product["qty_available"] + "quantities of " + product["item"])
TypeError: can only concatenate str (not "int") to str

When we run our code we get TypeError: can only concatenate str (not “int”) to str because we are trying to concatenate the string and integer value in our print statement.

How to fix TypeError: can only concatenate str (not “int”) to str

In Python, you should ensure that the values are of the same type before performing the concatenation. Usually, we come under into this situation often during computational or printing the dictionary value using the print statement.

In the above dictionary, the product[qty_available] is of type integer and we are concatenating the integer value with strings in the print statement which leads into TypeError: can only concatenate str (not “int”) to str.

Solution

Now that we know why the issue occurs in the first place the solution is pretty simple and straightforward.

We can resolve the issue by converting the value product[qty_available] to a string before concatenating with other strings in the print statement.

Let us modify the above example and run it once again.

product = {
	"item": "iPhone",
	"price": 1599,
	"qty_available": 40
}

print("We have total " + str(product["qty_available"]) + " quantities of Product " + product["item"])

Output

We have total 40 quantities of Product iPhone

Conclusion

The TypeError: can only concatenate str (not “int”) to str mainly occurs if you try to concatenate integer with a string. Python does not allow concatenating values of different types. We can resolve the issue by converting the integer values to strings before concatenating them in the print statement.

February 01, 2022 08:19 PM UTC

PyCoder’s Weekly

Issue #510 (Feb. 1, 2022)

#510 – FEBRUARY 1, 2022
View in Browser »

What Is a JIT and How Can Pyjion Speed Up Your Python?

How can you can speed up Python? Have you thought of using a JIT (Just-In-Time Compiler)? This week on the show, we have Real Python author and previous guest Anthony Shaw to talk about his project Pyjion, a drop-in JIT compiler for CPython 3.10.
REAL PYTHON podcast

10 Unknown Security Pitfalls for Python

“In this blog post, we share 10 security pitfalls we encountered in real-world Python projects. We chose pitfalls that we believe are less known in the developer community.”
DENNIS BRINKROLF

Monitor and Optimize Python Application Uptime and Trace App Requests in Real Time With Datadog APM

Trace requests across service boundaries and identify latency and other potential errors with end-to-end application tracing. Datadog’s unified platform enables you to correlate traces with related metrics and logs for fast troubleshooting. Track Python app performance by trying Datadog APM out →
DATADOG sponsor

Draw the Mandelbrot Set in Python

In this tutorial, you’ll visualize the famous Mandelbrot set using Python’s Matplotlib and Pillow libraries. You’ll learn how to draw the fractal in black and white, grayscale, and color.
REAL PYTHON

Apple Removes Python 2.7 in macOS Monterey 12.3

“Python 2.7 was removed from macOS in this update. Developers should use Python 3 or an alternative language instead.” Also see the related discussion on Hacker News.
APPLE.COM

Black, the Python Code Formatter, Is Stable

The popular Python auto-formatter Black is finally non-beta software. Related discussion on Hacker News and Black’s stability policy doc.
HACKER NEWS

Django Security Releases Issued: 4.0.2, 3.2.12, and 2.2.27

Includes fixes for a possible XSS via debug template tag and a denial-of-service possibility in file uploads.
DJANGO SOFTWARE FOUNDATION

Textual v0.1.14 Adds Windows Support

WILL MCGUGAN

Discussions

Should PyPI Deprecate `.egg` Uploads?

.egg uploads make up less than 1% of built distribution uploads in Dec 2021. Do you still build or install eggs, or maintain a tool that supports eggs?
DUSTIN INGRAM

What’re the Cleanest, Most Beautifully Written Projects in Github That Are Worth Studying the Code?

Python Jobs

Articles & Tutorials

Looping With Python `enumerate()`

Once you learn about for loops in Python, you know that using an index to access items in a sequence isn’t very Pythonic. So what do you do when you need that index value? In this course, you’ll learn all about Python’s built-in enumerate(), where it’s used, and how you can emulate its behavior.
REAL PYTHON course

PySide2 vs PySide6: What Are the Differences, and Is It Time to Upgrade?

If you are already developing Python GUI apps with PySide2, you might be asking yourself whether it’s time to upgrade to PySide6 and use the latest version of the Qt library.
MARTIN FITZPATRICK

Data Elixir: Data Science Newsletter

Data Elixir is a free newsletter that curates intermediate and advanced data science content from around the web. Covers analytics, machine learning, data visualization, analytics, and strategy →
DATA ELIXIR sponsor

The Fastest Way to Read a CSV in Pandas

This article covers: Pandas’ default CSV reading, the faster, more parallel CSV reader introduced in v1.4, and a different approach that can make things even faster.
ITAMAR TURNER-TRAURING

Executable Docs

It’s common to include interactive Python sessions inside documentation. With a small script we can turn these docs into Jupyter notebooks and serve them instantly.
IVO HOUBRECHTS • Shared by Ivo Houbrechts

Raining Outside? Build a Weather CLI App With Python

In this tutorial, you’ll write a nicely formatted Python CLI app that displays information about the current weather in any city you provide the name for.
REAL PYTHON

Find and Fix Performance Issues Fast With Scout APM

Scout APM is an application performance monitoring tool that ties performance anomalies directly to endpoints, saving developers hours of time spent on troubleshooting and debugging. Start your free trial today to see if Scout APM is right for you!
SCOUT APM sponsor

Using Pyenv and Pyenv-Virtualenv to Install Python and Create a Virtual Environment on macOS

ANDREA GRANDI

Unravelling Python’s Ellipsis Literal (`...`)

BRETT CANNON

Projects & Code

pip-secure-install: GitHub Action That Securely Installs Python Dependencies on CI

GITHUB.COM/MARKETPLACE

mureq: Single-File Alternative to `Requests`

GITHUB.COM/SLINGAMN

Python Graph Gallery: A Collection of Hundreds of Charts Made With Python

PYTHON-GRAPH-GALLERY.COM

Procrastinate: PostgreSQL-based Task Queue for Python

PROCRASTINATE.READTHEDOCS.IO

PyFlow: Visual and Modular Block Programing in Python

GITHUB.COM/BYCELIUM

konsole: Readable, Pleasing Console Output

GITHUB.COM/APPAREBIT • Shared by Robert Grimm

django-plpy: Django Toolkit for Python Stored Procedures in PostgreSQL

MEDIUM.COM/P • Shared by Thorin Schiffer

q: Quick-and-Dirty Debugging Output for Tired Programmers

PYPI.ORG

Events

Weekly Real Python Office Hours Q&A (Virtual)

February 2, 2022
REALPYTHON.COM

Canberra Python Meetup

February 3, 2022
MEETUP.COM

Sydney Python User Group (SyPy)

February 3, 2022
SYPY.ORG

Reunión Python Valencia

February 3, 2022
GOOGLE.COM

PyCascades Remote 2020

February 5 to February 7, 2022
PYCASCADES.COM

Happy Pythoning!
This was PyCoder’s Weekly Issue #510.
View in Browser »

[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]

February 01, 2022 07:30 PM UTC

PyCharm

Tutorial: Developing FastAPI Applications using K8s & AWS

We’re happy to announce the new PyCharm Guide tutorial by Mukul Mantosh, this time on FastAPI, Kubernetes, and AWS. In the video below he talks to Paul Everitt and shares a sneak peek at what you should expect from this amazing — almost 6 hours long — course. Let’s take a quick look!

REST APIs In The Cloud

In this free tutorial, you will build a sample e-commerce application with everything along the way and then deploy it to AWS with Kubernetes. Here’s what it looks like:

Introducing FastAPI, a modern, fast (high-performance), web framework
Setting up FastAPI project type in PyCharm Professional
Setting up the database connection and migrations
Implementing the REST APIs with routing and CRuD
Configuring Celery & Redis with FastAPI
Integrating FastAPI with JWT Tokens
Writing Unit Tests using Pytest & Asyncio
Writing K8s manifests & deploying in minikube
Deploying application using EKS, RDS, ElastiCache, Route53 & AWS Certificate Manager
Combine your K8s manifests into a single package using Helm Charts

Each of the 17 sections includes a narrated video demonstration, text explaining everything, and working code snippets.

Start now!

Previous Tutorials

This isn’t the first long-form tutorial series from Mukul. He’s done two previously on AWS in the PyCharm Guide:

Developing Serverless APIs using AWS Toolkit covers
shows you how to run, build, and deploy serverless functions in PyCharm. This tutorial also had a webinar session with Calvin Hendryx-Parker from Six Feet Up, joined by Michael Kennedy.
Developing Django Application using AWS goes through running, building, testing, and deploying Django application in PyCharm using NICE DCV. We had a
blog post with a longer introduction.

February 01, 2022 04:51 PM UTC

Python for Beginners

Arithmetic Sequence In Python

You might have heard about arithmetic sequences and geometric sequences in your mathematics classes. In this article, we will discuss arithmetic sequences. We will also implement programs to perform different operations on an arithmetic sequence in Python.

What is An Arithmetic Sequence?

An arithmetic sequence is a sequence of numbers in which any two consecutive numbers have a fixed difference. This difference is also known as the common difference between the terms in the arithmetic sequence.

For example, 3,5,7,9,11,13,… is an arithmetic sequence with a common difference of 2 between consecutive terms.

Nth Term In An Arithmetic Sequence

If we are given the first term A₁ and the common difference D, we can write the second term as A₁+D, the third term as A₁+2D, the fourth term as A₁+3D, and so on. The Nth term will be written as A₁+(N-1)D To find the Nth term of an arithmetic sequence in python, we can simply add the common difference (N-1) times to the first terms A₁ using a for loop as follows.

commonDifference = 2
print("Common Difference in the arithmetic sequence is:", commonDifference)
firstTerm = 3
print("First term in the arithmetic sequence is:", firstTerm)
# calculating 100th term
N = 100
nthTerm = firstTerm
for i in range(1, N):
    nthTerm = nthTerm + commonDifference
print("100th term in the arithmetic sequence is:", nthTerm)

Output:

Common Difference in the arithmetic sequence is: 2
First term in the arithmetic sequence is: 3
100th term in the arithmetic sequence is: 201

Alternatively, we can directly calculate the Nth term using the formulae as follows.

commonDifference = 2
print("Common Difference in the arithmetic sequence is:", commonDifference)
firstTerm = 3
print("First term in the arithmetic sequence is:", firstTerm)
# calculating 100th term
N = 100
nthTerm = firstTerm + (N - 1) * commonDifference
print("100th term in the arithmetic sequence is:", nthTerm)

Output:

Common Difference in the arithmetic sequence is: 2
First term in the arithmetic sequence is: 3
100th term in the arithmetic sequence is: 201

Sum Of N Terms In An Arithmetic Sequence In Python

To find the sum of N terms in an arithmetic expression, we can simply add each term using a for loop. In the for loop, we will first find each term using the formulae discussed above. After that, we will add the each term to calculate the sum of N terms as follows.

commonDifference = 2
print("Common Difference in the arithmetic sequence is:", commonDifference)
firstTerm = 3
print("First term in the arithmetic sequence is:", firstTerm)
# calculating sum of 50 terms
N = 50
sumOfTerms = 0
for i in range(1, N + 1):
    ithTerm = firstTerm + (i - 1) * commonDifference
    sumOfTerms = sumOfTerms + ithTerm
print("Sum of 50 terms in the arithmetic sequence is:", sumOfTerms)

Output:

Common Difference in the arithmetic sequence is: 2
First term in the arithmetic sequence is: 3
Sum of 50 terms in the arithmetic sequence is: 2600

Alternatively, we can also derive a mathematical expression for calculating the sum of N terms of the arithmetic sequence.

We know that the sum of N numbers will be equal to N * (average of all the terms). Here, we can find the average of all the terms very easily.

For an arithmetic sequence with the first term A₁ and the Nth term A_N, the average of all the terms is defined as (A₁+A_N)/2. As A₁ and common difference D will be given in the program, we can find A_N= A₁+ (N-1)*D.

Hence, the average of all the numbers in the arithmetic sequence will become (2A₁+ (N-1)*D)/2.

Subsequently, the sum of N terms of the arithmetic sequence will become N*((2A₁+ (N-1)*D)/2).

We can calculate the sum of N terms in the arithmetic equation using this formula in python as follows.

commonDifference = 2
print("Common Difference in the arithmetic sequence is:", commonDifference)
firstTerm = 3
print("First term in the arithmetic sequence is:", firstTerm)
# calculating sum of 50 terms
N = 50
sumOfTerms = (N * (2 * firstTerm + (N - 1) * commonDifference)) // 2
print("Sum of 50 terms in the arithmetic sequence is:", sumOfTerms)

Output:

Common Difference in the arithmetic sequence is: 2
First term in the arithmetic sequence is: 3
Sum of 50 terms in the arithmetic sequence is: 2600

Conclusion

In this article, we have discussed the basics and formulas of arithmetic sequences. We have also performed different operations like finding the Nth term and finding the sum of N terms of an arithmetic sequence in python. To learn more about numbers in python, you can read this article on decimal numbers in python. You might also like this article on complex numbers in python.

The post Arithmetic Sequence In Python appeared first on PythonForBeginners.com.

February 01, 2022 02:10 PM UTC

Real Python

Python's len() Function

In many situations, you’ll need to find the number of items stored in a data structure. Python’s built-in function len() is the tool that will help you with this task.

There are some cases in which the use of len() is straightforward. However, there are other times when you’ll need to understand how this function works in more detail and how to apply it to different data types.

In this course, you’ll learn how to:

Find the length of built-in data types using len()
Use len() with third-party data types
Provide support for len() with user-defined classes

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

February 01, 2022 02:00 PM UTC

Zero to Mastery

Python Monthly Newsletter 💻🐍 January 2022

26th issue of the Python Monthly Newsletter! Read by 20,000+ Python developers every month. This monthly Python newsletter covers the latest Python news so that you stay up-to-date with the industry and keep your skills sharp.

February 01, 2022 10:00 AM UTC

Django Weblog

Django security releases issued: 4.0.2, 3.2.12, and 2.2.27

In accordance with our security release policy, the Django team is issuing Django 4.0.2, Django 3.2.12, and Django 2.2.27. These release addresses the security issues detailed below. We encourage all users of Django to upgrade as soon as possible.

CVE-2022-22818: Possible XSS via `{% debug %}` template tag

The {% debug %} template tag didn't properly encode the current context, posing an XSS attack vector.

In order to avoid this vulnerability, {% debug %} no longer outputs an information when the DEBUG setting is False, and it ensures all context variables are correctly escaped when the DEBUG setting is True.

Thanks Keryn Knight for the report.

This issue has severity "medium" according to the Django security policy.

CVE-2022-23833: Denial-of-service possibility in file uploads

Passing certain inputs to multipart forms could result in an infinite loop when parsing files.

Thanks Alan Ryan for the report.

This issue has severity "medium" according to the Django security policy.

Affected supported versions

Django main branch
Django 4.0
Django 3.2
Django 2.2

Resolution

Patches to resolve the issue have been applied to Django's main branch and to the 4.0, 3.2, and 2.2 release branches. The patches may be obtained from the following changesets.

CVE-2022-22818:

CVE-2022-23833:

The following releases have been issued:

Django 4.0.2 (download Django 4.0.2 | 4.0.2 checksums)
Django 3.2.12 (download Django 3.2.12 | 3.2.12 checksums)
Django 2.2.27 (download Django 2.2.27 | 2.2.27 checksums)

The PGP key ID used for this release is Mariusz Felisiak: 2EF56372BA48CD1B.

General notes regarding security reporting

As always, we ask that potential security issues be reported via private email to security@djangoproject.com, and not via Django's Trac instance or the django-developers list. Please see our security policies for further information.

February 01, 2022 07:57 AM UTC

Tryton News

Newsletter February 2022

Time goes by and improvements to Tryton continue to be made. Here you can find the latest changes which have been included in the last month.

Changes for the User

The forecasts are now used for all supply calculations instead of only the purchase requests.

In the web client, the list of tabs no longer wraps on large screens but scrolls horizontally and each tab entry takes up the full width on small screens.

We now calculate an early date for the partial quantity if there isn’t one for the full quantity.

It is no longer possible to close (or remove) the type of an account that is already used in account move lines.

The auto-detection of CSV headers now stops on the first error in the web client.

We now support editing Shopify orders.

The accounts can have another type when their balance is a credit. This is the opposite of the existing debit type.

We do not create any more dunning for lines with pending payment.

Production orders with missing early moves are no more proposed for early planning.

We renamed the split lines accounting wizard into reschedule lines to be less confusing.

Changes for the System Administrator

The country module supports pycountry version 22.1.10.

We mirror the geonames zip files and use our mirror by default. This was needed because the original host has frequent downtime.

We removed the entropy check on user password. We found that it was not a good way to enforce good password. We recommend to use the forbidden list instead.

Changes for the Developer

We process sales for Shopify asynchronously as it can be quite slow due to the Shopify rate limit.

The view_id is now set in the context when parsing the view. So it can be used to apply attributes depending on the view in ModelView.view_attributes.

We replaced the deepcopy of the JSON-RPC result in the desktop client by a faster implementation based on the json types.

The country module can now load subdivisions with unknown types. This was needed to support future versions of pycountry.

The ORM uses now the already cached data to instantiate relational target records for which the context depends on other fields. This optimization prevents extra SQL queries for most of the cases.

The stock margin report retrieves the product’s unit from the SQL query instead of using a Function field which was triggering a second execution of the table query.

We replaced the back-off time on Shopify API calls by an automatic retry loop. This allows to make the first calls quickly until it consume the available bucket.

The board action domain is now limited to active_id and active_ids and they are stored in a dedicated _actions dictionary.

We added on ir.ui.view the view_get RPC method which can be used by the board to support inheritance like the other views.

The xpath inheritance of views applies now on all matching elements (instead of only the first) by default.

1 post - 1 participant

Read full topic

February 01, 2022 07:00 AM UTC

Wingware

Wing Python IDE Version 8.1.3 - February 1, 2022

Wing 8.1.3 adds support for using Unreal Engine with Wing, writes a preconfigured wingdbstub.py when creating new projects, expands Run Arguments and other dialog fields, correctly finds printers on Linux, and fixes a number of code analysis problems and other minor issues.

See the change log for details.

Download Wing 8 Now: Wing Pro | Wing Personal | Wing 101 | Compare Products

What's New in Wing 8.1

Support for Containers and Clusters

Wing 8 adds support for developing, testing, and debugging Python code that runs inside containers, such as those provided by Docker and LXC/LXD, and clusters of containers managed by a container orchestration system like Docker Compose. A new Containers tool can be used to start, stop, and monitor container services, and new Docker container environments may be created during project creation.

For details, see Working with Containers and Clusters.

New Package Management Tool

Wing 8 adds a new Packages tool that provides the ability to install, remove, and update packages found in the Python environment used by your project. This supports pipenv, pip, and conda as the underlying package manager. Packages may be selected manually from PyPI or by package specifications found in a requirements.txt or Pipfile.

For details, see Package Manager .

Improved Project Creation

Wing 8 redesigns New Project support so that the host, project directory, Python environment, and project type may all be selected independently. New projects may use either an existing or newly created source directory, optionally cloning code from a revision control repository. An existing or newly created Python environment may be selected, using virtualenv, pipenv, conda, or Docker.

Improved Python Code Analysis and Warnings

Wing 8 expands the capabilities of Wing's static analysis engine, by improving its support for f-strings, named tuples, and other language constructs. Find Uses, Refactoring, and auto-completion now work within f-string expressions, Wing's built-in code warnings work with named tuples, the Source Assistant displays more detailed and complete value type information, and code warning indicators are updated more cleanly during edits.

And More

Wing 8 also adds support for Python 3.10, native executable for Apple Silicon (M1) hardware, a new Nord style display theme, reduced application startup time, support for Unreal Engine, Delete Symbol and Rename Current Module refactoring operations, improved debug stepping and exception handling in async code, and much more.

For a complete list of new features in Wing 8, see What's New in Wing 8.

Try Wing 8 Now!

Wing 8 is an exciting new step for Wingware's Python IDE product line. Find out how Wing 8 can turbocharge your Python development by trying it today.

Downloads: Wing Pro | Wing Personal | Wing 101 | Compare Products

See Upgrading for details on upgrading from Wing 7 and earlier, and Migrating from Older Versions for a list of compatibility notes.

February 01, 2022 01:00 AM UTC

January 31, 2022

Test and Code

178: The Five Factors of Automated Software Testing

"There are five practical reasons that we write tests. Whether we realize it or not, our personal testing philosophy is based on how we judge the relative importance of these reasons." - Sarah Mei

This episode discusses the factors.

Sarah's order:

Verify the code is working correctly
Prevent future regressions
Document the code’s behavior
Provide design guidance
Support refactoring

Brian's order:

Verify the code is working correctly
Prevent future regressions
Support refactoring
Provide design guidance
Document the code’s behavior

The episode includes reasons why I've re-ordered them.

PyCharm

PyCharm 2022.1 EAP is open!

It’s that time of the year when we count on your early feedback to help us prepare for the next major PyCharm release. The Early Acces Program (EAP) is designed to give our users and community members the chance to contribute to a better PyCharm by increasing the amount of testing we do and helping us identify bugs and usability problems that would be hard to catch just through our internal processes.

Download the EAP builds via the Toolbox App or directly from our website.

Important: EAP builds are not fully tested and might be unstable.

Download PyCharm EAP

PyCharm 2022.1 EAP1 brings the new Run Targets implementation, which adds support for creating virtual environments inside different targets.

What’s new?

Targets are configured environments where PyCharm will execute your code. PyCharm Pro users already have built-in support for Docker, Docker-Compose, SSH, WSL, and other targets for a long time.

The new implementation brings two major benefits out of the box:

1. A simpler UI to configure your targets

Configuring your targets is now a quick process performed through a wizard. The first thing to do is go to Preferences/Settings > Python Interpreter > Add Interpreter, and choose the type of target that you want to configure.

In this example, we will configure a Docker target, but you can find more information about all the supported targets in our documentation.

As you select the target a dialog window pops up. In the case of Docker it’s a three-step process. You can build your image locally or pull it from a registry. In step 1 (1/3) we will pull the Python:latest image from Docker and click next.

As you can see, in the next step (2/3) PyCharm will launch an introspection container to check your environment and will remove this container as soon as the introspection process is over. You can, then, click next.

The third step (3/3) is where you can create your virtual environment inside your target. This is not necessary in this example, so we will select the System Interpreter option and click Create.

By now, you should have your target interpreter properly configured to run your application.

2. Creating virtual environments inside targets

Although in our previous example we were not intending to create a virtual environment inside our remote host (a Docker container in this case), this ability can be very useful in other types of targets, and this is one of the main reasons why we improved our Run Targets implementation.

From this EAP build oneards, you can create virtual environments inside WSL, Vagrant, and SSH hosts from the comfort of your IDE. The initial process is the same as demonstrated above, but in the last step, you will be able to select the path to the virtual environment of your choice.

The new support for targets is the main feature to be highlighted in this EAP1, but, of course, it’s not the only one. While we will talk more about other improvements in the following blog posts, we highly encourage you to try PyCharm now and discover them yourself.

You can also check the release notes for a complete list of features and bug fixes brought by this build.

Ready to join the EAP?

Ground rules

EAP builds are free to use and require a valid JetBrains account.
EAP builds expire 30 days after the build date.
You can install an EAP build side by side with your stable PyCharm version.
These builds are not fully tested and can be unstable.
Your feedback is always welcome. Please use our issue tracker to report any bugs or inconsistencies.

How to download?

Download this EAP from our website or through the JetBrains Toolbox App. Alternatively, if you’re on Ubuntu 16.04 or later you can use snaps.

The PyCharm team

January 31, 2022 04:39 PM UTC

Python Morsels

Supporting index and key lookups

In Python, you can customize how square brackets ([]) work on your objects.

Index and key lookups both use square brackets

Lists in Python support indexing using square brackets:

>>> colors = ["purple", "blue", "green", "pink"]
>>> colors[1]
'blue'

While dictionaries support key lookups using square brackets:

>>> fruit_counts = {"lime": 1, "apple": 4, "pear": 2}
>>> fruit_counts["pear"]
2

Putting square brackets after an object and passing something into those brackets is called subscript notation. This syntax is powered by the __getitem__ method (pronounced "dunder getitem"):

>>> colors.__getitem__(1)
'blue'

Any object that implements a __getitem__ method can be used with that subscript notation:

>>> fruit_counts.__getitem__("pear")
2

So if we want to make our own objects support this subscript notation, all we need to do is make sure we have a __getitem__ method.

Making square brackets work on your objects

This ForgivingIndexer class has a __init__ method and a __getitem__ method:

class ForgivingIndexer:

    def __init__(self, sequence):
        self.sequence = sequence

    def __getitem__(self, index):
        return self.sequence[int(index)]

ForgivingIndexer accepts a sequence and returns an object that can be indexed (which in turn, indexes the given sequence):

>>> colors = ForgivingIndexer(["purple", "blue", "green", "pink"])
>>> colors[1]
'blue'

But we also pass in a string to these square brackets, and our ForgivingIndexer object will convert that string into an integer:

>>> colors["2"]
'green'

Or we could even pass in a floating point number, and it'll convert that into an integer as well:

>>> colors[3.2]
'pink'

It's kind of a "forgiving index" in a sense. This all works because we've implemented a __getitem__ method on our class.

Summary

To allow square brackets to work on your object, you can implement a __getitem__ method. You'll most often see this done in list-like objects (a.k.a. sequences), or dictionary-like objects (a.k.a. mappings).

January 31, 2022 04:00 PM UTC

Real Python

Draw the Mandelbrot Set in Python

This tutorial will guide you through a fun project involving complex numbers in Python. You’re going to learn about fractals and create some truly stunning art by drawing the Mandelbrot set using Python’s Matplotlib and Pillow libraries. Along the way, you’ll learn how this famous fractal was discovered, what it represents, and how it relates to other fractals.

Knowing about object-oriented programming principles and recursion will enable you to take full advantage of Python’s expressive syntax to write clean code that reads almost like math formulas. To understand the algorithmic details of making fractals, you should also be comfortable with complex numbers, logarithms, set theory, and iterated functions. But don’t let these prerequisites scare you away, as you’ll be able to follow along and produce the art anyway!

In this tutorial, you’ll learn how to:

Apply complex numbers to a practical problem
Find members of the Mandelbrot and Julia sets
Draw these sets as fractals using Matplotlib and Pillow
Make a colorful artistic representation of the fractals

To download the source code used in this tutorial, click the link below:

Get Source Code: Click here to get the source code you’ll use to draw the Mandelbrot set.

Understanding the Mandelbrot Set

Before you try to draw the fractal, it’ll help to understand what the corresponding Mandelbrot set represents and how to determine its members. If you’re already familiar with the underlying theory, then feel free to skip ahead to the plotting section below.

The Icon of Fractal Geometry

Even if the name is new to you, you might have seen some mesmerizing visualizations of the Mandelbrot set before. It’s a set of complex numbers, whose boundary forms a distinctive and intricate pattern when depicted on the complex plane. That pattern became arguably the most famous fractal, giving birth to fractal geometry in the late 20th century:

Mandelbrot Set (Source: Wikimedia, Created by Wolfgang Beyer, CC BY-SA 3.0)

The discovery of the Mandelbrot set was possible thanks to technological advancement. It’s attributed to a mathematician named Benoît Mandelbrot. He worked at IBM and had access to a computer capable of what was, at the time, demanding number crunching. Today, you can explore fractals in the comfort of your home, using nothing more than Python!

Fractals are infinitely repeating patterns on different scales. While philosophers have argued for centuries about the existence of infinity, fractals do have an analogy in the real world. It’s a fairly common phenomenon occurring in nature. For example, this Romanesco cauliflower is finite but has a self-similar structure because each part of the vegetable looks like the whole, only smaller:

Fractal Structure of a Romanesco Cauliflower

Self-similarity can often be defined mathematically with recursion. The Mandelbrot set isn’t perfectly self-similar as it contains slightly different copies of itself at smaller scales. Nevertheless, it can still be described by a recursive function in the complex domain.

The Boundary of Iterative Stability

Formally, the Mandelbrot set is the set of complex numbers, c, for which an infinite sequence of numbers, z₀, z₁, …, z_n, …, remains bounded. In other words, there is a limit that the magnitude of each complex number in that sequence never exceeds. The Mandelbrot sequence is given by the following recursive formula:

In plain English, to decide whether some complex number, c, belongs to the Mandelbrot set, you must feed that number to the formula above. From now on, the number c will remain constant as you iterate the sequence. The first element of the sequence, z₀, is always equal to zero. To calculate the next element, z_n+1, you’ll keep squaring the last element, z_n, and adding your initial number, c, in a feedback loop.

By observing how the resulting sequence of numbers behaves, you’ll be able to classify your complex number, c, as either a Mandelbrot set member or not. The sequence is infinite, but you must stop calculating its elements at some point. Making that choice is somewhat arbitrary and depends on your accepted level of confidence, as more elements will provide a more accurate ruling on c.

Note: The entire Mandelbrot set fits in a circle with a radius of two when depicted on the complex plane. This is a handy fact that’ll let you skip many unnecessary calculations for points that certainly don’t belong to the set.

With complex numbers, you can imagine this iterative process visually in two dimensions, but you can go ahead and consider only real numbers for the sake of simplicity now. If you were to implement the above equation in Python, then it could look something like this:

>>>

>>> def z(n, c):
...     if n == 0:
...         return 0
...     else:
...         return z(n - 1, c) ** 2 + c

Your z() function returns the nth element of the sequence, which is why it expects an element’s index, n, as the first argument. The second argument, c, is a fixed number that you’re testing. This function would keep calling itself infinitely due to recursion. However, to break that chain of recursive calls, a condition checks for the base case with an immediately known solution—zero.

Try using your new function to find the first ten elements of the sequence for c = 1, and see what happens:

Read the full article at https://realpython.com/mandelbrot-set-python/ »

[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]

January 31, 2022 02:00 PM UTC

PyCharm

PyCharm 2021.3.2 Is Out

Bug fixes highlighted in this second minor release of PyCharm 2021.3:

We fixed the IDE’s freeze while indexing WSL projects [IDEA-286059]
We fixed a bug with Python string literals that were treated as ‘bytes’ in Markdown documents [PY-40313]

Download PyCharm 2021.3.2

For the full list of issues addressed in PyCharm 2021.3.2 please check the release notes.
Found a bug? Please report it using our bug tracker.

January 31, 2022 01:05 PM UTC

ItsMyCode

TypeError: only size-1 arrays can be converted to python scalars

We get this error generally while working with NumPy and Matplotlib. If you have a function that accepts a single value, but if you pass an array instead, you will encounter TypeError: only size-1 arrays can be converted to python scalars.

In this tutorial, we will learn what is TypeError: only size-1 arrays can be converted to python scalars and how to resolve this error with examples.

What is TypeError: only size-1 arrays can be converted to python scalars?

Python generally has a handful of scalar values such as int, float, bool, etc. However, in NumPy, there are 24 new fundamental Python types to describe different types of scalars.

Due to this nature, while working with NumPy, you should ensure to pass a correct type, else Python will raise a TypeError.

Let us take a simple example to reproduce this error.

# import numpy and matplotlib
import numpy as np
import matplotlib.pyplot as plt

# function which accepts scalar value


def my_function(x):
    return int(x)

data = np.arange(1, 22, 0.4)

# passing an array to function
plt.plot(data, my_function(data))
plt.show()

Output

Traceback (most recent call last):
  File "c:\Personal\IJS\Code\main.py", line 14, in <module>
    plt.plot(data, my_function(data))
  File "c:\Personal\IJS\Code\main.py", line 9, in my_function
    return int(x)
TypeError: only size-1 arrays can be converted to Python scalars

In the above example, we have an int function that accepts only single values. However, we are passing an array to the np.int() or int() method, which will not work and results in TypeError.

How to fix TypeError: only size-1 arrays can be converted to python scalars?

There are two different ways to resolve this error. Let us take a look at both solutions with examples.

Solution 1 – Vectorize the function using np.vectorize

If you are working with a simple array and then vectorizing, this would be the best way to resolve the issue.

The int() accepts a single parameter and not an array according to its signature. We can use np.vectorize() function, which takes a nested sequence of objects or NumPy arrays as inputs and returns a single NumPy array or a tuple of NumPy arrays.

Behind the scenes, it’s a for loop that iterates over each array element and returns a single NumPy array as output.

Let us modify our code to use the np.vectorize() method and run the program.

# import numpy and matplotlib
import numpy as np
import matplotlib.pyplot as plt

# function which accepts scalar value

def my_function(x):
    return int(x)


# vectorize the function
f = np.vectorize(my_function)

data = np.arange(1, 22, 0.4)

# passing an array to function
plt.plot(data, f(data))
plt.show()

Output

We can see that error is gone, the vectorize() function will loop through the array and returns a single array that is accepted by the int() function.

TypeError: only size-1 arrays can be converted to python scalars 3

Solution 2 – Cast the array using .astype() method

The np.vectorize() method is inefficient over the larger arrays as it loops through each element.

The better way to resolve this issue is to cast the array into a specific type (int in this case) using astype() method.

# import numpy and matplotlib
import numpy as np
import matplotlib.pyplot as plt

# function which accepts scalar value


def my_function(x):
    # csat the array into integer
    return x.astype(int)


data = np.arange(1, 22, 0.4)

# passing an array to function
plt.plot(data, my_function(data))
plt.show()

Output

TypeError: only size-1 arrays can be converted to python scalars 4

Conclusion

We get TypeError: only size-1 arrays can be converted to python scalars if we pass an array to the method that accepts only scalar values.

The issue can be resolved by using np.vectorize() function a nested sequence of objects or NumPy arrays as inputs and returns a single NumPy array or a tuple of NumPy arrays.

Another way to resolve the error is to use the astype() method to cast the array into an integer type.

January 31, 2022 12:00 PM UTC

Will McGugan

Textual for Windows

Textual adds Windows support

I've just released v0.1.15 of Textual, with Windows support.

The Windows support is somewhat experimental, but so far seems solid. You will get best results on the new Windows Terminal. On the classic command prompt you might find a reduced color palette. This works on VSCode, but is missing mouse input on Windows 10 due to an upstream issue which is apparently fixed in Windows 11. If you have Windows 11, I'd appreciate confirmation on that!

This is the code_viewer example in the Textual repo.

This is the first release under the Textualize umbrella, which is my tech startup funding development. We've been hard at work in a branch adding some very exciting new features which should push the boundaries of what you would think is possible in the terminal. Join the mailing list if you would like to be the first to hear about that.

Windows terminal apps

When it comes to developing terminal apps, MacOS and Linux are essentially the same. Textual shares code for both. Windows works differently, and requires an entirely different API to switch to application (fullscreen) mode and read keys without echo. Recently it got a little easier when Windows added support for virtual terminal sequences (~5 years ago), i.e. the same ansi codes that have been supported in Linux since forever.

The virtual terminal sequences support certainly helped. Textual can re-use the code that generates the display, but I couldn't avoid using the win32 api entirely. In particular, getting updates regarding the terminal size was problematic. Textual should update the display when resizing the window. In Linux that is done via a signal, in Windows that requires subscribing and listening to input events.

Fortunately if you build an app with Textual, you won't have to worry about the differences between these platforms. All the gnarly API details are abstracted with a driver system which ensures that by the time your code receives events any differences in platforms have been abstracted away.

January 31, 2022 11:55 AM UTC

Stack Abuse

Guide to enumerate() in Python - Easy for Loops with Counting

Introduction

Looping with a counter variable/index - a classic in Computer Science! Typically, you'd either explicitly define a counter variable/index, and manually increment it on each loop, or you'd use some sort of syntactic sugar to avoid this process through enhanced for loops:

some_list = ['Looping', 'with', 'counters', 'is', 'a', 'classic!']

# Manual counter incrementation
i = 0
for element in some_list:
    print(f'Element Index: {i}, Element: {element}')
    i += 1

# Automatic counter incrementation
for i in range(len(some_list)):
    print(f'Element Index: {i}, Element: {some_list[i]}')

Both of these snippets result in the same output:

Element Index: 0, Element: Looping
Element Index: 1, Element: with
Element Index: 2, Element: counters
Element Index: 3, Element: is
Element Index: 4, Element: a
Element Index: 5, Element: classic!

Due to how common looping like this is in day-to-day work - the enumerate() function was built into the Python namespace. You can, without any extra dependencies, loop through an iterable in Python, with an automatic counter variable/index with syntax as simple as:

for idx, element in enumerate(some_list):
     print(idx, element)

Note: It's common, but not necessary, convention to name the index as idx if no other label is applicable, since id is a reserved keyword. Commonly, based on the iterable you're working with, more meaningful names can be attributed, such as: batch_num, batch in enumerate(...).

This piece of code results in:

0 Looping
1 with
2 counters
3 is
4 a
5 classic!

Let's dive into the function and explore how it works! It's a classic and common one - and in true Python fashion, it simplifies a common, redundant operation down and improves readability of your code.

The enumerate() Function in Python

The enumerate() function accepts an iterable collection (such as a tuple, list or string), and returns an enumerate object, which consists of a key-set and value-set, where the keys correspond to a counter variable (starting at 0) and the values correspond to the original elements of the iterable collection:

obj = enumerate(some_list)
print(type(obj))
# <class 'enumerate'>

Note: The enumerate object is, itself, iterable! You can use the standard for syntax, unpacking the keys and values of the enumerate object.

Using Python's standard for syntax, we can unpack the keys and values from this object and inspect their types:

for key, value in obj:
    print(type(key), type(value))
    
# <class 'int'> <class 'str'>
# <class 'int'> <class 'str'>
# <class 'int'> <class 'str'>
# <class 'int'> <class 'str'>
# <class 'int'> <class 'str'>
# <class 'int'> <class 'str'>

The data types of the values (elements from the original collection) are retained, so even if you pass custom data types, as long as they're a valid iterable collection - they'll simply be annotated with a counter variable. If you were to collect the object itself into a list, its structure would become very clear:

print(list(obj))
# [(0, 'Looping'), (1, 'with'), (2, 'counters'), (3, 'is'), (4, 'a'), (5, 'classic!')]

It's just a set of tuples with two elements each - a counter variable, starting at 0, and each element of the original iterable mapped to the indices.

You can set an optional start argument, denoting not the starting index in the iterable, but the starting value for the first counter/index that the function will generate. For instance, say we'd like to start at 1 instead of 0:

obj = enumerate(some_list, 1)
print(list(obj))
# [(1, 'Looping'), (2, 'with'), (3, 'counters'), (4, 'is'), (5, 'a'), (6, 'classic!')]

Loop Through Iterable with enumerate()

Having said all that - looping through an enumerate object looks the same as looping through other iterables. The for loop comes in handy here as you can assign reference variables to the returned tuple values. Additionally, there's no need to reference the object explicitly, as it's very rarely used outside of a single loop so the returned value is typically used directly in the loop itself:

# No need to assign the returned `enumerate` object to a distinct reference variable
for idx, element in enumerate(some_list):
     print(f'{idx}, {element}')

This results in:

0, Looping
1, with
2, counters
3, is
4, a
5, classic!

If you'd like to read more about f-Strings and formatting output in Python, read our Guide to String Formatting with Python 3's f-Strings!

Annotating each element in an iterable - or rather, incrementing a counter and returning it, while accessing elements of iterables is as easy as that!

It's worth noting that nothing special really happens within the enumerate() function. It really is, functionally equivalent, to the initial loop we wrote, with an explicit counter variable being returned with an element. If you take a look at the note in the official documentation, the result of the function is functionally equivalent to:

def enumerate(sequence, start=0):
    n = start
    for elem in sequence:
        yield n, elem
        n += 1

You can see that the code is quite similar to the first implementation we've defined:

# Original implementation
i = 0
for element in some_list:
    print(f'Element Index: {i}, Element: {some_list[i]}')
    i += 1
    
# Or, rewritten as a method that accepts an iterable    
def our_enumerate(some_iterable, start=0):
    i = start
    for element in some_iterable:
        yield i, element
        i += 1

The key point here is - the yield keyword defines a generator, which is iterable. By yielding back the index and the element itself, we're creating an iterable generator object, which we can then loop over and extract elements (and their indices) from via the for loop.

If you'd like to read more about the usage of the yield keyword here, read our Guide to Understanding Python's "yield" Keyword!

If you were to use the our_enumerate() function instead of the built-in one, we'd have much the same results:

some_list = ['Looping', 'with', 'counters', 'is', 'a', 'classic!']

for idx, element in our_enumerate(some_list):
     print(f'{idx}, {element}')
        
obj = our_enumerate(some_list)
print(f'Object type: {obj}')

This results in:

0, Looping
1, with
2, counters
3, is
4, a
5, classic!
Object type: <generator object our_enumerate at 0x000002750B595F48>

The only difference is that we just have a generic generator object, instead of a nicer class name.

Conclusion

Ultimately, the enumerate() function is simply syntactic sugar, wrapping an extremely common and straightforward looping implementation.

In this short guide, we've taken a look at the enumerate() function in Python - the built-in convenience method to iterate over a collection and annotate the elements with indices.

January 31, 2022 11:30 AM UTC

ItsMyCode

How to Install Seaborn in Python using the Pip command

This tutorial will learn how to install Seaborn in Python using the Pip command.

Seaborn is a library for making statistical graphics in Python. It is built on top of matplotlib and closely integrated with pandas data structures

Pip is a recursive acronym for “Pip Installs Packages” or “Pip Installs Python.” Basically, it is a package manager that allows you to download and install packages.

Supported Python Version

Python 3.6+ (Recommended)

Note: If you are installing the latest version of Seaborn, the recommended version of Python is 3.6 and above.

Required dependencies

These are the dependent libraries that need to be present. If not already present, these libraries will be downloaded when you install seaborn.

Optional dependencies

statsmodels, for advanced regression plots
fastcluster, for clustering large matrices

How to Install Seaborn in Python using the Pip Command

Before installing the Seaborn, ensure you have the latest version of Pip installed on your computer.

If Pip is not installed correctly or not present, check out the article pip: command not found to resolve the issue.

To install the latest version of Seaborn, run the following pip command.

pip install seaborn

For Python version 3+, it is recommended to use the pip3 command to install the seaborn, as shown below.

pip3 install seaborn

If you would like to install a specific version of Seaborn, you can provide the version number in the pip command as shown below.

pip3 install seaborn==0.11.2

The library is also included as part of Anaconda distribution. You can run the below conda command to install Seaborn on Anaconda.

conda install seaborn

January 31, 2022 10:09 AM UTC

How to Create a Pie Chart in Seaborn

We do not have any built-in function to create Pie chart in seaborn, but with the help of Matplotlib, we can create a pie chart and leverage seaborn for color pallets.

A pie chart (or a circle chart) is a circular statistical graphic, which is divided into slices to illustrate numerical proportion. In this article, let us take a look at creating a pie chart using Matplotlib and Seaborn with examples.

Create a simple Pie chart in Seaborn

Let’s say we need to plot a pie chart for sales delivery of cars sold in different locations.

The Pie chart below demonstrates the proportion of numerical sales data for different locations. We represent the sales data in the form of slices to understand it easily.

In the below example, we have used Matplotlib to draw a pie chart, and we are using a pastel color pallet of seaborn.

The seaborn color_palette() method accepts the color pallet in the string format, and it has a wide variety of color pallets such as dark, bright, pastel, Paired etc.

import matplotlib.pyplot as plt
import seaborn as sns

# car sales data
total_sales = [3000, 2245, 1235, 5330, 4200]

location = ['Bangalore', 'Delhi', 'Chennai', 'Mumbai', 'Kolkatta']

# Seaborn color palette to plot pie chart
colors = sns.color_palette('pastel')

# create pie chart using matplotlib
plt.pie(total_sales, labels=location, colors=colors)
plt.show()

Output

Simple Pie chart in Seaborn

Create an advanced Pie chart in Seaborn

Now that we know how to create a Pie chart using Matplotlib and seaborn, let us explore the advanced features to customize the pie chart.

We have used autopct property to set the percentage of sales inside each slice, making it more effective.
We have the highest car sales in Mumbai. We can represent it differently by making it a standout slice using explode property.
We can rotate the labels in the direction of the slices using the rotatelabel property.
We have changed the color pallet to bright to make the pie chart look more attractive.

import matplotlib.pyplot as plt
import seaborn as sns

# car sales data
total_sales = [3000, 2245, 1235, 5330, 4200]
explode = [0, 0, 0, 0.2, 0]

location = ['Bangalore', 'Delhi', 'Chennai', 'Mumbai', 'Kolkatta']

# Seaborn color palette to plot pie chart
colors = sns.color_palette('bright')

# create pie chart using matplotlib
plt.pie(total_sales, labels=location, colors=colors,
        autopct='%.0f%%', explode=explode, shadow=True, rotatelabels='true')
plt.show()

Output

Advanced Pie chart in Seaborn

January 31, 2022 10:05 AM UTC

How to Add a Title to Seaborn Plots

There are four different methods to add a title to seaborn plots. Let us explore each of these methods in detail with examples.

Method 1: Using set() method

The set() method is used to add different elements to the plot, and we can use it to add the title by using the title parameter in the case of a single seaborn plot.

# import pandas library
import pandas as pd
from pyparsing import line
import seaborn as sns
import matplotlib.pyplot as plt

# create pandas DataFrame
df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'],
                   'points': [10, 8, 3, 5],
                   'runrate': [0.5, 1.4, 2, -0.6],
                   'wins': [5, 4, 2, 2]})

# plot the data frame
line_plt = sns.lineplot(data = df)
line_plt.set(title = "ICC World Cup Standings")
plt.show()

Output

Add a Title to Seaborn Plots using set() method

How to Add a Title to Seaborn Plots 9

Method 2: Using set_title() method

The seaborn plot returns an instance of Matplotlib axes, and for such objects, we can use the set_title() method, which accepts one parameter title as an argument.

# import pandas library
import pandas as pd
from pyparsing import line
import seaborn as sns
import matplotlib.pyplot as plt

# create pandas DataFrame
df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'],
                   'points': [10, 8, 3, 5],
                   'runrate': [0.5, 1.4, 2, -0.6],
                   'wins': [5, 4, 2, 2]})

# plot the data frame
line_plt = sns.lineplot(data = df)
line_plt.set_title("ICC World Cup Standings")
plt.show()

Output

Add a Title to Seaborn Plots

Method 3: Using title() method

We can also leverage matplotlib.pyplot.title() method, which offers a lot of customization such as location, color, font size, etc.

# import pandas library
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

# create pandas DataFrame
df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'],
                   'points': [10, 8, 3, 5],
                   'runrate': [0.5, 1.4, 2, -0.6],
                   'wins': [5, 4, 2, 2]})

# plot the data frame
line_plt = sns.lineplot(data = df)
# set title using matplotlib title
plt.title("ICC World Cup Standings")
plt.show()

Output

Add a Title to Seaborn Plots

Method 4: Using suptitle() method

We can also use suptitle() method to add an overall title to the seaborn plot. This is mainly useful if you have many subplots, but you still need to use one single title to represent the seaborn charts.

import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

# create pandas DataFrame
df = pd.DataFrame({'wins': [12, 11, 10, 3, 11, 20, 2, 30, 12,7],
                   'lost': [6, 4, 5, 3, 10, 7, 2, 12, 0, 6],
                   'team': ['A', 'A', 'A', 'A', 'A', 'B', 'B', 'B', 'B', 'B']})

# plot the data frame
rel = sns.relplot(data=df, x='wins', y='lost', col='team')

# add overall title
rel.fig.suptitle('ICC World Cup Standings')
plt.show()

Output

Add overall title to seaborn using suptitle() method

How to Add a Title to Seaborn Plots 10

January 31, 2022 10:05 AM UTC

Python GUIs

PySide2 vs PySide6: What are the differences, and is it time to upgrade?

If you are already developing Python GUI apps with PySide2, you might be asking yourself whether it's time to upgrade to PySide6 and use the latest version of the Qt library. In this article we'll look at the main differences between PySide2 and PySide6, benefits of upgrading and problems you might encounter when doing so.

Background

Qt is a GUI framework written in the C++ programming language created by Trolltech, now developed by The Qt Company. They also maintain the Qt for Python project, which provides the official Python binding for Qt under the name PySide.

The name PySide was chosen because the word side means binding in the Finnish language.

While the development of Qt started in 1992, it wasn't until 2009 that the Python binding PySide became available. Development of PySide lagged behind Qt for many years, and the other Python binding PyQt became more popular. However, in recent years The Qt Company have been putting increased resources into development, and it now tracks Qt releases closely.

The first version of PySide6 was released on December 10, 2020, just two days after the release of Qt6 itself.

Upgrading from PySide2 to PySide6

The upgrade path from PySide2 to PySide6 is very straightforward. For most applications, just renaming the imports from PySide2 to PySide6 will be enough to convert your application to work with the new library.

If you are considering upgrading, I recommend you try this first and see if works -- if not, take a look at the differences below and see if they apply to your project.

Where things might go wrong

Let’s get acquainted with a few differences between the two versions to know how to write code that works seamlessly with both. After reading this, you should be able to take any PySide2 example online and convert it to work with PySide6. These changes reflect underlying differences in Qt6 vs. Qt5 and aren't unique to PySide itself.

If you’re still using Python 2.x, note that PySide6 is available only for Python 3.x versions.

`QAction` moved

In Qt6 the QAction class, which is used for creation toolbars and menus, has been moved from the QtWidgets to the QtGui module.

PySide2
PySide6

python
from PySide2.QtWidgets import QAction

python
from PySide6.QtGui import QAction

This may seem strange, but the move makes sense since actions can also be used in QML (non-widgets) applications.

High DPI Scaling

The high DPI (dots per inch) scaling attributes Qt.AA_EnableHighDpiScaling, Qt.AA_DisableHighDpiScaling and Qt.AA_UseHighDpiPixmaps have been deprecated because high DPI is enabled by default in PySide6 and can’t be disabled.

QMouseEvent

QMouseEvent.pos() and QMouseEvent.globalPos() methods returning a QPoint object as well as QMouseEvent.x() and QMouseEvent.y() returning an int object have been deprecated – use QMouseEvent.position() and QMouseEvent.globalPosition() returning a QPointF object instead, so like QMouseEvent.position().x() and QMouseEvent.position().y().

Qt.MidButton has been renamed to Qt.MiddleButton

Platform specific

Finally, platform-specific methods in the QtWin and QtMac modules have been deprecated, in favor of using the native calls instead. In PySide applications the only likely consequence of this will be the setCurrentProcessExplicitAppUserModelID call to set an application ID, for taskbar icon grouping on Windows.

QtWin
Native

python
try:
    # Include in try/except block if you're also targeting Mac/Linux
    from PySide2.QtWinExtras import QtWin
    myappid = 'com.learnpyqt.examples.helloworld'
    QtWin.setCurrentProcessExplicitAppUserModelID(myappid)
except ImportError:
    pass

python
try:
    # Include in try/except block if you're also targeting Mac/Linux
    from ctypes import windll  # Only exists on Windows.
    myappid = 'mycompany.myproduct.subproduct.version'
    windll.shell32.SetCurrentProcessExplicitAppUserModelID(myappid)
except ImportError:
    pass

Miscellaneous

QDesktopWidget has been removed – use QScreen instead, which can be retrieved using QWidget.screen(), QGuiApplication.primaryScreen(), or QGuiApplication.screens().
.width() of QFontMetrics has been renamed to .horizontalAdvance().
.get() of QOpenGLVersionFunctionsFactory() has been recommended to be used instead of .versionFunctions() of QOpenGLContext() when obtaining any functions of the Open GL library.
QRegularExpression has replaced QRegExp.
QWidget.mapToGlobal() and QWidget.mapFromGlobal() now accept and return a QPointF object.
All methods named .exec_() (in classes QCoreApplication, QDialog, and QEventLoop) have been deprecated, and should be replaced with .exec() which became possible since Python 3. However, they are still available under the underscored names for backwards compatibility.

`snake_case` and the new `true_property`

PySide2 introduced the snake_case feature to write Qt method names – like .addWidget() – in a Python-friendly snake-case style like .add_widget(). This allows your PySide code to follow the Python standard PEP8 style.

Introduced in PySide6 is a new feature which allows direct access to Qt properties as Python object properties, eliminating the setter and getter methods. This can be enabled explicitly on a per-module basis by importing the true_property feature.

The example below demonstrates the effect on PySide6 code of applying both these features.

Standard API
Pythonic API

python
table = QTableWidget()
table.setColumnCount(2)

button = QPushButton("Add")
button.setEnabled(False)

layout = QVBoxLayout()
layout.addWidget(table)
layout.addWidget(button)

python
from __feature__ import snake_case, true_property

table = QTableWidget()
table.column_count = 2

button = QPushButton("Add")
button.enabled = False

layout = QVBoxLayout()
layout.add_widget(table)
layout.add_widget(button)

As you can see, the true_property feature allows you to assign a value to a Qt property directly – rather than using setters.

In the pre-PySide6 code, you could only do .setEnabled(False) to set the enabled property of a widget to the value False, hence disabling the widget. However, with true_property enabled, you can set a property directly with, for example, button.enabled = False. While this may seem like a cosmetic change, following Pythonic style in this way makes code easier for Python developers to understand and maintain.

PySide6 demo

Let’s demonstrate how these two features could be valuable in your PySide6 code.

python

import sys
import random

from PySide6.QtCore import Slot, Qt
from PySide6.QtWidgets import (
    QLabel,
    QWidget,
    QMainWindow,
    QPushButton,
    QVBoxLayout,
    QApplication,
)

# Import snake_case and true_property after PySide6 imports.
from __feature__ import snake_case, true_property


class MainWindow(QMainWindow):

    def __init__(self):
        super().__init__()

        # Since QMainWindow does not have a fixedSize property,
        # we use the setFixedSize() method but call it in the
        # snake-case style.
        self.set_fixed_size(300, 100)

        # However, QMainWindow does have a windowTitle property
        # for which we assign a value directly but must write
        # the property's name in snake-case style.
        self.window_title = "PySide6 Translator"

        # And this is our non-Qt Python property to which
        # we must assign a value just like above anyway,
        # so assigning values to properties uniformly
        # throughout our code could be intriguing.
        self.multilingual_greetings = (
            "&Pcy&rcy&icy&vcy&iecy&tcy &mcy&icy&rcy!",    # Russian ("Privet mir!" in Cyrillic)
            "Hallo Welt!",    # German
            "¡Hola Mundo!",   # Spanish
            "Hei maailma!",   # Finnish
            "Helló Világ!",   # Hungarian
            "Hallo Wereld!",  # Dutch
        )

        # We create a label with an English greeting by default.
        self.greeting = QLabel("Hello world!")

        # Instead of self.message.setAlignment(Qt.AlignCenter),
        # we set a value to the alignment property directly...
        self.greeting.alignment = Qt.AlignCenter

        # We now also create a button to translate our
        # English greeting and then connect it with
        # our translate_greeting() slot.
        self.translate_button = QPushButton("Translate")
        self.translate_button.clicked.connect(self.translate_greeting)

        self.vertical_layout = QVBoxLayout()
        self.vertical_layout.add_widget(self.greeting)
        self.vertical_layout.add_widget(self.translate_button)

        self.widget_container = QWidget()
        self.widget_container.set_layout(self.vertical_layout)

        # Instead of calling .setCentralWidget(),
        # we call it by its snake-case name...
        self.set_central_widget(self.widget_container)

    @Slot()
    def translate_greeting(self):
        # Here, instead of using the .setText() method,
        # we set a value to the text property directly...
        self.greeting.text = random.choice(self.multilingual_greetings)


if __name__ == "__main__":
    app = QApplication(sys.argv)

    main_window = MainWindow()
    main_window.show()

    app.exec()

`.exec()` or `.exec_()`?

The .exec() method in Qt starts the event loop of your QApplication or dialog boxes. In Python 2.7, exec was a keyword, meaning that it could not be used as a variable name, a function name, or a method name. The solution used in PySide was to name the method as .exec_() – adding a trailing underscore – to avoid a conflict.

Python 3.0 removed the exec keyword, freeing up the name to be used. And since PySide6 targets only Python 3.x versions, it currently deprecates the workaround name and will later remove it. The .exec() method is named just as in Qt itself. However, the .exec_() name only exists for short-term backward compatibility with old code.

If your code must target PySide2 and PySide6 libraries, you can use .exec_(), but beware that this method name will be removed.

Missing modules

This isn’t a concern anymore, but when Qt6 was new, not all of the Qt modules had been ported yet, and so were not available in PySide6. If you needed any of these modules, the upgrade to PySide6 was not desirable then. Fast forward to Qt 6.2 and PySide 6.2, the good news is that all of those missing modules are now back. You can upgrade with no hesitation.

Is it time to upgrade?

Whether or not it's time to upgrade depends on your project. If you're starting out learning PySide (or GUI programming in general), you may prefer to stick with PySide2 for the time being as there are more examples available for PySide2 online. While the differences are minor, anything not working is confusing when you learn. Anything you know using PySide2 will carry over when you choose to upgrade to PySide6.

However, if you're starting a new project and are reasonably familiar with PySide/Qt, I'd recommend jumping into PySide6 now.

If you want to get started with PySide6, the PySide6 book is available with all code examples updated for this latest PySide edition.

`PySide` Backwards compatibility

If you're developing software that's targeting both PySide2 and PySide6 you can use conditional imports to import the classes from whichever module is loaded.

python

try:
    from PySide6 import QtWidgets, QtGui, QtCore # ...
except ImportError:
    from PySide2 import QtWidgets, QtGui, QtCore # ...

If you add these imports to a file in the root of your project named qt.py. You can then, in your own code files import use from qt import QtWidgets and the available library will be imported automatically.

Note however, that importing in this way won't work around any of the other differences between PySide2 and PySide6 mentioned above. For that, we recommend using the QtPy library.

Universal compatibility

If you need to support all Python Qt libraries (PySide2, PySide6, PyQt5, PyQt6) or are dependent on features which have changed between versions of Qt, then you should consider using QtPy. This package is a small abstraction layer around all versions of the Qt libraries, which allows you to use them interchangeably (as far as possible) in your applications.

If you're developing Python libraries or applications that need to be portable across different versions it is definitely worth a look.

Conclusion

As we've discovered, there are no major differences between PySide2 and PySide6. The changes that are there can be easily worked around. If you are new to Python GUI programming with Qt you may find it easier to start with PySide2 still, but for any new project I'd suggest starting with PySide6. It is an LTS (Long Term Support) version. If you’re not upgrading for the benefits, which are not significant, do it for the long-term bug fixes.

For an in-depth guide to building GUIs with Python see my PySide6 book.

January 31, 2022 07:00 AM UTC

Mike Driscoll

PyDev of the Week: Sundeep Agarwal

This week we welcome Sundeep Agarwal (@learn_byexample) as our PyDev of the Week! Sundeep has authored more than 10 books about RegEx, Awk, Python and more! You can see what else Sundeep has been up to by checking out his blog or his GitHub profile.

Sundeep Agarwal

Let's spend some time getting to know Sundeep better!

Can you tell us a little about yourself (hobbies, education, etc):

Hello! My name is Sundeep Agarwal and I'm from India. I did my bachelors in Electronics and Communications, worked at Analog Devices (a semiconductor company) for six years and now write technical books for a living. I help programmers learn tricky topics like Regular Expressions and CLI tools with understandable examples.

I love reading novels, preferred genre these days being fantasy and science-fiction. I used to go on trekking and hiking a lot, but not much opportunities in the past few years.

If I had to choose one notable thing from my country, it'd be the melodious film music that I listen to all day long. Born in North India but raised in South India, I get to enjoy them in two languages!

Why did you start using Python?

I was familiar with Linux, Vim and Perl (for scripting and text processing tasks) while working at Analog Devices. Our college decided to introduce Scripting Course for Electronics students to help them prepare for such jobs. I've been part of the team that conducts such workshops. In 2016, based on industry trend, it was decided to shift to Python from Perl. Around that time, I was learning Python anyway, so I decided to dig deeper for these workshops and started using Python for my scripting tasks as well.

Which Python libraries are your favorite (core or 3rd party)?

I'm biased since they are part of my best selling ebook — the built-in "re" and third-party "regex" modules.

Working with "tkinter" was nice too. I hope to try out other GUI frameworks this year.

What other programming languages do you know and which is your favorite?

I've had to learn or use several programming languages as part of my education and work — C, C++, Java, MATLAB, Perl and Verilog. But I don't use them anymore and don't remember much either!

I dabbled with Ruby while writing ebooks and understood just enough JavaScript to write a Regular Expressions book.

These days, I primarily use Linux CLI one-liners and Vim for most of my programming tasks. I reach for Python if I need more than a few lines of code, so you could say that's my favorite. Last year I made a few GUI apps using Python and that was a nice experience. Hope to do more such projects this year too.

How did you decide to start writing technical books?

The materials I had prepared for college workshops played a role here too! And there were several other factors that led me to try authoring programming books.

I started using GitHub in 2016, attracted by the simplicity of markdown files for presenting programming concepts. I had also been learning and improving my programming skills by answering questions on stackoverflow. So, by the time I decided to write ebooks in 2018, I had more than two years worth of tutorials. I've published 11 ebooks by now, but I still have materials left for more ebooks!

What challenges have you had writing the books and how did you overcome them?

I found it difficult to brace myself for valid criticism, like grammar and cover design. They only made my ebooks better when I tried to incorporate the suggestions, but it isn't easy to face faults in your creative work.

What projects are you working on now?

I'm currently writing a Vim Reference Guide for beginner to intermediate level users.

Aiming to publish the ebook in February.

Is there anything else you’d like to say?

Thank you Mike for the opportunity to share my journey here.

Wishing all readers a great year ahead 🙂

Thanks for doing the interview, Sundeep!

The post PyDev of the Week: Sundeep Agarwal appeared first on Mouse Vs Python.

January 31, 2022 06:05 AM UTC

Podcast.init

Building A Detailed View Of Your Software Delivery Process With The Eiffel Protocol

The process of getting software delivered to an environment where users can interact with it requires many steps along the way. In some cases the journey can require a large number of interdependent workflows that need to be orchestrated across technical and organizational boundaries, making it difficult to know what the current status is. Faced with such a complex delivery workflow the engineers at Ericsson created a message based protocol and accompanying tooling to let the various actors in the process provide information about the events that happened across the different stages. In this episode Daniel Ståhl and Magnus Bäck explain how the Eiffel protocol allows you to build a tooling agnostic visibility layer for your software delivery process, letting you answer all of your questions about what is happening between writing a line of code and your users executing it.

Summary

Announcements

Hello and welcome to Podcast.__init__, the podcast about Python’s role in data and science.
When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
Your host as usual is Tobias Macey and today I’m interviewing Daniel Ståhl and Magnus Bäck about Eiffel, an open protocol for platform agnostic communication for CI/CD systems

Interview

Introductions
How did you get introduced to Python?
Can you describe what Eiffel is and the story behind it?
- What are the goals of the Eiffel protocol and ecosystem?
- What is the role of Python in the Eiffel ecosystem?
What are some of the types of questions that someone might ask about their CI/CD workflow?
- How does Eiffel help to answer those questions?
Who are the personas that you would expect to interact with an Eiffel system?
Can you describe the core architectural elements required to integrate Eiffel into the software lifecycle?
- How have the design and goals of the Eiffel protocol/architecture changed or evolved since you first began working on it?
What are some example workflows that an engineering/product team might build with Eiffel?
What are some of the challenges that teams encounter when integrating Eiffel into their delivery process?
What are the most interesting, innovative, or unexpected ways that you have seen Eiffel used?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on Eiffel?
When is Eiffel the wrong choice?
What do you have planned for the future of Eiffel?

Keep In Touch

Daniel
- d-stahl-ericsson on GitHub
- LinkedIn
Magnus
- LinkedIn
- magnusbaeck on GitHub

Picks

Tobias
- Red Notice
Daniel
- The Witcher
Magnus
- Lego

Closing Announcements

Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@podcastinit.com) with your story.
To help other people find the show please leave a review on iTunes and tell your friends and co-workers

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

January 31, 2022 02:04 AM UTC

January 30, 2022

ItsMyCode

How to Fix: module ‘pandas’ has no attribute ‘dataframe’

We get AttributeError: module ‘pandas’ has no attribute ‘dataframe’ when the Pandas module is unable to resolve and initialize the DataFrame class. The AttributeError usually occurs if the class is not defined in the module or if it has a different name.

In this article, let us take a look at what AttributeError: module ‘pandas’ has no attribute ‘dataframe’ error means and how to resolve this with examples.

AttributeError: module ‘pandas’ has no attribute ‘dataframe’ Solution

There are four main reasons behind this error.

Ignoring the case of while creating DataFrame
Declaring the module name as a variable name
Naming file as pd.py or pandas.py
Pandas module is not installed

Reason 1 – Ignoring the case of while creating DataFrame

The DataFrame class name is case-sensitive and, it is represented in camel-case, if you are using pd.dataframe() all in lower case then you will get module ‘pandas’ has no attribute ‘dataframe’ as shown below.

# import pandas library
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

# create pandas DataFrame
df = pd.dataframe({'team': ['India', 'South Africa', 'New Zealand', 'England'],
                   'points': [10, 8, 3, 5],
                   'runrate': [0.5, 1.4, 2, -0.6],
                   'wins': [5, 4, 2, 2]})

print(df)

Output

raceback (most recent call last):
  File "c:\Personal\IJS\Code\main.py", line 7, in <module>
    df = pd.dataframe({'team': ['India', 'South Africa', 'New Zealand', 'England'],
    raise AttributeError(f"module 'pandas' has no attribute '{name}'")

AttributeError: module 'pandas' has no attribute 'dataframe'. Did you mean: 'DataFrame'?

We can fix the issue by changing the dataframe to DataFrame(lowercase to camel-case) to create the Pandas DataFrame. When we run the program now, we don’t get any errors and, the code runs successfully.

# import pandas library
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

# create pandas DataFrame
df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'],
                   'points': [10, 8, 3, 5],
                   'runrate': [0.5, 1.4, 2, -0.6],
                   'wins': [5, 4, 2, 2]})

print(df)

Output

           team  points  runrate  wins
0         India      10      0.5     5
1  South Africa       8      1.4     4
2   New Zealand       3      2.0     2
3       England       5     -0.6     2

Reason 2 – Declaring the module name as a variable name

We get this error if you have named any variable as ‘pd‘ or ‘pandas‘ and try to create the DataFrame.

# import pandas library
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

# variable named as pd
pd = [2, 3]
# create pandas DataFrame
df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'],
                   'points': [10, 8, 3, 5],
                   'runrate': [0.5, 1.4, 2, -0.6],
                   'wins': [5, 4, 2, 2]})

print(df)

Output

Traceback (most recent call last):
  File "c:\Personal\IJS\Code\main.py", line 9, in <module>
    df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'],
AttributeError: 'str' object has no attribute 'DataFrame'

In our above code, we have a local variable named ‘pd‘ and Python will not be able to resolve the DataFrame as it is a local variable and not the Pandas module.

We can resolve this error by renaming the variable to something else.

# import pandas library
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

# variable named as pd
data = [2, 3]
# create pandas DataFrame
df = pd.DataFrame({'team': ['India', 'South Africa', 'New Zealand', 'England'],
                   'points': [10, 8, 3, 5],
                   'runrate': [0.5, 1.4, 2, -0.6],
                   'wins': [5, 4, 2, 2]})

print(df)

Output

           team  points  runrate  wins
0         India      10      0.5     5
1  South Africa       8      1.4     4
2   New Zealand       3      2.0     2
3       England       5     -0.6     2

Reason 3 – Naming file as pd.py or pandas.py

If you have named the script as pd.py or pandas.py then you will get module ‘pandas’ has no attribute ‘dataframe’ error.

This mainly happens because the file name will shadow the Pandas module and, it can mess up the module imports.

We can fix this issue by renaming the script to some other name such as “my_script.py”.

Reason 4- Pandas package is not installed

The last reason could be if Pandas library is not installed in the proper Python path. You can go and verify the Lib/site-packages path and see if the Pandas library exists.

If it’s not present then you can install pandas by running the below command.

pip install pandas

Installation instructions for Anaconda is found here

January 30, 2022 01:04 AM UTC

Armin Ronacher

Uninitialized Memory: Unsafe Rust is Too Hard

Rust is in many ways not just a modern systems language, but also quite a pragmatic one. It promises safety and provides an entire framework that makes creating safe abstractions possible with minimal to zero runtime overhead. A well known pragmatic solution in the language is an explicit way to opt out of safety by using unsafe. In unsafe blocks anything goes.

If you have read this article before you might be surprised that it looks quite different now. This article in itself was a victim of the author being confused by the rules surrounding unsafe. It has since been changed with an alternative example that better explains the pitfalls. A thank you goes to eddyb who pointed out my mistakes on reddit.

I made the case on Twitter a few days ago that writing unsafe Rust is harder than C or C++, so I figured it might be good to explain what I mean by that.

From C to Rust

So let's start with something simple: we have some struct that we want to initialize with some values. The interesting value here will be the name. It's a pointer to an allocated string. Other than that where it's allocated doesn't matter to us so we keep the struct itself on the stack. The idea is that after the initialization that thing can be passed around safely and printed.

#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>

struct role {
    char *name;
    bool disabled;
    int flag;
};

int main() {
    struct role r;
    r.name = strdup("basic");
    r.flag = 1;
    r.disabled = false;
    printf("%s (%d, %s)\n", r.name, r.flag, r.disabled ? "true" : "false");
    free(r.name);
}

Now let's write this in Rust. Let's not read the docs too much, let's just do a 1:1 translation to more or less the same but by using unsafe. One note here before you read the code: we're purposefully trying to create an object that looks familiar to Rust programmers and can be seen as public API. So we use a String here instead of a C string so there are some changes to the C code.

use std::mem;

struct Role {
    name: String,
    disabled: bool,
    flag: u32,
}

fn main() {
    let role = unsafe {
        let mut role: Role = mem::zeroed();
        role.name = "basic".to_string();
        role.flag = 1;
        role.disabled = false;
        role
    };

    println!("{} ({}, {})", role.name, role.flag, role.disabled);
}

So immediately one will ask why unsafe is needed here and the answer is that of course you don't need it here. However this code is also using a suboptimal function: std::mem::zeroed. If you run this on a recent Rust compiler you will get this result:

thread 'main' panicked at 'attempted to zero-initialize type `Role`,
  which is invalid', src/main.rs:11:30

On older Rust compilers this code will run but it was never really correct. So how do we solve this? The compiler already tells us that we need to use something else:

warning: the type `Role` does not permit zero-initialization
  --> src/main.rs:11:30
   |
11 | let mut role: Role = mem::zeroed();
   |                      ^^^^^^^^^^^^^
   |                      |
   |                      this code causes undefined behavior when executed
   |                      help: use `MaybeUninit<T>` instead, and only call
   |                         `assume_init` after initialization is done
   |

So why does this type not support zero initialization? What do we have to change? Can zeroed not be used at all? Some of you might think that the answer is #[repr(C)] on the struct to force a C layout but that won't solve the problem. We in fact need to reach for MaybeUninit as the compiler indicates. So let's try that first and then afterwards we figure out why we need it:

use std::mem::MaybeUninit;

struct Role {
    name: String,
    disabled: bool,
    flag: u32,
}

fn main() {
    let role = unsafe {
        let mut uninit = MaybeUninit::<Role>::zeroed();
        let role = uninit.as_mut_ptr();
        (*role).name = "basic".to_string();
        (*role).flag = 1;
        (*role).disabled = false;
        uninit.assume_init()
    };

    println!("{} ({}, {})", role.name, role.flag, role.disabled);
}

By swapping out zeroed for MaybeUninit::zeroed everything changes. We can no longer manipulate our struct directly, we now need to manipulate a raw pointer. Because that raw pointer does not implement deref and because Rust has no -> operator we now need to dereference the pointer permanently to assign the fields with that awkward syntax.

So first of all: does this work now? The answer is yes. But is it correct? The answer is not. But let's see what changed? The answer lies in the fact that any construct like a mutable reference (&mut) or value on the stack in itself (even in unsafe) that would be valid outside of unsafe code still needs to be in a valid state at all times. zeroed returns a zeroed struct and there is no guarantee that this is a valid representation of either the struct or the fields within it. In our case it happens that our String is valid with everything zeroed out but this is not guaranteed and undefined behavior.

One important note is that a mutable reference must also never point to an invalid object, so doing let role = &mut *uninit.as_mut_ptr() if that object is not fully initialized is also wrong.

So let's change from zeroed to uninit. If we run it again we're crashing. So why are we crashing? The answer is that by assigning a string to name we also drop the old string that was there before. We just happened to not encounter this before because Drop happened to be able to deal with a zeroed out string, but we were deep in undefined behavior there. Now how do we solve that? We need to somehow directly write to the pointer there.

So let's just accept that MaybeUninit is necessary and we need to deal with raw references here. It's somewhat cumbersome but it doesn't look too bad. So now we have two new problems: we know that &mut X is not allowed, but *mut X is. How do we get a *mut X without using &mut X first? Ironically until Rust 1.51 it was impossible to construct such a thing without breaking the rules. Today you can use the addr_of_mut! macro. So we can do this:

let name_ptr = std::ptr::addr_of_mut!((*role).name);

Great, so now we have this pointer. How do we write into it? We can use the write method instead:

addr_of_mut!((*role).name).write("basic".to_string());

Are we okay now? Remember how we used a regular struct? If we read the documentation we learn that there are no guarantees of such a struct at all. It turns out that despite what the documentation currently says we can rely on fields being aligned. If however we were dealing with #[repr(packed)] we would have to use write_unaligned instead which is legal if Rust were to pick for a member of the struct to be unaligned. So this could be the final version:

use std::mem::MaybeUninit;
use std::ptr::addr_of_mut;

struct Role {
    name: String,
    disabled: bool,
    flag: u32,
}

fn main() {
    let role = unsafe {
        let mut uninit = MaybeUninit::<Role>::uninit();
        let role = uninit.as_mut_ptr();
        addr_of_mut!((*role).name).write("basic".to_string());
        (*role).flag = 1;
        (*role).disabled = false;
        uninit.assume_init()
    };

    println!("{} ({}, {})", role.name, role.flag, role.disabled);
}

When to use addr_of_mut!

There are two cases to consider: uninitialized memory and unaligned references. You're not allowed to (even temporarily) create an unaligned reference to something and you're not allowed to create a reference to uninitialized memory. So when are these references created?

If you write (*role).flag = 1; this is fine by Rust rules if the type does not Drop. If it does, then we have more a problem: Drop::drop gets called and it gets called on uninitialized memory. So in that case we need to go via addr_of_mut!. This is why we can directly assign to flag, but we need to go via addr_of_mut! for the name as it is a String.

MaybeUninit

A meta issue is that the understanding of safety changed with time. At one point mem::uninitialized was considered a sound API. At a later point MaybeUninit was added to address the detected short comings. However MaybeUninit in practical terms not ideal because of partially initialized types. While MaybeUninit<T> and T are memory compatible thanks to #[repr(transparent)] this does not work well with nested use.

It's not uncommon that you need to have a MaybeUninit on a field of a struct, but at a later point you want this abstraction not to be there. Actually working with MaybeUninit in practice can be a very challenging experience which this blog post does not sufficiently capture.

Is my Unsafe Correct?

It's 2022 and I will admit that I no longer feel confident writing unsafe Rust code. The rules were probably always complex but I know from reading a lot of unsafe Rust code over many years that most unsafe code just did not care about those rules and just disregarded them. There is a reason that addr_of_mut! did not get added to the language until 1.53. Even today the docs both say there are no guarantees on the alignment on native rust struct reprs.

Over the last few years it seem to have happened that the Rust developers has made writing unsafe Rust harder in practice and the rules are so complex now that it's very hard to understand for a casual programmer and the documentation surrounding it can be easily misinterpreted. An earlier version of this article for instance assumed that some uses of addr_of_mut! were necessary that really were not. And that article got quite a few shares overlooking this before someone pointed that mistake out!

These rules have made one of Rust's best features less and less approachable and also harder to understand. The requirement for the existence MaybeUninit instead of “just” having the old mem::uninitialized API is obvious but shows how complex the rules of the language are.

I don't think this is good. In fact, I believe this is not at all a great trend that fewer and fewer people seem to understand unsafe rust. C interop is a bit part of what made Rust great, and that we're creating such massive barriers should be seen as undesirable. More importantly: the compiler is not helpful in pointing out when I'm doing something wrong.

Making unsafe more ergonomic is a hard problem for sure but it might be worth addressing. Because one thing is clear: people won't be stopping writing unsafe code any time soon.

January 30, 2022 12:00 AM UTC

Planet Python

February 01, 2022

What is TypeError: can only concatenate str (not “int”) to str

Example Scenario

How to fix TypeError: can only concatenate str (not “int”) to str

Solution

Conclusion

Discussions

Python Jobs

Articles & Tutorials

Projects & Code

Events

REST APIs In The Cloud

Previous Tutorials

What is An Arithmetic Sequence?

Nth Term In An Arithmetic Sequence

Sum Of N Terms In An Arithmetic Sequence In Python

Conclusion

CVE-2022-22818: Possible XSS via {% debug %} template tag

CVE-2022-23833: Denial-of-service possibility in file uploads

Affected supported versions

Resolution

General notes regarding security reporting

Changes for the User

Changes for the System Administrator

Changes for the Developer

What's New in Wing 8.1

Support for Containers and Clusters

New Package Management Tool

Improved Project Creation

Improved Python Code Analysis and Warnings

And More

Try Wing 8 Now!

January 31, 2022

What’s new?

1. A simpler UI to configure your targets

2. Creating virtual environments inside targets

Ready to join the EAP?

Ground rules

How to download?

Index and key lookups both use square brackets

Making square brackets work on your objects

Summary

Understanding the Mandelbrot Set

The Icon of Fractal Geometry

The Boundary of Iterative Stability

What is TypeError: only size-1 arrays can be converted to python scalars?

How to fix TypeError: only size-1 arrays can be converted to python scalars?

Solution 1 – Vectorize the function using np.vectorize

Solution 2 – Cast the array using .astype() method

Conclusion

Textual adds Windows support

Windows terminal apps

Introduction

The enumerate() Function in Python

Loop Through Iterable with enumerate()

Conclusion

Supported Python Version

Required dependencies

Optional dependencies

How to Install Seaborn in Python using the Pip Command

Create a simple Pie chart in Seaborn

Create an advanced Pie chart in Seaborn

Method 1: Using set() method

Method 2: Using set_title() method

Method 3: Using title() method

Method 4: Using suptitle() method

Background

Upgrading from PySide2 to PySide6

Where things might go wrong

CVE-2022-22818: Possible XSS via `{% debug %}` template tag