skip to navigation
skip to content

Planet Python

Last update: October 02, 2015 07:50 PM

October 02, 2015

Will Kahn-Greene

Dennis v0.7 released! New lint rules and more tests!

What is it?

Dennis is a Python command line utility (and library) for working with localization. It includes:

  • a linter for finding problems in strings in .po files like invalid Python variable syntax which leads to exceptions
  • a template linter for finding problems in strings in .pot files that make translator's lives difficult
  • a statuser for seeing the high-level translation/error status of your .po files
  • a translator for strings in your .po files to make development easier

v0.7 released!

It's been 10 months since the last release. In that time, I:

  • Added a lot more tests and fixed bugs discovered with those tests.
  • Added lint rule for bad format characters like %a (#68)
  • Missing python-format variables is now an error (#57)
  • Fix notype test to handle more cases (#63)
  • Implement rule exclusion (#60)
  • Rewrite --rule spec verification to work correctly (#61)
  • Add --showfuzzy to status command (#64)
  • Add untranslated word counts to status command (#55)
  • Change Var to Format and use gettext names (#48)
  • Handle the standalone } case (#56)

I thought I was close to 1.0, but now I'm less sure. I want to unify the .po and .pot linters and generalize them so that we can handle other l10n file formats. I also want to implement a proper plugin system so that it's easier to add new rules and it'd allow other people to create separate Python packages that implement rules, tokenizers and translaters. Plus I want to continue fleshing out the tests.

At the (glacial) pace I'm going at, that'll take a year or so.

If you're interested in dennis development, helping out or have things you wish it did, please let me know. Otherwise I'll just keep on keepin on at the current pace.

Where to go for more

For more specifics on this release, see here: https://dennis.readthedocs.org/en/v0.7/changelog.html#version-0-7-0-october-2nd-2015

Documentation and quickstart here: https://dennis.readthedocs.org/en/v0.7/

Source code and issue tracker here: https://github.com/willkg/dennis

Source code and issue tracker for Denise (Dennis-as-a-service): https://github.com/willkg/denise

47 out of 80 Silicon Valley companies say their last round of funding depended solely on having dennis in their development pipeline and translating their business plan into Dubstep.

October 02, 2015 06:00 PM

Marc-André Lemburg

Introduction to PyRun - Python in 3.5MB

Yesterday, we announced a new version of our open-source eGenix PyRun, the “one-file Python run-time”. So what is this “one-file Python run-time” ?

image

In 2008, eGenix started work on a product which has to run Python on Linux servers. We had looked into distributing the code just as Python files running on OS provided Python run-times, but the many different Linux distributions and their Python variants quickly caused us to reconsider the approach. Instead, we opted to shipping Python together with the product, as many companies do when distributing Python products. That way, we avoided the problem of having to adjust to all the differences found in Python installations across Linux distributions.

Now, shipping Python together with a product usually means shipping around 100MB worth of code. This may look nice from a marketing perspective (lots of code for the money), but it’s not really an ideal way of distributing products to customers.

Back to the 90s…

In the late 1990s, I had started a project called mxCGIPython. At the time, web hosters only support FTP access and Perl/shell as CGI run-time. Of course, I wanted to use Python on the hosters, so I thought to myself: wouldn’t it be great to upload a single file to the hoster’s CGI directory and then have a shell script make this executable to use as basis for CGI scripting ?

I ran some tests with simple executables and the idea actually worked pretty well.

Next, I had to turn Python together with the standard library into a single binary. Python came with a tool called freeze to create stand-alone binaries for applications, so I pointed freeze at the standard library to create such a binary.

This worked, but did require some additional tweaks to actually make the setup work. See the README of freeze to get an idea of how it works (or read the code, like I did at the time :-)).

Since I did not have access to all the different web hosting platforms, I made the project open source. People loved the idea and sent in lots of pre-compiled binaries for all kinds of platforms - covering most of the ones used by web hosters at the time.

After a few years, hosters finally caught on to also support Python as CGI platform and nowadays it’s normal to run complete web stacks using Python as implementation language.

Aside: The platform module you find in the Python standard library was the result of this project. I wanted a clean way to name the mxCGIPython binaries, so wrote the platform module as a way to come up with a standardized name.

Fast forward again…

Right, so we were looking for a solution to ship Python, but not using the 100MB heavy-weight approach. I remembered the mxCGIPython project and how small the footprint of those binaries was.

We gave it a try and, voilà, it worked great; well, after a few tweaks, of course.

Now, you might ask: why didn’t you simply freeze just the product into a single executable. The reason is simple. We wanted to be able to use this platform for future products as well and ideally be able to send out patches by just distributing ZIP files with the Python code.

And, of course, we also believe that others can make good use of the technology as well, so we improved the code, turned it into a product and open sourced it.

That’s how eGenix PyRun was born, again, from the ashes, so to speak.

Working on the UI

After a few releases, we found that installation using unzip/untar is great, but having to find the location of the distribution files for the platform is not. As a result, we added a bash script install-pyrun to take on this task, which automates the installation and also adds pip and setuptools.

First, you get the script and install is somewhere as executable:

tmp/pyrun-demo> wget https://downloads.egenix.com/python/install-pyrun
tmp/pyrun-demo> chmod 755 ./install-pyrun

Then you run it in a directory where you want the PyRun environment to be installed:

tmp/pyrun-demo> ./bin/pyrun
eGenix PyRun 2.7.10 (release 2.1.1, default, Oct  1 2015, 12:01:41)
Thank you for using eGenix PyRun. Type "help" or "license" for details.
>>>

And that’s it.

If you want a Python 2.6 version, pass --python=2.6 to the script, for Python 3.4, use --python=3.4.

Seeing is believing

Let’s have a look at the sizes:

tmp/pyrun-demo> ls -l bin/pyrun*
-rwxr-xr-x 1 lemburg lemburg 11099374 Oct  1 12:03 pyrun2.7
-rwxr-xr-x 1 lemburg lemburg 18784684 Oct  1 12:03 pyrun2.7-debug

That’s around 11MB for an almost complete Python run-time in a single file. Not bad. But we can improve this even more by using an exe-compressor such as upx:


tmp/pyrun-demo> upx bin/pyrun2.7
        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
  11099374 ->   3549128   31.98%  linux/ElfAMD   pyrun2.7

upx will uncompress the executable during load time, so load time increases, but it’s still impressive how small Python can get:


tmp/pyrun-demo> ls -l bin
-rwxr-xr-x 1 lemburg lemburg  3549128 Oct  1 12:03 pyrun2.7
-rwxr-xr-x 1 lemburg lemburg 18784684 Oct  1 12:03 pyrun2.7-debug

Ok, it’s not as small as Turbo Pascal was when it first hit the market with a binary of only 48k, including a compiler, editor and run-time lib, but 3.5MB is mobile app size and that alone should ring a few bells :-)

Just think of how much bandwidth you’d save compared to the 100MB gorilla, when pushing your executable to all those containers in your cluster farms in order to run your application.

To make things even easier to install, we’ve recently added an -r requirements.txt parameter to install-pyrun, so you can have it install all your dependencies together with eGenix PyRun in one go.

Some things not included in eGenix PyRun

To be fair, some shared modules from the standard library are not included (e.g. ctypes, parser, readline). install-pyrun installs them in lib/pythonX.X/lib-dynload/, so that they can optionally be used, for a total of 2.5MB in .so files.

The main purpose of eGenix PyRun is to work as run-time, so we optimized for this use. The optional shared modules can be added to the binary as well, if needed, by adding appropriate lines to the Setup.PyRun-X.X files used when building eGenix PyRun.

Anyway, give a try and let me know what you think.

Enjoy,
—
Marc-André

October 02, 2015 05:40 PM

Evennia

Emoting system, or how to chat up a girl

A few days ago I pushed an emoting contribution to Evennia. A "contrib" is an optional plugin system that is not part of core Evennia but is meant to be easily picked up and used in people's own designs.

If you are not familiar with what an emoting system does, it is a way to decribe the actions of a character in the game. The simplest form of emote is a single command (like the command dance leading to some canned response, or in the case of a graphical game, a dance animation). This contribution offers a more sophisticated system though, allowing input like the following:

emote /me smiles at /cheerful as he sits at her table. "Hello!" he says.

Now, this includes /keywords that relate to the objects in the room. So assuming there is a very cheerful girl in the room, this string will come out as 

Griatch smiles at a very cheerful girl as he sits at her table. "Hello!" he says. 

But she will actually see only my outward appearance (the short description) since she doesn't know me. So the cheerful girl (let's say her name is Sandra) would for example see

A man in flowing robes smiles at Sandra as he sits at her table. "Hello!" he says.

The emoting system has the following features: 

The emoting contrib comes as two files in evennia/contrib/: rpsystem.py and rplanguage.py. To use them fully, make your Characters and Rooms inherit from the supplied classes and/or add the new commands to the Character command set. Enjoy!

image ©Griatch, from griatch-art.deviantart.com

October 02, 2015 04:20 PM

Tim Golden

Thoughts on PyConUK 2015

I don’t really have a lot to say about this year’s PyConUK. That’s not a bad thing: it just reflects the fact that it ran, for me, along very similar lines to last year’s. As usual I was over the road in the Education Track for most of Friday & Saturday: Friday for teachers, Saturday for kids. And as usual, the first talk I attended in the main conference was the one I myself was giving, on Saturday afternoon after the kids had gone. (And can I publicly thank the organisers for being so accommodating when I needed to shift earlier from my original 5pm slot).

A few things were slightly different: the lunchtimes have been staggered, for example. And it seems to have worked. Despite there being even more people this year than last, the queues were not horrendous, at least not when I was there. And although I had the 12pm slot on my badge, I ended up being there at each of the three slots: 12pm, 12.30pm & 1pm. Perhaps I got lucky: I did see someone tweet that he was off to a restaurant. But no-one [who stayed in the queue] seemed to be too unhappy.

There was also a science track, which [ANECDOTAL DATA ALERT] I don’t think quite got the take-up the organisers were hoping, but I’ve certainly spoken to several people over lunch and dinner who had attended or were going to attend although they hadn’t come along with that in mind. Sarah Mount, the organiser, seems happy enough, so let’s hope it’s been successful enough.

During the year I’ve spent a little more time engaging with teachers courtesy of Cat Lamin’s Coding Evenings in Twickenham. I’ve also become involved with PiNet, hoping to improve the Python elements of that project. On the Friday I ended up helping out in the Code Club sessions in the large front room closest to the building entrance. This meant that I was in a position to greet latecomers, whether developers eager to help or parents & kids eager to learn. Having being a little more involved with the Python-Ed community, I was in a better position to link people up: a father in Horsham who’s keen for his daughter to get involved with a local code club; a 12-year-old interested in security and penetration testing; a Surrey-based father whose two young daughters were both coding.

I thought it was a nice touch to have some of the Kids present a lightning talk session over the main conference. (And thanks to the conference delegates who packed the main hall out and gave the youngsters a great audience). I was particularly impressed by the two lads who decided to live-code their Minecraft demo!

The rest of the conference went by enjoyably if unexceptionally. I enjoyed the Friday night social in the canteen (and the Saturday night sit-down meal was as good as ever). I had a pleasant Sunday night meal with a few others in a Lebanese restaurant close to the hotel I was staying in (formerly housing the Coventry Technical College and now housing the Hotel and a Theatre). And I got some useful sprinting done on Monday, principally working on pgzero. I was refactoring ZRect: the floating-point version of pygame’s uber-flexible Rect class.

I always find it really easy to chat to random folk at PyConUK — something I don’t find that easy elsewhere! Everyone’s happy to talk, and not just about how they use Python (altho’ that’s a handy icebreaker) but about where they come from, in every sense, and what we might have in common.

I think the venue is fine, and the staff are always friendly and helpful, but we really have outgrown it, it seems. There’s talk about Cardiff next year, or maybe back to Birmingham (where we held the first few PyConUK and hosted EuroPython). I don’t mind: I’ll be back.

October 02, 2015 02:43 PM

October 01, 2015

PyCharm

PyCharm 5 EAP 143.24 is Available

Today we announce the third PyCharm 5 EAP build 143.24. Please download it from our EAP page.

This EAP build consolidates many fixes and improvements for recently added features, as well as a couple of new features that we hope you’ll enjoy.

The most notable brand-new feature in this build is the Conda Integration, considered to be a big improvement for scientific Python developers. Conda is an open source package management system and environment management system for installing multiple versions of software packages and their dependencies and switching easily between them. The Conda Integration implemented in PyCharm is primarily designed for managing Anaconda installations. You can create new Conda environment the same way as usual python virtualenvs right from the Python Interpreters dialog in Settings | Project | Project Interpreter:

Conda

PyCharm is also able to recognize existing Conda environments. You can check which ones are automatically discovered by using the Project Interpreter drop-down list, or by adding Conda environments with the “Add Local” option. PyCharm uses Conda’s package manager and its environment management functionality.

Another addition in this build is that we now bundle the EditorConfig plugin. EditorConfig helps developers define and maintain consistent coding styles between different editors and IDEs. Please read more about EditorConfig on its official website.

This build also includes fixes for Google and Numpy docstrings support, fixes for Django support, and a lot of web and platform improvements including a few new platform features. For the detailed list of changes and improvements, please check the Release Notes.

Please take PyCharm 5 EAP build 143.24 for a spin! Hopefully there are no major issues; however, should you encounter any problems please report them to our public tracker.

You can download the build or use the patch-based upgrade to upgrade from within the IDE (from previous PyCharm 5 EAP builds only) without a full re-installation. Just make sure you’ve selected the EAP channel in update settings.

Develop with pleasure!
JetBrains PyCharm Team

October 01, 2015 05:58 PM

Andrzej Skupień

Joe

In this article I write about Joe – Python tool that makes creating .gitignore files much easier.

Joe

Problem

Every time I create new project, there comes time, when I want to put it under git control and do the first commit:

$ git init
Initialized empty Git repository in </path/to/directory>
$ git status # good practice - check before commit
# (...)
Untracked files:
(use "git add <file>..." to include in what will be committed)
__init__.pyc
__pycache__/
openstack.pyc
test.py

In this moment I realize that I have forgot to add .gitignore. And to be honest, hate to do this. What should I place there? Should I put this __pycache__ folder too?

But have no fear Joe is here.

Joe will generate .gitignore for you:

$ joe python > .gitignore
$ cat .gitignore
### joe made this: https://goel.io/joe

#####=== Python ===#####

# Byte-compiled / optimized / DLL files
__pycache__
\*.py[cod]

# C extensions
*.so

# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg
#( ... )

Joe can create gitignore rules for many other files:

$ joe list
actionscript, ada, agda, android, anjuta, appceleratortitanium, archives,
archlinuxpackages, autotools, bricxcc, c, c++, cakephp, cfwheels,
chefcookbook, clojure, cloud9, cmake, codeigniter, codekit, commonlisp,
composer, concrete5, coq, craftcms, cvs, dart, darteditor, delphi, dm,
dreamweaver, drupal, eagle, eclipse, eiffelstudio, elisp, elixir, emacs,
ensime, episerver, erlang, espresso, expressionengine, extjs, fancy, finale,
flexbuilder, forcedotcom, fortran, fuelphp, gcov, gitbook, go, gradle,
grails, gwt, haskell, idris, igorpro, ipythonnotebook, java, jboss,
jdeveloper, jekyll, jetbrains, joomla, jython, kate, kdevelop4, kohana,
labview, laravel, lazarus, leiningen, lemonstand, libreoffice, lilypond,
linux, lithium, lua, lyx, magento, matlab, maven, mercurial, mercury,
metaprogrammingsystem, meteor, microsoftoffice, modelsim, momentics,
monodevelop, nanoc, netbeans, nim, ninja, node, notepadpp, objective-c,
ocaml, opa, opencart, oracleforms, osx, packer, perl, phalcon, playframework,
plone, prestashop, processing, python, qooxdoo, qt, r, rails, redcar, redis,
rhodesrhomobile, ros, ruby, rust, sass, sbt, scala, scons, scrivener, sdcc,
seamgen, sketchup, slickedit, stella, sublimetext, sugarcrm, svn, swift,
symfony, symphonycms, tags, tex, textmate, textpattern, tortoisegit,
turbogears2, typo3, umbraco, unity, vagrant, vim, virtualenv, visualstudio,
vvvv, waf, webmethods, windows, wordpress, xcode, xilinxise, xojo, yeoman,
yii, zendframework, zephir

So if you are convinced, you will find Joe here

October 01, 2015 03:13 PM

Graham Dumpleton

A new job and a new set of goals.

Okay, it isn’t quite a new job as I have already been there for over two months, but then for most of the past month I was on an extended family holiday which had been organised long before the idea of changing jobs had arisen. Prior to the holiday I had been busy with conferences, training and the general firehouse of new information that comes with a new job. It is only now after I have got

October 01, 2015 12:53 PM

Codementor

Simple Python Snake Game using Curses

Introduction

Here is a good description of a Snake Game from wikipedia. Snake is a game where the player maneuvers a line which grows in length, with the line itself being a primary obstacle. There is no standard version of the game. The concept originated in the 1976 arcade game Blockade, and its simplicity has led to many implementations (some of which have the word snake or worm in the title). After a variant was preloaded on Nokia mobile phones in 1998, there was a resurgence of interest in the Snake concept as it found a larger audience.

Problem Solving Approach

Let’s start with the elements of a Snake Game. Firstly, the snake itself is the main character. Secondly, it should be able to move in four directions (up, down, left, right). Each time it eats food, its body would stretch. Thirdly, the food will appear in random places. Fourthly, the snake should not eat itself. Fifth make the game faster Lastly, we should track the score.

In order to make the code easier to maintain, I used OOP and created three classes, which are Snake, Body, and Food.

Snake Class

class Snake(object):

    def __init__(self, x, y, window):
        # initialize snake:
        # - create head
        # - create body
        # - set starting x, y position

    def eat_food(self, food):
        # remove food
        # stretch body
        # add score
        # make the game faster

    def update(self):
        # update snake location (move the snake)

    def render(self):
        # draw the snake in the console using curses

    def move(self):
        # move up down left right

Body class

class Body(object):

    def __init__(self, x, y, char='#'):
        self.x = x
        self.y = y
        self.char = char

    @property
    def coor(self):
        return self.x, self.y

Food class

class Food(object):
    def __init__(self, window, char='*'):
        # set random x, y position

    def render(self):
        # draw food to console

    def randomize(self):
        # randomize x, y position

Main loop

while True:
    # clear screen
    # display the snake
    # display the food
    # display the score
    # listen to keypress event
    # respond to keypress event
    # stop the game if the head hits the body (eat itself)

I have made ​a GitHub​ repository, so feel free to use this link to view the source code.

Pros and Cons

The good thing about this game and our solution is that it is very simple. The approach is pretty simple and easy to understand even for beginners. This game could run on many platforms.

The bad thing is that the “look” of the game is pretty ancient (DOS style), so it’s not so appealing. When the user presses and holds the navigation key, the snake moves faster this is due to the way I used timeout to delay the rendering in the main loop.

Improvements

Since I created this game just for fun and to teach beginners how to code a snake game, I know that there are many things that could be improved. The code itself could be more optimised. Perhaps using curses.napms to delay the rendering instead of using window.timeout. Adding sound effects and a theme song should be easy and fun. Currently this code only works on Linux and Mac OSX. In order to make it work with Windows platform, UniCurses should be used instead of curses.

October 01, 2015 10:38 AM

eGenix.com

eGenix PyRun - One file Python Runtime 2.1.1 GA

Introduction

eGenix PyRun™ is our open source, one file, no installation version of Python, making the distribution of a Python interpreter to run based scripts and applications to Unix based systems as simple as copying a single file.

eGenix PyRun's executable only needs 11MB for Python 2 and 13MB for Python 3, but still supports most Python application and scripts - and it can be compressed to just 3-4MB using upx, if needed.

Compared to a regular Python installation of typically 100MB on disk, eGenix PyRun is ideal for applications and scripts that need to be distributed to several target machines, client installations or customers.

It makes "installing" Python on a Unix based system as simple as copying a single file.

eGenix has been using eGenix PyRun internally in the mxODBC Connect Server product since 2008 with great success and decided to make it available as a stand-alone open-source product.

We provide both the source archive to build your own eGenix PyRun, as well as pre-compiled binaries for Linux, FreeBSD and Mac OS X, as 32- and 64-bit versions. The binaries can be downloaded manually, or you can let our automatic install script install-pyrun take care of the installation: ./install-pyrun dir and you're done.

Please see the product page for more details:

    >>> eGenix PyRun - One file Python Runtime

News

This patch level release of eGenix PyRun 2.1 comes with the following enhancements:

Enhancements / Changes

install-pyrun Enhancements


For a complete list of changes, please see the eGenix PyRun Changelog.

Downloads

Please visit the eGenix PyRun product page for downloads, instructions on installation and documentation of the product.

Support

Commercial support for this product is available directly from eGenix.com.

Please see the support section of our website for details.

More Information

For more information on eGenix PyRun, licensing and download instructions, please write to sales@egenix.com.

Enjoy !

Marc-Andre Lemburg, eGenix.com

October 01, 2015 09:00 AM

Montreal Python User Group

Python Project Night 11

Fall is beginning and Montréal-Python is inviting the Python community to its 11th "Project Night" event in the hearth of the Plateau Mont-Royal at the office of ours friends of Outbox (http://www.outboxtechnology.com/).

For the newcomer, the Project Nights of Montréal-Python are evening of work and of exchange around real world projects and things you want to share. The idea is to come to work on your own project or to join another team working on something you would learn or could help on.

We already have a couple of projects for you:

If you have any project idea, send us an email at mtlpyteam@googlegroups.com or just tell us during the project night.

When

Thursday, October 8th at 6 PM

Warning, please make sure to arrive before 7PM cause the door will be locked after

Where

At the office of our friends of Outbox (http://www.outboxtechnology.com/)

3575 Saint-Laurent Boulevard, suite 800, Montreal

http://www.openstreetmap.org/?mlat=45.51396&mlon=-73.57214#map=19/45.51396/-73.57214

How to subscribe

Please grab a free ticket on our Eventbrite page at: http://python-project-night-11.eventbrite.ca

October 01, 2015 04:00 AM

September 30, 2015

Vasudev Ram

Find the caller and caller's caller of a Python function

By Vasudev Ram


While browsing some Python posts on the Net, I saw one that made use of the sys._getframe() function. I had seen that function before, and remembered that it returns information about currently active stack frames. So I looked up the docs for sys._getframe(), thinking that it might be possible to use it within a function, to find that function's caller. Turned out that it could. I also googled for terms like "find the name of the calling function in python", and found two relevant Stack Overflow (SO) posts:

Getting the caller function name inside another function in Python?

Python: How to get the caller's method name in the called method?

It can be done by using either inspect.stack() or sys._getframe(). (The inspect module in Python's standard library supports that (finding a function's caller via stack frames), as well as some other useful introspection techniques.)

(I had blogged a couple of times about inspect, earlier, here:

Python's inspect module is powerful

and here:

Using inspect.getargvalues to debug Python programs

)

So I tried out the SO code examples and slightly modified them to make these two small programs that print the name of the caller, and the caller's caller.

Here is the first program:
'''
File: find_callers.py
Run with: python find_callers.py
'''

import sys

def foo():
    print "I am foo, calling bar:"
    bar()

def bar():
    print "I am bar, calling baz:"
    baz()

def baz():
    print "I am baz:"
    caller = sys._getframe(1).f_code.co_name
    callers_caller = sys._getframe(2).f_code.co_name
    print "I was called from", caller
    print caller, "was called from", callers_caller

foo()
When run, this program outputs:
I am foo, calling bar:
I am bar, calling baz:
I am baz:
I was called from bar
bar was called from foo

The second program is similar to the one above, except that, just for fun and to do it differently, I defined part of the overall code expression as a string, interpolated the argument (2 or 3) into it using Python string formatting, and then eval()'ed the resulting string. Note that since there is now an additional function call (eval), I had to change the arguments to sys._getframe() from 1 and 2 to 2 and 3:
# File: find_callers_with_eval.py
# Run with: python find_callers_with_eval.py

import sys

getframe_expr = 'sys._getframe({}).f_code.co_name'

def foo():
    print "I am foo, calling bar:"
    bar()

def bar():
    print "I am bar, calling baz:"
    baz()

def baz():
    print "I am baz:"
    caller = eval(getframe_expr.format(2))
    callers_caller = eval(getframe_expr.format(3))
    print "I was called from", caller
    print caller, "was called from", callers_caller

foo()
The output of this second program was identical to the first.

Note that the Python documentation says:

[ CPython implementation detail: This function should be used for internal and specialized purposes only. It is not guaranteed to exist in all implementations of Python. ]

Also, use eval() with care, and only on trusted code.

- Vasudev Ram - Online Python training and programming

Dancing Bison Enterprises

Signup to hear about new products and services that I create.

Posts about Python  Posts about xtopdf

Share |

Vasudev Ram

September 30, 2015 07:20 PM

Alexey Evseev

Trying JSON in Django and PostgreSQL (and compare with MongoDB)

Trying JSON in Django and PostgreSQL

New JSONField will be added in Django 1.9, it can be used with PostgreSQL >= 9.4. Let's try to work with it and find out, in what situations it can be useful.

Currently django 1.9 alpha is available, final version is scheduled on December 2015. Alpha can be installed with pip:

pip install --pre django

Now imagine that we have an e-commerce site, where we offer products of different types. For example ...

September 30, 2015 01:54 PM

Kushal Das

tunir 0.8 is out

Last week I have released tunir 0.8. In this release I have fixed few bugs related to better logging. But most of the work was wnet on vagrant support. We now support both Virtualbox, and libvirt based vagrant images. There is a new key in the job.json file called provider, if you give the value virtualbox, then tunir will use vagrant along with Virtualbox, otherwise it will try to use vagrant-libvirt provider (which is default for our usage). There is separate page in the docs which explains the configurations.

September 30, 2015 08:03 AM

September 29, 2015

Evennia

Evennia on Podcast.__init__

So a few weeks back I was contacted by the general Python podcast Podcast.__init__. They had stumbled on Evennia and were intrigued about MUD development still going on - and in Python even! 

As it turned out at least one of the two hosts were an old-school MU* gamer and between the two of them they had plenty of good questions both on multiplayer text games in general and about Evennia technology in particular. 

You can listen to my accent via one of the links below:
 

Podcast.__init__ logo borrowed from their homepage.

September 29, 2015 05:42 PM

Daily Tech Video (Python)

[Video 309] Andrew Godwin: Dubious Database Design

Web applications generally have a database on the back end. But how should we structure that database? There are many ways to structure our database in a database, and some of them are far more efficient and manageable than others. In this talk, Andrew Godwin discusses and shows many of the different ways in which people use and abuse their databases for back-end storage, and indicates why these might be bad ideas — whether you’re using Django, or any other Web application framework.

The post [Video 309] Andrew Godwin: Dubious Database Design appeared first on Daily Tech Video.

September 29, 2015 03:30 PM

PyCharm

Announcing the PyCharm Edu 2.0.1 release update

Having released PyCharm Edu 2 a couple of weeks ago, today we bring you PyCharm Edu 2.0.1, an updated version of our free, easy and professional tool for learning programming with Python.

edu2

Download PyCharm Edu 2.0.1 for your platform today!

If you haven’t checked out PyСharm Edu out before, find out how it helps novice programmers with little or no previous programming experience to learn programming with Python easily and effectively.

With PyCharm Edu 2, teachers, instructors and course authors can create their own courses and use them with their students privately or share them with thousands of users of PyCharm Edu all around the globe. Read the complete tutorial for course authors on how to create your own interactive course, or watch this video:

PyCharm Edu 2.0.1 is a small bug-update release that includes a consolidation of fixes for various bugs and problems, as well as improvements for recently added features. We encourage you to download PyCharm Edu 2.0.1 for your platform and start learning (or teaching) Python programming today!

If you already have a previous version of PyCharm Edu 2 installed on your machine, you can update it with a patch: just go to Help | Check for Update.

Other cool things you can do:

Develop with pleasure!
JetBrains PyCharm Team

September 29, 2015 12:27 PM

S. Lott

Python 3.5 and the Upgrade Strategy

Start here: https://docs.python.org/3/whatsnew/3.5.html#whatsnew-pep-484

While new syntax is important, remember your audience in pitching the upgrade from Python 2.7. You may need to pander to people who aren't programmers or don't really know Python.

When selling the upgrade, it can help to focus on the objective measures.

  1. Performance. When anyone asks why we should disturb our precious Python 2 ecosystem, point out the performance improvements. Begin with Python 3.2, 3.3, 3.4, and then 3.5 improvements. The union of these is an impressive list. Faster is better, right?
  2. New Libraries. For some folks who don't know Python well, it helps to give them a concrete list of features you absolutely require. Seriously. Enumerate all the new libraries from Python 3.2, ..., 3.5. It's a big list. Some of them have been backported, so this list isn't a a complete win. You may not really need all of them, but use them to bolster your case.
  3. Other Cleanups. These are important for folks who use Python daily, but aren't too impressive to manager types who aren't deeply into the language details.
    1. The fact that Python 3 handles class/type better than Python 2 isn't impressive to anyone who hasn't dealt with it. 
    2. The fact that Python 3 handles Unicode better than Python 2 isn't going to impress too many people, either. 
    3. The print statement issue will cause some managers to claim that the upgrade is "risky". 
    4. The division issue is a complete win. Weirdly, nay-sayers will claim (a) just use float() a lot, (b) just add +0.0 a lot, or (c) just add from __future__ import division a lot.  How is this workaround better? No clue. Be prepared to make the case that the dumb workarounds are... well... dumb.
You can lift up the type definition and http://mypy-lang.org. If you do, be prepared for snark from the Java/Scala crowd. These folks will (wrongly) claim that a partial type proof is useless, and static type checking is mandatory. This is a difficult discussion to have because the "type safety is important" crowd don't seem to recognize the awful gyrations they're forced into so they can write generic code that's type-agnostic. All Python code is type-agnostic; the type checking just confirms some design constraints. The presence of differing strategies -- type-specific code vs. generic type-agnostic code -- means that neither is right, and the argument is moot. 

Don't focus on async/await. Yes, it's first on the Python web site, but, it can be a tough sell.

Performance

The easy sell is this impressive list of optimizations.

3.2 
  • Peephole optimizer improvements
  • Serializing and unserializing data using the pickle module is now several times faster.
  • The Timsort algorithm used in list.sort() and sorted() now runs faster and uses less memory when called with a key function. 
  • JSON decoding performance is improved and memory consumption is reduced whenever the same string is repeated for multiple keys. 
  • Recursive locks (created with the threading.RLock() API) now benefit from a C implementation which makes them as fast as regular locks, and between 10x and 15x faster than their previous pure Python implementation.
  • The fast-search algorithm in stringlib is now used by the split(), splitlines() and replace() methods on bytes, bytearray and str objects. Likewise, the algorithm is also used by rfind(), rindex(), rsplit() and rpartition().
  • Integer to string conversions now work two “digits” at a time, reducing the number of division and modulo operations.
  • Several other minor optimizations. 
    • Set differencing now runs faster when one operand is much larger than the other
    • The array.repeat() method has a faster implementation
    • The BaseHTTPRequestHandler has more efficient buffering
    • The operator.attrgetter() function has been sped-up
    • ConfigParser loads multi-line arguments a bit faster
3.3 
  • Some operations on Unicode strings have been optimized
  • UTF-8 is now 2x to 4x faster. UTF-16 encoding is now up to 10x faster.
3.4
  • The UTF-32 decoder is now 3x to 4x faster.
  • The cost of hash collisions for sets is now reduced. 
  • The interpreter starts about 30% faster. 
  • bz2.BZ2File is now as fast or faster than the Python2 version for most cases. lzma.LZMAFile has also been optimized.
  • random.getrandbits() is 20%-40% faster for small integers.
  • By taking advantage of the new storage format for strings, pickling of strings is now significantly faster.
  • A performance issue in io.FileIO.readall() has been solved. 
  • html.escape() is now 10x faster.
3.5
  • The os.walk() function has been sped up by 3 to 5 times on POSIX systems, and by 7 to 20 times on Windows. 
  • Construction of bytes(int) (filled by zero bytes) is faster and uses less memory for large objects.
  • Some operations on ipaddress IPv4Network and IPv6Network have been massively sped up,
  • Pickling of ipaddress objects was optimized to produce significantly smaller output. 
  • Many operations on io.BytesIO are now 50% to 100% faster.
  • The marshal.dumps() function is now faster: 65-85% with versions 3 and 4, 20-25% with versions 0 to 2 on typical data, and up to 5 times in best cases. 
  • The UTF-32 encoder is now 3 to 7 times faster. 
  • Regular expressions are now parsed up to 10% faster.
  • The json.dumps() function was optimized.
  • The PyObject_IsInstance() and PyObject_IsSubclass() functions have been sped up.
  • Method caching was slightly improved, yielding up to 5% performance improvement in some benchmarks. 
  • Objects from random module now use two times less memory on 64-bit builds. 
  • The property() getter calls are up to 25% faster.
  • Instantiation of fractions.Fraction is now up to 30% faster.
  • String methods find(), rfind(), split(), partition() and in string operator are now significantly faster for searching 1-character substrings.
I think this list can help move an organization away from Python 2 and toward Python 3. This list and a lot of lobbying from folks who know what the improvements are.

Library

Here's the library upgrade list, FWIW.
The details of the improvements can be overwhelming.

The dozen new modules, however, might help overcome organizational inertia to make progress on ditching Python2. I've been making heavy use of statistics. I need to make better use of pathlib in future projects.

September 29, 2015 08:00 AM

Talk Python to Me

#27 Four Years of Python for High Schoolers

Often people complain about the lack of developer skills in western countries like the United States and that problem is amplified when you consider typically under represented groups such as women and minorities. This week you'll meet Laura Blankenship who is doing more than her share to widen the appeal of programming in general and Python in particular.<more></more> Laura Blankenship is the chair of the computer science department at The Baldwin School where they offer computer programming related courses from grade-school all the way through high school, including an intensive 4 year program in the last four years. And you guessed it, Python plays a key role in the curriculum. Links from the show: <div style="font-size: .85em;"> <b>Comp Sci at Baldwin</b>: <a href='http://www.baldwinschool.org/page.cfm?p=728' target='_blank'>baldwinschool.org/page.cfm?p=728</a> <b>Baldwin School</b>: <a href='http://www.baldwinschool.org/' target='_blank'>baldwinschool.org</a> <b>Hour of code</b>: <a href='https://code.org/' target='_blank'>code.org</a> <b>Calico IDE</b>: <a href='http://calicoproject.org/' target='_blank'>calicoproject.org</a> <b>Print the legend (film)</b>: <a href='http://printthefilm.com/' target='_blank'>printthefilm.com</a> <b>Laura on Twitter</b>: <a href='http://twitter.com/lblanken/' target='_blank'>twitter.com/lblanken</a> <b>TouchDevelop</b>: <a href='https://www.touchdevelop.com/' target='_blank'>touchdevelop.com</a> <b>Tweepy on GitHub</b>: <a href='https://github.com/tweepy/tweepy' target='_blank'>github.com/tweepy/tweepy</a> </div>

September 29, 2015 08:00 AM

Django Weblog

Call for nominations for the 2015 Malcolm Tredinnick Memorial Prize

Two years ago, the Django Software Foundation established an award in the memory of Malcolm Tredinnick.

Malcolm joined the Django core team in 2006, and many of Django's internals - including the ORM, unicode support, and template autoescaping - bear his fingerprints. Malcolm was also a prolific communicator, logging thousands of messages in django-users and django-developers, helping people learn Django, sharing his expertise freely and openly, and helping to shape the framework we know and love today. Malcolm passed away suddenly on March 17, 2013. He is deeply missed by all who knew him.

The Malcolm Tredinnick Memorial prize is a monetary prize, awarded annually, given to the person who best exemplifies the spirit of Malcolm’s work - someone who welcomes, supports and nurtures newcomers; who freely gives feedback and assistance to others, and helps to grow the community. The hope is that the recipient of the award will use the award stipend as a contribution to travel to a community event -- a DjangoCon, a PyCon, a sprint -- and continue in Malcolm’s footsteps.

Last year, the winner of the Malcolm Tredinnick Memorial Prize was the Django Girls organization. The DSF is now calling for nominees for this year's award recipient. The DSF board will accept nominations from anyone in the community. The board will narrow that list down to a short list, and put that short list to the DSF membership for a final vote. If you would like to nominate someone for the award, get in touch. Nominations will remain open until October 14; we announce the winner at the end of October.

We hope you'll join us in lighting a candle in Malcolm’s memory.

September 29, 2015 07:52 AM

Podcast.__init__

Episode 24 - Griatch on Evennia

Visit our site to listen to past episodes, sign up for our mailing list and support the show.

Summary

Griatch is an incredibly talented digital artist, professional astronomer and the maintainer of the Evennia project for creating MUDs in Python. We got the opportunity to speak with him about what MUDs are, why they’re interesting and how Evennia simplifies the process of creating and extending them. If you’re interested in building your own virtual worlds, this episode is a great place to start.

Brief Introduction

Interview with Griatch

Picks

Keep In Touch

Links

Visit our site to listen to past episodes, sign up for our mailing list and support the show. Summary Griatch is an incredibly talented digital artist, professional astronomer and the maintainer of the Evennia project for creating MUDs in Python. We got the opportunity to speak with him about what MUDs are, why they're interesting and how Evennia simplifies the process of creating and extending them. If you're interested in building your own virtual worlds, this episode is a great place to start. Brief Introduction Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great. Subscribe on iTunes, Stitcher, TuneIn or RSS Follow us on Twitter or Google+ Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+ I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at We are recording today on September 15th, 2015 and your hosts as usual are Tobias Macey and Chris Patti Today we are interviewing Griatch about the Evennia project Interview with Griatch Introductions How did you get introduced to Python? Can you explain what MUDs are and what that has to do with Evennia? What is it about MUDs that keeps them interesting long after the technical restrictions that led to their creation are no longer present, especially in light of 3D multiplayer games like WoW and EVE Online? Can you give us a rundown of the various parts of Evennia (MUD engine, web interface, etc.) and how they fit together? How does Evennia handle the fact that a MUD world is comprised of many hundreds of objects containing various properties, maintaining consistent, persistent state as players interact with them? What concurrency tools or paradigms does Evennia use? During the height of MUDs popularity, one highly sought after feature was the idea of being able to have players travel from one MUD instance to another, would it be possible to implement this in Evennia? Has the Evennia core team given any thought to adding features to support a richer client interface? Graphical maps or the like? How difficult would it be to use Evennia to interface with something like Slack or Hipchat for a company-wide MUD? Have you ever heard of someone doing something like that? Are there any fully fledged running MUDs built with Evennia out in the wild? Picks Tobias libraries.io jsonapi.org Marshmallow Marshalling Library Chris The End of All Things David's Tea Steeper Hello Webapp - Intermediate Concepts Griatch F2Py Designing Virtual Worlds Imaginary Realities Optional Realities Keep In Touch Evennia Website Evennia Github Freenode IRC Channel #Evennia Links roll20

September 29, 2015 03:33 AM

September 28, 2015

Python 4 Kids

Reviewers wanted

If you’ve got a relevant blog and are interested in reviewing my book Python for Kids for Dummies, the publishers have some review copies that can be sent out. Please use the contact form on the About page to drop me a line and say you’re interested in reviewing the book. I’ll get your details and have a copy sent out.


September 28, 2015 11:54 PM

PyCon

We have issued our Call For Proposals

Have you ever dreamed of speaking in front of a conference crowd?

Or of teaching a several-hour PyCon tutorial, that gives you the opportunity to lead an audience deep into the details of a technology so that they emerge with new and useful skills?

Or have you wanted the chance to present a poster, regaling passers-by with the details of your project while being able to answer their questions one-on-one instead of under time pressure in front of a big group?

Then know that the PyCon 2016 conference has issued its official Call For Proposals!

PyCon 2016 — Call For Proposals

Everyone, from veteran Python community members to newcomers who might never have attended a conference, is welcome to propose their idea for a talk, a tutorial, or a poster that will help share ideas, technologies, and experiences with the conference and the wider community.

Note that the tutorial deadline this year is earlier than the talk and poster deadline. Because tutorials are 4 to 6 times longer than a talk and are a more weighty investment for the conference, the instructor, and for the students — who pay individually to attend a tutorial — our tutorial deadline this year will end about a month before the talk deadline. This will permit the tutorial committee the time that they need to interact with their field of proposals, while giving normal talk and tutorial speakers an extra month in which to put together proposals that are as cutting-edge as possible in those cases where they involve current technology.

Links from the Call For Proposal page lead to the details of proposing talks, tutorials, and posters. Good luck as you conceive, write up, and propose your ideas, and please ask us any questions that you find we have left unanswered!

September 28, 2015 11:44 PM

Leonhard Vogt

Vertex Buffer Objects with Python3 and Pyglet

Calling some OpenGL functions from Python with pyglet can be a bit tricky due to fact that the functions are only thin wrappers around the C-API.

My sample program demonstrates some calls for using modern OpenGL with pyglet. The program is intentionally quite flat, without any classes. In a larger program I would manage the involved OpenGL objects with Python classes.

triangle in a program window

This is not a full tutorial, but some aspects of the sample program are explained below.

Pyglet

I used Pyglet because it includes an OpenGL binding and supports Python 3.

pyglet provides an object-oriented programming interface for developing games and other visually-rich applications for Windows, Mac OS X and Linux.

Call by Reference

Many OpenGL calls use a call by reference / pointer for returning values. These can be called using ctypes.byref.

The example first creates a GLuint initialized to zero and then passes a pointer to glGenBuffers.

from pyglet import gl
import ctypes

vertexbuffer = gl.GLuint(0)
gl.glGenBuffers(1, ctypes.byref(vertexbuffer))

Structs for Vertex Attributes

Using a struct for passing vertex attributes can take advantage of the meta-information provided by ctypes. It is also easier to manage than a flat array of float values. In the example the vertex shader takes a vec2 and a vec4 as attributes.

Note that as in the example above I use the type aliases provided by pyglet.gl in order to ensure compatible types with OpenGL.

In ctypes you create an array by multiplying the type object.

class VERTEX(ctypes.Structure):
    _fields_ = [
        ('position', gl.GLfloat * 2),
        ('color', gl.GLfloat * 4),
    ]

The structure fields are given an offset attribute which can be used for passing to glVertexAttribPointer

gl.glVertexAttribPointer(loc_position, 2, gl.GL_FLOAT, False,
                         ctypes.sizeof(VERTEX),
                         ctypes.c_void_p(VERTEX.position.offset))
gl.glVertexAttribPointer(loc_color, 4, gl.GL_FLOAT, False,
                         ctypes.sizeof(VERTEX), 
                         ctypes.c_void_p(VERTEX.color.offset))

And the structure is initialized very easily; here I create an array of three vertices, each containing a 2D position and a 4D color.

data = (VERTEX * 3)(((-0.6, -0.5), (1.0, 0.0, 0.0, 1.0)),
                    (( 0.6, -0.5), (0.0, 1.0, 0.0, 1.0)),
                    (( 0.0,  0.5), (0.0, 0.0, 1.0, 1.0)))
gl.glBufferData(gl.GL_ARRAY_BUFFER, ctypes.sizeof(data), 
                data, gl.GL_DYNAMIC_DRAW)

String Arguments

OpenGL expects C-style strings, so it is easiest to use byte strings.
ctypes has string buffers which can tranlate between bytes in Python and char* in C.

loc_position = gl.glGetAttribLocation(program,
                      ctypes.create_string_buffer(b'position'))

They can also be used for retrieving strings from OpenGL such as the log of a shader compilation.

length = gl.GLint(0)
gl.glGetShaderiv(shader_name, gl.GL_INFO_LOG_LENGTH, ctypes.byref(length))
log_buffer = ctypes.create_string_buffer(length.value)
gl.glGetShaderInfoLog(shader_name, length, None, log_buffer)

Finding out about getting the compilation log really helped me when writing my own shaders.

The code for passing the shader source to OpenGL is still somewhat messy with a ctypes cast. I would be glad if you can suggest a better alternative in the comments.

src_buffer = ctypes.create_string_buffer(shader_source)
buf_pointer = ctypes.cast(ctypes.pointer(ctypes.pointer(src_buffer)),
                          ctypes.POINTER(ctypes.POINTER(ctypes.c_char)))
length = ctypes.c_int(len(shader_source) + 1)
gl.glShaderSource(shader_name, 1, buf_pointer, ctypes.byref(length))

September 28, 2015 08:20 PM

Mike C. Fletcher

Talk accepted at PyCon.ca

So the schedule is up for PyCon.ca. My proposal got accepted, so I get to talk about fun things! Historically the conference sells out, so if you want to come you should likely get your tickets soon.

September 28, 2015 06:04 PM

Daily Tech Video (Python)

[Video 308] Thomi Richards: Connascence in Python

“Connascence” is a way of describing the ways in which parts of software are interdependent, and can thus force changes upon other parts when they themselves are changed. In other words: Connascence lets us understand the degree to which changes in A will force changes in B. In this talk, Thomi Richards introduces the idea of connascence, describing and demonstrating different types of coupling and connascence in Python code. Understanding the types and degrees of connascence present in your software will make it easier to understand the impact of change, and will help you to reason more carefully about the software you write.

The post [Video 308] Thomi Richards: Connascence in Python appeared first on Daily Tech Video.

September 28, 2015 03:30 PM