Explore All i18n Advantages of Babel for Your Python App

Python i18n

Python Babel is a well-known solution that assists in internationalizing and localizing Python applications, with an emphasis on web-based applications. If you are working with dates and times, numbers or getttext translations, then it will help you automate a part of the process. In this tutorial, we are going to explore the advanced usages of this library with practical examples.

In the past, we have given detailed tutorials on Gettext tools as well as integrating Gettext with Python. We are going to extend our knowledge by using Babel and see some practical examples of its usage in python i18n. We are also going to see how to integrate it with Jinja2 templates and how to integrate Phrase’s In-Context Editor into a Flask application to help with the translation process by simply browsing the website and edit texts along the way, making your python localization process a lot simpler. You can also find the code described in this tutorial on Github.

About Babel

Babel provides for internationalization (python i18n) and localization (python l10n) helpers and tools that work on two areas. The first is the gettext module that usegettext to update, extract and compile message catalogs and manipulate PO  files. The second one is the usage of CLDR (Common Locale Data Repository) to provide formatting methods for currencies, dates, numbers etc. based on a locale parameter. Both aspects aim to help to automate the process of internationalizing Python applications as well as providing convenient methods for accessing and using this data.

Babel, in essence, works as an abstraction mechanism for a larger message extraction framework, as you can extend it with your own extractors and strategies that are not tied to a particular platform.

Installing and using Babel

Installing Babel is simple using pip

$ pip install Babel

If you don’t have pip installed, you can get it with easy_install

$ sudo easy_install pip

Working with Locale Data

(CLDR) Unicode Common Locale Data Repository is a standardized repository of locale data used for formatting, parsing and displaying locale-specific information. Instead of translating, for example, day names or month names for a particular language or script, you can make use of the translations provided by the locale data included with Babel based on CLDR data.

Let’s see some examples…

Create a file name loc.py  and add the following code:

from babel import Locale

# Parsing
l = Locale.parse('de-DE', sep='-')
print("Locale name: {0}".format(l.display_name))

l = Locale.parse('und_GR', sep='_')
print("Locale name: {0}".format(l.display_name))

# Detecting
l = Locale.negotiate(['de_DE', 'en_AU'], ['de_DE', 'de_AT'])
print("Locale negociated: {0}".format(l.display_name))

print(Locale('it').english_name)
print(Locale('it').get_display_name('fr_FR'))
print(Locale('it').get_language_name('de_DE'))
print(Locale('de', 'DE').languages['zh'])

print(Locale('el', 'GR').scripts['Copt'])

# Calendar
locale = Locale('it')
month_names = locale.days['format']['wide'].items()
print(list(month_names))

We are showing some examples of the Locale class. It is used to print, negotiate or identify language tags. If you run this example, you will see the following output:

$ python loc.py
Locale name: Deutsch (Deutschland)
Locale name: Ελληνικά (Ελλάδα)
Locale negociated: Deutsch (Deutschland)
Italian
italien
Italienisch
Chinesisch
Κοπτικό
[(6, 'domenica'), (0, 'lunedì'), (1, 'martedì'), (2, 'mercoledì'), (3, 'giovedì'), (4, 'venerdì'), (5, 'sabato')]

These are useful as they are provided by the (CLDR) dataset and do not need translation.

Apart from that, there are several functions that format dates, times, currencies, units etc. Let’s see some examples:

from babel.dates import format_date, format_datetime, format_time
from babel.numbers import format_number, format_decimal, format_percent, parse_decimal
from babel.units import format_unit

from datetime import date, datetime, time

# Date, time

d = date(2010, 3, 10)
print(format_date(d, format='short', locale='it'))
print(format_date(d, format='full', locale='it'))

print(format_date(d, "EEEE, d.M.yyyy", locale='de'))

dt = datetime.now()
print(format_datetime(dt, "yyyy.MMMM.dd GGG hh:mm a", locale='en'))

# Numbers/ Units

print(format_decimal(123.45123, locale='en_US'))
print(format_decimal(123.45123, locale='de'))

print(format_unit(12, 'length-meter', locale='en_GB'))
print(format_unit(12, 'length-meter', locale='en_US'))

parse_decimal('2.029,98', locale='de')

The output is:

$ python loc.py
10/03/10
mercoledì 10 marzo 2010
Mittwoch, 10.3.2010
2018.August.28 AD 10:24 AM
123.451
123,451
12 metres
12 meters
2029.98

Message Extraction

Babel has an extraction mechanism similar to gettext. It works by walking through the specified directories and based on the configuration rules it applies extractor functions to those files matched. This way there is more flexible than gettext  as you can leverage the expression power of Python to extend the tool.

Babel comes with a few built-in extractors such as python, javascript, and ignore (which extracts nothing) and you can create your own extractors and there are two different front-ends to access this functionality:

  • A Command-Line Interface
  • Distutils/Setuptools Integration

In this tutorial, we are going to use the Command-Line Interface.

To use it just invoke the  pybabel   tool for example to print all known locales

$ pybabel --list-locales:

To actually use the tooling lets walk through the process of extracting messages using pybabel :

Create a file named main.py  and add the following code:

import gettext
_ = gettext.gettext

print(_('This is a translatable string.'))

Note that the usage of gettext  is only convenient because the default extractor uses gettext  behind the scenes, but Babel, in general, is not tied to that.

Use the  pybabel extract   command to create the initial message catalog:

$ mkdir locale
$ pybabel extract . -o locale/base.pot

That will createa base pot file that will contain the following messages:

#: main.py:4
msgid "This is a translatable string."
msgstr ""

You don’t have to edit this file now.

Using the  init  command, we can create a new translation catalog based on that POT  template file:

$ pybabel init -l el_GR de_DE en_US -i locale/base.pot -d locale
creating catalog locale/el_GR/LC_MESSAGES/messages.po based on locale/base.pot

$ pybabel init -l de_DE en_US -i locale/base.pot -d locale
creating catalog locale/de_DE/LC_MESSAGES/messages.po based on locale/base.pot

$ pybabel init -l en_US -i locale/base.pot -d locale
creating catalog locale/en_US/LC_MESSAGES/messages.po based on locale/base.pot

Those files are ready to be translated. When the translations are done, you can use the compile command to turn them into MO  files:

$ pybabel compile -d locale
compiling catalog locale/en_US/LC_MESSAGES/messages.po to locale/en_US/LC_MESSAGES/messages.mo
compiling catalog locale/el_GR/LC_MESSAGES/messages.po to locale/el_GR/LC_MESSAGES/messages.mo
compiling catalog locale/de_DE/LC_MESSAGES/messages.po to locale/de_DE/LC_MESSAGES/messages.mo

Now, if you make changes to the base.pot file you can update the rest using the  update  command. For example, add the following line to base.pot

msgid "Hello world."
msgstr ""

Then run the tool to update the rest of the PO  files:

$ pybabel update -i locale/base.pot -d locale
updating catalog locale/en_US/LC_MESSAGES/messages.po based on locale/base.pot
updating catalog locale/el_GR/LC_MESSAGES/messages.po based on locale/base.pot
updating catalog locale/de_DE/LC_MESSAGES/messages.po based on locale/base.pot

Integration with Jinja2 templates

Babel can integrate with Jinja templates using the  jinja2.ext.i18n  extension.

It can then be used to mark and translate messages from templates and it is useful for internationalizing HTML pages.

In order to integrate both of those tools together, you need to provide some config.

First, install jinja using pip:

$ pip install Jinja2

You need to instruct babel to parse jinja templates when extracting the messages and for that, you need to add a configuration file.

Create a file named babel-mapping.ini  and add the following text:

[python: **.py]
[jinja2: **/templates/**.html]
extensions=jinja2.ext.i18n,jinja2.ext.autoescape,jinja2.ext.with_

So now when you invoke pybabel  commands referencing that file it will also extract messages from Jinja templates.

Le’t see how we can load Babel and Jinja templates together:

Create a template called index.html  that will be used to extract our messages:

<title>{% trans %}{{title}}{% endtrans %}</title>
<p>{% trans count=mailservers|length %}
There is {{ count }} {{ name }} server.
{% pluralize %}
There are {{ count }} {{ name }} servers.
{% endtrans %}
</p>

Invoke the following command to extract the base messages:

¢ pybabel extract -F babel-mapping.ini -o locale/messages.pot ./

That will generate the following catalog:

# Translations template for PROJECT.
# Copyright (C) 2018 ORGANIZATION
# This file is distributed under the same license as the PROJECT project.
# FIRST AUTHOR <EMAIL@ADDRESS>, 2018.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PROJECT VERSION\n"
"Report-Msgid-Bugs-To: EMAIL@ADDRESS\n"
"POT-Creation-Date: 2018-08-29 10:24+0100\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 2.6.0\n"

#: templates/index.html:1
#, python-format
msgid "%(title)s"
msgstr ""

#: templates/index.html:2
#, python-format
msgid ""
"\n"
" There is %(count)s %(name)s server.\n"
" "
msgid_plural ""
"\n"
" There are %(count)s %(name)s servers.\n"
" "
msgstr[0] ""
msgstr[1] ""

Now initialize the Italian translations using the init command and provide the translations:

$ pybabel init -d locale -l it -i locale/messages.pot
creating catalog locale/it/LC_MESSAGES/messages.po based on locale/messages.pot

Run the compile command to generate the MO files:

$ pybabel compile -d locale -l it
compiling catalog locale/it/LC_MESSAGES/messages.po to locale/it/LC_MESSAGES/messages.mo

Create a file named app.py  to hook everything together:

from jinja2 import Environment, FileSystemLoader, select_autoescape
from babel.support import Translations

templateLoader = FileSystemLoader( searchpath="templates" )

env = Environment(
  loader=templateLoader,
  extensions=['jinja2.ext.i18n', 'jinja2.ext.autoescape'],
  autoescape=select_autoescape(['html', 'xml'])
)

translations = Translations.load('locale', ['it'])
env.install_gettext_translations(translations)

template = env.get_template('index.html')

print(template.render(mailservers=range(10), name='mail'))

We are using the  Translations component to load the message catalogs that we compiled earlier. Then we load them to the Jinja environment using the install_gettext_translations method. Then we render the template.

If you run this program, you will see the following output:

$ python app.py
<title>Titolo</title>
<p>Esistono 10 mail servers.
</p>

If we want to change the locale, we need to do the same procedure. For example:

translations = Translations.load('locale', ['en_US'])
env.install_gettext_translations(translations)

template = env.get_template('index.html')

print(template.render())

Adding Phrase in-context editor with Flask and Babel

We can also introduce Flask into the picture and integrate Phrase in-context editor by using a technique to replace the gettext callables for the jinja environment.

First, we need to install the required packages:

$ pip install flask, flask-babel

Create a file named web.py and add the following code:

from flask import Flask, render_template, request
from flask_babel import Babel
from phrase import Phrase, gettext, ngettext

class Config(object):
    LANGUAGES = {
        'it': 'Italian',
        'de_DE': 'Deutsch'
    },
    BABEL_DEFAULT_LOCALE= 'it'
    PHRASEAPP_ENABLED = True
    PHRASEAPP_PREFIX = '{{__'
    PHRASEAPP_SUFFIX = '__}}'

app = Flask(__name__)
app.config.from_object(Config)

babel = Babel(app)
phrase = Phrase(app)

@babel.localeselector
def get_locale():
    return request.accept_languages.best_match(app.config['LANGUAGES'][0].keys())

@app.route('/')
def index():
    return render_template('index.html', locale=get_locale() or babel.default_locale)

We set up some configuration first to define the list of supported languages and the PHRASEAPP_* specific keys. What the in-context editor needs is to wrap the translatable strings with specific tags ‘{{__’  and ‘__}}’ .

Now let’s see the contents of the phrase.py  file

from __future__ import print_function
try:
    from flask_babel import gettext as gettext_original, ngettext as ngettext_original
    from flask import current_app
except ImportError:
    print("Flask-Babel is required.")

class Phrase(object):

    def __init__(self, app=None):
        self.app = app
        app.jinja_env.install_gettext_callables(
            gettext,
            ngettext,
            newstyle=True
        )

def phrase_enabled():
    return current_app.config['PHRASEAPP_ENABLED']

def phrase_key(msgid):
    return current_app.config['PHRASEAPP_PREFIX'] + 'phrase_' + msgid + current_app.config['PHRASEAPP_SUFFIX']

def gettext(msgid):
    if phrase_enabled():
        return phrase_key(msgid)
    else:
        return gettext_original(msgid)

def ngettext(msgid1, msgid2, n, **kwargs):
    if phrase_enabled():
        return phrase_key(msgid1)
    else:
        return ngettext_original(msgid1, msgid2, n, **kwargs)

We need to make sure we forward the original parameters to the original gettext functions in case we disabled the editor.

Update the index.html  file to include the script to load the editor:

<!DOCTYPE html>
<html lang="{{locale}}">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta http-equiv="X-UA-Compatible" content="ie=edge">
  <title>{% trans %}title{% endtrans %}</title>
</head>
<body>
    <p>{% trans %}title{% endtrans %}</p>
    
        window.PHRASEAPP_CONFIG = {
            projectId: "YOUR-PROJECT-ID"
        };
        (function() {
            var phraseapp = document.createElement('script'); phraseapp.type = 'text/javascript'; phraseapp.async = true;
            phraseapp.src = ['https://', 'phraseapp.com/assets/in-context-editor/2.0/app.js?', new Date().getTime()].join('');
            var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(phraseapp, s);
        })();
      
</body>
</html>

If you haven’t done that already, go to https://phrase.com/ and signup to try it for free.

Once you set up your account, you can create a project and navigate to Project Settings to find your projectId key.

Use that to assign the PHRASE_APP_TOKEN  environment variable before you start the server.

When you navigate to the page you will see a login modal and once you are authenticated you will see the translated strings change to include edit buttons next to them. The In-Context editor panel will show also.

$ export FLASK_APP=web.py
$ flask run

 

Conclusion

In this article, we have seen how to do python localization using the Python babel library. We’ve also seen how we can integrate it with Jinja templates and Phrase’s In-Context Editor in our workflow. If you have any other questions left, do not hesitate to post a comment or drop me a line. Thank you for reading and see you again next time!

 

4.6 (92%) 5 votes
Comments