Software localization
Babel’s I18n Advantages for Multilingual Apps

We have given detailed tutorials on gettext tools and integrating gettext with Python. We are going to extend our knowledge by using Babel and see some practical examples of its usage in python i18n. We are also going to see how to integrate it with Jinja2 templates and how to integrate Phrase's in-context editor into a Flask application to help with the translation process by simply browsing the website and editing texts along the way, making your Python localization process a lot simpler. You can also find the code described in this tutorial on GitHub.
About Babel
Babel provides internationalization (python i18n) and localization (python l10n) helpers and tools that work in two areas. The first is the gettext module that uses gettext to update, extract, and compile message catalogs and manipulate PO files. The second one is the usage of CLDR (Common Locale Data Repository) to provide formatting methods for currencies, dates, numbers, etc. based on a locale parameter. Both aspects aim to help automate the process of internationalizing Python applications as well as provide convenient methods for accessing and using this data.
Babel, in essence, works as an abstraction mechanism for a larger message extraction framework, as you can extend it with your own extractors and strategies that are not tied to a particular platform.
Installing and using Babel
Installing Babel is simple using pip
$ pip install Babel
If you don't have pip installed, you can get it with easy_install
$ sudo easy_install pip
Working with Locale Data
(CLDR) Unicode Common Locale Data Repository is a standardized repository of locale data used for formatting, parsing, and displaying locale-specific information. Instead of translating, for example, day names or month names for a particular language or script, you can make use of the translations provided by the locale data included with Babel based on CLDR data.
Let's see some examples...
Create a file name loc.py and add the following code:
from babel import Locale # Parsing l = Locale.parse('de-DE', sep='-') print("Locale name: {0}".format(l.display_name)) l = Locale.parse('und_GR', sep='_') print("Locale name: {0}".format(l.display_name)) # Detecting l = Locale.negotiate(['de_DE', 'en_AU'], ['de_DE', 'de_AT']) print("Locale negociated: {0}".format(l.display_name)) print(Locale('it').english_name) print(Locale('it').get_display_name('fr_FR')) print(Locale('it').get_language_name('de_DE')) print(Locale('de', 'DE').languages['zh']) print(Locale('el', 'GR').scripts['Copt']) # Calendar locale = Locale('it') month_names = locale.days['format']['wide'].items() print(list(month_names))
We are showing some examples of the Locale class. It is used to print, negotiate or identify language tags. If you run this example, you will see the following output:
$ python loc.py Locale name: Deutsch (Deutschland) Locale name: Ελληνικά (Ελλάδα) Locale negociated: Deutsch (Deutschland) Italian italien Italienisch Chinesisch Κοπτικό [(6, 'domenica'), (0, 'lunedì'), (1, 'martedì'), (2, 'mercoledì'), (3, 'giovedì'), (4, 'venerdì'), (5, 'sabato')]
These are useful as they are provided by the (CLDR) dataset and do not need translation.
Apart from that, there are several functions that format dates, times, currencies, units etc. Let's see some examples:
from babel.dates import format_date, format_datetime, format_time from babel.numbers import format_number, format_decimal, format_percent, parse_decimal from babel.units import format_unit from datetime import date, datetime, time # Date, time d = date(2010, 3, 10) print(format_date(d, format='short', locale='it')) print(format_date(d, format='full', locale='it')) print(format_date(d, "EEEE, d.M.yyyy", locale='de')) dt = datetime.now() print(format_datetime(dt, "yyyy.MMMM.dd GGG hh:mm a", locale='en')) # Numbers/ Units print(format_decimal(123.45123, locale='en_US')) print(format_decimal(123.45123, locale='de')) print(format_unit(12, 'length-meter', locale='en_GB')) print(format_unit(12, 'length-meter', locale='en_US')) parse_decimal('2.029,98', locale='de')
The output is:
$ python loc.py 10/03/10 mercoledì 10 marzo 2010 Mittwoch, 10.3.2010 2018.August.28 AD 10:24 AM 123.451 123,451 12 metres 12 meters 2029.98
Message Extraction
Babel has an extraction mechanism similar to gettext. It works by walking through the specified directories and based on the configuration rules it applies extractor functions to those files matched. This way there is more flexibility than gettext as you can leverage the expression power of Python to extend the tool.
Babel comes with a few built-in extractors such as python, javascript, and ignore (which extracts nothing) and you can create your own extractors. There are two different front-ends to access this functionality:
- A Command-Line Interface
- Distutils/Setuptools Integration
In this tutorial, we are going to use the Command-Line Interface.
To use it just invoke the pybabel tool for example to print all known locales
$ pybabel --list-locales:
To actually use the tooling, let's walk through the process of extracting messages using pybabel :
Create a file named main.py and add the following code:
import gettext _ = gettext.gettext print(_('This is a translatable string.'))
Note that the usage of gettext is only convenient because the default extractor uses gettext behind the scenes, but Babel, in general, is not tied to that.
Use the pybabel extract command to create the initial message catalog:
$ mkdir locale $ pybabel extract . -o locale/base.pot
That will create a base pot file that will contain the following messages:
#: main.py:4 msgid "This is a translatable string." msgstr ""
You don't have to edit this file now.
Using the init command, we can create a new translation catalog based on that POT template file:
$ pybabel init -l el_GR de_DE en_US -i locale/base.pot -d locale creating catalog locale/el_GR/LC_MESSAGES/messages.po based on locale/base.pot $ pybabel init -l de_DE en_US -i locale/base.pot -d locale creating catalog locale/de_DE/LC_MESSAGES/messages.po based on locale/base.pot $ pybabel init -l en_US -i locale/base.pot -d locale creating catalog locale/en_US/LC_MESSAGES/messages.po based on locale/base.pot
Those files are ready to be translated. When the translations are done, you can use the compile command to turn them into MO files:
$ pybabel compile -d locale compiling catalog locale/en_US/LC_MESSAGES/messages.po to locale/en_US/LC_MESSAGES/messages.mo compiling catalog locale/el_GR/LC_MESSAGES/messages.po to locale/el_GR/LC_MESSAGES/messages.mo compiling catalog locale/de_DE/LC_MESSAGES/messages.po to locale/de_DE/LC_MESSAGES/messages.mo
Now, if you make changes to the base.pot file you can update the rest using the update command. For example, add the following line to base.pot:
msgid "Hello world." msgstr ""
Then run the tool to update the rest of the PO files:
$ pybabel update -i locale/base.pot -d locale updating catalog locale/en_US/LC_MESSAGES/messages.po based on locale/base.pot updating catalog locale/el_GR/LC_MESSAGES/messages.po based on locale/base.pot updating catalog locale/de_DE/LC_MESSAGES/messages.po based on locale/base.pot
Integration with Jinja2 templates
Babel can integrate with Jinja templates using the jinja2.ext.i18n extension.
It can then be used to mark and translate messages from templates and it is useful for internationalizing HTML pages.
In order to integrate both of those tools together, you need to provide some config.
First, install jinja using pip:
$ pip install Jinja2
You need to instruct babel to parse jinja templates when extracting the messages and for that, you need to add a configuration file.
Create a file named babel-mapping.ini and add the following text:
[python: **.py] [jinja2: **/templates/**.html] extensions=jinja2.ext.i18n,jinja2.ext.autoescape,jinja2.ext.with_
So now when you invoke pybabel commands referencing that file, it will also extract messages from Jinja templates.
Let's see how we can load Babel and Jinja templates together:
Create a template called index.html that will be used to extract our messages:
<title>{% trans %}{{title}}{% endtrans %}</title> <p>{% trans count=mailservers|length %} There is {{ count }} {{ name }} server. {% pluralize %} There are {{ count }} {{ name }} servers. {% endtrans %} </p>
Invoke the following command to extract the base messages:
¢ pybabel extract -F babel-mapping.ini -o locale/messages.pot ./
That will generate the following catalog:
# Translations template for PROJECT. # Copyright (C) 2018 ORGANIZATION # This file is distributed under the same license as the PROJECT project. # FIRST AUTHOR <EMAIL@ADDRESS>, 2018. # #, fuzzy msgid "" msgstr "" "Project-Id-Version: PROJECT VERSION\n" "Report-Msgid-Bugs-To: EMAIL@ADDRESS\n" "POT-Creation-Date: 2018-08-29 10:24+0100\n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" "Language-Team: LANGUAGE <LL@li.org>\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=utf-8\n" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.6.0\n" #: templates/index.html:1 #, python-format msgid "%(title)s" msgstr "" #: templates/index.html:2 #, python-format msgid "" "\n" " There is %(count)s %(name)s server.\n" " " msgid_plural "" "\n" " There are %(count)s %(name)s servers.\n" " " msgstr[0] "" msgstr[1] ""
Now initialize the Italian translations using the init command and provide the translations:
$ pybabel init -d locale -l it -i locale/messages.pot creating catalog locale/it/LC_MESSAGES/messages.po based on locale/messages.pot
Run the compile command to generate the MO files:
$ pybabel compile -d locale -l it compiling catalog locale/it/LC_MESSAGES/messages.po to locale/it/LC_MESSAGES/messages.mo
Create a file named app.py to hook everything together:
from jinja2 import Environment, FileSystemLoader, select_autoescape from babel.support import Translations templateLoader = FileSystemLoader( searchpath="templates" ) env = Environment( loader=templateLoader, extensions=['jinja2.ext.i18n', 'jinja2.ext.autoescape'], autoescape=select_autoescape(['html', 'xml']) ) translations = Translations.load('locale', ['it']) env.install_gettext_translations(translations) template = env.get_template('index.html') print(template.render(mailservers=range(10), name='mail'))
We are using the Translations component to load the message catalog, which we compiled earlier. Then we load them to the Jinja environment using the install_gettext_translations method. Then we render the template.
If you run this program, you will see the following output:
$ python app.py <title>Titolo</title> <p>Esistono 10 mail servers. </p>
If we want to change the locale, we need to do the same procedure. For example:
translations = Translations.load('locale', ['en_US']) env.install_gettext_translations(translations) template = env.get_template('index.html') print(template.render())
Adding Phrase in-context editor with Flask and Babel
We can also introduce Flask into the picture and integrate Phrase in-context editor by using a technique to replace the gettext callables for the jinja environment.
First, we need to install the required packages:
$ pip install flask, flask-babel
Create a file named web.py and add the following code:
from flask import Flask, render_template, request from flask_babel import Babel from phrase import Phrase, gettext, ngettext class Config(object): LANGUAGES = { 'it': 'Italian', 'de_DE': 'Deutsch' }, BABEL_DEFAULT_LOCALE= 'it' PHRASEAPP_ENABLED = True PHRASEAPP_PREFIX = '{{__' PHRASEAPP_SUFFIX = '__}}' app = Flask(__name__) app.config.from_object(Config) babel = Babel(app) phrase = Phrase(app) @babel.localeselector def get_locale(): return request.accept_languages.best_match(app.config['LANGUAGES'][0].keys()) @app.route('/') def index(): return render_template('index.html', locale=get_locale() or babel.default_locale)
We set up some configuration first to define the list of supported languages and the PHRASEAPP_* specific keys. What the in-context editor needs is wrapping the translatable strings with specific tags '{{__' and '__}}'.
Now let's see the contents of the phrase.py file
from __future__ import print_function try: from flask_babel import gettext as gettext_original, ngettext as ngettext_original from flask import current_app except ImportError: print("Flask-Babel is required.") class Phrase(object): def __init__(self, app=None): self.app = app app.jinja_env.install_gettext_callables( gettext, ngettext, newstyle=True ) def phrase_enabled(): return current_app.config['PHRASEAPP_ENABLED'] def phrase_key(msgid): return current_app.config['PHRASEAPP_PREFIX'] + 'phrase_' + msgid + current_app.config['PHRASEAPP_SUFFIX'] def gettext(msgid): if phrase_enabled(): return phrase_key(msgid) else: return gettext_original(msgid) def ngettext(msgid1, msgid2, n, **kwargs): if phrase_enabled(): return phrase_key(msgid1) else: return ngettext_original(msgid1, msgid2, n, **kwargs)
We need to make sure we forward the original parameters to the original gettext functions in case we disabled the editor.
Update the index.html file to include the script to load the editor:
<!DOCTYPE html> <html lang="{{locale}}"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta http-equiv="X-UA-Compatible" content="ie=edge"> <title>{% trans %}title{% endtrans %}</title> </head> <body> <p>{% trans %}title{% endtrans %}</p> window.PHRASEAPP_CONFIG = { projectId: "YOUR-PROJECT-ID" }; (function() { var phraseapp = document.createElement('script'); phraseapp.type = 'text/javascript'; phraseapp.async = true; phraseapp.src = ['https://', 'phraseapp.com/assets/in-context-editor/2.0/app.js?', new Date().getTime()].join(''); var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(phraseapp, s); })(); </body> </html>
If you haven't done that already, go to phrase.com and signup to try it for free.
Once you set up your account, you can create a project and navigate to Project Settings to find your projectId key.
Use that to assign the PHRASE_APP_TOKEN environment variable before you start the server.
$ export FLASK_APP=web.py $ flask run
Conclusion
In this article, we have seen how to do python localization using the Python babel library. We’ve also seen how we can integrate it with Jinja templates and Phrase's in-context editor in our workflow. If you have any other questions left, do not hesitate to get in touch. Thank you for reading and see you again next time!