Friday, January 20, 2017

Fabric how to set environment variable to fix encoding

Hi there,

Today I've encountered with one weird problem and decided to share it with you.

Here is what happens. When I ssh to server manually and run python script - everything works fine. But if I try to run the same script using fabric script, which connects to the same server, then it fails. In particular, it was encoding error:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xaa' in position bb: ordinal not in range(128)

Then I dug a bit into the problem and decided to print system encoding from python script:
import sys
print sys.getfilesystemencoding()
When I ran it manually from the shell the output was "en_US.utf8", but when I ran the same using fabric it was: "ANSI_X3.4-1968".

OK gotcha. In addition to that locale environment variables were not set when running python script with fabric. So in the end I decided to set LC_CTYPE environment variable in fabric, which in turn fixed the problem.

To set LC_CTYPE or any other environment variable in fabric, whether you're using new-style tasks or basic tasks, you can use this code:
from fabric.api import env
env.command_prefixes=["export LC_CTYPE=en_US.UTF-8",]

This worked well for me. But if you know a better approach how to set environment variable globally in fabric, please do let me know.

No comments:

Post a Comment