Problem with postgresql encoding installing Airtime
  • Hi,
    we're helping a collegue to move to Airtime from a Windows software.
    The enviroment is similar to the one we've been using sin January:

    • Debian 6.0 Squeeze running on a VPS;
    • Same datacenter;
    • Same installation procedure (Automated installation);
    Everything looks to go but at the end of the process we got this message never seen before:

    "* Checking database for correct encoding
     * Unfortunately your postgresql database has not been created using a UTF-8 encoding.
     * As of Airtime 2.1, installs will fail unless the encoding has been set to UTF-8. Please verify this is the case
     * and try the install again".

    No way to overcome this issue, we tried several time. Any idea? Thanks

  • 25 Comments sorted by
  • Does he have anything else stored in his postgresql database? You can check by doing

    sudo -u postgres psql -l

    If you only see "postgres, template0, and template1", then you should be safe to completely remove the database and install it again:

    sudo apt-get purge postgresql
    sudo apt-get install postgresql

    This should solve the problem. If he has other software that is using postgresql, he will need to upgrade his database to UTF-8 which is a little more complex:


    A UTF-8 database is required since it allows storage of files with special characters in their filenames.
    Airtime Pro Hosting: http://airtime.pro
  • Hi Martin,

    It's a fresh installation on a new Debian 6 VPS. No other applications installed just like in our enviroment.
    I did what you suggested but the outcome is the same.

    I also tried the installation of postgresql as Airtime dependency (I first udated the sources list, then installed rabbitmq and Icecast and finally typed apt-get install airtime). but no way. It's the first time I face an issue like this installing Airtime.

    Thank you
  • Ok please give me the output of "locale".
    Airtime Pro Hosting: http://airtime.pro
  • Here you are:

    root@www:~# locale
    LANG=en_US.UTF-8
    LC_CTYPE="en_US.UTF-8"
    LC_NUMERIC="en_US.UTF-8"
    LC_TIME="en_US.UTF-8"
    LC_COLLATE="en_US.UTF-8"
    LC_MONETARY="en_US.UTF-8"
    LC_MESSAGES="en_US.UTF-8"
    LC_PAPER="en_US.UTF-8"
    LC_NAME="en_US.UTF-8"
    LC_ADDRESS="en_US.UTF-8"
    LC_TELEPHONE="en_US.UTF-8"
    LC_MEASUREMENT="en_US.UTF-8"
    LC_IDENTIFICATION="en_US.UTF-8"
    LC_ALL=

  • Strange. Please also provide output of

    sudo -u postgres psql -c "SHOW SERVER_ENCODING"

    Airtime Pro Hosting: http://airtime.pro
  • Hi Martin,

    I cannot say how but the issue is fixed. I did a fresh installation after reloaded the OS on the VPS and this specific issue disappeared. Probably it was related to the VPS.

    Now I'm facing another problem but I'll write in a new discussion. Thanks
  • what's wrong ?
    apt-get install airtime
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    The following extra packages will be installed:
      odbc-postgresql
    Suggested packages:
      airtime-audio-samples alsa-utils unixodbc-bin
    The following NEW packages will be installed:
      airtime odbc-postgresql
    0 upgraded, 2 newly installed, 0 to remove and 1 not upgraded.
    Need to get 0 B/9,149 kB of archives.
    After this operation, 38.1 MB of additional disk space will be used.
    Do you want to continue [Y/n]? y
    Preconfiguring packages ...
    Selecting previously deselected package odbc-postgresql.
    (Reading database ... 47632 files and directories currently installed.)
    Unpacking odbc-postgresql (from .../odbc-postgresql_1%3a08.03.0200-1.2_amd64.deb) ...
    Selecting previously deselected package airtime.
    Unpacking airtime (from .../airtime_2.1.3-1_all.deb) ...
    Setting up odbc-postgresql (1:08.03.0200-1.2) ...
    Setting up airtime (2.1.3-1) ...
    Site airtime-vhost already disabled
    Setting up apache2...
    Enabling site airtime-vhost.
    Run '/etc/init.d/apache2 reload' to activate new configuration!
    Module rewrite already enabled
    Configuring php5...
    Setting up monit configuration...
    Stopping daemon monitor: monit.
    Starting daemon monitor: monit.
    Warning: rabbitmq-server is not configured to start after a reboot!
    Fix Default-Start and Default-Stop lines in /etc/init.d/rabbitmq-server
    then run this command as root: update-rc.d rabbitmq-server defaults
    Restarting web server: apache2 ... waiting .
    Creating symlink for Liquidsoap...

    Debian package of Airtime detected...

    * Making sure /etc/default/locale is set properly
    LANG=en_US.UTF-8
     * None found.
    * Temporarily stopping any previous running services

    ******************************** Install Begin *********************************
    Ensuring python-virtualenv version > 1.4.8...Success!

    *** Creating Virtualenv for Airtime ***
    Already using interpreter /usr/bin/python
    New python executable in /usr/lib/airtime/airtime_virtualenv/bin/python
    Installing distribute.................................................................................................................................................................................done.

    *** Installing Python Libraries ***
    Unpacking /var/lib/airtime/tmp/python_apps/python-virtualenv/airtime_virtual_env.pybundle
    Downloading/unpacking pyinotify
      Running setup.py egg_info for package pyinotify
    Downloading/unpacking pytz
      Running setup.py egg_info for package pytz
        warning: no files found matching '*.pot' under directory 'pytz'
        warning: no previously-included files found matching 'test_zdump.py'
    Downloading/unpacking mutagen
      Running setup.py egg_info for package mutagen
    Downloading/unpacking anyjson
      Running setup.py egg_info for package anyjson
    Downloading/unpacking configobj
      Running setup.py egg_info for package configobj
    Downloading/unpacking amqplib
      Running setup.py egg_info for package amqplib
    Downloading/unpacking kombu
      Running setup.py egg_info for package kombu
    Downloading/unpacking poster
      Running setup.py egg_info for package poster
    Installing collected packages: amqplib, anyjson, configobj, kombu, mutagen, poster, pyinotify, pytz
      Running setup.py install for amqplib
      Running setup.py install for anyjson
      Running setup.py install for configobj
      Running setup.py install for kombu
      Running setup.py install for mutagen
        changing mode of build/scripts-2.6/mutagen-inspect from 644 to 755
        changing mode of build/scripts-2.6/mutagen-pony from 644 to 755
        changing mode of build/scripts-2.6/moggsplit from 644 to 755
        changing mode of build/scripts-2.6/mid3iconv from 644 to 755
        changing mode of build/scripts-2.6/mid3v2 from 644 to 755
        changing mode of /usr/lib/airtime/airtime_virtualenv/bin/mutagen-inspect to 755
        changing mode of /usr/lib/airtime/airtime_virtualenv/bin/mutagen-pony to 755
        changing mode of /usr/lib/airtime/airtime_virtualenv/bin/moggsplit to 755
        changing mode of /usr/lib/airtime/airtime_virtualenv/bin/mid3iconv to 755
        changing mode of /usr/lib/airtime/airtime_virtualenv/bin/mid3v2 to 755
      Running setup.py install for poster
      Running setup.py install for pyinotify
      Running setup.py install for pytz
        warning: no files found matching '*.pot' under directory 'pytz'
        warning: no previously-included files found matching 'test_zdump.py'
    Successfully installed amqplib anyjson configobj kombu mutagen poster pyinotify pytz
    Cleaning up...

    *** Patching Python Libraries ***
     * Patching virtualenv libraries in /usr/lib/airtime/airtime_virtualenv/lib/python2.6

       ---Applying Patches for mutagen---
    patching file site-packages/mutagen/mp3.py
    Hunk #1 succeeded at 205 (offset -4 lines).
    patching file site-packages/mutagen/mp3.py
    Hunk #2 succeeded at 169 (offset -4 lines).
    * Checking for user pypo
     * Creating user pypo
    * Creating INI files
    * Initializing INI files
    * Airtime Version: 2.1.3
    * Storage directory setup
    * Skipping directory already exists: /srv/airtime/stor
    * Giving Apache permission to access /srv/airtime/stor
    * Skipping directory already exists: /srv/airtime/stor/organize
    * Giving Apache permission to access /srv/airtime/stor/organize
    * Checking database for correct encoding

    * Database Installation
     * Creating Airtime database user
      * Database user 'airtime' created.
     * Creating Airtime database
      * Database 'airtime' created.
     * Installing Postgres scripting language
     * Creating database tables
     * Setting Airtime version
     * Inserting stor directory location /srv/airtime/stor/ into music_dirs table
    * Creating /etc/airtime
    * Creating /etc/monit/conf.d/monit-airtime-generic.cfg
    * Creating /etc/cron.d/airtime-crons
    * Creating /usr/lib/airtime
    * Creating symbolic links in /usr/bin
    * Creating /var/log/airtime
    * Creating /usr/share/airtime
    * Creating /var/log/airtime
    * Creating /var/tmp/airtime
    * Waiting for media-monitor processes to start...
    Starting Airtime Media Monitor: Done.
    * Detecting OS: ...  Found Debian GNU/Linux 6.0.5 (squeeze) (squeeze) on amd64 architecture
    * Clearing previous pypo cache
    * Waiting for pypo processes to start...
    Stopping daemon monitor: monit.
    Starting daemon monitor: monit.

    *** Verifying your system environment, running airtime-check-system ***
    AIRTIME_SERVER_RESPONDING      = OK
    KERNEL_VERSION                 = UNKNOWN
    MACHINE_ARCHITECTURE           = UNKNOWN
    TOTAL_MEMORY_MBYTES            = UNKNOWN
    TOTAL_SWAP_MBYTES              = UNKNOWN
    AIRTIME_VERSION                = UNKNOWN
    OS                             = Debian GNU/Linux 6.0.5 (squeeze) x86_64
    CPU                            = Intel(R) Core(TM) i5-2320 CPU @ 3.00GHz
    WEB_SERVER                     = Apache
    PLAYOUT_ENGINE_PROCESS_ID      = FAILED
    PLAYOUT_ENGINE_RUNNING_SECONDS = 0
    PLAYOUT_ENGINE_MEM_PERC        = 0%
    PLAYOUT_ENGINE_CPU_PERC        = 0%
    LIQUIDSOAP_PROCESS_ID          = FAILED
    LIQUIDSOAP_RUNNING_SECONDS     = 0
    LIQUIDSOAP_MEM_PERC            = 0%
    LIQUIDSOAP_CPU_PERC            = 0%
    MEDIA_MONITOR_PROCESS_ID       = FAILED
    MEDIA_MONITOR_RUNNING_SECONDS  = 0
    MEDIA_MONITOR_MEM_PERC         = 0%
    MEDIA_MONITOR_CPU_PERC         = 0%
    RABBITMQ_PROCESS_ID            = FAILED
    RABBITMQ_RUNNING_SECONDS       = 0
    RABBITMQ_MEM_PERC              = 0%
    RABBITMQ_CPU_PERC              = 0%

    -- There appears to be a problem with your Airtime installation.

    dpkg: error processing airtime (--configure):
     subprocess installed post-installation script returned error exit status 1
    configured to not write apport reports
                                          Errors were encountered while processing:
     airtime
    E: Sub-process /usr/bin/dpkg returned an error code (1)

  • What VPN is this?

    Please post the output of /var/log/airtime/media-monitor/media-monitor.log
    and
    /var/log/airtime/pypo/pypo.log
    Airtime Pro Hosting: http://airtime.pro
  • My fix was to just blow away the default cluster and re-create it with a UTF8 encoding. It should go without saying that you shouldn’t do this on a server with data you’d actually like to keep, but on a new server this works fine:


    pg_dropcluster --stop 8.4 main
    pg_createcluster --start -e UTF-8 8.4 main

    If someone knows of an easier way — one that doesn’t require destroying data — I’d love to know about it.

  • As far as I'm aware there are no simple ways to do this. You could try dumping the database to a file, while forcing a certain character encoding and then restore from that file, but this seems real hacky. 

    Almost all OS'es these days are created with UTF-8 as the default locale meaning postgresql by default uses UTF-8. I've seen a few users on virtual hosts have this problem, so it seems that those providers are setting ASCII locale as default for some reason.
    Airtime Pro Hosting: http://airtime.pro
  • I had to remake the cluster as above too in 2.5 - but substituting 9.1 for 8.4

    It seems this could be fixed by the install script forcing UTF-8 at database creation time - or so the PostgreSQL Docs infer.

    Post edited by John Chewter at 2013-10-25 16:16:04
    Resurrected: 16 May 2017.
  • Vote Up0Vote Down hoerichhoerich
    Posts: 627Member, Airtime Moderator

    I had to remake the cluster as above too in 2.5 - but substituting 9.1 for 8.4

    It seems this could be fixed by the install script forcing UTF-8 at database creation time - or so the PostgreSQL Docs infer.



    worked for me too
    Official Airtime Forum Manager
    --------------------------
    Most of the time an issue is located between keyboard and chair.
  • Yes. I got flamed for saying it also.
    Post edited by John Chewter at 2013-11-24 07:51:50
    Resurrected: 16 May 2017.
  • Vote Up0Vote Down Albert SantoniAlbert Santoni
    Posts: 68Member, Sourcefabric Team, Airtime Moderator
    Does someone feel like volunteering a patch for this?

    Thanks,
    Albert
    Airtime Developer @ Sourcefabric
  • Method A

    Here is a quick workaround that worked for me using my server control panel tools.
    I have a better fix - see 'Method B' below - but that needs the install script modifying.
    Install PostgreSQL, if not installed.
    Delete the Airtime database - if already created.
    Create a new database called airtime with a username and password of airtime.
    It asked me what collation I wanted. I chose UTF8.
    IMPORTANT. It asked what Template to use- use template0
    A blank database was created.

    Run the Airtime Installer again. It will use the database already created.
    Worked for me.


    Post edited by John Chewter at 2013-11-29 11:01:30
    Resurrected: 16 May 2017.
  • Method B

    Modifying the installer.

    Copy /var/lib/airtime/tmp/install_minimum/include/airtime_install.php to a safe place as a backup.

    In /var/lib/airtime/tmp/install_minimum/include/airtime_install.php

    Around line 228 find:

    $command = "su postgres -c \"createdb $database --encoding UTF8 --owner $username\"";

    Replace with:

    $command = "sudo -i -u postgres psql postgres -c \"CREATE DATABASE ".$database." WITH ENCODING 'UTF8' TEMPLATE template0 OWNER ".$username."\"";

    Re-run the airtime install.

    This fix will be in the next Airtime release.
    Post edited by John Chewter at 2013-11-29 10:54:08
    Resurrected: 16 May 2017.
  • Vote Up0Vote Down Albert SantoniAlbert Santoni
    Posts: 68Member, Sourcefabric Team, Airtime Moderator
    Thanks for the patch John, I've committed it in our 2.5.x branch so it'll be in our next bugfix release.

    To be fair, the original code looks correct on first glance because we're explicitly passing "--encoding UTF8" to postgres. It's only once you dive into the postgres docs that you find out there are circumstances where that flag is ignored and you have to deal with this template0 nonsense. So I don't think this bug is gross negligence on anyone's part...

    Thanks again,
    Albert
    Airtime Developer @ Sourcefabric
  • Yes Albert - it is not obvious that the locale is changed by the SSH client.
    Resurrected: 16 May 2017.
  • Vote Up0Vote Down hoerichhoerich
    Posts: 627Member, Airtime Moderator
    awesome job
    Official Airtime Forum Manager
    --------------------------
    Most of the time an issue is located between keyboard and chair.
  • sudo -u postgres pg_dumpall > /tmp/postgres.sql
    sudo pg_dropcluster --stop 9.1 main
    sudo pg_createcluster --locale en_US.UTF-8 --start 9.1 main
    sudo -u postgres psql -f /tmp/postgres.sql

  • * Checking for user pypo
    User already exists.
    * Creating INI files
    * Initializing INI files
    * Airtime Version: 2.5.1
    * Storage directory setup
    * Skipping directory already exists: /srv/airtime/stor
    * Giving Apache permission to access /srv/airtime/stor
    * Skipping directory already exists: /srv/airtime/stor/organize
    * Giving Apache permission to access /srv/airtime/stor/organize
    * Checking database for correct encoding
     * Unfortunately your postgresql database has not been created using a UTF-8 encoding.
     * As of Airtime 2.1, installs will fail unless the encoding has been set to UTF-8. Please verify this is the case
     * and try the install again
    There was an error during install. Exit code 1
    dpkg: error processing package airtime (--configure):
     subprocess installed post-installation script returned error exit status 1
    Errors were encountered while processing:
     airtime
    E: Sub-process /usr/bin/dpkg returned an error code (1)
    root@s1:~#

  • that is my problem.. help me
  • It tells you what is wrong.
    * Unfortunately your postgresql database has not been created using a UTF-8 encoding.
     * As of Airtime 2.1, installs will fail unless the encoding has been set to UTF-8. Please verify this is the case
     * and try the install again

    Resurrected: 16 May 2017.
  • Most Airtime issues have a fix in the forum. Use search.
    Resurrected: 16 May 2017.
  • Trung tâm giống cây trồng Công Nghệ Cao Việt Nam là đơn vị chuyên cung cấp các loại giống cây trồng mới, cây giống ăn quả, hạt giống năng suất cao, hoa cây cảnh, vật tư nông nghiệp... Tư vấn và chuyển giao nông nghiệp.