Liquid Soap problem. Streaming doesn't work
  • Vote Up0Vote Down Marco BRUNOMarco BRUNO
    Posts: 12Member
    Unexpectedly airtime no longer able to connect to the streaming servers.
    It has happened several times already and when reinstalling "airtime" everything is back to work.
    Since this is a big problem give me a hand to do debugging?


    Airtime is running on a vps gandi.net along with other vhost for a drupal cms and drupal mobile cms.
    I have three streams.
    the first on localhost and is normally disabled because I use a relay server.


    The only combinations of events (but not repeatable) was a large size file uploadin through "add media" (100mb)

    airtime-check-system says...
    root@########:~# airtime-check-system
    AIRTIME_SERVER_RESPONDING      = OK
    KERNEL_VERSION                 = 3.2.39-xenU-7726-x86_64
    MACHINE_ARCHITECTURE           = x86_64
    TOTAL_MEMORY_MBYTES            = 741640
    TOTAL_SWAP_MBYTES              = 524284
    AIRTIME_VERSION                = 2.3.1
    OS                             = Ubuntu 12.04.2 LTS x86_64
    CPU                            = Intel(R) Xeon(R) CPU           L5640  @ 2.27GHz
    WEB_SERVER                     = Apache/2.2.22 (Ubuntu)
    PLAYOUT_ENGINE_PROCESS_ID      = 1403
    PLAYOUT_ENGINE_RUNNING_SECONDS = 1622
    PLAYOUT_ENGINE_MEM_PERC        = 2.0%
    PLAYOUT_ENGINE_CPU_PERC        = 0.0%
    LIQUIDSOAP_PROCESS_ID          = NOT MONITORED
    LIQUIDSOAP_RUNNING_SECONDS     = 1
    LIQUIDSOAP_MEM_PERC            = 0%
    LIQUIDSOAP_CPU_PERC            = 0%
    MEDIA_MONITOR_PROCESS_ID       = 1647
    MEDIA_MONITOR_RUNNING_SECONDS  = 1619
    MEDIA_MONITOR_MEM_PERC         = 2.2%
    MEDIA_MONITOR_CPU_PERC         = 0.0%
    -- There appears to be a problem with your Airtime installation.

    \\\\\\\\\\\\\\\\\\\\\

    the suggested wiki wasn't help me


    [message in Italian language]
    Inaspettatamente airtime non riesce più a connettersi ai server di streaming.
    Mi è già successo altre volte e reinstallando airtime tutto ritorna a funzionare.
    Essendo questo un grosso problema mi date una mano a fare debug?


    Airtime gira su un vps gandi.net insieme ad altri vhost per un cms drupal e un cms drupal mobile.
    Ho tre flussi di stream.
    il primo su localhost e normalmente è disabilitato perchè utilizzo un server relay.

    L'unica combinazioni di eventi (ma non ripetibile) è stata un caricamente di file di grosse dimensione tramite "add media"

    Streamproblem.png
    863 x 315 - 15K
  • 14 Comments sorted by
  • Vote Up0Vote Down Marco BRUNOMarco BRUNO
    Posts: 12Member
    With these commands returns to work everything.
    I do not think that manual intervention is the solution. :-)
    Also, when restart does not work.
    give me a hand to understand why crashes randomly?

    sudo -u pypo /usr/lib/airtime/pypo/bin/airtime-liquidsoap &
    sudo service rabbitmq-server restart
    sudo service airtime-playout restart
    sudo service airtime-media-monitor restart 

    [Italian message]
    Con questi comandi ritorna a funzionare tutto.
    Io non penso che un intervento manuale sia la soluzione.:-)
    Inoltre quando riavvia non funziona.
    mi date una mano a capire perchè si blocca casualmente?


  • Vote Up0Vote Down Martin KonecnyMartin Konecny
    Posts: 1,860Member
    What happens when you type

    "sudo service airtime-liquidsoap restart"

    Also, take a look into /etc/airtime/liquidsoap.cfg to ensure expected
    shoutcast/icecast server parameters are in there.


    On Mon, May 6, 2013 at 6:50 AM, Marco BRUNO <<br />airtime-support@lists.sourcefabric.org> wrote:

    > Unexpectedly airtime no longer able to connect to the streaming servers.
    > It has happened several times already and when reinstalling "airtime"
    > everything is back to work.
    > Since this is a big problem give me a hand to do debugging?
    >
    >
    > Airtime is running on a vps gandi.net along with other vhost for a drupal
    > cms and drupal mobile cms.
    > I have three streams.
    > the first on localhost and is normally disabled because I use a relay
    > server.
    >
    >
    > The only combinations of events (but not repeatable) was a large size file
    > uploadin through "add media" (100mb)
    >
    > airtime-check-system says...
    > root@########:~# airtime-check-system
    > AIRTIME_STATUS_URL = http://
    > #####.#############.###:80/api/status/format/json/api_key/%%api_key%%
    > AIRTIME_SERVER_RESPONDING = OK
    > KERNEL_VERSION = 3.2.39-xenU-7726-x86_64
    > MACHINE_ARCHITECTURE = x86_64
    > TOTAL_MEMORY_MBYTES = 741640
    > TOTAL_SWAP_MBYTES = 524284
    > AIRTIME_VERSION = 2.3.1
    > OS = Ubuntu 12.04.2 LTS x86_64
    > CPU = Intel(R) Xeon(R) CPU L5640 @
    > 2.27GHz
    > WEB_SERVER = Apache/2.2.22 (Ubuntu)
    > PLAYOUT_ENGINE_PROCESS_ID = 1403
    > PLAYOUT_ENGINE_RUNNING_SECONDS = 1622
    > PLAYOUT_ENGINE_MEM_PERC = 2.0%
    > PLAYOUT_ENGINE_CPU_PERC = 0.0%
    > LIQUIDSOAP_PROCESS_ID = NOT MONITORED
    > LIQUIDSOAP_RUNNING_SECONDS = 1
    > LIQUIDSOAP_MEM_PERC = 0%
    > LIQUIDSOAP_CPU_PERC = 0%
    > MEDIA_MONITOR_PROCESS_ID = 1647
    > MEDIA_MONITOR_RUNNING_SECONDS = 1619
    > MEDIA_MONITOR_MEM_PERC = 2.2%
    > MEDIA_MONITOR_CPU_PERC = 0.0%
    > -- There appears to be a problem with your Airtime installation.
    > -- Please visit http://wiki.sourcefabric.org/x/HABQ
    >
    > \\\\\\\\\\\\\\\\\\\\\
    >
    > the suggested wiki wasn't help me
    >
    >
    > [message in Italian language]
    > Inaspettatamente airtime non riesce più a connettersi ai server di
    > streaming.
    > Mi è già successo altre volte e reinstallando airtime tutto ritorna a
    > funzionare.
    > Essendo questo un grosso problema mi date una mano a fare debug?
    >
    >
    > Airtime gira su un vps gandi.net insieme ad altri vhost per un cms drupal
    > e un cms drupal mobile.
    > Ho tre flussi di stream.
    > il primo su localhost e normalmente è disabilitato perchè utilizzo un
    > server relay.
    >
    > L'unica combinazioni di eventi (ma non ripetibile) è stata un caricamente
    > di file di grosse dimensione tramite "add media"
    >
    >
    > - Attachment: Streamproblem.png(Size: 15.35Kb)
    >
    >
    Airtime Pro Hosting: http://airtime.pro
  • Vote Up0Vote Down Marco BRUNOMarco BRUNO
    Posts: 12Member
    This is the output
    Restarting Liquidsoap: start-stop-daemon: warning: failed to kill 2957: No such process

    #########################################
    #           *** pypo  ***               #
    #     pypo notification gateway         #
    #########################################
    Done.

    The /etc/airtime/liquidsoap.cfg file is correct. In the past cases this file disappeared. the only way to have  consistency file and  right access was reinstall airtime or use the previous 4 commands. 
    I'm not sure that all commands and the order is important... 'cause the crash is random.
    I think the file is regenerated with data from postgres
  • Vote Up0Vote Down Marco BRUNOMarco BRUNO
    Posts: 12Member
    heeeeelp!!!
     i do 
    sudo service airtime-liquidsoap restart
    and all it's ok and airtime front end works.
    after that i want try a system reboot and the server doesn't boot.
    I reinstall vps whit only airtime and drupal many time and now crash again!
    Disaster....
    Post edited by Marco BRUNO at 2013-05-06 17:57:56
  • Vote Up0Vote Down Martin KonecnyMartin Konecny
    Posts: 1,860Member
    At what times did the playout crash? Please give us these times and post

    /var/log/airtime/pypo/pypo.log
    /var/log/airtime/pypo-liquidsoap/ls_script.log
    Airtime Pro Hosting: http://airtime.pro
  • Vote Up0Vote Down Marco BRUNOMarco BRUNO
    Posts: 12Member
    at 23:39 I did restart the server to make sure that Liquidsoap start correctly.
    For some reason the vps did not reboot.
    at 23:58 I decided to restart the vps.
    attached the two log files.
    I removed some sensitive information such as passwords and addresses streaming which are reported in the logs.



    ls_script.log.txt
    72K
    pypo.log.txt
    1M
  • Vote Up0Vote Down Roger WilcoRoger Wilco
    Posts: 784Banned
    Friend!! You're asking WAY too much putting a full-blown CMS and Airtime on the very same VPS. Scale back your expectations. You could be maxing out your share of the CPU. The server could also be intermittently overbooked, too.
  • Vote Up0Vote Down Marco BRUNOMarco BRUNO
    Posts: 12Member
    the use of the processor when it is active only airtime (Liquidsoap) is 15%.
    The crash, random and are not related to the use of the cms than however we intend to migrate on a separate machine.
    With the cms I noticed peaks not more than 28%
  • Vote Up0Vote Down Roger WilcoRoger Wilco
    Posts: 784Banned
    How many cores do you have available? I've noticed anything less than 2 is bad bad news. 4 is much better, especially if you're running 3 streams. That 15% seems WAY off in my opinion and shouldn't be trusted.
  • Vote Up0Vote Down john drakejohn drake
    Posts: 2Member
    hope someone knows the answer, I have the same situation. It was connected to the icecast server then went form being connected to "Getting information from the server...". Tried the suggestions in this thread and nothing worked. In fact it seems that the liquidsoad,cfg no longer exists.
  • Vote Up0Vote Down Martin KonecnyMartin Konecny
    Posts: 1,860Member
    From the log files:

    2013/05/07 00:43:06 [decoder:3] Method "MAD" accepted "/var/tmp/airtime/pypo/cache/scheduler/112.mp3".
    2013/05/07 00:43:06 [cue_cut_5386:3] Cueing out...
    2013/05/07 00:43:06 [queue:3] Finished with "/var/tmp/airtime/pypo/cache/scheduler/242.mp3".

    All your files are finishing after 1 second.

    What is the output of

    sudo -u postgres psql -c "select count(*) from cc_files where cuein is null" airtime
    Airtime Pro Hosting: http://airtime.pro
  • Vote Up0Vote Down Marco BRUNOMarco BRUNO
    Posts: 12Member
    I had to reinstall the server because for some reason there were no more free inodes on the disk.
    This is very strange and I do not understand which component of airtime can consume all these inodes ...
    after reinstalling, and restore the airtime database the result of the query is as follows

     count
    -------
         0
    (1 row)

    I am very concerned that this might happen again. 3 is the reinstallation I have to do for problems that do not understand.

  • Vote Up0Vote Down Marco BRUNOMarco BRUNO
    Posts: 12Member
    Any ideas? 
  • Vote Up0Vote Down Martin KonecnyMartin Konecny
    Posts: 1,860Member
    Ok the next step is to run Liquidsoap manually from the terminal and see what is happening.

    Please do the following:

    Open /usr/lib/airtime/pypo/bin/airtime-liquidsoap and change the following line from

        ls_path="/usr/bin/airtime-liquidsoap --verbose -f -d"                       

    to

        ls_path="/usr/bin/airtime-liquidsoap --verbose -f"                       

    Save the changes and exit. Now run the following:

    sudo -u pypo /usr/lib/airtime/pypo/bin/airtime-liquidsoap

    If Liquidsoap crashes, it should give a better error message here.
    Airtime Pro Hosting: http://airtime.pro
Add a Comment