Booktype 2.0 -- can't export...problem with the json data?
  • Vote Up0Vote Down Quinn MurphyQuinn Murphy
    Posts: 5Member
    Hi,

    I've used the Digtial Ocean instructions (I've even put a pull request to point ppl at it), and am having some problems exporting books in my dev environment.

    When I export, it just freezes. In the logs, I see:

    [31/Dec/2015 23:52:05] ERROR [booktype:43] Could not load JSON data.
    Traceback (most recent call last):
      File "/usr/local/src/booktype/lib/booktype/utils/download.py", line 41, in fetch_url
        dta = req.json()
      File "/usr/lib/python2.7/dist-packages/requests/models.py", line 793, in json
        return json.loads(self.text, **kwargs)
      File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
        return _default_decoder.decode(s)
      File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
        raise ValueError("No JSON object could be decoded")
    ValueError: No JSON object could be decoded
    [31/Dec/2015 23:52:05] ERROR [booktype:102] Could not fetch the book from [http://shinto.
    thoughtcrimegames.net/_convert/]

    I've looked at the data and it seems OK...I've been going at this in depth and if appears that, according to the error message it prints from download.py:

    try:
            dta = req.json()
        except ValueError:
            logger.exception('Could not load JSON data.')
            return None

    might be due to malformed json. Has this been reported as a problem?  I will include a sample output in the logs for a json object to illustrate.

    [31/Dec/2015 23:52:05] DEBUG [booktype:50] {'username': u'admin', 'bookid': u'2', 'client
    id': u'27', 'sputnikid': u'j4l0wcosrozfcg8j0z3h6pbv9sca6hyg:27', 'version': u'1.0', 'form
    ats': [u'screenpdf']}
    [31/Dec/2015 23:52:05] DEBUG [booktype:93] {'input': 'input.epub', 'assets': {'input.epub
    ': u'http://shinto.thoughtcrimegames.net/testing/_export/'}, 'outputs': {u'screenpdf': {'
    profile': u'screenpdf', 'output': 'testing_2015-12-31_23-52-05.pdf', 'config': {'theme': 
    {'id': u'custom', 'custom': u'{"fontP": "serif", "lineHeightP": 100, "fontH1": "serif", "
    fontSizeH4": 100, "fontSizeH2": 140, "fontSizeH3": 120, "fontSizeH1": 160, "indentP": 0, 
    "alignH1": "left", "colorH1": "000000"}'}, 'project_id': u'testing', 'settings': {u'botto
    m_margin': u'20', u'header_margin': u'10', u'custom_height': u'', u'footer_margin': u'10'
    , u'top_margin': u'20', u'styling': u'', u'show_header': u'on', u'gutter': u'20', u'custo
    m_width': u'', u'cover_image': u' ', u'show_footer': u'on', u'side_margin': u'20', u'size
    ': u'A4'}}}}}
    [31/Dec/2015 23:52:05] ERROR [booktype:43] Could not load JSON data.
    Traceback (most recent call last):
      File "/usr/local/src/booktype/lib/booktype/utils/download.py", line 41, in fetch_url
        dta = req.json()
      File "/usr/lib/python2.7/dist-packages/requests/models.py", line 793, in json
        return json.loads(self.text, **kwargs)
      File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
        return _default_decoder.decode(s)
      File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
        raise ValueError("No JSON object could be decoded")
    ValueError: No JSON object could be decoded
  • 7 Comments sorted by
  • Vote Up0Vote Down Quinn MurphyQuinn Murphy
    Posts: 5Member
    So the first problem seemed to be that my Export_allowed_hosts looked like this: 

    EXPORT_ALLOWED_HOSTS = ['127.0.0.1']

    I changed it to this:

    EXPORT_ALLOWED_HOSTS = ['127.0.0.1','*']

    which avoids the previous problem but leaves me here:

    [04/Jan/2016 21:17:34] DEBUG [booktype:93] {'input': 'input.epub', 'assets': {'input.epub
    2a346/cover.jpg'}, 'outputs': {u'screenpdf': {'profile': u'screenpdf', 'output': 'underwo
    rld_2016-01-04_21-17-34.pdf', 'config': {'cover_image': 'screenpdf_cover_image', 'theme':
     {'id': u'custom', 'custom': u'{"fontP": "serif", "lineHeightP": 100, "fontH1": "serif", 
    "fontSizeH4": 100, "fontSizeH2": 140, "fontSizeH3": 120, "fontSizeH1": 160, "indentP": 0,
     "alignH1": "left", "colorH1": "000000"}'}, 'project_id': u'underworld', 'settings': {u'b
    ottom_margin': u'20', u'header_margin': u'10', u'custom_height': u'297', u'footer_margin'
    : u'10', u'top_margin': u'20', u'styling': u'', u'show_header': u'on', u'gutter': u'20', 
    u'custom_width': u'210', u'cover_image': u'6774045b68e1d1728653c11043c696494152a346', u's
    how_footer': u'on', u'side_margin': u'20', u'size': u'A4'}}}}}
    [04/Jan/2016 21:17:36] DEBUG [booktype.convert:35] php /var/www/booktype/instance1/bookty
    pe2mpdf.php --mpdf=/var/www/mdpf60/ --dir=/var/www/booktype/instance1/data/tmp/7b617a02-b
    352-11e5-aa94-04019394c201/screenpdf --output=/var/www/booktype/instance1/data/tmp/7b617a
    02-b352-11e5-aa94-04019394c201/screenpdf/body.pdf
    php returned 255 and produced
    stdout:
    stderr:
    [04/Jan/2016 21:17:36] ERROR [booktype.convert.pdf:745] MPDF Converter::Fail running the 
    command "No JSON object could be decoded".
  • Vote Up0Vote Down Daniel JamesDaniel James
    Posts: 844Member, Sourcefabric Team
    Hi Quinn, I have not personally experienced this error and I don't recall it being reported either. We'll do our best to get it figured out!
  • Vote Up0Vote Down Daniel JamesDaniel James
    Posts: 844Member, Sourcefabric Team
    Have you looked in the web server error log for any tell-tale lines at the same time as the export, for example under /var/log/apache2/ ? Please paste any relevant lines here.
  • Vote Up0Vote Down Aleksandar ErkalovićAleksandar Erkalović
    Posts: 207Member
    Hi Quinn,

    go to your "/var/www/booktype/instance1/bookty
    pe2mpdf.php" file:

    Modify these 2 lines:
    <?php
    error_reporting(0);

    into:
    <?php
    //error_reporting(0);

    run the publishing again and you should see errors which are reported by the mpdf. What is happening is that booktype2mpdf.php script is receiving some errors from mpdf but it is ignoring it. Once you comment it, you will get more verbose output from it.

    There are many reasons why mpdf might fail. For instance:
    - no support for png or some other image format
    - can not support given font files

    If you get no error in the output, the issue might be somewhere else.

    Aco
  • Vote Up0Vote Down GraabeinGraabein
    Posts: 94Member
    Also, I discovered errors when having too little memory on the droplet.  I had 1GB, then upgraded to 2GB and errors went away.  I can not say definately it was a memory problem, but problems went away when I upgraded to 2GB.  I only have Booktype and Postfix mailserver installed on this droplet.

    Graabein
    Post edited by Graabein at 2016-01-08 02:37:11
  • Vote Up0Vote Down Quinn MurphyQuinn Murphy
    Posts: 5Member
    Sorry it's been so long, but I figured it out!

    So I resized to no avail (but I don't think it's going to mind the extra memory).

    set up error reporting in the booktype2mpdf.  I ran the publish and got (in the celery logs):

    php /var/www/booktype/instance1/booktype2mpdf.php --mpdf=/var/www/mdpf60/ --dir=/var/www/booktype/instance1/data/tmp/f41341ce-c56d-11e5-bb81-04019394c201/screenpdf --output=/var/www/booktype/instance1/data/tmp/f41341ce-c56d-11e5-bb81-04019394c201/screenpdf/body.pdf

    Hopefully you caught the error faster than I did....

    --mpdf=/var/www/mdpf60/ 

    A typo in my base.py !!!!

    I changed it in the base.py, restarted apache to recompile, but also had to reload celery (supervisorctl reload), and now it works!!!!

    Thanks everyone for your help. It was taking me forever to get this working, but the upside is I've gotten a lot more familiar with how the program runs :)
  • Vote Up0Vote Down Daniel JamesDaniel James
    Posts: 844Member, Sourcefabric Team
    Hi Quinn, glad to hear you got it working! Please let us have your feedback, pull requests are always welcome too.

    Cheers!

    Daniel
Add a Comment