Alex Raichev

Deploying a Flask Application on Webfaction

A few months ago i had to deploy for the first time a Flask application on a Webfaction server. After studying this Webfaciton Community thread on the topic, issuing a Webfaction support ticket for help ---Webfaction customer service is excellent--- and trying and failing several times, i figured it out. Here's what i did.

Suppose that your Flask project structure looks like this:

APPNAME
├── APPNAME
│   ├── main.py
│   ...
│   └── wsgi.py
...
├── Pipfile
├── Pipfile.lock
└── README.rst

that you are tracking it with Git, that you run it locally in a Python virtual environment, e.g. using Pipenv, and that it works. Now, let's deploy it to Webfaction.

  1. Log in to your Webfaction control panel and create a new mod_wsgi/Python application. Also create a corresponding domain and website for the app. I assume you know how to do that.

  2. Throughout the remainder of these notes, suppose that your Webfaction username is USERNAME and that your Flask application is called APPNAME

  3. SSH into your Webfaction server and run these commands:

    > mkdir home/USERNAME/webapps/APPNAME/APPNAME
    > rm -r home/USERNAME/webapps/APPNAME/htdocs
    > rm -r home/USERNAME/webapps/APPNAME/lib
    

    You don't need the lib directory in the last line, because you will be using a virtual environment.

  4. Set up Git, Git hooks, and push your local repository to Webfaction. See the Webfaction instructions here and here.

  5. Create your virtual environment ---let's call it VENVNAME here--- and install your app requirements. I think the easiest way to do this is to install Pip to install Pipenv to manage your virtual environments and packages.

  6. If your local repo has a .env environment file containing (Git-ignored) secrets, then create a corresponding .env file in your Webfaction repo.

  7. If you haven't done so already, create the file home/USERNAME/webapps/APPNAME/APPNAME/wsgi.py containing the single line from main import server as application.

    This assumes that your Flask code lies in the module main.py and that the Flask app you create in that module is call server. If that's not the case, then change the above snippet as needed.

  8. Edit the file home/USERNAME/webapps/APPNAME/apache2/conf/httpd.conf to the following, where PORT is the port number that Webfaction assigned to your application (which you can view in your Webfaction control panel for the app):

    ServerRoot "/home/USERNAME/webapps/APPNAME/apache2"
    
    LoadModule authz_core_module modules/mod_authz_core.so
    LoadModule dir_module        modules/mod_dir.so
    LoadModule env_module        modules/mod_env.so
    LoadModule log_config_module modules/mod_log_config.so
    LoadModule mime_module       modules/mod_mime.so
    LoadModule rewrite_module    modules/mod_rewrite.so
    LoadModule setenvif_module   modules/mod_setenvif.so
    LoadModule wsgi_module       modules/mod_wsgi.so
    LoadModule unixd_module      modules/mod_unixd.so
    
    LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    CustomLog /home/USERNAME/logs/user/access_APPNAME.log combined
    ErrorLog /home/USERNAME/logs/user/error_APPNAME.log
    
    Listen PORT
    KeepAlive Off
    SetEnvIf X-Forwarded-SSL on HTTPS=1
    ServerLimit 1
    StartServers 1
    MaxRequestWorkers 5
    MinSpareThreads 1
    MaxSpareThreads 3
    ThreadsPerChild 5
    
    WSGIPythonHome /home/USERNAME/.virtualenvs/VENVNAME
    WSGIDaemonProcess APPNAME processes=2 threads=12 python-path=/home/USERNAME/webapps/APPNAME/APPNAME:/home/USERNAME/webapps/APPNAME/APPNAME/APPNAME
    WSGIScriptAlias / /home/USERNAME/webapps/APPNAME/APPNAME/APPNAME/wsgi.py
    WSGIProcessGroup APPNAME
    WSGIRestrictEmbedded On
    WSGILazyInitialization On
    
  9. Restart your app via home/USERNAME/webapps/APPNAME/apache2/bin/restart.

These instructions will probably go obsolete in a few months, as all things web related do. Still, they'll help me while they last and maybe help you too.

Author: Alex Raichev
Date: 2018-05-17
Tags: instructions, tech
Permalink, Comment

Quick and Easy Fruit Picker

Description and build instructions here.

Author: Alex Raichev
Date: 2017-02-25
Tags: tech
Permalink, Comment

The Walk Radiant Dryer

Solar food dehydration without extra gadgets, even in humid climates.

Description and build instructions for the Walk Radiant Dryer here.

Author: Alex Raichev
Date: 2017-02-05
Tags: tech
Permalink, Comment

Bicycle Repair Kit

Author: Alex Raichev
Date: 2014-12-07
Tags: bicycle, tech
Permalink, Comment

Adding Celery with RabbitMQ to a Django project on Webfaction

I recently added Celery with RabbitMQ to one of my Django projects on Webfaction. Along the way, i read lots of documentation, encountered difficulties, and got advice from the helpful staff at Webfaction. What worked for me in the end are the instructions below. May they might work for you too and save you some pain!

I'll assume that you've already added Celery with RabbitMQ to your Django project in your development environment (encapsulated within a virtual environment) and have that working as desired. If not, follow the Celery docs. By the way, the latest version of Celery, version 3.1.13, works with Django out of the box and no longer requires installing the additional django-celery Django app.

Now, let's add Celery with RabbitMQ to your production environment (encapsulated within a virtual environment) on Webfaction.

Preparation

  • Ssh into your Webfaction server. All the command line instructions below take place in that session.
  • Also log in to your Webfaction control panel on the web
  • When following the instructions below, replace text of the form <stuff> with text appropriate to your situation. For example, replace <username> by your Webfaction username.

Install and configure Erlang

  1. From your Webfaction control panel, create a new application with App category: Custom, App type: Custom app (listening on port). Note the port number that gets assigned to your app.
  2. Go to Erlang's download webpage and copy the link to the latest .tar.gz version of Erlang
  3. In your ssh session, download the latest version of Erlang (currently OTP 17.1): wget <the Erlang link you copied>
  4. Unzip it: gunzip -c otp_src_17.1.tar.gz | tar xf -
  5. Change to the directory created: cd otp_src_17.1
  6. Configure the build: ./configure --prefix=/home/<username>/
  7. Make it: make
  8. Install it: make install

Test Erlang

  1. Run Erlang on the port assigned to your Erlang app (noted in the first step): epmd -port <your Erlang port> -daemon
  2. Check that it is running by looking for an epmd process: ps aux | grep epmd

Install and configure RabbitMQ

  1. From your Webfaction control panel, create a new application with App category: Custom, App type: Custom app (listening on port). Note the port number that gets assigned to your app.

  2. Go to RabbitMQ's download webpage and copy the link to the latest binary .tar.gz version of RabbitMQ (currently 3.3.4)

  3. In your ssh session, download the latest binary version of RabbitMQ: wget <the RabbitMQ link you copied>

  4. Unzip it: gunzip -c rabbitmq-server-generic-unix-3.3.4.tar.gz | tar xf -

  5. Simlink RabbitMQ to the Erlang lib directory: cd ~/lib/erlang/lib/; ln -s ../src/rabbitmq_server-3.3.4 rabbitmq_server-3.3.4

  6. Edit $HOME/rabbitmq_server-3.3.4/sbin/rabbitmq-defaults:

    ...
    # comment these lines:
    #CONFIG_FILE=${SYS_PREFIX}/etc/rabbitmq/rabbitmq
    #LOG_BASE=${SYS_PREFIX}/var/log/rabbitmq
    #MNESIA_BASE=${SYS_PREFIX}/var/lib/rabbitmq/mnesia
    # add these lines:
    CONFIG_FILE=/home/<username>/rabbitmq_server-3.1.0/sbin/
    LOG_BASE=/home/<username>/logs/user/rabbitmq # create this directory!
    MNESIA_BASE=/home/<username>/rabbitmq_server-3.1.0/sbin/
    ...
    
  7. Create the RabbitMQ log directory: mkdir /home/<username>/logs/user/rabbitmq

  8. Edit $HOME/rabbitmq_server-3.3.4/sbin/rabbitmq-env:

    ...
    # add to the end
    export ERL_EPMD_PORT=<your Erlang port>
    export RABBITMQ_NODE_PORT=<your RabbitMQ port>
    export ERL_INETRC=$HOME/.erl_inetrc
    
  9. Create $HOME/hosts, replacing dweb125 with your Webfaction hostname prefix:

    127.0.0.1 developmenthost.developmentdomain developmenthost
    ::1      developmenthost6.developmentdomain6 developmenthost6
    127.0.0.1 dweb125 dweb125.webfaction.com
    
  10. Create $HOME/.erl_inetrc:

    {hosts_file, "/home/<username>/hosts"}.
    {lookup, [file,native]}.
    

Test RabbitMQ

  1. Make sure you have Erlang running. Start RabbitMQ via ./rabbitmq_server-3.3.4/sbin/rabbitmq-server -detached, then check on its status via ./rabbitmq_server-3.3.4/sbin/rabbitmqctl status, and look for a line with {amqp,<your RabbitMQ port>,"::"}.

Install and configure Celery

  1. Within your ssh session, activate the virtual environment for your Django project and install Celery via pip install celery.

  2. Assuming you've already configured Celery to work in your development environment, all that's left to do is add the following line to the production version of your Django settings file:

    BROKER_URL = 'amqp://guest:guest@localhost:<your RabbitMQ port>//'
    

Test Celery

  1. Within your ssh session and Django project directory, run celery -A <your Django project> worker -l info

That's it. Fyew! You now have Erlang, RabbitMQ, and Celery installed and running on Webfaction.

Todo

I've heard that controlling Erlang + RabbitMQ + Celery with Supervisor is a good idea. Will look into that.

Author: Alex Raichev
Date: 2014-08-02
Tags: instructions, tech
Permalink, Comment

Blohg to Pelican

Over the past few days i moved this website's backend from Blohg to Pelican, both of which are RST-capable website generators. I did so, because i don't need the dynamic Flask-application capabilities of Blohg and because Pelican is more mature and equally simple to use.

The transition was mostly smooth, but i did encounter a few issues that required me to search beyond Pelican's documentation to resolve. To save new Pelicaneers the effort, here are those issues and resolutions.

Issue 1: Custom paginated templates.

I wanted a separate landing page and blog page, and the Pelican docs don't elaborate on this point. Querying the Pelican developers through the project's Github page, i received instructions from justinmayer. First, he said, i needed to make a custom template to house my blog, which i called blog.html. It contains the code

{% extends "base.html" %}
{% set active_page = "blog" %}
{% block title %}{{ SITENAME }} - Blog{% endblock %}
{% block content %}
{% for article in (articles_page.object_list if articles_page else articles) %}
    <div class="blogItem">
    <h1><a href="{{ SITEURL }}/{{ article.url }}">{{ article.title }}</a></h1>
    {{ article.content }}

    <div class="blogMeta">
    Author: <a href="mailto: {{ AUTHOR_EMAIL }}">{{ article.author }}</a><br>
    Date: {{ article.locale_date }}<br>
    {% if article.tags %}
        Tags:
        {% for tag in article.tags %}
            <a href="{{ SITEURL }}/{{ tag.url }}">
            {{ tag }}</a>{% if not loop.last %}, {% endif %}
        {% endfor %}<br />
    {% endif %}
    <a href="{{ SITEURL }}/{{ article.url }}#disqus_thread">Comments</a> -
    <a href="{{ SITEURL }}/{{ article.url }}">Permalink</a>
    </div>
    </div><!-- end #blogItem -->
{% endfor %}
{% include 'pagination.html' %}
{% endblock content %}

By the way, here's the code for the pagination template pagination.html:

{% if articles_page and articles_paginator.num_pages > 1 %}
    <div class="pagination">
    <ul>
    {% if articles_page.has_previous() %}
        {% set num = articles_page.previous_page_number() %}
        <li class="prev"><a href="
            {{ SITEURL }}/{{ page_name }}{{ num if num > 1 else ''}}.html"
            >&laquo;</a></li>
    {% endif %}
    {% for num in range( 1, 1 + articles_paginator.num_pages ) %}
        <li><a href="
            {{ SITEURL }}/{{ page_name }}{{ num if num > 1 else '' }}.html"
            class="{{ 'active' if num == articles_page.number else '' }}"
            >{{ num }}</a></li>
    {% endfor %}
    {% if articles_page.has_next() %}
        <li class="next"><a href="
            {{ SITEURL }}/{{ page_name }}{{
               articles_page.next_page_number() }}.html"
            >&raquo;</a></li>
    {% endif %}
    </ul>
    </div>
{% endif %}

I got it from the tuxlit_tbs Pelican theme and modified it to my liking.

Second, i had to add the following lines to my pelicanconf.py.

DIRECT_TEMPLATES = (('index', 'blog', 'tags', 'categories', 'archives'))
PAGINATED_DIRECT_TEMPLATES = (('blog',))

Issue 2: Disqus comments.

This is another topic the Pelican docs don't elaborate on. To embed Disqus comments in my blog pages, i read Disqus's help article on embedding comments and Disqus's help article on adding comment counts and did the following.

First, i made the template disqus_comments.html which contains the code

{% if DISQUS_SITENAME %}
    <div class="blogItem">
    <h2>Comments</h2>
    <div id="disqus_thread"></div>
    <script type="text/javascript">
        var disqus_shortname = '{{ DISQUS_SITENAME }}';
        (function() {
            var dsq = document.createElement('script');
            dsq.type = 'text/javascript';
            dsq.async = true;
            dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
            (document.getElementsByTagName('head')[0] ||
             document.getElementsByTagName('body')[0]).appendChild(dsq);
        })();
    </script>
    <noscript>
        Please enable JavaScript to view the
        <a href="http://disqus.com/?ref_noscript={{ DISQUS_SITENAME }}">
            comments powered by Disqus.
        </a>
    </noscript>
    <a href="http://disqus.com" class="dsq-brlink">
        blog comments powered by <span class="logo-disqus">Disqus</span>
    </a>
    </div>
{% endif %}

and in my article.html template added the line {% include 'disqus_comments.html' %} before the line {% endblock %}. Doing so embeds Disqus comments for each blog post (article).

Second, i made the template disqus_comment_counts.html which contains the code

{% if DISQUS_SITENAME %}
    <script type="text/javascript">
        var disqus_shortname = '{{ DISQUS_SITENAME }}';
        (function () {
            var s = document.createElement('script'); s.async = true;
            s.type = 'text/javascript';
            s.src = 'http://' + disqus_shortname + '.disqus.com/count.js';
            (document.getElementsByTagName('HEAD')[0] ||
             document.getElementsByTagName('BODY')[0]).appendChild(s);
        }());
    </script>
{% endif %}

and in my base.html template added the line {% include 'disqus_comment_counts.html' %} before the </body> tag. That enables blog post comment counts via the tags <a href="{{ SITEURL }}/{{ article.url }}#disqus_thread">Comments</a>.

Issue 3: Typesetting math.

I wanted to use MathJax to typeset math on this site. Pelican has a LaTeX plugin, which uses MathJax, but doesn't work properly at present. It didn't typeset math in the list view of my blog posts, only in the detail view of each post. So i searched the web for an alternative and found this helpful blog post from another Pelican user. Following its instructions, i added this Javascript code into the <head> section of my base.html template:

<!-- Using MathJax, with the delimiters $ -->
<!-- Conflict with pygments for the .mo and .mi -->
<script type="text/x-mathjax-config">
    MathJax.Hub.Config({
    "HTML-CSS": {
    styles: {
    ".MathJax .mo, .MathJax .mi": {color: "black ! important"}}
    },
    tex2jax: {inlineMath: [['$','$'], ['\\(','\\)']],processEscapes: true}
    });
    MathJax.Hub.Register.StartupHook("HTML-CSS Jax Ready",function () {
    var VARIANT = MathJax.OutputJax["HTML-CSS"].FONTDATA.VARIANT;
    VARIANT["normal"].fonts.unshift("MathJax_SansSerif");
    VARIANT["bold"].fonts.unshift("MathJax_SansSerif-bold");
    VARIANT["italic"].fonts.unshift("MathJax_SansSerif-italic");
    VARIANT["-tex-mathit"].fonts.unshift("MathJax_SansSerif-italic");
    });
    MathJax.Hub.Register.StartupHook("SVG Jax Ready",function () {
    var VARIANT = MathJax.OutputJax.SVG.FONTDATA.VARIANT;
    VARIANT["normal"].fonts.unshift("MathJax_SansSerif");
    VARIANT["bold"].fonts.unshift("MathJax_SansSerif-bold");
    VARIANT["italic"].fonts.unshift("MathJax_SansSerif-italic");
    VARIANT["-tex-mathit"].fonts.unshift("MathJax_SansSerif-italic");
    });
</script>

<script type="text/javascript"
src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS_HTML">
</script>

I removed the ['$','$'] item which allows dollar signs to delimit math text, as is standard in LaTeX documents, but it requires escaping every backslash in the math text with an additional backslash. For now i'll stick with the standard RST role :math: for inline math and the standard RST directive .. math:: for block math display. The area of a circle is Ac = (π ⁄ 4)d2, that is,

Ac = (π)/(4)d2.
Author: Alex Raichev
Date: 2013-05-11
Tags: instructions, tech
Permalink, Comment

Science and Python: retrospective of a (mostly) successful decade

youtube~F4rFuIb1Ie4

Video sourced from YouTube here.

Author: Alex Raichev
Date: 2013-02-27
Tags: video, Python, science, tech
Permalink, Comment


Why no comments? I used to do public comments but found that moderating and maintaining them took too much time in front of the computer, time better spent playing outdoors. So these days I only do private comments, that is, you can email me comments regarding a post by clicking the 'Comment' link at the bottom of the post.