Search This Blog

Monday, November 24, 2014

django project + gunicorn + supervisord + logrotate

/etc/supervisord.conf

[supervisord]
http_port=/var/tmp/supervisor.sock
logfile=/var/log/supervisor/supervisord.log
logfile_maxbytes=50MB
logfile_backups=20
loglevel=info
pidfile=/var/run/supervisord.pid
nodaemon=false
minfds=1024
minprocs=200

[supervisorctl]
serverurl=unix:///var/tmp/supervisor.sock

[program:guni_prj]
command=/usr/bin/gunicorn prj.wsgi -c /opt/prj/gunicorn.conf.py
directory=/opt/prj
user=prjuser
autostart=true
autorestart=true
log_stdout=true
log_stderr=true
#redirect_stdout=true
redirect_stderr=true
logfile=/var/log/supervisor/guni_prj.log
logfile_maxbytes=50MB
logfile_backups=20


/opt/prj/gunicorn.conf.py

bind = ['127.0.0.1:8090']
workers = 2
worker_class = 'gevent'
user = 'prjuser'
group = 'prjgroup'
daemon = False

import os
chdir = os.path.dirname(os.path.abspath(__file__))

# logging.handlers.WatchedFileHandler  # python logging file handler, safe for logrotate scripts

# logconfig = '%s/gunicorn.log.conf' % os.path.dirname(os.path.abspath(__file__))  # path to standart logging conf
# accesslog = '-'  # logging to stderr
# errorlog = '-'  # logging to stderr

# accesslog = '/var/log/prj/guni_prj_access.log'  # simple logging to file
# errorlog = '/var/log/prj/guni_prj_error.log'  # simple logging to file


/etc/logrotate.d/supervisor

/var/log/supervisor/*.log {
    missingok
    rotate 60
    daily
    compress
    delaycompress
    notifempty
    postrotate
        /bin/kill -SIGUSR2 $(cat /var/run/supervisord.pid 2>/dev/null) 2>/dev/null
    endscript
}


If you dont want use supervisord, gunicorn can write logs by it self (with standard logging)
Don't forget set "logconfig" option in gunicorn configuration
Logging config for example:

 /opt/prj/gunicorn.log.conf

[loggers]
keys=root, gunicorn.error, gunicorn.access

[handlers]
keys=console, error_file, access_file

[formatters]
keys=generic, access

[logger_root]
level=INFO
handlers=console

[logger_gunicorn.error]
level=INFO
handlers=error_file
propagate=0
qualname=gunicorn.error

[logger_gunicorn.access]
level=INFO
handlers=access_file
propagate=0
qualname=gunicorn.access

[handler_console]
class=StreamHandler
formatter=generic
args=(sys.stdout, sys.stderr, )

[handler_error_file]
class=logging.handlers.WatchedFileHandler
formatter=generic
args=('/var/log/prj/guni_prj_error.log',)

[handler_access_file]
class=logging.handlers.WatchedFileHandler
formatter=access
args=('/var/log/prj/guni_prj_access.log',)

[formatter_generic]
format=%(asctime)s [%(process)d] [%(levelname)s] %(message)s
datefmt=%Y-%m-%d %H:%M:%S
class=logging.Formatter

[formatter_access]
format=%(message)s
class=logging.Formatter



Wednesday, November 12, 2014

Python RESTful webservices with Python: Flask & Django solutions


Maximum results with minimum cost
- that’s ideal of all business and software development processes. How to get such a results in project with tight deadline and big performance expectations? Unfortunately there is no easy way: quality of software needs time and money, but .. choice of technologies, tools, solutions can be critical for product lifecycle. We can improve total time and cost of development and maintenance by choosing appropriate solutions.

The basis of effective software development is deep research and experience built on previous projects and  also mistakes. In our firm we carry out systematic research of technologies dedicated web and mobile development. We create internal projects and test to detect potential problems. Today I wont to present you short review of verified RESTful solutions for Python which really speed up development time.
Technology research and benchmarks
The picture below presents a benchmark of  peak JSON response per second for many technologies also Python. There are 3 Python approaches with the following results:
  • Django-stripped (Django without context processors and middlewares) – 13 269 per sec
  • Flask – 11 506 per sec
  • Django – 7122 per sec