elephant cheatsheet

table rows

-- works bad

SELECT schemaname,relname,n_live_tup
  FROM pg_stat_user_tables
  ORDER BY n_live_tup DESC;



Python cheatsheet

www.pythonsheets.com

python anti-patterns

django tweaks

linux cheat sheet

# resources
ps -eo size,rss,pid,user,comm --sort -rss | head -n 30
ps -eo size,rss,pcpu,pid,user,comm --sort -pcpu | head -n 30

# memstat
atop -L160 1 1 | grep MEM

# caches

# chmod directories

# awk epoch to timestamp in logs
awk '{a = strftime("%F %T", $1); $1 = "" ; print a" "$0}'

# safe file delete

# some magic in resolv.conf

# default browser ( set default apps )
xdg-mime default firefox.desktop x-scheme-handler/http
xdg-mime default firefox.desktop x-scheme-handler/https

# keyboard
setxkbmap -layout us,ru -option grp:toggle

# qt crutch
export QT_GRAPHICSSYSTEM=native | raster | opengl

# swap usage by pid

# sqlite

# get url
wget -qO- http://foo/response
wget -qO- http://bar/api/json/response | python -m json.tool
wget -qO- http://bar/api/json/response | jq
wget -qO- http://bar/api/json/response | jq '.[] | select((.k_1 == 1) or (.k_2 == 2)) | "k1:\(.k_1) k2:\(.k_2)"'

# ssh/sshfs
sshfs -o reconnect -o auto_unmount 192.168.5.5:/mnt/data/usorted/ ~/sshfs/unsorted/


to remember

source

CLUSTERFUCK

Military term for an operation in which multiple things have gone wrong. Related to "SNAFU" (Situation Normal, All Fucked Up") and "FUBAR" (Fucked Up Beyond All Repair).

In radio communication or polite conversation (i.e. with a very senior officer with whom you have no prior experience) the term "clusterfuck" will often be replaced by the NATO phonetic acronym "Charlie Foxtrot."

Wonderful!

Bill Gates: “So, how’s heaven, Steve?”
Steve Jobs: “Great! It just doesn’t have any wall or fence.”
Bill Gates: “So…?”
Steve Jobs: “So, we don’t need any Windows and Gates. I’m sorry, Bill, I didn’t mean to offend you.”
Bill Gates: “It’s ok Steve, but I heard a rumor.”
Steve Jobs: “Oh, what rumor?”
Bill Gates: “That nobody is allowed to touch Apple there, and there are no Jobs in heaven.”
Steve Jobs : “Oh no, definitely there are, but only no–pay Jobs. Therefore definitely no Bill in heaven as everything will be provided free….”

How to chmod all directories except files (recursively)

source


find /path/to/base/dir -type d -print0 | xargs -0 chmod 755
find /path/to/base/dir -type f -print0 | xargs -0 chmod 644

PHP vs Python, ru only

Если вы решили перейти с PHP на Python

Web-разработка на Python глазами PHP-программиста

Delete file safe

shred -vuzn 5 /path/to/file

Cisco AIR LAP-1141 to AIR AP-1141, converting AP from CAPWAP to autonomous

Cisco AIR LAP-1141 - Lightweight Access Point (L letter before AP) controller-based (WLC)
Cisco AIR AP-1141 - Standalone (Autonomous) Access Point

I have at work two Cisco AP-1141. Thats great wireless stations and i like how it works. Few days ago we bought two more. But by mistake new access points were LAP. I dont have Cisco wireless controller at work, its very expensive and useless for us. So i was faced with task of reconfigure LAP 1141 to AP 1141. After searching some time i found that my task can be solved by changing IOS image on device.

First of all, i needed standalone AP IOS image archive. If you has original Cisco image for standalone AP downloaded from cisco.com thats greate. I did not have it. So i download it from my access point. To store image I use tftp server.

1. Setup TFTP server
I'm using Arch linux on laptop so config file for Arch(from official wiki)
I added only -c in ExecStart to allow create new files when uploading

*TFTP server configuration has been moved to /etc/conf.d/tftpd. Please update /etc/conf.d/tftpd and remove /etc/systemd/system/tftpd.service

/etc/systemd/system/tftpd.service
[Unit]
Description=hpa's original TFTP daemon

[Service]
ExecStart=/usr/bin/in.tftpd -c -s /srv/tftp/
StandardInput=socket
StandardOutput=inherit
StandardError=journal


/etc/conf.d/tftpd
TFTPD_ARGS="-c -v -s /srv/tftp/"


2. Uploading image archive to tftp (Cisco manual)
archive upload-sw tftp://10.1.1.64/c1140-k9w7-mx.124-21a.JA1.tar


3. When upload finished, everithing like here, connect to LAP Cisco device and run commands:

debug capwap con cli
conf t
boot manual
reload

After device reboot you should see the ap: prompt. If you issue a set command you'll see a few variables that you can change.

set IP_ADDR 10.1.1.21
set NETMASK 255.255.255.0
set DEFAULT_ROUTER 10.1.1.1
tftp_init
ether_init
flash_init
tar -xtract tftp://10.1.1.64/c1140-k9w7-mx.124-21a.JA1.tar flash:
set BOOT flash:/c1140-k9w7-mx.124-21a.JA1/c1140-k9w7-mx.124-21a.JA1
set MANUAL_BOOT no
set MODE_BUTTON yes
set
boot


After boot command device will boot uploaded image, and here we go, Cisco AIR AP-1141 stanbalone autonomous access point

Have fun ;)

IOS images:
*.k9w7.* - autonomous IOS
*.k9w8.* - full lightweight IOS (this is what is bundled in the WLC .aes image, and is factory installed on "mesh" APs)
*.rcvk9w8.* - lightweight recovery image - this is factory installed on lightweight APs, unless a "mesh" image is specified; it lacks radio firmware

resolv.conf

nameserver 8.8.8.8
nameserver 4.2.2.2
# some magical options
options timeout:1 attempts:1

django project + gunicorn + supervisord + logrotate

Django 

Gunicorn

Supervisor 

/etc/supervisord.conf

[supervisord]
http_port=/var/tmp/supervisor.sock
logfile=/var/log/supervisor/supervisord.log
logfile_maxbytes=50MB
logfile_backups=20
loglevel=info
pidfile=/var/run/supervisord.pid
nodaemon=false
minfds=1024
minprocs=200

[supervisorctl]
serverurl=unix:///var/tmp/supervisor.sock

[program:guni_prj]
command=/usr/bin/gunicorn prj.wsgi -c /opt/prj/gunicorn.conf.py
directory=/opt/prj
user=prjuser
autostart=true
autorestart=true
log_stdout=true
log_stderr=true
#redirect_stdout=true
redirect_stderr=true
logfile=/var/log/supervisor/guni_prj.log
logfile_maxbytes=50MB
logfile_backups=20


/opt/prj/gunicorn.conf.py

bind = ['127.0.0.1:8090']
workers = 2
worker_class = 'gevent'
user = 'prjuser'
group = 'prjgroup'
daemon = False

import os
chdir = os.path.dirname(os.path.abspath(__file__))

# logging.handlers.WatchedFileHandler  # python logging file handler, safe for logrotate scripts

# logconfig = '%s/gunicorn.log.conf' % os.path.dirname(os.path.abspath(__file__))  # path to standart logging conf
# accesslog = '-'  # logging to stderr
# errorlog = '-'  # logging to stderr

# accesslog = '/var/log/prj/guni_prj_access.log'  # simple logging to file
# errorlog = '/var/log/prj/guni_prj_error.log'  # simple logging to file


/etc/logrotate.d/supervisor

/var/log/supervisor/*.log {
    missingok
    rotate 60
    daily
    compress
    delaycompress
    notifempty
    postrotate
        /bin/kill -SIGUSR2 $(cat /var/run/supervisord.pid 2>/dev/null) 2>/dev/null
    endscript
}


If you dont want use supervisord, gunicorn can write logs by it self (with standard logging)
Don't forget set "logconfig" option in gunicorn configuration
Logging config for example:

 /opt/prj/gunicorn.log.conf

[loggers]
keys=root, gunicorn.error, gunicorn.access

[handlers]
keys=console, error_file, access_file

[formatters]
keys=generic, access

[logger_root]
level=INFO
handlers=console

[logger_gunicorn.error]
level=INFO
handlers=error_file
propagate=0
qualname=gunicorn.error

[logger_gunicorn.access]
level=INFO
handlers=access_file
propagate=0
qualname=gunicorn.access

[handler_console]
class=StreamHandler
formatter=generic
args=(sys.stdout, sys.stderr, )

[handler_error_file]
class=logging.handlers.WatchedFileHandler
formatter=generic
args=('/var/log/prj/guni_prj_error.log',)

[handler_access_file]
class=logging.handlers.WatchedFileHandler
formatter=access
args=('/var/log/prj/guni_prj_access.log',)

[formatter_generic]
format=%(asctime)s [%(process)d] [%(levelname)s] %(message)s
datefmt=%Y-%m-%d %H:%M:%S
class=logging.Formatter

[formatter_access]
format=%(message)s
class=logging.Formatter


Python RESTful webservices with Python: Flask & Django solutions

source

Maximum results with minimum cost
- that’s ideal of all business and software development processes. How to get such a results in project with tight deadline and big performance expectations? Unfortunately there is no easy way: quality of software needs time and money, but .. choice of technologies, tools, solutions can be critical for product lifecycle. We can improve total time and cost of development and maintenance by choosing appropriate solutions.

The basis of effective software development is deep research and experience built on previous projects and  also mistakes. In our firm we carry out systematic research of technologies dedicated web and mobile development. We create internal projects and test to detect potential problems. Today I wont to present you short review of verified RESTful solutions for Python which really speed up development time.
Technology research and benchmarks
The picture below presents a benchmark of  peak JSON response per second for many technologies also Python. There are 3 Python approaches with the following results:

• Django-stripped (Django without context processors and middlewares) – 13 269 per sec
• Flask – 11 506 per sec
• Django – 7122 per sec

Swap usage by PID

source

#> for i in `find /proc -maxdepth 1 -type d|xargs -i basename {}`; do echo -n "PID: $i SWAP: "; cat /proc/$i/smaps|fgrep Swap|awk '{s=s+$2}END{print s}'; done|sort -k4n

WINE + winetricks + WoT + Steam

Small note about installation and configuration process

#> WINEARCH=win32 winecfg

#> wget http://winetricks.org/winetricks
#> chmod +x winetricks
#> sudo cp winetricks /usr/local/bin

#> cd .wine/drive_c/
#> winetricks  d3dx9_36 vcrun2008 corefonts wininet msls31
#> winetricks  msxml3
#> winetricks  ie8

#> wine  WoT_internet_install_ru.exe
#> winecfg
Libraries:  Add: msvcp110 (native, buildin), msvcr110 (native, buildin)

#> wine msiexec /i SteamInstall.msi
#> wine Steam/Steam.exe -no-dwrite

SQLite...

Client configuration - rc file for sqlite3
~/.sqliterc
.headers ON
.timer ON



Create table with autoincrement primary key - id

CREATE TABLE table_name (id INTEGER PRIMARY KEY AUTOINCREMENT);


Order of field option important, don't swap it.

Xorg - adding undetected resolutions

source

Due to buggy hardware or drivers, your monitor's correct resolutions may not always be detected. For example, the EDID data block queried from your monitor may be incorrect.

If the mode already exists, but just isn't associated for the particular output, you can add it like this:

• $ xrandr --addmode S-video 800x600

If the mode doesn't yet exist, you'll need to create it first by specifying a modeline:

• $ xrandr --newmode <Mode``Line>

You may create a modeline using the gtf or cvt utility. For example, if you want to add a mode with resolution 800x600 at 60 Hz, you can enter the following command: (The output is shown following.)

• $ cvt 800 600 60
  # 800x600 59.86 Hz (CVT 0.48M3) hsync: 37.35 kHz; pclk: 38.25 MHz
  Modeline "800x600_60.00"   38.25  800 832 912 1024  600 603 607 624 -hsync +vsync

Then copy the information after the word "Modeline" into the xrandr command:

$ xrandr --newmode "800x600_60.00"   38.25  800 832 912 1024  600 603 607 624 -hsync +vsync

X tuning

Some useful software for low level tuning X

xprop - WM classes and etc
xwininfo - window id*, position, style, owner and etc

 * xkill -id window_id_from_xwininfo

quote

You say you love rain, but you use an umbrella to walk under it.
You say you love sun, but you seek shelter when it is shining.
You say you love wind, but when it comes, you close your window.
So that's why I'm scared when you say you love me.


Bob Marley

small notes about django and etc

source 

Just to remember, how to get project bse directory in django projects: 

settings.py
import os
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(__file__))
print(BASE_DIR)



# How to tell django ignore requests like /favicon.ico and etc.

settings.py

import re
IGNORABLE_404_URLS = (
    re.compile(r'^/favicon\.ico$'),
    re.compile(r'^/robots\.txt$'),
)


# How to serve static files in dev

settings.py

import os
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

STATIC_URL = '/s/'
STATIC_ROOT = ''
STATICFILES_DIRS = (
    os.path.join(BASE_DIR, 'static'),  # <- only one static files directory
)



Some magic in widget configuration

widget = forms.Select(attrs = {'onchange': 'this.form.submit();'})



pip freeze --local | grep -v '^\-e' | cut -d = -f 1  | xargs pip install -U 

Self-signed SSL Certificate with OpenSSL

I lost previous post :(

openssl req -x509 -nodes -newkey rsa:2048 -days 3650 -keyout server.pem -out server.pem