Hello everyone!

We have been investing plenty of personal time and energy for many years to share our knowledge with you all. However, we now need your help to keep this blog running. All you have to do is just click one of the adverts on the site, otherwise it will sadly be taken down due to hosting etc. costs. Thank you.

In this example we are going to deal with supervisor when deploying our application with capistrano. Supervisor will control beanstalk and RabbitMQ console command so that the workers are always up to date and never down. Follow example below to see how it can be done.


Application


Supervisor structure


Setup environment specific supervisor configuration files.


# Structure
app
config
supervisor
production.conf
staging.conf

The commands mentioned in config files below would normally be run manually as php bin/console job:processor country_create --limit={5 or any number} --env={prod or stag} for beanstalk and php bin/console rabbitmq:consumer -m {5 or any number} country_create --env={prod or stag} for RabbitMQ under symfony application. The critical parts in the config below are -DFOREGROUND flag and startsecs=0 option. They are needed to prevent supervisor crashing with errors. Such as # INFO gave up: ... entered FATAL state, too many start retries too quickly or Exited too quickly (process log may have details). For more info, read startsecs and Nondaemonizing of Subprocesses pages.


# app/config/supervisor/production.conf
[program:football-prod-country-create]
command=php bin/console job:processor country_create --limit=5 --env=prod -DFOREGROUND # Beanstalk
#command=php bin/console rabbitmq:consumer -m 5 country_create --env=prod -DFOREGROUND # RabbitMQ
directory=/srv/www/football/current
autostart=true
autorestart=true
startretries=5
startsecs=0
user=deployer # Our deployment user
numprocs=1
process_name=%(program_name)s_%(process_num)02d
stderr_logfile=/srv/www/football/current/var/logs/%(program_name)s_stderr.log
stderr_logfile_maxbytes=10MB
stdout_logfile=/srv/www/football/current/var/logs/%(program_name)s_stdout.log
stdout_logfile_maxbytes=10MB

# app/config/supervisor/staging.conf
[program:football-stag-country-create]
command=php bin/console job:processor country_create --limit=5 --env=stag -DFOREGROUND # Beanstalk
#command=php bin/console rabbitmq:consumer -m 5 country_create --env=stag -DFOREGROUND # RabbitMQ
directory=/srv/www/football/current
autostart=true
autorestart=true
startretries=5
startsecs=0
user=deployer # Our deployment user
numprocs=1
process_name=%(program_name)s_%(process_num)02d
stderr_logfile=/srv/www/football/current/var/logs/%(program_name)s_stderr.log
stderr_logfile_maxbytes=10MB
stdout_logfile=/srv/www/football/current/var/logs/%(program_name)s_stdout.log
stdout_logfile_maxbytes=10MB

Deployment files


deploy.rb


# deploy/deploy.rb

set :application, "football"
set :supervisor_srv_conf_path, "/etc/supervisor/conf.d"
set :supervisor_app_conf, "#{fetch(:app_config_path)}/supervisor/"+fetch(:stage).to_s+".conf"
after "deploy:cleanup", "application:supervisor:setup"

namespace :application do
namespace :supervisor do
desc "Sets up supervisor"
task :setup do
on roles (:app) do
puts "-" * 6
puts "Setting up supervisor"
ask :answer, "Would you like to restart supervisor? [y/N]"
if fetch(:answer) == "y"
source = "#{release_path}/#{fetch(:supervisor_app_conf)}"
destination = "#{fetch(:supervisor_srv_conf_path)}/#{fetch(:application)}.conf"
execute :cp, "#{source} #{destination}"
execute :supervisorctl, "reread"
execute :supervisorctl, "update"
execute :supervisorctl, "reload"
end
puts "-" * 6
end
end
end
end

Supervisor remote server permissions


In order for "deployer" user to interact with supervisor in deployment process, we need to grant him some permissions.


conf.d


This is where our application configuration files will be put by "deployer" user so do the following to grant him W permissions.


Current status.


root@prod:~# ls -l /etc/supervisor/
drwxr-xr-x 2 root root 4096 Jul 30 2013 conf.d
-rw-r--r-- 1 root root 1245 May 4 20:54 supervisord.conf

root@prod:~# ls -l /etc/supervisor/conf.d/
total 0

Create dedicated linux group named as "supervisor" and add "deployer" user to it.


root@prod:~# groupadd supervisor
root@prod:~# usermod -a -G supervisor deployer

root@prod:~# groups deployer
deployer : deployer supervisor

Let "supervisor" group own /etc/supervisor/conf.d/ directory.


root@prod:~# chgrp -R supervisor /etc/supervisor/conf.d

root@prod:~# ls -l /etc/supervisor
drwxr-xr-x 2 root supervisor 4096 Jul 30 2013 conf.d
-rw-r--r-- 1 root root 1245 May 4 20:54 supervisord.conf

Grant 775 permissions to the folder so that "deployer" can write to it.


root@prod:~# chmod 775 /etc/supervisor/conf.d

root@prod:~# ls -l /etc/supervisor
drwxrwxr-x 2 root supervisor 4096 Jul 30 2013 conf.d
-rw-r--r-- 1 root root 1245 May 4 20:54 supervisord.conf

supervisord.conf


Update config file to match section below.


# Replace this part
[unix_http_server]
file=/var/run/supervisor.sock
chmod=0700

# With this part
[unix_http_server]
file=/var/run/supervisor.sock
chmod=0770
chown=root:supervisor

Restart


To reflect to changes we need to logout and login again.


root@prod:~# logout
vagrant@prod:~# su -l root

root@prod:~# service supervisor restart
Restarting supervisor: supervisord.

Switch to "deployer" user and see if "deployer" user can reload supervisorctl.


root@prod:~# su -l deployer
deployer@prod:~$ supervisorctl reload
Restarted supervisord

If we didn't carry out steps above, we would get error below instead.


deployer@prod:~$ supervisorctl reload
error: , [Errno 13] Permission denied: file: /usr/lib/python2.7/socket.py line: 224

Test


This is the output of capistrano for supervisor.


deployer@deploy:/projects/football$ bundle exec cap production deploy
Setting up supervisor
00:18 application:supervisor:setup
01 cp /srv/www/football/releases/20170501121638/app/config/supervisor/production.conf /etc/supervisor/conf.d/football.conf
✔ 01 deployer@192.168.99.50 0.009s
02 supervisorctl reread
02 football-prod-country-create: available
✔ 02 deployer@192.168.99.50 0.107s
03 supervisorctl update
03 football-prod-country-create: added process group
✔ 03 deployer@192.168.99.50 0.088s
04 supervisorctl reload
04 Restarted supervisord
✔ 04 deployer@192.168.99.50 0.141s

And this is what file permission looks like.


deployer@prod:~$ ls -l /etc/supervisor/conf.d/
-rw-rw-r-- 1 deployer deployer 508 May 5 20:03 football.conf

Our configuration file settings defined to run only one worker for given tube so lets confirm it.


deployer@prod:~$ supervisorctl 
football-prod-country-create:football-prod-country-create_00 RUNNING pid 4906, uptime 0:03:18 # Beanstalk
#football-prod-country-create:football-prod-country-create_00 RUNNING pid 15664, uptime 0:03:18 # RabbitMQ
supervisor>

GUI shows the same result.




As you can see below, the beanstalk tube has been created and ready for incoming jobs as well as RabbitMQ queue and exchange.