I have Django application and it uses Celery for asynchronous tasks. Additionally, I use RabbitMQ locally, but on Heroku I'm trying to setup CloudAMQP plugin. The application starts without any error, however any task sent to Celery fails with the error: [Errno 111] Connection refused Here is how heroku logs look like at beginning:
heroku[celery.1]: Process exited with status 0
heroku[celery.1]: Starting process with command `celery worker -A mysite -l info --beat`
heroku[web.1]: Starting process with command `gunicorn mysite.wsgi`
heroku[celery.1]: State changed from starting to up
app[web.1]: 2014-03-31 15:10:13 [2] [INFO] Starting gunicorn 18.0
app[web.1]: 2014-03-31 15:10:13 [2] [INFO] Listening at: http://0.0.0.0:51389 (2)
app[web.1]: 2014-03-31 15:10:13 [2] [INFO] Using worker: sync
app[web.1]: 2014-03-31 15:10:13 [7] [INFO] Booting worker with pid: 7
heroku[web.1]: State changed from starting to up
app[celery.1]: - ** ---------- .> transport: amqp://xxx@tiger.cloudamqp.com:5672/xxx
app[celery.1]: - ** ---------- .> results: amqp
app[celery.1]: -------------- celery@124124-23a8-sdf3-bd25-basdjah v3.1.10 (Cipater)
app[celery.1]: --- * *** * -- Linux-3.8.11-ec2-x86_64-with-debian-squeeze-sid
app[celery.1]: - ** ---------- .> app: mysite:0x2846324123
app[celery.1]: . tasks.add
app[celery.1]: [tasks]
app[celery.1]: -- ******* ----
app[celery.1]: . mysite.celery.debug_task
app[celery.1]: - *** --- * --- .> concurrency: 4 (prefork)
app[celery.1]: . tasks.a
app[celery.1]: -------------- .> celery exchange=celery(direct) key=celery
app[celery.1]: . tasks.k
app[celery.1]: --- ***** ----- [queues]
app[celery.1]: [2014-03-31 17:10:16,332: INFO/Beat] beat: Starting...
app[celery.1]: [2014-03-31 17:10:16,592: INFO/MainProcess] Connected to amqp://xxx@tiger.cloudamqp.com:5672/xxx
app[celery.1]: [2014-03-31 17:10:16,737: INFO/MainProcess] mingle: searching for neighbors
app[celery.1]: [2014-03-31 17:10:17,785: INFO/MainProcess] mingle: all alone
app[celery.1]: [2014-03-31 17:10:17,929: WARNING/MainProcess] /app/.heroku/python/lib/python2.7/site-packages/celery/fixups/django.py:233: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
app[celery.1]: warnings.warn('Using settings.DEBUG leads to a memory leak, never '
app[celery.1]: /app/.heroku/python/lib/python2.7/site-packages/celery/fixups/django.py:233: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
app[celery.1]: warnings.warn('Using settings.DEBUG leads to a memory leak, never '
WARNING/MainProcess] celery@124124-23a8-sdf3-bd25-basdjah ready.
I have a simple task, which fails while calling it:
@app.task(name='tasks.add')
def add(x, y):
return x * y
Here is heroku logs again:
Internal Server Error: /configurator/perf_test
app[web.1]: Traceback (most recent call last):
app[web.1]: response = wrapped_callback(request, *callback_args, **callback_kwargs)
app[web.1]: File "/app/configurator/views.py", line 58, in perf_test
[web.1]: job_id = tasks.add.delay(2,3)
[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/task.py", line 453, in delay
[web.1]: return self.apply_async(args, kwargs)
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/amqp.py", line 302, in publish_task
app[web.1]: **kwargs
app[web.1]: reply_to=reply_to or self.oid, **options
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/base.py", line 114, in get_response
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 461, in _ensured
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/task.py", line 555, in apply_async
app[web.1]: return self.connection
app[web.1]: return fun(*args, **kwargs)
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 241, in connect
app[web.1]: routing_key, mandatory, immediate, exchange, declare)
app[web.1]: **dict(self._get_exec_options(), **options)
app[web.1]: interval_start, interval_step, interval_max, callback)
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/messaging.py", line 168, in publish
app[web.1]: self._connection = self._establish_connection()
app[web.1]: conn = self.transport.establish_connection()
app[web.1]: interval_max)
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/base.py", line 323, in send_task
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/utils/__init__.py", line 230, in retry_over_time
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 713, in _establish_connection
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 754, in connection
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 111, in establish_connection
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 373, in ensure_connection
app[web.1]: self.transport = create_transport(host, connect_timeout, ssl)
app[web.1]: error: [Errno 111] Connection refused
app[web.1]: conn = self.Connection(**opts)
app[web.1]: raise socket.error(last_err)
app[web.1]: return TCPTransport(host, connect_timeout)
Part of settings.py:
DEBUG = True
TEMPLATE_DEBUG = DEBUG
import djcelery
djcelery.setup_loader()
WSGI_APPLICATION = 'mysite.wsgi.application'
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'configurator',
'django.contrib.admin',
'kombu.transport.django',
'djcelery',
)
CELERY_IMPORTS = ("tasks",)
import dj_database_url
DATABASES = {'default': dj_database_url.config()}
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']
CELERY_RESULT_BACKEND = "amqp"
BROKER_POOL_LIMIT=1
BROKER_URL = os.environ.get('CLOUDAMQP_URL')
Thank you in advance.
Limit gunicorn workers with the -w flag, and celery workers with -c. With the free Little Lemur plan you can only have 3 concurrent connections.
I have Django application and it uses Celery for asynchronous tasks. Additionally, I use RabbitMQ locally, but on Heroku I'm trying to setup CloudAMQP plugin. The application starts without any error, however any task sent to Celery fails with the error: [Errno 111] Connection refused Here is how heroku logs look like at beginning:
heroku[celery.1]: Process exited with status 0
heroku[celery.1]: Starting process with command `celery worker -A mysite -l info --beat`
heroku[web.1]: Starting process with command `gunicorn mysite.wsgi`
heroku[celery.1]: State changed from starting to up
app[web.1]: 2014-03-31 15:10:13 [2] [INFO] Starting gunicorn 18.0
app[web.1]: 2014-03-31 15:10:13 [2] [INFO] Listening at: http://0.0.0.0:51389 (2)
app[web.1]: 2014-03-31 15:10:13 [2] [INFO] Using worker: sync
app[web.1]: 2014-03-31 15:10:13 [7] [INFO] Booting worker with pid: 7
heroku[web.1]: State changed from starting to up
app[celery.1]: - ** ---------- .> transport: amqp://xxx@tiger.cloudamqp.com:5672/xxx
app[celery.1]: - ** ---------- .> results: amqp
app[celery.1]: -------------- celery@124124-23a8-sdf3-bd25-basdjah v3.1.10 (Cipater)
app[celery.1]: --- * *** * -- Linux-3.8.11-ec2-x86_64-with-debian-squeeze-sid
app[celery.1]: - ** ---------- .> app: mysite:0x2846324123
app[celery.1]: . tasks.add
app[celery.1]: [tasks]
app[celery.1]: -- ******* ----
app[celery.1]: . mysite.celery.debug_task
app[celery.1]: - *** --- * --- .> concurrency: 4 (prefork)
app[celery.1]: . tasks.a
app[celery.1]: -------------- .> celery exchange=celery(direct) key=celery
app[celery.1]: . tasks.k
app[celery.1]: --- ***** ----- [queues]
app[celery.1]: [2014-03-31 17:10:16,332: INFO/Beat] beat: Starting...
app[celery.1]: [2014-03-31 17:10:16,592: INFO/MainProcess] Connected to amqp://xxx@tiger.cloudamqp.com:5672/xxx
app[celery.1]: [2014-03-31 17:10:16,737: INFO/MainProcess] mingle: searching for neighbors
app[celery.1]: [2014-03-31 17:10:17,785: INFO/MainProcess] mingle: all alone
app[celery.1]: [2014-03-31 17:10:17,929: WARNING/MainProcess] /app/.heroku/python/lib/python2.7/site-packages/celery/fixups/django.py:233: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
app[celery.1]: warnings.warn('Using settings.DEBUG leads to a memory leak, never '
app[celery.1]: /app/.heroku/python/lib/python2.7/site-packages/celery/fixups/django.py:233: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in production environments!
app[celery.1]: warnings.warn('Using settings.DEBUG leads to a memory leak, never '
WARNING/MainProcess] celery@124124-23a8-sdf3-bd25-basdjah ready.
I have a simple task, which fails while calling it:
@app.task(name='tasks.add')
def add(x, y):
return x * y
Here is heroku logs again:
Internal Server Error: /configurator/perf_test
app[web.1]: Traceback (most recent call last):
app[web.1]: response = wrapped_callback(request, *callback_args, **callback_kwargs)
app[web.1]: File "/app/configurator/views.py", line 58, in perf_test
[web.1]: job_id = tasks.add.delay(2,3)
[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/task.py", line 453, in delay
[web.1]: return self.apply_async(args, kwargs)
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/amqp.py", line 302, in publish_task
app[web.1]: **kwargs
app[web.1]: reply_to=reply_to or self.oid, **options
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/base.py", line 114, in get_response
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 461, in _ensured
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/task.py", line 555, in apply_async
app[web.1]: return self.connection
app[web.1]: return fun(*args, **kwargs)
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 241, in connect
app[web.1]: routing_key, mandatory, immediate, exchange, declare)
app[web.1]: **dict(self._get_exec_options(), **options)
app[web.1]: interval_start, interval_step, interval_max, callback)
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/messaging.py", line 168, in publish
app[web.1]: self._connection = self._establish_connection()
app[web.1]: conn = self.transport.establish_connection()
app[web.1]: interval_max)
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/celery/app/base.py", line 323, in send_task
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/utils/__init__.py", line 230, in retry_over_time
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 713, in _establish_connection
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 754, in connection
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 111, in establish_connection
app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/kombu/connection.py", line 373, in ensure_connection
app[web.1]: self.transport = create_transport(host, connect_timeout, ssl)
app[web.1]: error: [Errno 111] Connection refused
app[web.1]: conn = self.Connection(**opts)
app[web.1]: raise socket.error(last_err)
app[web.1]: return TCPTransport(host, connect_timeout)
Part of settings.py:
DEBUG = True
TEMPLATE_DEBUG = DEBUG
import djcelery
djcelery.setup_loader()
WSGI_APPLICATION = 'mysite.wsgi.application'
INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'configurator',
'django.contrib.admin',
'kombu.transport.django',
'djcelery',
)
CELERY_IMPORTS = ("tasks",)
import dj_database_url
DATABASES = {'default': dj_database_url.config()}
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']
CELERY_RESULT_BACKEND = "amqp"
BROKER_POOL_LIMIT=1
BROKER_URL = os.environ.get('CLOUDAMQP_URL')
Thank you in advance.
Limit gunicorn workers with the -w flag, and celery workers with -c. With the free Little Lemur plan you can only have 3 concurrent connections.
0 commentaires:
Enregistrer un commentaire