Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
forked from chexov/queueit

Simple cli tools for processing workers. (redis or beanstalkd)

License

Notifications You must be signed in to change notification settings

udayanw/queueit

Open more actions menu
 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

74 Commits
74 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

INTRODUCTION
============

Queueit is a CLI interface tool
which helps to integrate beanstalkd[1] queue service into my shell scripts.
This could be usefull for the cloud processing or task paralelling.

[1] http://kr.github.com/beanstalkd/

I am also using redis-server in one of the projects so there are scripts which
will do the same for redis backend. Please check redis/redis-queue script.

ENVIROMENT VARIABLES
====================

You can redefine variables in a such way:
export QUEUEIT_HOST=btd.local
export QUEUEIT_PORT=11300
export QUEUEIT_TIMEOUT=100
export QUEUEIT_TTR=5000


USING q-put AND q-get
=====================
1. Run the beanstalkd server
beanstalkd -d

2. Push new tasks to process
q-put todownload "http://site.com/video1.avi" "http://video2.avi"

3. Run workers in the cloud nodes

# Downloading files using wget and putting task into 'encoding' queue
while true; do
   url=$(q-get todownload);
   wget -c $url -O $$.out
   q-put toencode $$.out
done

# Get task from the 'encoding' queue and encode video file using HandbrakeCLI
while true; do
   in_file=$(q-get toencode)
   HandbrakeCLI -i $in_file -o ${in_file%.avi}.mp4
one



USING q-wrapper
===============
1. Run beanstalkd queue server:
beanstalkd -d

2. Push new task
q-put download "http://site.com/video1.avi"

3. Run q-wrapper to process the tasks
q-wrapper download encode wget {}


USING q-stat
============
q-stat shows nubmer of ready and buried tasks per queue
tube                     watching      buried        ready
default                  24    +1      0             0
t-start                  3             124           570   -3
stats                    1             3             0
t-ir                     1             6             67    +2
t-ip                     1             6             0
t-upl                    0             0             9907
t-t350                   1             0             0
t-t1350                  1             0             0

About

Simple cli tools for processing workers. (redis or beanstalkd)

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 93.6%
  • Shell 4.8%
  • Makefile 1.6%
Morty Proxy This is a proxified and sanitized view of the page, visit original site.