Laravel Queue Manager

The Laravel Queue Manager, manage the queue process worker.

It uses Supervisor as Process Control System.

It also has a scheduler system built-in.

Installation

Composer

$ composer require pierophp/laravel-queue-manager

Publish

$ php artisan vendor:publish --provider="LaravelQueueManager\Providers\LaravelQueueManagerServiceProvider"

Running the migration

If you have MySQL version before 5.7, change in the migration the field "schedule_config" from "json" to "text".

$ php artisan migrate

Adding the provider

You need to add the provider at the file config/app.php

LaravelQueueManager\Providers\LaravelQueueManagerServiceProvider::class,

Configuration

Generating a job

You need generate a class that extends LaravelQueueManager\AbstractJob.

It's necessary to implement 2 methods:

| Method | Description | | --------- | ---------------------------- | | getName() | The name of the job | | execute() | The code of the job yourself |

Dispatching a new job

You need create a new instance of your job and call the dispatch() method.

Or use the CLI:

$ php artisan queue-manager:generate-queue queue_name

You can set optional params too:

$ php artisan queue-manager:generate-queue queue_name foo=test,bar=test

Database

To the job work correctly, it is necessary generate a row in the queue_config table.

| Field | Description | | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | name | It's the same of the return of the getName() method. | | class_name | The full path with namespace of your job class (\App\Jobs\TestJob) | | active | If the job is active or not | | schedulable | If the job is schedulable or not | | schedule_config | A JSON config of the schedule. {"method" : "The schedule methods from laravel", "params": "The params to the schedule method (optional)", "props": [ { "my_job_prop": 1 }, { "my_job_prop": 2 } ]} | | max_attemps | The max attempts of the queue | | max_instances | The max parallel instances of the queue | | timeout | The timeout of the queue | | delay | The delay to the next execution (Not implemented yet) | | connection | The connection name of the queue provider. (If null = default) | | aggregator | The aggregator is used to group the queues in a report | | config | The config is used to configure a dynamic functionality. Example json below |

Config field example

{
  "nextQueues":{
    "onError":[
      {
        "url":"url/test",
        "data":{
          "param":value
        },
        "name":"QUEUE_NAME",
        "method":"POST",
        "service":"SERVICE",
        "delay_seconds":1
      },
      {
        "url":"url/test",
        "data":{
          "param":value
        },
        "name":"QUEUE_NAME",
        "method":"GET",
        "service":"SERVICE",
        "delay_seconds":1
      }
    ],
    "onSuccess":[
      {
        "url":"url/test",
        "data":{
          "param":value
        },
        "name":"QUEUE_NAME",
        "method":"POST",
        "service":"SERVICE",
        "delay_seconds":1
      }
    ]
  }
}

Config

At the queue_manager.php config file you can configure:

| Field | Description | Default | | ------------------------- | ---------------------------------------------------------------------------------------- | ----------------------------------------- | | artisan_path | The artisan path | base_path('artisan') | | log_path | The log path | storage_path('logs/worker.log') | | supervisor_config_file | The supervisor config file | /etc/supervisor/conf.d/laravel-queue.conf | | supervisor_bin | The supervisor bin path | /usr/bin/supervisorctl | | supervisor_user | The supervisor user | docker | | supervisor_update_timeout | The supervisor update timeout to gracefully stop the process when a configuration change | 600 | | execute_as_api | Enable the queue as API mode | false | | api_url | URL to run the queue as API mode | http://127.0.0.1/queue/process | | fallback_connections | Array of fallback connections when first provider fails to dispatch | [] |

Showing all available jobs

$ php artisan queue-manager:show-jobs

Getting error events

You need add to your AppServiceProvider and log as you like:

$this->app['events']->listen(\LaravelQueueManager\Events\ScheduleError::class, function(\LaravelQueueManager\Events\ScheduleError $error){
    // my code
});

$this->app['events']->listen(\LaravelQueueManager\Events\DispatchQueueError::class, function(\LaravelQueueManager\Events\DispatchQueueError $error){
    // my code
});

Deploying

Supervisor config

You need configure a cron to run as root every minute to generate the supervisor config

$ php artisan queue-manager:generate-config

Scheduler

You need configure a cron to run every minute to generate the scheduler

$ php artisan schedule:run

Queue Restart

Every time you change the PHP code, it's necessary to restart the queues. Put this at your deploy script.

$ php artisan queue:restart

API Mode

Introduction

To easily scale your jobs machine, you can run the queues in API mode. An API is much more easy to apply auto-scale.

Configuration

In your route configuration file add:

$api->post('queue/process', 'LaravelQueueManager\Http\Controllers\QueueController@process');

Edit in your "queue_manager.php" config file the execute_as_api and api_url options.