Skip to content

Add job queue support for mongodb. #623

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from

Conversation

jeremy-brenner
Copy link

This adds job queue support by making a few small changes to the Database job queue libraries. It has a separate service provider making it's usage completely optional.

@manan-jadhav
Copy link
Contributor

Instead of having a table config parameter, we should have a collection parameter, since this is Mongo.

@manan-jadhav
Copy link
Contributor

And, why not have a seperate mongodb queue driver, instead of extending the database driver?

@jeremy-brenner
Copy link
Author

It isn't a driver per se. The 'database' setting uses whatever database driver Laravel is already using. If you set your queue driver to database while the laravel-mongodb driver is in use it will try to use your mongo database and fail. This really just patches it so that it can work as intended. I have not really looked at how the other queues are implemented but since mongodb isn't really a queue then I doubt anybody would want a standalone mongodb queue while using some other database as default laravel db driver. That is my reasoning anyway. :)

tl;dr: the queue drivers are for different "queues" and "database" is to use the Laravel database which in this case happens to be mongodb.

@johngieselmann
Copy link

Any progress?

@jenssegers
Copy link
Contributor

@jeremy-brenner What did you change in the MongoQueue class? It looks almost identical as the DatabaseQueue class.

@sajadghawami
Copy link

Any news on this? Does it work?

@jenssegers
Copy link
Contributor

Currently waiting one @jeremy-brenner's response.

@sajadghawami
Copy link

hmm... ok well, since this post is pretty old, i guess there wont be any response...
has anyone tried it out yet? i havent really worked with qeues, so i cant confirm this!

@jeremy-brenner
Copy link
Author

The difference is that Mongo can't "select for update" so it was replaced with a simple loop. As for the naming, the "database" and "table" nomenclature was kept because really the "database" driver has nothing to do with the db and really means "use the ORM" which in this case happens to be Moloquent rather than Eloquent. If I switch then name then I have to do a little bit of redirecting instead of just using the ORM directly like it already does. It's a trivial difference so I left it. It works fine, I use it personally. I have not done a recent merge and test but I use it in a couple projects.

@sajadghawami
Copy link

Sweet! Pretty excited to try it out :)

One more question, which i didnt really understand from your post: Do you use Eloquent or Moloquent for this?

@jeremy-brenner
Copy link
Author

This is a simple connector to make job queues work with Moloquent. It just adds the necessary files and makes no changes to Moloquent directly. You just need to load them as outlined in the docs to make it work.

@jeremy-brenner
Copy link
Author

Is there any more I can do to help this go more smoothly? Any other outstanding questions that I have not covered?

@jenssegers
Copy link
Contributor

I prefer extending original Laravel classes rather than copy pasting all their code. So I need to figure out what was changed before I can create an extended class. Or if you want to refactor your code that's good as well :)

@jeremy-brenner
Copy link
Author

The issue is pretty straightforward.
In releaseJobsThatHaveBeenReservedTooLong in Illuminate/Queue/DatabaseQueue.php the query chains an update. This doesn't work with mongo.

protected function releaseJobsThatHaveBeenReservedTooLong($queue)
{
    $expired = Carbon::now()->subSeconds($this->expire)->getTimestamp();

    $this->database->table($this->table)
                ->where('queue', $this->getQueue($queue))
                ->where('reserved', 1)
                ->where('reserved_at', '<=', $expired)
                ->update([
                    'reserved' => 0,
                    'reserved_at' => null,
                    'attempts' => new Expression('attempts + 1'),
                ]);
}

Thats really it. I tried extending the Illuminate classes but had little luck. Another approach might be to simply emulate the ->update() behavior in the db driver leaving the database queue driver alone entirely.

jenssegers added a commit that referenced this pull request Feb 10, 2016
jenssegers added a commit that referenced this pull request Feb 10, 2016
jenssegers added a commit that referenced this pull request Feb 10, 2016
@jenssegers
Copy link
Contributor

Thanks @jeremy-brenner, I have added your code using an extending class in my latest commits.

@sajadghawami
Copy link

Nice, that was fast :)

How about using and setting everything else up? Is it basically the same as:

https://laravel.com/docs/5.1/queues

Nothing else to keep in mind?

Thanks guys!

@jenssegers
Copy link
Contributor

@leakingminds I updated the readme as well.

@jenssegers jenssegers closed this Feb 11, 2016
mnphpexpert added a commit to mnphpexpert/laravel-mongodb that referenced this pull request Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants