small medium large xlarge

15 Jul 2012, 15:10
James Nachbar (12 posts)

I have been following the twitalytics demo under trinidad on my Mac using Lion.

I had no trouble getting the scheduler to work. However I could not compile the redis version (2.4.2), getting this compile error:

hiredis.c:700:31: error: second argument to 'va_arg' is of incomplete type 'void'

The current stable redis version (2.4.15) compiled and started without problem, and that is what I have been using

However, the trinidad_resque_extension does not start the worker. I have gone over my Gemfile and config/trinidad.yml, and they match the code in the book. The Resque overview console works (and shows that no workers are started). The only console errors I see are deprecation notices:

NOTE: Gem::GemPathSearcher#initialize is deprecated with no replacement. It will be removed on or after 2011-10-01.
Gem::GemPathSearcher#initialize called from /Users/nachbar/.rvm/gems/jruby-

The redis server is accepting the connection from the client when I start trinidad:

[39847] 15 Jul 07:44:39 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[39847] 15 Jul 07:44:39 - 0 clients connected (0 slaves), 923216 bytes in use
[39847] 15 Jul 07:44:41 - Accepted
[39847] 15 Jul 07:44:44 - DB 0: 2 keys (0 volatile) in 4 slots HT.
[39847] 15 Jul 07:44:44 - 1 clients connected (0 slaves), 931840 bytes in use

If I stop the redis server before stopping trinidad, the trinidad process hangs and has to be kill -9’d, as you describe in the text.

The work/resque/log/production.log gets created and written to as the application starts:

2012-07-15 14:44:15 INFO: No global web.xml found
2012-07-15 14:44:29 INFO: Starting ProtocolHandler ["http-bio-3000"]

(on termination, I get errors about threads not being stopped)

However, although the work/log/production.log is being used, I think I am in development mode in Rails, and log/development.log is the rails log being used:

2012-07-15 14:44:29 INFO: Starting ProtocolHandler ["http-bio-3000"]
2012-07-15 14:44:54 INFO: Pausing ProtocolHandler ["http-bio-3000"]
2012-07-15 14:44:54 INFO: Stopping service Tomcat

</code> Each time I add a job by visiting customers/index, the number of jobs in the ‘normal’ queue increases, but the status screen says ‘0 of 0 Workers Working’. The job count persists over stopping and restarting both redis and trinidad.

I am at a loss as to what to check next. Any ideas what I could be doing wrong?

20 Jul 2012, 14:14
Joe Kutner (86 posts)

Yes, I have seen this before, but I do not know exactly what is going on. It appears that the worker is actually “doing it’s thing” but the console doesn’t see it. I’m almost certain this has something to do with the worker running in a thread instead of a separate process.

If that is true, then I believe this falls in the category of “resque bug” – but the resque developers are usually not interested in supporting workers as threads (I believe David Calavera did all of the work to get it that way).

In terms of JRuby messaging and background processing, I think things are moving in the direction of Sidekiq unfortunately, there is no Trinidad extension for that.

I’ll keep looking into this, but it’s probably worth opening a trinidad_resque_extension issue for it

17 Aug 2012, 07:25
Ygor Geurts (1 post)

Do you know if anyone has already made progress with the sidekiq extension? And does the resque extension indeed use multiple threads instead of processes, as you mention in your book ?

17 Aug 2012, 20:10
Joe Kutner (86 posts)

I’m not aware of a “sidekiq extension”, but that would be an AWESOME contribution to Trinidad. As far as I know Sidekiq is officially supported on JRuby 1.6.x, and mperham seems to be very interested in continuing this.

The trinidad_resque_ext seems to have gone back and forth from using threads to not using threads. I think this is due to resque issues. The last version I used was indeed using threads. But as described above, there are issues.

31 Oct 2012, 08:21
Karol Bucek (1 post)

Hey, I wanted to add a few bits here on Resque :

Worker extensions such as trinidad_resque_extension as well as the quartz extension are a bi tricky to get right (with hot deploys etc). I’ve been pretty happy with running DJ in a Java Thread with JRuby::Rack::Worker (supports any JRuby::Rack app server) along side my app for quite a while.

There’s also pretty good Resque support built-in to run in thread-ed that aims to support recent master (should end up as resque-2.0.0) builds as well.

Recently I’ve setup a Trinidad extension (on top of JRuby::Rack::Worker) that should work “better” than the resque extension (and possibly all other worker extensions). It works a bit different - does not load a rake task or anything similar but sub-classes Resque::Worker and boots a Resque::JRubyWorker for each thread.