Plasticx Blog

If you are running your Rails application with Phusion Passenger AND you are caching using Interlock AND/OR cach_fu AND you are using the memcache-client library to connect to your memcache server, then you’ll be seeing plenty of MemCache::MemCacheError errors that might look like these

MemCache::MemCacheError: No connection to server (localhost:11211 DEAD (Timeout::Error: IO timeout), will retry at Mon Dec 10 07:47:23 -0800 2010)

or

MemCache::MemCacheError: IO timeout

If you are using the memcached library to connect to your memcache server then you might be seeing a number of Memcached::ATimeoutOccurred errors that look like this:

Memcached::ATimeoutOccurred: Key {"interlock::controller:action:action:id:tag"=>"localhost:11211:8"}

If you are the former with memcache-client errors then don’t believe the examples you’ve seen, memcache-client doesn’t work well with the way Passenger spawns Rails processes. Don’t even try it, use the memcached library instead.

If you are using Interlock and cache_fu in the same application then you need to have the Interlock plugin loaded before cache_fu. Do so in config/environment.rb like so

# load the Interlock plugin first so it will load the memcache client specified
# in memcache.yml otherwise cache_fu will load memcache-client
config.plugins = [ :interlock, :all ]

Also, when Passenger spawns a new instance of your application you must reconnect your memcache client from within Passgener’s starting_worker_process event However that code example is vague, here is what it should look like within the Rails::Initializer block in config/environment.rb

Rails::Initializer.run do |config|
  # gem and plugin configs above ....
  if defined?(PhusionPassenger)
    PhusionPassenger.on_event(:starting_worker_process) do |forked|
      if forked
        # We're in smart spawning mode ...
        if defined?(CACHE)
          Rails.logger.info('resetting memcache client')
          CACHE.reset
          Object.send(:remove_const, 'CACHE')
          Interlock::Config.run!
        end
      else
        # We're in conservative spawning mode. We don't need to do anything.
      end
    end
  end
end

What is happening in the code above is that we are closing (with reset) the current memcached connection and are then forcing Interlock to initiate a new memcached connection within it’s helper Interlock::Config.run! method. run! will not fire if the global constant CACHE has already been assigned.

The last thing we need to do is put some timeout protection around the Mecached::Rails client when it is getting and setting values from the memcache server. Interlock has a locking mechanism when its writing to the memcache server and will try to perform the write up to five times if the server doesn’t acknowledge that the write has occured. If a timeout exception bubbles up from the runtime then the purpose of the lock is defeated and it is not able to be retried. The same can be said for reads. Your application shouldn’t have a rendering error if a single read fails to complete from the memcache server. With Interlock if a memached read returns a nil value then all that happends is the code in the behavior_cache and view_cache blocks are executed. Read and write caching errors should not be imposed upon the user’s experience, in my opinion.

To do this make an initializer named config/initializers/memcached_rails.rb as the name reveals the purpose of the file. It will alias method chain Memcached::Rails get and set operations so that they only return nil instead of bubbling up a timeout error when they occur. As I already pointed out if Interlock receives a nil value from a read or a write it will proceed and execute the view_cache and/or behavior_cache blocks you have specified in your application. Memcached::Rails get and set operations underpin Interlock’s reads and writes.

class Memcached::Rails
  def get_with_timeout_protection(*args)
    begin
      get_without_timeout_protection(*args)
    rescue Memcached::ATimeoutOccurred => e
      if (RAILS_ENV == "production" ||  RAILS_ENV == "staging")
        nil
      else
        raise e
      end 
    end   
  end     
          
  def set_with_timeout_protection(*args)
    begin
      set_without_timeout_protection(*args)
    rescue Memcached::ATimeoutOccurred => e
      if (RAILS_ENV == "production" ||  RAILS_ENV == "staging")
        nil
      else
        raise e
      end
    end
  end
  
  alias_method_chain :get, :timeout_protection
  alias_method_chain :set, :timeout_protection
end

If you are using cache_fu only, you might not have to be so forceful as I’ve been with Interlock to explicitly reset the mecached client. I’m not certain how to explicitly set memcached as the client library in a cache_fu-only environment either, it seems like it has a preference for memcache-client from the its code I’ve reviewed. Post your experiences in the comments for others to learn from if you are in a cache_fu only environment and use these techniques to overcome timeout errors.

Synopsis

I’ve been in a writers block for useful bits of information to post on my blog. I guess I’ll give a summary of things I’ve seen and done lately and perhaps that will inspire others.

Work

I started my own S-Corp. to work freelance out of over the summer. I was motivated to do so after attending the The Business of Rails session at RubyConf 2007. And I’ve gotten a lot of great tips on how to be a business person in Rails from the Google group Ruby on Rails meets the business world

I started consulting through Contentfree over the summer. They are a Rails consulting shop and have been working on a startup for their Eachday photo and memories sharing site. Its been great to working with Dave Myron there. Dave is one of the best coders and software designers I’ve ever worked with and I’m not saying that just to get more work!

RubyConf

I went to RubyConf 2007. It was fun. I played a lot of Werewolf (see #53) in the evenings. Sadly, it felt like the close knit party was/is over in the Ruby community since all the momentum that Rails has brought to Ruby is bringing in the masses (I’m in that group, one of “those guys”). My guess is that RailsConf 2008 is going to feel like a JavaOne, and RubyConf 2008 will feel like RailsConf 2007 with to many tracks.

#fauna

At RubyConf I got to meet the people I’ve met in the #fauna channel on irc.freenode.net. I think some of the greatest Ruby code and ideas I’ve been exposed to are from people in that channel. Shout outs to adamblock, agile, evn, lifo, heaveysixer, loincloth, defunkt, and others.

Evan Weaver

Evan is a genius and pretty cool dude.

Pratik

Pratik is a genius and is very active on RailsCore contributions. He says Just Say No To Named Spaced Models so I guess you should ignore this post: Rails Models in a Namespace

Chris Wanstrath

Chris is a genius is full of ambition Ambition Google Group

ditching Typo

I’m probably going to ditch this Typo blog when I can make time to do it. I’ll either go with an another Rails based blog called Mephisto or Evan Weaver’s Bax blog which uses scripts and Apache SSI and is hidden on fauna’s SVN on Ruby Forge: ‘svn co svn://rubyforge.org/var/svn/fauna/bax’

like Ozimodo

I’ve been thinking about doing a Camping based tumblelog. I even paid a designer to make a template for it. I’ll blog about that latter. It will be like Ozimodo and probably steel code from it.

imPOSTor

I’m sitting on a Gem called imPOSTor that will post comments to forums such as phpBB and Web Wiz Forums . I’ll probably release within the month. It has been working inside a production grade private Rails app for over a month so I think its ready to be released. “The imPOSTor library is used to automate the act of posting comments and data to forums such as phpBB and WWF. impostor encapsulates the work of posting to these forums using a common (ruby) interface.”

MMS2R

I’m about to finish the next major version of MMS2R (2.0) . I think I’ve found the best architecture for it be maintained for the long haul. Each release of MMS2R is named after a character in the Metalocalypse cartoon.

Speaking of MMS2R Luke Francl and I submitted a MMS + Rails proposal for RubyConf 2008 called “Mobile Messaging with Rails”. Luke and I are also writing a PeepCode book about Rails+MMS+Mobile phones.

This is how I’m doing a 302 redirect to Feedburner from my Apache2 virtual host settings for this blog. Its only for the main articles syndication in RSS and Atom . Everyone is redirected to Feedburner except for their web crawling bot. I’m doing this so people can see Feedburner’s shiny widget ( ) in the sidebar showing that only 1 or 2 people subscribe to my blog.

I put the following in between the RewriteEngine On statement and the static maintenance statement RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f covered in this post describing an Apache vhosts setup for a Rails app

# 302 temporary, 301 permanent
RewriteCond %{HTTP_USER_AGENT}  !^FeedBurner/.*
RewriteRule ^/xml/rss20/feed.xml$ http://feeds.feedburner.com/mondragon/PQVR [R=302,L]
RewriteRule ^/xml/atom/feed.xml$ http://feeds.feedburner.com/mondragon/PQVR [R=302,L]

This is the combined experience from my previous entry Typo 4.0 + Apache2 + MySQL + Gentoo and Chapter 27 Deployment and Production / Agile Web Development with Rails and Capistrano: Automating Application Deployment

What I am illustrating here is how to maintain a Typo blog. What I mean by maintain is that you are creating your own
source repository for your instance of Typo. This will allow you to patch the application with fixes and your own
modifications. You will also integrate Capistrano into your repository so that you deploy your updates with ease.
Also, I show you my implementation of a Mongrel init.d script on Gentoo that will ensure Mongrel starts back up on
server reboot.

IMPORTANT!!! IMPORTANT!!! IMPORTANT!!!

This article deprecated by my Maintaining Your Own Typo 4.0.3 article.

The section in this article about the apache proxy is flawed, it doesn’t actually balance the load. There is also a better way to craft an init.d script generically for Mongrel in my new article.

I’ll leave the article in place for reference only.

IMPORTANT!!! IMPORTANT!!! IMPORTANT!!!

I’ve almost completed my Typo 4.0 + Apache2 + MySQL + Gentoo configuration. Some system administration is Gentoo flavored but I think this is a good rececipe for anyone looking to do setup of Typo 4.0 within an Apache2 instance with MySQL for the datasource.