Archive for January, 2007

Bizarre Rails problem

Tuesday, January 16th, 2007

Something in changeset r5951 caused my RoR project to break under Linux in subtle ways.  I get the following error when starting the console:

Loading development environment.
/usr/lib/ruby/1.8/date/format.rb:593:TypeError: superclass mismatch for class DateTime
./script/../config/../config/../vendor/rails/railties/lib/dispatcher.rb:90:NoMethodError: undefined method `attr_accessor_with_default’ for #<Class:Dispatcher>
./script/../config/../config/../app/controllers/application.rb:3:NameError: uninitialized constant ActionController

I can reproduce this problem with a fresh copy of Rails:

# cd /tmp
# rails rails
# cd rails
# script/console # this works
# rake rails:freeze:edge
# script/console # this fails

Using svn:externals on an existing project, I have tracked the problem down to the file vendor/rails/activesupport/lib/active_support/core_ext/date_time.rb.  It opens the DateTime class like this:

class DateTime
  include ActiveSupport::CoreExtensions::Time::Behavior
end

And in so doing, provokes the error.  It actually doesn’t matter what, if anything, is done with the DateTime class.  Simply reopening it provokes the failure.

Interestingly, my existing projects continue to work fine as long as they reside in the /home hierarchy (which happens to be on a separate filesystem). 
But if the project (new or existing) is in any other directory, I get the error.

Debug Info: Prior to freezing to Edge, script/about shows this:

[cch1@bmw script]$ ./about
About your application’s environment
Ruby version                 1.8.5 (i386-linux)
RubyGems version             0.9.0
Rails version                1.1.6
Active Record version        1.14.4
Action Pack version          1.12.5
Action Web Service version   1.1.6
Action Mailer version        1.2.5
Active Support version       1.3.1
Application root             /tmp/rails
Environment                  development
Database adapter             mysql
[cch1@bmw script]$

NB: The problem also happens with RubyGems 0.8.11.

I have raised case 7094 on the RoR developer site.

Benchmarking Rails

Wednesday, January 10th, 2007

I decided to take a detour and look at benchmarking my Rails application. Some background: I am running on a Compaq Proliant 3000 2×550 Pentium III Xeon with 2GB of RAM and a RAID 5 array using the old-style (white) Compaq drives. Software is Fedora Core 5 (Linux kernel 2.6.18-1.2200.fc5smp) with Apache 2.2.2 balancing across a mongrel cluster (v0.2.1) of two mongrel servers (v0.13.4). I am running on Edge Rails (r5875), and the database is MySQL 5.0.27.

To start, I used standard apache benchmark tools to get a feel for performance:

# ab -n 100 -C _MemoryMiner=b2b53b4c3141af2ed7e871291f9959f8 http://bmw.hapgoods.com/login

# ab -n 100 -C _MemoryMiner=b2b53b4c3141af2ed7e871291f9959f8 http://bmw.hapgoods.com/people/b7a2d7c0-4b49-11db-bf36-0011855ee3ff

# ab -n 100 -C _MemoryMiner=b2b53b4c3141af2ed7e871291f9959f8 http://bmw.hapgoods.com/places

Note that I am providing a session cookie to ensure that the expensive authentication methods are not run. To test the authentication code, I use this:

# ab -n 100 -A alice:XXX http://bmw.hapgoods.com/places

To get the cookie, you can either cheat (use the Rails console, look in the session store or examine the log files) or you can build a session dynamically with wget and get the session key:

# wget –save-cookies cookies.txt –keep-session-cookies -O /dev/null –post-data ‘user[login]=alice&user[password]=XXX’ http://bmw.hapgoods.com/sessions

Results from the testing show that even a simple (no DB access) GET request like the first one in the list above takes about 100ms. During the benchmarking run the CPU on the server is pegged at nearly 100% running the ruby process; this respresents the Rails overhead for instantiating a controller instance and rendering a simple view. Pretty disappointing. Interestingly, Rails log file shows the request being processed in about 35ms. Why the big discrepancy?

Moving to more complex requests like the second one show that mysql quickly gets hit hard. The response time jumps up to nearly 1700ms and mysqld starts to claim a substantial chunk of CPU. Based on the SQL query info in the Rails log file, I presume my authorization-enabled finds (using nasty joins) are the likely culprit.

Doubling the concurrency from one to two (-c 2) almost doubles the response time. Conclusion -few resources are available to service extra queries.

Next Step: Rails Benchmarking.

ActiveRecord HasManyThroughAssociation proxies

Monday, January 8th, 2007

The HasManyThroughAssociation is awesome.  But the documentation is a bit thin.  I wanted to create an builder method on the association proxy that would check for the presence of an existing join model record before creating a new one with the << method (aliased as push and concat).  This could be done trivially with a validates_uniqueness_of validation on the association model, but that approach produces an AR validation exception when there is an existing record.  To combat this, I decided to write an association proxy extension called add_if_missing.  The trick is to make it compact and to reuse as much of the existing code as possible.  To meet those objectives, I decided

  1. Adding the join model record would come with all the restrictions currently in place for the << method (no join model attributes, just foreign keys!)
  2. The uniqueness test would use the through association’s find method.

I relied heavily on three relatively new variables available to the association proxy.  In the list below, consider the example Man :has_many Grandchildren :through Children:

  • proxy_reflection - the reflection describing the relationship (Man to Grandchildren)
  • proxy_target - the association member(s) (set of Grandchildren)
  • proxy_owner - the AR instance to which the association is related (@man).

This was my first cut:

def add_if_missing(a)
  through = proxy_reflection.through_reflection
  klass = through.klass
  klass.exists?(through.primary_key_name.to_s => proxy_owner.id, proxy_reflection.association_foreign_key.to_s => a.id) ? self : self << a
end

Unfortunately, it fails to take into consideration any options of the association such as finder and counter sql, uniq, etc.

def add_if_missing(a)
  through = proxy_reflection.through_reflection
  count = proxy_owner.send(through.name).count(:conditions => {proxy_reflection.association_foreign_key.to_s => a.id})
  count == 0 ? self << a : self
end

This version falls back on the through association itself to determine if the record already exists by counting the number of records in the association.

Question: Why, with all the nice query methods available on the association, is there not an exists? method?

References:

  1. activerecord/lib/activerecord/associations.rb
  2. activerecord/lib/activerecord/reflection.rb
  3. activerecord/lib/activerecord/associations/has_many_through_association.rb
  4. activerecord/lib/activerecord/associations/has_many_association.rb
  5. http://blog.hasmanythrough.com/2006/8/19/magic-join-model-creation

Subfolders with Versamail, dovecot and Palm Treo

Monday, January 8th, 2007

I’m trying to use the following:

    Versamail (version 3.1E on a Treo650 and version 3.5 on a Treo680)
    dovecot version 1.0

Potentially a nice combo, but with the following issues:
    Subfolders can’t be detected by Versamail (formerly known at Palm as Multimail).  Consequently, you can only add them by “creating” them in Versamail with the following syntax:

    ParentFolder.SubFolder

The ParentFolder must be visible (a child of the root folder).  I have not tried deeper constructs (ParentFolder.SubFolder.SubSubFolder), but I suspect they work.

Palm’s technical resources, as usual, are aimed at the “average” user and, as usual, completely lacking in any detail of how to get the job done in the real world (where most average users have some extra-ordinary requirements).

Rails and XMLSimple

Monday, January 8th, 2007

I’ve had problems getting Rail’s automatic parsing of form-encoded XML parameters to handle arrays.  After reading up on XMLSimple I thought I had it figured out: use the anon tag to create the array.  Alas, it’s just not that simple.  RoR takes the input from XMLSimple and recursively invokes ActiveSupport::CoreExtensions::Conversions.typecast_xml_value against the result.  This nasty method forces all zero element arrays to nil and all single element arrays to a scalar value.  Thus, even if I could override RoR’s invocation of XMLSimple (in Hash.from_xml), I would get bitten by the nasty restriction in typecast_xml_value if I tried to use that method’s otherwise desirable functionality.

My lament: why does RoR not allow consistent handling of arrays parsed from the XML?  You really can’t rely on the default behavior of Rails to produce reliable arrays from XML.