Triggering Sprockets on Rails init

[toolmantim]

Whilst trying to reduce our app boot time and memory I noticed Sprockets extension name parsing was being triggered on Rails app load.

$ ruby -rmemory_profiler -I. -e 'MemoryProfiler.report { require "config/environment" }.pretty_print'
Total allocated 2057572
Total retained 390073

allocated memory by gem
-----------------------------------
activesupport-4.2.1 x 60149050 (note: this is just because dependency require'ing gets reported under here)
sprockets-3.0.1 x 56626099
2.2.0/lib x 28229776
rubygems x 13925583

retained memory by gem
-----------------------------------
sprockets-3.0.1 x 14024689
2.2.0/lib x 7370079
activesupport-4.2.1 x 5285833

I added a raise inside Sprockets#extname_map that is causing all this, and the only place it's called during Rails init is from this part of railtie.rb:

      config.react.react_js = lambda {File.read(::Rails.application.assets.resolve('react.js'))}

Removing react-rails from the Gemfile brings things back to normal (extname is never called):

$ ruby -rmemory_profiler -I. -e 'MemoryProfiler.report { require "config/environment" }.pretty_print'
Total allocated 1614438
Total retained 257955

allocated memory by gem
-----------------------------------
activesupport-4.2.1 x 60005108
2.2.0/lib x 28192938
rubygems x 13108443
json-1.8.2 x 9075397
mime-types-2.4.3 x 7098487
erubis-2.7.0 x 5427205

So config.react.react_js is called from in the config.after_initialize like so:

          React::Renderer.setup!( cfg.react_js, cfg.components_js, cfg.replay_console,
                                {:size => cfg.max_renderers, :timeout => cfg.timeout})

Is there a way to avoid this call to ::Rails.application.assets.resolve('react.js') in cfg.react_js?

[mchristen]

This causes extreme app bootup times.

When hosted on Heroku which has a 60s timeout for app boot you can get yourself into some trouble after a server reboot where your servers are marked as crashed and no requests are processed.

I find the app boot time to be right around 60s and it basically makes deploying to heroku a complete crapshoot on whether or not your app will boot back up.

[toolmantim]

We've just removed this gem from our app altogether…

[rmosolgo]

Thanks for your detailed report & info about how to measure the issue. This is new turf for me!

Looks like this was resolved by #253, here's a comparison of 1.0 vs master, running:

ruby -rmemory_profiler -I. -e 'MemoryProfiler.report { require "config/environment" }.pretty_print' > memory_report.txt

1.0

Lots of Sprockets

Total allocated 1447748
Total retained 463404

allocated memory by gem
-----------------------------------
  81814733  sprockets-3.0.3
  42794787  rubygems
  17847106  activesupport-4.2.1
  12381094  mime-types-2.5
  10138094  sass-3.4.13
   8725717  json-1.8.2
   8011839  2.2.2/lib
   4887473  actionpack-4.2.1
   4643550  railties-4.2.1
   4627274  bundler-1.9.6

master

Now with 20x less Sprockets :P

Total allocated 949417
Total retained 182311

allocated memory by gem
-----------------------------------
  42817946  rubygems
  17619861  activesupport-4.2.1
  12381094  mime-types-2.5
  10138254  sass-3.4.13
   8725717  json-1.8.2
   7971678  2.2.2/lib
   4887201  actionpack-4.2.1
   4844431  bundler-1.9.6
   4643519  railties-4.2.1
   4439556  sprockets-3.0.3 

Looks good to me, does that look good to you? I'll cut 1.1 after addressing a few more points on #252

[mchristen]

I tried using the current version on master, but I'm receiving the following error during bootup.

rake aborted!
TypeError: can't modify immutable index
/home/matt/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/sprockets-2.12.3/lib/sprockets/index.rb:81:in `expire_index!'
/home/matt/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/sprockets-2.12.3/lib/sprockets/base.rb:49:in `version='
/home/matt/.rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/bundler/gems/react-rails-75248975f838/lib/react/rails/railtie.rb:47:in `block in <class:Railtie>'

It seems like this was encountered in the past according to #31

[rmosolgo]

Oh i see it's here, I imagine it's a sprockets 2.12 thing. I thought one of the Appraisals ran Sprockets 2 but I guess not.

Looking now...

[rmosolgo]

btw what rake task was that?

[rmosolgo]

Yeah, I think the Rails 3.1 Appraisal uses Sprockets 2, eg this job

[mchristen]

rake db:test:prepare

Other rake tasks appear to work just fine though and my server boots up just fine, so it appears it's something to do with that rake task and possibly running under the TEST environment

[rmosolgo]

Ah, I reproduced also with rake db:migrate RAILS_ENV=test

Also:

$ rails c
Loading development environment (Rails 4.2.1)
irb(main):001:0>
# ... 
$ rails c production
/Users/rmosolgo/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/sprockets-2.12.3/lib/sprockets/index.rb:81:in `expire_index!': can't modify immutable index (TypeError)

I guess we should either test Sprockets version or make a dependency on Sprockets 3. But Sprockets 3 is really new (released April 12, 2015), so I don't think we should add that dependency.

I'm surprised the tests passed!

Paging @byroot , I guess you were running Sprockets 3?

[byroot]

I wasn't using sprockets 3 AFAIK. I'll try to take a look at this issue during the week end.

[mchristen]

Unfortunately the compass-rails gem forces Sprockets < 2.13 and lots of front-end gems tend to depend on compass-rails so I'm stuck with using Sprockets 2.13 for the time being, at least until Compass/compass-rails#227 is finished and merged.

At least the server bootup times are much quicker now, thanks for the work on this 👍

[byroot]

yeah forcing sprockets 3 is not realistic before a year or 2. But don't worry I'm fairly certain this can be fixed.

[byroot]

Ok, so I'm kinda stupid: https://github.com/rails/sprockets-rails/blob/627db8302914666ee212c823e50ed3bc1fc4a8f8/lib/sprockets/railtie.rb#L116-L118

sprockets-rails freeze the manifest on after_initialize, so it was working for us because react-rails afer_initialize callback was evaled before sprockets-rails.

Unfortunately we can't do it in before_initialize because the sprockets environment is not yet initialize at that point.

It look like doing from a regular initializer work, but I'm not 100% sure. @mchristen could you please test this branch: #267 ?

[mchristen]

I just tested your branch @byroot and that particular issue is fixed, all rake tasks I've tested seem to work fine now, thanks!

Now I'm running into an entirely separate issue related to the decoupled ServerRendering code, I'll make a comment on that pull request with some details.

[rmosolgo]

thanks for testing, glad this is cleared up!