Loading Constants - Ruby VS Rails (Part 2)

This is the second part of a two-parts blog post about how Ruby and Ruby on Rails treats constants. On Part I, we have studied the way Ruby deals with constants. On this second part, we do the same for Ruby on Rails.

Table of Contents

Part I

1. What Is a Constant in Ruby
2. Ruby Keeps Track Of Constants Defined
3. Object.constants
4. Accessing Undefined Constants
5. Which Piece of Code Raises the NameError?
6. Another Method To Check that a Constant is Defined Or Not
7. Removing A Defined Constant
8. Classes And Modules Are Constants Too
9. Nesting And Constants
10. Getting the Nesting Path
11. How Constants Are Resolved When References Are Nested?
    1. Resolving Constants
       1. Case #1 - Found in Any of Nesting Points
       2. Case #2 - Found in Ancestors of First Nesting Point
       3. Case #3 - Found in Object
       4. Case #4 - Constant is Not Resolved
    2. References that are Qualified
       1. Constant Found in the Parent
       2. Constant Found in the Ancestors of Parent
       3. Constant is not looked at at Object
12. Overriding const_missing on Nested Cases
13. const_missing for Classes and Modules

Part II

1. Ruby On Rails - Constant Resolution
2. Application Entry Point
3. Ruby Parsing of Files - load vs require
4. Rails Parsing of Files - load vs require
5. Qualified Constants - Autoloading With Nesting
6. Add Folders to Autoload Paths
7. How does Rails Reloads Constants in Development Environment?
8. config.eager_load
9. How Does Rails Attaches The Autoloading Mechanism to Constant Missing Hook?
10. Watch Out for requires and require_relatives

Ruby On Rails - Constant Resolution

All the things that we have learnt in Part I apply to Ruby. But not to Ruby on Rails. Or, at least, not exactly as we have described it above. This is because RoR overrides the Module#const_missing method and uses another complex algorithm to finally load the constants, including an autoloading mechanism.

Application entry point

Every Ruby application, even if it is a simple Ruby script, has a file which is its starting, entry point. Traditionally, this is called main in other languages, like Java or C. The Ruby interpreter will need this file in order to load and execute the commands of the application.

Assume the following file application-1/main.rb:

#  File: application-1/main.rb
#
puts 'Hello World'

This file is the application entry point and we start the application by calling Ruby interpreter and giving as argument this specific file:

ruby application-1/main.rb
Hello World

Ruby interpreter, takes one-by-one the Ruby statements inside the file and executes them, till it parses the whole file.

Usually, we execute a Ruby application by giving the starting point Ruby file as the first argument to the ruby program. But, we can also make the application entry file be the actual executable Ruby program, i.e. calling that would internally invoke the Ruby interpreter, which would parse its content. This is how we do that:

# !/usr/bin/env ruby
#
#  File: application-1/main.rb
#
puts 'Hello World'

Watch for the line 1, i.e. #!/usr/bin/env ruby. This will tell Unix shell to invoke ruby interpreter to further process the contents of the file.

Important You need to have made the file application-1/main executable. This is how you can do that: chmod +x application-1/main.

Now that you have done that, you can execute the main file itself:

application-1/main

and it will return exactly what the previous ruby application-1/main.rb did.

Ruby Parsing of Files - load vs require

Nevertheless, applications may be composed of thousands of Ruby statements. It is not practical to have all the application statements inside the same file. For that reason, we usually have different Ruby files hosting functional components and utilities of our application. When the Ruby statements we want to use are hosted in separate files, then, the main.rb file needs to ask Ruby interpreter to parse those files too. And, then, we say that main.rb file is dependent on these files, or these files are the dependencies of the main.rb file. This dependency may be recursive of course. Files that main.rb depends on might depend on other files too.

How a Ruby file asks Ruby interpreter to include in its parsing another file, a dependent file? This is usually done with one of the 2 commands:

1. Kernel#require (or Kernel#require_relative)
2. Kernel#load

The main difference between the 2 being that require will not re-parse a file if asked multiple times. It parses it only once. On the other hand, load will parse the file multiple times, every time we are trying to parse a file with load, it will parse the content of the file.

Let's see the following example. It is composed of 3 files as follows:

#  File application-2/main.rb
#
$LOAD_PATH.unshift '.'

require 'application-2/addition'
load 'application-2/multiplication.rb'

require 'application-2/addition'
load 'application-2/multiplication.rb'

puts Addition.new(5, 2).do
puts Multiplication.new(5, 2).do

#  File: application-2/addition.rb
#
puts 'Parsing addition.rb'

class Addition
  def initialize(a, b)
    @a, @b = a, b
  end

  def do
    @a + @b
  end
end

#  File: application-2/multiplication.rb
#
puts 'Parsing multiplication.rb'

class Multiplication
  C = 5

  def initialize(a, b)
    @a, @b = a, b
  end

  def do
    @a * @b + C
  end
end

If you run ruby application-2/main.rb you will get the following:

Parsing addition.rb
Parsing multiplication.rb
Parsing multiplication.rb
/Users/pmatsino/projects/blog_posts/loading_constants_ruby_vs_rails/code_samples/application-2/multiplication.rb:6: warning: already initialized constant Multiplication::C
/Users/pmatsino/projects/blog_posts/loading_constants_ruby_vs_rails/code_samples/application-2/multiplication.rb:6: warning: previous definition of C was here
7
15

This proves that the 2 load statements on the same file, i.e. the application-2/multiplication.rb asked Ruby interpreter to parse the file twice. On the other hand, the 2nd instance of require on the file application-2/addition.rb didn't actually had any impact, i.e. the file was not parsed for a second time.

Note also that the parsing of the application-2/multiplication.rb file for a second time had the side-effect of the constant Multiplication:C to be redefined, a fact that threw a warning on our console.

Rails Parsing of Files - load vs require

Rails is not different to any other Ruby application with regards to how it parses files. It has a starting point which is the rails script and which first invokes the Ruby interpreter and then tries to parse the dependent files.

It is not in the scope of this post to precisely teach you what is the exact Rails boot sequence. But here are some details:

1. First, the file config/boot.rb is required.
2. Some time later the file config/application.rb is required.
3. Then the rails/all.rb file requires some of the most important Rails files.
4. Then all the files that are specified in the Gemfile are required.
5. Then the config/environment.rb file is required.
6. Then all the initializers are being executed. One of them has to do with requiring the correct config/environments/*.rb file depending on the environment Rails has been bootstrapped in.

The last step is critical, because inside the environment specific files, you configure the cache_classes application configuration variable.

By default:

1. On development environment:
   1. cache_classes = false
2. On production environment:
   1. cache_classes = true

Is this related to how Rails parses the files? Yes it is.

cache_classes false, means that when a file is not explicitly required with a require statement, but it is parsed due to the autoloading mechanism ^{(see later on)}, it will be loaded and not required. This is very useful while in development environment. You change something in your code and Rails reloads (and does not re-require) the files, reading the new content into Ruby memory, taking into account the changes, without you having to restart the Rails server/process.

cache_classes true is for the production environment. That makes sure that all code that is autoloaded it is loaded only once. Since no changes take place at the production environment, there is no reason not to cache the classes. The caching is actually implemented by the fact that whenever a file needs to be autoloaded, then it is required and not loaded.

We will return back to this a little bit later. For now, keep in mind that

1. Whichever file is explicitly required or loaded, it is parsed as instructed (the require command requires the file, the load command loads the file).
2. There is an autoloading mechanism implemented by Rails that does either require or load depending on the cache_classes value.

Now it's time to talk about the autoloading mechanism.

Constants and Autoloading Mechanism

We are returning back to constants and, having talked about the differences between require and load, we are now ready to talk about how Rails resolves constants.

Let's suppose now that Rails parses the file app/models/apple.rb. Let's assume also that the app/models/apple.rb file has the following content:

#  File rails-1/app/models/apple.rb
#
class Apple < Fruit
end

See the folder rails-1 for this. The app/models folder does not contain any other file.

If you run the command:

rails-1/bin/rails runner "require 'apple'"

you will get this:

rails-1/app/models/apple.rb:3:in `<top (required)>': uninitialized constant Fruit (NameError)
... backtrace ...

which is expected, because the Fruit class/constant has never been defined in the past.

Now look at the rails-2 application. This one, has a fruit.rb file that defines the Fruit class. The file exists inside the folder app/models, and it has the following content:

#  File: rails-2app/models/fruit.rb
#
class Fruit
end

Now, both rails-2/app/models/apple.rb and rails-2/app/models/fruit.rb exist together inside rails-2/app/models.

Try to run this:

rails-2/bin/rails runner "require 'apple'"

Everything will run successfully, without any error this time. This means that Rails has managed to find and autoload the constant Fruit, from the file app/models/fruit.rb.

How has this been achieved?

Rails has an autoloading mechanism for missing constants. First, it has a set of paths that are called autoload_paths. By default, the paths are whatever is returned by this Rails command: ActiveSupport::Dependencies.autoload_paths

Let's try that on the rails-2 application:

rails-2/bin/rails runner 'puts ActiveSupport::Dependencies.autoload_paths'

It will return this list here:

rails-2/app/assets
rails-2/app/controllers
rails-2/app/helpers
rails-2/app/mailers
rails-2/app/models
rails-2/app/controllers/concerns
rails-2/app/models/concerns
rails-2/test/mailers/previews

As you can see, all the folders inside the rails-2/app folder are considered autoload folders. This means that any constant that is not found it is being looked inside the files on these folders.

What exactly does Rails look in these folders? It looks for a file with name that matches the name of the constant that is trying to load. What does it mean matches? Well, it is very simple. The constant name is being send to .underscore command and the resulting string is suffixed with the filename extension .rb.

Example: The constant Fruit is being turned to fruit.rb. Then Rails looks to find one of the following files, in this sequence:

rails-2/app/assets/fruit.rb
rails-2/app/controllers/fruit.rb
rails-2/app/helpers/fruit.rb
rails-2/app/mailers/fruit.rb
rails-2/app/models/fruit.rb
rails-2/app/controllers/concerns/fruit.rb
rails-2/app/models/concerns/fruit.rb
rails-2/test/mailers/previews/fruit.rb

When it finds a file that matches, stops looking further and parses the file, expecting this file to define the constant Fruit. If the constant is not defined, even if the file is there, then an error is raised. On our rails-2 example, the fruit.rb file has been found inside app/models folder. Also, the content of this file, app/models/fruit.rb defined the class, hence the constant Fruit, and everything was a success.

Let's see what happens if the file is found, but it does not define the expected constant. File rails-3/app/models/fruit.rb does not define the constant Fruit.

Try to run this:

rails-3/bin/rails runner "require 'apple'"

...activesupport-4.2.7.1/lib/active_support/dependencies.rb:495:in `load_missing_constant': Unable to autoload constant Fruit,
expected ...rails-3/app/models/fruit.rb to define it (LoadError)

It is clear from the message, that the autoloading mechanism failed to load the constant. It was not defined inside the expected file rails-3/app/models/fruit.rb.

Qualified Constants - Autoloading With Nesting

Let's look now at the following version of the apple.rb file:

#  File: rails-4/app/models/apple.rb
#
class Apple < Com::Acme::DomainModel::Fruit
end

As you can see, there is a reference to a qualified constant: Com::Acme::DomainModel::Fruit. What will happen if we try to parse apple.rb file?

Try to run:

rails-4/bin/rails runner "require 'apple'"

You will get this:

.../rails-4/app/models/apple.rb:3:in `<top (required)>': uninitialized constant Com (NameError)

Rails tried first to locate the definition of the constant Com and didn't find it. This is reasonable. It is the first constant mentioned inside the qualified reference Com::Acme::DomainModel::Fruit and it needs to be found first. Rails autoloading mechanism was triggered and Rails tried to find the file com.rb inside any of the autoload folders. It failed to find it and raised the NameError.

We have various options to help Rails find the definition of this constant Com. One that we will use here is the implicit Module definition that is triggered when, instead of a file com.rb that would define the constant, a folder com is found. When the folder com is found as a sub-folder of one of the autoload paths, then Rails implicitly instantiates a module with name Com, and hence, the constant Com, from this point on, is defined.

rails-5 project defines the folder com as a sub-folder of rails-5/app/models, which is part of the autoload paths. Let's try to parse the same apple.rb content now.

Try to run:

rails-5/bin/rails runner "require 'apple'"

You will get this:

.../rails-5/app/models/apple.rb:3:in `<top (required)>': uninitialized constant Com::Acme (NameError)

A new error that the constant Com::Acme is not defined. Great! The Com is now defined, thanks to the folder rails-5/app/models/com. We will use the same technique to define the constant Com::Acme. We will create the sub-folder acme inside the com folder. And, in order to avoid getting NameError for Com::Acme::DomainModel, we will also add the sub-folder domain_model inside the acme folder. Hence, we will have the folder branch: rails-6/app/models/com/acme/domain_model. This is where we will also create the file fruit.rb that would finally define the constant Fruit. Note that the new fruit.rb content, needs to be:

#  File: rails-6/app/models/com/acme/domain_model/fruit.rb
#
module Com
  module Acme
    module DomainModel
      class Fruit
      end
    end
  end
end

since the Fruit should be defined inside the module DomainModel, which is defined inside the moduleAcme, which is defined inside the moduleCom`.

Try to run:

rails-6/bin/rails runner "require 'apple'"

You should not get any error. And this proves how the autoloading mechanism works on Rails when we have qualified constants.

Add Folders to Autoload Paths

We have explained how the autoloading mechanism works on Rails. One of the critical ingredients here is the list of autoloaded paths. This can be altered at application configuration level. If you look inside your config/application.rb file, you will find the configuration variable

config.autoload_paths

Usually, one would like to add the config/lib folder as one more autoloading path. This can be done with a statement like:

config.autoload_paths << "#{config.root}/lib"

How does Rails Reloads Constants in Development Environment?

As we said earlier Rails watches files for changes, while on development environment, and reloads them in order to avoid you having to restart the Rails server every time you do a change.

Which piece of software does the watching of files? It is the middleware ActionDispatch::Reloader which is registered when cache_classes is false. This piece of code watches all the files inside the autoload_paths folders and their sub-folders. It also watches the i18n files and the routes definition files.

Whenever any of these files changes, then, on next request to rails server, the files in these folders will be reloaded, whenever they are needed.

How is this related to constants. Well, if any file is changed, then Rails, will call the prepare callbacks, before actually processing the incoming request. This will call the ActiveSupport::FileUpdateChecker#execute method. This will call

ActiveSupport::DescendantsTracker.clear ActiveSupport::Dependencies.clear

Both statements above, will remove the autoloaded constants, i.e. the constants that have been loaded with the autoloading mechanism described above.

Hence, after every request that follows a change in any of the autoloaded paths files, all constants are being removed and will be redefined on-demand.

Remember that on development environment the autoload files will be re-parsed using load commands and not require commands.

config.eager_load

On development environment, there is a configuration variable that is called config.eager_load and has the value false by default. Whereas on test and production environment, this value is true. What does this mean?

There is a Rails initializer that deals with the eager_load value. With this value set to true, Rails will load all the Ruby files inside the eager_load_paths as defined in config/application.rb file. Note that by default, all the sub-folders of the app folder as inside the eager_load_paths folders. The load will be done with either load or require commands, depending on the cache_classes configuration variable. Note that, while Rails will be doing that, any constants that will need be resolved will be resolved with the autoloading mechanism as described above.

How Does Rails Attaches The Autoloading Mechanism to Constant Missing Hook?

As we have learnt earlier on this post, in order to customize the constant missing behaviour, one needs to redefine the const_missing method.

As of today, this is done here: .../ruby/gems/2.2.0/gems/activesupport-4.2.7.1/lib/active_support/dependencies.rb. Inside this file, there is a module called ModuleConstMissing that defines the method const_missing as follows:

def const_missing(const_name)
  from_mod = anonymous? ? guess_for_anonymous(const_name) : self
  Dependencies.load_missing_constant(from_mod, const_name)
end

Then, later on on this file, there is a statement like this:

Module.class_eval { include ModuleConstMissing }

which basically includes the ModuleConstMissing module inside the Module module and hence overriding/redefining the const_missing implementation.

Watch Out for requires and require_relatives

Due to the fact that Rails drops all constants and re-parses the files with a load statement while on development environment, this might give you some scratching-your-head moments wondering what is going wrong.

For example, look at the rails project 7, rails-7. File rails-7/app/models/fruit.rb requires the file rails-7/app/color.rb, which defines the constant ANOTHER_COLOR. Everything goes well the first time the constant is referenced, inside the file rails-7/app/models/fruit.rb line 8.

Start the server and send curl http://localhost:3000. You will see this on the Rails logs:

Started GET "/" for ::1 at 2016-10-04 14:46:31 +0100 Processing by WelcomeController#index as / start of fruit************* Printing ANOTHER_COLOR: red Completed 200 OK in 8ms (ActiveRecord: 0.0ms)

But, if you change any of your code on any of the autoload paths files and reissue a request to rails server being on development environment, it will throw away all the constants defined and will try to parse the files again. It will try to parse the rails-7/app/models/fruit.rb file and it will fail to find the constant on line 8.

Change file fruit.rb, by adding an extra line at the end, for example. This will make sure that on the next request the reloading will take place. Send the curl http://localhost:3000 request again. You will see the following on the Rails logs:

Started GET "/" for ::1 at 2016-10-04 14:53:11 +0100
Processing by WelcomeController#index as */*
start of fruit*************
Completed 500 Internal Server Error in 2ms (ActiveRecord: 0.0ms)

NameError (uninitialized constant ANOTHER_COLOR):
  app/models/fruit.rb:8:in `<top (required)>'
  app/models/apple.rb:3:in `<top (required)>'
  app/controllers/welcome_controller.rb:3:in `index'
  ...

This is because the statement require_relative '../color.rb' will not parse the file again. Remember that require and require_relative commands parse the file once. Without the file color.rb being parsed while parsing the fruit.rb file for a second time, then constant will not be defined when parsing will reach line 8.

The lesson learned here is that you should avoid using require statements to load the files. Try to rely on the autoloading mechanism of Rails. Alternatively, use require_dependency instead of require. require_dependency comes from ActiveSupport::Dependencies module and will actually do either 'load' or 'require' depending on the cache_classes variable. On development this is false, which means the load statement will be used, and hence, the file will be parsed again.

Try to change the require_relative to require_dependency on the rails-7 example (file fruit.rb). Do the experiment with Rails server on development environment again. You will see that second request does not fail anymore.

Closing Note

That was a long journey to Ruby and Ruby on Rails Constants handling. Hope that it has contributed to your knowledge around the subject.

Also, don't forget that your comments below are more than welcome. I am willing to answer any questions that you may have and give you feedback on any comments that you may post. I would like to have your feedback because I learn from you as much as you learn from me.

About the Author

Panayotis Matsinopoulos works as Lead Developer at Simply Business and, on his free time, enjoys giving and taking classes about Web development at Tech Career Booster.