Make is solid. It's been around since before the internet and still one of the most important tools used today. As such - it's tried, proven, tested, and was used in some of the most successful projects out there.
…but what are the chances that something so old got it right the first time? What are the chances that good old Make is the be-all, end-all, perfect ultimate solution for every possible development project? Of course not, and noone says it is - so let's try an exercise in frustration and compile some C using Rake - the cool fresh ruby little brother to big pappy Make.
Rake is a Make-like program implemented in Ruby
Rake is Ruby's version of the Make build system. Created in 2003 by the late Jim Weirich, it's goal is to get away from the idiosyncratic build system syntax and express build rules in plain Ruby. According to wikipedia, it's the most downloaded ruby gem of all time and has been bundled with OSX since 2011.
Rake has a diverse set of functions but in my experience it's primarily used to drive a single command line interface for a ton of unrelated scripts and tools. By invoking your scripts from rake tasks you can create a single unified point of access for both developers and other tools to work with. This is cool for a variety of reasons, e.g. having a Continuous Integration server run the rake task directly gives a build system that is both under version control and available locally. But this is true of many build systems… Where Rake really shines is in it's incredibly powerful File, Rule and Task functionality.
I was lucky enough to see Jim's Power Rake presentation at the Scottish Ruby Conference in 2012 where he spoke at length about the awesomeness of the pathmap method and the FileList class. Why not use these tools for modern C development? Even though one of the first examples ever provided was using Rake to compile a basic C program, it still lacks behind Make‘s out of the box no-config compilation rules. Let's see what it takes to get a Rake to walk like a Make.
Even without a Makefile, Make can still work out how to compile and link your C files. Without a Rakefile, Rake is a thing that isn't useful.
This is because of the huge library of implicit rules and behaviour that Make has collected over it's lifetime in the battlefield. Type make -p in a directory without a Makefile to list what it already knows before you've told it anything. Rake has none of this, so we'll need to replicate some functionality before we have a contender on our hands.
What we want is the same “no config” approach in Rake that Make has. Given a directory that looks like
├── Rakefile
├── foo.cWe want to be able to run rake foo and end up with an executable named foo. Additionally we could also run rake foo.o and create the intermediate object file directly.
The functionality we need to steal from Make to get our basic compilation working is
We can see from Rake's task manager source that a file task is generated when rake is given an argument which matches an existing file in the pwd. Now that we know this fact, we can add rules that will force arbitrary filenames passed to Rake result in compilation.
The first step will be to compile an object ( .o) file from a corresponding .c file. Make has this defined as $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c, stripping this down and expressing it in Rake‘s rule syntax, we get
rule ".o" => ".c" do |t|
`cc -c #{t.source} -o #{t.name}`
endThe first line basically says “whenever I need a *.o file, I also need an existing .c file with the same name”. Now in our example - when we run rake foo.o, the foo.o will match against the .o section, and then following the rule rake will look for a corresponding foo.c file, find it, and finally execute the block.
Rake passes a Task instance into the block, which is populated with a lot of information about the rule we just followed. From this task we take out the source (foo.c) and the output name (foo.o) and then pass them into the compiler via a shell invocation.
What we want to do here is solve the situation of rake foo and directly create an executable without the intermediate fluff hanging around. This can be done by matching the rule against a regular expression.
Rather awkwardly, we want something that matches against a string without a file extension. To specify that we match against any string would end up creating a circular dependency with the rule above. We can naively match a file extension with the expression /\w*\.\w*/ (we're not going to take multiple dots in filename into account), then we can invert that pattern and get the horrible looking /^((?!\w*\.\w*).)*$/which should match any single word without a file extension attached.
Throwing that into Rake and adding the compilation + linking will give us
rule /^((?!\w*\.\w*).)*$/ => ".c" do |t|
`cc #{t.source} -o #{t.name}`
endSo far so good. We can now compile very simple C executables with nothing but Ruby. Granted, we need the presence of a Rakefile and some arcane rules to compile everything correctly, but we can compile arbitrary programs nonetheless.
But what if we want to do something non trivial?? With a single line in a Makefile like x: y.o z.o we can compile and link against any number of object files. This makes it actually useful! How can we go about adding this into Rake?
Pretty easily actually, thanks to the Ruby standard library. As mentioned above, the Task object in Rake contains a lot of useful information, including a list of dependencies for the task. We can extract any object files and then pass them into the call to cc by changing the rule above as follows.
rule /^((?!\w*\.\w*).)*$/ => ".c" do |t|
deps = t.prerequisites
objects = deps.select { |d| d.pathmap == ".o" }
`cc #{t.source} #{objects.join ' '} -o #{t.name}`
endNow if we add a similar rule to our Rakefile (in our case file 'x' => ["y.o',"z.o"]) Rake will resolve the dependencies, compiling y.c and z.c into object files before compiling x.c and linking against y.o and z.o. Cool!
Now (after some minor tidying) we have a very basic Rakefile like this which will let us compile just like we're using Make
OK, now we have our contender. Let's throw him in the ring and see how he stands against a pro. Let's not start small, let's go all out brave or grave. We're going to take the Makefile from Zed A. Shaw's awesome Learn C The Hard Way series, and see how we can trick out our Rakefile to do the same tasks. Then we'll decide which solution is more convenient, readable and maintainable.
After following through Learn C The Hard Way you end up with a Makefile that should look like this. There's quite a lot going on here - a lot of tasks and targets with multiple dependencies and outputs. What we are trying to do here is replicate the exact same functionality, all tasks, in our Rakefile. Let's quickly breakdown what's going on and see what's there.
I'm going to give a very brief overview of the cool Rake features that we're going to use to get the same result as what's going on in this Makefile.
Make has a lot of variables built into it's compilation rules, which allows for a very flexible and direct means of passing parameters into the build commands. You can see this is used in the first line of the Makefile. Since Rake doesn't have any of this by default, we're going to take a simple means of keeping our config in constants, without optional environment overrides.
In lines 5-9 of the Makefile, we can see that applicable file lists are being constructed using Make‘s built in filesystem functions, wildcard and pathsubst. This is where we can use Rake‘s awesome FileList class to construct these collections. In addition, Rake extends the String class with some nice methods that let us interact with standard strings as if they are filepaths. Changing file extensions and getting path components is really easy with this functionality. We'll use the same approach for the target definitions too.
Here's where we start getting into the actual work of the build system. Make‘s default task is simply the first declared in the Makefile, Rake has an explicit :default task that can be set to any desired value. We can set dependencies to have the default task build everything, much like in Zed's Makefile.
The interesting part comes to when we encounter the programmatic tasks, like the $(PROGRAMS) task on line 22. Here the PROGRAMS variable has been set to the contents of the bin directory, but with all suffixes removed - to indicate executable files that need to be created. This is already handled by the rules we made earlier, but the Makefile also modifies the CFLAGS variable to link against the entire built library.
Here we can see that Make has a pretty interesting feature where it exploits a the way it variables, and essentially changes the existing CFLAGS variable within the scope of a certain task. This means that CFLAGS refers to something different for each task, namely it has the built library appended to it for the PROGRAMS and SO_TARGET tasks.
For more info on how make does this, you can check out Make's manual in the section Two Flavours of Variable {: .notice}
This is another feature we're going to have to replicate in Rake, and since we are just looking to prove a concept - unfortunately it's going to be a bit dirty. Instead of the variable replacement functionality, we'll just add the ability to specify additional CFLAGS for a task. Having task specific options implies a place to store task specific information, so let's do the bad thing and open up the Task class and add a field to it. In doing this we are tying the abstract concept of a build task to concrete C compiler options - and in a larger project that would be totally inexcusable - but in the context of a single Rakefile let's just go for it.
module Rake
class Task
attr_accessor :task_flags
def task_flags
@task_flags ||= []
@task_flags
end
end
endSimply by adding the line require 'rake/clean', we get clean and clobber tasks for free (clobber is clean's big brother, removing all generated files instead of just intermediate files). This is supported via the addition of two built in FileList instances with the appropriate names. By adding files to this list we can ensure that they'll be cleaned up with the correct task.
By default in Make, every task is a file task. This means that for every task you declare, Make wants to create a file with the same name. If a file with that name already exists, then nothing will happen. In this case we have a “tests” directory and a task with the same name. To override this feature in Make you can add the file to the .PHONY target, which basically means “run this task even if it's file exists anyway”. In Rake we don't have such problems, so we can just define standard task's and skip along our merry way.
One final quirk that we need to address is how Learn C the Hard Way tells it's test script to run the unit tests under Valgrind. From Make we can export variables directly to the environment, whilst with Rake we will need to do this explicitly in our shell invocations. We can use the convenience method we implemented above, and due to the way Kernel#system takes an optional hash as the first parameter for environment vars, we can just add our stuff here.
Awesome! We can now build and link our library using nothing but Rake. Following the steps above, we now have a Rakefile that looks like this, and can do everything our Makefile initially did.
Except uh… we have almost doubled the LOC… we've made awful compromises to one of the core classes of Rake itself, and we don't nearly have the flexibility and control that Make gives us from the get go. It's pretty clear from this that we are forcing Rake to do something that it's not totally designed for. Not to mention the dependencies and portablity concerns we're placing on having Ruby + Rake installed on every target build machine. Were we to take this further we'd ideally have a set of Task / FileTask subclasses to encapsulate any C specific options, and we'd have a set of rules that match Make‘s impressive repertoire that we could painlessly import at the top.
It looks like Make is the Apollo Creed to Rake‘s Rocky Balboa. Make has ultimately won out in the first instance through technical prowess and experience, but perhaps in a series of unsatisfying followup movies (read: blog posts) Rake will come back to win the world heavyweight build system title. It's clear that Rake can go the distance once you put in enough effort.