F#: Building compiler from sources.

I think every curious person couldn’t resist from peeking when he met folder named Source inside F# installation directory. And what can be funnier that browsing through the source code of real-world compiler? Of course, compiling them sources and observing all internal compiler activities, so to say “in action’.

You need F# 2.0 and F#PowerPack to be installed. Additionaly we’ll build a few necessary utilities from PowerPack sources (in fact we can build entire PowerPack from scratch, but to save time we’ll take prepared binaries).

Initial ingredients:

• F# source code (taken from F# installation package)
• F# PowerPack sources (from CodePlex)

So, let’s start:

1. Create new folder (denoted below as $ROOT)
2. Copy F# sources into $ROOT\source
3. Copy PowerPack sources into $ROOT\powerpack
4. Create auxiliary directory $ROOT\lkg\FSharp-2.0.50726.900\bin\. Copy files listed below to newly created folder
   • From PowerPack installation folder
     • FSharp.PowerPack.targets
     • FSharp.PowerPack.Build.Tasks.dll
     • Fslex.exe
     • Fsyacc.exe
   • From F# installation folder
     • Fsc.exe
     • FSharp.Core.dll
     • FSharp.Core.optdata
     • FSharpCore.sigdata
5. Briefly about remaining steps: build FsSrGen utility from PowerPack, build proto compiler and finally build result compiler with proto (obtained on step 2). To automate the build process you can use script given below (copy it to $ROOT folder)

   set MSB35="c:\WINDOWS\Microsoft.NET\Framework\v3.5\MSBuild.exe"
   
   SET ROOT=%cd%
   
   @rem Delete previous build results
   rmdir %ROOT%\powerpack\head\Debug  /S /Q
   
   @rem Build FsResGen
   cd %ROOT%\powerpack\head\workyard\FsSrGen\FsSrGen
   %MSB35% FsSrGen.fsproj
   
   @rem Build FsResGen task
   cd %ROOT%\powerpack\head\workyard\FsSrGen\FSharp.SRGen.Build.Tasks
   %MSB35% FSharp.SRGen.Build.Tasks.fsproj
   
   @rem Copy build results
   copy %ROOT%\powerpack\head\Debug\bin\*.* %ROOT%\lkg\FSharp-2.0.50726.900\bin\ /Y
   
   rem proto
   cd %ROOT%\source
   %MSB35% /p:TargetFrameworkVersion=v3.5 fsharp-proto-build.proj
   %MSB35% /p:TargetFrameworkVersion=v3.5 fsharp-compiler-build.proj
   
   cd %ROOT%\source\Debug\bin\

Overall compilation time will be something about 20 min.

Testing the result

For the test we’'ll take well-known sample with async downloading of web pages, but this time it will be examined from the inside.

open System.IO
open System.Net

let downloadPage (uri : string) = async {
    let request = WebRequest.Create(uri)
    let! response = request.AsyncGetResponse()
    use stream = response.GetResponseStream()
    use reader = new StreamReader(stream)
    return reader.ReadLine()
    }

let content = Async.RunSynchronously <| downloadPage "http://microsoft.com"
printfn "%d" content.Length

 

Create new C# ConsoleApplication project in VS, open Project Properties –> Debug page and select Start external program.

External program here: out freshly built fsc.

Save settings, press F10 and behold… you are debugging the compiler

 

After compiler finish its job modify Project Properties and replace external program file from  fsc.exe to the name of new executable file.

Start debugging and set breakpoint inside downloadPage function.

Press F11 several times and you step into source code of FSharp.Core

Having your own tamed version of the compiler is pretty convinient, you can not only browse static source code but also inspect behavior dynamically. Imagination is the only limitation, enjoy :).