Ruby with a Side of Sorbet

I come from a world of types. The languages I know best (C, C++ and Go) are all statically typed languages. Living in the dynamically typed word of Ruby, PHP or Javascript you would think would feel freeing. Instead, I feel annoyed and frustrated at my loss of a security blanket.

PHP

As early as PHP 5.0, types started to slowly creep into the language by way of type-hinting. As of PHP 7, typing is becoming more prevalent and I, for one, applauded it. My first job was working on a Managed Services team whose job was to fix broken sites and we worked in PHP. A vast majority of bugs would have been caught and fixed with typing or proper index checking.

Other developers who had never worked in another language, or only flirted with typed languages in school, didn't understand why I cursed PHP. Dynamic typing was freeing and easy, why would they want to use typing?


At that time, I extolled the virtues of statically typed languages and urged many to at least start using type hinting. I gained many a convert and soon most of the company started to adopt it, certainly for any new software we wrote.

Ruby

When I joined Shopify, I was returned to the dynamically typed world with Ruby. Ruby was like PHP on steroids. Much of what it does worries me greatly but, thankfully, there's a lot of excellent developers at Shopify and most of the code I deal with is pretty well written. We have style guides that forbid, or at least strongly discourage, some of the gnarly things you can do with the language. There's a heavy emphasis on writing good quality tests, a gambit of code review, and a concerted effort to improve anything we do.

That said, some of the same types of bugs I encountered in the PHP world were repeated by developers in the Ruby world. Issues that, once again, proper typing could have caught. However, this time around, I decided not to fight it. I am surrounded by some pretty strong developers and I was willing to continue to learn Ruby to see if proper techniques could further reduce bugs.

To be honest, they can do a pretty good job and I can certainly see why, when you know what you're doing, you can avoid most of the pitfalls of a dynamically typed language.

And yet...

Sorbet

There's a growing number of developers within our core product that have been pushing for the adoption of static type analysis in Ruby. I'll spare any details on history and alternatives; suffice to say, Sorbet is the solution currently being promoted.

The first time I tried it, I hated it. Then, many months later I bumped into one of our core developers during a conference and had lunch with them. During our conversation, I brought up Sorbet and we got into an interesting discussion in which he encouraged me to revisit it.

You see, when I first tried it, I was still a relative novice at Ruby. Sorbet felt foreign and tacked on. Some of the errors it gave were cryptic and, quite frankly, bizarre. I also had made an important mistake. I'd turned on the strictest level of typing.

Ya, don't do that. Not yet, anyway.

This time, I took the recommendation that gradual typing be used, even on a new project. Slowly increase the typing level as you gain stability and see how it goes. At the current maturity of the typing system, I think this is the wise choice.

The best part? It made some recommendations to write better Ruby. Not only that, it caught several bugs that my tests hadn't exposed! It even taught me, and a peer who is a long-time Ruby developer, something that neither of us knew about the language (that the initialize method should be private)!

Conclusion

Use Sorbet! It feels weird, like a new pair of underwear but once you get used to it, you'll be thankful for it!

React - A Follow Up

Note: This is a follow-up article to my first experiences working with react.

Shopify is full of many intelligent and skilled developers and, now that I work with them, I've gained the benefit of the experience of those much more experienced with React. We maintain are own React component library called Polaris but, that said, many of us are still novices at writing React apps. Now that I've gained more experience, I have more thoughts to share!

1. App Structure

I thought I'd gotten better at this but many screw ups later I've realized I still have much to learn. Having a src/ directory is standard for React apps but it isn't always clear how to organize the sub-directories. There's also a lesson in writing components that I had to learn.

A components/ directory is create and you can nest components in whatever way works best for you, provided there's sound logic to it, but knowing how to write those components can be pretty confusing for the novice React developer. The key is understanding how to utilize dumb, generic components alongside domain specific components.

A single page application can usually be broken down into separate pages and it makes sense to have a pages/ directory. These pages are typically pretty domain specific and might contain sub-directories for components that only ever relate to that particular page.

Another option might be to create domain specific directories. A blog app could potentially organize different views into directories, like posts/, comments/ and users/.

Continuing the blog example, since you might have a list of comments, a list of users or a list of posts, it stands to reason you should only have one component to display them, wrapped by a domain specific version if necessary. Creating List and ListItem components that are wrapped by CommentList and Comment components respectively, lets you optimize your component usage. It stands to reason that, for the most part, your lists will have some common properties.

2. Per-Component CSS

So, now we have CSS-modules. A blessing and a curse. On one hand, it removes the problem of namespace collisions and unwanted style-sheet cascading issues. It makes your CSS much simpler and removes much of the necessity of depending on SASS or LESS. On the other, it's more of a pain in the ass when you want to override a style. Good structure and organization of your CSS and where you apply certain styles is key.

Overall, I'd call them a net gain but not without their own warts. Regardless, I still highly recommend per-component CSS!

3. Network Communication

As nice as promises and the Fetch API are, I have to say I now prefer Axios. "Fetching" a POST just feels odd. Not being able to catch non-network errors (40x responses) within Promises feels non-intuitive; it requiring extra boilerplate.

Axios just feels more natural. The verbs for performing actions make sense. 40x responses can be caught instead of being checked for. Browser compatibility, if that's a concern, is covered.

And now...

That's about it! Nothing too revolutionary but still, some nice tidy lessons from being in the trenches.

Ta-ta for now!

My First React App And What I Did Wrong

I recently had the pleasure of using React to build an internal use only app for Acro Media. I say pleasure because it genuinely was despite the fact I was met with many frustrations with using React for the first time. To be fair, these were more my own shortcomings trying to learn a framework I'd never used before, rather than the framework itself.

As with building any app, my failures taught me far more than my successes. Here are those failures:

1. Poor Structure

This stems mainly from the fact I didn't understand what React really was. I was aware, of course, that parts of my page should be abstracted into components but I didn't understand how to truly utilize that. I didn't, for example, build my components in a way that would make them reusable in a concrete way. Not only were they not reusable within the app but they couldn't be copied into another app, either.

I didn't understand that even though I created a table component, with header and body sub-components abstracted out, I made them very specific to the app. While this app only needed the one table, and so this worked out fine in this instance, I would have had to do a lot of refactoring when I had the need to add a second one. Instead, I should have created a generic table component and used props or higher order components to customize it.

Finally, I used a single src/ directory to hold my handful of components. Even though this worked fine in this very specific case, if I'd laid out the project, and thereby the components in a more discrete way, I would have required a better defined structure.

Generally speaking, you'd have a Components/ directory separated into components, like Table/.

2. Poor Per-Component CSS

When I began the project, I was aware you could import CSS into a React component. I assumed, wrongly, this was just a nice way to abstract CSS into smaller chunks. While this is indeed a benefit, how wrong was I?

Thankfully, before the project's end, I realized the power of using per-component CSS. I was further relieved to discover how I could leverage SASS and Webpack with React to make a very powerful combination of tools.

3. Poor Understanding of State

This was highly frustrating. I had only two main components: a form in which you could set parameters with a button to submit them and a table where fetched data should be displayed. A classic "report" app. The problem was, these components were sister components, on the same level, with a common parent but I didn't understand how to react (lul) to an event in one component so that the other would automatically get updated.

Searches repeated suggested using Redux but I was convinced that route was completely overkill for what I was trying to achieve. I wasted a heck of a lot of time looking into Redux and other solutions. Shouldn't it be simple to link components up? Of course, I could use props from the parent to provide callbacks to update state in the App but that felt so wrong.

How can something that feels so wrong be so right?

Maybe there is an even better way but as it turns out, using Hooks (or state in a class) is really the easiest way. Store the state of the app in the top level component, pass a state update function into the form component and pass the state of the app as a props to the table component. Easy-peasy.

Another instance might be to create a global "state" class and you could dispatch updates from components but, honestly, you may as well use Redux at that point.

4. Lots of Classes

The documentation confused me. If I used a function to create a stateless component, and I used a class to create a stateful component, why not just make every component a class? Why did React prefer or encourage functions over classes?

Well, in 16.8 at least, you almost never need classes any more and using functions is actually a lot more straight forward, requiring fair less boilerplate. After many hours of use, I came to rewrite all my class based components to functions and I couldn't be happier. Viva function components!

5. No TypeScript

Not everyone may be a lover of TypeScript, and I'm certainly not an evangelist either, but I also appreciate what it's trying to do. I am a fan of strong, static typing and TypeScript is a good step in this direction.

When I started the app, I had some logic only portions written in TypeScript. Unfortunately, as development ground on, I ended up converting this code to pure Javascript because of a frustrating bug. I had concluded, wrongly, that something was being happening in the transpiling that was causing my issue.

As it turned out, this was not the case and I regret that I never converted the code back to TypeScript. Whoops.

6. No Object Destructering

Prior to this project, I wasn't familiar with destructuring in Javascript. After learning about it, I used array destructuring any time I used a React state Hook but, for some reason, I still didn't know about object destructuring. This in spite of the fact I came across examples that used it. I just didn't comprehend what I was seeing.

Now that I know about it, I wish I'd used it. Ah, well...

Things That Went Right

Here is a list of things, without explanation, that went right:

1. Yarn dependency management.
2. SASS.
3. Fetch API and Promises.
4. Webpack

That's all the wisdom I shall pass on.

Adieu!

Trying Ruby

I've never used Ruby. I have friends who swear by it and the Rails framework is incredibly well known. All I know about the language is that it's dynamically typed and incredibly flexible. A million ways to do any given task.

Now, I am not typically a fan of dynamically typed languages. I use PHP in my current job and while its "typeless-ness" can be useful and allow for quick prototyping of ideas, it is also fraught with many pitfalls that bite you in mysterious ways.

That being said, you can program in PHP, and I suspect Ruby, in a safe and responsible manner and I find myself interested to see what I can do.

Installation

To get started, I've gone to Ruby's main website. I normally work in Ubuntu but today I'm on my Windows machine. A quick skim over the page and that leads me to the Windows Ruby Installer page. I chose the default of installing Ruby with the devkit, which includes MSYS2 so gems that require C can be compiled. It seemed like a reasonable choice since I was new to the language.

After running the installer, I was given the choice to run an MSYS2 utility for further installation configuration. I chose the defaults for all options.

I also decided I wanted to install Ruby on Rails. The tutorial page for the framework warns you should get to know Ruby first, and I probably should, but I'm going to go ahead and install the framework anyway. I am already aware of the gems package manager for Ruby and used it to install Rails per the Getting Started document on the website. It also looks like using the Windows installer was a good choice as it includes sqlite3 by default which the Rails tutorial recommends. Excellent!

Getting Started

These tutorials can be a mixed bag. With programming languages, they tend to be slow, boring and uninformative; targeted at new programmers and not experienced ones. That said, I've opted to give the Ruby quick-start Ruby in Twenty Minutes a whirl to see what's up.

It encouraged me to start up the REPL for Ruby and do some basic tasks. Some notes:

• There's an exponential operator (**).
• Variables are dynamically declared, as expected, and require no special characters to denote a variable.
• Function declaration are reminiscent of Pascal where you declare the function start (def) and close the block with end. An interesting choice.
• Functions can be called with or without brackets. I'm not a fan of that and I think in my own code I would always include the brackets.
• String interpolation is nice if not slightly unconventional compared to other languages with the feature.
• I was glad to see there's some kind of notion of private/hidden class fields/properties.
• The ease of use of reflection is interesting.
• Accounting for variables types will likely require a lot of boiler plate. 
• Using `if __FILE__ == $0` reminds me of Python.

Initial Musings

There are certainly things I like and things I don't. I prefer statically typed and strongly typed languages. Ruby is dynamically typed and weakly typed. Like with PHP, it's just something you have to live with or work around.

It might be a bit premature to say after only spending a few minutes with the language, but it seems that like with most dynamically typed and weakly typed language you have to expend a fair amount of effort with boilerplate to determine what type any given variable is to take action on it. That said, the duck typing and using reflection does cut down on this a bit.

Coming bundled with an industry-wide accepted package manager is a refreshing change of pace. Package management is a tough topic and many languages still struggle with it. NodeJS, for example, has multiple competing tools. Go is only just starting to figure out its own management system. C and C++ don't really have one unless you could system tools like yum or apt.

Honestly, this is my second look at Ruby. In my first attempt, many years ago, I can quite confidently say I did not give it a fair chance and I didn't like. That's not Ruby's fault but my own. This second go around has left me with a different impression and I look forward to delving in further.

Going off the Rails

Where's the Rails portion I talked about? Well, that will come later.

Ta-ta for now!

Docker for Developers - An Introduction

While Docker has been around for a few years, it is fairly new to me. The organization I work for has only been using it for about a year but I've never had to delve to deep into using it.

In most cases, I run a command to start my containers and start working. It all just works out of box, as it was intended.

However, I recently had a need to setup a new project on my own and using our standard setup wouldn't work and I realized I really don't know much about Docker. So, this article about a few key parts of Docker that I struggled with in the hopes that others might learn something of benefit.

Images and Words

Dream Theater references aside, one key aspect of Docker I didn't understand was the different between an image and a container. I thought the words were interchangeable but, alas, they most certainly are not.

One metaphor I read about was that an image is to a class as a container is to an instance. I think that's fairly accurate. Saying that a container is just a running image isn't quite right because it doesn't capture the full picture. A container is not only a running image but also has additional configuration and may have a filesystem attached.

You can have two containers, two instances of an image, with different settings. This is important to note.

Monolithic vs Multiple Containers

Another question was: "why one would want to run multiple containers orchestrated to work together rather than a single, monolithic image."

At first, I thought the only real difference was the argument between statically linked binaries and dynamically linked ones. Why go through all the headache require the use of additional tools, like docker compose, just to achieve the same thing as can be done with a single image? Turns out, that's not the case.

Of course, reproducibility is a key part of developing an application. The ideal situation is having a development environment that perfectly reflects the production environment so that any bugs or issues can be replicated easily. A single docker image does that job just fine. In fact, in many organizations, web applications are deployed using containers so that the development environment and production environment are, quite literally, identical from a software point of view.

However, unlike a binary, where libraries are linked at compile time and become static dependencies, Docker containers behave more like plug and play peripherals.

A perfect example is adding a container to trap emails sent by an application. In a development environment, you can quickly spin up a container containing a tool like Mailhog to capture emails without affecting the rest of the environment.

Another advantage is that you can mock application services with a dummy container or even swap different versions of a dependent service without having to rebuild the image each time.

Imagine needing to swap between different versions of PHP. You could have 3 complete images of 500Mb each to cover PHP 5.6, 7.0 and 7.2 or, you could have 3 different PHP images that are 50 Mb each. Need to add a new PHP version to support? Easy! Just pull down another PHP image rather than be force to rebuild an entirely new image for your app.

Configuring Containers

I'm a bit embarrassed to admit it but I didn't understand how to configure a container. I assumed that you must have to build an image, instantiate a container, open an interactive terminal and perform the configuration. The idea felt all wrong but I didn't understand. The reason it felt wrong was because, well, it was wrong.

The problem was, I didn't know what Volumes were or how to use them. I also realized I didn't understand how containers were persistent or how to access files on a local computer. I assumed, again incorrectly, that you must have to copy the files into the image or container. Again, how wrong I was.

When starting an Nginx or Apache server, you need to be able to configure it. Whether it's core configuration or virtual host configuration, you need to supply this information to the web service. Did you create the file outside the image (yes) and then copy it in (no; well, not exactly)?

This is when the brilliance of Docker set in. When I understood how this worked.

You can configure the web server using the configuration files for the deployment server, whether they use containers or not, by attaching them to a container via volumes.

Volumes are exactly like file system mounting. Take a directory, or file, on your local system and mount it to a directory in the container's file system. In the case of Nginx, you might attach a "myapp/nginx.conf" to "/etc/nginx/nginx.conf" of the container. Then, when the container is started, it uses your local filesystem's file as if it were part of its own filesystem. A little like a chroot. It's brilliant!

This, of course, opens all kinds of wondrous doors! Need to import a database or assets for a web application? Use volumes to mount them where the live system would expect them to be! It's great!

This was a crucial part of the system I was missing and explains one aspect of why images are so reusable. You can have a single image whereby container A uses volumes for project A and container B uses volumes for project B. This comes at a major size savings.

Communication is Key
If configuration, volumes specifically, was the biggest hurdle I needed to get over, then the second largest would be intercommunication. How in the hell do different containers communicate?

If one image contains Apache, a second image contains PHP, a third image contains MySQL, how in the world do they talk to one another? The web server acts as a proxy to PHP (presume php-fpm) and thereby forwards paths to scripts to that container. PHP runs the code attached to the container in a volume but that code needs access to MySQL which is, again, in another container.

Within a single image or server, all these programs live in one space and so intercommunication happens (typically, but not always) through localhost. So, how in heck does it work with multiple images that have no knowledge of each other?

Enter networks and some Docker magic. To use multiple containers, you need to use docker compose, hereinafter referred to only as compose. Compose does two things:

1. Create a network to communicate by.
2. Sets the hosts file up in a container with the network names.

Now, with some simple setup, different image can communicate with each other as if they were all together on a single environment. Far out, solid and right on!

Alpine Skiing

Last, and I think a bit more minor, might be what these Alpine images are. Alpine is a small Linux distro specifically created for Docker. An Ubuntu image is quite large. Unless you have a specific need to perfectly imitate an Ubuntu system, Alpine is an excellent choice.

However, I have discovered a potential best of both words that I haven't tested yet: https://blog.ubuntu.com/2018/07/09/minimal-ubuntu-released.

Anyway, that's the gist of Alpine and its primary use.

Wrapping Up

That about covers the things I wanted to talk about. With any luck, a fellow newb will learn something and help them on their way to devops greatness!

So...ya. Bye!

C++ Text Template Engine - Part 1: Overview

This post will be the first in a series of undetermined length in my journey to write a text template engine in C++. No doubt, something like this already exists but those I happened to look at didn't really wet my whistle. No only that, there are aspects of C++ I want to explore and this would give me an excellent opportunity to do just that.

Motivation

I would classify myself as an intermediate programmer overall and a somewhat novice to C++. Certainly, I am drastically behind the times of modern C++ and I was never a strong programmer in the language to begin with. So, the first motivation is to help improve my C++ chops a bit. Why?

Well, that leads me to my second motivation: Not long after I started working for my current employer, I was tasked with maintaining our in-house systems and they were authored almost 20 years ago. The main software functions much as a modern web-server except that it runs like a CGI script. Each page load invokes a program to build the page and output it over HTTP. Some of these pages are ludicrously complex and are all done through print (std::cout/std::ostream) statements. It's horrific to maintain.

As part of the effort to make the code base more maintainable I desire to have a template engine so I can create templates and inject data into them. Due to budgets and our main focus of development, completely replacing the system isn't currently in the cards but we are frequently making changes to it. So, in the meantime, I need to maintain it.

Recently, I created a stop-gap by using regex to perform search and replace on a template document. It has drastically improved the situation but falls far short of the mark. I'd like to further improve the system.

General Overview

So, the plan is to document the entire process. Including the mistakes. I want to give a sort of free form exploration as I meander through the process and hopefully you, the reader, learn something along the way.

The idea here is that I want to demonstrate a little about what agile programming looks like and demonstrate how the thought process works when scoping and building out a moderately sized piece of software.

I've already gone on long enough, so it's time to find a place to start.

User Stories

A good place would be to create user stories. These are just statements that describes what it is the user/client (in this case, me) wants. I used to think them quite silly and pointless but I've recently come to appreciate them.

They give you a starting point to begin a discussion and they help keep you on track. Too often I've found myself over-engineering or falling short of client expectations and it usually has been a result of not fully understanding their needs. That's where the user story comes in. It serves as a jumping off point to understanding what they want.

Story 1: As a developer, I would like to create a document, to act as a template, that can have portions of it replaced dynamically to create a final document. This would ease creating pages for my application by avoiding the need to construct these pages with print statements.

Story 2: As a developer, I will need to be able to use any basic data type as well as some standard library containers.

Story 3: As a developer, I need a method to perform conditionals and loops. I have lists of users and accounts I will need to loop over and print.

Story 4: As a front end developer, I want the syntax should be easy to learn and familiar to me. I don't want to spend a lot of time learning another

Story 5: As a developer, it would be useful that any classes I currently have in my code base be compatible or could be integrated easily into this system.

Initial Thoughts

Perfect. So, what can we glean from that?

I need to have a source document, maybe a file on disk or a string. Reading files was not part of the user story so I think I should dismiss that for now. All I care about is the actual source and either a data stream or string will suffice as that lets me take data from any source and allow me to transform it. I worry a little that a data stream might be a pain to work with so I'm thinking that I'll want the data source to be passed in as a string. That more or less covers story number one.

In the second story, I need to be able to access basic data types like an integer, float or character string. I also need to be concerned with more complex data structures like vectors or maps. Classes will be addressed in the last user story so I'll forget about that for now. Off the top of my head, I'm thinking I'll need to have some kind of base value type with several derived classes for each basic type. I'm thinking ahead a bit here so I'll leave it at that for the moment.

User story number three asks about a conditional. That could be something like an if statement or switch statement. I want to keep things as simple as possible, I will likely just use an if statement with and else clause. This story could use some fleshing out.

For loops, I feel am faced with the classic for and while statements. In the interest of simplicity, I will only pick one. Since I will likely be passing data structures into the template engine a for loop with an each or range behavior likely makes the most sense. In the Go standard library, they eschew the for and while keywords for the range keyword and I kind of like that but that may be problematic for user story four. Perhaps a keyword of foreach might be apropos.

A familiar syntax for front end developers would be one like Mustache, Django or Twig. I think if I adopt something similar, that will ease any mind-share. While some systems use different tokens for commands and identifiers I think I can safely use one and just enforce some simple naming rules common to almost every programming language.

Finally, I need to be able to support existing classes. To my knowledge, C++ does not have reflection and converting large classes of data into maps would not be fun or efficient. I'm thinking that I can maybe create an interface by which my existing classes could fulfill in order to make them compatible with this system. I'm going to have to think more on it but I need more information first.

The Stage is Set

That completes the initial breakdown and user stories. The next phase will be discovery and I'll cover in the next article.

Writing Interfaces in C++

After my last article, I got to thinking about utilizing interfaces in C++. I am not an expert in C++ by any means and most of the code I have to work on is both antiquated (C++ 98) and poorly written (even for its time). Most of my time is spent writing PHP and Go, so using interfaces is quite common.

Interfaces, abstract classes in C++, are not used at all in the code bases I work on regularly. It got me to thinking: "could Go or PHP style interfaces be done in C++?"

Virtual Recap

A virtual function is one that is defined in a class that is intended to be redefined by derived classes. When a base class calls a virtual function, the derived version (if it exists) is called.

A key point to make is that virtual functions do NOT need to be defined by a derived class.

To see an example, check out my previous post.

Pure Virtual Goodness

In PHP, you would call these abstract functions. In C++, they are called pure virtual functions. Perhaps a better thing to call them would be purely virtual. I feel like that term makes it more clear to the purpose of this feature. Declare that a function exists as a method on a class but leave the definition for later. It exists purely in a virtual sense.

Abstract functions are those that are defined in a lower, base class and implemented by a higher, derived class. Class A provides a prototype for a function but class B, which extends class A, actually implements it.

A pure virtual function is a contract. A class which extends a class with an abstract, pure virtual function must implement it before it can be instantiated. This guarantees that the function will exist somewhere in the class hierarchy. Otherwise, it would be no different than calling a function that has not been defined.

A pure virtual function looks like this:

virtual void Read(std::string& s) = 0;

This declares a virtual function Read. The notation of assigning zero to the function designates that the function is purely virtual.

Abstract classes as Interfaces

A class with only pure virtual functions is considered to be an abstract class. Since C++ does not have interfaces in the same way other languages do, abstract classes can fulfill the same role.

class Reader {
public:
    virtual void Read(std::string& s) = 0;};

The above class is fully abstract. Any classes that extend it will need to implement the method Read.

Or does it?

Compound Interfaces

I am a proponent of small, simple interfaces that can be combined to create ever more complex ones. This is the Interface segregation principle from SOLID design principles. The problem lies in the fact that C++ requires derived classes implement pure virtual functions. Thankfully, there is a way to get around that restriction.

Extending a class with the virtual keyword tells the compiler that the derived class will not be implementing the pure virtual function either, making it abstract as well. This allows multiple interfaces to be combined.

class Reader {
public:
    virtual void Read(std::string& s) = 0;};
class Writer {
public:
    virtual void Write(const std::string& s) = 0;};
class ReadWriter : public virtual Reader, public virtual Writer {};

Here, the interfaces Reader and Writer get combined into a third abstract class ReadWriter. It too, could add further pure virtual functions if desired.

Implementing an Interface

Implementing the interface is the same as any deriving any class. So, to tie everything together, he's a complete example:

#include 
#include 

class Reader {
public:
    virtual void Read(std::string& s) = 0;};
class Writer {
public:
    virtual void Write(const std::string& s) = 0;};
class ReadWriter : public virtual Reader, public virtual Writer {};
class SomeClass: public ReadWriter {
    std::string buf;public:
    void Read(std::string& s) override { s = buf; }
    void Write(const std::string& s) override { buf = s; }
};
void readAndWrite(ReadWriter& rw) {
    rw.Write("Hello");    std::string buf;    rw.Read(buf);    std::cout << buf << std::endl;}

int main() {
    SomeClass c;    readAndWrite(c);    return 0;}

The on caveat is that abstract classes must be some kind of pointer, either as a standard pointer or a reference pointer. This requirement makes sense since it can not be instantiated.

Summary

Using pure virtual functions and virtual classes, it is indeed possible to describe behaviour as would be done in other languages with interfaces. Pure virtual functions have a further use in multiple inheritance. To learn more, check out this StackOverflow answer.

Happy programming!

Understanding Virtual Functions in C++

Until I had to explain it to someone, I never appreciated how confusing the virtual keyword can be. After all, in certain situations there seems to be no functional difference between virtual and non-virtual functions except that your IDE or editor might complain at you.

This article is aimed at programmers who are new to C++ or an initiate to programming in general.

Virtual

The term conjures up a vision of something that is non-tangible. That is, something that doesn't exist in the physical world. In C++, this meaning is extended to describe functions which may not exist. By that, we mean that it may not be defined.

A virtual function, therefore, is a function which may not be real. While a class may call a function it defines, a virtual function could also exist somewhere else in it's object hierarchy. The class doesn't know what function it's actually calling until it is compiled or possibly until runtime.

Essentially, the virtual keyword signals to a developer that the function is intended to be able to be implemented or overridden in a derived class.

The Base Class (Non-Virtual)

To demonstrate, start by creating a simple base class with two public functions: Foo() and Bar(). Inside the Foo() function, call Bar(). It may be helpful to add some output so you know which function is being called.

Something like this:

class Base {
public:
    void Foo() { cout << "Base::Foo" << endl; Bar(); }
    void Bar() { cout << "Base::Bar" << endl; };};

In a main function, instantiate a new Base class and call Foo(). You should get ouput similar to this:

Base::Foo
Base::Bar

This is pretty standard fair and works as expected.

The Derived Class

Next, create a derived function that inherits from Base. In it, create a function with the same name and signature as Bar(). If printing out a statement, make sure you update it to report the new class name.

class Derived : public Base {
public:
    void Bar() { cout << "Derived::Bar" << endl; }
};

Call Foo() on this new class and you'll see that you get the same output. The function Bar() is only called on the Base class.

The Base Class (Virtual)

Add the virtual keyword to the function Bar() in the Base class then try running the code again. The code should look like this:

class Base {
public:
    void Foo() { cout << "Base::Foo" << endl; Bar(); }
    virtual void Bar() { cout << "Base::Bar" << endl; };};

You should get output like this:

Base::Foo
Derived::Bar

This time, Bar() was called in the Derived class instead of the Base class. Why?

Virtual Explained

The virtual keyword signaled a change in visibility. A non-virtual function tells the compiler not to bother looking for another function with the same name, just to look for the function in the class where it is called. A virtual function tells the compiler to see if a function with a matching signature exists in any derived classes first, before calling the one in the same class.

Summary

Virtual functions are those whom are intended to exist higher up in the class hierarchy in derived classes and should be called if they are defined.

Domains, IPs, Ports and Virtual Hosts - How it All Fits Together

Developing web applications might seem fairly simple and writing a basic web page is. If all you are doing is writing a static web page then you can freely go ahead and start writing code. What if you want to develop a dynamic web page or mimic a production server?

Servers

Any single computer on any network, whether a local network or the Internet, is associated with an address. An address may be assigned to only one computer at any given time but multiple addresses could potentially point to the same server. Without going into the semantics of addresses, two common ones that will be encountered are:

• 192.168.X.X - These are local, intranet addresses. Each computer on a home network would have an address in this range.
• 127.0.0.1 - This is a special address called the loop-back address. It is used by a server in order to allow it to contact itself. Hence, loop back.

An IP (Internet Protocol) address is a 32 bit number that is usually displayed as a series of numbers separated by dots, as seen above.

There are a lot of complexities to how addressing works, including internal (intranet) to external network addresses and how they interact.

Ports

To serve files or data, a program has to be setup to listen for new connections. However, if a computer may only have a single address then how does a server know how to associate any given connection with a program?

Ports allow connections routed to routed to the correct program. Ports for a few common web services include:

• 20 and 21 - File Server, FTP (File Transfer Protocol)
• 25 - Mail Server, SMTP (Simple Mail Transfer Protocol)
• 80 - Web Server, HTTP (Hyper-Text Transfer Protocol)
• 443 - Web Server, HTTPS (Hyper-Text Transfer Protocol over SSL/TLS)

A list of reserved ports can be found on Wikipedia.

Only one program may listen to any one port but for web servers, at least, there may exist a way to get around this limitation.

Also of note, is that evne though many clients/services don't explicitly require you to add a port it is always there. If no port is supplied, web browsers will always use port 80. When testing web applications from a local system, it is common to use port 8080 or 8880 like so: www.example.com:8080.

Domains

A domain is usually a human readable string of characters that act as an alias to an IP address. Any single domain may be associated with a single IP address. Domains on the Internet must be registered with an authority called a Domain Registrar. You supply the registrar with the address to your host and they will associate it with the name of your domain.

A top level domain (TLD) are those such as: com, gov, net and dev, to name just a few. There are many others. As long as they are used on your local machine and a client, like a web browser, never accesses a name server to resolve it (as it would for domains on the internet) then there are no real restrictions.

Unfortunately, some web browsers, Chrome and Firefox for example, can complicate matters and it is generally recommended to steer away from common TLDs.

Multiple domains may point to the same address.

Sub Domains

Any other domains are sub-domains. Without going into detail, when someone refers to a subdomain they usually refer to the prefix domain, namely www or mobile. Moving from right to left, the least significant domain is on the right.

• www.example.com - root, com (top level domain), example, www (least significant domain)

Virtual Hosts

A web server can host many different websites. If a web server is listening on port 80 but multiple different domains point to the same address, how would the web server know which site to route the request to?

Virtual hosts associate domain names to locations on the server. Even though both www.example.com and mobile.example.com might resolve to 192.168.0.1:80 the web server is given the value of the domain. The web server takes that domain and, provided an entry for it exists in its configuration, routes communication for each domain to the correct web directory.

Hosts File

When a client like a web server is supplied a domain name, normally it contacts a service called DNS (Domain Name Service) to resolve the address the name is associated with. When doing local development, this isn't desirable as an internal web server is usually not publicly accessible.

To overcome this limitation, most operating systems have a file that is used to define local domain names. If a domain exists in this list, then the associated IP address is used without ever connecting to a DNS server to resolve the name.

In this capacity, you could even do things like route www.google.com back at your own web server.

On most GNU/Linux systems you'll find the hosts file under the /etc directory. You would add an entry like:

www.example.com 127.0.0.1

When putting that address into your browser, the system will first check for a matching entry in /etc/hosts and, if it find one, provide the given address. In this case, the loop back address which sends the connection to the corresponding port on your system. Remember, a port is always required even though they often don't need to be supplied explicitly.

Bringing It All Together

Armed with this knowledge, a developer can setup a testing environment that mimics a production server.

If developing web sites and are in need of setting up a full stack setup for each one (by that I mean, a web server, database server and any other accoutrements) then a process similar to this could be followed:

1. Install/Copy any files from either an existing production or a framework.
2. Add a VHost entry for your web server.
3. Add a Domain entry to your hosts file.
4. Create or Import a database.
5. Work like crazy.

With such a setup, a developer can connect to a local web server as if it were a remote server.


Of course, there are a lot more things that can be done to provide even better environments. Here are a few closing thoughts to whet your appetite.

Docker

You can use Docker to mimic a production server almost exactly or, in some cases, run docker on both production and in development to ensure 100% compatibility. This can be very useful for addressing issues with software packages and libraries being incompatible versions.

Reverse Proxy

Some environments, like PHP for instance, may require using a technique called a reverse proxy. Go development can also benefit one in instances where you need both a Go application and a web server like nginx or Apache running, too. This involves having one server accept an incoming request then forwarding the request to the same address but on a different port.

Been a While

It's been a long while since my last post. That's not to say I have been idle. In fact, it's quite the opposite.

After working 10 years first as a sales representative and another 10 years as a credit professional I decided it was time to finally pursue my true passion: programming. I have been programming since I was in my early teens and it has always been a passion of mine.

At one point I thought maybe I'd like to write games but I soon became disillusioned by the video game industry. I became confused as what I wanted to do and one thing leading to another, my career path headed in another direction.

I was quickly approaching a time in my life where either I commit to my path or pursue my passions. It was time to make a choice. However, being married with kids, making a life change comes with a tremendous amount of pressure and fear. If I fail or make the wrong choice, it's not just me who suffers the consequences.

The opportunity that presented itself also required us moving roughly 300km away. We had been wanting to move back to the Okanagan Valley for quite some time and now we could do it. My children are still young enough that lasting friendships haven't formed yet but still it would be hard on them.

So, I accepted a position at Acro Media where we primarily build custom Drupal e-commerce websites. I, specifically, work on supporting existing sites and maintaining our in-house infrastructure. Luckily, this means I don't exclusively program in PHP, Javascript, HTML and CSS. I also get to work in C++ and Go.

I wasn't sure how I would feel about working in PHP for much of my day. I am not a fan of the language for many reasons but I've been pleasantly surprised to discover that they joy of programming has thus-far eclipsed working in a language I dislike. I think this is mainly due to the fact that my day is quite varied. My job is about solving problems. It's not so much what language I'm using.

So where does that leave the blog?

Well, I hope that it means that'll be writing more again. Compilers are still a passion and I'm working on another project. It's rather ambitious so we'll see what happens there.

On that note, I think I'll call it a day. Until next time!

The Accumulator Register

At all curious about the accumulator register? I was. Here's a little summation of what I discovered.

Overview

In modern, general purpose CPUs, the accumulator register has lost some of its meaning and much of its use has been relegated to convention rather than necessity.

I got to wondering why the ax/eax/rax register was called the accumulator and why it was used so much since it seemed that any of the general purpose registers would do. Does it just come down to convention or is there more to the story?

All the MIPS documentation I’ve read to date indicates the only register considered the accumulator is a hidden, double-word register accessed via the MFHI and MFLO instructions. This speical register is used when doing multiplication and division.

In developing my own virtual machine, and delving into electrical engineering, I got to be more intimately familiar with this happy little register.

A Little History

If my sources are correct, the first commercially available CPU from Intel was the 4004. It had a single register that was implictly used in the vast majority of its instructions. You can see for yourself on the data sheet (PDF). Can you guess which one?

The 4004 was designed in such a way that the output of the ALU fed directly into this single register to accumulate data. Operations that expect two operands always implicitly use the accumulator as the first operand.

The ADD instruction, for example, takes a single register as an operand. It performs addition on the value stored in the accumulator plus the given register and then stores the result back into the accumulator.

To my knowledge, the 4004 did not pioneer the use of Accumulator but it serves as a nice example.

Example

The design seems simple but consider how calculators or adding machines work. How about an example?

Assume the calculator has just been turned on or a clear instruction has been set so that the accumulator has a value of zero.

The user types in a value then hits the addition key. The accumulator and the value on screen get added together. There’s no operands just a simple ADD instruction. ACC += DISPLAY.

Arithmetic instructions would not need explicit operands at this level of design. It makes the accumulator a very important facet in the design of these machines.

In modern implementations, even if a CPU can use any general purpose register to execute a particular instruction, it may be optimized to perform certain operations in fewer clock cycles when using the accumulator. You’ll need to read documentation to find out.

Legacy

The result is that the importance of this register is historic and thereby remains important to standard convention. We probably no longer need to do it this way but it does have a logical, and historic, significance.

Happy programming!

Compiler 3 Part 8 - Finale

This will end up being the longest, and final, post in the series. So let's tuck in!

Testing and Everything Else

The scanner and most of the infrastructure stayed exactly as it was.

The parser and AST did see some changes. The parser, in particular, saw several bug fixes and simplification. Some of the error checking that had been done during parsing has been passed on to the semantic analyzer. Similarly, anything related to type checking could be removed.

Also, the AST-related scoping code could be simplified and confined to the parser since it is discarded during semantic analysis.

Detecting “true” and “false” was added for boolean the type.

With those changes out of the way, I can say that if there is one area that Calc has truly lacked, it was in testing.

A compiler project should, in my opinion at least, be concerned with correctness. Correctness and consistency. The IR was one step in shoring up Calc in this area and testing is the other. Calc has always incorporated at least a little testing but it has never been as comprehensive as it should be.

Go provides great testing facilities right out of the box. The cover tool has also proven helpful in sussing out corner cases that need testing.

I have, I think, made large strides have been made in making Calc a much more stable compiler.

That isn’t to say that it’s perfect. I’m sure within 5 minutes of playing with it someone will discover some bugs. Hopefully, those than find them will take the time to submit an issue so I can create tests and fix them.

This is the value of having an open source project.

The Future...

Calc has continued to evolve along with my own skills and I hope that I’ve helped someone, in some way, to further their own growth. Whether it be teaching someone how to, or how NOT to code, the end result remains the same. Someone, somewhere, has benefitted from my efforts.

Now that I’ve had a chance to regroup from the herculean effort of writing and implementing the Calc2 spec and teaching series, I feel like I can continue to produce more content.

I do hope to continue talking about compiler construction. In the future, I hope that by using a smaller format, like this one, that I can produce more posts more quickly. I can certainly say that this third series has been much more enjoyable to write!

I have also been working on another side project to create a virtual machine with a complete software stack. It has an assembler, linker and virtual machine running it’s own bytecode. If there is interest, I’ll write some blog posts about it. You can find the project here.

I have concluded both previous series talking about future plans and I shall do no less today. In series two I had a list of things I wanted to implement and change. Let’s go through the list:

• growable stack - well, this is null and void. Calc no longer manages it’s own stack.
• simple garbage collector - I no longer plan on implementing a garbage collector myself. See below.
• loops - not implemented
• generated code optimizations - done! 
• library/packages - not implemented
• assembly - null and void. I don’t plan on generating assembly.
• objects/structures - not implemented

Only one of the seven features I wanted to implement was actually completed. Of the remaining six features, three of them no longer make sense. The groundwork for other three has been laid down. In particular, loops and structures only await language design since implementation should be almost trivial.

Incorporating the remaining three features, I now present you with an updated list:

• #line directives for source-debug mapping - I think this is a very reasonable and easily obtainable goal. It will make debugging Calc with the GNU debugger easier.
• logical && and || operators - are easily added with minimal work
• importing libraries - thanks to the x/debug/elf package, importing libraries into code may not be unreasonable to achieve in the near future. The scope of adding imports is likely a series in of itself.
• garbage collection - Calc 2.x does not need garbage collection. It does not have pointers and it does no memory allocation. However, when it does, I will probably use the Beohm garbage collector rather than attempting to spin my own. Additional info can be found on Wikipedia. This feature will be far in the future so don't expect it any time soon.
• structs, loops and arrays - with the new code generator, implementing structs and arrays on the back end ought to be trivial. Work on this has actually already begun and you can view some of the changes here. Sorry, the specification is not published publicly right now.
• stack tracing - there isn’t much nothing holding me back from implementing stack traces on crashes now that the new code generator is in place. Time and effort are all that remains.

And that, as they say, is that! Thank you for taking the time to read through this series! At some point I’d like to put everything into a PDF but that would be a large task since large parts of both series would need to be re-written entirely. In fact, I might even want to start again from scratch.


Until next time!

Compiler 3 Part 7 - Code Generation

Here we are at the final step.

As mentioned in the introduction, I cut out a massive amount of code by offloading much of the work on the IR.

So here is what has changed:

I introduced a small function to map Calc types to C types. This could probably exist in ir.Types, too, but I chose to keep it coupled with the code generator since that’s the only code that uses it.

Any object with an ID has its original named stripped and is replaced by a generic variable name. These names start with an underscore and the lower-case letter “v” followed by the ID of the object.

Each binary operation is assigned to a new variable (a reason for why C99 is required). Don’t worry about this being wasteful. Even if you chose to output a chain of infix arithmetic (1 + 2 + … + N) the underlying assembly instructions usually take no more than two operands anyway. This is why using the above method works so well since it more closely matches the machine code.

Note: I've spent a lot of time comparing the assembly generated from C to see how things work. This is a fun (am I sick?) and interesting exercise to try yourself.
Even if you didn't follow along with the last series, I encourage you to view the previous binary code generation function. 58 lines of confusing mess now pared down to a simple 5 line function. Perhaps more importantly, the generated code is easier to read and follow even though it’s not really intended for visual parsing.

Another important change is using C-style function calling and function declarations. This was made possible by the new IR and type system. With every object being assigned a valid type, we can easily create proper C function prototypes and definitions.

By utilizing the SSA-like code generation and the new IR it also becomes trivial to use C-style calling convention. Types have already been checked, the number of arguments verified, and sub-expressions have been assigned ID’s. Therefore, only raw values and object IDs are passed into the function.

All in all, fairly simple.

Compiler 3 Part 6 - Tagging

This step is crucial to Calc’s code generator but may not exist at all in other compilers. Regardless, it makes code generation for Calc dead simple. Before I can get into the process, you do need a little background information.

In the introduction I mentioned something about 3AC and SSA. Make sure you check out the articles but the cut and dry is thus:

In SSA, each calculation is given a unique ID. These ID’s replace variable names and other objects.

So, what does this mean to us?

Consider the following two example infix arithmetic operations:

1. a + b + c + d 
2. a * (b + c) / d 

These two operations could be translated into something like the following:

Example 1: a + b + c + d

r1 := a + b
r2: := r1 + c
r3 := r2 + d

Example 1: a * (b + c) / d
r1 := b + c
r2 := a * r1
r3 := r2 / d

As you can see, the result of each binary operation is assigned to a new, unique variable. While verbose, it is much more akin to assembly and easy for C compilers to optimize.

It also ensures that calculations are done in the correct order and removes the necessity of pushing and popping operands to the stack.

Armed with knowledge, we can now get on with what I call tagging.

Tagging is the process by which we attach these unique identifiers to each operation. Variables, parameters, binary operations and unary operations all need to be tagged.

Oddly, perhaps, even if statements need to get tagged and you may wonder why that is. Well, if statements in Calc are not statements. They’re expressions. Like if expressions in functional languages, if expressions in Calc return a value.

As part of escape analysis and type checking, the value of any branch in an if expression must be checked and tagged since it’s result may be used elsewhere in the code.

As you can see, Folding constants can potentially save us time by reducing the number of operations needing an ID.

We can traverse the tree in any manner we chose provided that every ID is unique.

One more stop left to go. Code generation!

Compiler 3 Part 5 - Constant Folding

The only optimization included in series 2 was constant folding. Unlike in the previous series, we’re now much better equipped to handle this optimization.

I feel that I should point out that most C compilers do this step, too. I was somewhat reluctant to keep it in at first. However, my reasoning for keeping it is two-fold: one, I think learning a bit about optimizations on an IR is a worthy lesson; and two, it can help make the next step a little quicker.

Once again, the IR comes to the rescue! In the first step of transforming the AST into the IR we created constants. These were objects representing actual values. This crucial step makes it much easier on us now.

Unary and Binary operations are ideal candidates for folding. We have already done the work of verifying types and correctness so we can ignore all that now. When we check if a value is a certain type we can be confident that information is correct.

Looking at the code you can see that it’s pretty simple. Traverse the tree depth first.  If both operands of a binary object, or the single operand of a unary object, are values of the correct type we can fold them together. We then return the result as a new constant value and replace the previous object with it.

Moving back up the tree we repeat the process until we exhaust all the foldable values.

You can see the value in converting basic literals into constants when building the initial IR tree. It makes folding constants together much, much simpler.

Onward and upward!

Compiler 3 Part 4 - Type Checking

Whether you chose to produce assembly or an intermediate language, the job of the IR is to take us ever closer to generating actual output. We need to do the crucial step of type checking. Since Calc’s type system is strong and static it is necessary to ensure type safety before moving on.

Each struct in the IR tree has an underlying object. This object provides a Pos() function that maps the remaining constructs back into the original source code. This will be useful to us when we are doing type checking so we can provide well formed error messages.

We must traverse the tree and verify that expected types match. Not all objects currently have a type, like variable declarations using type inference. This means that we don’t actually know the type of the variable when parsing the source language since the type isn't explicitly declared but inferred from the object being assigned to the variable. 

Similarly, binary expressions do not have a defined type when parsing. Your language may allow adding (concatenating) strings, in which case an add binary operation on strings would have the type string. In the case of Calc, logical comparison operations make a binary expression type bool. Don't forget that most languages have different integer types so your binary expressions will need to take on and verify these types.

This requires we traverse the tree depth first to identify the types of certain structures prior to verifying their correctness.

In addition to verifying type correctness, the type-checker also does some additional tests…

• checks that function parameters match the number of arguments in function calls;
• verifies that an identifier matches the right kind (variable vs. function names);
• checks for undeclared variable and function names.

While not true of the current version of Calc, many languages have varying bit sizes of integer types. Somewhere during this process you would will need to verify if a particular value fits within the bounds of a particular type. If the source has a value greater than 255 being stored into an unsigned 8 bit type, this is a problem. When and how you chose to handle this error is up to you.

Type checking may also involve a level of escape analysis. You need to verify that any value being returned anywhere in the tree conforms to the parent type. In imperative languages, this means looking for statements like “return”. For functional languages, you need to search for terminal points like the "then" and "else" clauses in an "if" expression.

Once we are sure that everything is type safe and correct, we can move on.

Compiler 3 Part 3 - Intermediate Representation

After presenting Calc2 I came to discover that many parts of the compiler were quite fragile. The type checking was particularly bad, occasionally reporting type errors (false positives) when run over the exact same source code multiple times. 

That’s a serious problem.

I had assumed that using an intermediate representation was strictly for doing optimizations. While they certainly help with, and may be crucial for, optimization they can be useful for much, much more.

The IR used by Calc is nothing special. In fact, it looks very much like the AST and the objects it uses share many of the same names. So what gives?

If you look more closely, there are some crucial differences. For one, each object is represented a little differently. Second, there is much more in common between each object. In fact, there is a struct embedded into every one: object.

This struct gives me access to almost everything needed to perform various tasks, including: type checking, error reporting and code generation.

While the AST and IR may both be trees, and represent much of the same data, they represent two very distinct things. The AST, as the name implies, represents the actual syntax of the language. Using it, you can somewhat faithfully recreate the original source code. The IR, on the other hand, sits much closer to code generation and thereby represents the code we wish to output.

I look at the IR like this: it acts as a bridge between (intermediate) the syntax and code (representation) generation.

The first step is transforming the AST into IR. This is done with a simple tree walking algorithm, converting each node into the new form. Typically, the compiler starts with MakePackage. From there, you can follow the calls along to see how the tree is built.

There are a few things worth noting.

First, notice that the parameters of functions get separated into objects stored in a function's scope and only a string slice of parameter names is stored with the declaration. This will become important in the next few steps.

Second, we can also begin a simple optimization of converting literals, like number and boolean strings, into actual values called constants. This saves us the step of doing these conversions later on and provides an opportunity to do constant folding in a much more efficient manner.

Once we’re done converting the AST into the IR tree, the real work can begin. Stay tuned!

Compiler 3 Part 2 - C Runtime and Code Generation

Before diving into the real meat and potatoes of the changes that led to the runtime being (temporarily) removed, I thought I should discuss the motivations.

The runtime irked me greatly because I felt that I did not leverage the power of C nor the C compilers. There were many, many improvements needing to be made and as I began working on them I came to a realization.

The runtime, at least in its current form, is not very useful.

Unlike in a language like Go where the runtime serves a very real purpose (managing goroutines, the garbage collector, etc) the Calc C runtime merely tried to mimic what it’s like to program in assembly.

Of course, there are reasons to re-add a runtime further down the road. Runtime checks, like bounds checking, are one such reason. Stack tracing is another wonderful motivation to implement a runtime. Built-in functions are also common and useful.

I still maintain this was a worthy exercise, long-term it just wasn't viable. The more I looked at the runtime the more I knew I could do better.

The largest hurdle was removing the stack. How was I to handle the call stack? As it turns out, it’s not too hard to use C-style function calls but it did require some changes that the series 2 code base just wasn't equipped to handle elegantly.

I also needed to push and pop data on and off the stack. So how could I rid myself of it?

I have been researching on how to use C as an intermediate representation for many months. Generating C function calls and declarations were proved difficult with the Calc2 compiler and that was one of the motivations for using an assembly-like approach.

Ultimately, it became more and more clear that I was missing a step that would make my life a lot easier.

That secret, if you will, was intermediate representation.

After much research I ran into some potential solutions: three address code and static single assignment. I have been aware of these languages and others like them (Gimple and RTL) for a while.

Using them seemed complicated and I didn't think they were worth the effort of learning about. I wasn't writing an optimizing compiler so what advantages did they provide that I would need.

While not using any of these representations directly I do take advantage of what they provide. More on that next post.

Compiler 3 Part 1 - Introduction

I return to you once again with continued improvements and changes to the Calc programming language compiler. In the intervening months (almost a year!) since I completed the last series I have been busy with many other projects. However, I keep coming back to the compiler and in the last few months have been busy tinkering away.

I have taken a different route in this next iteration of the series and you could consider it a partial rewrite of the previous one. There are several knobbly sections in series two that I wasn't particularly happy with and in this third series I hope to address most, if not all, of them.

I also intend to make this third, and any potential future series, much shorter and narrower in scope. Please feel free to ask me questions as I will be glossing over some of the finer points.

So what has changed?

Well, the language itself changed almost not at all. The only additional language feature is the addition of the boolean type. So, nothing too exciting there.

What did change was the entire back-end.

In series two, the code generator operated directly on the AST. It proved to be cumbersome and error prone. The compiler now uses an intermediate representation for the code generator that reduced the LOC from 544 to 255. Less than half the LOC while producing better, more easily optimized code.

That does not even include the removal of the runtime library!

Yes, the entire runtime has been removed which included many hundreds of lines of code being deleted. The Calc language went on a diet!

The net gain is a much better compiler and I want to share those efforts with you all.

Stay tuned!

Series 1 - Novice Compiler Design

Series 2 - Apprentice Compiler Design