C++ Text Template Engine - Part 1: Overview

This post will be the first in a series of undetermined length in my journey to write a text template engine in C++. No doubt, something like this already exists but those I happened to look at didn't really wet my whistle. No only that, there are aspects of C++ I want to explore and this would give me an excellent opportunity to do just that.

Motivation

I would classify myself as an intermediate programmer overall and a somewhat novice to C++. Certainly, I am drastically behind the times of modern C++ and I was never a strong programmer in the language to begin with. So, the first motivation is to help improve my C++ chops a bit. Why?

Well, that leads me to my second motivation: Not long after I started working for my current employer, I was tasked with maintaining our in-house systems and they were authored almost 20 years ago. The main software functions much as a modern web-server except that it runs like a CGI script. Each page load invokes a program to build the page and output it over HTTP. Some of these pages are ludicrously complex and are all done through print (std::cout/std::ostream) statements. It's horrific to maintain.

As part of the effort to make the code base more maintainable I desire to have a template engine so I can create templates and inject data into them. Due to budgets and our main focus of development, completely replacing the system isn't currently in the cards but we are frequently making changes to it. So, in the meantime, I need to maintain it.

Recently, I created a stop-gap by using regex to perform search and replace on a template document. It has drastically improved the situation but falls far short of the mark. I'd like to further improve the system.

General Overview

So, the plan is to document the entire process. Including the mistakes. I want to give a sort of free form exploration as I meander through the process and hopefully you, the reader, learn something along the way.

The idea here is that I want to demonstrate a little about what agile programming looks like and demonstrate how the thought process works when scoping and building out a moderately sized piece of software.

I've already gone on long enough, so it's time to find a place to start.

User Stories

A good place would be to create user stories. These are just statements that describes what it is the user/client (in this case, me) wants. I used to think them quite silly and pointless but I've recently come to appreciate them.

They give you a starting point to begin a discussion and they help keep you on track. Too often I've found myself over-engineering or falling short of client expectations and it usually has been a result of not fully understanding their needs. That's where the user story comes in. It serves as a jumping off point to understanding what they want.

Story 1: As a developer, I would like to create a document, to act as a template, that can have portions of it replaced dynamically to create a final document. This would ease creating pages for my application by avoiding the need to construct these pages with print statements.

Story 2: As a developer, I will need to be able to use any basic data type as well as some standard library containers.

Story 3: As a developer, I need a method to perform conditionals and loops. I have lists of users and accounts I will need to loop over and print.

Story 4: As a front end developer, I want the syntax should be easy to learn and familiar to me. I don't want to spend a lot of time learning another

Story 5: As a developer, it would be useful that any classes I currently have in my code base be compatible or could be integrated easily into this system.

Initial Thoughts

Perfect. So, what can we glean from that?

I need to have a source document, maybe a file on disk or a string. Reading files was not part of the user story so I think I should dismiss that for now. All I care about is the actual source and either a data stream or string will suffice as that lets me take data from any source and allow me to transform it. I worry a little that a data stream might be a pain to work with so I'm thinking that I'll want the data source to be passed in as a string. That more or less covers story number one.

In the second story, I need to be able to access basic data types like an integer, float or character string. I also need to be concerned with more complex data structures like vectors or maps. Classes will be addressed in the last user story so I'll forget about that for now. Off the top of my head, I'm thinking I'll need to have some kind of base value type with several derived classes for each basic type. I'm thinking ahead a bit here so I'll leave it at that for the moment.

User story number three asks about a conditional. That could be something like an if statement or switch statement. I want to keep things as simple as possible, I will likely just use an if statement with and else clause. This story could use some fleshing out.

For loops, I feel am faced with the classic for and while statements. In the interest of simplicity, I will only pick one. Since I will likely be passing data structures into the template engine a for loop with an each or range behavior likely makes the most sense. In the Go standard library, they eschew the for and while keywords for the range keyword and I kind of like that but that may be problematic for user story four. Perhaps a keyword of foreach might be apropos.

A familiar syntax for front end developers would be one like Mustache, Django or Twig. I think if I adopt something similar, that will ease any mind-share. While some systems use different tokens for commands and identifiers I think I can safely use one and just enforce some simple naming rules common to almost every programming language.

Finally, I need to be able to support existing classes. To my knowledge, C++ does not have reflection and converting large classes of data into maps would not be fun or efficient. I'm thinking that I can maybe create an interface by which my existing classes could fulfill in order to make them compatible with this system. I'm going to have to think more on it but I need more information first.

The Stage is Set

That completes the initial breakdown and user stories. The next phase will be discovery and I'll cover in the next article.