Upcoming Changes in Goncurses 0.2

I wrote and maintain an ncurses wrapper library for the Go programming language. Much like this blog, it went through a long dormancy with a few minor spurts of action while I was wrapped up in my regular job and my family. Back in June I finally tagged version 0.1.

Since that time I've implemented a fair number of changes and, as I warned at the time, a couple are somewhat breaking. The library is quite young as far as implementation goes and I want to avoid any API breaking changes if I can but I felt these were necessary.

Here is a list of changes coming up in 0.2:

Breaking Changes

• As I announced in the 0.1 release, the Print statement has now been broken up into separate Print and MovePrint functions to be more ncurses-like and each has been further separated to be more Go-like: Print,  Println, Printf, MovePrint, MovePrintln, and MovePrintf. A regex replacement ought to fix most instances but some manual editing will probably be required.
• The Character type has been changed to the shorter Char type. A simple gofmt or sed regex will easily fix any existing code.
• Several functions now use the Char type like: AttrOn, AttrOff, Box, Border,ColorPair,  etc
• Color related functions now take an int16 rather than an int (32bit) type. This is to mirror the C short type to protect against any type conversion errors.

Non-Breaking Changes

• Implemented the slk soft-keys functions.
• Added screen functions (newterm, set_term, etc).
• Implemented a StdScr function which returns the underlying stdscr window object. This was mainly a requirement to using the screen functions because there was no other way to get the stdscr but it may be useful for other situations.
• Implemented AttrSet, Standend and Standout text attribute functions.
• Implemented Timeout for input on Windows

Version Control for Go

When I first heard of version control, the most popular ones at the time were probably CVS and subversion. It seemed like every open source project used one. I understood their value for collaboration but not as a sole developer so I steered clear of them for a long while. However, they kept coming up and eventually there were some new kids on the block which were vying for attention: Git, Mercurial and Bazaar. Now, this isn't a history lesson of any kind or some kind of detailed comparison of them all. There’s plenty of that already out there. Instead, this is a short explanation of which one I chose to use when programming in Go and why.

I recall reading a book on C programming which had a section on Subversion. I tried it out. I didn't understand it. There were a lot of do’s and don’ts and I found it very intimidating. I understood the value but hated using it. Why did I have to keep track of changes anyway if I wasn't sharing the code? As time went on and I wanted to finally work on my first major project, however, I knew I was going to need a method of tracking my changes even if I didn't share my code. Too many times I'd get part way through working through a feature, realize I didn't like it or it didn't work, and realize it was too late to go back (or would take a lot to revert the changes. This left me little choice but to look at version control, again.

Fast forward a few years and I finally installed a GNU/Linux distribution that really clicked with me: Ubuntu. Long story short, I discovered Ubuntu used a version control system called Bazaar and after reading a few articles telling me how easy it was to use I decided to give it a try. A lot of mistakes later I finally figured out how to use it and I loved it. Bazaar was simple and easy to use. I really liked that the commands were clear in their purpose and the defaults felt natural. I could clone a repository into a new directory with a specific name, make changes (like a hot-fix or new feature), then merge that directory back into the main repo. and the name stuck with it! Forever I would know that a series of commits belonged to a fix or feature with that name. Loved it!

As time went on, I began to discover that Bazaar was the least favoured of the 'big three' distributed version control systems. At first this didn't matter to me but it started to make me wonder if I'd made the right choice. Indeed, Git and Mercurial both had much larger share of the market, as it were. But, Git scared the hell out of me. It had so many commands and features that it still made me feel truly lost, even after using Bazaar for close to a year.

I had just discovered Go and, perhaps naively, I thought that if I were to create a Go library I should use Google Code hosting, just like Go did! The problem being: no support for Bazaar. This was a problem I found everywhere except Launchpad and Sourceforge. No other popular code hosting site supported Bazaar. But, many of them did support Mercurial and guess what? Mercurial was just like Bazaar but, perhaps, even better.

Mercurial welcomed me with open arms! It supported the same workflow and the commands were all almost identical. Now I had a wider choice in code hosting too.

Again, as I stated already, I was discovering Go. And one thing that Go doesn't like is when sub-directories change. Let me give an example. Lets say you’re using Google Code to host your project. Let’s call the project an imaginative name. Let’s call it: foo. So you’d import that library from “code.google.com/p/foo”. Maybe I want to make a fix. What I would typically do is clone my project into a new directory called foo-name-of-fix. Now, let’s say I want to run the test framework to make sure it passes and I didn't mess something up. If I just run ‘go test’ it will import from “code.google.com/p/foo” NOT from the repo. I just created, which would be “code.google.com/p/foo-name-of-fix”. That meant, every time I made a new directory I’d need to change everything. And, make sure you change it all back before merging! A headache to be sure!

Now, this isn't the only way to do it in Mercurial. You can used named branches or tags. There are ways to overcome this problem. But again, these were workarounds and it was like “a splinter in my mind” driving me crazy. I was also aware that the most popular version control was Git.

So, I tried it. And this time I’d accumulated enough experience that I wasn't completely overwhelmed. Yes, Git has a shit-ton of features. But, you don’t need to use them. You don’t even have to know they’re there. You just need to know the basic commands and ask questions (most questions have already been asked on Stack Overflow so a quick search is usually the best thing to do) when you get stuck or if something bothers you. Because in Git, there’s almost always an answer.

I didn't immediately like Git but one thing I really liked was that named, in-directory branches were the natural way to do things. It made working with Go brutally easy. I didn't like that the name is discarded once you merge and delete said branch. I really liked that Git doesn't assume that just because you made a change to a file you want to commit it. This is a feature I didn't like at first but that I've grown to like. You can use a shortcut when committing to get the same behaviour as Mercurial and Bazaar but otherwise it will force you to manually add those files you want to commit. Very nice for cherry picking changes.

There are things I don’t like about Git. Many of it's features are reserved for corner cases and it’s still a bit overwhelming at first even though the Git community has done a tremendous job alleviating that particular problem. There are a million and one ways to do the same thing with their own pros and cons making it sometimes difficult to know which is the right solution for you. I’m not overly fond that branch names aren't attached to a commit series after merging, either.

Conclusion

I don’t want to mislead you. I still like, and use, Mercurial. As a matter of fact, my main project goncurses uses it still and I have no plans to change that. The Go project itself also uses it. I do, though, find myself missing Git sometimes and the reverse is not true when using other version control systems. I can’t really say why I find myself liking Git better but its just a feeling I get that I can’t put my finger on. Were I to make a recommendation it would be Git because it plays nicely with Go within its 'default' workflow but to be perfectly honest very similar behaviour can be done in Mercurial so you can't really go wrong with it either. It's a very close second. You just need to be aware about the issue with using cloned branches.

Depending on your workflow and habits, you may need to be flexible when using a VCS like Mercurial with Go. My needs are simple and the adjustment wasn't hard. Depending on what you’re used to, no adjustment at all may be necessary.

Proposal for a GUI Toolkit for Go

The Situation

This isn't a unique topic. It's come up in the mailing list before.

Let’s face it, while Go is a general purpose programming language it is ultimately a systems language. It was designed to be concurrent. It was designed to write servers. It was designed to provide a back-end to other services. It does those things pretty well. There is, therefore, an area where Go is lacking and that’s as an application programming language.

This isn't because Go as a language isn't suited it to but that it's standard library doesn't include a crucial component: a UI toolkit. A few people have blamed Google (which makes no sense because Go is not actually a Google project) and the Go developers for this but honestly, it's not their problem. It's not what Go was designed to do nor are the current devs experts in UI design.

Who’s responsible then?

Us.

To be honest, there have been a few attempts already, each with their own merits, but I think all these solutions still fall short of an ideal GUI for Go. So, let me tell you why I think this.


What We Have and Why They Don’t Work

1. GTK+ or another UI Toolkit Wrapper

The easiest, most obvious choice is to create a wrapper for an existing UI Toolkit because of cgo and SWIG. GTK+ fits the bill and mattn has begun this work already. However, there's a problem. For one, GTK+ is C-based. No matter how well the wrapper is designed I don't think it will every feel very Go-like. The toolkit itself is a beast, too. It has thousands of function signatures in it’s API and will take a fairly large concerted effort to completely implement it. More importantly, however, I don’t think this is the right choice and I'll tell you why.

Using GTK+ means that users will need to install it if they don’t have it already. For Linux users, this usually isn't a problem but what about Windows and Mac users? Now we have a problem. I’m a firm believer that you shouldn't force users to install things onto their computer just to run your application unless it's absolutely necessary; and, it makes the installation process more complicated. There is, of course, a solution by including the GTK+ source into the dev tree but we're not talking something like sqlite3 where this can be done easily. GTK+ is massive and I just don't see this as being a feasible solution.

2. GoWut or another HTML Server

I think GoWut is a great idea with what I think is one glaring flaw: it’s a web server. For a single user application, it doesn't feel right. For one, the server doesn't shut down after you’re done using it. Yes, this could be considered a feature because it would reduce load-time of future invocations and you can have user sessions but to me it’s a problem. As an end-user, I don’t want a terminal hanging open with a server running in it. I certainly don’t want all my applications each running it’s own server hogging up my resources, just waiting to be accessed again. I'm sure you could have a 'quit' button to shut down the server but that just doesn't feel right.

3. Go UIK

A native UI Toolkit designed to work like an idiomatic Go library from the start? Sounds awesome! The problem? It’s not done, yet. It’s still in the early design stages but certainly something I’m going to watch for. I think this project has a lot of promise and I look forward to seeing how it turns out. Another issue might that it is going to use a custom widget design rather than a look and feel native to the OS. If, however, it looks pretty enough I’m sure I can overlook that.

4. GoWebkit

Now we’re talking. This to me feels like a more natural solution than a wrapper for another Toolkit. Open a basic application window with webkit embedded in it? Fantastic! This solution comes very close to the mark with one glaring issue: it uses GTK+ as a back-end. The other issue is that it's not a UI. You will have to have some HTML and CSS skills to actually create anything with it and I don't think that provides a very unified solution.


The Solution

Webkit + Native UI Backend + HTML + CSS + (YA/X)ML

Existing technologies mixed together to provide a hybrid result. I think we need a UI Toolkit that uses the OS’s native windowing mechanism (Win API for Windows, Cocoa for Mac, X, GTK+ or Qt for Linux), maybe using John’s WDE library, an embedded webkit renderer, with an API akin to any traditional UI Toolkit complete with event handling. It should allow for customizable CSS elements and, the icing on top, a markup language UI layout design utilizing something like XML. Hrm...it feels like I've just described Android’s UI toolkit...

Of course, this needs some rationale.

1. Native UI Backend - Go is multi-platform. A UI toolkit should be as multi-platform as possible and not require end-users install additional toolkits onto their system. Many application designers will want to control window sizing and else-wise have control over them, like having modal windows. Javascript could provide some of this behavior as well.

2. Webkit - Creating a custom toolkit from scratch, including widgets and all the rest is a colossal undertaking. Qt, GTK+, Win API and Cocoa took an incredible, collective effort to create. Webkit would allow us to circumvent this effort by using a UI that already exists: HTML5 and Javascript. Is this an ideal solution? Maybe not but it’s a reasonable solution that wouldn't require the herculean effort to create a full UI from scratch.

3. HTML + CSS - I think GoWut has a good idea here. Default widgets and creation mechanism so devs don’t have to write HTML and CSS if they aren't skilled in those areas. There is a request in another toolkit's issue tracker requesting ‘nicer’ CSS style to be used by default which I think is a good idea. All a dev would have to do to make their UI unique or make some minor alterations is edit the existing CSS. Or better yet, have an override system in place so that the built-in CSS wouldn't need to be altered. I think adding a custom CSS file shouldn't require injecting an html string with a link element as it does with GoWut but rather a simple API call of SetCSSFile() for a much more elegant and less error prone solution.

4. XML or some other markup language - Android uses this mechanism and so does GTK with GtkBuilder/Glade. I think the reason for this is twofold: a) a UI can be quickly generated by hand without cluttering up the application itself with hordes of API calls for creating widgets; and b) it allows for the use of visual GUI creation tools for building UI’s, akin to Qt’s QtCreater or GTK+’s aforementioned Glade. Like Qt and GTK+, I’m not sure this should be the sole method of building an application UI but rather a complementary one to a standard API. I suggest XML only because I know it's used in existing solutions like Glade/GtkBuilder and Android.

Is this the best solution? Good question!

Conventional languages for writing applications seem to rely on either an existing native toolkit (Tk for TCL, Swing/JavaFX for Java, GTK+ for C or Qt for C++) or a language wrapper for a non-native toolkit. However, most of those tools are either language specific (Swing/JavaFX) or are encumbered by methods for handling situations native to their own languages that don’t play nice with Go (concurrency/threading). There might be ways to overcome these limitations but there you have it.

There are a lot of existing resources for HTML and CSS. I think it would be far easier to find someone proficient in these technologies to help in such an endeavor than it would be to work on some of the other solutions. As a further plus, most of the groundwork has already been done. By combining something like wde for a backend and webkit to render pages then we already have the canvas on which we can paint. By adding in elements of gowut on top, minus the server aspect of it, we essentially have the end product. Add in a dash of CSS to pretty things up, XML for widget creation and layout and we’re done. Sounds easy huh?

Probably isn’t but I think it’s easier than the other solutions with a better end result. It would feel more idiomatic to Go, build on existing technologies, and make for an easy transition for a wide audience (those who are already familiar with HTML, XML and CSS).

I'd love to see some input.

Go: Interfaces and Type Assertion

Hot on the heals of my last post, I feel I have to share a little trick I just discovered about type assertions. While it is documented within the language specs I handn't really seen anyone talk about this in a formal manner until coming across a post in the Go mailing list.

In the pursuit of implementing Scheme-like functional behaviour in Go in order to solve some exercises in SICP I discovered a problem: Scheme is dynamically typed and it's internal data structures (cons, list) allow for mixed types. While this is obviously handy it creates some problems, most important of which is type safety. Thankfully, Go provides an awesome way to handle this: an empty interface.

Go's implementation of interfaces is awesome. It might be a bit foreign to those who come from other languages with interfaces, like Java, where you have to specifically state that an object implements an interface. No so in Go. If you want an object to implement an interface just implement it. Objects can then be used anywhere that accepts that interface. You don't need to explicitly state anything. An object either implements an interface or it doesn't. However, that's not what this post is about. No, this is about type assertions and the empty interface.

An empty interface is just that: Empty. That means that any type matches an interface because the empty interface has no requirements, no functions to implement. You could think of the empty interface as a generic type but I think that's misleading. Interfaces are interfaces and types are types. And this leads us to what I want to talk about. Implementing a list type which accepts multiple types, all at once:

type List []interface{}

We've created a slice type which accepts variables of any type. As I've already stated, the empty interface doesn't have any requirements so every type implements its interface, which is why we can use it like this. What's truly brilliant, is that we can use all the slice operations on this new type just like regular slices like ranging through the slice or creating sub-slices. This absolves us from even creating a constructor. If you want to simply create a list of integers do:

l := List{1, 2, 3, 4}

Fan-freakin-tastic, I say! So we've created a list type that behaves like it's dynamically typed. The icing on the cake is that we don't even need to create a custom String() function for our List type, even with nested Lists!

This is great and all except that Go isn't dynamically typed. So, what happens if we create a list of integers and try to add them up? Well, since an interface{} isn't a type, and we try to use the + operator on it, the compiler will fail with an error. So, how can we get around this? Type assertions. We can assert that an element in the list is of a specific type by using a type assertion:

l[0].(int)

But, there's a problem with that. We don't know what type we might have and we can't guarantee what we've received is what we expect. If we assert that an element in the list is a specific type, and we're wrong, our program will panic and, if not caught, will crash our program. And this, finally, is what I've been wanting to talk about. Thanks to our intrepid Go developers, functions can have multiple return types. In the case of type assertions, we can check to see if the assertion succeeded, which suppresses the panic if we're wrong. So let's write an Accumulate function which performs an operation on our list:

type Operation func(int, int) int

func (l List) Accumulate(f Operation) interface{} {
res := 0
for _, v := range(l) {
if i, ok := v.(int); ok {
res = f(res, i)
continue
}
if lst, ok := v.(List); ok {
res = f(res, lst.Accumulate(f).(int))
}
}
return res
}

Voila! Now, you'll notice that I've cheated a bit where I assert that the value returned by Accumulate is an integer on line 11, after I test whether the current item in the list is another list. Since I know that Accumulate can only ever return an integer I can be safe in that assumption even though Accumulate returns another empty interface. I could have just made Accumulate return an integer and avoided the type assertion completely but the point was to create a function which could accept and return any type. We might want to be able to concatenate strings or operate on floats. However, I cheated a bit in the implementation details for simplicity's sake. Obviously, if you wanted to write an Accumulate function which accepts lists containing many types different types you'd have to do a bit more work to test what type you received back.

Thankfully, there's another way you can test what type a variable is. If you need your function to accept many different types, you can use an assertion switch. In my case, I didn't need to go that far because I knew I was only dealing with integers and lists of lists. However, I'll show you an example of what it might look like:

func (l List) Accumulate2(f Operation) interface{} {
var res interface{} for _, v := range(l) { switch t := v.(type) { case int: if res == nil { res = 0 } res = f(res.(int), t) case List: if i, ok := t.Accumulate2(f).(int); ok { res = f(res.(int), i) } } } return res
}

And that, as they say as that! Strictly speaking, you'd have to do a heck of a lot more work if you were to add more types. For one, what would happen if there were strings mixed with ints? Or even floats and bytes? This function only really works if all the elements are of the same type or if elements are other lists. If, though, you do need to wrestle with mixed types in a single list you might need to rethink the problem you're trying to solve!

Hopefully, this has provided you with another tool to add to your toolbox, as it did for me. For the complete source code, go here.

Data as Procedures

* Updated: Made the example a little more idiomatic and used a more intuitive type for differentiating between the numerator and denominator. Thanks to the excellent comments/suggestions! *

It has been a long while since my last post. Having a stressful full-time job, two kids under the age of five, and all the other fun and foibles that come along with life has been a natural distraction. Hopefully, I can get back into the swing of things again but I make no promises!

I've begun reading Structure and Interpretation of Computer Programs. It has proved, thus far, to be truly enlightening and I feel the need to talk about my first revelation, of a sorts. Coming from a primarily imperative based background, functional programming is somewhat foreign to me. For example, the concept of data as procedures came as truly mind boggling. Even the book pokes fun at the reader about it! I have only ever known data as container constructs. In C and Go these would be structs and in C++ and Java, classes. However, through the use of closures you can do away with data structures and use procedures instead.

SICP uses the example of representing a rational number. A rational number consists of a numerator and a denominator and we normally write them as a faction (i.e. 3/4). In essence, they're a number pair and you could represent them as a structure quite easily:


type RationalNumber struct {
numerator, denominator int
}

You would then create constructor and selector procedures to operator on the data structure. Nothing new here. However, there is a way to do away with the data structure. First we'll create the constructor. This is the hard part:


type Fraction func(bool) int

func NewFraction(n, d int) Fraction {
return func(z bool) int {
if z == true {
return n
}
return d
}
}

Utilizing a closure, we create a function which returns a function containing our data. To make this a little more idiomatic Go we've made the return function it's own type. If we supply this function with a value of true it returns the numerator and if we supply false it returns the denominator.

The rest becomes trivial. We just create our selectors to accept the closure as a parameter and tell it what we want to return:

func Numerator(f Fraction) int {

return f(true)

}

func Denominator(f Fraction) int {

return f(false)

}

Voila! We've just represented data without using a data structure! Colour me impressed! In a language like Scheme the implementation is a lot cleaner looking. Because Go emphasis type-safety we have to do define a lot of data types making thing look a lot more complicated than they really are.

Full source code for this example can be found here. Click run to see it in action.

If you want a more thorough, professional explanation I highly encourage you to read SICP yourself. If you want a PDF you can find a link on the Wikipedia page.