Showing posts with label Golang. Show all posts
Showing posts with label Golang. Show all posts

Monday, 13 May 2013

Go: Interfaces and Type Assertion

Hot on the heals of my last post, I feel I have to share a little trick I just discovered about type assertions. While it is documented within the language specs I handn't really seen anyone talk about this in a formal manner until coming across a post in the Go mailing list.

In the pursuit of implementing Scheme-like functional behaviour in Go in order to solve some exercises in SICP I discovered a problem: Scheme is dynamically typed and it's internal data structures (cons, list) allow for mixed types. While this is obviously handy it creates some problems, most important of which is type safety. Thankfully, Go provides an awesome way to handle this: an empty interface.

Go's implementation of interfaces is awesome. It might be a bit foreign to those who come from other languages with interfaces, like Java, where you have to specifically state that an object implements an interface. No so in Go. If you want an object to implement an interface just implement it. Objects can then be used anywhere that accepts that interface. You don't need to explicitly state anything. An object either implements an interface or it doesn't. However, that's not what this post is about. No, this is about type assertions and the empty interface.

An empty interface is just that: Empty. That means that any type matches an interface because the empty interface has no requirements, no functions to implement. You could think of the empty interface as a generic type but I think that's misleading. Interfaces are interfaces and types are types. And this leads us to what I want to talk about. Implementing a list type which accepts multiple types, all at once:

type List []interface{}

We've created a slice type which accepts variables of any type. As I've already stated, the empty interface doesn't have any requirements so every type implements its interface, which is why we can use it like this. What's truly brilliant, is that we can use all the slice operations on this new type just like regular slices like ranging through the slice or creating sub-slices. This absolves us from even creating a constructor. If you want to simply create a list of integers do:

l := List{1, 2, 3, 4}

Fan-freakin-tastic, I say! So we've created a list type that behaves like it's dynamically typed. The icing on the cake is that we don't even need to create a custom String() function for our List type, even with nested Lists!

This is great and all except that Go isn't dynamically typed. So, what happens if we create a list of integers and try to add them up? Well, since an interface{} isn't a type, and we try to use the + operator on it, the compiler will fail with an error. So, how can we get around this? Type assertions. We can assert that an element in the list is of a specific type by using a type assertion:

l[0].(int)

But, there's a problem with that. We don't know what type we might have and we can't guarantee what we've received is what we expect. If we assert that an element in the list is a specific type, and we're wrong, our program will panic and, if not caught, will crash our program. And this, finally, is what I've been wanting to talk about. Thanks to our intrepid Go developers, functions can have multiple return types. In the case of type assertions, we can check to see if the assertion succeeded, which suppresses the panic if we're wrong. So let's write an Accumulate function which performs an operation on our list:

type Operation func(int, int) int

func (l List) Accumulate(f Operation) interface{} {
res := 0
for _, v := range(l) {
if i, ok := v.(int); ok {
res = f(res, i)
continue
}
if lst, ok := v.(List); ok {
res = f(res, lst.Accumulate(f).(int))
}
}
return res
}

Voila! Now, you'll notice that I've cheated a bit where I assert that the value returned by Accumulate is an integer on line 11, after I test whether the current item in the list is another list. Since I know that Accumulate can only ever return an integer I can be safe in that assumption even though Accumulate returns another empty interface. I could have just made Accumulate return an integer and avoided the type assertion completely but the point was to create a function which could accept and return any type. We might want to be able to concatenate strings or operate on floats. However, I cheated a bit in the implementation details for simplicity's sake. Obviously, if you wanted to write an Accumulate function which accepts lists containing many types different types you'd have to do a bit more work to test what type you received back.

Thankfully, there's another way you can test what type a variable is. If you need your function to accept many different types, you can use an assertion switch. In my case, I didn't need to go that far because I knew I was only dealing with integers and lists of lists. However, I'll show you an example of what it might look like:

func (l List) Accumulate2(f Operation) interface{} {
var res interface{} for _, v := range(l) { switch t := v.(type) { case int: if res == nil { res = 0 } res = f(res.(int), t) case List: if i, ok := t.Accumulate2(f).(int); ok { res = f(res.(int), i) } } } return res
}

And that, as they say as that! Strictly speaking, you'd have to do a heck of a lot more work if you were to add more types. For one, what would happen if there were strings mixed with ints? Or even floats and bytes? This function only really works if all the elements are of the same type or if elements are other lists. If, though, you do need to wrestle with mixed types in a single list you might need to rethink the problem you're trying to solve!

Hopefully, this has provided you with another tool to add to your toolbox, as it did for me. For the complete source code, go here.

Saturday, 21 January 2012

Build Google Go on Windows

** Important Note: The Go team now provides a Windows port of Go. This is the recommended way of installing Go on that platform but I will leave this here as an exercise to whomever would prefer to install Go this way. **

I've seen countless posts requesting a Windows port of Go. For quite some time there hasn't been one available but, thankfully, there is an experimental port available. However, as it turns out, you can also compile it yourself without waiting for someone else to do it for you. This gives you the advantage of always having an up-to-date version of Go by having immediate access to the latest release without having to wait for a build, or even using the weekly release. In order to do this, we'll need MinGW.

So, without further adieu, here is a step by step installation procedure to start using Go on Windows.

1) Install Msys and MinGW. You will need a POSIX compliant shell in order to build Go. I'm sure Cygwin would also be capable of building Go but I prefer Msys. For anyone new to Msys, or unfamiliar with using the CLI, I would recommend using the GUI installer. As per the golang documentation, you'll need to make sure you install the following tools: gcc,  glibc or some other C standard library, Bison, Make and Awk. All of these tools are available via the apt-like mingw-get tool. If you forget to download one of these tools during installation, don't worry. The mingw-get tool is accessible from within Msys.

2) Install Mercurial. Here you have two choices. One, you can try and build Mercurial from source on MinGW or you can do the sensible thing and download the Windows version. I highly recommend doing the latter. You can also get TortoiseHg if you require a GUI but if your only intention is to use it to download Go then I wouldn't bother. It's much simpler just using the CLI.

3) Open the Msys terminal. Assuming all is right with the world you will be in your home directory. You should now be able to use the following command, as taken from the Golang installation guide:

$ hg clone -u release https://go.googlecode.com/hg/ go

4) Change to the go/src directory and run all.bash.

$ cd go/src
$ ./all.bash

5) If all goes well, Go is now installed and ready to go! That's it! I recommend creating a .bashrc file and add the recommended environmental variables. Something simple like:

$ cat > ~/.bashrc << EOF
# .bashrc
export GOROOT="$HOME/go"
export PATH="$PATH:$GOROOT/bin"
EOF
$ . ~/.bashrc

Msys does not come with any kind of editor installed (not even ed). Vim is available through the repositories via mingw-get or you can just use your normal text editor for Windows to edit the file. There are plenty of other things you might wish put into your .bashrc, too, but that is beyond the scope of this article. If you installed Msys into the default installation directory, you would save it in "C:\MinGW\msys\1.0\<username>\.bashrc" where <username> is replaced with your name. Obviously.

I should mention that on my first attempt on building Go, the build script hung while trying to compile gprof. Sadly, I didn't have the presence of mind at the time to record on which file it got stuck on, so there's not much I can do to see if this is a Go or MinGW/Msys bug. Irregardless, I had to force-quit Msys by closing the window because no other method would unlock the process. Re-running the all.bash script a second time completed successfully and finally reported that, "ALL TESTS PASSED!" Now THAT is what I wanted to hear! Or, in this case, see.

You should now be able to use the Go tools from the 'cmd' shell but in order for the shell to find it you will need to manually adjust your %PATH% environmental variable in Windows, too. A simple way to do this on Windows 7 is to open the start menu, right-click on 'Computer' and select properties. Then, click the 'Advanced Settings' link on the left to open the advanced settings window. A button near the bottom of the window will allow you to alter your PATH variable. A simple search can show you how to do it on other versions of windows or via an alternate method.

And that, as they say is that. A native Windows build of Go. Provided you adjust some environmental variables, as noted in the 'Getting Started' documentation for Go, you are free to relocate the go tree to another location like "C:\Program Files (x86)". You may also want to build the 'hello world' application as an extra sanity test. You can install a Go IDE or add Go syntax highlighting to your favourite editor.

Sunday, 8 January 2012

A Path to Python

I've always liked the saying, "The only truly stupid person is the one who has nothing left to learn." I have no idea who said it, or if I even have the quote right, but it's always stuck with me. So, I am always trying to learn new programming techniques and new languages.

Call it a fault of mine but I always learn the best when I'm enjoying what I'm learning. I know I don't stand alone in that. So, I've discovered that when I'm going to learn a new language I need to know more about it first; and, to really know a language you have to see it in action. Feature's really don't mean a lot unless the language "speaks" to you. So, I always browse the source code of some applications written in the language. I always take a look at a minimal "hello, world" application, too.

Python has been recommended to me in the past on several occasions but I've never really paid it much interest. My first hang up was the need for the Python interpreter. I didn't like the idea that a user of my product would have to install the Python VM just to use my application. I prefer an all-in-one package and bundling Python with the installer would increase an installer's size dramatically. Of course, there are other solutions but at the time that's how I saw it. I was also worried that Python was very slow. Maybe, at the beginning, it was slow but in relation to what? Will the speed difference even matter or be noticeable? And is that still the case? For application level programming it would probably be a complete non-issue but, again, I couldn't see that at the time.

Go also challenged my programming dogmas. For example, take dynamic linking. I had grown to believe that dynamically linking libraries (not to be confused with dynamic loading) was the only way to go. I just couldn't wrap my head around why anyone would ever use static linking (make sure to follow the links at the bottom of the page) and chalked it up as the old, antiquated way of doing things. How wrong was I? Both methods of linking have advantages and disadvantages and I can certainly appreciate why Go does much of it's linking statically.

Go does struggle with a couple issues and one of the big ones is shared with C. Because most third-party libraries utilized by Go are linked to C libraries portability becomes an issue. I've been looking for a "write-once, run-anywhere" solution ever since writing my first non-trivial application. There is only one platform that immediately jumps out at me to fit that bill: Java.

Initially, I thought Java might be a great language. It's C-like, the JVM is installed on most platforms, and has a huge collection of libraries built right in! Unfortunately, despite it's many strengths, I didn't enjoy programming in the language. My feelings about Oracle aside, I felt that it is very, very verbose. Take even a simple "Hello World!" program for example:


class HelloWorld {
  static public void main( String args[] ) {
    System.out.println( "Hello World!" );
  }
}


Now, compare that to Go:


package main
import "fmt"
func main() {
  fmt.Printf("Hello World\n")
}


And, finally, Python:


print "Hello World"


As I began to work with the language I started to feel like it was more of a patched together, band-aid solution, mess. Don't get me wrong, Java is a decent language but it just doesn't work for me. C is painful enough to program in, Java seems to be worse. Thankfully, other avenues to the JVM platform exist.

I considered Clojure, Jython, Groovy and Scala. I want to learn to learn a LISP-ish language at some point so I thought Clojure may work. After looking at some source code and documentation for the other languages, the only one which spoke to me was Jython. Python + the JVM? How can you go wrong?

Documentation for Jython is somewhat lacking if you don't already know Python. Good thing there's an easy solution to that. The Jython documentation is good for learning how it differs from Python and how to utilize the JVM, and Java, from within Jython but not much on learning the language itself. So? Learn Python.

My first observation has been that it is remarkably similar to Go. Go has clearly been heavily influenced by Python and I feel right at home with it. It does differ from Go in several distinct ways but it's turning out to be a completely fantastic language and I can see why so many programmers are using it. It's expressive, concise and an altogether eloquent language.

Time is important to me. I have precious little of it. So getting bogged down with nonsense is something I do my best to avoid. Python, like Go, does away with a lot of the nonsense and let's you get down to producing functional projects. I can see myself working with it a lot in future projects and I look forward to that time. I think, too, it will help me bridge the mental gap with being able to work in a LISP-like language.

After learning the basics of Python I then started to work with Jython. The amount of time it took for the Jython interpreter to load took me aback. I had gotten used to how quickly the Python interpreter loaded. I then discovered that Jython is based on Python 2.5. Not only is it two CPython releases old (as of this writing CPython 2.7 is the current stable 2.x branch) but Python 3 has also been released. While this hasn't proved to be particularly handy-capping thus far I am worried about the future of Jython as it falls further and further behind it's parent project. I am also concerned about it's speed. Is the slowness of it's interpreter indicative of the language itself? I suppose time will tell, literally.

Sunday, 23 October 2011

Experimenting with Cgo - Part 3: Errno

In this third installment of introducing cgo to novice Go programmers I want to talk about error handling. More specifically, how to handle C functions that utilize errno. At first glance one would think that this is something no harder to implement than anything else. Not so. There are a couple of "gotchas" with errno in particular you need to be aware of and little to no documentation to help you out. In particular, there was absolutely no documentation describing how to extract errno from os.Error.

  1. errno can't be accessed directly in Go. The cgo documentation clearly states why this is the case. In two words, 'thread safety'.
  2. There is a handy way to trap errno as an os.Error in Go but no obvious way to extract the actual error number.
After a lot of trial and error I finally pieced the puzzle together. First, if the C functions you're implementing set a common system error, as a function like fopen() would, then cgo provides a simple and easy way to get a Go-like error. Again, I refer you to the cgo documentation and you don't really need to look any further. If, however, you're using a library that doesn't you'll need to follow these steps, or at least some of them.

1. Map the error codes to sensible strings describing the error. Documentation on the library you're implementing usually does a good job helping here.

var errText = map[int]string {
    C.EINVAL: "Invalid mode specified",
}

2. Create a helper function to handle printing the error.

func error(e os.Error) os.Error {
    s, ok := errText[int(e.(os.Errno))]
    if ok {
        return os.NewError(s)
    }
    return os.NewError(fmt.Sprintf("Unknown error: %d", int(e.(os.Errno))))
}

This was the hard part. The os package provides Errno, which is Go's equivalent to errno. In order to extract Errno from Error you have to use a type assertion. Errno is typed to int64 which won't work with our C.int error codes so we need to then type cast it to an int.

3. Catch the error value returned by the function utilizing errno to describe errors.

    f, err := C.fopen(cpath, cmode)
    if f == nil {
        return nil, error(err)
    }

It's as simple as that. Complete source code follows:

Makefile:
include $(GOROOT)/src/Make.inc

TARG=cgoexample
CGOFILES=cgofopen.go

include $(GOROOT)/src/Make.pkg

cgofopen.go:
package cgoexample

//#include <stdio.h>
//#include <stdlib.h>
//#include <errno.h>
import "C"

import (
    "fmt"
    "os"
    "unsafe"
)

var errText = map[int]string {
    C.EINVAL: "Invalid mode specified",
}

func error(e os.Error) os.Error {
    s, ok := errText[int(e.(os.Errno))]
    if ok {
        return os.NewError(s)
    }
    return os.NewError(fmt.Sprintf("Unknown error: %d", int(e.(os.Errno))))
}

type File C.FILE

func Open(path, mode string) (*File, os.Error) {
    cpath, cmode := C.CString(path), C.CString(mode)
    defer C.free(unsafe.Pointer(cpath))
    defer C.free(unsafe.Pointer(cmode))
    
    f, err := C.fopen(cpath, cmode)
    if f == nil {
        return nil, error(err)
    }
    return (*File)(f), nil
}

func (f *File) Close() {
    if f != nil {
        C.fclose((*C.FILE)(f))
    }
}

example/Makefile:
include $(GOROOT)/src/Make.inc

TARG=cgoexample
GOFILES=fopen.go

include $(GOROOT)/src/Make.cmd

example/fopen.go
package main

import (
    "cgoexample"
    "fmt"
)

func main() {
    f, e := cgoexample.Open("foobar", "qq")
    defer f.Close()
    
    if e != nil {
        fmt.Println("Error: ", e)
    }
}

Tuesday, 4 October 2011

Experimenting with Cgo - Part 2

In this second cgo article, I am going to demonstrate how to handle strings. There is already an excellent post by Andrew Gerrand, one of the Go developers at Google, on using strings in cgo. My own post will provide a slightly more complete example.

The first thing to understand is that C has no concept of a string. Unlike in Go, where a string is an actual type, a string in C is just a pointer to an array of characters. Thankfully, the "C" package has two simple functions to convert between the two types. However, there are two problems: One, converting from a Go string to a C string (character array) allocates memory to the string which is NOT garbage collected by Go. This is important! The reason for this is because Go has no way of knowing when the C library you're creating a language binding for will be done with the string. Once your function exists, any local variables will go out of scope and Go's garbage collector will at some point free the allocated memory even though the C library involved may not be finished with the string. Therefore, you have to free the memory when you know the C library is finished with it. Second, you still need a means of storing characters into a buffer and there isn't an immediately obvious way to do that.

This article assumes you've read my previous post and already know how to start a basic cgo project and have a working Go development environment. I am going to provide a very simple wrapper for the following functions from stdio.h: fopen, fclose, fputs, fgets and frewind.

Important: Absolutely no attempt was made to provide error handling in this code in order to provide absolute clarity for the code itself. While I believe proper examples should always contain error handling, handling errno and NULL return values is not easily handled in Go.

1. Include any C headers and import any libraries you will need:


// #include <stdio.h>
// #include <stdlib.h>
import "C"
import "unsafe"

Note that, when converting to a C string, you will need to free the allocated memory. Free() from stdlib.h is required to do this. You will also need to the "unsafe" library to handle C pointers.


2. In order to make stdio feel like Go, type the FILE structure:

type File C.FILE

While embedding the FILE type in a struct would allow us to create much clearer code because we could remove all the casting to/from the C and Go types Go does not allow C types to be embedded into Go structs. So, we must do things the hard way.

3. Implement the fopen() function:


func Open(path, mode string) *File {
    cpath, cmode := C.CString(path), C.CString(mode)
    defer C.free(unsafe.Pointer(cpath))
    defer C.free(unsafe.Pointer(cmode))
    
    return (*File)(C.fopen(cpath, cmode))
}

Notice first that, like the original C code, we accept strings as arguments to indicate the path and mode. The CString() function from the "C" package allocates enough memory for the Go string and then converts it to a C string. You must make sure you free the memory after using it, and the defer statement is the ideal way to accomplish this. Not only does it guarantee to free up the memory it does so after we return the value from fopen(). free() take a void pointer as an argument, so variables passed to the function must be cast to an unsafe.Pointer().


4. Implement the fgets() function:


func (f *File) Get(n int) string {
    cbuf := make([]C.char, n)
    return C.GoString(C.fgets(&cbuf[0], C.int(n), (*C.FILE)(f)))
}

I found this to be tricky to figure out until I thought about it. You need to provide a pointer to a C character array but Go provides no obvious ways to do so. Use Go's make() function to build an array of characters. You could also use a static array size but the dynamic version provides a lot more flexibility and is no more complex to implement. Use C.GoString() to convert the C string to a Go string.

5. Implement the rest of the functions. See below for the complete code.

6. Compile and install your library.

7. Compile and run a test program to complete your proof of concept.


The complete code follows:

godev/src/gostdio/gostdio.h:

package gostdio

// #include <stdio.h>
// #include <stdlib.h>
import "C"
import "unsafe"

type File C.FILE

func Open(path, mode string) *File {
    cpath, cmode := C.CString(path), C.CString(mode)
    defer C.free(unsafe.Pointer(cpath))
    defer C.free(unsafe.Pointer(cmode))
    
    return (*File)(C.fopen(cpath, cmode))
}

func (f *File) Close () {
    C.fclose((*C.FILE)(f))
    return
}

func (f *File) Get(n int) string {
    cbuf := make([]C.char, n)
    return C.GoString(C.fgets(&cbuf[0], C.int(n), (*C.FILE)(f)))
}

func (f *File) Put(str string) {
    cstr := C.CString(str)
    defer C.free(unsafe.Pointer(cstr))
    
    C.fputs(cstr, (*C.FILE)(f))
    return
}

func (f *File) Rewind() {
    C.rewind((*C.FILE)(f))
}


godev/src/cgoexample/cgoexample.go:

package main

import "gostdio"
import "fmt"

func main() {
    f := gostdio.Open("test.txt", "w+")    
    defer f.Close()
    
    f.Put("Some example text\n")
    f.Rewind()
    
    fmt.Println(f.Get(13))
}



Monday, 26 September 2011

Experimenting with Cgo - Part 1

** Modified: Updated to comply with Go 1.1 release. **

This is the first in a planned series of posts on cgo. Cgo is a program in the Go language tool suite that allows programmers to wrap C language libraries for use in Go. I have found it to be an incredibly useful tool and I am sure will help provide Go with the ability to grow it's third-party, non-standard library more quickly.

I am currently designing some software for personal business use which would utilize the ncurses library as an interface. There is already a curses wrapper library available for Go but I wasn't entirely satisfied by the completeness of the library. For one, the mouse isn't implemented at all, along with a myriad of other functions. Nor are the form, menu or panel libraries. So I decided that this would the perfect time to learn cgo.

To create a wrapper library for a C library you need to take the following steps:

1. Setup your build environment. I recommend using a sub-directory in your home directory. On Linux you would use:


$ cd ~
$ mkdir -p godev/src/mylibwrapper


Of course, you can change the 'godev' to whatever you want and 'mylibwrapper' to the name of the library you're creating a wrapper for but 'src' must be a sub-directory of your development hierarchy.

You will also need to set your $GOPATH to this new directory. On Linux you might use:


$ export GOPATH="$GOPATH:/home/username/godev"


Take care not to add a slash at the end of the directory name as the go tools won't like it.

2.  Within your wrapper library you need to include the C library header(s) you need to use followed by the C import directive:



// #include <math.h>
import "C"



It is essential that 'import "C"' is on its own, directly after the commented C code in order for cgo to work properly. Or, at all for that matter. You can write your own C code prior to the import call and use either of Go's comment styles. 


3. If you need to pass any compile flags or link to the C library, like the C math library, you would use cgo's LDFLAGS and CFLAGS:



// #cgo LDFLAGS: -lm



4. To access any of the C functions from your Go code, you use the C namespace. Same goes for C types. As a matter of fact, no Go types are available in C and vice versa. Therefore, everything you need to pass to or from a C function must be cast to or from a C or Go type. Unfortunately, strings aren't so easy. In order to convert a Go string to a C string, you need to use the CString function. For a good example and explanation, see Andrew Gerrards blog post on the subject.



func Pow(b, e float64) float64 {
    return float64(C.pow(C.double(b), C.double(e)))
}



5. Run 'go build' to compile your code. This step is not strictly necessary as installing automatically builds the library for you and will not install it if the build stage fails.

6. In order to test your new wrapper, you'll need to run 'go install' and then import your library into some Go source.

That's it, start to finish. To view a complete example:


godev/src/mylibwrapper/mylibwrapper.go:

package mylibwrapper

// #cgo LDFLAGS: -lm
// #include <math.h>
import "C"

func Pow(b, e float64) float64 {
    return float64(C.pow(C.double(b), C.double(e)))
}


godev/src/cgoexample/example.go:

package main

import (
    "mylibwrapper"
    "fmt"
)

func main() {
    b, e := 5.0, 2.0
    f := mylibwrapper.Pow(b, e)
    fmt.Println(b, "raised to the power of", e, "is", f)
}