Why?

Well, I want to understand the project, so that I can get involved with it because it seems complex, at least at this point, Yes it does

I spent a reasonable amount of time learning Go, but I didn't harness it by building projects and contributing to open source projects, I believe this can help me understand Golang even better.

I'm unfamiliar with the terminology of the project, I don't understand the idea and why we have it.

How To Use Yaegi CLI

Installation steps

# you may get permission denied error so it's better to execute it as sudo
# Output:
$ sudo curl -sfL https://raw.githubusercontent.com/traefik/yaegi/master/install.sh | sudo bash -s -- -b $GOPATH/bin v0.9.0
traefik/yaegi info checking GitHub for tag 'v0.9.0'
traefik/yaegi info found version: 0.9.0 for v0.9.0/linux/amd64
traefik/yaegi info installed /bin/yaegi

Despite being static and strongly typed, Go feels like a dynamic language. The standard library even provides the Go parser used by the compiler and the reflection system to interact dynamically with the runtime. So why not just take the last logical step and finally build a complete Go interpreter? - Announcing Yaegi

• Here's an example of it, it's a simple program that prints the average of a slice of numbers, and I've added a new function that gives the median of that slice of ints, the code is pretty self-explanatory

  Sample code: RUN

    # Creating a go file
    $ cd 
    $ mkdir yaegi-test && cd yaegi-test
    $ code main.go 
    # pasting the sample code given above and saving it(ctrl + s)
    package main
  
    import "fmt"
  
    var nums = []int{1, 2, 3, 4, 5}
  
    func main() {
        sum := 0
        for _, num := range nums {
            sum += num
        }
        average := float64(sum) / float64(len(nums))
        fmt.Println("The average is", average)
    }
  

  output:

    # opening the terminal in vscode(ctrl + `)
    $ go run main.go
  
    The average is 3
  

  yaegi run main.go interprets the program, and compiles at runtime, one way to check is to declare a variable and don't use it, and print a simple message

    package main
  
    func main() {
        var k int
        println("Hello, 世界")
    }
  

    $ go run main.go
    # command-line-arguments
    ./main.go:4:6: k declared and not used
    $ yaegi run main.go
    Hello, 世界
  

  Note that, you need to have the file saved first before you run the interactive yaegi REPL on the program

  Now in the program's home directory, I'm executing this

  yaegi run -i main.go

  i - the program runs and this flag starts a new interactive REPL within the program's context, any program we run within the reply belong to this interpreter context

  sample program: RUN

    $ cd yaegi-test
    $ yaegi run -i main.go
    The average is 3
    # a new function within the context of main.go
    > func median(nums []int) float64 {
        sorted := make([]int, len(nums))
        copy(sorted, nums)
        sort.Ints(sorted)
        length := len(sorted)
        if length%2 == 0 {
            return float64(sorted[length/2]+sorted[(length/2)-1]) / 2.0
        }
        return float64(sorted[length/2])
    }
    The average is 3
    # this is the reflect value of function median
    : 0xc0001bf5f0
    > fmt.Println("The median is", median(nums))
    The median is 3
    The average is 3
    : 16
  

  In interactive mode, the stdlib packages("sort" in this case) are pre-imported so that I can use them directly

  the code in REPL is evaluated and it returns a reflect.value , value interface reflects the type and value of the Go object, we'll see how we are comparing values using the Interface() method when we are using yaegi interpreter and evaluating code with Eval()method in its dynamic extension framework

• so far I see yaegi CLI is for executing a standalone script within the program's context, when I specifically know the task I'm automating, I can experiment with the existing program and also be able to test the new functionality in the interactive shell

• In the above context, I'm making changes in real-time(while it's executing) and performing operations on the existing data of the program

Yaegi specs

• The interpreter provides an interactive and dynamic way to run the code, I don't have to recompile the program each time I make a change, I can see the results immediately, and I am also able to test and debug individuals lines of code

• Here I'm running the program or it can be a code snippet in an interactive shell on a command-line-interface(a REPL=Read-Eval-Print Loop) like the way we do with python(it's just an example)

> python
Python 3.10.7 (main, Dec 30 2022, 10:22:51) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print("Hello, world")
Hello, world
>>> name = "spike"
>>> print(name)
spike

Motivation

• Make Go programs extensible

• Existing script engines less fit than Go itself

• Unify scripting and implementation languages - GoLab 2019 - Marc Vertes - YAEGI, Yet Another Elegant Go Interpreter

• The dynamic nature of Go and its efficiency makes it a suitable language for scripting, an interpreter like yaegi puts up a solid point to consider it as a scripting engine and use Go for scripting

Yaegi as an embedded interpreter

# command-line executable
# for adding the dependency package or upgrading it to latest version
$ cd yaegi-test
$ go mod init github.com/KiranSatyaRaj/yaegi-test
go: creating new go.mod: module github.com/KiranSatyaRaj/yaegi-test
$ go get -u github.com/traefik/yaegi/cmd/yaegi
go: added github.com/traefik/yaegi v0.15.0
$ go get -u github.com/traefik/yaegi/interp

// import path
import "github.com/traefik/yaegi/interp"

• Yaegi only relies on the go standard library and no external dependencies, the only API is New(), Use(), and Eval()

• For creating customs scripts on the fly, yaegi as embedded interpreter cuts in, the program itself is less tightly integrated into the source code making it easier to modify and update it

• Even though I'm executing the go run command each time I run it, the program isn't recompiled every time, it's being interpreted at runtime, we'll see that in action in this article itself

New() method to create a new interpreter context

• New() is a part of the yaegi interp library which provides a complete Go interpreter, interp.New() method creates a new interpreter context and returns it

• It takes interp.Options{} as input, Options{} is a custom type of interp package it's a struct type, and the values it holds are:

  Run

  Output:

    # pasting code in given from RUN link and executing it
    $ go run main.go 
    {GoPath: BuildTags:[] Stdin:<nil> Stdout:<nil> Stderr:<nil> Args:[] Env:[] SourcecodeFilesystem:<nil> Unrestricted:false}
  

Use() method to import standard library packages into the interpreter context

It takes stlib.Symbols as input, here Symbols can be functions, variables, and data structures, the Use() method takes the binary representation of these symbols and makes them available for the code in the interpreter context, to simply put the standard library packages are imported natively as given in the import path

It throws an error if this isn't configured, we'll see this in action soon

Eval() method to evaluate the code in the interpreter context

It evaluates the source code

The Go code is represented as a string and taken as input, returns the result computed by the interpreter or else we get a non-nil error

Let's RUN this

output:

$ go run main.go
panic: 4:9: import "fmt" error: unable to find source related to: "fmt". Either the GOPATH environment variable, or the Interpreter.Options.GoPath needs to be set

goroutine 1 [running]:
main.main()
    /tmp/sandbox4073505706/prog.go:22 +0x9a

Program exited.

RUN with Use() method

$ go get -u github.com/traefik/yaegi/stdlib

import "github.com/traefik/yaegi/stdlib"

Output:

$ go run main.go 
Hello Yaegi

This is a simple implementation of the embedded interpreter

Yaegi dynamic extension framework

A dynamic function run from static code, what does that mean?

The code is being compiled at runtime, I'm not altering the behavior of the static code which is compiled ahead of runtime, and the code that's defined is only available within the interpreter context

i.Eval() returns a reflect.value

RUN

output:

$ go run main.go 
The median is 3
The average is 3

In the above code, the v stores the reflect.value which is the interface value to access the specific method providing a similar function signature as in v.Interface().(func([]int) int), where we are comparing the value types with Interface() method, which was the average function

End thoughts

The goal is to provide a Go dynamic interpreter embeddable, simple, secure and fast enough to be used as plugin engine

this is not a toy, it's for use in production - GoLab-talk-slides

• It aims to unify Go as the scripting language and these are simple implementations of yaegi, it's specifically for production use cases and faster development so I'm looking forward to using yaegi on projects with large codebases(more likely to use it on an open source project)

So far I've tried my hands on the yaegi-CLI, yaegi as an embedded interpreter, and its dynamic-extension framework, and yet to be explored are its production use cases, architecture, security, and performance too, I need to understand them in detail

Do check this GitHub discussion, the issues I faced and how it was resolved

A series on Traefik-Yaegi would be a good way of continuing it Thanks for staying till the end, I hope this helped you and I'll see you in the next one. HAPPY LEARNING