Introduction To Golang

Concurrency is not parallelism

some quotes to support this statement :)

Concurrency is a property of the code ; parallelism is a property of the running program

Concurrency is about dealing with lot of things at once . Parallelism is about doing lot of things at once

Go is known for its first class support for concurrency code in a program , and channels solve the problem of communicating safely between concurrently running code .

Go also supports more traditional tools for writing concurrency code . Mutexes , pools , locks they are implemented in sync package

Go provides

Concurrency execution ( goroutines)
synchronisation and messaging ( channels )
Muti-way concurrent control ( select )

1. Channels in short

Channel are one of the synchronization primitives in Go . Channel can be imagined as a pipe for stream of data . they are used to communicate information between goroutines .

Create a Channel is simple

channel := make( chan interface {})

there are a few types of channel that you can create

2. Bidirectional Channels

by default you create channel that can be read and written .

package main

import "fmt"

func main() {
    messages := make(chan string)
    go func() { messages <- "ping" }()

    msg := <-messages
    fmt.Println(msg)

}

Anyone can read and write to this channel and this can cause problem in concurrent environments .

3. Directional Channels

you can specify if a channel is meant to be only read or written data . This increases the type-safety of the program

the sender cam know when there's no more data to send , and its the receivers responsibility to watch for the channel

package main

import "fmt"

func ping(pings chan<- string, msg string) {
    pings <- msg

}

func pong(pings <-chan string, pongs chan<- string) {
    msg := <-pings
    pongs <- msg
}

func main() {
    pings := make(chan string, 1)
    pongs := make(chan string, 1)
    ping(pings, "passed message")
    pong(pings, pongs)
    fmt.Println(<-pongs)
}

4. Buffered Channels

Default channel are unbuffered , they will only accept sends (chan <-) if there is a receive (<- chan ) ready to read the send value .

Buffered channel accept a limited number of values without a corresponding receiver for those values .

package main

import "fmt"

func main() {

    messages := make(chan string, 2)
    messages <- "buffered"
    messages <- "channel"

    fmt.Println(<-messages)
    fmt.PrintIn(<-messages)

}

Lets understand Concurrency in depth

Don't Communicate by sharing memory , share memory by communicating

5. Creating goroutine

package main

import "fmt"

func main() {
    fmt.Println("Hello,world ")
}
// here func main itself act as goroutine

this is very small program not covering much about groutine

let add one function and try to print something

package main

import "fmt"

func printSomething(s string) {

    fmt.Println(s)

}

func main() {

    printSomething("my 1st line ")
    printSomething("my 2nd line ")

}

output :

my 1st line 
my 2nd line 

Program exited.

so lets add go keyword to function it can start its own goroutine


    go printSomething("my 1st line ")

what happened here go compiler schedule its own goroutine and if your run you see only my 2nd line printed as output . this is so lightweight and fast but there is no error lets put timer after go routine

go printSomething("my 1st line ")

    time.Sleep(1 * time.Second)

it will get you output but this is not best way to do it !

6 . WaitGroups is the right way


package main

import (
    "fmt"
    "time"
)

func printSomething(s string) {

    fmt.Println(s)

}

func main() {

    words := []string{

        "one",
        "two",
        "five",
    }

    for i, x := range words {

        go printSomething(fmt.Sprintf("%d: %s", i, x))

    }

    time.Sleep(1 * time.Second)
    printSomething("my 2nd line ")

}

output

0: one
2: five
1: two
my 2nd line 

Program exited.

if you see here the strings index is not in order that's why time.Sleep(1 * time.Second) is not right way !

       var wg sync.WaitGroup

so we are printing 3 time that is so basically we need to wait each time so we will add wg.add(3) also after running go routine we need to set it to zero . so we will add another parameter to printSomething function wg *sync.WaitGroup and then add defer to run surrounding function

package main

import (
    "fmt"
    "sync"
)

func printSomething(s string, wg *sync.WaitGroup) {
    defer wg.Done()
    fmt.Println(s)

}

func main() {

    var wg sync.WaitGroup

    words := []string{

        "one",
        "two",
        "five",
    }

    wg.Add(len(words))

    for i, x := range words {

        go printSomething(fmt.Sprintf("%d: %s", i, x), &wg)

    }

    wg.Wait()
    wg.Add(1)

    printSomething("my 2nd line ", &wg)

}

7 . Writing tests with WaitGroups


   wg.Add(5)

lets assume you have added value more then the length of string it show error


2: five
1: two
0: one
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc00005c060?)
    /usr/local/go-faketime/src/runtime/sema.go:56 +0x25
sync.(*WaitGroup).Wait(0x496148?)
    /usr/local/go-faketime/src/sync/waitgroup.go:136 +0x52
main.main()
    /tmp/sandbox3776579600/prog.go:33 +0x1c8

Program exited.

package main

import (
    "io"
    "os"
    "strings"
    "sync"
    "testing"
)

func Test_printSomething(t *testing.T) {
    stdOut := os.Stdout

    r, w, _ := os.Pipe()
    os.Stdout = w

    var wg sync.WaitGroup
    wg.Add(1)
    go printSomething("ok print", &wg)
    wg.Wait()

    _ = w.Close()

    result, _ := io.ReadAll(r)
    output := string(result)
    os.Stdout = stdOut
    if !strings.Contains(output, "ok print") {
        t.Errorf("Expected output to contain 'ok print' but got %s", output)
    }
}

output

go test *.go
ok      command-line-arguments  0.099s

let write sync.WaitGroup to print everything in order

package main

import (
    "fmt"
    "sync"
)

var msg string

func updateMessage(s string) {
    defer wg.Done()
    msg = s
}

func printMessage() {
    fmt.Println(msg)
}

var wg sync.WaitGroup

func main() {

    msg = "Hello, world!"
    wg.Add(1)

    go updateMessage("Hello, One !")
    wg.Wait()
    printMessage()

    wg.Add(1)
    go updateMessage("Hello, two !")
    wg.Wait()
    printMessage()

    wg.Add(1)
    go updateMessage("Hello, three !")
    wg.Wait()
    printMessage()
}

if you see above program we have different go routine that print different values now here we set defer wg.Done() that set value to -1 and we add the wg.Add(1) and then whenever the new go routine start we put wg.Wait() and print value and set back to wg.Add(1) so this way we get the exact same out no matter you run n number of time

output


~/Documents/golabs ❯ go run .
Hello, One !
Hello, two !
Hello, three !

~/Documents/golabs ❯ go run .
Hello, One !
Hello, two !
Hello, three !

8. Race Condition

package main

import (
    "fmt"
    "sync"
)

var msg string
var wg sync.WaitGroup

func updateMessage(s string) {
    defer wg.Done()
    msg = s

}

func main() {

    msg = "Hello, world!"
    wg.Add(2)
    go updateMessage("Hello, one")
    go updateMessage("Hello, two")
    wg.Wait()

    fmt.Println(msg)

}

if you see above program we have two variables one in msg string and another sync.WaitGroup and defer wg.Done() we are decrementing Waitgroup by 1 and then in main function we have two go routine running and if you run this program .


~/Documents/golabs ❯ go run .      
Hello, one

but as some point you may get different output

go run -race .
==================
WARNING: DATA RACE
Write at 0x0000011d09b0 by goroutine 8:
  main.updateMessage()
      /Users/sangam/Documents/golabs/main.go:13 +0x78
  main.main·dwrap·3()
      /Users/sangam/Documents/golabs/main.go:22 +0x47

Previous write at 0x0000011d09b0 by goroutine 7:
  main.updateMessage()
      /Users/sangam/Documents/golabs/main.go:13 +0x78
  main.main·dwrap·2()
      /Users/sangam/Documents/golabs/main.go:21 +0x47

Goroutine 8 (running) created at:
  main.main()
      /Users/sangam/Documents/golabs/main.go:22 +0x164

Goroutine 7 (finished) created at:
  main.main()
      /Users/sangam/Documents/golabs/main.go:21 +0xeb
==================
Hello, two
Found 1 data race(s)
exit status 66

if I run this time with go run -race it will give warning . in this condition its accessing same data but we are not sure which go routine going to finish first so you can run into problem , now how we can fix this problem !

9. Mutex

package main

import (
    "fmt"
    "sync"
)

var msg string
var wg sync.WaitGroup

func updateMessage(s string, m *sync.Mutex) {
    defer wg.Done()
    m.Lock()
    msg = s
    m.Unlock()

}

func main() {

    msg = "Hello, world!"
    var mutex sync.Mutex
    wg.Add(2)
    go updateMessage("Hello, one", &mutex)
    go updateMessage("Hello, two", &mutex)
    wg.Wait()

    fmt.Println(msg)

}

let create var mutex with of mutex is sync.Mutex and it will receive as a parameter func updateMessage(s string, m *sync.Mutex) and then add reference to the mutex go updateMessage("Hello, one", &mutex) now to rescue the race data we will add m.Lock() no one access it until it use and then unlock it m.Unlock()

~/Documents/golabs ❯ go run .
Hello, one

~/Documents/golabs ❯ go run -race .
Hello, two

so here we are accessing data safely and there is no race data warning .

package main

import (
    "testing"
)

func Test_updatemesasge(t *testing.T) {
    msg = "Hello, world!"

    wg.Add(1)
    go updateMessage("Hello, one")
    wg.Wait()

    if msg != "Hello, one" {
        t.Error("Expected ")
    }
}

output :


~/Documents/golabs ❯ go test .
ok      example2        0.262s

let duplicate go routine with same parameter

package main

import (
    "testing"
)

func Test_updatemesasge(t *testing.T) {
    msg = "Hello, world!"

    wg.Add(2)
    go updateMessage("x ")
    go updateMessage("Hello, one")
    wg.Wait()

    if msg != "Hello, one" {
        t.Error("Expected ")
    }
}

output

~/Documents/golabs ❯ go test .
ok      example2        0.439s

let check data race

go test -race .
==================
WARNING: DATA RACE
Write at 0x0000012c0650 by goroutine 9:
  example2.updateMessage()
      /Users/sangam/Documents/golabs/main.go:14 +0x78
  example2.Test_updatemesasge·dwrap·5()
      /Users/sangam/Documents/golabs/main_test.go:12 +0x47

Previous write at 0x0000012c0650 by goroutine 8:
  example2.updateMessage()
      /Users/sangam/Documents/golabs/main.go:14 +0x78
  example2.Test_updatemesasge·dwrap·4()
      /Users/sangam/Documents/golabs/main_test.go:11 +0x47

Goroutine 9 (running) created at:
  example2.Test_updatemesasge()
      /Users/sangam/Documents/golabs/main_test.go:12 +0x164
  testing.tRunner()
      /usr/local/Cellar/go/1.17.3/libexec/src/testing/testing.go:1259 +0x22f
  testing.(*T).Run·dwrap·21()
      /usr/local/Cellar/go/1.17.3/libexec/src/testing/testing.go:1306 +0x47

Goroutine 8 (finished) created at:
  example2.Test_updatemesasge()
      /Users/sangam/Documents/golabs/main_test.go:11 +0xeb
  testing.tRunner()
      /usr/local/Cellar/go/1.17.3/libexec/src/testing/testing.go:1259 +0x22f
  testing.(*T).Run·dwrap·21()
      /usr/local/Cellar/go/1.17.3/libexec/src/testing/testing.go:1306 +0x47
==================
--- FAIL: Test_updatemesasge (0.00s)
    testing.go:1152: race detected during execution of test
FAIL
FAIL    example2        0.270s
FAIL

if you see here I get race data warning !

package main

import (
    "fmt"
    "sync"
)

var wg sync.WaitGroup

type Income struct {
    Source string
    Amount int
}

func main() {
    // variable for bank balance
    var bankBalance int
    var balance sync.Mutex

    // print out starting values
    fmt.Printf("Initial account balance: $%d.00", bankBalance)
    fmt.Println()

    // define weekly revenue
    incomes := []Income{
        {Source: "google", Amount: 500},
        {Source: "amazon ", Amount: 10},
        {Source: "docker ", Amount: 50},
        {Source: "tenable", Amount: 100},
    }

    wg.Add(len(incomes))

    // loop through 52 weeks and print out how much is made; keep a running total
    for i, income := range incomes {

        go func(i int, income Income) {
            defer wg.Done()

            for week := 1; week <= 52; week++ {
                balance.Lock()
                temp := bankBalance
                temp += income.Amount
                bankBalance = temp
                balance.Unlock()

                fmt.Printf("On week %d, you earned $%d.00 from %s\n", week, income.Amount, income.Source)
            }
        }(i, income)
    }

    wg.Wait()

    // print out final balance
    fmt.Printf("Final bank balance: $%d.00", bankBalance)
    fmt.Println()
}

here in above program we have struct type of Income with two field one in Source and another Amount

type Income struct {
    Source string
    Amount int
}

lets declare length of Waitgroup

    wg.Add(len(incomes))

here we have inner loop which consists of weekly earnings

    for i, income := range incomes {

        go func(i int, income Income) {
            defer wg.Done()

            for week := 1; week <= 52; week++ {
                balance.Lock()
                temp := bankBalance
                temp += income.Amount
                bankBalance = temp
                balance.Unlock()

                fmt.Printf("On week %d, you earned $%d.00 from %s\n", week, income.Amount, income.Source)
            }
        }(i, income)

here we get print final balance remaining same as previous programs ``` wg.Wait()

// print out final balance fmt.Printf("Final bank balance: $%d.00", bankBalance) fmt.Println() }


output

~/Documents/golabs ❯ go run . Initial account balance: $0.00 On week 1, you earned $100.00 from tenable On week 2, you earned $100.00 from tenable On week 3, you earned $100.00 from tenable On week 4, you earned $100.00 from tenable On week 5, you earned $100.00 from tenable On week 6, you earned $100.00 from tenable On week 7, you earned $100.00 from tenable On week 8, you earned $100.00 from tenable On week 9, you earned $100.00 from tenable On week 10, you earned $100.00 from tenable On week 11, you earned $100.00 from tenable On week 12, you earned $100.00 from tenable On week 13, you earned $100.00 from tenable On week 14, you earned $100.00 from tenable On week 15, you earned $100.00 from tenable On week 16, you earned $100.00 from tenable On week 1, you earned $50.00 from docker On week 2, you earned $50.00 from docker On week 3, you earned $50.00 from docker On week 4, you earned $50.00 from docker On week 5, you earned $50.00 from docker On week 6, you earned $50.00 from docker On week 7, you earned $50.00 from docker On week 8, you earned $50.00 from docker On week 9, you earned $50.00 from docker On week 10, you earned $50.00 from docker On week 11, you earned $50.00 from docker On week 12, you earned $50.00 from docker On week 13, you earned $50.00 from docker On week 14, you earned $50.00 from docker On week 15, you earned $50.00 from docker On week 16, you earned $50.00 from docker On week 17, you earned $50.00 from docker On week 18, you earned $50.00 from docker On week 19, you earned $50.00 from docker On week 20, you earned $50.00 from docker On week 21, you earned $50.00 from docker On week 22, you earned $50.00 from docker On week 23, you earned $50.00 from docker On week 24, you earned $50.00 from docker On week 25, you earned $50.00 from docker On week 26, you earned $50.00 from docker On week 27, you earned $50.00 from docker On week 28, you earned $50.00 from docker On week 29, you earned $50.00 from docker On week 30, you earned $50.00 from docker On week 31, you earned $50.00 from docker On week 32, you earned $50.00 from docker On week 33, you earned $50.00 from docker On week 34, you earned $50.00 from docker On week 35, you earned $50.00 from docker On week 36, you earned $50.00 from docker On week 37, you earned $50.00 from docker On week 38, you earned $50.00 from docker On week 39, you earned $50.00 from docker On week 40, you earned $50.00 from docker On week 41, you earned $50.00 from docker On week 42, you earned $50.00 from docker On week 43, you earned $50.00 from docker On week 44, you earned $50.00 from docker On week 45, you earned $50.00 from docker On week 46, you earned $50.00 from docker On week 1, you earned $500.00 from google On week 47, you earned $50.00 from docker On week 48, you earned $50.00 from docker On week 49, you earned $50.00 from docker On week 50, you earned $50.00 from docker On week 51, you earned $50.00 from docker On week 52, you earned $50.00 from docker On week 1, you earned $10.00 from amazon On week 2, you earned $10.00 from amazon On week 3, you earned $10.00 from amazon On week 4, you earned $10.00 from amazon On week 5, you earned $10.00 from amazon On week 17, you earned $100.00 from tenable On week 18, you earned $100.00 from tenable On week 19, you earned $100.00 from tenable On week 20, you earned $100.00 from tenable On week 21, you earned $100.00 from tenable On week 22, you earned $100.00 from tenable On week 23, you earned $100.00 from tenable On week 24, you earned $100.00 from tenable On week 25, you earned $100.00 from tenable On week 26, you earned $100.00 from tenable On week 27, you earned $100.00 from tenable On week 28, you earned $100.00 from tenable On week 29, you earned $100.00 from tenable On week 30, you earned $100.00 from tenable On week 31, you earned $100.00 from tenable On week 32, you earned $100.00 from tenable On week 33, you earned $100.00 from tenable On week 34, you earned $100.00 from tenable On week 35, you earned $100.00 from tenable On week 6, you earned $10.00 from amazon On week 7, you earned $10.00 from amazon On week 8, you earned $10.00 from amazon On week 9, you earned $10.00 from amazon On week 10, you earned $10.00 from amazon On week 11, you earned $10.00 from amazon On week 12, you earned $10.00 from amazon On week 13, you earned $10.00 from amazon On week 14, you earned $10.00 from amazon On week 2, you earned $500.00 from google On week 3, you earned $500.00 from google On week 4, you earned $500.00 from google On week 5, you earned $500.00 from google On week 6, you earned $500.00 from google On week 7, you earned $500.00 from google On week 8, you earned $500.00 from google On week 9, you earned $500.00 from google On week 10, you earned $500.00 from google On week 36, you earned $100.00 from tenable On week 37, you earned $100.00 from tenable On week 38, you earned $100.00 from tenable On week 39, you earned $100.00 from tenable On week 40, you earned $100.00 from tenable On week 41, you earned $100.00 from tenable On week 42, you earned $100.00 from tenable On week 43, you earned $100.00 from tenable On week 44, you earned $100.00 from tenable On week 45, you earned $100.00 from tenable On week 46, you earned $100.00 from tenable On week 47, you earned $100.00 from tenable On week 48, you earned $100.00 from tenable On week 49, you earned $100.00 from tenable On week 50, you earned $100.00 from tenable On week 51, you earned $100.00 from tenable On week 52, you earned $100.00 from tenable On week 11, you earned $500.00 from google On week 12, you earned $500.00 from google On week 13, you earned $500.00 from google On week 14, you earned $500.00 from google On week 15, you earned $500.00 from google On week 16, you earned $500.00 from google On week 17, you earned $500.00 from google On week 18, you earned $500.00 from google On week 19, you earned $500.00 from google On week 20, you earned $500.00 from google On week 15, you earned $10.00 from amazon On week 16, you earned $10.00 from amazon On week 17, you earned $10.00 from amazon On week 18, you earned $10.00 from amazon On week 19, you earned $10.00 from amazon On week 20, you earned $10.00 from amazon On week 21, you earned $10.00 from amazon On week 22, you earned $10.00 from amazon On week 23, you earned $10.00 from amazon On week 24, you earned $10.00 from amazon On week 25, you earned $10.00 from amazon On week 26, you earned $10.00 from amazon On week 27, you earned $10.00 from amazon On week 28, you earned $10.00 from amazon On week 29, you earned $10.00 from amazon On week 30, you earned $10.00 from amazon On week 31, you earned $10.00 from amazon On week 32, you earned $10.00 from amazon On week 33, you earned $10.00 from amazon On week 34, you earned $10.00 from amazon On week 35, you earned $10.00 from amazon On week 36, you earned $10.00 from amazon On week 37, you earned $10.00 from amazon On week 38, you earned $10.00 from amazon On week 39, you earned $10.00 from amazon On week 40, you earned $10.00 from amazon On week 41, you earned $10.00 from amazon On week 42, you earned $10.00 from amazon On week 43, you earned $10.00 from amazon On week 44, you earned $10.00 from amazon On week 45, you earned $10.00 from amazon On week 46, you earned $10.00 from amazon On week 47, you earned $10.00 from amazon On week 48, you earned $10.00 from amazon On week 49, you earned $10.00 from amazon On week 50, you earned $10.00 from amazon On week 51, you earned $10.00 from amazon On week 52, you earned $10.00 from amazon On week 21, you earned $500.00 from google On week 22, you earned $500.00 from google On week 23, you earned $500.00 from google On week 24, you earned $500.00 from google On week 25, you earned $500.00 from google On week 26, you earned $500.00 from google On week 27, you earned $500.00 from google On week 28, you earned $500.00 from google On week 29, you earned $500.00 from google On week 30, you earned $500.00 from google On week 31, you earned $500.00 from google On week 32, you earned $500.00 from google On week 33, you earned $500.00 from google On week 34, you earned $500.00 from google On week 35, you earned $500.00 from google On week 36, you earned $500.00 from google On week 37, you earned $500.00 from google On week 38, you earned $500.00 from google On week 39, you earned $500.00 from google On week 40, you earned $500.00 from google On week 41, you earned $500.00 from google On week 42, you earned $500.00 from google On week 43, you earned $500.00 from google On week 44, you earned $500.00 from google On week 45, you earned $500.00 from google On week 46, you earned $500.00 from google On week 47, you earned $500.00 from google On week 48, you earned $500.00 from google On week 49, you earned $500.00 from google On week 50, you earned $500.00 from google On week 51, you earned $500.00 from google On week 52, you earned $500.00 from google Final bank balance: $34320.00


lets write test case for above program

package main

import ( "io" "os" "strings" "testing" )

func Test_main(t *testing.T) {

stdOut := os.Stdout r, w, _ := os.Pipe() os.Stdout = w main()

_ = w.Close()

result, _ := io.ReadAll(r) output := string(result) os.Stdout = stdOut if !strings.Contains(output, "$34320.00") { t.Errorf("Expected output") }

} output ~/Documents/golabs ❯ go test . ok example2 0.464s

test race

~/Documents/golabs ❯ go test -race . ok example2 0.291s


#### 10 . Producer-Consumer problem

[Producer-Consumer problem](https://en.wikipedia.org/wiki/Producer–consumer_problem) 

here its take example of pizza as producer and consumer problem

package main

const NumberofPizzas = 10

var pizzaMade, pizzasFails, total int

type Producer struct { data chan Pizzaorder quit chan chan error }

type Pizzaorder struct { pizzaNumber int message string success bool }

func main() {

// seed the random number generator

// print out a message

// create a producer

// run the producer in the background

// create and run consumer

// print out ending messages

}

lets started with functions

package main

import ( "math/rand" "time"

color "github.com/fatih/color" )

const NumberofPizzas = 10

var pizzaMade, pizzasFails, total int

type Producer struct { data chan Pizzaorder quit chan chan error }

type Pizzaorder struct { pizzaNumber int message string success bool }

func (p *Producer) close() error { ch := make(chan error) p.quit <- ch return <-ch }

func pizzaria(pizzaMaker *Producer) { // keep track of the number of pizzas made // run forver or until we get a quit signal // try to make a pizza // if we can't make a pizza, send a message to the consumer for { // try to make pizza //decision }

}

func main() {

// seed the random number generator rand.Seed(time.Now().UnixNano())

// print out a message color.Cyan("Pizza delivery service") color.Cyan("======================")

// create a producer

pizzaJob := &Producer{ data: make(chan Pizzaorder), quit: make(chan chan error), }

// run the producer in the background go pizzaria(pizzaJob)

// create and run consumer

// print out ending messages

} ``` if I run this program

~/Documents/golabs ❯ go run .
Pizza delivery service


package main

import (
    "fmt"
    "math/rand"
    "time"

    "github.com/fatih/color"
)

const NumberOfPizzas = 10

var pizzasMade, pizzasFailed, total int

type Producer struct {
    data chan PizzaOrder
    quit chan chan error
}

type PizzaOrder struct {
    pizzaNumber int
    message     string
    success     bool
}

func (p *Producer) Close() error {
    ch := make(chan error)
    p.quit <- ch
    return <-ch
}

func makePizza(pizzaNumber int) *PizzaOrder {
    pizzaNumber++
    if pizzaNumber <= NumberOfPizzas {
        delay := rand.Intn(5) + 1
        fmt.Printf("Received order #%d!\n", pizzaNumber)

        rnd := rand.Intn(12) + 1
        msg := ""
        success := false

        if rnd < 5 {
            pizzasFailed++
        } else {
            pizzasMade++
        }
        total++

        fmt.Printf("Making pizza #%d. It will take %d seconds....\n", pizzaNumber, delay)
        // delay for a bit
        time.Sleep(time.Duration(delay) * time.Second)

        if rnd <= 2 {
            msg = fmt.Sprintf("*** We ran out of ingredients for pizza #%d!", pizzaNumber)
        } else if rnd <= 4 {
            msg = fmt.Sprintf("*** The cook quit while making pizza #%d!", pizzaNumber)
        } else {
            success = true
            msg = fmt.Sprintf("Pizza order #%d is ready!", pizzaNumber)
        }

        p := PizzaOrder{
            pizzaNumber: pizzaNumber,
            message:     msg,
            success:     success,
        }

        return &p

    }

    return &PizzaOrder{
        pizzaNumber: pizzaNumber,
    }
}

func pizzeria(pizzaMaker *Producer) {
    // keep track of which pizza we are making
    var i = 0

    // run forever or until we receive a quit notification
    // try to make pizzas
    for {
        currentPizza := makePizza(i)
        if currentPizza != nil {
            i = currentPizza.pizzaNumber
            select {
            case pizzaMaker.data <- *currentPizza:
            // we tried to make a pizza (send someting to data channel )
            case pizzaMaker.data <- *currentPizza:
                // we tried to make a pizza (send someting to data channel )
            case quitChan := <-pizzaMaker.quit:
                // we received a quit request
                close(pizzaMaker.data)
                close(quitChan)
                return

            }
        }

    }
}

func main() {
    // seed the random number generator
    rand.Seed(time.Now().UnixNano())

    // print out a message
    color.Cyan("Pizza delivery service!")
    color.Cyan("----------------------------------")

    // create a producer
    pizzaJob := &Producer{
        data: make(chan PizzaOrder),
        quit: make(chan chan error),
    }

    // run the producer in the background
    go pizzeria(pizzaJob)

    for i := range pizzaJob.data {
        if i.pizzaNumber <= NumberOfPizzas {
            if i.success {
                color.Green("%s", i.message)
                color.Green("order #%d is out of deliver !", i.pizzaNumber)
            } else {
                color.Red("%s", i.message)
                color.Red("customer not happy ")
            }
        } else {
            color.Cyan("done making pizzas")
            err := pizzaJob.Close()
            if err != nil {
                color.Red("Error closing channel ", err)
            }
        }
    }

    // create and run consumer

    // print out the ending message
}

output ```

~/Documents/golabs ❯ go run . 30s Pizza delivery service!

Received order #1! Making pizza #1. It will take 1 seconds.... Received order #2! Making pizza #2. It will take 3 seconds.... *** The cook quit while making pizza #1! customer not happy Received order #3! Making pizza #3. It will take 4 seconds.... Pizza order #2 is ready! order #2 is out of deliver ! Received order #4! Making pizza #4. It will take 2 seconds.... *** The cook quit while making pizza #3! customer not happy Received order #5! Making pizza #5. It will take 3 seconds.... Pizza order #4 is ready! order #4 is out of deliver ! Received order #6! Making pizza #6. It will take 3 seconds.... Pizza order #5 is ready! order #5 is out of deliver ! Received order #7! Making pizza #7. It will take 4 seconds.... Pizza order #6 is ready! order #6 is out of deliver ! Received order #8! Pizza order #7 is ready! order #7 is out of deliver ! Making pizza #8. It will take 5 seconds.... Received order #9! Making pizza #9. It will take 1 seconds.... *** The cook quit while making pizza #8! customer not happy Received order #10! Making pizza #10. It will take 2 seconds.... Pizza order #9 is ready! order #9 is out of deliver ! *** The cook quit while making pizza #10! customer not happy done making pizzas


if you see above output its not in sequence

package main

import ( "fmt" "math/rand" "time"

"github.com/fatih/color" )

const NumberOfPizzas = 10

var pizzasMade, pizzasFailed, total int

type Producer struct { data chan PizzaOrder quit chan chan error }

type PizzaOrder struct { pizzaNumber int message string success bool }

func (p *Producer) Close() error { ch := make(chan error) p.quit <- ch return <-ch }

func makePizza(pizzaNumber int) *PizzaOrder { pizzaNumber++ if pizzaNumber <= NumberOfPizzas { delay := rand.Intn(5) + 1 fmt.Printf("Received order #%d!\n", pizzaNumber)

rnd := rand.Intn(12) + 1 msg := "" success := false

if rnd < 5 { pizzasFailed++ } else { pizzasMade++ } total++

fmt.Printf("Making pizza #%d. It will take %d seconds....\n", pizzaNumber, delay) // delay for a bit time.Sleep(time.Duration(delay) * time.Second)

if rnd <= 2 { msg = fmt.Sprintf("*** We ran out of ingredients for pizza #%d!", pizzaNumber) } else if rnd <= 4 { msg = fmt.Sprintf("*** The cook quit while making pizza #%d!", pizzaNumber) } else { success = true msg = fmt.Sprintf("Pizza order #%d is ready!", pizzaNumber) }

p := PizzaOrder{ pizzaNumber: pizzaNumber, message: msg, success: success, }

return &p

}

return &PizzaOrder{ pizzaNumber: pizzaNumber, } }

func pizzeria(pizzaMaker *Producer) { // keep track of which pizza we are making var i = 0

// run forever or until we receive a quit notification // try to make pizzas for { currentPizza := makePizza(i) if currentPizza != nil { i = currentPizza.pizzaNumber select { case pizzaMaker.data <- *currentPizza: // we tried to make a pizza (send someting to data channel ) case pizzaMaker.data <- *currentPizza: // we tried to make a pizza (send someting to data channel ) case quitChan := <-pizzaMaker.quit: // we received a quit request close(pizzaMaker.data) close(quitChan) return

} }

func main() { // seed the random number generator rand.Seed(time.Now().UnixNano())

// print out a message color.Cyan("Pizza delivery service!") color.Cyan("----------------------------------")

// create a producer pizzaJob := &Producer{ data: make(chan PizzaOrder), quit: make(chan chan error), }

// run the producer in the background go pizzeria(pizzaJob)

for i := range pizzaJob.data { if i.pizzaNumber <= NumberOfPizzas { if i.success { color.Green("%s", i.message) color.Green("order #%d is out of deliver !", i.pizzaNumber) } else { color.Red("%s", i.message) color.Red("customer not happy ") } } else { color.Cyan("done making pizzas") err := pizzaJob.Close() if err != nil { color.Red("Error closing channel ", err) } } }

// print out the ending message color.Cyan("----------------------------------") color.Cyan("Done for the day !\n") color.Cyan("we made %d pizzas , but failed to make %d , with %d attempts in total ", pizzasMade, pizzasFailed, total) switch { case pizzasFailed > 9: color.Red("We are in trouble !") case pizzasFailed >= 6: color.Green("We are doing great !") case pizzasFailed >= 4: color.Yellow("We are doing okay !") case pizzasFailed >= 2: color.Red("We are pretty well !") default: color.Green("We are doing great !")

} }


final output :

~/Documents/golabs ❯ go run . 33s Pizza delivery service!

Received order #1! Making pizza #1. It will take 1 seconds.... Received order #2! Making pizza #2. It will take 4 seconds.... Pizza order #1 is ready! order #1 is out of deliver ! Received order #3! Making pizza #3. It will take 1 seconds.... * The cook quit while making pizza #2! customer not happy Received order #4! Making pizza #4. It will take 5 seconds.... Pizza order #3 is ready! order #3 is out of deliver ! Received order #5! Making pizza #5. It will take 5 seconds.... * We ran out of ingredients for pizza #4! customer not happy Received order #6! Making pizza #6. It will take 3 seconds.... * We ran out of ingredients for pizza #5! customer not happy Received order #7! Making pizza #7. It will take 4 seconds.... * The cook quit while making pizza #6! customer not happy Received order #8! Making pizza #8. It will take 1 seconds.... Pizza order #7 is ready! order #7 is out of deliver ! Received order #9! Making pizza #9. It will take 4 seconds.... Pizza order #8 is ready! order #8 is out of deliver ! Received order #10! Making pizza #10. It will take 4 seconds.... Pizza order #9 is ready! order #9 is out of deliver ! Pizza order #10 is ready! order #10 is out of deliver ! done making pizzas

Done for the day ! we made 6 pizzas , but failed to make 4 , with 10 attempts in total We are doing okay !


####  11 . Range , Buffered Channel

for value := range ch {

....

}

- iterate over values received from channel 
- loop automatically breaks when a channel is close 
- range does not return the second boolean value 


Range over the channel, the receiver goroutine can use range to receive a sequence of values from the channel. range over the channel will iterate over the values received from a channel.
The loop automatically breaks when the channel is closed. So once the sender goroutine has sent all of its values, it will close the channel and the receiver
goroutine will break out of the range loop. The range does not return the second boolean value.



##### 12 . Unbuffered channels 

synchronous channel 

![](https://raw.githubusercontent.com/cloudnativefolks/Graphics/main/unbuffered-channel.png)

package main

func main() { ch := make(chan int) go func() { for i := 0; i < 6; i++ { // send iterator to channel ch ch <- i } close(ch) }() // range over channel to receive values

for v := range ch { println(v) } }

output

~/Documents/golabs ❯ go run main.go 0 1 2 3 4 5


Normally the receive returns the second boolean value, but range just returns value, as on close, the range will automatically break out of the loop.
Unbuffered channels, the channels that we have been creating till now are unbuffered channels. There is no buffer between the sender goroutine and the receiver goroutine.
Since there is no buffer, the sender goroutine will block until there is a receiver, to receive the value, and the receiver goroutine will block until there is a sender, sending the value.



##### 13.  buffered channels 

![](https://raw.githubusercontent.com/cloudnativefolks/Graphics/main/buffered.png)

In buffered channels, there is a buffer between the sender and the receiver goroutine, and we can specify the capacity, that is the buffer size, which indicates the number of elements that can be sent without the receiver being ready to receive the values.
The sender can keep sending the values without blocking, till the buffer gets full, when the buffer gets full, the sender
will block.

The receiver can keep receiving the values without blocking till the buffer gets empty, when the buffer gets empty, the receiver will block.
The buffered channels are in-memory FIFO queues, so the element that is sent first, will be the element

package main

import "fmt"

func main() { ch := make(chan int, 6) go func() { for i := 0; i < 6; i++ { // send iterator to channel ch fmt.Printf("Sending %d to channel\n", i) ch <- i } close(ch) }() // range over channel to receive values

for v := range ch { fmt.Printf("Received %d from channel\n", v) println(v) } }

output

go run main.go Sending 0 to channel Sending 1 to channel Sending 2 to channel Sending 3 to channel Sending 4 to channel Sending 5 to channel Received 0 from channel 0 Received 1 from channel 1 Received 2 from channel 2 Received 3 from channel 3 Received 4 from channel 4 Received 5 from channel 5 ```

14 . Channel Direction

when using channels as function parameters , you can specify if a channel is meant to only send or receive values

this specificity increases the type-safety of the program


func pong( in <- chan string , out chan <- string){}

in <- chan string - receive only channel
out chan <- string - send only channel

package main

import "fmt"

func getMsg(ch1 chan<- string) {

    // send message to channel 1

    ch1 <- "Hello CloudNativeFolks"
}
func relayMsg(ch1 <-chan string, ch2 chan<- string) {

    // receive message on channel 1
    m := <-ch1
    ch2 <- m

    // send it on channel 2
}

func main() {
    // create channels ch1 and ch2
    ch1 := make(chan string)
    ch2 := make(chan string)

    // spine goroutine to getMsg() and relayMsg()
    go getMsg(ch1)
    go relayMsg(ch1, ch2)
    // recv message on ch2
    v := <-ch2
    fmt.Println(v)
}

output ``` go run main.go Hello CloudNativeFolks


#### 15 .Channel Ownership 


Default  value for channel is nil

var ch chan interface{}

reading / writing to a nil channel will block forever

var ch chan interface{} <-ch ch <- struct{}{} closing nil channel will panic

var ch chan interface {} close(ch) ``` ensure the channels are initialised first

owner of channel is a goroutine that instantiates writes and closes a channel channel utilisers only have a read-only view into the channel

ownership of channel avoids

deadlocking by writing to a nil channel
closing a nil channel
writing to a closed channel
closing a channel more than once

package main

import "fmt"

func main() {
    // return recieve only channel to caller
    // spin a goroutine , which
    // writes data into channel and
    // close the channel when done

    owner := func() <-chan int {
        ch := make(chan int)

        go func() {
            defer close(ch)
            for i := 0; i < 10; i++ {
                ch <- i
            }
        }()
        return ch
    }

    consumer := func(in <-chan int) {
        // read data from channel
        for v := range in {
            fmt.Printf("received: %d\n", v)
        }
        fmt.Println("done")
    }
    ch := owner()
    consumer(ch)
}

output go run main.go received: 0 received: 1 received: 2 received: 3 received: 4 received: 5 received: 6 received: 7 received: 8 received: 9 done

16 .Pipeline

Pipline streams or batches of data

Go's concurrency primitives makes it easy to construct streaming pipelines. That enables us to make an efficient use of the I/O and the multiple CPU cores available on the machine, to run our computation faster. Pipelines are often used to process streams or batches of data. Pipeline is a series of stages that are connected by the channels, where each stage is represented by a goroutine.

A goroutine takes the data from an in-bound channel, performs an operation on it and sends the data

on the out-bound channel, that can be used by the next stage. By using pipelines, we can separate the concerns of each stage, and process individual stages concurrently. Stages could consume and return the same type. For example, a square stage can take, receive only channel of type int and return receive only channelof type int as output. func square(in <-chan int) <-chan int { This enables the composability of the pipeline. square(square (generator(2,3)))

For example, a generator stage can return a receiver only channel of type int, which a square stage can take as input, and we can compose the output of the square stage as input to another square stage.


package main

func generator(nums ...int) <-chan int {

    out := make(chan int)
    go func() {
        for _, n := range nums {
            out <- n
        }
        close(out)
    }()
    return out
}

// square - receive on inbound channel
// square the number
//output on outbound channel
func square(in <-chan int) <-chan int {
    out := make(chan int)
    go func() {
        for n := range in {
            out <- n * n
        }
        close(out)
    }()
    return out
}

func main() {

    //    ch := generator(2, 3)
    //    out := square(ch)
    //    for n := range out {
    //        println(n)
    //    }

    for n := range square(square(generator(2, 3))) {
        println(n)
    }
    // setup pipeline
    // run the last stage of pipeline
    // receive the value from square stage
    // print each value , until channel is closed
}

output ``` go run main.go 16 81


#### 17 . fan out & fan in 

![](https://raw.githubusercontent.com/cloudnativefolks/Graphics/main/fan-out-fan-in.png) 

what is fan-out?

- multiple goroutine are started read data from the single channel 

- Distribute work amongst a group of workers coroutines to parallelise the CPU usage and the I/O usages 

- Helps computational intensive stage to run faster 

What is fan In ?

- Process ion combining multiple results into one channel 

- we create merge go routine to read data from multiple input channel and send the data to a single output channel .

package main

import ( "sync" )

func generator(nums ...int) <-chan int {

out := make(chan int) go func() { for _, n := range nums { out <- n } close(out) }() return out }

// square - receive on inbound channel // square the number //output on outbound channel func square(in <-chan int) <-chan int { out := make(chan int) go func() { for n := range in { out <- n * n } close(out) }() return out }

func merge(cs ...<-chan int) <-chan int { // implement fan in // merge a list of channels into a single channel out := make(chan int) var wg sync.WaitGroup output := func(c <-chan int) { for n := range c { out <- n } wg.Done() } wg.Add(len(cs)) for _, c := range cs { go output(c) } go func() { wg.Wait() close(out) }() return out }

func main() {

// ch := generator(2, 3) // out := square(ch) // for n := range out { // println(n) // }

in := generator(2, 3) ch1 := square(in) ch2 := square(in)

for n := range merge(ch1, ch2) { println(n) } // setup pipeline // run the last stage of pipeline // receive the value from square stage // print each value , until channel is closed }

output

go run main.go 4 9


#### 18 .cancelling go routine 

downstream stages keep receiving values from inbound channel until the channel is closed .

func square(in <-chan int) <-chan int { out := make(chan int) go func() { for n := range in { out <- n * n } close(out) }() return out }


All go routines exit once all values have been successfully sent downstream


-  Real Pipelines 

- Real Pipelines - Receiver stages may only need a subset of values to make progress 
- A stage can exit early because an inbound value represents an error in an earlier stage
- Receiver should not have to wait for the remaining values to arrives 
- We want earlier stages to stop producing values that later stages don't need 

![](https://raw.githubusercontent.com/cloudnativefolks/Graphics/main/goroutineleak.png)


Cancellation of goroutine 

- Pass a read-only 'done' channel to goroutine 
- Close the channel, to send broadcasts signal to all goroutine 
-  on receiving the signal on done channel , Goroutines needs to abandon their work and terminate 
- we use 'select` to make send/receive operation on channel pre-emptible

select { case out <- n: case <-done return }

package main

import ( "fmt" "runtime" "sync" "time" )

func generator(done <-chan struct{}, nums ...int) <-chan int {

out := make(chan int) go func() { defer close(out) for _, n := range nums { select { case out <- n: case <-done: return } } }() return out }

func square(done <-chan struct{}, in <-chan int) <-chan int { out := make(chan int) go func() { defer close(out) for n := range in { select { case out <- n * n: case <-done: return } }

}() return out }

func merge(done <-chan struct{}, cs ...<-chan int) <-chan int { // implement fan in // merge a list of channels into a single channel out := make(chan int) var wg sync.WaitGroup output := func(c <-chan int) { defer wg.Done() for n := range c { select { case out <- n: case <-done: return } } } wg.Add(len(cs)) for _, c := range cs { go output(c) } go func() { wg.Wait() close(out) }() return out }

func main() {

done := make(chan struct{}) in := generator(done, 2, 3)

ch1 := square(done, in) ch2 := square(done, in)

out := merge(done, ch1, ch2) fmt.Println(<-out) close(done) time.Sleep(10 * time.Millisecond) g := runtime.NumGoroutine() fmt.Printf("number of active go routines = %d\n", g)

}


output :-

~/Documents/golabs ❯ go run main.go 4 number of active go routines = 1


#### 19 . Context Package 


we often need to manage goroutines because of timeouts , cancellations or failures in related goroutine 

also we needs to pass request-specific values across API boundaries and between processes

the content package serves two primary purpose 
 - to provide an API for Canceling branches of your call-graph 
- to provide a data-bag for transporting request scoped data through your call-graph 

Cancelation

func WithCancel(parent Context)(ctx Context, cancelFunc)

func WithDeadline(parent Context,deadline time.Time)(Context , cancelFunc)

func WithTimeOut(parent Context, timeout time.Duration)(Conrxr , cancelFunc) ```

WithCancel returns a Context that closed its done channel when the cancel function is called
WithDeadline return a Context the closes it done channel when the machine's clock passed the given deadline
WithTimeout returns a Context that closes its done channel after the given time duration

Value

func WithValue(parent Context, Key , val interface{} ) Context 

value(key interface{}) interface

WithValue returns a copy of parent context in which the value associated with key
value returns the associated with this context for key or nil if no value associated with key
use context Values only for request-specific data , not for passing optionals parameters to functions

20 . What are the limits of channels

Message Size

the maximum message size ( orhannel type ) is 2^16 bytes 0r 64 kilobytes

// https://github.com/golang/go/blob/release-branch.go1.18/src/runtime/chan.go#L72
func makechan(t *chantype, size int) *hchan {
    elem := t.elem

    // compiler checks this but be safe.
    if elem.size >= 1<<16 {
        throw("makechan: invalid channel element type")
    }
....
...
}

Maximum allocation

The maximum allocation that is allowed by the compile 2^47 byte(140 Terabyte) in 64 bit unix like system

// https://github.com/golang/go/blob/release-branch.go1.18/src/runtime/malloc.go#L225 
    // maxAlloc is the maximum size of an allocation. On 64-bit,
    // it's theoretically possible to allocate 1<<heapAddrBits bytes. On
    // 32-bit, however, this is one less than 1<<32 because the
    // number of bytes in the address space doesn't actually fit
    // in a uintptr.
    maxAlloc = (1 << heapAddrBits) - (1-_64bit)*1

but you'll meet memory issues if you will try to allocate maximum size

Speed

the time for the speed send and receive operations os dominated by the price of goroutine context switching ( which should be consistently <+ 200ms

go-ch go test -bench=.

unbuffered channels are slightly faster then buffered

Introduction To Golang - Part 2

Comments

GopherLabs