In this part we will cover standard libraries
1. Retrieving the current working directory
Another useful source of information for the application is the directory, where the program binary is located. With this information, the program can access the assets and files collocated with the binary file.
Note: Go since version 1.8. This one is the preferred one.
Create the main.go file with the following content:
package main
import (
"fmt"
"os"
"path/filepath"
)
func main() {
ex, err := os.Executable()
if err != nil {
panic(err)
}
fmt.Println(ex)
exPath := filepath.Dir(ex)
fmt.Println("Executable path :" + exPath)
realPath, err := filepath.EvalSymlinks(exPath)
if err != nil {
panic(err)
}
fmt.Println("Symlink evaluated:" + realPath)
}
output:
Biradars-MacBook-Air-4:golang-daily sangam$ go build -o binary
Biradars-MacBook-Air-4:golang-daily sangam$ ./binary
/Users/sangam/Documents/golang-daily/binary
Executable path :/Users/sangam/Documents/golang-daily
Symlink evaluated:/Users/sangam/Documents/golang-daily
Biradars-MacBook-Air-4:golang-daily sangam$
How it works…
Since Go 1.8, the Executable function from the os package is the preferred way of resolving the path of the executable. The Executable function returns the absolute path of the binary that is executed (unless the error is returned).
To resolve the directory from the binary path, the Dir from the filepath package is applied. The only pitfall of this is that the result could be the symlink or the path it pointed to.
To overcome this unstable behavior, the EvalSymlinks from the filepath package could be applied to the resultant path. With this hack, the returned value would be the real path of the binary.
The information about the directory where the binary is located could be obtained with the use of the Executable function in the os library.
Note that if the code is run by the command go run, the actual executable is located in a temporary directory.
2 .Getting the current process PID
Getting to know the PID of the running process is useful. The PID could be used by OS utilities to find out the information about the process itself. It is also valuable to know the PID in case of process failure, so you can trace the process behavior across the system in system logs, such as /var/log/messages, /var/log/syslog.
This program shows you how to use the os package to obtain a PID of the executed program, and use it with the operating system utility to obtain some more information.
package main
import (
"fmt"
"os"
"os/exec"
"strconv"
)
func main() {
pid := os.Getpid()
fmt.Printf("Process PID: %d \n", pid)
prc := exec.Command("ps", "-p", strconv.Itoa(pid), "-v")
out, err := prc.Output()
if err != nil {
panic(err)
}
fmt.Println(string(out))
}
output:
Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go Process PID: 48094 PID STAT TIME SL RE PAGEIN VSZ RSS LIM TSIZ %CPU %MEM COMMAND 48094 S+ 0:00.01 0 0 0 4373552 2072 - 0 0.0 0.0 /var/folders/mg/\_355pdvd741cz0z99ys9s66h0000gn/T/go-build581430461/b001/exe/main
```rust
How it works…
- The function Getpid from the os package returns the PID of a process. The sample code shows how to get more information on the process from the operating system utility ps.
- It could be useful to print the PID at the start of the application, so at the time of the crash, the cause could also be investigated by the retrieved PID.
#### 3 . Handling operating system signals
- Signals are the elementary way the operating systems communicate with the running process. Two of the most usual signals are called SIGINT and SIGTERM.These cause the program to terminate.
- There are also signals such as SIGHUP. SIGHUP indicates that the terminal which called the process was closed and, for example, the program could decide to move to the background.
- Go provides a way of handling the behavior in case the application received the signal. This recipe will provide an example of implementing the handling.
package main
import ( "fmt" "os" "os/signal" "syscall" )
func main() {
// Create the channel where the received // signal would be sent. The Notify // will not block when the signal // is sent and the channel is not ready. // So it is better to // create buffered channel. sChan := make(chan os.Signal, 1)
// Notify will catch the // given signals and send // the os.Signal value // through the sChan. // If no signal specified in // argument, all signals are matched. signal.Notify(sChan, syscall.SIGHUP, syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT)
// Create channel to wait till the // signal is handled. exitChan := make(chan int) go func() { signal := <-sChan switch signal { case syscall.SIGHUP: fmt.Println("The calling terminal has been closed") exitChan <- 0
case syscall.SIGINT: fmt.Println("The process has been interrupted by CTRL+C") exitChan <- 1
case syscall.SIGTERM: fmt.Println("kill SIGTERM was executed for process") exitChan <- 1
case syscall.SIGQUIT: fmt.Println("kill SIGQUIT was executed for process") exitChan <- 1 } }()
code := <-exitChan os.Exit(code) } ``` output:
Note: after running program press CTRL+C
Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go
^CThe process has been interrupted by CTRL+C
exit status 1
Biradars-MacBook-Air-4:golang-daily sangam$
How it works…
In an application, where the resources are acquired, a resource leak could happen in the case of an instant termination. It is better to handle the signals and take some necessary steps to release the resources. The preceding code shows the concept of how to do that.
The Notify function from the signal package would be the one that helps us to handle the received signals.
Note that the Notify function of the signal package is communicating with the goroutine by the sChan channel. Notify then catches the defined signals and sends these to goroutine to be handled. Finally, exitChan is used to resolve the exit code of the process.
The important information is that the Notify function will not block the signal if the assigned channel is not ready. This way the signal could be missed. To avoid missing the signal, it is better to create the buffered channel.
Note that the SIGKILL and SIGSTOP signals may not be caught by the Notify function, thus it is not possible to handle these.
4 . calling an external proces
The Go binary could also be used as a tool for various utilities and with use of go run as a replacement for the bash script. For these purposes, it is usual that the command-line utilities are called.
In this recipe, the basics of how to execute and handle the child process will be provided.
Create the run.go file with the following content:
package main
import (
"bytes"
"fmt"
"os/exec"
)
func main() {
prc := exec.Command("ls", "-a")
out := bytes.NewBuffer([]byte{})
prc.Stdout = out
err := prc.Run()
if err != nil {
fmt.Println(err)
}
if prc.ProcessState.Success() {
fmt.Println("Process run successfully with output:\n")
fmt.Println(out.String())
}
}
output: ``` Biradars-MacBook-Air-4:golang-daily sangam$ go run run.go Process run successfully with output:
. .. binary main main.go run.go test util
How it works…
- The Go standard library provides a simple way of calling the external process. This could be done by the Command function of the os/exec package.
- The simplest way is to create the Cmd struct and call the Run function. The Run function executes the process and waits until it completes. If the command exited with an error, the err value is not null.
- This is more suitable for calling the OS utils and tools, so the program does not hang too long.
- The process could be executed asynchronously too. This is done by calling the Start method of the Cmd structure. In this case, the process is executed, but the main goroutine does not wait until it ends. The Wait method could be used to wait until the process ends. After the Wait method finishes, the resources of the process are released.
- This approach is more suitable for executing long-running processes and services that the program depends on.
#### 5. Retrieving child process information
- The recipe Calling an external process describes how to call the child process, synchronously and asynchronously. Naturally, to handle the process behavior you need to find out more about the process. This recipe shows how to obtain the PID and elementary information about the child process after it terminates.
- The information about the running process could be obtained only via the syscall package and it is highly platform-dependent.
Getting ready
- Test if the sleep (timeout for Windows) command exists in the Terminal.
Create the `main.go` file with the following content:
package main
import ( "fmt" "os/exec" "runtime" "time" )
func main() {
var cmd string if runtime.GOOS == "windows" { cmd = "timeout" } else { cmd = "sleep" }
proc := exec.Command(cmd, "1") proc.Start()
// Wait function will // wait till the process ends. proc.Wait()
// After the process terminates // the *os.ProcessState contains // simple information // about the process run fmt.Printf("PID: %d\n", proc.ProcessState.Pid()) fmt.Printf("Process took: %dms\n", proc.ProcessState.SystemTime()/time.Microsecond) fmt.Printf("Exited sucessfuly : %t\n", proc.ProcessState.Success()) }
output:
Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go PID: 46936 Process took: 1406ms Exited sucessfuly : true
How it works…
- The os/exec standard library provides the way to execute the process. Using Command, the Cmd structure is returned. The Cmd provides the access to process the representation. When the process is running, you can only find out the PID.
- There is only a little information that you can retrieve about the process. But by retrieving the PID of the process, you are able to call the utilities from the OS to get more information.
> Note
Remember that it is possible to obtain the PID of the child process, even if it is running. On the other hand, the ProcessState structure of the os package is available, only after the process terminates.
#### 6 . Reading writing from the child process
- Every process, that is executed, has the standard output, input and error output. The Go standard library provides the way to read and write to these.
- This recipe will walk through the approaches on how to read the output and write to the input of the child process.
Create the `main_read_output.go` file with the following content:
package main
import ( "fmt" "os/exec" "runtime" )
func main() {
var cmd string
if runtime.GOOS == "windows" { cmd = "dir" } else { cmd = "ls" }
proc := exec.Command(cmd)
// Output will run the process // terminates and returns the standard // output in a byte slice. buff, err := proc.Output()
if err != nil { panic(err) }
// The output of child // process in form // of byte slice // printed as string fmt.Println(string(buff))
}
output
Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go binary main main.go run.go start.go test util
Create the main_read_stdout.go file with the following content:
package main
import ( "bytes" "fmt" "os/exec" "runtime" )
func main() {
var cmd string
if runtime.GOOS == "windows" { cmd = "dir" } else { cmd = "ls" }
proc := exec.Command(cmd)
buf := bytes.NewBuffer([]byte{})
// The buffer which implements // io.Writer interface is assigned to // Stdout of the process proc.Stdout = buf
// To avoid race conditions // in this example. We wait till // the process exit. proc.Run()
// The process writes the output to // to buffer and we use the bytes // to print the output. fmt.Println(string(buf.Bytes()))
}
output
Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go binary main main.go run.go start.go test
Create the `main_read_read.go` file with the following content:
package main
import ( "bufio" "context" "fmt" "os/exec" "time" )
func main() { cmd := "ping" timeout := 2 * time.Second
// The command line tool // "ping" is executed for // 2 seconds ctx, _ := context.WithTimeout(context.TODO(), timeout) proc := exec.CommandContext(ctx, cmd, "example.com")
// The process output is obtained // in form of io.ReadCloser. The underlying // implementation use the os.Pipe stdout, _ := proc.StdoutPipe() defer stdout.Close()
// Start the process proc.Start()
// For more comfortable reading the // bufio.Scanner is used. // The read call is blocking. s := bufio.NewScanner(stdout) for s.Scan() { fmt.Println(s.Text()) } }
output
Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go PING example.com (93.184.216.34): 56 data bytes 64 bytes from 93.184.216.34: icmp_seq=0 ttl=51 time=411.571 ms 64 bytes from 93.184.216.34: icmp_seq=1 ttl=51 time=217.384 ms Biradars-MacBook-Air-4:golang-daily sangam$
Create the sample.go file with the following content:
package main
import ( "bufio" "fmt" "os" )
func main() { sc := bufio.NewScanner(os.Stdin)
for sc.Scan() { fmt.Println(sc.Text()) } }
Create the main.go file with the following content:
package main
import ( "bufio" "fmt" "io" "os/exec" "time" )
func main() { cmd := []string{"go", "run", "sample.go"}
// The command line tool // "ping" is executed for // 2 seconds proc := exec.Command(cmd[0], cmd[1], cmd[2])
// The process input is obtained // in form of io.WriteCloser. The underlying // implementation use the os.Pipe stdin, _ := proc.StdinPipe() defer stdin.Close()
// For debugging purposes we watch the // output of the executed process stdout, _ := proc.StdoutPipe() defer stdout.Close()
go func() { s := bufio.NewScanner(stdout) for s.Scan() { fmt.Println("Program says:" + s.Text()) } }()
// Start the process proc.Start()
// Now the following lines // are written to child // process standard input fmt.Println("Writing input") io.WriteString(stdin, "Hello\n") io.WriteString(stdin, "Golang\n") io.WriteString(stdin, "is awesome\n")
time.Sleep(time.Second * 2)
proc.Process.Kill()
}
output
Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go Writing input Program says:Hello Program says:Golang Program says:is awesome Biradars-MacBook-Air-4:golang-daily sangam$
How it works…
- The Cmd structure of the os/exec package provides the functions to access the output/input of the process. There are a few approaches to read the output of the process.
- One of the simplest ways to read the process output is to use the Output or CombinedOutput method of the Cmd structure (gets Stderr and Stdout). While calling this function, the program synchronously waits till the child process terminates and then returns the output to a byte buffer.
- Besides the Output and OutputCombined methods, the Cmd structure provides the Stdout property, where the io.Writer could be assigned. The assigned writer then serves as a destination for the process output. It could be a file, byte buffer or any type implementing the io.Writer interface.
- The last approach to read the process output is to get the io.Reader from the Cmd structure by calling the StdoutPipe method. The StdoutPipe method creates the pipe between the Stdout, where the process writes the output, and provides Reader which works as the interface for the program to read the process output. This way the output of the process is piped to the retrieved io.Reader .
- Writing to a process stdin works the same way. Of all the options, the one with io.Writer will be demonstrated.
- As could be seen, there are a few ways to read and write from the child process. The use of stderr and stdin is almost the same as described in steps 6-7. Finally, the approach of how to access the input/output could be divided this way:
- Synchronous (wait until the process ends and get the bytes): The Output and CombinedOutput methods of Cmd are used.
- IO: The output or input are provided in the form of io.Writer/Reader. The XXXPipe and StdXXX properties are the right ones for this approach.
- The IO type is more flexible and could also be used asynchronously.
#### 7 . Shutting down the application gracefully
- Servers and daemons are the programs that run for a long time (typically days or even weeks). These long-running programs usually allocate resources (database connections, network sock) at the start and keep these resources as long as they exist. If such a process is killed and the shutdown is not handled properly, a resource leak could happen. To avoid that behavior, the so-called graceful shutdown should be implemented.
- Graceful, in this case, means that the application catches the termination signal, if possible, and tries to clean up and release the allocated resources before it terminates. This recipe will show you how to implement the graceful shutdown.
- The recipe, Handling operating system signals describes the catching of OS signals. The same approach will be used for implementing the graceful shutdown. Before the program terminates, it will clean up and carry out some other activities.
Create the `main.go` file with the following content:
package main
import ( "fmt" "io" "log" "os" "os/signal" "syscall" "time" )
var writer *os.File
func main() {
// The file is opened as // a log file to write into. // This way we represent the resources // allocation. var err error writer, err = os.OpenFile(fmt.Sprintf("test_%d.log", time.Now().Unix()), os.O_RDWR|os.O_CREATE, os.ModePerm) if err != nil { panic(err) }
// The code is running in a goroutine // independently. So in case the program is // terminated from outside, we need to // let the goroutine know via the closeChan closeChan := make(chan bool) go func() { for { time.Sleep(time.Second) select { case <-closeChan: log.Println("Goroutine closing") return default: log.Println("Writing to log") io.WriteString(writer, fmt.Sprintf("Logging access %s\n", time.Now().String())) }
} }()
sigChan := make(chan os.Signal, 1) signal.Notify(sigChan, syscall.SIGTERM, syscall.SIGQUIT, syscall.SIGINT)
// This is blocking read from // sigChan where the Notify function sends // the signal. <-sigChan
// After the signal is received // all the code behind the read from channel could be // considered as a cleanup close(closeChan) releaseAllResources() fmt.Println("The application shut down gracefully") }
func releaseAllResources() { io.WriteString(writer, "Application releasing all resources\n") writer.Close() }
output
Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go 2019/12/02 03:45:13 Writing to log 2019/12/02 03:45:14 Writing to log 2019/12/02 03:45:16 Writing to log 2019/12/02 03:45:17 Writing to log 2019/12/02 03:45:18 Writing to log 2019/12/02 03:45:19 Writing to log 2019/12/02 03:45:20 Writing to log 2019/12/02 03:45:21 Writing to log 2019/12/02 03:45:22 Writing to log 2019/12/02 03:45:23 Writing to log 2019/12/02 03:45:24 Writing to log 2019/12/02 03:45:25 Writing to log 2019/12/02 03:45:26 Writing to log 2019/12/02 03:45:27 Writing to log ^CThe application shut down gracefully Biradars-MacBook-Air-4:golang-daily sangam$ ```
also cat logs by following commands :
Biradars-MacBook-Air-4:golang-daily sangam$ cat test_1575238512.log
Logging access 2019-12-02 03:45:13.996115 +0530 IST m=+1.004724107
Logging access 2019-12-02 03:45:14.998721 +0530 IST m=+2.007300282
Logging access 2019-12-02 03:45:16.001896 +0530 IST m=+3.010445676
Logging access 2019-12-02 03:45:17.289594 +0530 IST m=+4.298105159
Logging access 2019-12-02 03:45:18.290419 +0530 IST m=+5.298899897
Logging access 2019-12-02 03:45:19.294715 +0530 IST m=+6.303165286
Logging access 2019-12-02 03:45:20.295224 +0530 IST m=+7.303644943
Logging access 2019-12-02 03:45:21.298422 +0530 IST m=+8.306812201
Logging access 2019-12-02 03:45:22.301538 +0530 IST m=+9.309898565
Logging access 2019-12-02 03:45:23.303273 +0530 IST m=+10.311603579
Logging access 2019-12-02 03:45:24.306044 +0530 IST m=+11.314343818
Logging access 2019-12-02 03:45:25.309499 +0530 IST m=+12.317769379
Logging access 2019-12-02 03:45:26.311555 +0530 IST m=+13.319795135
Logging access 2019-12-02 03:45:27.314678 +0530 IST m=+14.322888324
Application releasing all resources
How it works…
The reading from a sigChan is blocking so the program keeps running until the Signal is sent through the channel. The sigChan is the channel where the Notify function sends the signals.
The main code of the program runs in a new goroutine. This way, the work continues while the main function is blocked on the sigChan. Once the signal from operation system is sent to process, the sigChan receives the signal and the code below the line where the reading from the sigChan channel is executed. This code section could be considered as the cleanup section.
8 . file configuration with functional options
- This recipe is not directly related to the Go standard library but includes how to handle an optional configuration for your application. The recipe will use the functional options pattern in a real case with a file configuration.
Create the main.go file with the following content:
package main
import (
"encoding/json"
"fmt"
"os"
)
type Client struct {
consulIP string
connString string
}
func (c *Client) String() string {
return fmt.Sprintf("ConsulIP: %s , Connection String: %s",
c.consulIP, c.connString)
}
var defaultClient = Client{
consulIP: "localhost:9000",
connString: "postgres://localhost:5432",
}
type ConfigFunc func(opt *Client)
func FromFile(path string) ConfigFunc {
return func(opt *Client) {
f, err := os.Open(path)
if err != nil {
panic(err)
}
defer f.Close()
decoder := json.NewDecoder(f)
fop := struct {
ConsulIP string `json:"consul_ip"`
}{}
err = decoder.Decode(&fop)
if err != nil {
panic(err)
}
opt.consulIP = fop.ConsulIP
}
}
func FromEnv() ConfigFunc {
return func(opt *Client) {
connStr, exist := os.LookupEnv("CONN_DB")
if exist {
opt.connString = connStr
}
}
}
func NewClient(opts ...ConfigFunc) *Client {
client := defaultClient
for _, val := range opts {
val(&client)
}
return &client
}
func main() {
client := NewClient(FromFile("config.json"), FromEnv())
fmt.Println(client.String())
}
In the same folder, create the file config.json with content:
{
"consul_ip":"127.0.0.1"
}
output : ``` Biradars-MacBook-Air-4:golang-daily sangam$ CONN_DB=oracle://local:5921 go run main.go ConsulIP: 127.0.0.1 , Connection String: oracle://local:5921 Biradars-MacBook-Air-4:golang-daily sangam$
How it works...
- The core concept of the functional options pattern is that the configuration API contains the functional parameters. In this case, the NewClient function accepts a various number of ConfigFunc arguments, which are then applied one by one on the defaultClient struct. This way, the default configuration is modified with huge flexibility.
- See the FromFile and FromEnv functions, which return the ConfigFunc, that is in fact, accessing the file or environmental variables.
- Finally, you can check the output which applied both the configuration options and resulting Client struct that contains the values from the file and environmental variables.
#### 9 . Finding the substring in a string
- Finding the substring in a string is one of the most common tasks for developers. Most of the mainstream languages implement this in a standard library. Go is not an exception. This recipe describes the way Go implements this.
Create the main.go file with the following content:
package main
import ( "fmt" "strings" )
const refString = "Mary had a little lamb"
func main() {
lookFor := "lamb" contain := strings.Contains(refString, lookFor) fmt.Printf("The "%s" contains "%s": %t \n", refString, lookFor, contain)
lookFor = "wolf" contain = strings.Contains(refString, lookFor) fmt.Printf("The "%s" contains "%s": %t \n", refString, lookFor, contain)
startsWith := "Mary" starts := strings.HasPrefix(refString, startsWith) fmt.Printf("The "%s" starts with "%s": %t \n", refString, startsWith, starts)
endWith := "lamb" ends := strings.HasSuffix(refString, endWith) fmt.Printf("The "%s" ends with "%s": %t \n", refString, endWith, ends)
}
output:-
Biradars-MacBook-Air-4:Documents sangam$ cd golang-daily/ Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go The "Mary had a little lamb" contains "lamb": true The "Mary had a little lamb" contains "wolf": false The "Mary had a little lamb" starts with "Mary": true The "Mary had a little lamb" ends with "lamb": true Biradars-MacBook-Air-4:golang-daily sangam$ ``` How it works...
The Go library strings contain functions to handle the string operations. This time the function Contains could be used. The Contains function simply checks whether the string has a given substring. In fact, the function Index is used in Contains function.
To check whether the string begins with the substring, the HasPrefix function is there. To check whether the string ends with the substring, the function HasSuffix will work.
In fact, the Contains function is implemented by use of the Index function from the same package. As you can guess, the actual implementation works like this: if the index of the given substring is greater than -1, the Contains function returns true.
The HasPrefix and HasSuffix functions work in a different way: the internal implementation just checks the length of both the string and substring, and if they are equal or the string is longer, the required part of the string is compared.
10 . breaking the string into words
- Breaking the string into words could be tricky. First, decide what the word is, as well as what the separator is, and if there is any whitespace or any other characters. After these decisions have been made, you can choose the appropriate function from the strings package. This recipe will describe common cases.
Create the main.go file with the following content:
package main
import (
"fmt"
"strings"
)
const refString = "Mary had a little lamb"
func main() {
words := strings.Fields(refString)
for idx, word := range words {
fmt.Printf("Word %d is: %s\n", idx, word)
}
}
output: ``` Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go Word 0 is: Mary Word 1 is: had Word 2 is: a Word 3 is: little Word 4 is: lamb Biradars-MacBook-Air-4:golang-daily sangam$
Create the main.go file with the following content:
package main
import ( "fmt" "strings" )
const refString = "Mary_had a little_lamb"
func main() {
words := strings.Split(refString, "_") for idx, word := range words { fmt.Printf("Word %d is: %s\n", idx, word) }
}
output:
Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go Word 0 is: Mary Word 1 is: had a little Word 2 is: lamb Biradars-MacBook-Air-4:golang-daily sangam$
Create another file called main.go with the following content:
package main
import ( "fmt" "strings" )
const refString = "Mary*had,a%little_lamb"
func main() {
// The splitFunc is called for each // rune in a string. If the rune // equals any of character in a "%,_" // the refString is split. splitFunc := func(r rune) bool { return strings.ContainsRune("%,_", r) }
words := strings.FieldsFunc(refString, splitFunc) for idx, word := range words { fmt.Printf("Word %d is: %s\n", idx, word) }
}
output :
Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go Word 0 is: Mary Word 1 is: had Word 2 is: a Word 3 is: little Word 4 is: lamb Biradars-MacBook-Air-4:golang-daily sangam$
Create another file called main.go with the following content:
this problems asked in many interviews to understand your basics conects around strings
package main
import ( "fmt" "regexp" )
const refString = "Mary*had,a%little_lamb"
func main() {
words := regexp.MustCompile("[*,%_]{1}").Split(refString, -1) for idx, word := range words { fmt.Printf("Word %d is: %s\n", idx, word) }
}
output:
Biradars-MacBook-Air-4:golang-daily sangam$ go run main.go Word 0 is: Mary Word 1 is: had Word 2 is: a Word 3 is: little Word 4 is: lamb Biradars-MacBook-Air-4:golang-daily sangam$
How it works...
The simplest form of how to split the string into words considers any whitespace as a separator. In detail, the whitespace is defined by the IsSpace function in the unicode package:
'\t', '\n', '\v', '\f', '\r', ' ', U+0085 (NEL), U+00A0 (NBSP).
- The Fields function of the strings package could be used to split the sentence by the whitespace chars as mentioned earlier. The steps 1 – 5 cover this first simple case.
- If any other separator is needed, the Split function comes into play. Splitting by another separator is covered in steps 6 – 8. Just note that the whitespace in the string is omitted.
- If you need a more complex function to decide whether to split the string at a given point, FieldsFunc could work for you. One of the function's argument is the function that consumes the rune of the given string and returns true if the string should split at that point. This option is covered by steps 9 – 11.
- The regular expression is the last option mentioned in the example. The Regexp structure of the regexp package contains the Split method, which works as you would expect. It splits the string in the place of the matching group. This approach is used in steps 12 – 14.
There's more...
- The strings package also provides the various SplitXXX functions that could help you to achieve more specific tasks.
#### 11 . Joining the string slice with a separator
- The recipe, Breaking the string into words, led us through the task of splitting the single string into substrings, according to defined rules. This recipe, on the other hand, describes how to concatenate the multiple strings into a single string with a given string as the separator.
- A real use case could be the problem of dynamically building a SQL select statement condition.
Create the join.go file with the following content:
package main
import ( "fmt" "strings" )
const selectBase = "SELECT * FROM user WHERE %s "
var refStringSlice = []string{ " FIRST_NAME = 'Jack' ", " INSURANCE_NO = 333444555 ", " EFFECTIVE_FROM = SYSDATE "}
func main() {
sentence := strings.Join(refStringSlice, "AND") fmt.Printf(selectBase+"\n", sentence)
}
ouput:
sangam$ go run join.go SELECT * FROM user WHERE FIRST_NAME = 'Jack' AND INSURANCE_NO = 333444555 AND EFFECTIVE_FROM = SYSDATE
Biradars-MacBook-Air-4:pico sangam$
Create the join_manually.go file with the following content:
package main
import ( "fmt" "strings" )
const selectBase = "SELECT * FROM user WHERE "
var refStringSlice = []string{ " FIRST_NAME = 'Jack' ", " INSURANCE_NO = 333444555 ", " EFFECTIVE_FROM = SYSDATE "}
type JoinFunc func(piece string) string
func main() {
jF := func(p string) string { if strings.Contains(p, "INSURANCE") { return "OR" }
return "AND" } result := JoinWithFunc(refStringSlice, jF) fmt.Println(selectBase + result) }
func JoinWithFunc(refStringSlice []string, joinFunc JoinFunc) string { concatenate := refStringSlice[0] for _, val := range refStringSlice[1:] { concatenate = concatenate + joinFunc(val) + val } return concatenate }
output:
$ go run join_manually.go SELECT * FROM user WHERE FIRST_NAME = 'Jack' OR INSURANCE_NO = 333444555 AND EFFECTIVE_FROM = SYSDATE Biradars-MacBook-Air-4:pico sangam$ ``` How it works...
For the purpose of joining the string slice into a single string, the Join function of the strings package is there. Simply, you need to provide the slice with strings that are needed to be joined. This way you can comfortably join the string slices. The use of the Join function is shown in steps 1 – 5.
Naturally, the joining could be implemented manually by iterating over the slice. This way you can customize the separator by some more complex logic. The steps 6 – 8 just represent how the manual concatenation could be used with more complex decision logic, based on the string that is currently processed.
There's more...
The Join function is provided by the bytes package, which naturally serves to join the slice of bytes
12. Concatenating a string with writer
- Besides the built-in + operator, there are more ways to concatenate the string. This recipe will describe the more performant way of concatenating strings with the bytes package and the built-in copy function.
Create the concat_buffer.go file with the following content:
package main
import (
"bytes"
"fmt"
)
func main() {
strings := []string{"This ", "is ", "even ",
"more ", "performant "}
buffer := bytes.Buffer{}
for _, val := range strings {
buffer.WriteString(val)
}
fmt.Println(buffer.String())
}
output:
sangam:golang-daily sangam$ go run concat_buffer.go
This is even more performant
sangam:golang-daily sangam$
Create the concat_copy.go file with the following content:
package main
import (
"fmt"
)
func main() {
strings := []string{"This ", "is ", "even ",
"more ", "performant "}
bs := make([]byte, 100)
bl := 0
for _, val := range strings {
bl += copy(bs[bl:], []byte(val))
}
fmt.Println(string(bs[:]))
}
output:
sangam:golang-daily sangam$ go run concat_copy.go
This is even more performant
sangam:golang-daily sangam$
How it works...
The steps 1 - 5 cover the use of the bytes package Buffer as a performance-friendly solution to string concatenation. The Buffer structure implements the WriteString method, which could be used to effectively concatenate the strings into an underlying byte slice.
There is no need to use this improvement in all situations, just think about this in cases where the program is going to concatenate a big number of strings (for example, in-memory CSV exports and others).
The built-in copy function presented in steps 6 - 8 could be used to accomplish the string concatenation. This method requires some assumption about the final string length, or it could be done on the fly. However, if the capacity of the buffer, where the result is written, is smaller than the sum of the already written part and the string to be appended, the buffer must be expanded (usually by the allocation of a new slice with bigger capacity).
13. Aligning text with tabwriter
- In certain cases, the output (usually data output) is done via tabbed text, which is formatted in well-arranged cells. This format could be achieved with the text/tabwriter package. The package provides the Writer filter, which transforms the text with the tab characters into properly formatted output text.
Create the main.go file with the following content:
package main
import (
"fmt"
"os"
"text/tabwriter"
)
func main() {
w := tabwriter.NewWriter(os.Stdout, 15, 0, 1, ' ',
tabwriter.AlignRight)
fmt.Fprintln(w, "username\tfirstname\tlastname\t")
fmt.Fprintln(w, "sohlich\tRadomir\tSohlich\t")
fmt.Fprintln(w, "novak\tJohn\tSmith\t")
w.Flush()
}
output:
sangam:golang-daily sangam$ go run main.go
username firstname lastname
sohlich Radomir Sohlich
novak John Smith
sangam:golang-daily sangam$
How it works...
The NewWriter function call creates the Writer filter with configured parameters. All data written by this Writer is formatted according to the parameters. os.Stdout is used here for demonstration purposes.
The text/tabwriter package also provides a few more configuration options, such as the flag parameter. The most useful is tabwriter.AlignRight, which configures the writer to align the content to the right in each column.
14. Replacing part of the string
- Another very common task related to string processing is the replacement of the substring in a string. Go standard library provide the Replace function and Replacer type for the replacement of multiple strings at once.
Create the replace.go file with the following content:
package main
import (
"fmt"
"strings"
)
const refString = "Mary had a little lamb"
const refStringTwo = "lamb lamb lamb lamb"
func main() {
out := strings.Replace(refString, "lamb", "wolf", -1)
fmt.Println(out)
out = strings.Replace(refStringTwo, "lamb", "wolf", 2)
fmt.Println(out)
}
output:
sangam:golang-daily sangam$ go run replace.go
Mary had a little wolf
wolf wolf lamb lamb
sangam:golang-daily sangam$
Create the replacer.go file with the following content:
package main
import (
"fmt"
"strings"
)
const refString = "Mary had a little lamb"
func main() {
replacer := strings.NewReplacer("lamb", "wolf", "Mary", "Jack")
out := replacer.Replace(refString)
fmt.Println(out)
}
output: ``` sangam:golang-daily sangam$ go run replacer.go Jack had a little wolf sangam:golang-daily sangam$
Create the regexp.go file with the following content:
package main
import ( "fmt" "regexp" )
const refString = "Mary had a little lamb"
func main() { regex := regexp.MustCompile("l[a-z]+") out := regex.ReplaceAllString(refString, "replacement") fmt.Println(out) }
output:
sangam:golang-daily sangam$ go run regexp.go Mary had a replacement replacement sangam:golang-daily sangam$
How it works...
- The Replace function of a strings package is widely used for simple replacement. The last integer argument defines how many replacements will be done (in case of -1, all strings are replaced. See the second use of Replace, where only the first two occurrences are replaced.) The use of the Replace function is presented in steps 1 - 5.
- Besides the Replace function, the Replacer structure also has the WriteString method. This method will write to the given writer with all replacements defined in Replacer. The main purpose of this type is its reusability. It can replace multiple strings at once and it is safe for concurrent use; see steps 6 - 8.
- The more sophisticated method of replacing the substring, or even the matched pattern, is naturally the use of the regular expression. The Regex type pointer method ReplaceAllString could be leveraged for this purpose. Steps 9 - 11 illustrate the use of the regexp package.
There's more...
- If more complex logic for the replacement is needed, the regexp package is probably the one that should be used.
#### 15 .finding the substring in text by the regex pattern
- There are always tasks such as validating the input, searching the document for any information, or even cleaning up a given string from unwanted escape characters. For these cases, regular expressions are usually used.
- The Go standard library contains the regexp package, which covers the operations with regular expressions.
Create the main.go file with the following content:
package main
import ( "fmt" "regexp" )
const refString = [{ \"email\": \"email@example.com\" \"phone\": 555467890}, { \"email\": \"other@domain.com\" \"phone\": 555467890}]
func main() {
// This pattern is simplified for brevity emailRegexp := regexp.MustCompile("[a-zA-Z0-9]{1,}@[a-zA-Z0-9]{1,}\.[a-z]{1,}") first := emailRegexp.FindString(refString) fmt.Println("First: ") fmt.Println(first)
all := emailRegexp.FindAllString(refString, -1) fmt.Println("All: ") for _, val := range all { fmt.Println(val) }
}
output:-
sangam:golang-daily sangam$ go run main.go First: email@example.com All: email@example.com other@domain.com sangam:golang-daily sangam$
How it works...
- The FindString or FindAllString functions are the simplest ways to find the matching pattern in the given string. The only difference is that the FindString method of Regexp will return only the first occurrence. On the other hand, the FindAllString, as the name suggests, returns a slice of strings with all occurrences.
- The Regexp type offers a rich set of FindXXX methods. This recipe describes only the String variations that are usually most useful. Note that the preceding code uses the MustCompile function of the regexp package, which panics if the compilation of the regular expression fails.
#### 16 .controlling case
- There are a lot of practical tasks where the modification of case is the most common approach. Let's pick a few of these:
- Case-insensitive comparison
- Beginning the sentence with an automatic first capital
- Camel-case to snake-case conversion
For these purposes, the strings package offers functions ToLower, ToUpper, ToTitle, and Title.
How to do it...
Create the main.go file with the following content:
package main
import ( "fmt" "strings" "unicode" )
const email = "ExamPle@domain.com" const name = "isaac newton" const upc = "upc" const i = "i"
const snakeCase = "first_name"
func main() {
// For comparing the user input // sometimes it is better to // compare the input in a same // case. input := "Example@domain.com" input = strings.ToLower(input) emailToCompare := strings.ToLower(email) matches := input == emailToCompare fmt.Printf("Email matches: %t\n", matches)
upcCode := strings.ToUpper(upc) fmt.Println("UPPER case: " + upcCode)
// This digraph has different upper case and // title case. str := "dz" fmt.Printf("%s in upper: %s and title: %s \n", str, strings.ToUpper(str), strings.ToTitle(str))
// Use of XXXSpecial function title := strings.ToTitle(i) titleTurk := strings.ToTitleSpecial(unicode.TurkishCase, i) if title != titleTurk { fmt.Printf("ToTitle is defferent: %#U vs. %#U \n", title[0], []rune(titleTurk)[0]) }
// In some cases the input // needs to be corrected in case. correctNameCase := strings.Title(name) fmt.Println("Corrected name: " + correctNameCase)
// Converting the snake case // to camel case with use of // Title and ToLower functions. firstNameCamel := toCamelCase(snakeCase) fmt.Println("Camel case: " + firstNameCamel)
}
func toCamelCase(input string) string { titleSpace := strings.Title(strings.Replace(input, "_", " ", -1)) camel := strings.Replace(titleSpace, " ", "", -1) return strings.ToLower(camel[:1]) + camel[1:] }
output:
sangam:golang-daily sangam$ go run main.go Email matches: true UPPER case: UPC dz in upper: DZ and title: Dz ToTitle is defferent: U+0049 'I' vs. U+0130 'İ' Corrected name: Isaac Newton Camel case: firstName sangam:golang-daily sangam$
How it works...
- Note that the title-case mapping in Unicode differs from the uppercase mapping. The difference is that the number of characters requires special handling. These are mainly ligatures and digraphs such as fl, dz, and lj, plus a number of polytonic Greek characters. For example, U+01C7 (LJ) maps to U+01C8 (Lj) rather than to U+01C9 (lj).
#### 17 . parsing comma-separated data
- There are multiple table data formats. CSV (comma-separated values) is one of the most basic formats largely used for data transport and export. There is no standard that defines CSV, but the format itself is described in RFC 4180.
- This recipe introduces how to parse CSV-formatted data comfortably.
Create a file named data.csv with the following content:
"Name","Surname","Age"
"sangam","biradar",24 arjun,singh,21
Create the main.go file with the following content:
package main
import ( "encoding/csv" "fmt" "os" )
func main() {
file, err := os.Open("data.csv") if err != nil { panic(err) } defer file.Close()
reader := csv.NewReader(file) reader.FieldsPerRecord = 3 reader.Comment = '#'
for { record, e := reader.Read() if e != nil { fmt.Println(e) break } fmt.Println(record) } }
output:
sangam:golang-daily sangam$ go run main.go [Name Surname Age] [sangam biradar 24] [arjun singh 21] EOF sangam:golang-daily sangam$
Create a file named data_uncommon.csv with the following content:
Name;Surname;Age "sangam";biradar;24 "arjun";singh;21
Create a file named main.go with the following content:
package main
import ( "encoding/csv" "fmt" "os" )
func main() {
file, err := os.Open("data_uncommon.csv") if err != nil { panic(err) } defer file.Close()
reader := csv.NewReader(file) reader.Comma = ';'
for { record, e := reader.Read() if e != nil { fmt.Println(e) break } fmt.Println(record) } }
output:
sangam:golang-daily sangam$ go run main.go [Name Surname Age] [sangam biradar 24] [arjun singh 21] EOF sangam:golang-daily sangam$ ``` How it works...
Instead of simply scanning the input line by line and using strings.Split and other methods to parse the CSV format, Go offers a better way. The NewReader function in the encoding/csv package returns the Reader structure, which provides the API to read the CSV file. The Reader struct keeps variables to configure the read parameters, according to your needs.
The FieldsPerRecord parameter of Reader is a significant setting. This way the cell count per row could be validated. By default, when set to 0, it is set to the number of records in a first line. If a positive value is set, the number of records must match. If a negative value is set, there is no cell count validation.
Another interesting configuration is the Comment parameter, which allows you to define the comment characters in the parsed data. In the example, a whole line is ignored this way.
18. managing whitespace in a string
- The string input could contain too much whitespace, too little whitespace, or unsuitable whitespace chars. This recipe includes tips on how to manage these and format the string to your needs.
Create a file named main.go with the following content:
package main
import (
"fmt"
"math"
"regexp"
"strconv"
"strings"
)
func main() {
stringToTrim := "\t\t\n Go \tis\t Awesome \t\t"
trimResult := strings.TrimSpace(stringToTrim)
fmt.Println(trimResult)
stringWithSpaces := "\t\t\n Go \tis\n Awesome \t\t"
r := regexp.MustCompile("\\s+")
replace := r.ReplaceAllString(stringWithSpaces, " ")
fmt.Println(replace)
needSpace := "need space"
fmt.Println(pad(needSpace, 14, "CENTER"))
fmt.Println(pad(needSpace, 14, "LEFT"))
}
func pad(input string, padLen int, align string) string {
inputLen := len(input)
if inputLen >= padLen {
return input
}
repeat := padLen - inputLen
var output string
switch align {
case "RIGHT":
output = fmt.Sprintf("% "+strconv.Itoa(-padLen)+"s", input)
case "LEFT":
output = fmt.Sprintf("% "+strconv.Itoa(padLen)+"s", input)
case "CENTER":
bothRepeat := float64(repeat) / float64(2)
left := int(math.Floor(bothRepeat)) + inputLen
right := int(math.Ceil(bothRepeat))
output = fmt.Sprintf("% "+strconv.Itoa(left)+"s%
"+strconv.Itoa(right)+"s", input, "")
}
return output
}
output:
sangam:golang-daily sangam$ go run main.go
Go is Awesome
Go is Awesome
need space
need space
sangam:golang-daily sangam$
How it works...
Trimming the string before it is handled by the code is pretty common practice, and as the preceding code demonstrates, it is easily done by the standard Go library. The strings library also provides more variations of the TrimXXX function, which also allows the trimming of other chars from the string.
To trim the leading and ending whitespace, the TrimSpace function of the strings package can be used. This typifies the following part of a code, which was also included in the example earlier: stringToTrim := "\t\t\n Go \tis\t Awesome \t\t" stringToTrim = strings.TrimSpace(stringToTrim)
The regex package is suitable for replacing multiple spaces and tabs, and the string can be prepared for further processing this way. Note that, with this method, the break lines are replaced with a single space.
This part of the code represents the use of the regular expression to replace all multiple whitespaces with a single space:
r := regexp.MustCompile("\\s+")
replace := r.ReplaceAllString(stringToTrim, " ")
- Padding is not an explicit function for the strings package, but it can be achieved by the Sprintf function of the fmt package. The pad function in code uses the formatting pattern % <+/-padding>s and some simple math to find out the padding. Finally, the minus sign before the padding number works as the right pad, and the positive number as the left pad.
19. indenting a text document
- The previous recipe how to do string padding and whitespace trimming. This one will guide you through the indentation and unindentation of a text document. Similar principles from the previous recipes will be used.
Create the file main.go with the following content:
package main
import (
"fmt"
"strconv"
"strings"
"unicode"
)
func main() {
text := "Hi! Go is awesome."
text = Indent(text, 6)
fmt.Println(text)
text = Unindent(text, 3)
fmt.Println(text)
text = Unindent(text, 10)
fmt.Println(text)
text = IndentByRune(text, 10, '.')
fmt.Println(text)
}
func IndentByRune(input string, indent int, r rune) string {
return strings.Repeat(string(r), indent) + input
}
func Indent(input string, indent int) string {
padding := indent + len(input)
return fmt.Sprintf("% "+strconv.Itoa(padding)+"s", input)
}
func Unindent(input string, indent int) string {
count := 0
for _, val := range input {
if unicode.IsSpace(val) {
count++
}
if count == indent || !unicode.IsSpace(val) {
break
}
}
return input[count:]
}
output: sangam:golang-daily sangam$ go run main.go Hi! Go is awesome. Hi! Go is awesome. Hi! Go is awesome. ..........Hi! Go is awesome. sangam:golang-daily sangam$
How it works...
The indentation is as simple as padding. In this case, the same formatting option is used. The more readable form of the indent implementation could use the Repeat function of the strings package. The IndentByRune function in the preceding code applies this approach.
Unindenting, in this case, means removing the given count of leading spaces. The implementation of Unindent in the preceding code removes the minimum number of leading spaces or given indentation.
20. converting strings to numbers
Create the main.go file with the following content:
package main
import (
"fmt"
"strconv"
)
const bin = "00001"
const hex = "2f"
const intString = "12"
const floatString = "12.3"
func main() {
res, err := strconv.Atoi(intString)
if err != nil {
panic(err)
}
fmt.Printf("Parsed integer: %d\n", res)
res64, err := strconv.ParseInt(hex, 16, 32)
if err != nil {
panic(err)
}
fmt.Printf("Parsed hexadecima: %d\n", res64)
resBin, err := strconv.ParseInt(bin, 2, 32)
if err != nil {
panic(err)
}
fmt.Printf("Parsed bin: %d\n", resBin)
resFloat, err := strconv.ParseFloat(floatString, 32)
if err != nil {
panic(err)
}
fmt.Printf("Parsed float: %.5f\n", resFloat)
}
output:
sangam:golang-daily sangam$ go run main.go
Parsed integer: 12
Parsed hexadecima: 47
Parsed bin: 1
Parsed float: 12.30000
sangam:golang-daily sangam$
How it works...
The dominant function in the preceding sample code is the ParseInt function of package strconv. The function is called with three arguments: input, the base of input, and bit size. The base determines how the number is parsed. Note that the hexadecimal has the base (second argument) of 16 and the binary has the base of 2. The function Atoi of package strconv is, in fact, the ParseInt function with the base of 10.
The ParseFloat function converts the string to a floating-point number. The second argument is the precision of bitSize. bitSize = 64 will result in float64. bitSize = 32 will result in float64, but it is convertible to float32 without changing its value.
21. comparing floating-point numbers
- Because of how floating-point numbers are represented, there can be inconsistencies while comparing two numbers that appear to be identical. Unlike integers, IEEE floating-point numbers are only approximated. The need to convert the numbers to a form the computer can store in binary leads to minor precision or round-off deviations. For example, a value of 1.3 could be represented as 1.29999999999. The comparison could be done with some tolerance. To compare numbers with arbitrary precision, the big package is here.
Create the tolerance.go file with the following content:
package main
import (
"fmt"
"math"
)
const da = 0.29999999999999998889776975374843459576368331909180
const db = 0.3
func main() {
daStr := fmt.Sprintf("%.10f", da)
dbStr := fmt.Sprintf("%.10f", db)
fmt.Printf("Strings %s = %s equals: %v \n", daStr,
dbStr, dbStr == daStr)
fmt.Printf("Number equals: %v \n", db == da)
fmt.Printf("Number equals with TOLERANCE: %v \n",
equals(da, db))
}
const TOLERANCE = 1e-8
func equals(numA, numB float64) bool {
delta := math.Abs(numA - numB)
if delta < TOLERANCE {
return true
}
return false
}
output:
sangam:golang-daily sangam$ go run tolerance.go
Strings 0.3000000000 = 0.3000000000 equals: true
Number equals: false
Number equals with TOLERANCE: true
sangam:golang-daily sangam$
Create the file big.go with the following content:
package main
import (
"fmt"
"math/big"
)
var da float64 = 0.299999992
var db float64 = 0.299999991
var prec uint = 32
var prec2 uint = 16
func main() {
fmt.Printf("Comparing float64 with '==' equals: %v\n", da == db)
daB := big.NewFloat(da).SetPrec(prec)
dbB := big.NewFloat(db).SetPrec(prec)
fmt.Printf("A: %v \n", daB)
fmt.Printf("B: %v \n", dbB)
fmt.Printf("Comparing big.Float with precision: %d : %v\n",
prec, daB.Cmp(dbB) == 0)
daB = big.NewFloat(da).SetPrec(prec2)
dbB = big.NewFloat(db).SetPrec(prec2)
fmt.Printf("A: %v \n", daB)
fmt.Printf("B: %v \n", dbB)
fmt.Printf("Comparing big.Float with precision: %d : %v\n",
prec2, daB.Cmp(dbB) == 0)
}
output:
sangam:golang-daily sangam$ go run big.go
Comparing float64 with '==' equals: false
A: 0.299999992
B: 0.299999991
Comparing big.Float with precision: 32 : false
A: 0.3
B: 0.3
Comparing big.Float with precision: 16 : true
sangam:golang-daily sangam$
How it works...
The first approach for the floating-point numbers comparison without the use of any built-in package (steps 1-5) requires the use of a so-called EPSILON constant. This is the value chosen to be a sufficient small delta (difference) between two numbers to consider the values as equal. The delta constant could be on the order of 1e-8, which is usually sufficient precision.
The second option is more complex, but also more useful for further work with floating-point numbers. The package math/big offers the Float type that could be configured for a given precision. The advantage of this package is that the precision could be much higher than the precision of the float64 type. For illustrative purposes, the small precision values were used to show the rounding and comparison in the given precision.
Note that the da and db numbers are equal when using the precision of 16-bits and not equal when using the precision of 32-bits. The maximal configurable precision can be obtained from the big.MaxPrec constant.
22 .rounding floating-point numbers
The rounding of a floating-point number to an integer or to a particular precision has to be done properly. The most common error is to cast the floating-point type float64 to an integer type and consider it as well-handled.
An example could be casting the number 3.9999 to an integer and expect it to become an integer of value 4. The real result would be 3. ], in version of Go (1.9.2) does not contain the Round function. However, in version 1.10, the Round function was already implemented in the math package.
Create the round.go file with the following content:
package main
import (
"fmt"
"math"
)
var valA float64 = 3.55554444
func main() {
intVal := int(valA)
fmt.Printf("Bad rounding by casting to int: %v\n", intVal)
fRound := Round(valA)
fmt.Printf("Rounding by custom function: %v\n", fRound)
}
func Round(x float64) float64 {
t := math.Trunc(x)
if math.Abs(x-t) >= 0.5 {
return t + math.Copysign(1, x)
}
return t
}
Go playground
Bad rounding by casting to int: 3
Rounding by custom function: 4
How it works ...
Casting the float to integer actually just truncates the float value. Let's say the value 2 is represented as 1.999999; in this case, the output would be 1, which is not what you expected.
The proper way of rounding the float number is to use the function that would also consider the decimal part. The commonly used method of rounding is to half away from zero (also known as commercial rounding). Put simply, if the number contains the absolute value of the decimal part which is greater or equal to 0.5, the number is rounded up, otherwise, it is rounded down.
In the function Round, the function Trunc of package math truncates the decimal part of the number. Then, the decimal part of the number is extracted. If the value exceeds the limit of 0.5 than the value of 1 with the same sign as the integer value is added.
23. floating-point arithmetics
- As described in previous recipes, the representation of the floating-point numbers also complicates the arithmetic. For general purposes, the operations on the built-in float64 are sufficient. In case more precision is needed, the math/big package comes into play. This recipe will show you how to handle this.
package main
import (
"fmt"
"math/big"
)
const (
PI = `3.1415926535897932384626433832795028841971693
993751058209749445923078164062862089986280348253
421170679821480865132823066470938446095505822317
253594081284811174502841027019385211055596446229
4895493038196`
diameter = 3.0
precision = 400
)
func main() {
pi, _ := new(big.Float).SetPrec(precision).SetString(PI)
d := new(big.Float).SetPrec(precision).SetFloat64(diameter)
circumference := new(big.Float).Mul(pi, d)
pi64, _ := pi.Float64()
fmt.Printf("Circumference big.Float = %.400f\n",
circumference)
fmt.Printf("Circumference float64 = %.400f\n", pi64*diameter)
sum := new(big.Float).Add(pi, pi)
fmt.Printf("Sum = %.400f\n", sum)
diff := new(big.Float).Sub(pi, pi)
fmt.Printf("Diff = %.400f\n", diff)
quo := new(big.Float).Quo(pi, pi)
fmt.Printf("Quocient = %.400f\n", quo)
}
output: sangam:golang-daily sangam$ go run main.go Circumference big.Float = 9.4247779607693797153879301498385086525915081981253174629248337769234492188586269958841044760263512039 Circumference float64 = 9.4247779607693793479938904056325554847717285156250000000000000000000000000000000000000000000000000000 Sum = 6.2831853071795864769252867665590057683943387987502116419498891846156328125724179972560696506842341360 Diff = 0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 Quocient = 1.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 How it works...
The big package provides support for the arithmetic of floating-point numbers with high precision. The previous example illustrates the basic operations over the numbers. Note that the code compares the operation with the float64 type and the big.Float type.
By working with numbers with a high precision, it is crucial to use the big.Float type. When big.Float is converted back to the built-in float64 type, high precision is lost.
The big package contains more operations of the Float type. See the documentation (https://golang.org/pkg/math/big/#Float) of this package for more details.
- If the numbers are converted to the string, they usually need to be reasonably formatted. The formatting of a number means the number is printed with a given number, made up of digits and decimals. The representation of a value can also be chosen. A closely related problem with this, however, is the localization of number formatting. For example, some languages use comma-separated zeros.
Create the format.go file with the following content:
package main
import (
"fmt"
)
var integer int64 = 32500
var floatNum float64 = 22000.456
func main() {
fmt.Printf("%d \n", integer)
fmt.Printf("%+d \n", integer)
fmt.Printf("%X \n", integer)
fmt.Printf("%#X \n", integer)
fmt.Printf("%010d \n", integer)
fmt.Printf("% 10d \n", integer)
fmt.Printf("% -10d \n", integer)
fmt.Printf("%f \n", floatNum)
fmt.Printf("%.5f \n", floatNum)
fmt.Printf("%e \n", floatNum)
fmt.Printf("%g \n", floatNum)
}
output:
sangam:golang-daily sangam$ go run format.go
32500
+32500
7EF4
0X7EF4
0000032500
32500
32500
22000.456000
22000.45600
2.200046e+04
22000.456
Create the file localized.go with the following content:
package main
import (
"golang.org/x/text/language"
"golang.org/x/text/message"
)
const num = 100000.5678
func main() {
p := message.NewPrinter(language.English)
p.Printf(" %.2f \n", num)
p = message.NewPrinter(language.German)
p.Printf(" %.2f \n", num)
}
output:
sangam:golang-daily sangam$ go run localized.go
100,000.57
100.000,57
sangam:golang-daily sangam$
How it works...
The code example shows the most commonly used options for integers and floating-point numbers.
Note:The formatting in Go is derived from C's printf function. The so-called verbs are used to define the formatting of a number. The verb, for example, could be %X, which in fact is a placeholder for the value.
Besides the basic formatting, there are also rules in formatting that are related to the local manners. With formatting, according to the locale, the package golang.org/x/text/message could help. See the second code example in this recipe. This way, it is possible to localize the number formatting.
There's more...
- For all formatting options, see the fmt package. The strconv package could also be useful in case you are looking to format numbers in a different base. The following recipe describes the possibility of number conversion, but as a side effect, the options of how to format numbers in a different base are presented
25. Converting between binary, octal, decimal, and hexadecimal
- In some cases, the integer values can be represented by other than decimal representations. The conversion between these representations is easily done with the use of the strconv package
Create the main.go file with the following content: ``` package main
import ( "fmt" "strconv" )
const bin = "10111" const hex = "1A" const oct = "12" const dec = "10" const floatNum = 16.123557
func main() {
// Converts binary value into hex v, _ := ConvertInt(bin, 2, 16) fmt.Printf("Binary value %s converted to hex: %s\n", bin, v)
// Converts hex value into dec v, _ = ConvertInt(hex, 16, 10) fmt.Printf("Hex value %s converted to dec: %s\n", hex, v)
// Converts oct value into hex v, _ = ConvertInt(oct, 8, 16) fmt.Printf("Oct value %s converted to hex: %s\n", oct, v)
// Converts dec value into oct v, _ = ConvertInt(dec, 10, 8) fmt.Printf("Dec value %s converted to oct: %s\n", dec, v)
//... analogically any other conversion // could be done.
}
// ConvertInt converts the given string value of base // to defined toBase. func ConvertInt(val string, base, toBase int) (string, error) { i, err := strconv.ParseInt(val, base, 64) if err != nil { return "", err } return strconv.FormatInt(i, toBase), nil }
output:
sangam:golang-daily sangam$ go run main.go Binary value 10111 converted to hex: 17 Hex value 1A converted to dec: 26 Oct value 12 converted to hex: a Dec value 10 converted to oct: 12 sangam:golang-daily sangam$
How it works...
- The strconv package provides the functions ParseInt and FormatInt which are the, let's say, complementary functions. The function ParseInt is able to parse the integer number in any base representation. The function FormatInt, on the other hand, can format the integer into any given base.
- Finally, it is possible to parse the string representation of the integer to the built-in int64 type and subsequently, format the string of the parsed integer into the given base representation.
#### 26. formatting with the correct plurals
- When displaying messages for the user, the interaction is more pleasant if the sentences feel more human.
The Go package golang.org/x/text, which is the extension package, contains this feature for formatting plurals
in the correct way.
Getting ready
- Execute go get -x golang.org/x/text to obtain the extension package in case you don't have it already.
sangam:golang-daily sangam$ go get -x golang.org/x/text WORK=/var/folders/mg/_355pdvd741cz0z99ys9s66h0000gn/T/go-build380769388 mkdir -p $WORK/b001/ cat >$WORK/b001/importcfg << 'EOF' # internal
import config
EOF cd /Users/sangam/go/src/golang.org/x/text /usr/local/go/pkg/tool/darwin_amd64/compile -o $WORK/b001/pkg.a -trimpath "$WORK/b001=>" -p golang.org/x/text -complete -buildid Z8aQLviy4QTdfm0L5UnR/Z8aQLviy4QTdfm0L5UnR -goversion go1.13.4 -D "" -importcfg $WORK/b001/importcfg -pack -c=4 ./doc.go /usr/local/go/pkg/tool/darwin_amd64/buildid -w $WORK/b001/pkg.a # internal cp $WORK/b001/pkg.a /Users/sangam/Library/Caches/go-build/05/05d6df64800e241eb0cc32e8cdacdcdead37e6c5e5edfdb1b20826824e255b13-d # internal mkdir -p /Users/sangam/go/pkg/darwin_amd64/golang.org/x/ mv $WORK/b001/pkg.a /Users/sangam/go/pkg/darwin_amd64/golang.org/x/text.a rm -r $WORK/b001/ sangam:golang-daily sangam$ go get -x golang.org/x/text WORK=/var/folders/mg/_355pdvd741cz0z99ys9s66h0000gn/T/go-build711045050 sangam:golang-daily sangam$
Create the `plurals.go` file with the following content:
package main
import ( "golang.org/x/text/feature/plural" "golang.org/x/text/language" "golang.org/x/text/message" )
func main() {
message.Set(language.English, "%d items to do", plural.Selectf(1, "%d", "=0", "no items to do", plural.One, "one item to do", "<100", "%[1]d items to do", plural.Other, "lot of items to do", ))
message.Set(language.English, "The average is %.2f", plural.Selectf(1, "%.2f", "<1", "The average is zero", "=1", "The average is one", plural.Other, "The average is %[1]f ", ))
prt := message.NewPrinter(language.English) prt.Printf("%d items to do", 0) prt.Println() prt.Printf("%d items to do", 1) prt.Println() prt.Printf("%d items to do", 10) prt.Println() prt.Printf("%d items to do", 1000) prt.Println()
prt.Printf("The average is %.2f", 0.8) prt.Println() prt.Printf("The average is %.2f", 1.0) prt.Println() prt.Printf("The average is %.2f", 10.0) prt.Println()
}
output:
sangam:golang-daily sangam$ go run plurals.go no items to do one item to do 10 items to do lot of items to do The average is zero The average is one The average is 10.000000 sangam:golang-daily sangam$ ``` How it works...
The package golang.org/x/text/message contains the function NewPrinter which accepts the language identification and creates the formatted I/O, the same as the fmt package does, but with the ability to translate messages based on gender and plural forms.
The Set function of the message package adds the translation and plurals selection. The plural form itself is selected based on rules set via the Selectf function. The Selectf function produces the catalog.Message type with rules based on the plural.Form or selector.
The preceding sample code uses plural.One and plural.Other forms, and =x, <x selectors. These are matched against the formatting verb %d (other verbs can also be used). The first matching case is chosen.
There's more...
For more information about the selectors and forms, see the documentation for the (golang.org/x/text/message package).
27. generating random numbers
how to generate random numbers. This functionality is provided by the math/rand package. The random numbers generated by math/rand are considered cryptographically insecure because the sequences are repeatable with given seed.
To generate cryptographically secure numbers, the crypto/rand package should be used. These sequences are not repeatable.
Create the rand.go file with the following content:
package main
import (
crypto "crypto/rand"
"fmt"
"math/big"
"math/rand"
)
func main() {
sec1 := rand.New(rand.NewSource(10))
sec2 := rand.New(rand.NewSource(10))
for i := 0; i < 5; i++ {
rnd1 := sec1.Int()
rnd2 := sec2.Int()
if rnd1 != rnd2 {
fmt.Println("Rand generated non-equal sequence")
break
} else {
fmt.Printf("Math/Rand1: %d , Math/Rand2: %d\n", rnd1, rnd2)
}
}
for i := 0; i < 5; i++ {
safeNum := NewCryptoRand()
safeNum2 := NewCryptoRand()
if safeNum == safeNum2 {
fmt.Println("Crypto generated equal numbers")
break
} else {
fmt.Printf("Crypto/Rand1: %d , Crypto/Rand2: %d\n",
safeNum, safeNum2)
}
}
}
func NewCryptoRand() int64 {
safeNum, err := crypto.Int(crypto.Reader, big.NewInt(100234))
if err != nil {
panic(err)
}
return safeNum.Int64()
}
output:
sangam:golang-daily sangam$ go run rand.go
Math/Rand1: 1512982403 , Math/Rand2: 1512982403
Math/Rand1: 938371728 , Math/Rand2: 938371728
Math/Rand1: 1727441275 , Math/Rand2: 1727441275
Math/Rand1: 492959835 , Math/Rand2: 492959835
Math/Rand1: 1031730807 , Math/Rand2: 1031730807
Crypto/Rand1: 1156 , Crypto/Rand2: 75947
Crypto/Rand1: 69204 , Crypto/Rand2: 426
Crypto/Rand1: 14141 , Crypto/Rand2: 57210
Crypto/Rand1: 62348 , Crypto/Rand2: 85520
Crypto/Rand1: 77442 , Crypto/Rand2: 91391
How it works...
The previous code presents two possibilities on how to generate random numbers. The first option uses the math/rand package, which is cryptographically insecure, and allows us to generate the same sequence with the use of Source with the same seed number. This approach is usually used in tests. The reason for doing so is for the reproducibility of the sequence.
The second option, the cryptographically secure one, is the use of the crypto/rand package. The API uses the Reader to provide the instance of a cryptographically strong pseudo-random generator. The package itself has the default Reader which is usually based on the system-based random number generator.
28. operating complex numbers
Complex numbers are usually used for scientific applications and calculations. Go implements complex numbers as the primitive type.
The specific operations on complex numbers are part of the math/cmplx package.
Create the complex.go file with the following content:
package main
import (
"fmt"
"math/cmplx"
)
func main() {
a := complex(2, 3)
fmt.Printf("Real part: %f \n", real(a))
fmt.Printf("Complex part: %f \n", imag(a))
b := complex(6, 4)
c := a - b
fmt.Printf("Difference : %v\n", c)
c = a + b
fmt.Printf("Sum : %v\n", c)
c = a * b
fmt.Printf("Product : %v\n", c)
c = a / b
fmt.Printf("Product : %v\n", c)
conjugate := cmplx.Conj(a)
fmt.Println("Complex number a's conjugate : ", conjugate)
cos := cmplx.Cos(b)
fmt.Println("Cosine of b : ", cos)
}
output: ``` sangam:golang-daily sangam$ go run complex.go Real part: 2.000000 Complex part: 3.000000 Difference : (-4-1i) Sum : (8+7i) Product : (0+26i) Product : (0.46153846153846156+0.19230769230769232i) Complex number a's conjugate : (2-3i) Cosine of b : (26.220553750072888+7.625225809442885i)
How it works...
- The basic operators are implemented for the primitive type complex. The other operations on complex numbers are provided by the math/cmplx package. In case high precision operations are needed, there is no big implementation.
- On the other hand, the complex number could be implemented as real, and the imaginary part expressed by the big.Float type.
#### 29. converting between degrees and radians
- The trigonometric operations and geometric manipulation are usually done in radians; it is always useful to be able to convert these into degrees and vice versa.
show you some tips on how to handle the conversion between these units.
Create the radians.go file with the following content:
package main
import ( "fmt" "math" )
type Radian float64
func (rad Radian) ToDegrees() Degree { return Degree(float64(rad) * (180.0 / math.Pi)) }
func (rad Radian) Float64() float64 { return float64(rad) }
type Degree float64
func (deg Degree) ToRadians() Radian { return Radian(float64(deg) * (math.Pi / 180.0)) }
func (deg Degree) Float64() float64 { return float64(deg) }
func main() {
val := radiansToDegrees(1) fmt.Printf("One radian is : %.4f degrees\n", val)
val2 := degreesToRadians(val) fmt.Printf("%.4f degrees is %.4f rad\n", val, val2)
// Conversion as part // of type methods val = Radian(1).ToDegrees().Float64() fmt.Printf("Degrees: %.4f degrees\n", val)
val = Degree(val).ToRadians().Float64() fmt.Printf("Rad: %.4f radians\n", val) }
func degreesToRadians(deg float64) float64 { return deg * (math.Pi / 180.0) }
func radiansToDegrees(rad float64) float64 { return rad * (180.0 / math.Pi) }
output:
sangam:golang-daily sangam$ go run radians.go One radian is : 57.2958 degrees 57.2958 degrees is 1.0000 rad Degrees: 57.2958 degrees Rad: 1.0000 radians
How it works...
- The Go standard library does not contain any package with a function converting radians to degrees and vice versa. But at least the Pi constant is a part of the math package, so the conversion could be done as shown in the sample code.
- The preceding code also presents the approach of defining the custom type with additional methods. These are simplifying the conversion of values by handy API.
#### 30. taking logarithms
- Logarithms are used in scientific applications as well as in data visualizations and measurements.
The built-in math package contains the commonly used bases of the logarithm. Using these, you are able to get all bases.
Create the log.go file with the following content:
package main
import ( "fmt" "math" )
func main() {
ln := math.Log(math.E) fmt.Printf("Ln(E) = %.4f\n", ln)
log10 := math.Log10(-100) fmt.Printf("Log10(10) = %.4f\n", log10)
log2 := math.Log2(2) fmt.Printf("Log2(2) = %.4f\n", log2)
log_3_6 := Log(3, 6) fmt.Printf("Log3(6) = %.4f\n", log_3_6)
}
// Log computes the logarithm of // base > 1 and x greater 0 func Log(base, x float64) float64 { return math.Log(x) / math.Log(base) }
output:
sangam:golang-daily sangam$ go run log.go Ln(E) = 1.0000 Log10(10) = NaN Log2(2) = 1.0000 Log3(6) = 1.6309
How it works...
- The standard package, math, contains functions for all commonly used logarithms,
and so you can easily get binary, decimal, and natural logarithms. See the Log function which counts any
logarithm of y with base x through the helper-defined formula
- The internal implementation of the logarithm in standard lib is naturally based on approximation.
This function can be seen in the $GOROOT/src/math/log.go file.
#### 31. generating checksums
- The hash, or so-called checksum, is the easiest way to quickly compare any content. This recipe demonstrates how to create the checksum of the file content. For demonstration purposes,
the MD5 hash function will be used.
Create the content.dat file with the following content:
This is content to check ``` Create the checksum.go file with the following content:
package main
import (
"crypto/md5"
"fmt"
"io"
"os"
)
var content = "This is content to check"
func main() {
checksum := MD5(content)
checksum2 := FileMD5("content.dat")
fmt.Printf("Checksum 1: %s\n", checksum)
fmt.Printf("Checksum 2: %s\n", checksum2)
if checksum == checksum2 {
fmt.Println("Content matches!!!")
}
}
func MD5(data string) string {
h := md5.Sum([]byte(data))
return fmt.Sprintf("%x", h)
}
func FileMD5(path string) string {
h := md5.New()
f, err := os.Open(path)
if err != nil {
panic(err)
}
defer f.Close()
_, err = io.Copy(h, f)
if err != nil {
panic(err)
}
return fmt.Sprintf("%x", h.Sum(nil))
}
output: sangam:golang-daily sangam$ go run checksum.go Checksum 1: e44f5ac2d500bde35ace3dcc34cc6bf1 Checksum 2: 958ba356a2c98156d367a14b86226a1c sangam:golang-daily sangam$ How it works...
The crypto package contains implementations of well-known hash functions. The MD5 hash function is located in the crypto/md5 package. Each hash function in the crypto package implements the Hash interface. Note that Hash contains the Write method. With the Write method, it can be utilized as a Writer. This can be seen in the FileMD5 function. The Sum method of Hash accepts the argument of byte slice, where the resulting hash should be placed.
Beware of this. The Sum method does not compute the hash of the argument, but computes the hash into an argument
On the other hand, md5.Sum, the package function, can be used to produce the hash directly. In this case, the argument of the Sum function is the one from the hash values computed.
Naturally, the crypto package implements the SHA variants and other hash functions as well. These are usually used in the same way. The hash functions can be accessed through the crypto package constant crypto.Hash (for example, crypto.MD5.New()), but this way, the package with the given function must be linked to a built binary as well (the blank import could be used, import _ "crypto/md5"), otherwise the call for New will panic.
The hash package itself contains the CRC checksums and more.
Every process owns its standard input, output, and error file descriptor. The stdin serves as the input of the process. This recipe describes how to read the data from the stdin
Create the fmt.go file with the following content:
package main
import (
"fmt"
)
func main() {
var name string
fmt.Println("What is your name?")
fmt.Scanf("%s\n", &name)
var age int
fmt.Println("What is your age?")
fmt.Scanf("%d\n", &age)
fmt.Printf("Hello %s, your age is %d\n", name, age)
}
output: ``` sangam:golang-daily sangam$ go run fmt.go What is your name? sangam What is your age? 24 Hello sangam, your age is 24 sangam:golang-daily sangam$
Create the file scanner.go with the following content:
package main
import ( "bufio" "fmt" "os" )
func main() {
// The Scanner is able to // scan input by lines sc := bufio.NewScanner(os.Stdin)
for sc.Scan() { txt := sc.Text() fmt.Printf("Echo: %s\n", txt) }
}
output:
sangam:golang-daily sangam$ go run scanner.go welcome to gopherlabs Echo: welcome to gopherlabs // Press CTRL + C to send SIGINT ``` Create the file reader.go with the following content:
package main
import (
"fmt"
"os"
)
func main() {
for {
data := make([]byte, 8)
n, err := os.Stdin.Read(data)
if err == nil && n > 0 {
process(data)
} else {
break
}
}
}
func process(data []byte) {
fmt.Printf("Received: %X %s\n", data, string(data))
}
output: ``` sangam:golang-daily sangam$ echo 'Gopherlabs is awesome!' | go run reader.go Received: 476F706865726C61 Gopherla Received: 6273206973206177 bs is aw Received: 65736F6D65210A00 esome! sangam:golang-daily sangam$
How it works...
- The stdin of the Go process could be retrieved via the Stdin of the os package.
In fact, it is a File type which implements the Reader interface. Reading from the Reader is then very easy.
The preceding code shows three very common ways of how to read from the Stdin.
- The first option illustrates the use of the fmt package, which provides the functions Scan, Scanf, and Scanln.
The Scanf function reads the input into given variable(s). The advantage of Scanf is that you can determine the format of the scanned value. The Scan function just reads the input into a variable (without predefined formatting) and Scanln, as its name suggests, reads the input ended with the line break.
- The Scanner, which is the second option shown in the sample code,
provides a convenient way of scanning larger input. The Scanner contains the
function Split by which the custom split function could be defined. For example, to scan the words from stdin, you can use bufio.ScanWords predefined SplitFunc.
- The reading via the Reader API is the last presented approach.
This one provides you with more control of how the input is read.
#### 33. Writing standard output and error
- each process has stdin, a stdout and stderr file descriptors.
The standard approach is the use of stdout as a process output and stderr as process error output.
As these are the file descriptors, the destination where the data is written could be anything,
from the console to the socket.This recipe will show you how to write the stdout and stderr.
Create the stdouterr.go file with the following content:
package main
import ( "fmt" "io" "os" )
func main() {
// Simply write string io.WriteString(os.Stdout, "This is string to standard output.\n")
io.WriteString(os.Stderr, "This is string to standard error output.\n")
// Stdout/err implements // writer interface buf := []byte{0xAF, 0xFF, 0xFE} for i := 0; i < 200; i++ { if _, e := os.Stdout.Write(buf); e != nil { panic(e) } }
// The fmt package // could be used too fmt.Fprintln(os.Stdout, "\n") }
output:
sangam:golang-daily sangam$ go run stdouterr.go This is string to standard output. This is string to standard error output. ???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
How it works...
- As with the Stdin from the previous recipe, the Stdout and Stderr are the file descriptors. So these are implementing the Writer interface.
- The preceding example shows a few ways of how to write into these via the io.WriteString function, with the use of the Writer API and by the fmt package and FprintXX functions.
#### 34. opening a file by name
- File access is a very common operation used to store or read the data.
This recipe illustrates how to open a file by its name and path, using the standard library.
Create the directory temp and create the file file.txt in it.
Edit the file.txt file and write This file content into the file.
Create the openfile.go file with the following content:
package main
import ( "fmt" "io" "io/ioutil" "os" )
func main() {
f, err := os.Open("temp/file.txt") if err != nil { panic(err) }
c, err := ioutil.ReadAll(f) if err != nil { panic(err) }
fmt.Printf("### File content ###\n%s\n", string(c)) f.Close()
f, err = os.OpenFile("temp/test.txt", os.O_CREATE|os.O_RDWR, os.ModePerm) if err != nil { panic(err) } io.WriteString(f, "Test string") f.Close()
}
output:
sangam:golang-daily sangam$ go run openfile.go
File content
This file content
sangam:golang-daily sangam$
See the output there should also be a new file, test.txt, in the temp folder:
How it works...
- The os package offers a simple way of opening the file. The function Open opens the file by the path, just in read-only mode. Another function, OpenFile, is the more powerful one and consumes the path to the file, flags, and permissions.
- The flag constants are defined in the os package and you can combine them with use of the binary OR operator |. The permissions are set by the os package constants (for example, os.ModePerm ) or by the number notation such as 0777 (permissions: -rwxrwxrwx).
#### 35. reading the file into a string
- In the previous recipes, we saw the reading from Stdin and the opening of the file.
In this recipe, we will combine these two a little bit and show how to read the file into a string.
Create the directory temp and create the file file.txt in it.
Edit the file.txt file and write multiple lines into the file.
Create the readfile.go file with the following content:
package main
import "os" import "bufio"
import "bytes" import "fmt" import "io/ioutil"
func main() {
fmt.Println("### Read as reader ###") f, err := os.Open("temp/file.txt") if err != nil { panic(err) } defer f.Close()
// Read the // file with reader wr := bytes.Buffer{} sc := bufio.NewScanner(f) for sc.Scan() { wr.WriteString(sc.Text()) } fmt.Println(wr.String())
fmt.Println("### ReadFile ###") // for smaller files fContent, err := ioutil.ReadFile("temp/file.txt") if err != nil { panic(err) } fmt.Println(string(fContent))
}
output:
sangam:golang-daily sangam$ go run readfile.go
Read as reader
This file content
ReadFile
This file content
How it works...
- The reading from the file is simple because the File type implements both the Reader and Writer interfaces.
This way, all functions and approaches applicable to the Reader interface are applicable to the File type.
The preceding example shows how to read the file with the use of Scanner and write the content to the bytes buffer (which is more performant than string concatenation). This way, you are able to control the volume of content read from a file.
- The second approach with ioutil.ReadFile is simpler but should be used carefully, because it reads the whole file. Keep in mind that the file could be huge and it could threaten the stability of the application
#### 36. Reading writing a different charset
It is not an exception that the input from various sources could come in various charsets.
Note that a lot of systems use the Windows operating system but there are others.
Go, by default, expects that the strings used in the program are UTF-8 based. If they are not,
then decoding from the given charset must be done to be able to work with the string.
This recipe will show the reading and writing of the file in a charset other than UTF-8.
Create the charset.go file with the following content:
package main
import ( "fmt" "io/ioutil" "os"
"golang.org/x/text/encoding/charmap" )
func main() {
// Write the string // encoded to Windows-1252 encoder := charmap.Windows1252.NewEncoder() s, e := encoder.String("This is sample text with runes Š") if e != nil { panic(e) } ioutil.WriteFile("example.txt", []byte(s), os.ModePerm)
// Decode to UTF-8 f, e := os.Open("example.txt") if e != nil { panic(e) } defer f.Close() decoder := charmap.Windows1252.NewDecoder() reader := decoder.Reader(f) b, err := ioutil.ReadAll(reader) if err != nil { panic(err) } fmt.Println(string(b)) }
output:
sangam:golang-daily sangam$ go run charset.go This is sample text with runes Š sangam:golang-daily sangam$
How it works...
- The golang.org/x/text/encoding/charmap package contains the Charmap type pointer constants that
represent the widely used charsets. The Charmap type provides the methods for creating
the encoder and decoder for the given charset.
- The Encoder creates the encoding Writer which encodes the written bytes into the chosen charset.
Similarly, the Decoder can create the decoding Reader, which decodes all read data from the chosen charset.
#### 37. Seeking a position within a file
- In some cases, you need to read from or write to a particular location in a file, such as an indexed file.
The recipe will show you how to use the position seeking in the context of flat file operations.
Create the file `flatfile.txt` with the following content:
123.Jun.......Wong...... 12..Novak.....Jurgen.... 10..Thomas....Sohlich... ``` Create the fileseek.go file with the following content:
package main
import (
"errors"
"fmt"
"os"
)
const lineLegth = 25
func main() {
f, e := os.OpenFile("flatfile.txt", os.O_RDWR|os.O_CREATE,
os.ModePerm)
if e != nil {
panic(e)
}
defer f.Close()
fmt.Println(readRecords(2, "last", f))
if err := writeRecord(2, "first", "Radomir", f); err != nil {
panic(err)
}
fmt.Println(readRecords(2, "first", f))
if err := writeRecord(10, "first", "Andrew", f); err != nil {
panic(err)
}
fmt.Println(readRecords(10, "first", f))
fmt.Println(readLine(2, f))
}
func readLine(line int, f *os.File) (string, error) {
lineBuffer := make([]byte, 24)
f.Seek(int64(line*lineLegth), 0)
_, err := f.Read(lineBuffer)
return string(lineBuffer), err
}
func writeRecord(line int, column, dataStr string, f *os.File) error {
definedLen := 10
position := int64(line * lineLegth)
switch column {
case "id":
definedLen = 4
case "first":
position += 4
case "last":
position += 14
default:
return errors.New("Column not defined")
}
if len([]byte(dataStr)) > definedLen {
return fmt.Errorf("Maximum length for '%s' is %d",
column, definedLen)
}
data := make([]byte, definedLen)
for i := range data {
data[i] = '.'
}
copy(data, []byte(dataStr))
_, err := f.WriteAt(data, position)
return err
}
func readRecords(line int, column string, f *os.File) (string, error) {
lineBuffer := make([]byte, 24)
f.ReadAt(lineBuffer, int64(line*lineLegth))
var retVal string
switch column {
case "id":
return string(lineBuffer[:3]), nil
case "first":
return string(lineBuffer[4:13]), nil
case "last":
return string(lineBuffer[14:23]), nil
}
return retVal, errors.New("Column not defined")
}
output:
sangam:golang-daily sangam$ go run fileseek.go
Sohlich.. <nil>
Radomir.. <nil>
Andrew... <nil>
10..Radomir...Sohlich... <nil>
sangam:golang-daily sangam$
Display the file in hex xxd flatfile.txt.
sangam:golang-daily sangam$ xxd flatfile.txt
00000000: 3132 332e 4a75 6e2e 2e2e 2e2e 2e2e 576f 123.Jun.......Wo
00000010: 6e67 2e2e 2e2e 2e2e 0a31 322e 2e4e 6f76 ng.......12..Nov
00000020: 616b 2e2e 2e2e 2e4a 7572 6765 6e2e 2e2e ak.....Jurgen...
00000030: 2e0a 3130 2e2e 5261 646f 6d69 722e 2e2e ..10..Radomir...
00000040: 536f 686c 6963 682e 2e2e 0a00 0000 0000 Sohlich.........
00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000080: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000f0: 0000 0000 0000 0000 0000 0000 0000 416e ..............An
00000100: 6472 6577 2e2e 2e2e drew....
sangam:golang-daily sangam$
How it works...
The preceding example uses the flatfile as an illustration of how to seek, read and write at the position in the file. In general, for moving the position of the current pointer in the File, the Seek method can be used. It takes two arguments and these are, position and how to count the position, 0 - relative to file origin, 1 - relative to current position, 2 - relative to the end of file. This way you are able to move the cursor within the file.
Note : The flatfile is the most basic form of how to store the data. The record structure has a fixed length and the same for the record parts. The structure of the flat file in the example is: ID - 4 chars, FirstName - 10 chars, LastName - 10 chars. The whole record is 24 chars long, ended by a line break which is the 25th character. The Seek method is used in the implementation of the readLine function in the preceding code.
The os.File also contains the ReadAt and WriteAt methods. These methods consume that the bytes to be written/read and the offset where to start. These simplify the writing and reading to a certain position in a file.
38. reading and writing binary data
- describes how to write and read any type in the binary form.
Create the rwbinary.go file with the following content:
package main
import (
"bytes"
"encoding/binary"
"fmt"
)
func main() {
buf := bytes.NewBuffer([]byte{})
if err := binary.Write(buf, binary.BigEndian, 1.004);
err != nil {
panic(err)
}
if err := binary.Write(buf, binary.BigEndian,
[]byte("Hello")); err != nil {
panic(err)
}
var num float64
if err := binary.Read(buf, binary.BigEndian, &num);
err != nil {
panic(err)
}
fmt.Printf("float64: %.3f\n", num)
greeting := make([]byte, 5)
if err := binary.Read(buf, binary.BigEndian, &greeting);
err != nil {
panic(err)
}
fmt.Printf("string: %s\n", string(greeting))
}
output: sangam:golang-daily sangam$ go run rwbinary.go float64: 1.004 string: Hello sangam:golang-daily sangam$ How it works...
The binary data could be written with the use of the encoding/binary package. The function Write consumes the Writer where the data should be written, the byte order (BigEndian/LittleEndian) and finally, the value to be written into Writer.
To read the binary data analogically, the Read function could be used. Note that there is no magic in reading the data from the binary source. You need to be sure what data you are fetching from the Reader. If not, the data could be fetched into any type which fits the size.
39. writing to multiple writers at once
- When you need to write the same output into more than one target, there is a helping hand available in the built-in package. This recipe shows how to implement writing simultaneously into multiple targets.
Create the multiwr.go file with the following content: ``` package main
import "io" import "bytes" import "os" import "fmt"
func main() {
buf := bytes.NewBuffer([]byte{}) f, err := os.OpenFile("sample.txt", os.O_CREATE|os.O_RDWR, os.ModePerm) if err != nil { panic(err) } wr := io.MultiWriter(buf, f) _, err = io.WriteString(wr, "Hello, Go is awesome!") if err != nil { panic(err) }
fmt.Println("Content of buffer: " + buf.String()) }
output:
sangam:golang-daily sangam$ go run multiwr.go Content of buffer: Hello, Go is awesome! sangam:golang-daily sangam$
Check the content of the created file: `sample.txt`
Hello, Go is awesome! ``` How it works...
The io package contains the MultiWriter function with variadic parameters of Writers.
When the Write method on the Writer is called, then the data is written to all underlying Writers.
40. piping between writer and reader
- The pipes between processes are the easy way to use the output of the first process as the input of other processes. The same concept could be done in Go, for example, to pipe data from one socket to another socket, to create the tunneled connection. This recipe will show you how to create the pipe with use of the Go built-in library.
Create the pipe.go file with the following content:
package main
import (
"io"
"log"
"os"
"os/exec"
)
func main() {
pReader, pWriter := io.Pipe()
cmd := exec.Command("echo", "Hello Go!\nThis is example")
cmd.Stdout = pWriter
go func() {
defer pReader.Close()
if _, err := io.Copy(os.Stdout, pReader); err != nil {
log.Fatal(err)
}
}()
if err := cmd.Run(); err != nil {
log.Fatal(err)
}
}
output:
sangam:golang-daily sangam$ go run pipe.go
Hello Go!
This is example
sangam:golang-daily sangam$
How it works...
The io.Pipe function creates the in-memory pipe and returns both ends of the pipe, the PipeReader on one side and PipeWriter on the other side. Each Write to PipeWriter is blocked until it is consumed by Read on the other end.
The example shows the piping output from the executed command to the standard output of the parent program. By assigning the pWriter to cmd.Stdout, the standard output of the child process is written to the pipe, and the io.Copy in goroutine consumes the written data, by copying the data to os.Stdout.
- Besides the well-known JSON and XML, Go also offers the binary format, gob. This recipe goes through the basic concept of how to use the gob package.
Create the gob.go file with the following content: ``` package main
import ( "bytes" "encoding/gob" "fmt" )
type User struct { FirstName string LastName string Age int Active bool }
func (u User) String() string { return fmt.Sprintf({"FirstName":%s,"LastName":%s, "Age":%d,"Active":%v }, u.FirstName, u.LastName, u.Age, u.Active) }
type SimpleUser struct { FirstName string LastName string }
func (u SimpleUser) String() string { return fmt.Sprintf({"FirstName":%s,"LastName":%s}, u.FirstName, u.LastName) }
func main() {
var buff bytes.Buffer
// Encode value enc := gob.NewEncoder(&buff) user := User{ "sangam", "biradar", 24, true, } enc.Encode(user) fmt.Printf("%X\n", buff.Bytes())
// Decode value out := User{} dec := gob.NewDecoder(&buff) dec.Decode(&out) fmt.Println(out.String())
enc.Encode(user) out2 := SimpleUser{} dec.Decode(&out2) fmt.Println(out2.String())
}
output:
sangam:golang-daily sangam$ go run gob.go 40FF81030101045573657201FF82000104010946697273744E616D65010C0001084C6173744E616D65010C0001034167650104000106416374697665010200000018FF82010673616E67616D0107626972616461720130010100 {"FirstName":sangam,"LastName":biradar, "Age":24,"Active":true } {"FirstName":sangam,"LastName":biradar} sangam:golang-daily sangam$
How it works...
- The gob serialization and deserialization need the Encoder and Decoder.
The gob.NewEncoder function creates the Encoder with the underlying Writer.
- Each call of the Encode method will serialize the object into a gob format.
The gob format itself is the self-describing binary format. This means each serialized struct is preceded by its description.
- To decode the data from the serialized form, the Decoder must be created by calling the gob.NewDecoder with the underlying Reader. The Decode then accepts the pointer to the structure where the data should be deserialized.
#### 42. reading and writing ZIP files
- ZIP compression is a widely used compression format. It is usual to use the ZIP format for an application to upload a file set or, on the other hand, export zipped files as output.
This recipe will show you how to handle ZIP files programmatically with the use of the standard library.
Create the zip.go file with the following content:
package main
import ( "archive/zip" "bytes" "fmt" "io" "io/ioutil" "log" "os" )
func main() {
var buff bytes.Buffer
// Compress content zipW := zip.NewWriter(&buff) f, err := zipW.Create("newfile.txt") if err != nil { panic(err) } _, err = f.Write([]byte("This is my file content")) if err != nil { panic(err) } err = zipW.Close() if err != nil { panic(err) }
//Write output to file err = ioutil.WriteFile("data.zip", buff.Bytes(), os.ModePerm) if err != nil { panic(err) }
// Decompress the content zipR, err := zip.OpenReader("data.zip") if err != nil { panic(err) }
for _, file := range zipR.File { fmt.Println("File " + file.Name + " contains:") r, err := file.Open() if err != nil { log.Fatal(err) } _, err = io.Copy(os.Stdout, r) if err != nil { panic(err) } err = r.Close() if err != nil { panic(err) } fmt.Println() }
}
output:
sangam:golang-daily sangam$ go run zip.go File newfile.txt contains: This is my file content sangam:golang-daily sangam$
How it works...
- The built-in package zip contains the NewWriter and NewReader functions to create the zip.Writer to compress, and the zip.Reader to decompress the zipped content.
- Each record of the ZIP file is created with the Create method of the created zip.Writer . The returned Writer is then used to write the content body.
- To decompress the files, the OpenReader function is used to create the ReadCloser of the records in the zipped file. The File field of the created ReaderCloser is the slice of zip.File pointers.
The content of the file is obtained by calling the Open method and by reading the returned ReadCloser
#### 43. Parsing a large XML file effectively
- XML is a very common format for data exchange. The Go library contains support for parsing XML files the same way as the JSON. Usually,
the struct which corresponds to the XML scheme is used and with this help, the XML content is parsed at once. The problem is when the XML file is too large to fit into memory and so you need to parse the file in chunks.
This recipe will reveal how to handle a large XML file and parse the required information.
Create the`data.xml` file with the following XML content:
?xml version="1.0"? Gambardella, Matthew Computer 44.95 <publish_date>2000-10-01</publish_date> An in-depth look at creating applications with XML. Galos, Mike Computer 49.95 <publish_date>2001-04-16</publish_date> Microsoft Visual Studio 7 is explored in depth, looking at how Visual Basic, Visual C++, C#, and ASP+ are integrated into a comprehensive development environment.
Create the `xml.go` file with the following content:
package main
import ( "encoding/xml" "fmt" "os" )
type Book struct { Title string xml:"title" Author string xml:"author" }
func main() {
f, err := os.Open("data.xml") if err != nil { panic(err) } defer f.Close() decoder := xml.NewDecoder(f)
// Read the book one by one books := make([]Book, 0) for { tok, _ := decoder.Token() if tok == nil { break } switch tp := tok.(type) { case xml.StartElement: if tp.Name.Local == "book" { // Decode the element to struct var b Book decoder.DecodeElement(&b, &tp) books = append(books, b) } } } fmt.Println(books) }
output:
sangam:golang-daily sangam$ go run xml.go [{XML Developer's Guide Gambardella, Matthew} {Visual Studio 7: A Comprehensive Guide Galos, Mike}] sangam:golang-daily sangam$
How it works...
- With the NewDecoder function of the xml package, the Decoder for the XML content is created.
- By calling the Token method on the Decoder, the xml.Token is received. The xml.Token is the interface which holds
the token type. The behavior of the code can be defined, based on the type. The sample code tests
if the parsed xml.StartElement is one of the book elements.
- Then it partially parses the data into a Book structure.
This way, the position of the pointer in the underlying Reader in the Decoder is shifted by the struct data, and
the parsing can continue.
#### 44. extracting data from an incomplete JSON array
- This recipe contains a very specific use case, where your program consumes the JSON from an unreliable source and the JSON contains an array of objects which has the beginning token
[ but the number of items in the array is very large, and the end of the JSON could be corrupted.
Create the json.go file with the following content:
package main
import ( "encoding/json" "fmt" "strings" )
const js = [ { "name":"Axel", "lastname":"Fooley" }, { "name":"Tim", "lastname":"Burton" }, { "name":"Tim", "lastname":"Burton"
type User struct { Name string json:"name" LastName string json:"lastname" }
func main() {
userSlice := make([]User, 0) r := strings.NewReader(js) dec := json.NewDecoder(r) for { tok, err := dec.Token() if err != nil { break } if tok == nil { break } switch tp := tok.(type) { case json.Delim: str := tp.String() if str == "[" || str == "{" { for dec.More() { u := User{} err := dec.Decode(&u) if err == nil { userSlice = append(userSlice, u) } else { break } } } } }
fmt.Println(userSlice) }
output:
sangam:golang-daily sangam$ go run json.go [{Axel Fooley} {Tim Burton}] sangam:golang-daily sangam$
How it works...
- Besides the Unmarshall function, the json package also contains the Decoder API. With NewDecoder, the Decoder could be created. By calling the Token method on the decoder, the underlying Reader is read and returns the Token interface. This could hold multiple values.
- One of these is the Delim type, which is a rune containing one of the {, [, ], } characters. Based on this, the beginning of the JSON array is detected. With the More method on the decoder, more objects to decode could be detected.
#### 45. getting file information
If you need to discover basic information about the accessed file,
Go's standard library provides a way on how you can do this. This recipe shows how you can access this information.
Create the sample test.file with the content `This is test file`
Create the `fileinfo.go` file with the following content:
package main
import ( "fmt" "os" )
func main() {
f, err := os.Open("test.file") if err != nil { panic(err) } fi, err := f.Stat() if err != nil { panic(err) }
fmt.Printf("File name: %v\n", fi.Name()) fmt.Printf("Is Directory: %t\n", fi.IsDir()) fmt.Printf("Size: %d\n", fi.Size()) fmt.Printf("Mode: %v\n", fi.Mode())
}
output:
sangam:golang-daily sangam$ go run fileinfo.go File name: test.file Is Directory: false Size: 18 Mode: -rw-r--r-- sangam:golang-daily sangam$
How it works...
The os.File type provides access to the FileInfo type via the Stat method.
The FileInfo struct contains all the basic information about the file.
#### 46 .creating temporary files
- Temporary files are commonly used while running test cases or if your application needs to have a place to store short-term content such as user data uploads and currently processed data.
This recipe will present the easiest way to create such a file or directory.
Create the tempfile.go file with the following content:
package main
import "io/ioutil" import "os" import "fmt"
func main() { tFile, err := ioutil.TempFile("", "gopherlabs") if err != nil { panic(err) } // The called is responsible for handling // the clean up. defer os.Remove(tFile.Name())
fmt.Println(tFile.Name())
// TempDir returns // the path in string. tDir, err := ioutil.TempDir("", "gopherlabs") if err != nil { panic(err) } defer os.Remove(tDir) fmt.Println(tDir)
}
output:
sangam:golang-daily sangam$ go run tempfile.go /var/folders/mg/_355pdvd741cz0z99ys9s66h0000gn/T/gopherlabs622911386 /var/folders/mg/_355pdvd741cz0z99ys9s66h0000gn/T/gopherlabs447325745 sangam:golang-daily sangam$
How it works...
- The ioutil package contains the functions TempFile and TempDir. The TempFile function consumes the directory and the file prefix. The os.File with the underlying temporary file is returned. Note that the caller is responsible for cleaning out the file. The previous example uses the os.Remove function to do that.
- The TempDir function works the same way. The difference is that the string with the path to the directory is returned.
#### 47 .writing the file
- Writing a file is an essential task for every programmer;
Go supports multiple ways on how you can do this. This recipe will show a few of them.
Create the writefile.go file with the following content:
package main
import ( "io" "os" "strings" )
func main() {
f, err := os.Create("sample.file") if err != nil { panic(err) } defer f.Close()
_, err = f.WriteString("Go is awesome!\n") if err != nil { panic(err) }
_, err = io.Copy(f, strings.NewReader("Yeah! Go is great.\n")) if err != nil { panic(err) } }
output:
sangam:golang-daily sangam$ go run writefile.go ```
Check the content of the created sample.file:
sangam:golang-daily sangam$ cat sample.file
Go is awesome!
Yeah! Go is great.
sangam:golang-daily sangam$
How it works...
- The os.File type implements the Writer interface, so writing to the file could be done by any option that uses the Writer interface. The preceding example uses the WriteString method of the os.File type. The
io.WriteString method can also be used in general.
48 .writing the file from multiple goroutines
- how to safely write to the file from multiple goroutines.
Create the syncwrite.go file with the following content:
package main
import (
"fmt"
"io"
"os"
"sync"
)
type SyncWriter struct {
m sync.Mutex
Writer io.Writer
}
func (w *SyncWriter) Write(b []byte) (n int, err error) {
w.m.Lock()
defer w.m.Unlock()
return w.Writer.Write(b)
}
var data = []string{
"Hello!",
"Ola!",
"Ahoj!",
}
func main() {
f, err := os.Create("sample.file")
if err != nil {
panic(err)
}
wr := &SyncWriter{sync.Mutex{}, f}
wg := sync.WaitGroup{}
for _, val := range data {
wg.Add(1)
go func(greetings string) {
fmt.Fprintln(wr, greetings)
wg.Done()
}(val)
}
wg.Wait()
}
output:
sangam:golang-daily sangam$ go run syncwrite.go
sangam:golang-daily sangam$ cat sample.file
Ahoj!
Hello!
Ola!
sangam:golang-daily sangam$
How it works...
Writing concurrently to a file is a problem that can end up with inconsistent file content. It is better to synchronize the writing to the file by using Mutex or any other synchronization primitive. This way, you ensure that only one goroutine at a time will be able to write to the file.
The preceding code creates a Writer with Mutex, which embeds the Writer (os.File, in this case), and for each Write call, internally locks the Mutex to provide exclusivity. After the write operation is complete, the Mutex primitive is unlocked naturally.
49 .listing a directory
Create a directory named folder. Create the listdir.go file with the following content:
package main
import (
"fmt"
"io/ioutil"
"os"
"path/filepath"
)
func main() {
fmt.Println("List by ReadDir")
listDirByReadDir(".")
fmt.Println()
fmt.Println("List by Walk")
listDirByWalk(".")
}
func listDirByWalk(path string) {
filepath.Walk(path, func(wPath string, info os.FileInfo,
err error) error {
if wPath == path {
return nil
}
if info.IsDir() {
fmt.Printf("[%s]\n", wPath)
return filepath.SkipDir
}
if wPath != path {
fmt.Println(wPath)
}
return nil
})
}
func listDirByReadDir(path string) {
lst, err := ioutil.ReadDir(path)
if err != nil {
panic(err)
}
for _, val := range lst {
if val.IsDir() {
fmt.Printf("[%s]\n", val.Name())
} else {
fmt.Println(val.Name())
}
}
}
output:
sangam:golang-daily sangam$ go run main.go
List by ReadDir
.DS_Store
binary
config.json
content.dat
List by Walk
.DS_Store
binary
config.json
content.dat
data.csv
data.xml
data.zip
How it works...
The folder listing the example above uses two approaches. The first, simpler one is implemented by using the listDirByReadDir function and utilizes the ReadDir function from the ioutil package. This function returns the slice of FileInfo structs that represent the actual directory content. Note that the ReadDir function does not read the folders recursively. In fact, the ReadDir function internally uses the Readdir method of the File type in the os package.
On the other hand, the more complicated listDirByWalk uses the filepath.Walk function which consumes the path to be walked and has a function that processes each file or folder in any given path. The main difference is that the Walk function reads the directory recursively. The core part of this approach is the WalkFunc type, where its function is to consume the results of the listing. Note that the function blocks the recursive call on underlying folders by returning the filepath.SkipDir error. The Walk function also processes the called path at first, so you need to handle this as well (in this case, we skip the printing and return nil because we need to process this folder recursively).
50. Changing file permissions
how file permissions can be changed programmatically.
Create the filechmod.go file with the following content:
package main
import (
"fmt"
"os"
)
func main() {
f, err := os.Create("test.file")
if err != nil {
panic(err)
}
defer f.Close()
fi, err := f.Stat()
if err != nil {
panic(err)
}
fmt.Printf("File permissions %v\n", fi.Mode())
err = f.Chmod(0777)
if err != nil {
panic(err)
}
fi, err = f.Stat()
if err != nil {
panic(err)
}
fmt.Printf("File permissions %v\n", fi.Mode())
}
output:
sangam:golang-daily sangam$ go run filechmod.go
File permissions -rw-r--r--
File permissions -rwxrwxrwx
sangam:golang-daily sangam$
How it works...
The Chmod method of the File type in the os package can be used to change file permissions. The preceding example just creates the file and changes the permissions to 0777.
Just note that the fi.Mode() is called twice because it extracts the permissions (os.FileMode) for the current state of the file.
The shortest way to change the permissions is by using the os.Chmod function, which does the same, but you do not need to obtain the File type in the code.
51.Creating files and directories
- few general ways you can create files and directories in code.
Create the create.go file with the following content:
package main
import (
"os"
)
func main() {
f, err := os.Create("created.file")
if err != nil {
panic(err)
}
f.Close()
f, err = os.OpenFile("created.byopen", os.O_CREATE|os.O_APPEND,
os.ModePerm)
if err != nil {
panic(err)
}
f.Close()
err = os.Mkdir("createdDir", 0777)
if err != nil {
panic(err)
}
err = os.MkdirAll("sampleDir/path1/path2", 0777)
if err != nil {
panic(err)
}
}
output:
sangam:golang-daily sangam$ create.go
sangam:golang-daily sangam$ tree
.
├── binary
├── config.json
├── content.dat
├── created.byopen
├── created.file
├── createdDir
├── data.csv
├── data.xml
├── data.zip
├── example.txt
├── flatfile.txt
├── main.go
├── sample.file
├── sample.txt
├── sampleDir
│ └── path1
│ └── path2
How it works...
The previous example represents four ways you can create a file or directory. The os.Create function is the simplest way to create the file. By using this function, you will create the file with permissions such as 0666.
If you need to create the file with any other configuration of permissions, then the OpenFile function of the os package is the one to be used.
The directories can be created by using the Mkdir function of the os package. This way, a directory with given permissions is created. The second option is to use the MkdirAll function. This function also creates the directory, but if the given path contains non-exiting directories, then all directories in the path are created (it works the same as the -p option of Unix's mkdir utility).
52 . Filtering file listings
- you how to list the file paths, matching a given pattern. The list does not have to be from the same folder.
Create the filter.go file with the following content:
package main
import (
"fmt"
"os"
"path/filepath"
)
func main() {
for i := 1; i <= 6; i++ {
_, err := os.Create(fmt.Sprintf("./test.file%d", i))
if err != nil {
fmt.Println(err)
}
}
m, err := filepath.Glob("./test.file[1-3]")
if err != nil {
panic(err)
}
for _, val := range m {
fmt.Println(val)
}
for i := 1; i <= 6; i++ {
err := os.Remove(fmt.Sprintf("./test.file%d", i))
if err != nil {
fmt.Println(err)
}
}
}
output:
sangam:golang-daily sangam$ go run filter.go
test.file1
test.file2
test.file3
sangam:golang-daily sangam$
How it works...
To get the filtered file list which corresponds to the given pattern, the Glob function from the filepath package can be used. For the pattern syntax, see the documentation of the filepath.Match function (https://golang.org/pkg/path/filepath/#Match).
Note that the returning result of filepath.Glob is the slice of strings with matching paths.
53 .comparing two files
- a hint on how to compare two files. The recipe will show you how to quickly determine whether the files are identical. The recipe will also present you with a way to find differences between the two.
Create the comparison.go file with the following content:
package main
import (
"bufio"
"crypto/md5"
"fmt"
"io"
"os"
)
var data = []struct {
name string
cont string
perm os.FileMode
}{
{"test1.file", "Hello\nGolang is great", 0666},
{"test2.file", "Hello\nGolang is great", 0666},
{"test3.file", "Not matching\nGolang is great\nLast line",
0666},
}
func main() {
files := []*os.File{}
for _, fData := range data {
f, err := os.Create(fData.name)
if err != nil {
panic(err)
}
defer f.Close()
_, err = io.WriteString(f, fData.cont)
if err != nil {
panic(err)
}
files = append(files, f)
}
checksums := []string{}
for _, f := range files {
f.Seek(0, 0)
sum, err := getMD5SumString(f)
if err != nil {
panic(err)
}
checksums = append(checksums, sum)
}
fmt.Println("### Comparing by checksum ###")
compareCheckSum(checksums[0], checksums[1])
compareCheckSum(checksums[0], checksums[2])
fmt.Println("### Comparing line by line ###")
files[0].Seek(0, 0)
files[2].Seek(0, 0)
compareFileByLine(files[0], files[2])
for _, val := range data {
os.Remove(val.name)
}
}
func getMD5SumString(f *os.File) (string, error) {
file1Sum := md5.New()
_, err := io.Copy(file1Sum, f)
if err != nil {
return "", err
}
return fmt.Sprintf("%X", file1Sum.Sum(nil)), nil
}
func compareCheckSum(sum1, sum2 string) {
match := "match"
if sum1 != sum2 {
match = " does not match"
}
fmt.Printf("Sum: %s and Sum: %s %s\n", sum1, sum2, match)
}
func compareLines(line1, line2 string) {
sign := "o"
if line1 != line2 {
sign = "x"
}
fmt.Printf("%s | %s | %s \n", sign, line1, line2)
}
func compareFileByLine(f1, f2 *os.File) {
sc1 := bufio.NewScanner(f1)
sc2 := bufio.NewScanner(f2)
for {
sc1Bool := sc1.Scan()
sc2Bool := sc2.Scan()
if !sc1Bool && !sc2Bool {
break
}
compareLines(sc1.Text(), sc2.Text())
}
}
output:
sangam:golang-daily sangam$ go run comparison.go
### Comparing by checksum ###
Sum: 5A07C1538087CD5B5C365DE52970E0A3 and Sum: 5A07C1538087CD5B5C365DE52970E0A3 match
Sum: 5A07C1538087CD5B5C365DE52970E0A3 and Sum: FED2EADA5D1D1EBF745DFDC7D1385E6C does not match
### Comparing line by line ###
x | Hello | Not matching
o | Golang is great | Golang is great
x | | Last line
sangam:golang-daily sangam$
How it works...
The comparison of the two files can be done in a few ways. This recipe describes the two basic ones. The first one is by doing a comparison of the whole file by creating the checksum of the file.
The Generating checksum recipe of (Generating Checksums) Dealing with Numbers shows how you can create the checksum of the file. This way, the getMD5SumString function generates the checksum string, which is a hexadecimal representation of the byte result of MD5. The strings are then compared.
The second approach compares the files line by line (in this case, the string content).
In case the lines are not matching, the x sign is included. This is the same way you can compare the binary content, but you will need to scan the file by blocks of bytes (byte slices).
54 .Resolving the user home directory
- It could be beneficial for the program to know the user's home directory, for example, in case you need to store a custom user configuration or any other data related to the user. This recipe will describe how you can find out the current user's home directory.
Create the home.go file with the following content:
package main
import (
"fmt"
"log"
"os/user"
)
func main() {
usr, err := user.Current()
if err != nil {
log.Fatal(err)
}
fmt.Println("The user home directory: " + usr.HomeDir)
}
output:
sangam:golang-daily sangam$ go run home.go
The user home directory: /Users/sangam
sangam:golang-daily sangam$
How it works...
- The os/user package contains the Current function, which provides the os.User type pointer. The User contains the HomeDir property, which contains the path of the current user's home directory.
