goroutine chan 通道 多线程批量上传文件 通过有缓冲的通道控制Go程数量 并发 执行顺序

 

go协程之间的执行顺序以及和主协程的执行顺序_藓类少女的博客-CSDN博客_go协程执行顺序 https://blog.csdn.net/a13545564067/article/details/115065658

 

在控制Go程数量、任务数量最大值的情况下，使动态数量的任务并发执行

 6 8 15 4

 

 

package main

import (
	"log"
	"time"
)

func f() int64 {
	const CN, n = 256, 100000
	ch := make(chan struct{}, CN)
	start0 := time.Now().UnixMilli()
	for i := 0; i < n; i++ {
		ch <- struct{}{}
		go func(i int) {
			defer func() {
				<-ch
			}()
			/*
				// Sleep pauses the current goroutine for at least the duration d.
				// A negative or zero duration causes Sleep to return immediately.
				func Sleep(d Duration)
			*/
			time.Sleep(time.Millisecond)
			log.Print(i)
		}(i)
	}
	close(ch)
	end0 := time.Now().UnixMilli()
	start1 := time.Now().UnixMilli()
	for i := 0; i < n; i++ {
		time.Sleep(time.Millisecond)
		log.Print(i)
	}

	end1 := time.Now().UnixMilli()
	return (end1 - start1) / (end0 - start0)
}

func main() {
	s := []int64{}
	for i := 0; i < 4; i++ {
		s = append(s, f())
	}
	log.Print(s)
}

　　单核

执行4次，花费的时间的倍数：

 

3、

多线程批量上传文件

对象存储 上传对象 - SDK 文档 - 文档中心 - 腾讯云 https://cloud.tencent.com/document/product/436/65644

package main

import (
    "context"
    "github.com/tencentyun/cos-go-sdk-v5"
    "net/http"
    "net/url"
    "os"
    "sync"
)

func upload(wg *sync.WaitGroup, c *cos.Client, files <-chan string) {
    defer wg.Done()
    for file := range files {
        name := "folder/" + file
        fd, err := os.Open(file)
        if err != nil {
            //ERROR
            continue
        }
        _, err = c.Object.Put(context.Background(), name, fd, nil)
        if err != nil {
            //ERROR
        }
    }
}
func main() {
    u, _ := url.Parse("https://examplebucket-1250000000.cos.ap-guangzhou.myqcloud.com")
    b := &cos.BaseURL{BucketURL: u}
    c := cos.NewClient(b, &http.Client{
        Transport: &cos.AuthorizationTransport{
            SecretID:  os.Getenv("SECRETID"),
            SecretKey: os.Getenv("SECRETKEY"),
        },
    })
    // 多线程批量上传文件
    filesCh := make(chan string, 2)
    filePaths := []string{"test1", "test2", "test3"}
    var wg sync.WaitGroup
    threadpool := 2
    for i := 0; i < threadpool; i++ {
        wg.Add(1)
        go upload(&wg, c, filesCh)
    }
    for _, filePath := range filePaths {
        filesCh <- filePath
    }
    close(filesCh)
    wg.Wait()
}

　　

 

 

 

2、

顺序打印
go程之间通信
收发阻塞
通过通道的阻塞实现控制go程执行顺序

 

1、

fatal error: all goroutines are asleep - deadlock!

所有的协程都休眠了 - 死锁！

 

package main
import("fmt")
func f1(in chan int){
fmt.Println(<-in)
}
func main(){
out :=make(chan int)
out<-2
go f1(out)
}

 

 

package main
import("fmt")
func f1(in chan int){
fmt.Println(<-in)
}
func main(){
out :=make(chan int)
// out<-2
go f1(out)
out<-2
}

 

Understanding the context package in golang - Parikshit Agnihotry http://p.agnihotry.com/post/understanding_the_context_package_in_golang/

 

  package main

  import "fmt"
  
  //prints to stdout and puts an int on channel
  func printHello(ch chan int) {
    fmt.Println("Hello from printHello")
    //send a value on channel
    ch <- 2
  }

  func main() {
    //make a channel. You need to use the make function to create channels.
    //channels can also be buffered where you can specify size. eg: ch := make(chan int, 2)
    //that is out of the scope of this post.
    ch := make(chan int)
    //inline goroutine. Define a function and then call it.
    //write on a channel when done
    go func(){
      fmt.Println("Hello inline")
      //send a value on channel
      ch <- 1
    }()
    //call a function as goroutine
    go printHello(ch)
    fmt.Println("Hello from main")

    //get first value from channel.
    //and assign to a variable to use this value later
    //here that is to print it
    i := <- ch
    fmt.Println("Recieved ",i)
    //get the second value from channel
    //do not assign it to a variable because we dont want to use that
    <- ch
  }

　　

Hello from main
Hello from printHello
Recieved 2
Hello inline

 

golang channel 使用总结 - litang.me http://litang.me/post/golang-channel/

• http://www.usingcsp.com/
• https://go101.org/article/channel.html
• https://go101.org/article/channel-use-cases.html
• https://go101.org/article/channel-closing.html
• https://www.jianshu.com/p/819aa9b9af86

 

package main

import (
   "fmt"
   "time"
)

func fA(ch chan bool) {
   defer func() {
      fmt.Println("fA-defer")
   }()
   fmt.Println("fA-doing")
   ch <- true
   time.Sleep(time.Duration(123) * time.Millisecond)
   
}

func fB(ch chan bool) {
   defer func() {
      fmt.Println("fB-defer")
   }()
   fmt.Println("fB-doing")
   ch <- true
   
}

func main() {
   defer func() {
      fmt.Println("main-defer")
   }()
   ch := make(chan bool)
   go fA(ch)
   go fB(ch)
   <-ch
   <-ch
}

 

目前打印结果为：
fB-doing
fB-defer
fA-doing
main-defer
修改代码打印结果为：
fB-doing
fB-defer
fA-doing
fA-defer
main-defer

 

交换

ch <- true
time.Sleep(time.Duration(123) * time.Millisecond)

位置

 

package main

import (
   "sync"
)

func main() {
   ch := make(chan string)
   var wg sync.WaitGroup
   wg.Add(2)
   go func() {
      s := []string{"A", "B", "C", "D", "E"}
      for _, v := range s {
         ch <- v
      }
      wg.Done()
   }()
   go func() {
      s := []string{"1", "2", "3", "4", "5"}
      for _, v := range s {
         v1 := <-ch
         print(v1, v)
      }
      wg.Done()
   }()
   wg.Wait()
}

 

 Channel in Golang - GeeksforGeeks https://www.geeksforgeeks.org/channel-in-golang/

Channel in Golang

• Difficulty Level : Medium
• Last Updated : 20 Nov, 2019

In Go language, a channel is a medium through which a goroutine communicates with another goroutine and this communication is lock-free. Or in other words, a channel is a technique which allows to let one goroutine to send data to another goroutine. By default channel is bidirectional, means the goroutines can send or receive data through the same channel as shown in the below image:

Creating a Channel

In Go language, a channel is created using chan keyword and it can only transfer data of the same type, different types of data are not allowed to transport from the same channel.

Syntax:

var Channel_name chan Type

You can also create a channel using make() function using a shorthand declaration.

 

 

Syntax:

channel_name:= make(chan Type)

Example:

 

// Go program to illustrate

// how to create a channel

package main

  

import "fmt"

  

func main() {

  

    // Creating a channel

    // Using var keyword

    var mychannel chan int

    fmt.Println("Value of the channel: ", mychannel)

    fmt.Printf("Type of the channel: %T ", mychannel)

  

    // Creating a channel using make() function

    mychannel1 := make(chan int)

    fmt.Println("\nValue of the channel1: ", mychannel1)

    fmt.Printf("Type of the channel1: %T ", mychannel1)

}

Output:

Value of the channel:  
Type of the channel: chan int 
Value of the channel1:  0x432080
Type of the channel1: chan int 

Send and Receive Data From a Channel

In Go language, channel work with two principal operations one is sending and another one is receiving, both the operations collectively known as communication. And the direction of <- operator indicates whether the data is received or send. In the channel, the send and receive operation block until another side is not ready by default. It allows goroutine to synchronize with each other without explicit locks or condition variables.

1. Send operation: The send operation is used to send data from one goroutine to another goroutine with the help of a channel. Values like int, float64, and bool can safe and easy to send through a channel because they are copied so there is no risk of accidental concurrent access of the same value. Similarly, strings are also safe to transfer because they are immutable. But for sending pointers or reference like a slice, map, etc. through a channel are not safe because the value of pointers or reference may change by sending goroutine or by the receiving goroutine at the same time and the result is unpredicted. So, when you use pointers or references in the channel you must make sure that they can only access by the one goroutine at a time.

   Mychannel <- element

   The above statement indicates that the data(element) send to the channel(Mychannel) with the help of a <- operator.

2. Receive operation: The receive operation is used to receive the data sent by the send operator.

   element := <-Mychannel

   The above statement indicates that the element receives data from the channel(Mychannel). If the result of the received statement is not going to use is also a valid statement. You can also write a receive statement as:

   <-Mychannel

Example:

 

// Go program to illustrate send

// and receive operation

package main

  

import "fmt"

  

func myfunc(ch chan int) {

  

    fmt.Println(234 + <-ch)

}

func main() {

    fmt.Println("start Main method")

    // Creating a channel

    ch := make(chan int)

  

    go myfunc(ch)

    ch <- 23

    fmt.Println("End Main method")

}

Output:

start Main method
257
End Main method

Closing a Channel

You can also close a channel with the help of close() function. This is an in-built function and sets a flag which indicates that no more value will send to this channel.

 

Syntax:

close()

You can also close the channel using for range loop. Here, the receiver goroutine can check the channel is open or close with the help of the given syntax:

ele, ok:= <- Mychannel

Here, if the value of ok is true which means the channel is open so, read operations can be performed. And if the value of is false which means the channel is closed so, read operations are not going to perform.

Example:

 

// Go program to illustrate how

// to close a channel using for

// range loop and close function

package main

  

import "fmt"

  

// Function

func myfun(mychnl chan string) {

  

    for v := 0; v < 4; v++ {

        mychnl <- "GeeksforGeeks"

    }

    close(mychnl)

}

  

// Main function

func main() {

  

    // Creating a channel

    c := make(chan string)

  

    // calling Goroutine

    go myfun(c)

  

    // When the value of ok is

    // set to true means the

    // channel is open and it

    // can send or receive data

    // When the value of ok is set to

    // false means the channel is closed

    for {

        res, ok := <-c

        if ok == false {

            fmt.Println("Channel Close ", ok)

            break

        }

        fmt.Println("Channel Open ", res, ok)

    }

}

Output:

Channel Open  GeeksforGeeks true
Channel Open  GeeksforGeeks true
Channel Open  GeeksforGeeks true
Channel Open  GeeksforGeeks true
Channel Close  false

Important Points

• Blocking Send and Receive: In the channel when the data sent to a channel the control is blocked in that send statement until other goroutine reads from that channel. Similarly, when a channel receives data from the goroutine the read statement block until another goroutine statement.
• Zero Value Channel: The zero value of the channel is nil.
• For loop in Channel: A for loop can iterate over the sequential values sent on the channel until it closed.

  Syntax:

  for item := range Chnl { 
       // statements..
  }
  

  Example:

   

  // Go program to illustrate how to

  // use for loop in the channel

    

  package main

    

  import "fmt"

    

  // Main function

  func main() {

    

      // Creating a channel

      // Using make() function

      mychnl := make(chan string)

    

      // Anonymous goroutine

      go func() {

          mychnl <- "GFG"

          mychnl <- "gfg"

          mychnl <- "Geeks"

          mychnl <- "GeeksforGeeks"

          close(mychnl)

      }()

    

      // Using for loop

      for res := range mychnl {

          fmt.Println(res)

      }

  }

  Output:

  GFG
  gfg
  Geeks
  GeeksforGeeks
  
  

• Length of the Channel: In channel, you can find the length of the channel using len() function. Here, the length indicates the number of value queued in the channel buffer.

  Example:

   

  // Go program to illustrate how to

  // find the length of the channel

    

  package main

    

  import "fmt"

  a

  // Main function

  func main() {

    

      // Creating a channel

      // Using make() function

      mychnl := make(chan string, 4)

      mychnl <- "GFG"

      mychnl <- "gfg"

      mychnl <- "Geeks"

      mychnl <- "GeeksforGeeks"

    

      // Finding the length of the channel

      // Using len() function

      fmt.Println("Length of the channel is: ", len(mychnl))

  }

  Output:

  Length of the channel is:  4

• Capacity of the Channel: In channel, you can find the capacity of the channel using cap() function. Here, the capacity indicates the size of the buffer.

  Example:

   

  // Go program to illustrate how to

  // find the capacity of the channel

    

  package main

    

  import "fmt"

    

  // Main function

  func main() {

    

      // Creating a channel

      // Using make() function

      mychnl := make(chan string, 5)

      mychnl <- "GFG"

      mychnl <- "gfg"

      mychnl <- "Geeks"

      mychnl <- "GeeksforGeeks"

    

      // Finding the capacity of the channel

      // Using cap() function

      fmt.Println("Capacity of the channel is: ", cap(mychnl))

  }

  Output:

  Capacity of the channel is:  5

• Select and case statement in Channel: In go language, select statement is just like a switch statement without any input parameter. This select statement is used in the channel to perform a single operation out of multiple operations provided by the case block.