WebRequest抓取网页数据出现乱码问题

今天项目里突然有个功能用不起来了，本机确实好的 ，这个很无语 不知道为啥

经过写日志发现html 变成了这样的东西，很是头疼，刚开始各种编码转换，发现这并不是编码的问题

 

 

 

后面观察目标网站多了一个gzip压缩标识，开始尝试解压Gzip 

找到AutomaticDecompression属性:

 

request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;

 

官方说明：https://docs.microsoft.com/en-us/dotnet/api/system.net.httpwebrequest.automaticdecompression?redirectedfrom=MSDN&view=netframework-4.8

 

 

写出日志变正常了，正常写出html源代码 

 

但是一直搞不明白 ，没加这句话之前为啥我本机正常，而服务器却不行。。有点无解啊

 

 

public static string Fetch(string sourceUrl)
        {
            string html = string.Empty;
            HttpWebRequest request = HttpWebRequest.Create(sourceUrl) as HttpWebRequest;
            request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate; //加入了自动减压特性
            using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
            {
                if (response.StatusCode != HttpStatusCode.OK)
                {
                    throw new Exception("Url returns " + response.StatusCode + ", " + response.StatusDescription);
                }

                StreamReader reader = new StreamReader(response.GetResponseStream(), Encoding.GetEncoding("gb2312"));

                try
                {
                    html = reader.ReadToEnd();
                }
                catch (Exception ex)
                {
                    throw ex;
                }
            }
            return html;
        }