客户端开发小结

因为项目需要，改web端项目到客户端，就开始了第一次客户端开发。

采用c#语言开发，其实还是蛮简单的，直接拉控件摆放到面板上，需要发生点击事件就点两下，出来一个双击函数，在函数里写要操作的方法就好了。

这次主要遇到的难点有两方面吧：1、c#调用外部可执行程序gawk；2、awk的使用

1、c#调用外部可执行程序

这里为什么要调用gawk咧？主要是考虑到，程序的主要用途是进行文本的处理，而对于文本的处理awk（gawk是awk的windows版本）处理起来相当简单。

怎么调用gawk咧？

Process p = new Process();
p.StartInfo.FileName = "cmd.exe";
p.StartInfo.UseShellExecute = false;
p.StartInfo.RedirectStandardInput = true;
p.StartInfo.RedirectStandardOutput = true;
p.StartInfo.RedirectStandardError = true;
p.StartInfo.CreateNoWindow = true;
p.Start();
p.StandardInput.WriteLine("cd " + newLocation);
//按照qq号进行过滤
p.StandardInput.WriteLine("gawk ..../*此处省略*/);
p.StandardInput.WriteLine("exit");
string strRst = p.StandardOutput.ReadToEnd();
p.Close();

这里犯了一个错误就是我最开始以为可以直接调用gawk.exe执行文件，但是发现完全行不通，最后还是采用这种本办法，先调用cmd.exe，再进入gawk的目录，调用gawk.exe，执行gawk的命令。

这里的

p.StandardInput.WriteLine("gawk ..../*此处省略*/);

可以不断的write新的语句进去，感觉应该是同步执行的。但是这里如果出现c#代码，就会异步执行，注意。

2、awk的使用总结

awk的使用总结分为一下几点：文件操作、多文件操作

文件操作：
awk 双引号+大括号+执行语句+反大括号+反双引号；（在windows下需用双引号，linux下用单引号）。

gawk "{if(!(++a[substr($1,0,1)]>1)){print $0 >> \"temp2013-04-22-04-14-48.txt\"}else{print $0 >> \".\\sadQQData\\sadQQData2013-04-22-04-14-48.txt\"}}" C:\\Users\\share\\Desktop\\数据包过滤系统\\test.txt

对于去重的方法,网上有很多啦,比如:

gawk "!(a[$1]++)" 22.txt  去掉重复的数据条,会保留一条
gawk "!(++a[$1]>6)" 22.txt  处理能包涵超过6个重复数据的文本,会保留6条

但是对于要去掉所有重复的数据，即只要重复了，一条也不保留的情况。就有点够呛了。我的处理方式是：

首先按照去重的方式过滤一遍，把干净数据包和脏数据包放到两个文件中，这时候脏数据包里的所有数据都表示在源数据包中重复出现超过次数的数据，干净数据包里的数据有正常数据和出现超过次数的数据的几条（看去重的时候阈值）。这个时候再用两个文件比较，把在臧数据包中出现过的数据从干净数据包中去掉。这个时候就是完全干净的数据包了。

多文件处理：
这个一定要看下NR(Number of Record)、FNR(File Number of Record)、NF(NUmber of Field)。

通过对NR=FNR，表示读取第一个文件，NR>FNR，表示读取第二个文件内容。

了解了这几个东东对于多文件操作就很简单啦。看查看：http://www.letuknowit.com/topics/20120329/sample-about-awk-build-in-variable-nr-fnr-nf.html

附上客户端软件打包源码：

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.IO;
using System.Collections;
using System.Runtime.InteropServices;
using System.Threading;
using System.Diagnostics;
namespace FileWatch
{
    public partial class Form1 : Form
    {
        // File System Watcher object.
        private FileSystemWatcher watcher;
        private delegate void UpdateWatchTextDelegate(string newText);

        public Form1()
        {
            InitializeComponent();

            this.watcher = new FileSystemWatcher();
            this.watcher.Deleted +=
               new FileSystemEventHandler(this.OnDelete);
            this.watcher.Renamed +=
               new RenamedEventHandler(this.OnRenamed);
            this.watcher.Changed +=
               new FileSystemEventHandler(this.OnChanged);
            this.watcher.Created +=
               new FileSystemEventHandler(this.OnCreate);

            DirectoryInfo aDir = new DirectoryInfo(@"C:\FileLogs");
            if (!aDir.Exists)
                aDir.Create();
        }

              // Utility method to update watch text.
      public void UpdateWatchText(string newText)
      {
         //lblWatch.Text = newText;
      }

      // Define the event handlers.
      public void OnChanged(object source, FileSystemEventArgs e)
      {
         try
         {
             lblWatch.Text = "123";
            StreamWriter sw =
               new StreamWriter("C:/FileLogs/Log.txt", true);
            sw.WriteLine("File: {0} {1}", e.FullPath,
                         e.ChangeType.ToString());
            sw.Close();
            this.BeginInvoke(new UpdateWatchTextDelegate(UpdateWatchText),
               "Wrote change event to log");
         }
         catch (IOException)
         {
            this.BeginInvoke(new UpdateWatchTextDelegate(UpdateWatchText),
               "Error Writing to log");
         }
      }

      public void OnRenamed(object source, RenamedEventArgs e)
      {
         try
         {
            StreamWriter sw =
               new StreamWriter("C:/FileLogs/Log.txt", true);
            sw.WriteLine("File renamed from {0} to {1}", e.OldName,
                         e.FullPath);
            sw.Close();
            this.BeginInvoke(new UpdateWatchTextDelegate(UpdateWatchText),
               "Wrote renamed event to log");
         }
         catch (IOException)
         {
            this.BeginInvoke(new UpdateWatchTextDelegate(UpdateWatchText),
               "Error Writing to log");
         }
      }

      public void OnDelete(object source, FileSystemEventArgs e)
      {
         try
         {
            StreamWriter sw =
               new StreamWriter("C:/FileLogs/Log.txt", true);
            sw.WriteLine("File: {0} Deleted", e.FullPath);
            sw.Close();
            this.BeginInvoke(new UpdateWatchTextDelegate(UpdateWatchText),
               "Wrote delete event to log");
         }
         catch (IOException)
         {
            this.BeginInvoke(new UpdateWatchTextDelegate(UpdateWatchText),
               "Error Writing to log");
         }
      }

      public void OnCreate(object source, FileSystemEventArgs e)
      {
         try
         {
            StreamWriter sw =
               new StreamWriter("C:/FileLogs/Log.txt", true);
            sw.WriteLine("File: {0} Created", e.FullPath);
            sw.Close();
            this.BeginInvoke(new UpdateWatchTextDelegate(UpdateWatchText),
               "Wrote create event to log");
         }
         catch (IOException)
         {
            this.BeginInvoke(new UpdateWatchTextDelegate(UpdateWatchText),
               "Error Writing to log");
         }
      }

      private void cmdBrowse_Click(object sender, EventArgs e)
      {
          if (FileDialog.ShowDialog() != DialogResult.Cancel)
          {
              txtLocation.Text = FileDialog.FileName;
              cmdWatch.Enabled = true;
          }
      }

      private void cmdWatch_Click(object sender, EventArgs e)
      {
          watcher.Path = Path.GetDirectoryName(txtLocation.Text);
          watcher.Filter = Path.GetFileName(txtLocation.Text);
          watcher.NotifyFilter = NotifyFilters.LastWrite |
          NotifyFilters.FileName | NotifyFilters.Size;
          //先进行文件格式校验
          bool result = checkFile();
          checkNum();
          int i, qqi, ipi, vki, happyi;
          List<string> qq = new List<string>();
          List<string> ip = new List<string>();
          List<string> vk = new List<string>();
          if (result == true) {
              String line;
              StreamReader fileStream = new StreamReader(txtLocation.Text, Encoding.Default);
              for (i = 0; (line = fileStream.ReadLine()) != null;i++){}
              string taskId = DateTime.Now.ToString("yyyy-MM-dd-hh-mm-ss");
              dataAddr.Text = taskId;
              string happyPath = @".\\happyData\\happyData" + taskId + ".txt";
              //dataAddr.Text = happyPath;
              string sadQQPath = @".\\sadQQData\\sadQQData" + taskId + ".txt";
              string sadIPPath = @".\\sadIPData\\sadIPData" + taskId + ".txt";
              string sadVKPath = @".\\sadVKData\\sadVKData" + taskId + ".txt";
              string suspectPath = @".\\suspectData\\suspectData" + taskId + ".txt";
              string temp = @"temp" + taskId + ".txt";
              if (!File.Exists(happyPath))
              {
                  using (StreamWriter sw = File.CreateText(happyPath))
                  {
                      //sw.WriteLine("Hello");
                  }
              }
              if (!File.Exists(sadQQPath))
              {
                  using (StreamWriter sw = File.CreateText(sadQQPath))
                  {
                      //sw.WriteLine("Hello");
                  }
              }
              if (!File.Exists(sadIPPath))
              {
                  using (StreamWriter sw = File.CreateText(sadIPPath))
                  {
                      //sw.WriteLine("Hello");
                  }
              }
              if (!File.Exists(sadVKPath))
              {
                  using (StreamWriter sw = File.CreateText(sadVKPath))
                  {
                      //sw.WriteLine("Hello");
                  }
              }
              string strRst = awk(txtLocation.Text.Replace("\\","\\\\"), happyPath, sadQQPath, sadIPPath, sadVKPath, temp,suspectPath);
              Console.WriteLine(strRst);
              Console.ReadLine();

              //执行完后开始计数
              StreamReader qqStream = new StreamReader(sadQQPath, Encoding.Default);
              StreamReader ipStream = new StreamReader(sadIPPath, Encoding.Default);
              StreamReader vkStream = new StreamReader(sadVKPath, Encoding.Default);
              StreamReader happyStream = new StreamReader(happyPath, Encoding.Default);
              for (qqi = 0; (qqStream.ReadLine()) != null; qqi++){}
              for (ipi = 0; (ipStream.ReadLine()) != null; ipi++){}
              for (vki = 0; (vkStream.ReadLine()) != null; vki++){}
              for (happyi = 0; (happyStream.ReadLine()) != null; happyi++) { }
              //文件总行数
              totalNum.Text = i.ToString();
              ipSusNum.Text = ipi.ToString();
              QQSusNum.Text = qqi.ToString();
              vkSusNum.Text = vki.ToString();
              if (ipSusNum.Text == totalNum.Text || QQSusNum.Text == totalNum.Text || vkSusNum.Text == totalNum.Text)
              {
                  normalNum.Text = "0";
              }
              else {
                  normalNum.Text = happyi.ToString();
              }

              suspectNum.Text = (i-Int32.Parse(normalNum.Text)).ToString();
              fileStream.Close();
          }
          else if (result == false) {

          }
          watcher.EnableRaisingEvents = true;
      }


      private bool checkFile() {
          if (txtLocation.Text!= "")
          {
              if(ipNum.Text==""&&vkNum.Text==""&&QQFirstNum.Text==""&&QQNum.Text==""){
                  MessageBox.Show("请填写过滤规则！");
                  return false;
              }
              return true;
          }
          else {
              MessageBox.Show("请选择文件！");
              return false;
          }

      }
      private void checkNum() {
          if(QQFirstNum.Text == ""){
              QQFirstNum.Text = "9";

          }
          if (QQNum.Text == "") {
              QQNum.Text = "1";

          }
          if (vkNum.Text == "")
          {
              vkNum.Text = "10000000";

          }
          if (ipNum.Text == "")
          {
              ipNum.Text = "10000000";

          }
      }

      private  string awk(string path,string happyPath,string sadQQPath,string sadIPPath,string sadVKPath,string temp,string suspectPath)
      {
          string location = System.AppDomain.CurrentDomain.SetupInformation.ApplicationBase;

          string newLocation = location.Replace("\\","\\\\");
          //MessageBox.Show(newLocation);
          Process p = new Process();
          p.StartInfo.FileName = "cmd.exe";
          p.StartInfo.UseShellExecute = false;
          p.StartInfo.RedirectStandardInput = true;
          p.StartInfo.RedirectStandardOutput = true;
          p.StartInfo.RedirectStandardError = true;
          p.StartInfo.CreateNoWindow = true;
          p.Start();
          p.StandardInput.WriteLine("cd " + newLocation);
          //按照qq号进行过滤
          p.StandardInput.WriteLine("gawk \"{if(!(++a[substr($1,0," + Int32.Parse(QQFirstNum.Text) + ")]>" + Int32.Parse(QQNum.Text) + ")){print $0 >> \\\"" + temp + "qq\\\"}else{print $0 >> \\\"" + sadQQPath + "\\\"}}\" "+path+"");
          //按照ip进行过滤
          p.StandardInput.WriteLine("gawk \"{if(!(++a[$2]>" + Int32.Parse(ipNum.Text) + ")){print $0 >> \\\"" + temp + "ip\\\"}else{print $0 >> \\\"" + sadIPPath + "\\\"}}\" " + path);

          //p.StandardInput.WriteLine("gawk \"{if(NR==FNR){a[$2]++}if(NR>FNR){if(a[$2]>=1){print $0 >>\\\""+sadIPPath+"\\\"}else{print $0 >> \\\""+temp+"ip2\\\"}}}\" " + sadIPPath +" "+ temp+"ip");
          //按照vk进行过滤
          p.StandardInput.WriteLine("gawk \"{if(!(++a[$3]>" + Int32.Parse(vkNum.Text) + ")){print $0 >> \\\"" + temp + "vk\\\"}else{print $0 >> \\\"" + sadVKPath + "\\\"}}\" " + path);
          //对qq好完全去重
          p.StandardInput.WriteLine("gawk \"{if(NR==FNR){a[substr($1,0," + Int32.Parse(QQFirstNum.Text) + ")]++}if(NR>FNR){if(a[substr($1,0," + Int32.Parse(QQFirstNum.Text) + ")]>=1){print $0 >> \\\"" + sadQQPath + "\\\"}else{print $0 >> \\\"" + temp + "qq2\\\"}}}\" " + sadQQPath + " " + temp + "qq");

          //对重复ip完全去重
          p.StandardInput.WriteLine("gawk \"{if(NR==FNR){a[$2]++}if(NR>FNR){if(a[$2]>=1){print $0 >> \\\"" + sadIPPath + "\\\"}else{print $0 >> \\\"" + temp + "ip2\\\"}}}\" " + sadIPPath + " " + temp + "ip");

          //对vk完全去重
          p.StandardInput.WriteLine("gawk \"{if(NR==FNR){a[$3]++}if(NR>FNR){if(a[$3]>=1){print $0 >> \\\"" + sadVKPath + "\\\"}else{print $0 >> \\\"" + temp + "vk2\\\"}}}\" " + sadVKPath + " " + temp + "vk");
          //过滤完之后将tempqq、tempip、tempvk进行合并去重得到干净数据包
          DialogResult dialogResult = MessageBox.Show("数据包运行可能会比较慢，请不要着急~~", "温馨提示", MessageBoxButtons.OK, MessageBoxIcon.Question);
          if (dialogResult == DialogResult.OK)
          {
              //Application.ExitThread();

          FileInfo tempqq2 = new FileInfo(temp + "qq2");
          if (tempqq2.Exists)
          {
              //MessageBox.Show("qq2");
              p.StandardInput.WriteLine("gawk \"{print $0 >> \\\"" + temp + "total\\\"}\" " + temp + "qq2");
          }
          else
          {
              //MessageBox.Show("qq");
              p.StandardInput.WriteLine("gawk \"{print $0 >> \\\"" + temp + "total\\\"}\" " + temp + "qq");
          }
          FileInfo tempip2 = new FileInfo(temp + "ip2");
          if (tempip2.Exists)
          {
              p.StandardInput.WriteLine("gawk \"{print $0 >> \\\"" + temp + "total\\\"}\" " + temp + "ip2");
          }
          else
          {
              p.StandardInput.WriteLine("gawk \"{print $0 >> \\\"" + temp + "total\\\"}\" " + temp + "ip");
          }
          FileInfo tempvk2 = new FileInfo(temp + "vk2");
          if (tempvk2.Exists)
          {
              p.StandardInput.WriteLine("gawk \"{print $0 >> \\\"" + temp + "total\\\"}\" " + temp + "vk2");
          }
          else {
              p.StandardInput.WriteLine("gawk \"{print $0 >> \\\"" + temp + "total\\\"}\" " + temp + "vk");
          }
          //p.StandardInput.WriteLine("gawk \"{print $0 >> \\\"" + temp + "total\\\"}\" " + temp + "ip2");
          //p.StandardInput.WriteLine("gawk \"{print $0 >> \\\"" + temp + "total\\\"}\" " + temp + "vk2");
          p.StandardInput.WriteLine("gawk \"{if(++a[$0]>=3){print $0 >> \\\"" + happyPath + "\\\"}}\" " + temp + "total");
          }
          p.StandardInput.WriteLine("exit");
          string strRst = p.StandardOutput.ReadToEnd();
          p.Close();
          //删除临时文件
          FileInfo tempqq = new FileInfo(temp+"qq");
          if (tempqq.Exists)
          {
             tempqq.Delete();
          }
          FileInfo tempip = new FileInfo(temp + "ip");
          if (tempip.Exists)
          {
              tempip.Delete();
          }
          FileInfo tempvk = new FileInfo(temp + "vk");
          if (tempvk.Exists)
          {
             tempvk.Delete();
          }
          FileInfo tempqq3 = new FileInfo(temp + "qq2");
          if (tempqq3.Exists)
          {
              tempqq3.Delete();
          }
          FileInfo tempip3 = new FileInfo(temp + "ip2");
          if (tempip3.Exists)
          {
              tempip3.Delete();
          }
          FileInfo tempvk3 = new FileInfo(temp + "vk2");
          if (tempvk3.Exists)
          {
              tempvk3.Delete();
          }
          FileInfo temptotal = new FileInfo(temp + "total");
          if (temptotal.Exists)
          {
             temptotal.Delete();
          }
          return strRst;
      }
      private void txtLocation_TextChanged(object sender, EventArgs e)
      {

      }

      private void Form1_Load(object sender, EventArgs e)
      {

      }

      private void FileDialog_FileOk(object sender, CancelEventArgs e)
      {

      }

      private void ipNum_TextChanged(object sender, EventArgs e)
      {

      }

      private void fileToolStripMenuItem_Click(object sender, EventArgs e)
      {

      }

      private void 使用说明ToolStripMenuItem_Click(object sender, EventArgs e)
      {
          MessageBox.Show("");
      }

      private void 文本规则ToolStripMenuItem_Click(object sender, EventArgs e)
      {
          MessageBox.Show("上传数据包为TXT格式！");
      }

      private void 联系ToolStripMenuItem_Click(object sender, EventArgs e)
      {
          MessageBox.Show("如果问题可联系sharexie、lesliezhang！");
      }

      private void 关于ToolStripMenuItem_Click(object sender, EventArgs e)
      {
          MessageBox.Show("本软件为数据作弊分包处理，版本v1.0.0");
      }

      private void 个数规则ToolStripMenuItem_Click(object sender, EventArgs e)
      {
          MessageBox.Show("个数设置为很大（超过数据总量），表示不对当前列过滤。");
      }

      private void 时间编号ToolStripMenuItem_Click(object sender, EventArgs e)
      {
          MessageBox.Show("时间标号方便提取干净数据和灰名单。");
      }
    }
}