博客园-智慧掩盖真相http://www.cnblogs.com/birdshover/天之道，不争而善胜。 zh-cnThu, 24 Jul 2008 05:58:02 GMTThu, 24 Jul 2008 05:58:02 GMT60Lucene.Net 2.3.1版变化之TokenStream（搞笑了）http://www.cnblogs.com/birdshover/archive/2008/07/23/1249591.htmlBirdshoverBirdshoverWed, 23 Jul 2008 06:59:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/07/23/1249591.htmlhttp://www.cnblogs.com/birdshover/comments/1249591.htmlhttp://www.cnblogs.com/birdshover/archive/2008/07/23/1249591.html#Feedback0http://www.cnblogs.com/birdshover/comments/commentRss/1249591.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1249591.html//2.1版publicab  阅读全文

[新闻]Mozilla将于本周五发布Firefox 3.1第一个预览版]]>如何在Lucene.Net中一个Document使用不同的分词（二）http://www.cnblogs.com/birdshover/archive/2008/07/22/1249013.htmlBirdshoverBirdshoverTue, 22 Jul 2008 12:36:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/07/22/1249013.htmlhttp://www.cnblogs.com/birdshover/comments/1249013.htmlhttp://www.cnblogs.com/birdshover/archive/2008/07/22/1249013.html#Feedback3http://www.cnblogs.com/birdshover/comments/commentRss/1249013.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1249013.html昨天说了索引时，如何在Lucene.Net中一个Document使用不同的分词。

 

前文地址：如何在Lucene.Net中一个Document使用不同的分词（一）

 

今天讲的搜索时使用不同分词。

 

 

方法一、使用两个Parser就可以使用两个分词

 

            IndexSearcher searcher = new IndexSearcher("C:\\LuceneIndex");
            BooleanQuery bquery = new BooleanQuery();
            QueryParser titlequery = new QueryParser("title",new Lucene.Net.Analysis.Standard.StandardAnalyzer());
            QueryParser contentquery = new QueryParser("content", new Lucene.Net.Analysis.SimpleAnalyzer());

            bquery.Add(titlequery.Parse("标题"), BooleanClause.Occur.MUST);
            bquery.Add(titlequery.Parse("内容"), BooleanClause.Occur.MUST);

            Hits hits = searcher.Search(bquery);

            searcher.Close();

 

分析下表达式就是 +title:"标题" +content:"内容".

 

方法二、而如果要使用MultiFieldQueryParser来解析呢。

 

简单的做法就是用MultiFieldQueryParser类的静态Parser方法得到Query,形式上和用QueryParser 差不多。

而如果要 用MultiFieldQueryParser.Parse(new string[] { }, new string[] { }, new Lucene.Net.Analysis.Standard.StandardAnalyzer());这种实例化，那么又要进行框架改造了。

 

MultiFieldQueryParser是继承自QueryParser 类的，显然我们不需要在QueryParser 使用多种分词。但是MultiFieldQueryParser实例的Parse方法却是使用的父类的。那就不得不改造父类了。

 

分词处理是在QueryParser 类470行，由GetFieldQuery方法的TokenStream source = analyzer.TokenStream(field, new System.IO.StringReader(queryText));完成的。

 

这个地方改造就要注意技巧了，因为这个动作执行的方法被很多地方都用到了，改造成本比较高，而且一不小心就出错了。

重载一个，或者干脆直接换的方法的参数并不友好。因为QueryParser 类本身还有可能被使用，你换掉了，用MultiFieldQueryParser没问题，但是用QueryParser 就可能会出问题。

 

其实这个时候你的眼睛已经被代码迷惑，呵呵，GetFieldQuery方法在MultiFieldQueryParser被循环调用了。

MultiFieldQueryParser 的GetFieldQuery方法，循环调用了Query q = base.GetFieldQuery(fields[i], queryText);

 

是不是想到了什么呢？

 

首先给MultiFieldQueryParser类增加一个字段

private Analysis.Analyzer[] analyzers;

然后增加一个构造函数

        public MultiFieldQueryParser(Analysis.Analyzer[] analyzers, System.String[] fields)
            : this(fields, null)
        {
            if (fields.Length != analyzers.Length)
                throw new Exception("赋予的字段跟分词器数量不匹配");
            this.analyzers = analyzers;
        }

 

为什么要把新构造函数的参数调换位置？是因为this(fields, null)引用另外一个构造函数时候，如果不调换，会导致无法识别用的是哪个方法。

 

接着在GetFieldQuery方法Query q = base.GetFieldQuery(fields[i], queryText);之前切换掉分词器：

this.analyzer = this.analyzers[i];

 

代码就是

        public override Query GetFieldQuery(System.String field, System.String queryText, int slop)
        {
            if (field == null)
            {
                System.Collections.ArrayList clauses = System.Collections.ArrayList.Synchronized(new System.Collections.ArrayList(10));
                for (int i = 0; i < fields.Length; i++)
                {
                    //切换分词器
                    this.analyzer = this.analyzers[i];
                    Query q = base.GetFieldQuery(fields[i], queryText);
                    if (q != null)
                    {
                        //If the user passes a map of boosts
                        if (boosts != null)
                        {
                            //Get the boost from the map and apply them
                            System.Single boost = (System.Single) boosts[fields[i]];
                            //if (boost != null)      // {{Aroush-2.1 there is no null for 'boost'
                            if (boosts[fields[i]] != null)      // {{Aroush-2.1 will this line do?
                            {
                                q.SetBoost((float) boost);
                            }
                        }
                        if (q is PhraseQuery)
                        {
                            ((PhraseQuery) q).SetSlop(slop);
                        }
                        if (q is MultiPhraseQuery)
                        {
                            ((MultiPhraseQuery) q).SetSlop(slop);
                        }
                        clauses.Add(new BooleanClause(q, BooleanClause.Occur.SHOULD));
                    }
                }
                if (clauses.Count == 0)
                // happens for stopwords
                    return null;
                return GetBooleanQuery(clauses, true);
            }
            return base.GetFieldQuery(field, queryText);
        }

 

至此，就改造完毕了。

 

就可以像下面一样调用了：

 

 

            IndexSearcher searcher = new IndexSearcher("C:\\LuceneIndex");
            //BooleanQuery bquery = new BooleanQuery();
            //QueryParser titlequery = new QueryParser("title",new Lucene.Net.Analysis.Standard.StandardAnalyzer());
            //QueryParser contentquery = new QueryParser("content", new Lucene.Net.Analysis.SimpleAnalyzer());
            MultiFieldQueryParser multiparser = new MultiFieldQueryParser(new Lucene.Net.Analysis.Analyzer[] { new Lucene.Net.Analysis.Standard.StandardAnalyzer(), new Lucene.Net.Analysis.SimpleAnalyzer() }, new string[] { "title","content" });
            //bquery.Add(titlequery.Parse("标题"), BooleanClause.Occur.MUST);
            //bquery.Add(contentquery.Parse("内容"), BooleanClause.Occur.MUST);
            Hits hits = searcher.Search(multiparser.Parse("标题"));

            searcher.Close();

 

 

全文完。 by yurow @http://www.cnblogs.com/birdshover/



[新闻]瑞星将向个人用户免费1年]]>如何在Lucene.Net中一个Document使用不同的分词（一）http://www.cnblogs.com/birdshover/archive/2008/07/21/1247966.htmlBirdshoverBirdshoverMon, 21 Jul 2008 13:01:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/07/21/1247966.htmlhttp://www.cnblogs.com/birdshover/comments/1247966.htmlhttp://www.cnblogs.com/birdshover/archive/2008/07/21/1247966.html#Feedback4http://www.cnblogs.com/birdshover/comments/commentRss/1247966.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1247966.html 说明：代码没有经过测试。


在Lucene.Net中一个Document对象创建并不涉及到分词，分词是跟IndexWriter相关的。

以类库自带的StandardAnalyzer分词举例，创建一个索引的过程如下

/* 实例化写入 */
IndexWriter writer = new IndexWriter(new Lucene.Net.Analysis.Standard.StandardAnalyzer(),true);
/* 建立文档 */
Document doc = new Document();
doc.Add(new Field("id", "123", Field.Store.YES, Field.Index.TOKENIZED));
doc.Add(new Field("title", "这个是标题", Field.Store.YES, Field.Index.TOKENIZED));
doc.Add(new Field("content", "这个是内容", Field.Store.YES, Field.Index.TOKENIZED));
/*  写入文档 */
writer.AddDocument(doc);
/* 优化索引结构 */
writer.Optimize();
/* 关闭写入 */
writer.Close();

在这样的代码里想一个文档不同字段使用不同分词是不可能的。如果要让每个字段使用不同分词，那么当然是给每个字段加上分词器属性。

先找到Lucene.Net的Field类，发现Field类的字段是继承自AbstractField父类的。在AbstractField先加上
protected internal Analysis.Analyzer analyzer;

然后增加一个构造函数：
        protected internal AbstractField(string name, Field.Store store, Field.Index index, Field.TermVector termVector, Analysis.Analyzer analyzer)
            : this(name, store, index, termVector)
        {
            this.analyzer = analyzer;
        }

在Field类增加构造函数：
        public Field(System.String name, System.String value_Renamed, Store store, Index index, TermVector termVector, Analysis.Analyzer analyzer)
            : base(value_Renamed, store, index, termVector, analyzer)
        {

        }


在接口Fieldable中增加方法GetAnalyzer;
Analysis.Analyzer GetAnalyzer();


并且在AbstractField类中实现GetAnalyzer方法：

        public virtual Analysis.Analyzer GetAnalyzer()
        {
            return this.analyzer;
        }

修改DocumentWriter类的 179行 ，如果因为版本不一样不在179行，请查找
“TokenStream stream = analyzer.TokenStream(fieldName, reader);”

然后换成

TokenStream stream = field.GetAnalyzer().TokenStream(fieldName, reader);

OK，写入的部分搞定了，读取的部分明天再弄。

现在就可以这么用了：

/* 实例化写入 */
IndexWriter writer = new IndexWriter(new Lucene.Net.Analysis.Standard.StandardAnalyzer(),true);
/* 建立文档 */
Document doc = new Document();
doc.Add(new Field("id", "123", Field.Store.YES, Field.Index.TOKENIZED, Field.TermVector.NO, new Lucene.Net.Analysis.KeywordAnalyzer()));
doc.Add(new Field("title", "这个是标题", Field.Store.YES, Field.Index.TOKENIZED, Field.TermVector.NO, new Lucene.Net.Analysis.Standard.StandardAnalyzer()));
doc.Add(new Field("content","这个是内容", Field.Store.YES, Field.Index.TOKENIZED, Field.TermVector.NO, new Lucene.Net.Analysis.SimpleAnalyzer()));
/*  写入文档 */
writer.AddDocument(doc);
/* 优化索引结构 */
writer.Optimize();
/* 关闭写入 */
writer.Close();

当然，这个时候，IndexWriter 的分词器实际上是无用的，也可以判定Field给定的分词器是否为null，然后用IndexWriter 的作为默认分词器。

by yurow @ http://www.cnblogs.com/birdshover/

更新第二部分地址：如何在Lucene.Net中一个Document使用不同的分词（二）

[新闻]中国互联网历史上最伟大的产品TOP10（二）]]>Lucene.Net 已经更新到2.3.1 beat版http://www.cnblogs.com/birdshover/archive/2008/07/20/1247029.htmlBirdshoverBirdshoverSun, 20 Jul 2008 06:33:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/07/20/1247029.htmlhttp://www.cnblogs.com/birdshover/comments/1247029.htmlhttp://www.cnblogs.com/birdshover/archive/2008/07/20/1247029.html#Feedback1http://www.cnblogs.com/birdshover/comments/commentRss/1247029.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1247029.html今天更新了下才注意到，Lucene.Net已经更新到2.3.1版了，而java版的是2.3.2
2.3.1要通过svn下载

地址：
http://incubator.apache.org/lucene.net/

svn地址：
https://svn.apache.org/repos/asf/incubator/lucene.net/


更新历史：

Apache Lucene.Net History
-------------------------


15JJul08:
 - Release:  Apache Lucene.Net.2.3.1 build 002 "Beta"
 - Port: Prted the "Test" code from Java to C#.
 - Fix: A number of issues in Lucene.Net Core code -- Thanks to Doug Sale
 - Fix: A number of issues in Lucene.Net Test code -- Thanks to Doug Sale


24Jun08:
 - Dev Release:  Apache Lucene.Net.2.3.1 build 001 "Alpha"
 - Port: Prted the "Core" and "Demo" code from Java to C#.
 - Issues: Test code is not released yet.
 - Note: Requires Visual Studio 2005 or later and .NET 2.0 or later.
 - Note: There is no 2.3.0 or 2.2 release (to correspond with the Java Lucene release).


10Dec07:
 - Release:  Apache Lucene.Net.2.1 build 003 "Release Candidate"
 - Fix: LUCENENET-55 "Documents.DateTools has issue creating a Date in StringToDate()"
 - Fix: LUCENENET-56 "Incorrect file in TestLockFactory.RmDir()"
 - Fix: LUCENENET-57 "DocHelper in Tests not creating UTF8 Cleanly"
 - Fix: LUCENENET-58 "Issue in CheckHits c# doesn't perform an Assert against a hashtable"
 - Fix: LUCENENET-59 "QueryUtils has some invalid Asserts"
 - Fix: LUCENENET-61 "Issue testing Backwards Compatibility"
 - Fix: LUCENENET-62 "IndexReader.IndexExists() Fails if directory doesn't exists."
 - Fix: LUCENENET-63 "FieldCacheImpl tries to parse a float in f format"
 - Fix: LUCENENET-64 "TestDateFilter incorrectly gets total milliseconds"
 - Fix: LUCENENET-65 "Test case "TestSerializable" uses the stream after closing"
 - Fix: LUCENENET-66 "TestMergeAfterCopy fails in IndexFileDeleter"
 - Fix: LUCENENET-67 "Bug in TestIndexWriter.TestAddIndexOnDiskFull"
 - Fix: LUCENENET-68 "Bug in TestIndexWriterDelete.TestOperationsOnDiskFull"
 - Fix: LUCENENET-69 "FSIndexInput.isFDValid() not ported correctly"
 - Fix: LUCENENET-70 "TestParallelReader.TestDocument fails because of bug in "ParallelReader.Document(int n, FieldSelector fieldSelector)" method"
 - Fix: LUCENENET-71 "TestParallelTermEnum.Test1 fails because of bug in "ParallelReader.Next" method"
 - Fix: LUCENENET-72 "TestIndexReader bugs"
 - Fix: LUCENENET-73 "TestDoc.cs --> access to a closed stream"
 - Fix: LUCENENET-74 "SimpleFSLockFactory can not obtain lock correctly."
 - Fix: LUCENENET-75 "FSDirectory does not correctly handle directory cache "DIRECTORIES""
 - Fix: LUCENENET-76 "DisjunctionMaxQuery has unnecessary clone which causes it to fail unit tests"
 - Fix: LUCENENET-77 "Bug in TestBinaryDocument.cs"
 - Fix: LUCENENET-81 "TestTermVectorsWriter.cs bug"
 - Fix: LUCENENET-82 "NUnite test for TestSimpleExplanations"
 - Fix: LUCENENET-83 "NUnite test for TestComplexExplanations"
 - Fix: LUCENENET-84 "Nunit test for TestMultiFieldQueryParser"
 - Fix: LUCENENET-85 "SupportClass.Parse and System.Globalization.CultureInfo.CurrentCulture.NumberFormat.NumberDecimalSeparator"
 - Fix: LUCENENET-87 "NUnite test for TestQueryParser"
 - Fix: LUCENENET-88 "NUnit test for TestQueryParser -2"
 - Fix: LUCENENET-89 "NUnit test for TestQueryParser -3"
 - Fix: LUCENENET-90 "Nunit test for TestIndexModifier.TestIndex"
 - Fix: LUCENENET-91 "NUnit test for TestQueryParser.TestStarParsing"
 - Fix: LUCENENET-92 "NUnite test for QueryParser.TestMultiAnalyzer."
 - Fix: LUCENENET-93 "NUnite test for Search.TestRemoteSearchable"
 - Fix: LUCENENET-94 "NUnit test for Search.TestSort (RemoteSearchable issues)" (only TestSort.patch applied)
 - Fix: LUCENENET-96 "NUnit test for Lucene.Net.Store.TestLockFactory.TestLockClassProperty"
 - Fix: LUCENENET-101 "Using incorrect base when opening index"
 - Fix: LUCENENET-100 "Problem with remoting of IComparable[] in FieldDoc.cs"
 - Fix: LUCENENET-104 "Name of first index segment is empty string"


11Aug07:
 - Release:  Apache Lucene.Net.2.1 build 002 "Beta"
 - Port: Ported the "Test" code from Java to C#
 - Fix: LUCENENET-47: "Make up for Constansts.cs"
 - Fix: LUCENENET-48 "Clone method of SegmentInfos.cs does'nt copy local fields/variables."
 - Fix: LUCENENET-50 "Improvement for FSDirectory."
 - Fix: LUCENENET-52 "IndexFileDeleter in svn trunk"
 - Fix: LUCENENET-53 "SegmentsInfos.GetCurrentSegmentGeneration works incorrectly"
 - Issues: A number of NUnit tests are failing.


01May07:
 - Dev Release:  Apache Lucene.Net.2.1 build 001 "early-Alpha"
 - Issues: Optimizing an index will cause an exception.
 - Issues: The "Test" code has not yet be ported for this release.
 - Issues: Code in "contrib" have not been validated to work with this release.


30Apr07:
 - Patch:  Apache Lucene.Net.2.0 build 005 "final"
 - Fix: LUCENENET-37 "Exception while search in Lucene.Net and Index prepared by Lucene Java"


11Mar07:
 - Release:  Apache Lucene.Net.2.0 build 004 "final"
 - Fix: LUCENENET-36 "Countries using "," as decimal separator gets an exception in QueryParser.cs with a query like color~0.5"
 - Fix: LUCENENET-35 "Tokenizer.Close should check if input is null"
 - Fix: LUCENENET-33 "Frequent exceptions at Single Parse(String s)"
 - Fix: LUCENENET-32 "Check hashtable in PhraseQuery.ExtractTerms for existing keys"
 - Fix: LUCENENET-31 "elimate exception when casting TermFreqVector"
 - Fix: LUCENENET-30 "Unnecessary boxing of bytes"


27Dec06:
 - Release:  Apache Lucene.Net.2.0 build 003 "final"
 - Fix: Lucene.Net.Search.TestSort.TestInternationalMultiSearcherSort -- NUnit test now passes
 - Fix: Lucene.Net.Search.TestSort.TestInternationalSort -- NUnit test now passes


27Nov06:
 - Release:  Apache Lucene.Net.2.0 build 002 "Beta"
 - Lucene.Net.Demo.SearchFiles.cs -- ported new code
 - Lucene.Net.Index.SegmentReader.Get() -- changed Exception to SystemException
 - Lucene.Net.Search.StringIndex.cs -- added a Close() method (to fix sort memory leak defect)
 - Lucene.Net.Search.FieldCacheImpl.cs -- added a Close() method (to fix sort memory leak defect)
 - Lucene.Net.Search.FieldSortHitQueue.cs -- added a Close() method (to fix sort memory leak defect)
 - Lucene.Net.Search.IndexSearcher.cs -- added a Close() method (to fix sort memory leak defect)
 - Lucene.Net.Search.MatchAllDocsQuery.Clone() -- removed this unused methods
 - Lucene.Net.Search.MultiPhraseQuery.Clone() -- removed this unused methods
 - Lucene.Net.Search.PrefixQuery.Clone() -- removed this unused methods
 - Lucene.Net.Search.RangeQuery.Clone() -- removed this unused methods
 - Lucene.Net.Index.FieldInfos.FieldName()/FieldInfo() -- avoid the use of exception throwing
 - Issues: Lucene.Net.Search.TestSort.TestInternationalMultiSearcherSort -- NUnit test FAILS
 - Issues: Lucene.Net.Search.TestSort.TestInternationalSort -- NUnit test FAILS


17Aug06:
 - Release:  Apache Lucene.Net.2.0 build 001 "Alpha"


13Jul06:
 - Release:  Apache Lucene.Net.1.9.1 build 001
 - Port: Lucene.Net.Documents.TimeToString() -- re-based with the Java version
 - Port: Lucene.Net.Index.IndexWriter.SetMaxBufferedDocs() -- re-based with the Java version
 - Port: Lucene.Net.Store.BufferedIndexOutput.WriteBytes() -- re-based with the Java version
 - Port: Lucene.Net.Store.RAMOutputStream.FlushBuffer() -- re-based with the Java version
 - Port: Lucene.Net.Demo.* -- re-based with the Java version
 - Port: Test.Lucene.Net.Index.TestCompoundFile.TestLargeWrites() -- new test case added
 - Port: Test.Lucene.Net.StoreTest.Test() -- re-based with the Java version


09Jul06:
 - Release:  Apache Lucene.Net.1.9 RC1 build 005 "final"
 - Fix: Lucene.Net.Search.MultiPhraseQuery.ToString() -- was skipping one too many item during the iteration
 - Fix: Lucene.Net.Index.ParallelReader.GetFieldNames() -- was adding the object instead of the dictionary value
 - Fix: Lucene.Net.Index.ParallelReader.Add() -- was adding the object instead of the dictionary value
 - Fix: Lucene.Net.Store.Obtain() -- changed name from obtain() to Obtain() (lower case to upper case)
 - Fix: Lucene.Net.Index.SegmentReader.~SegmentReader() -- removed: System.Threading.Thread.SetData()
 - Fix: Lucene.Net.Index.TermInfosReader.~TermInfosReader() -- removed: System.Threading.Thread.SetData()
 - Fix: Lucene.Net.Documents.DateField.DATE_LEN -- must use SupportClass.Number.ToString() to get the length
 - Fix: Lucene.Net.Util.ToStringUtils.Boost() -- wasn't adding ".0" when the value doesn't have a remainder
 - Fix: Lucene.Net.Index.SegmentReader.CreateFakeNorms() -- was returning the wrong data member
 - Fix: Lucene.Net.Documents.NumberTools -- value of MIN_STRING_VALUE and MAX_STRING_VALUE were wrong
 - Fix: Test.Lucene.Net.Analysis.TestISOLatin1AccentFilter.TestU() -- file was not saved as unicode; thanks to Ben Tregenna
 - Fix: Test.Lucene.Net.TestSearchForDuplicates.TestRun() -- wasn't using MemoryStream correctly
 - Fix: Test.Lucene.Net.TestSearch.TestSearch_Renamed_Method() -- wasn't using MemoryStream correctly
 - Fix: Test.Lucene.Net.* -- replaced "[TestFixtureSetUp]" to "[SetUp]" and "[FixtureTearDown]" to "[TearDown]"
 - Fix: Test.Lucene.Net.Index.TestParallelReader -- was comparing objects instead of the dictionary value
 - Fix: Test.Lucene.Net.Index.TestSegmentReader -- was comparing objects instead of the dictionary value
 - Fix: Test.Lucene.Net.Index.TestTermVectorWriter -- was not calling InitBloc()
 - Fix: Test.Lucene.Net.Analysis.Setup() -- was adding to the hash without checking if the key already exist
 - Fix: Test.Lucene.Net.Index.TestMultiReader/TestSegmentReader/TestSegmentTermDoc.cs -- all those needed their class member variables re-initialized


13Jun06:
 - Release:  Apache Lucene.Net.1.9 RC1 build 004 Beta
 - Fix: Lucene.Net.Search.FieldCacheImpl.GetFloats() -- must remove 'f" or "F" for System.Single.Parse() to work
 - Fix: Lucene.Net.Index.GetFieldnames() -- was storing the object instead the value in the object
 - Fix: Test.Lucene.Net.Index.CollectionContains() -- need to compare strings, not objects
 - Fix: Test.Lucene.Net.Serch.TestKnownSetOfDocuments() -- don't fail if an item doesn't exist


03Jun06:
 - Release: Apache Lucene.Net.1.9 RC1 build 003 Alpha
 - Note: This is the first release of Lucene.Net 1.9 to SVN
 - Note: Added ZIP compression support via reflection.  Thanks to Eyal Post
 - Note: Fixed bugs in the code which were expose via the NUnit "Test" code
 - Note: NUnit "Test" code has been ported to Lucene.Net.  Out of 307 tests 58 are failing
 - Note: There are still some remaining port work to be done; look for the text "Aroush" in the code
 - Issue: There are still some code not fully ported; search for "Aroush" to find those codes
 - Issue: The NUnit test code has not been ported yet
 - Issue: Demo.IndexHtml won't work due to some bug in the area of in-memory stream porting


07Feb06:
 - Release: Lucene.Net.1.9 RC1 build 002 Alpha
 - Note: This release is based on the current Java code release of 1.9 RC1
 - Note: This release contains all the fixes currently implemented for 1.4.3 build 004
 - Note: There are still some remaining port work to be done; look for the text "Aroush" in the code
 - Issue: The NUnit test code has not been ported yet
 - Issue: Demo.IndexHtml won't work due to some bug in the area of in-memory stream porting


26May05:
 - Release: Lucene.Net.1.9 RC1 build 001 Alpha
 - Issue: There are still some code not fully ported; search for "Aroush" to find those codes.
 - Issue: The NUnit test code has not been ported yet (the current code is still based on 1.4.3 final)


15Dec05:
 - Release: Lucene.Net.1.4.3 final build 004 to the Apache incubator site
 - Clean-up: VS.Net project settings.


21Nov05:
 - Release: Lucene.Net.1.4.3 final build 004
 - Fix: Fixed System.Object cast in Lucene.Net.Search.RangeQuery and Lucene.Net.Analysis.PorterStemFilter -- Thanks to Jason
 - Note: Moved project to Apache at: http://svn.apache.org/repos/asf/incubator/lucene.net/


23Oct05:
 - Release: Lucene.Net.1.4.3 final build 003
 - Fix: Memory leak -- Thanks to Marcus.
 - Fix: Remoting class and other misused of System.Object cast -- Thanks to Jason


26May05:
 - Release: Lucene.Net.1.9 RC1 build 001 Alpha
 - Issue: There are still some code not fully ported; search for "Aroush" to find those codes.
 - Issue: The NUnit test code has not been ported yet (the current code is still based on 1.4.3 final)


22Feb05:
 - Release: Lucene.Net.1.4.3 final build 002
 - Fix: Lucene.Net.Index.MultiReader.cs -- fixed GetIndexedFieldNames()


13Feb05:
 - Release: Lucene.Net.1.4.3 final build 001
 - Fix: Lucene.Net.Search.TermQuery.Clone() -- was returning null, so removed it.
 - Fix: Lucene.Net.Documents.Field.cs -- replaced StreamReader() to TextReader()
 - Fix: Lucene.Net.Search.RemoteSearchable.cs -- added InitializeLifetimeService()
 - Fix: Lucene.Net.Document.DateField.cs -- fixed StringToDate()
 - Fix: Lucene.Net.Store.RAMDirectory.cs -- fixed RAMDirectory()
 - Issue: Demo.IndexHtml won't work due to some bug in the area of in-memory stream porting


05Jan05:
 - Release: Lucene.Net.1.4.3 RC2 build 001
 - Fix: Lucene.Net.Search.Query.MergeBooleanQueries(); cast type was wrong.
 - Fix: Demo.IndexHtml; can index now but searching on the index file won't work, yet.


21Dec04:
 - Release: Lucene.Net.1.4.3 RC1 build 001
 - Fix: Document.DateField.StringToTime() -- can't use System.Convert.ToInt64()


06Dec04:
 - Release: Lucene.Net.1.4.3 beta build 001


22Nov04:
 - Release: Lucene.Net.1.4.0 RC1 build 001
 - Fix: Changed some data types from 'int' to 'long'
 - Fix: changed some codes from base.Clone() to this.Clone()


10Nov04:
 - Release: Lucene.Net.1.4.0 beta build 002 (targeted for "final")
 - Fix: Document.Fields() now returns an IEnumerable.  Thanks to Marcus.
 - Fix: Use SupportClass.Single.Parse() to parse numbers ending with 'f'


03Nov04:
 - Release: Lucene.Net.1.4.0 beta build 001
 - Release Note: 159 test run, 7 failed due to non-existing remote-Lucene setup
 - Failed Test:
 - Search.TestRemoteSearchable.*
 - Search.TestSort.TestNormalizedScores/TestRemoteCustomSort/TestRemoteSort
 - Fix: Removed 'ref' keyword usage from DateTime.
 - Port: TestQueryParser.cs -- done. thanks to Radu.
 - Fix: TestQueryParser.cs -- date-range formatting issues.
 - Fix: Lucene.Net.QueryParsers.GetRangeQuery() -- date-range formatting issues.
 - Fix: Use: System.IO.Path.Combine("path-1", "path-2") instead of "path-1" + "\\" + "path-2"


29Oct04:
 - Port: Ported "test" out of ~160 tests, only ~14 fail.
 - Port: SupportClass.Character.ForDigit() -- done. thanks to Monsur
 - Port: Lucene.Net.Documents.DateField.cs -- done. thanks to Monsur
 - Fix: Lucene.Net.Search.BooleanScorer
 - Fix: Lucene.Net.Search.FilteredQuery
 - Fix: Lucene.Net.Search.SortScorers


25Oct04:
 - Port: Ported "test" more than 2/3 of the NUnit test now pass
 - Fix: Fixed query (ToString()) formatting issue ("2" -> "2.0") to pass NUnit test
 - Fix: Field items iteration


19Oct04:
 - Fix: TermInfoReader.get_Renamed() -> TermInfoReader.Get()
 - Fix: Searching now works


18Oct04:
 - Fix: Indexing now works; indexing with Lucene.Net and searching on the same index via Jakarta Lucene
 - Fix: System.IO.TextReader.Read() returns 0 not -1 so check for '<= 0'


16Oct04:
 - Port: QueryTermVector.cs -- done. thanks to Monsur


14Oct04:
 - Port: SpanNearQuery.cs -- done. thanks to Radu
 - Port: SpanOrQuery.cs -- done. thanks to Radu
 - Port: FSDirectory.cs -- done. thanks to Seth & Bill
 - Port: IndexReader.cs -- done. thanks to Bill
 - Port: IndexWriter.cs -- done. thanks to Bill


13Oct04 Lucene.Net.1.4-alpha build 002
 - Port: SpanNearQuery.cs & SpanOrQuery.cs -- thanks to Radu
 - Fix: FSDirectory.cs, IndexReader.cs & IndexWriter.cs -- thanks to Seth and Bill
 - Fix: FSDirectory.RenameFile()


13Oct04 Lucene.Net.1.4-alpha build 002
 - Note: First effort to port Jakarta Lucene 1.4-final to C#
 - Note: Ported emements are: "src" and "demo"
 - Note: There are still outstanding un-ported code (12 in all) which are being looked at
 - Note: The code compiles and runs as is, but you can't index or search
 - Fix: PriorityQueue.Out() -> PriorityQueue.Put()
 - Port: MultiReader.cs & Document.cs -- thanks to Radu


15Sep04 Lucene.Net.1.3-rc3 build 001
 - Revival of Lucene by repackaging 1.3rc1-001 and 1.3rc3-001


29Sep04 Lucene.Net 1.3-rc3 build 001
 - initial release of jakarta lucene 1.3-rc3
  for changes: http://cvs.apache.org/viewcvs.cgi/*checkout*/jakarta-lucene/CHANGES.txt?rev=1.58
 - fix: DateToString bug
   for more info: https://sourceforge.net/tracker/?func=detail&atid=582307&aid=910832&group_id=87200


18Apr04 Lucene.Net 1.3-rc2 build 002
 - add: demos, web demos and tests
 - fix: add FileAccess.Read key with FileMode.Open when open
        file stream for read
 - fix: name style updated
    Document.cs:GetField()
   QueryParser.cs: GenerateParseException()
   Search/BooleanQuery.cs:GetMaxClauseCount(),SetMaxClauseCount()
  - new: Lucene.Net National Language Support Pack (Lucene.Net.NLS Pack)
   support: Brazilian, CJK, Chinese, Czech,  French and Dutch Analysers


20Dec03 Lucene.Net 1.3-rc2 build 001
 - initial release for 1.3-rc2
   only Lucene.Net without demo and tests


09Jan03 Lucene.Net 1.3-rc1 build 002
 - update: documentation and comments
 - bug fixed: wildcard search crash lucene - fixed
                 look TestQueryFSDirectory project for details
 - bug fixed: all tests and demos work


08May03 Lucene.Net 1.3-rc1 build 001
 - initial release



[新闻]盖茨官方否认天价租楼看奥运 纯属地产商炒作]]>javascript 操作cookie类http://www.cnblogs.com/birdshover/archive/2008/07/15/1243007.htmlBirdshoverBirdshoverMon, 14 Jul 2008 16:12:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/07/15/1243007.htmlhttp://www.cnblogs.com/birdshover/comments/1243007.htmlhttp://www.cnblogs.com/birdshover/archive/2008/07/15/1243007.html#Feedback0http://www.cnblogs.com/birdshover/comments/commentRss/1243007.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1243007.html 1 String.prototype.Trim = function()
 2 {
 3     return this.replace(/^\s+/g,"").replace(/\s+$/g,"");
 4 }
 5 
 6 function JSCookie()
 7 {
 8     this.GetCookie = function(key)
 9     {
10         var cookie = document.cookie;
11         var cookieArray = cookie.split(';');
12         var getvalue = "";
13         for(var i = 0;i<cookieArray.length;i++)
14         {
15             
16             if(cookieArray[i].Trim().substr(0,key.length) == key)
17             {
18                 getvalue = cookieArray[i].Trim().substr(key.length + 1);
19                 break;
20             }
21         }
22 
23         return getvalue;
24     };
25     this.GetChild = function(cookiekey,childkey)
26     {
27         var child = this.GetCookie(cookiekey);
28         var childs = child.split('&');
29         var getvalue = "";
30         
31         for(var i = 0;i < childs.length;i++)
32         {
33             if(childs[i].Trim().substr(0,childkey.length) == childkey)
34             {
35                 getvalue = childs[i].Trim().substr(childkey.length + 1);
36                 break;
37             }
38         }
39         return getvalue;
40     };
41     this.SetCookie = function(key,value,expire,domain,path)
42     {
43         var cookie = "";
44         if(key != null && value != null)
45             cookie += key + "=" + value + ";";
46         if(expire != null)
47             cookie += "expires=" + expire.toGMTString() + ";";
48         if(domain != null)
49             cookie += "domain=" + domain + ";";
50         if(path != null)
51             cookie += "path=" + path + ";";
52         document.cookie = cookie;
53     };
54     this.Expire = function(key)
55     {
56         expire_time = new Date();
57         expire_time.setFullYear(expire_time.getFullYear() - 1);
58         var cookie = " " + key + "=e;expires=" + expire_time + ";"
59         document.cookie = cookie;
60     }
61 }


用法：
一、设置cookie
var cookie = new JSCookie();
//普通设置
cookie .SetCookie("key1","val1");

//过期时间为一年
var expire_time = new Date();
expire_time.setFullYear(expire_time.getFullYear() + 1);
cookie .SetCookie("key2","val2",expire_time);

//设置域及路径，带过期时间
cookie .SetCookie("key3","val3",expire_time,".cnblogs.com","/");

//设置带子键的cookie,子键分别是k1,k2,k3
cookie .SetCookie("key4","k1=1&k2=2&k3=3");

二、读取cookie
//简单获取
cookie .GetCookie("key1");
cookie .GetCookie("key2");
cookie .GetCookie("key3");
cookie .GetCookie("key4");
//获取key4的子键k1值
cookie .GetChild("key4","k1");

三、删除
cookie .Expire("key1");
cookie .Expire("key2");
cookie .Expire("key3");
cookie .Expire("key4");

[新闻]2008年7月24日IT博客精选]]>C# 62进制类型实现http://www.cnblogs.com/birdshover/archive/2008/07/04/1235543.htmlBirdshoverBirdshoverFri, 04 Jul 2008 05:16:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/07/04/1235543.htmlhttp://www.cnblogs.com/birdshover/comments/1235543.htmlhttp://www.cnblogs.com/birdshover/archive/2008/07/04/1235543.html#Feedback4http://www.cnblogs.com/birdshover/comments/commentRss/1235543.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1235543.html阅读全文

[新闻]TOM在线与Joost正式组建合资公司]]>话说passport的整合http://www.cnblogs.com/birdshover/archive/2008/05/16/1200282.htmlBirdshoverBirdshoverFri, 16 May 2008 03:51:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/05/16/1200282.htmlhttp://www.cnblogs.com/birdshover/comments/1200282.htmlhttp://www.cnblogs.com/birdshover/archive/2008/05/16/1200282.html#Feedback4http://www.cnblogs.com/birdshover/comments/commentRss/1200282.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1200282.html二、我已经有了3个系统，现在做了个passport，怎么整进去才好？  阅读全文

[新闻]财富:谷歌副总裁称其新闻搜索值1亿美元]]>地震了http://www.cnblogs.com/birdshover/archive/2008/05/12/1193823.htmlBirdshoverBirdshoverMon, 12 May 2008 08:08:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/05/12/1193823.htmlhttp://www.cnblogs.com/birdshover/comments/1193823.htmlhttp://www.cnblogs.com/birdshover/archive/2008/05/12/1193823.html#Feedback3http://www.cnblogs.com/birdshover/comments/commentRss/1193823.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1193823.html
同事老黄问我，你感觉楼在晃吗？才有猛然想到地震了。

[新闻]Ubuntu创始人呼吁开发Linux桌面软件对抗苹果]]>SharpCached小规模.Net缓存源码（VS2008项目）http://www.cnblogs.com/birdshover/archive/2008/03/29/1128383.htmlBirdshoverBirdshoverFri, 28 Mar 2008 17:41:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/03/29/1128383.htmlhttp://www.cnblogs.com/birdshover/comments/1128383.htmlhttp://www.cnblogs.com/birdshover/archive/2008/03/29/1128383.html#Feedback23http://www.cnblogs.com/birdshover/comments/commentRss/1128383.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1128383.html
SharpCached是用Remoting实现的IIS外缓存服务。
相当于是MemCached的C#实现，不过是个迷你版的。因为有很多功能没有实现。  阅读全文

[新闻]央视国际联手微软打造数字奥运媒体平台]]>话说搜索引擎分词http://www.cnblogs.com/birdshover/archive/2008/03/27/1125614.htmlBirdshoverBirdshoverThu, 27 Mar 2008 08:52:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/03/27/1125614.htmlhttp://www.cnblogs.com/birdshover/comments/1125614.htmlhttp://www.cnblogs.com/birdshover/archive/2008/03/27/1125614.html#Feedback19http://www.cnblogs.com/birdshover/comments/commentRss/1125614.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1125614.html
比如有句话:

"我是中国人"

看看这句话有多少种划分

我 我是 我是中 我是中国 我是中国人
  是 是中 是中国 是中国人
    中 中国 中国人
      国 国人
        人

按照灰色区域划分，无疑是正确的。但是从全部分词划分上来说，分词的分法达到了 2^n种。

在我见过的分发中，很多种都是大同小异。而很多种都提到了一个概念——最大词匹配。

比如句子：“通信信息报：瑞星以技术和服务开拓网络安全市场”

按最大划分 ，"通信信息报"是一个名词，“瑞星”是一个名词，“网络安全市场”是一个名词。那么就是索引为：

"通信信息报 :  瑞星  以 技术 和 服务 开拓 网络安全市场"

这样理解上确实是好的，但是实际效果却未必如此。按照这种分词索引数据，将造成，搜索“通信”是搜索不到结果的。因为划分后并没有“通信”这个词，而只有“通信信息报”。

也许你会觉得这样就排除了 “通信”这个词对“通信信息报”这个词的干扰。但是在很多场合中，用户不可能那么精确得输入。比如，用户很可能搜索“通信报”。

所以我认为搜索引擎中的分词技术，并不是要实现精确索引。中文分词最大的用处是排除特殊的干扰。

比如还是这句话"通信信息报 :  瑞星  以 技术 和 服务 开拓 网络安全市场"，如果从词义匹配上划分，还可能被分成"通信信息报 :  瑞星  以 技术 和服 务 开拓 网络安全市场"。注意，这句话和“和服”完全无关，但是这种分法就把它给相关上了。

无疑这种歧义在整合海量数据的索引中将会产生非常多。消除特定词汇的歧义肯定是可以处理的，但是你无法知道整个过程中会产生多少歧义。比如百度，在这个词的消除歧义过程中，也会有失误的地方：

宁波华和服装有限公司
备案序号:浙ICP备06009618号...
www.nbhuahe.com/ 2K 2007-12-28 - 百度快照
www.nbhuahe.com 上的更多结果

http://www.baidu.com/s?lm=0&si=&rn=10&ie=gb2312&ct=0&wd=%BA%CD%B7%FE&pn=20&ver=0&cn=baidu03&cl=3

现在我看到的最能识别歧义的方法，可能就是基于词频的统计。而不到一定的规模，没有足够的资金，肯定不会做这种事情去的。所以小的站内全文搜索引擎应该把更多的目光投到站内数据挖掘上面。不但要结果准，而且要结果多。

我认为基于最大词匹配的最小切分可以在很大程度上提升搜索引擎的性能和搜索精确度。

比如：“此次大会在香港特别行政区召开”

那么最大匹配就是“此次 大会 在 香港特别行政区 召开”

如果发现了最大匹配“香港特别行政区”，你不可能让别人搜索“香港”，而找不到这个结果。所以，有效的办法就是对最大匹配进行最小划分。划分为"香港 特别 行政 区"。

在搜索引擎中很多地方需要这么做。

当然，也可以用同意词的方法来解决.。但是定义同意词，无疑是一件长期而繁琐的事情。而且，对于“香港特别行政区”这样一个词，一般预期查询的词就是“香港”，划分就可以，没必要做同义词解析。

而对于“日本人常用的口头禅”，如果有人搜索“日语口头禅”，你认为是不是要匹配呢？最好当然是做下“日语”对“日本人”的同义词。而如果有人搜索“日语口头”，那么最小划分将会起到意向不到的作用。要知道，用户凭自己想出来的关键字，经常是不可预期的。

全文完。 by yurow

birdshover.cnblogs.com

___________________________________________________________________
把文章转成随笔了，汗～～！

[新闻]Google在线百科工具Knol今日正式上线]]>SharpICTCLASAnalyzer——基于吕震宇老师翻译的SharpICTCLAS 1.0开发的Lucene.Net 2.1分词接口源码http://www.cnblogs.com/birdshover/archive/2008/03/26/1122305.htmlBirdshoverBirdshoverTue, 25 Mar 2008 17:49:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/03/26/1122305.htmlhttp://www.cnblogs.com/birdshover/comments/1122305.htmlhttp://www.cnblogs.com/birdshover/archive/2008/03/26/1122305.html#Feedback6http://www.cnblogs.com/birdshover/comments/commentRss/1122305.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1122305.htmlhttp://www.cnblogs.com/birdshover/

关于SharpICTCLAS 1.0的信息请参考博客园吕震宇老师的博客：
http://www.cnblogs.com/zhenyulu/archive/2007/04/18/718383.html

以前不是做这个的，一直没在意，今天想想好长时间没上吕震宇老师的博客了，上去看看，看到了这个，呵呵。所以包装了一下。

为了增加Lucene.Net的分词接口，因此修改了部分源码。

1、增加了WordResult类sLocation变量，为了记住当前分词在文本中的位置；
代码：
 1    public class WordResult
 2    {
 3       //The word 
 4       public string sWord;
 5 
 6       //the POS of the word
 7       public int nPOS;
 8 
 9       //The -log(frequency/MAX)
10       public double dValue;
11 
12       public int sLocation;
13    }


2、修改了Segment类GenerateWord方法；
增加了下面的红色代码
 1   while (pCur != null)
 2          {
 3             WordResult item = new WordResult();
 4             item.sWord = pCur.theWord.sWord;
 5             item.nPOS = pCur.theWord.nPOS;
 6             item.dValue = pCur.theWord.dValue;
 7             item.sLocation = pCur.row;
 8             result[i] = item;
 9 
10             m_graphOptimum.SetElement(pCur.row, pCur.col, new ChainContent(item.dValue, item.nPOS, pCur.sWordInSegGraph));
11 
12             pCur = pCur.next;
13             i++;
14          }

3、修正了一个因为空格造成的bug;
红色部分是由
 1          //Position for the delimeters
 2          int nType;
 3 
 4 
 5          if (sWord.Length != 0)
 6          {
 7              nType = Utility.charType(sWord.ToCharArray()[0]);
 8 
 9              //Chinese word
10              if (nType == Predefine.CT_CHINESE)
11              {
12                  //Get the inner code of the first Chinese Char
13                  byte[] byteArray = Utility.String2ByteArray(sWord);
14                  nId = Utility.CC_ID(byteArray[0], byteArray[1]);
15 
16                  //store the word,not store the first Chinese Char
17                  sWordRet = sWord.Substring(1);
18                  return true;
19              }
20 
21              //Delimiter
22              if (nType == Predefine.CT_DELIMITER)
23              {
24                  nId = 3755;
25                  //Get the inner code of the first Chinese Char
26                  sWordRet = sWord; //store the word, not store the first Chinese Char
27                  return true;
28              }
29          }
30          else
31              nType = 0;


源码下载：
http://www.cnblogs.com/Files/birdshover/SharpICTCLASAnalyzer.rar

下载完注意修改里面的字典路径在测试。

测试文本：
 1             string testwords = @"1、修改了原子分词代码，使得对于全角字母有较好的识别
 2 
 3 2、修改了部分词性标注部分的代码
 4 
 5 因为词性标注部分的代码存在问题（应当是从ICTCLAS就存在的问题），主要表现在如果某个汉字没有词性，则在词性标注时会出现异常。例如：“这些是永远也没有现成的答桉的”其中“答案”写错了，当对这个有错别字的句子分词时，“桉”字是没有词性的，程序在此时将出现错误。
 6 
 7 目前的解决办法是对于这些没有词性的词在最终标注时标注为“字符串”。
 8 
 9 2、修改了地名识别的一些问题
10 
11 这个问题出现在Span类的PlaceRecognize方法中，nStart与nEnd在某些时候会计算错误。在测试版SharpICTCLAS中，句子“明定陵是明十三陵中第十座陵墓”在分词时会因为这个问题导致异常。 
12 
13 3、修改了基于CCID的字符串比较代码
14 
15 原有代码没有很好考虑对全角、半角混合字符串的比较问题，现在修正过来了。
16 
17 4、修改了向词库添加词汇的代码
18 
19 原有代码存在错误，现在改正了过来。
20 
21 ";



测试结果：

1 0 1
、 1 2
修改 2 4
了 4 5
原子 5 7
分词 7 9
代码 9 11
， 11 12
使得 12 14
对于 14 16
全角字 16 19
母 19 20
有 20 21
较 21 22
好 22 23
的 23 24
识别 24 26


2 26 31
、 27 28
修改 28 30
了 30 31
部分 31 33
词性 33 35
标注 35 37
部分 37 39
的 39 40
代码 40 42


 42 46
因为 43 45
词性 45 47
标注 47 49
部分 49 51
的 51 52
代码 52 54
存在 54 56
问题 56 58
（ 58 59
应当 59 61
是 61 62
从 62 63
ICTCLAS 63 70
就 64 65
存在 65 67
的 67 68
问题 68 70
） 70 71
， 71 72
主要 72 74
表现 74 76
在 76 77
如果 77 79
某个 79 81
汉字 81 83
没有 83 85
词 85 86
性 86 87
， 87 88
则 88 89
在 89 90
词性 90 92
标注 92 94
时 94 95
会 95 96
出现 96 98
异常 98 100
。 100 101
例如 101 103
： 103 104
“ 104 105
这些 105 107
是 107 108
永远 108 110
也 110 111
没有 111 113
现成 113 115
的 115 116
答 116 117
桉 117 118
的 118 119
” 119 120
其中 120 122
“ 122 123
答案 123 125
” 125 126
写 126 127
错 127 128
了 128 129
， 129 130
当 130 131
对 131 132
这个 132 134
有 134 135
错别字 135 138
的 138 139
句子 139 141
分词 141 143
时 143 144
， 144 145
“ 145 146
桉 146 147
” 147 148
字 148 149
是 149 150
没有 150 152
词性 152 154
的 154 155
， 155 156
程序 156 158
在 158 159
此时 159 161
将 161 162
出现 162 164
错误 164 166
。 166 167


 167 171
目前 168 170
的 170 171
解决 171 173
办法 173 175
是 175 176
对于 176 178
这些 178 180
没有 180 182
词性 182 184
的 184 185
词 185 186
在 186 187
最终 187 189
标注 189 191
时 191 192
标注 192 194
为 194 195
“ 195 196
字符串 196 199
” 199 200
。 200 201


2 201 206
、 202 203
修改 203 205
了 205 206
地名 206 208
识别 208 210
的 210 211
一些 211 213
问题 213 215


 215 219
这个 216 218
问题 218 220
出现 220 222
在 222 223
Span 223 227
类 224 225
的 225 226
PlaceRecognize 226 240
方法 227 229
中 229 230
， 230 231
nStart 231 237
与 232 233
nEnd 233 237
在 234 235
某些 235 237
时候 237 239
会 239 240
计算 240 242
错误 242 244
。 244 245
在 245 246
测试 246 248
版 248 249
SharpICTCLAS 249 261
中 250 251
， 251 252
句子 252 254
“ 254 255
明定陵 255 258
是 258 259
明 259 260
十三陵 260 263
中 263 264
第十 264 266
座 266 267
陵墓 267 269
” 269 270
在 270 271
分词 271 273
时 273 274
会 274 275
因为 275 277
这个 277 279
问题 279 281
导致 281 283
异常 283 285
。 285 286


3 286 291
、 287 288
修改 288 290
了 290 291
基于 291 293
CCID 293 297
的 294 295
字符串 295 298
比较 298 300
代码 300 302


 302 306
原有 303 305
代码 305 307
没有 307 309
很 309 310
好 310 311
考虑 311 313
对 313 314
全角 314 316
、 316 317
半 317 318
角 318 319
混合 319 321
字符串 321 324
的 324 325
比较 325 327
问题 327 329
， 329 330
现在 330 332
修正 332 334
过来 334 336
了 336 337
。 337 338


4 338 343
、 339 340
修改 340 342
了 342 343
向 343 344
词库 344 346
添加 346 348
词汇 348 350
的 350 351
代码 351 353


 353 357
原有 354 356
代码 356 358
存在 358 360
错误 360 362
， 362 363
现在 363 365
改正 365 367
了 367 368
过来 368 370
。 370 371


 371 375



[新闻]马云致信阿里巴巴全体员工号召准备过冬]]>基于.Net Framework 3.5的Lucene.Net 中文词组匹配分词器http://www.cnblogs.com/birdshover/archive/2008/03/25/1120517.htmlBirdshoverBirdshoverMon, 24 Mar 2008 17:32:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/03/25/1120517.htmlhttp://www.cnblogs.com/birdshover/comments/1120517.htmlhttp://www.cnblogs.com/birdshover/archive/2008/03/25/1120517.html#Feedback4http://www.cnblogs.com/birdshover/comments/commentRss/1120517.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1120517.html由于没有实现完整的一套字典机制，而是普通的文本字典，所以就不提供完整源码下载了，贴出核心的源码。  阅读全文

[新闻]comScore 公布 6 月美国 TOP10 社会化网络排名]]>.Net 3.5环境下常用数组性能测试http://www.cnblogs.com/birdshover/archive/2008/03/19/1113949.htmlBirdshoverBirdshoverWed, 19 Mar 2008 15:31:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/03/19/1113949.htmlhttp://www.cnblogs.com/birdshover/comments/1113949.htmlhttp://www.cnblogs.com/birdshover/archive/2008/03/19/1113949.html#Feedback21http://www.cnblogs.com/birdshover/comments/commentRss/1113949.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1113949.html阅读全文

[新闻]万众期待 iPhone 3G 16GB上市两天破万]]>控制‘控制台应用程序’的关闭操作http://www.cnblogs.com/birdshover/archive/2008/03/17/1110138.htmlBirdshoverBirdshoverMon, 17 Mar 2008 08:48:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/03/17/1110138.htmlhttp://www.cnblogs.com/birdshover/comments/1110138.htmlhttp://www.cnblogs.com/birdshover/archive/2008/03/17/1110138.html#Feedback5http://www.cnblogs.com/birdshover/comments/commentRss/1110138.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1110138.html
有没有办法和WinForm一样，对控制台的退出事件进行控制呢？有的！  阅读全文

[新闻]张宝全正式声明中国版HD-DVD涉嫌盗用EVD技术]]>用IHttpModule解决输入中文地址乱码问题（二）http://www.cnblogs.com/birdshover/archive/2008/03/14/1106333.htmlBirdshoverBirdshoverFri, 14 Mar 2008 10:52:00 GMThttp://www.cnblogs.com/birdshover/archive/2008/03/14/1106333.htmlhttp://www.cnblogs.com/birdshover/comments/1106333.htmlhttp://www.cnblogs.com/birdshover/archive/2008/03/14/1106333.html#Feedback2http://www.cnblogs.com/birdshover/comments/commentRss/1106333.htmlhttp://www.cnblogs.com/birdshover/services/trackbacks/1106333.html用IHttpModule解决输入中文地址乱码问题（一） 的续文。

上文说到，需要对已有的地址进行GB2312编码，这样大大增加了工作量，有没更好的办法呢？

 1     public class HookModule : IHttpModule
 2     {
 3 
 4         #region IHttpModule 成员
 5 
 6         public void Dispose()
 7         {
 8             
 9         }
10 
11         public void Init(HttpApplication context)
12         {
13             context.BeginRequest += new EventHandler(context_BeginRequest);
14         }
15 
16         void context_BeginRequest(object sender, EventArgs e)
17         {
18             
19             HttpApplication application = (HttpApplication)sender;
20             HttpContext context = application.Context;
21             IdentifyEncoding ie = new IdentifyEncoding();
22 
23             
24             string rawurl = context.Request.RawUr