前几天，我把词法分析器打包了，命名为BKLexer。目前，BKLexer分别支持Go/C++/Python。程序代码放在Github项目中：点击项目页面可学习try_lexer各版本代码。我们以Go为例：packagemainimport("fmt""strconv""./bklexer")funcmain(){fmt.Println("TestCode:")code:="declarevariable=PI*100-fda\n1024*4*3.14###\n123"fmt.Println(代码)fmt.Println("-------------------------------")lexer:=BKLexer.NewLexer()lexer.AddRule("\\d+\\.\\d*","FLOAT")lexer.AddRule("\\d+","INT")词法分析器.AddRule("[\\p{L}\\d_]+","NAME")词法分析器.AddRule("\\+","PLUS")词法分析器。AddRule("\\-","MINUS")词法分析器。AddRule("\\*","MUL")词法分析器。AddRule("/","DIV")词法分析器。AddRule("=","ASSIGN")词法分析器。AddRule("#[^\\r\\n]*","COMMENT")词法分析器。AddIgnores("[\\f\\t]+")词法分析器。构建（代码）为真{token:=lexer.NextToken()if(token.TType!=BKLexer.TOKEN_TYPE_EOF){fmt.Printf("%s\t%s\tt%d\t%d\t%d,%d\n",token.Name,strconv.Quote(token.Source),token.TType,token.Position,token.Row,token.Col)}if(token.TType==BKLexer.TOKEN_TYPE_EOF||token.TType==BKLexer.TOKEN_TYPE_ERROR){break}}}先导入包含bklexer的包import("fmt""strconv""./bklexer")fmt用于打印输出strconv用于优化文字的显示。/bklexer引入BKLexer包实例化词法分析器并设置规则lexer:=BKLexer.NewLexer()lexer.AddRule("\\d+\\.\\d*","FLOAT")lexer.AddRule("\\d+","INT")lexer.AddRule("[\\p{L}\\d_]+","NAME")lexer.AddRule("\\+","PLUS")lexer.AddRule("\\-","MINUS")lexer.AddRule("\\*","MUL")lexer.AddRule("/","DIV")词法分析器。AddRule("=","ASSIGN")lexer.AddRule("#[^\\r\\n]*","COMMENT")lexer.AddIgnores("[\\f\\t]+")NewLexer实例词法分析AddRule添加匹配规则，参数为正则表达式，对应的类型名AddIgnores用于设置需要忽略的字符内容。构建和循环匹配lexer.Build(code)fortrue{token:=lexer.NextToken()if(token.TType!=BKLexer.TOKEN_TYPE_EOF){fmt.Printf("%s\t%s\tt%d\t%d\t%d,%d\n",token.Name,strconv.Quote(token.Source),token.TType,token.Position,token.Row,token.Col)}如果(token.TType==BKLexer.TOKEN_TYPE_EOF||token.TType==BKLexer.TOKEN_TYPE_ERROR){break}}使用Build方法构建代码code作为参数，然后调用NextToken方法获取下一个Token，并打印相关信息。需要注意的是，Token应该是类型检测，判断是EOF还是ERROR来判断是否终止。运行结果如下测试代码：声明变量=PI*100-fda1024*4*3.14###123-------------------------------NAME"声明"t300,0NAME"变量"t370,3ASSIGN"="t8140,6NAME"PI"t3160,8MUL"*"t6190,11INT"100"t2210,13MINUS"-"t5250,17NAME"fda"t3270,19NEWLINE"\n"t0300,22INT"1024"t2311,0MUL"*"t6361,5INT"4"t2381,7MUL"*"t6401,9FLOAT"3.14"t1421,11COMMENT"###"t9471,16NEWLINE"\n"t0511,20INT"123"t2522,0Part《递归向下算法实现Calc》，欢迎关注。
