本文已参与「新人创作礼」活动,一起开启掘金创作之路。
通过AST访问Java代码
- 创建ASTParser解析Java代码为抽象语法树(ASTNode)结构
- 继承ASTVisitor类,重写访问AST各个节点
// 创建抽象语法书解析器,按照Java语言规范Java SE 11 Edition (JLS11)解析,
// 支持操作所有jdk11版本的Java代码操作
ASTParser astParser = ASTParser.newParser(AST.JLS11);
// Java编译参数,默认astKind类型为编译单元K_COMPILATION_UNIT,解析器将源码会解析为一个编译单元类型CompilationUnit
// 包含所有的类型声明、import、package等等节点
// K_CLASS_BODY_DECLARATIONS:仅将源码的class body(含子节点)部分解析。不包含类型、import、package等信息
// K_STATEMENTS:源码将被解析为一个语句序列statements数组,package、import节点会被解析成assert语句,其余节点正常解析
// K_EXPRESSION:源码将被解析为一个表达式,如果将一个类的源码传入,返回的将会是一对解析失败的报错,类的import等节点均不是表达式
Map<String, String> compilerOptions = JavaCore.getOptions();
// 设置编译依照的合规参数为jdk 1.8
JavaCore.setComplianceOptions(JavaCore.VERSION_1_8, compilerOptions);
astParser.setCompilerOptions(compilerOptions);
// 设置支持编译器在创建ASTNode节点时需要提供绑定信息(节点名称、类型信息),默认为false,即使为true也有可能不提供绑定信息
// if (this.resolveBindings) {
// recordNodes(typeDecl, typeDeclaration);
// recordNodes(typeName, typeDeclaration);
// typeDecl.resolveBinding();
// }
// 还需要:unitName不为空、compilerOptions不为空、(project或classpaths或sourcepaths不为空或开启了INCLUDE_RUNNING_VM_BOOTCLASSPATH)
// 最终是否解析绑定信息判断逻辑见下方的needToResolveBindings属性的赋值逻辑
// binding信息主要记录了节点的全限定名称、类型已经类型的绑定信息等。案例见下方截图
astParser.setResolveBindings(true);
// 如果不设置setResolveBindings是无效的,默认null,unitName:源文件名(绝对路径)
astParser.setUnitName("myUnit");
// 如果不设置 setResolveBindings是无效的,四个属性配置其中给一个即可,如果不配置全部默认为null
// 第1个参数 classpathEntries:当没有Java project时(直接解析某个文件),解析绑定信息使用的classpath,例如:String[] classpathEntries = new String[]{"D:\\Applications\\apache-maven-3.5.0\\conf\\repository\\org\\springframework\\spring-beans\\4.3.20.RELEASE\\spring-beans-4.3.20.RELEASE.jar"};
// 第2个参数 sourcepathEntries:当没有Java project时(直接解析某个文件),解析绑定信息使用的sourcepath,例如:String[] sourcepathEntries = new String[]{"D:\\Applications\\apache-maven-3.5.0\\conf\\repository\\org\\springframework\\spring-beans\\4.3.20.RELEASE\\spring-beans-4.3.20.RELEASE-sources.jar"};
// 第3个参数 sourcepathsEncodings:sourcepaths条目解析使用编码
// 第4个参数 includeRunningVMBootclasspath:运行时VM的bootclasspath追加至给定的classpath
astParser.setEnvironment(null, null, null, true);
// 前提必须要开启绑定信息,是否允许返回修复过的绑定,例如:RecoveredTypeBinding,结合isRecovered判断绑定信息是否已修复过的绑定类型
// 如果没有开启,当使用node节点获取绑定信息时,如果类型是未解析过的MissingType,则会直接返回null,
// 如果开启则会从解析器bindingTables中获取创建绑定信息返回,即可以修复绑定信息
// DefaultBindingResolver.getVariableBinding
// if (!this.isRecoveringBindings && ((variableType.tagBits & TagBits.HasMissingType) != 0)) {
// return null;
// }
// IVariableBinding binding = (IVariableBinding) this.bindingTables.compilerBindingsToASTBindings.get(variableBinding);
// if (binding != null) {
// return binding;
// }
// binding = new VariableBinding(this, variableBinding);
// this.bindingTables.compilerBindingsToASTBindings.put(variableBinding, binding);
// return binding;
astParser.setBindingsRecovery(true);
// 如果开启则允许编译器创建包含语法错误的语句,当解析时发现语法错误会保留statementsRecoveryData,
// 用于语句恢复,获取恢复数据的方法不希望被客户端使用,仅供框架内部使用
astParser.setStatementsRecovery(true);
// 绑定源码
astParser.setSource(src.toCharArray());
// 按照配置将源码解析为抽象语法树(AST)
CompilationUnit astRoot = (CompilationUnit) astParser.createAST(null);
// 创建AST访问对象
ASTVisitor astVisitor = new MyASTVisitor();
// 使用AST访问器访问语法树
astRoot.accept(astVisitor);
解析绑定信息ResolveBindings
needToResolveBindings =
((this.bits & CompilationUnitResolver.RESOLVE_BINDING) != 0)
&& this.unitName != null
&& (this.project != null
|| this.classpaths != null
|| this.sourcepaths != null
|| ((this.bits & CompilationUnitResolver.INCLUDE_RUNNING_VM_BOOTCLASSPATH) != 0))
&& this.compilerOptions != null;
创建抽象语法树
- 将源码解析为一个编译单元ASTParser.createAST
- 案例是按照rawSource源码解析
- 创建编译单元BasicCompilationUnit
- 如果需要解析绑定needToResolveBindings则调用CompilationUnitResolver.resolve解析,返回编译单元声明CompilationUnitDeclaration类型
- 否则调用CompilationUnitResolver.parse解析,根据配置将源码BasicCompilationUnit解析CommentRecorderParser.dietParse("节食解析"不解析方法体),返回编译单元声明CompilationUnitDeclaration类型
- 案例按照需要解析绑定分析源码
- 创建CompilationUnitResolver解析器,执行解析resolve,入参unit为空,no existing compilation unit declaration
- 默认不忽略方法body的解析ignoreMethodBodies,默认NodeSearcher为null
- internalBeginToCompile遍历compilationUnitDeclaration类型声明types,解析每个类型的方法body
- 编译源码beginToCompile,对每一个源文件创建一个编译单元声明实例CompilationUnitDeclaration
- 如果totalUnit数量大于parseThreshold阈值则调用父类Parser.dietParse"节食解析"方法,否则调用父类Parser.parse方法解析,创建CompilationUnitDeclaration实例
- 将源码与Scanner绑定。Parser.parse解析。按照token分隔符一个个解析消费token:Parser.consumeRule(根据token类型创建对应的节点,例如:consumeEnterVariable、consumeBlock、consumeClassHeaderName1、consumeSingleTypeImportDeclarationName),如果是diet模式则跳过方法体解析,返回compilationUnitDeclaration实例
- 记录解析语法树花费时间:this.stats.parseTime += resolveStart - parseStart;
- 创建类型绑定信息,使用LookupEnvironment.buildTypeBindings为CompilationUnitDeclaration创建设置scope属性,类型CompilationUnitScope,后面查找属性的类型会使用到该scope的fPackage中的knownTypes
- 记录绑定信息解析花费时间:this.stats.resolveTime += System.currentTimeMillis() - resolveStart;
- 完成信息绑定,完成类型解析this.lookupEnvironment.completeTypeBindings();,根据类型的import声明、超类superinterface声明、字段声明、方法声明信息创建绑定信息,注意此时field、method属性创建的是基础的绑定信息(例如声明类等信息),具体属性的类型信息还没有填充还是null,例如:对一个jar包中的类的引用,引用的类型还没有解析出来填充进去,后面unit.scope.faultInTypes();解析完成后再填充正确的类型信息。底层调用字段声明中的类型引用对象解析出实际类型fieldDecl.type.resolveType,例如:SingleTypeReference.resolveType,父类回调子类SingleTypeReference.getTypeBinding
- 遍历编译单元中类型声明节点TypeDeclaration解析方法体parseMethods
- 将CompilationUnitDeclaration转为编译单元CompilationUnit返回CompilationUnitResolver.convert
- 根据jdk版本创建AST.newAST(apiLevel)
- 默认绑定信息解析器实现DefaultBindingResolver,默认不启用解析绑定信息:BindingResolver
// class:CompilationUnitResolver.resolve
if (unit.scope != null) {
CompilationUnitDeclaration previousUnit = this.lookupEnvironment.unitBeingCompleted;
this.lookupEnvironment.unitBeingCompleted = unit;
try {
// fault in fields & methods
unit.scope.faultInTypes();
if (unit.scope != null && verifyMethods) {
// http://dev.eclipse.org/bugs/show_bug.cgi?id=23117
// verify inherited methods
unit.scope.verifyMethods(this.lookupEnvironment.methodVerifier());
}
// type checking
unit.resolve();
// flow analysis
if (analyzeCode) unit.analyseCode();
// code generation
if (generateCode) unit.generateCode();
// finalize problems (suppressWarnings)
unit.finalizeProblems();
} finally {
this.lookupEnvironment.unitBeingCompleted = previousUnit; // paranoia, always null in org.eclipse.jdt.core.tests.dom.RunAllTests
}
}
// SourceTypeBinding.resolveTypeFor,例如:SingleTypeReference.resolveType,父类回调子类SingleTypeReference.getTypeBinding
// 调用scope获取类型scope.getType,scope为initializerScope,类型MethodScope
TypeBinding fieldType =
fieldDecl.getKind() == AbstractVariableDeclaration.ENUM_CONSTANT
? initializationScope.environment().convertToRawType(this, false /*do not force conversion of enclosing types*/) // enum constant is implicitly of declaring enum type
: fieldDecl.type.resolveType(initializationScope, true /* check bounds*/);
field.type = fieldType;
转换CompilationUnitDeclaration为CompilationUnit,ASTConverter.convert(compilationUnitDeclaration, source)
- 解析注释
- 记录节点resolveBindings
- 如果currentPackage不为空转换为包节点
- 如果imports不为空将ImportReference转换为import节点
- 如果moduleDeclaration不为空则转换为ModuleDeclaration
- 否则,则如果types不为空则转换TypeDeclaration
转换TypeDeclaration
- 创建TypeDeclaration节点
- 创建body节点
- 遍历memberTypes、fields、methods
- 如果是FieldDeclaration类型,判断是否EnumConstantDeclaration,是则创建
- 如果是FieldDeclaration类型,判断是否Initializer,是则创建
- 如果是FieldDeclaration类型,判断是否VariableDeclarationFragment,是则创建
- 如果是FieldDeclaration类型,否则创建FieldDeclaration类型节点
- 如果是AbstractMethodDeclaration类型,判断是否AnnotationMethodDeclaration,是则创建
- 如果是AbstractMethodDeclaration类型,判断是否构造器是则创建构造器节点,是则创建
- 如果是AbstractMethodDeclaration类型,否则创建MethodDeclaration节点
- 如果是TypeDeclaration成员类型,递归调用类型转换convert(org.eclipse.jdt.internal.compiler.ast.TypeDeclaration typeDeclaration)
返回编译单元
- 设置源码范围setSourceRange
- 如果存在编译错误转换
- 如果启用解析绑定信息lookupForScopes
- 初始化注释映射表initCommentMapper
- 返回编译单元compilationUnit
访问语法树
访问前置动作ASTVisitor.preVisit2
访问ASTNode.accept0
- 类型声明节点访问实现accept0
- 调用ASTVisitor.visit方法访问当前节点
- 如果返回true则继续访问子节点,TypeDeclaration节点访问顺序与源码中声明顺序一致
- 访问节点后的后置动作endVisit
// CompilationUnit
@Override
void accept0(ASTVisitor visitor) {
boolean visitChildren = visitor.visit(this);
if (visitChildren) {
// visit children in normal left to right reading order
if (this.ast.apiLevel >= AST.JLS9_INTERNAL) {
acceptChild(visitor, getModule());
}
acceptChild(visitor, getPackage());
acceptChildren(visitor, this.imports);
acceptChildren(visitor, this.types);
}
visitor.endVisit(this);
}
// TypeDeclaration
@Override
void accept0(ASTVisitor visitor) {
boolean visitChildren = visitor.visit(this);
if (visitChildren) {
// visit children in normal left to right reading order
...
if (this.ast.apiLevel >= AST.JLS3_INTERNAL) {
acceptChild(visitor, getJavadoc());
acceptChildren(visitor, this.modifiers);
acceptChild(visitor, getName());
acceptChildren(visitor, this.typeParameters);
acceptChild(visitor, getSuperclassType());
acceptChildren(visitor, this.superInterfaceTypes);
// 循环调用body体的各个节点,递归遍历访问所有节点
acceptChildren(visitor, this.bodyDeclarations);
}
}
visitor.endVisit(this);
}
访问后置动作ASTVisitor.postVisit
访问流程图
重写语法树
private static VisitorDispatchDTO switchClean(String src)
throws BadLocationException {
VisitorDispatchDTO visitorDispatchDTO = new VisitorDispatchDTO();
String newSrc = null;
Document document= new Document(src);
// creation of DOM/AST from a ICompilationUnit
ASTParser astParser = ASTParser.newParser(AST.JLS11);
Map<String, String> compilerOptions = JavaCore.getOptions();
JavaCore.setComplianceOptions(JavaCore.VERSION_1_8, compilerOptions);
astParser.setCompilerOptions(compilerOptions);
astParser.setStatementsRecovery(true);
astParser.setSource(src.toCharArray());
CompilationUnit astRoot = (CompilationUnit) astParser.createAST(null);
// 创建语法树重写器
ASTRewrite astRewrite = ASTRewrite.create(astRoot.getAST());
SwitchesCleanerVisitor switchesCleanerVisitor = new SwitchesCleanerVisitor(astRewrite, astRoot);
// 3.访问并修改语法树
astRoot.accept(switchesCleanerVisitor);
// computation of the text edits
TextEdit edits = astRewrite.rewriteAST(document, compilerOptions);
// 根据新语法树更新代码
edits.apply(document);
// 获取更新后的源码
String newSrc=document.get()
return visitorDispatchDTO;
}
- ASTRewriteAnalyzer访问AST所有节点根据eventStore创建对应的InsertEdit、ReplaceEdit节点
- MultiTextEdit遍历子节点执行节点更新
- Document根据更新节点的offset与length以及新代码字符串fText更新Document文档
- 最后通过Document.get()方法获取更新后的代码
总结
- 成员变量的类型通过Scope.getTypeOrPackage方法查找,首先通过当前类声明的import、class、block中查找。如果找不到再从当前包下查找PackageBinding.getTypeOrPackage,注意此处的包下类型是遍历CompilationUnitDeclaration中types类型所得,并不会自动加载源码中当前包下未被指定的其他源码类型。如果找不到尝试从包中查找PackageBinding.getPackage0查找对应的包,因为可能是包类型而不是Type类型。如果依然找不到,则使用LookupEnvironment.askForType查找类型。如果依然找不到尝试使用findPackage查找包。找不到最后返回problemBinding存在问题的绑定信息
PackageBinding currentPackage = unitScope.fPackage;
unitScope.recordReference(currentPackage.compoundName, name);
Binding binding = currentPackage.getTypeOrPackage(name, module());
// ComiplationUnitScope
TypeDeclaration[] types = this.referenceContext.types;
int typeLength = (types == null) ? 0 : types.length;
this.topLevelTypes = new SourceTypeBinding[typeLength];
int count = 0;
nextType: for (int i = 0; i < typeLength; i++) {
TypeDeclaration typeDecl = types[i];
if (this.environment.root.isProcessingAnnotations && this.environment.isMissingType(typeDecl.name))
throw new SourceTypeCollisionException(); // resolved a type ref before APT generated the type
...
ClassScope child = new ClassScope(this, typeDecl);
SourceTypeBinding type = child.buildType(null, this.fPackage, accessRestriction);
if (firstIsSynthetic && i == 0)
type.modifiers |= ClassFileConstants.AccSynthetic;
if (type != null)
this.topLevelTypes[count++] = type;
// ClassScope.buildType
SourceTypeBinding buildType(SourceTypeBinding enclosingType, PackageBinding packageBinding, AccessRestriction accessRestriction) {
// provide the typeDeclaration with needed scopes
this.referenceContext.scope = this;
this.referenceContext.staticInitializerScope = new MethodScope(this, this.referenceContext, true);
this.referenceContext.initializerScope = new MethodScope(this, this.referenceContext, false);
if (enclosingType == null) {
char[][] className = CharOperation.arrayConcat(packageBinding.compoundName, this.referenceContext.name);
this.referenceContext.binding = new SourceTypeBinding(className, packageBinding, this);
} else {
char[][] className = CharOperation.deepCopy(enclosingType.compoundName);
className[className.length - 1] =
CharOperation.concat(className[className.length - 1], this.referenceContext.name, '$');
ReferenceBinding existingType = packageBinding.getType0(className[className.length - 1]);
if (existingType != null) {
if (existingType instanceof UnresolvedReferenceBinding) {
// its possible that a BinaryType referenced the member type before its enclosing source type was built
// so just replace the unresolved type with a new member type
} else {
// report the error against the parent - its still safe to answer the member type
this.parent.problemReporter().duplicateNestedType(this.referenceContext);
}
}
this.referenceContext.binding = new MemberTypeBinding(className, this, enclosingType);
}
SourceTypeBinding sourceType = this.referenceContext.binding;
sourceType.module = module();
environment().setAccessRestriction(sourceType, accessRestriction);
TypeParameter[] typeParameters = this.referenceContext.typeParameters;
sourceType.typeVariables = typeParameters == null || typeParameters.length == 0 ? Binding.NO_TYPE_VARIABLES : null;
sourceType.fPackage.addType(sourceType);
checkAndSetModifiers();
buildTypeVariables();
buildMemberTypes(accessRestriction);
return sourceType;
}