JDT-core学习

291 阅读9分钟

本文已参与「新人创作礼」活动,一起开启掘金创作之路。

通过AST访问Java代码

  1. 创建ASTParser解析Java代码为抽象语法树(ASTNode)结构
  2. 继承ASTVisitor类,重写访问AST各个节点
// 创建抽象语法书解析器,按照Java语言规范Java SE 11 Edition (JLS11)解析,
// 支持操作所有jdk11版本的Java代码操作
ASTParser astParser = ASTParser.newParser(AST.JLS11);
// Java编译参数,默认astKind类型为编译单元K_COMPILATION_UNIT,解析器将源码会解析为一个编译单元类型CompilationUnit
// 包含所有的类型声明、import、package等等节点
// K_CLASS_BODY_DECLARATIONS:仅将源码的class body(含子节点)部分解析。不包含类型、import、package等信息
// K_STATEMENTS:源码将被解析为一个语句序列statements数组,package、import节点会被解析成assert语句,其余节点正常解析
// K_EXPRESSION:源码将被解析为一个表达式,如果将一个类的源码传入,返回的将会是一对解析失败的报错,类的import等节点均不是表达式
Map<String, String> compilerOptions = JavaCore.getOptions();
// 设置编译依照的合规参数为jdk 1.8
JavaCore.setComplianceOptions(JavaCore.VERSION_1_8, compilerOptions);
astParser.setCompilerOptions(compilerOptions);
// 设置支持编译器在创建ASTNode节点时需要提供绑定信息(节点名称、类型信息),默认为false,即使为true也有可能不提供绑定信息
// if (this.resolveBindings) {
// 	recordNodes(typeDecl, typeDeclaration);
// 	recordNodes(typeName, typeDeclaration);
// 	typeDecl.resolveBinding();
// }
// 还需要:unitName不为空、compilerOptions不为空、(project或classpaths或sourcepaths不为空或开启了INCLUDE_RUNNING_VM_BOOTCLASSPATH)
// 最终是否解析绑定信息判断逻辑见下方的needToResolveBindings属性的赋值逻辑
// binding信息主要记录了节点的全限定名称、类型已经类型的绑定信息等。案例见下方截图
astParser.setResolveBindings(true);
// 如果不设置setResolveBindings是无效的,默认null,unitName:源文件名(绝对路径)
astParser.setUnitName("myUnit");
// 如果不设置 setResolveBindings是无效的,四个属性配置其中给一个即可,如果不配置全部默认为null
// 第1个参数 classpathEntries:当没有Java project时(直接解析某个文件),解析绑定信息使用的classpath,例如:String[] classpathEntries = new String[]{"D:\\Applications\\apache-maven-3.5.0\\conf\\repository\\org\\springframework\\spring-beans\\4.3.20.RELEASE\\spring-beans-4.3.20.RELEASE.jar"};
// 第2个参数 sourcepathEntries:当没有Java project时(直接解析某个文件),解析绑定信息使用的sourcepath,例如:String[] sourcepathEntries = new String[]{"D:\\Applications\\apache-maven-3.5.0\\conf\\repository\\org\\springframework\\spring-beans\\4.3.20.RELEASE\\spring-beans-4.3.20.RELEASE-sources.jar"};
// 第3个参数 sourcepathsEncodings:sourcepaths条目解析使用编码
// 第4个参数 includeRunningVMBootclasspath:运行时VM的bootclasspath追加至给定的classpath
astParser.setEnvironment(null, null, null, true);
// 前提必须要开启绑定信息,是否允许返回修复过的绑定,例如:RecoveredTypeBinding,结合isRecovered判断绑定信息是否已修复过的绑定类型
// 如果没有开启,当使用node节点获取绑定信息时,如果类型是未解析过的MissingType,则会直接返回null,
// 如果开启则会从解析器bindingTables中获取创建绑定信息返回,即可以修复绑定信息
// DefaultBindingResolver.getVariableBinding
// if (!this.isRecoveringBindings && ((variableType.tagBits & TagBits.HasMissingType) != 0)) {
// 	return null;
// }
// IVariableBinding binding = (IVariableBinding) this.bindingTables.compilerBindingsToASTBindings.get(variableBinding);
// if (binding != null) {
// 	return binding;
// }
// binding = new VariableBinding(this, variableBinding);
// this.bindingTables.compilerBindingsToASTBindings.put(variableBinding, binding);
// return binding;
astParser.setBindingsRecovery(true);
// 如果开启则允许编译器创建包含语法错误的语句,当解析时发现语法错误会保留statementsRecoveryData,
// 用于语句恢复,获取恢复数据的方法不希望被客户端使用,仅供框架内部使用
astParser.setStatementsRecovery(true);
// 绑定源码
astParser.setSource(src.toCharArray());
// 按照配置将源码解析为抽象语法树(AST)
CompilationUnit astRoot = (CompilationUnit) astParser.createAST(null);
// 创建AST访问对象
ASTVisitor astVisitor = new MyASTVisitor();
// 使用AST访问器访问语法树
astRoot.accept(astVisitor);

解析绑定信息ResolveBindings

needToResolveBindings = 
((this.bits & CompilationUnitResolver.RESOLVE_BINDING) != 0)
&& this.unitName != null
&& (this.project != null
		|| this.classpaths != null
		|| this.sourcepaths != null
		|| ((this.bits & CompilationUnitResolver.INCLUDE_RUNNING_VM_BOOTCLASSPATH) != 0))
&& this.compilerOptions != null;

在这里插入图片描述

创建抽象语法树

  1. 将源码解析为一个编译单元ASTParser.createAST
  2. 案例是按照rawSource源码解析
  3. 创建编译单元BasicCompilationUnit
  4. 如果需要解析绑定needToResolveBindings则调用CompilationUnitResolver.resolve解析,返回编译单元声明CompilationUnitDeclaration类型
  5. 否则调用CompilationUnitResolver.parse解析,根据配置将源码BasicCompilationUnit解析CommentRecorderParser.dietParse("节食解析"不解析方法体),返回编译单元声明CompilationUnitDeclaration类型
  6. 案例按照需要解析绑定分析源码
  7. 创建CompilationUnitResolver解析器,执行解析resolve,入参unit为空,no existing compilation unit declaration
  8. 默认不忽略方法body的解析ignoreMethodBodies,默认NodeSearcher为null
  9. internalBeginToCompile遍历compilationUnitDeclaration类型声明types,解析每个类型的方法body
  10. 编译源码beginToCompile,对每一个源文件创建一个编译单元声明实例CompilationUnitDeclaration
  11. 如果totalUnit数量大于parseThreshold阈值则调用父类Parser.dietParse"节食解析"方法,否则调用父类Parser.parse方法解析,创建CompilationUnitDeclaration实例
  12. 将源码与Scanner绑定。Parser.parse解析。按照token分隔符一个个解析消费token:Parser.consumeRule(根据token类型创建对应的节点,例如:consumeEnterVariable、consumeBlock、consumeClassHeaderName1、consumeSingleTypeImportDeclarationName),如果是diet模式则跳过方法体解析,返回compilationUnitDeclaration实例
  13. 记录解析语法树花费时间:this.stats.parseTime += resolveStart - parseStart;
  14. 创建类型绑定信息,使用LookupEnvironment.buildTypeBindings为CompilationUnitDeclaration创建设置scope属性,类型CompilationUnitScope,后面查找属性的类型会使用到该scope的fPackage中的knownTypes
  15. 记录绑定信息解析花费时间:this.stats.resolveTime += System.currentTimeMillis() - resolveStart;
  16. 完成信息绑定,完成类型解析this.lookupEnvironment.completeTypeBindings();,根据类型的import声明、超类superinterface声明、字段声明、方法声明信息创建绑定信息,注意此时field、method属性创建的是基础的绑定信息(例如声明类等信息),具体属性的类型信息还没有填充还是null,例如:对一个jar包中的类的引用,引用的类型还没有解析出来填充进去,后面unit.scope.faultInTypes();解析完成后再填充正确的类型信息。底层调用字段声明中的类型引用对象解析出实际类型fieldDecl.type.resolveType,例如:SingleTypeReference.resolveType,父类回调子类SingleTypeReference.getTypeBinding
  17. 遍历编译单元中类型声明节点TypeDeclaration解析方法体parseMethods
  18. 将CompilationUnitDeclaration转为编译单元CompilationUnit返回CompilationUnitResolver.convert
  19. 根据jdk版本创建AST.newAST(apiLevel)
  20. 默认绑定信息解析器实现DefaultBindingResolver,默认不启用解析绑定信息:BindingResolver
// class:CompilationUnitResolver.resolve
if (unit.scope != null) {
	CompilationUnitDeclaration previousUnit = this.lookupEnvironment.unitBeingCompleted;
	this.lookupEnvironment.unitBeingCompleted = unit;
	try {
		// fault in fields & methods
		unit.scope.faultInTypes();
		if (unit.scope != null && verifyMethods) {
			// http://dev.eclipse.org/bugs/show_bug.cgi?id=23117
			// verify inherited methods
			unit.scope.verifyMethods(this.lookupEnvironment.methodVerifier());
		}
		// type checking
		unit.resolve();

		// flow analysis
		if (analyzeCode) unit.analyseCode();

		// code generation
		if (generateCode) unit.generateCode();

		// finalize problems (suppressWarnings)
		unit.finalizeProblems();
	} finally {
		this.lookupEnvironment.unitBeingCompleted = previousUnit; // paranoia, always null in org.eclipse.jdt.core.tests.dom.RunAllTests
	}
}

// SourceTypeBinding.resolveTypeFor,例如:SingleTypeReference.resolveType,父类回调子类SingleTypeReference.getTypeBinding
// 调用scope获取类型scope.getType,scope为initializerScope,类型MethodScope
TypeBinding fieldType =
	fieldDecl.getKind() == AbstractVariableDeclaration.ENUM_CONSTANT
		? initializationScope.environment().convertToRawType(this, false /*do not force conversion of enclosing types*/) // enum constant is implicitly of declaring enum type
		: fieldDecl.type.resolveType(initializationScope, true /* check bounds*/);
field.type = fieldType;

转换CompilationUnitDeclaration为CompilationUnit,ASTConverter.convert(compilationUnitDeclaration, source)

  1. 解析注释
  2. 记录节点resolveBindings
  3. 如果currentPackage不为空转换为包节点
  4. 如果imports不为空将ImportReference转换为import节点
  5. 如果moduleDeclaration不为空则转换为ModuleDeclaration
  6. 否则,则如果types不为空则转换TypeDeclaration

转换TypeDeclaration

  1. 创建TypeDeclaration节点
  2. 创建body节点
  3. 遍历memberTypes、fields、methods
  4. 如果是FieldDeclaration类型,判断是否EnumConstantDeclaration,是则创建
  5. 如果是FieldDeclaration类型,判断是否Initializer,是则创建
  6. 如果是FieldDeclaration类型,判断是否VariableDeclarationFragment,是则创建
  7. 如果是FieldDeclaration类型,否则创建FieldDeclaration类型节点
  8. 如果是AbstractMethodDeclaration类型,判断是否AnnotationMethodDeclaration,是则创建
  9. 如果是AbstractMethodDeclaration类型,判断是否构造器是则创建构造器节点,是则创建
  10. 如果是AbstractMethodDeclaration类型,否则创建MethodDeclaration节点
  11. 如果是TypeDeclaration成员类型,递归调用类型转换convert(org.eclipse.jdt.internal.compiler.ast.TypeDeclaration typeDeclaration)

返回编译单元

  1. 设置源码范围setSourceRange
  2. 如果存在编译错误转换
  3. 如果启用解析绑定信息lookupForScopes
  4. 初始化注释映射表initCommentMapper
  5. 返回编译单元compilationUnit

访问语法树

访问前置动作ASTVisitor.preVisit2

访问ASTNode.accept0

  1. 类型声明节点访问实现accept0
  2. 调用ASTVisitor.visit方法访问当前节点
  3. 如果返回true则继续访问子节点,TypeDeclaration节点访问顺序与源码中声明顺序一致在这里插入图片描述
  4. 访问节点后的后置动作endVisit
// CompilationUnit
@Override
void accept0(ASTVisitor visitor) {
	boolean visitChildren = visitor.visit(this);
	if (visitChildren) {
		// visit children in normal left to right reading order
		if (this.ast.apiLevel >= AST.JLS9_INTERNAL) {
			acceptChild(visitor, getModule());
		}
		acceptChild(visitor, getPackage());
		acceptChildren(visitor, this.imports);
		acceptChildren(visitor, this.types);
	}
	visitor.endVisit(this);
}
// TypeDeclaration
@Override
void accept0(ASTVisitor visitor) {
	boolean visitChildren = visitor.visit(this);
	if (visitChildren) {
		// visit children in normal left to right reading order
		...
		if (this.ast.apiLevel >= AST.JLS3_INTERNAL) {
			acceptChild(visitor, getJavadoc());
			acceptChildren(visitor, this.modifiers);
			acceptChild(visitor, getName());
			acceptChildren(visitor, this.typeParameters);
			acceptChild(visitor, getSuperclassType());
			acceptChildren(visitor, this.superInterfaceTypes);
            // 循环调用body体的各个节点,递归遍历访问所有节点
			acceptChildren(visitor, this.bodyDeclarations);
		}
	}
	visitor.endVisit(this);
}

访问后置动作ASTVisitor.postVisit

访问流程图

在这里插入图片描述

重写语法树

private static VisitorDispatchDTO switchClean(String src)
        throws BadLocationException {
    VisitorDispatchDTO visitorDispatchDTO = new VisitorDispatchDTO();
    String newSrc = null;
    Document document= new Document(src);

    // creation of DOM/AST from a ICompilationUnit
    ASTParser astParser = ASTParser.newParser(AST.JLS11);
    Map<String, String> compilerOptions = JavaCore.getOptions();
    JavaCore.setComplianceOptions(JavaCore.VERSION_1_8, compilerOptions);
    astParser.setCompilerOptions(compilerOptions);
    astParser.setStatementsRecovery(true);
    astParser.setSource(src.toCharArray());
    CompilationUnit astRoot = (CompilationUnit) astParser.createAST(null);

    // 创建语法树重写器
    ASTRewrite astRewrite = ASTRewrite.create(astRoot.getAST());

    SwitchesCleanerVisitor switchesCleanerVisitor = new SwitchesCleanerVisitor(astRewrite, astRoot);
    // 3.访问并修改语法树
    astRoot.accept(switchesCleanerVisitor);

    // computation of the text edits
    TextEdit edits = astRewrite.rewriteAST(document, compilerOptions);

    // 根据新语法树更新代码
    edits.apply(document);

    // 获取更新后的源码
    String newSrc=document.get()
    return visitorDispatchDTO;
}

在这里插入图片描述

  1. ASTRewriteAnalyzer访问AST所有节点根据eventStore创建对应的InsertEdit、ReplaceEdit节点
  2. MultiTextEdit遍历子节点执行节点更新
  3. Document根据更新节点的offset与length以及新代码字符串fText更新Document文档
  4. 最后通过Document.get()方法获取更新后的代码

总结

  1. 成员变量的类型通过Scope.getTypeOrPackage方法查找,首先通过当前类声明的import、class、block中查找。如果找不到再从当前包下查找PackageBinding.getTypeOrPackage,注意此处的包下类型是遍历CompilationUnitDeclaration中types类型所得,并不会自动加载源码中当前包下未被指定的其他源码类型。如果找不到尝试从包中查找PackageBinding.getPackage0查找对应的包,因为可能是包类型而不是Type类型。如果依然找不到,则使用LookupEnvironment.askForType查找类型。如果依然找不到尝试使用findPackage查找包。找不到最后返回problemBinding存在问题的绑定信息
PackageBinding currentPackage = unitScope.fPackage;
unitScope.recordReference(currentPackage.compoundName, name);
Binding binding = currentPackage.getTypeOrPackage(name, module());

// ComiplationUnitScope
TypeDeclaration[] types = this.referenceContext.types;
int typeLength = (types == null) ? 0 : types.length;
this.topLevelTypes = new SourceTypeBinding[typeLength];
int count = 0;
nextType: for (int i = 0; i < typeLength; i++) {
	TypeDeclaration typeDecl = types[i];
	if (this.environment.root.isProcessingAnnotations && this.environment.isMissingType(typeDecl.name))
		throw new SourceTypeCollisionException(); // resolved a type ref before APT generated the type
	...
	ClassScope child = new ClassScope(this, typeDecl);
	SourceTypeBinding type = child.buildType(null, this.fPackage, accessRestriction);
	if (firstIsSynthetic && i == 0)
		type.modifiers |= ClassFileConstants.AccSynthetic;
	if (type != null)
		this.topLevelTypes[count++] = type;
		
// ClassScope.buildType
SourceTypeBinding buildType(SourceTypeBinding enclosingType, PackageBinding packageBinding, AccessRestriction accessRestriction) {
	// provide the typeDeclaration with needed scopes
	this.referenceContext.scope = this;
	this.referenceContext.staticInitializerScope = new MethodScope(this, this.referenceContext, true);
	this.referenceContext.initializerScope = new MethodScope(this, this.referenceContext, false);

	if (enclosingType == null) {
		char[][] className = CharOperation.arrayConcat(packageBinding.compoundName, this.referenceContext.name);
		this.referenceContext.binding = new SourceTypeBinding(className, packageBinding, this);
	} else {
		char[][] className = CharOperation.deepCopy(enclosingType.compoundName);
		className[className.length - 1] =
			CharOperation.concat(className[className.length - 1], this.referenceContext.name, '$');
		ReferenceBinding existingType = packageBinding.getType0(className[className.length - 1]);
		if (existingType != null) {
			if (existingType instanceof UnresolvedReferenceBinding) {
				// its possible that a BinaryType referenced the member type before its enclosing source type was built
				// so just replace the unresolved type with a new member type
			} else {
				// report the error against the parent - its still safe to answer the member type
				this.parent.problemReporter().duplicateNestedType(this.referenceContext);
			}
		}
		this.referenceContext.binding = new MemberTypeBinding(className, this, enclosingType);
	}

	SourceTypeBinding sourceType = this.referenceContext.binding;
	sourceType.module = module();
	environment().setAccessRestriction(sourceType, accessRestriction);
	
	TypeParameter[] typeParameters = this.referenceContext.typeParameters;
	sourceType.typeVariables = typeParameters == null || typeParameters.length == 0 ? Binding.NO_TYPE_VARIABLES : null;
	sourceType.fPackage.addType(sourceType);
	checkAndSetModifiers();
	buildTypeVariables();
	
	buildMemberTypes(accessRestriction);
	return sourceType;
}