预编译阶段扫描 R class

预编译阶段扫描 R class

10 July 2017


最近阅读 butterknife 8.6.0 源码,发现一段很神奇的代码。其实这种黑科技代码不常见,毕竟用了一些 JVM 内的 tools jar,欠缺 JVM 知识的人,很容易摸不着头脑。对于这块知识点,资料也少之又少。


这段神奇的代码仅仅是从 butterknife 8.2.0 开始引入。 committer 写了一段名为 "Generate nicer code for precompiled R class references" 的 commit,来表达他的喜悦心情。


作用:预编译阶段 扫描 R class解析 R class 逻辑,是一段利用 sun.tools.javacJVM 虚拟机。 运用了 org.sun.tools jar,也用到了其中的 JCTreeTreeScanner


由于之前对 JCTree 没有什么了解,查阅资料发现其是 JVM 在处理 javac 的过程中的 语法分析关键先生。在语法分析中的 有 两个流程

Token 流分析:检查代码的关键词组合是否符合 Java 语法规范。比如 if, while 后面是否是 boolean 表达式之类的。

构建语法树:单词组成的句子,区分主谓宾定语,语法树上的每个节点都是一个 JCTree。然后通过给 JCTree 添加 Visitor( TreeScanner 实现类 )。 往 JCTree 内添加 数据。




butterknife 魔法代码


实际上,在解析那些 butterknife 的注解之前,会进行一趟 R class 的扫描工作:

1.预编译阶段扫描 butterknife 相关注解所在 packageName 所属的的 资源 idR class 内的 ),这里就是黑魔法代码运用了 JCTreeTreeScanner

2.缓存下来的 QualifiedIdIdMap 缓存QualifiedId 是一个拥有 packageName 的 id,Id 是一个拥有 Javapoet CodeBlock 的 id。都是资源 id 的形象化封装产物


在解析那些 butterknife 的注解时

3.butterknife 会通过 Elementid 转换为 QualifiedId,因为 Element 可以通过 Elements 拿到 packageName

4.在解析注解中,就能拿到一个 QualifiedId,通过上述的扫描结果缓存,可以根据 QualifiedId 取到一个 Id。从而拿到了 Javapoet CodeBlock,用于自动生成代码时,进行代码拼凑。




引入 org.sun.tool


dependencies {  
    compileOnly files(org.gradle.internal.jvm.Jvm.current().getToolsJar())
}




修整 魔法代码


结构类

/**
 * @author CaMnter
 */

final class Id {

    private static final ClassName ANDROID_R = ClassName.get("android", "R");

    final int value;
    final CodeBlock code;
    final boolean qualifed;


    Id(int value) {
        this.value = value;
        this.code = CodeBlock.of("$L", value);
        this.qualifed = false;
    }


    Id(int value, ClassName className, String resourceName) {
        this.value = value;
        this.code = className.topLevelClassName().equals(ANDROID_R)
                    ? CodeBlock.of("$L.$N", className, resourceName)
                    : CodeBlock.of("$T.$N", className, resourceName);
        this.qualifed = true;
    }


    @Override
    public boolean equals(Object o) {
        return o instanceof Id && value == ((Id) o).value;
    }


    @Override
    public int hashCode() {
        return value;
    }


    @Override
    public String toString() {
        throw new UnsupportedOperationException("Please use value or code explicitly");
    }

}

封装的 Id,可以识别是不是 android 资源 id,根据 ClassName.get("android", "R") 来判断传入进来的 id,是 android.R 还是 int最主要的,提供了该 id 对应的 Javapoet CodeBlock


/**
 * @author CaMnter
 */

final class QualifiedId {

    final String packageName;
    final String enclosingElementName;
    final String elementSimpleName;
    final int id;


    QualifiedId(String packageName, String enclosingElementName, String elementSimpleName, int id) {
        this.packageName = packageName;
        this.enclosingElementName = enclosingElementName;
        this.elementSimpleName = elementSimpleName;
        this.id = id;
    }


    @Override
    public String toString() {
        return "QualifiedId{packageName='" + packageName + "', id=" + id + '}';
    }


    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (!(o instanceof QualifiedId)) return false;
        QualifiedId other = (QualifiedId) o;
        return id == other.id
            && packageName.equals(other.packageName);
    }


    @Override
    public int hashCode() {
        int result = packageName.hashCode();
        result = 31 * result + id;
        return result;
    }

}

原本是只有一个 封装一个具有 packageNameId。后来,我添加了 enclosingElementNameelementSimpleName,分别表示了 该资源所在的内部类名资源名


扫描类

public class RClassScanner extends TreeScanner {

    // Maps the currently evaulated rPackageName to R Classes
    private final Map<String, Set<String>> rClasses = new LinkedHashMap<>();
    private String currentPackageName;


    @Override
    public void visitSelect(JCTree.JCFieldAccess jcFieldAccess) {
        Symbol symbol = jcFieldAccess.sym;
        if (symbol != null
            && symbol.getEnclosingElement() != null
            && symbol.getEnclosingElement().getEnclosingElement() != null
            && symbol.getEnclosingElement().getEnclosingElement().enclClass() != null) {
            Set<String> rClassSet = this.rClasses.get(this.currentPackageName);
            if (rClassSet == null) {
                rClassSet = new HashSet<>();
                this.rClasses.put(this.currentPackageName, rClassSet);
            }
            rClassSet.add(
                symbol.getEnclosingElement().getEnclosingElement().enclClass().className());
        }
    }


    Map<String, Set<String>> getRClasses() {
        return this.rClasses;
    }


    void setCurrentPackageName(String respectivePackageName) {
        this.currentPackageName = respectivePackageName;
    }

}

每个 package name 都会有一个 R class,依赖包下的 R 也包含在内( v4, v7 等 ),对这些 R class 进行扫描。全部 package nameR 扫描,即等于相关注解所在 packageName 所属的 R 的资源 id


the_compile_phase_scans_the_R_class_1


import static com.camnter.smartsave.compiler.scanner.ScannerManager.SUPPORTED_TYPES;

/**
 * @author CaMnter
 */

public class IdScanner extends TreeScanner {

    private final Map<QualifiedId, Id> ids;
    private final String rPackageName;
    private final String respectivePackageName;


    IdScanner(Map<QualifiedId, Id> ids, String rPackageName, String respectivePackageName) {
        this.ids = ids;
        this.rPackageName = rPackageName;
        this.respectivePackageName = respectivePackageName;
    }


    @Override
    public void visitClassDef(JCTree.JCClassDecl jcClassDecl) {
        for (JCTree tree : jcClassDecl.defs) {
            if (tree instanceof ClassTree) {
                ClassTree classTree = (ClassTree) tree;
                String className = classTree.getSimpleName().toString();
                if (SUPPORTED_TYPES.contains(className)) {
                    ClassName rClassName = ClassName.get(rPackageName, "R", className);
                    VarScanner scanner = new VarScanner(ids, rClassName, respectivePackageName);
                    ((JCTree) classTree).accept(scanner);
                }
            }
        }
    }

}

R class id 扫描。同时,会继续调用 VarScanner 接着进行扫描。


/**
 * @author CaMnter
 */

class VarScanner extends TreeScanner {

    private final Map<QualifiedId, Id> ids;
    private final ClassName className;
    private final String respectivePackageName;


    VarScanner(Map<QualifiedId, Id> ids, ClassName className,
               String respectivePackageName) {
        this.ids = ids;
        this.className = className;
        this.respectivePackageName = respectivePackageName;
    }


    @Override
    public void visitVarDef(JCTree.JCVariableDecl jcVariableDecl) {
        if ("int".equals(jcVariableDecl.getType().toString())) {
            int id = Integer.valueOf(jcVariableDecl.getInitializer().toString());
            String resourceName = jcVariableDecl.getName().toString();
            QualifiedId qualifiedId = new QualifiedId(
                this.respectivePackageName,
                this.className.toString(),
                jcVariableDecl.getName().toString(),
                id
            );
            this.ids.put(qualifiedId, new Id(id, this.className, resourceName));
        }
    }

}

R class 变量 扫描。会保存 QualifiedId。此时就具备了 packageNameid。由于我扩展了 QualifiedId所在内部类名变量名,所以在这也得进行扩展: this.className.toString()jcVariableDecl.getName().toString()




ScannerManager

/**
 * 编译阶段扫描指定注解所在 packageName 所属的的 资源 id( R class 内的 )
 *
 * @author CaMnter
 */

public final class ScannerManager {

    static final List<String> SUPPORTED_TYPES = Arrays.asList(
        "array", "attr", "bool", "color", "dimen", "drawable", "id", "integer", "string"
    );

    private final ProcessingEnvironment processingEnvironment;
    private final Elements elementUtils;
    private final Types typeUtils;

    private final Map<QualifiedId, Id> symbols = new LinkedHashMap<>();
    private final Messager messager;
    private Trees trees;


    private ScannerManager(ProcessingEnvironment processingEnvironment) {
        this.processingEnvironment = processingEnvironment;
        this.elementUtils = this.processingEnvironment.getElementUtils();
        this.typeUtils = this.processingEnvironment.getTypeUtils();
        this.messager = processingEnvironment.getMessager();
        try {
            this.trees = Trees.instance(this.processingEnvironment);
        } catch (IllegalArgumentException ignored) {
        }
    }


    public static ScannerManager get(ProcessingEnvironment processingEnvironment) {
        return new ScannerManager(processingEnvironment);
    }


    /**
     * 获取 注解 在元素上对应的 AnnotationMirror
     * 目前仅为了生成 JCTree
     *
     * @param element 注解元素
     * @param annotation 注解 class 类型
     * @return AnnotationMirror
     */
    private AnnotationMirror getMirror(Element element,
                                       Class<? extends Annotation> annotation) {
        for (AnnotationMirror annotationMirror : element.getAnnotationMirrors()) {
            if (annotationMirror.getAnnotationType()
                .toString()
                .equals(annotation.getCanonicalName())) {
                return annotationMirror;
            }
        }
        return null;
    }


    /**
     * 扫描 R class
     * 解析 R class
     *
     * @param env RoundEnvironment
     * @param supportedAnnotations 注解集合
     */
    public void scanForRClasses(RoundEnvironment env,
                                Set<Class<? extends Annotation>> supportedAnnotations) {
        if (trees == null) return;

        RClassScanner scanner = new RClassScanner();

        /*
         * 每个注解类型 与 被注解的元素,生成一棵树
         * 然后设置 R class 扫描这个元素的 package name
         * 最后让这棵树 会自动调用扫描类中的方法去,扫描 R class
         */
        for (Class<? extends Annotation> annotation : supportedAnnotations) {
            for (Element element : env.getElementsAnnotatedWith(annotation)) {
                JCTree tree = (JCTree) this.trees.getTree(element, getMirror(element, annotation));
                // tree can be null if the references are compiled types and not source
                if (tree != null) {
                    String respectivePackageName =
                        this.elementUtils.getPackageOf(element).getQualifiedName().toString();
                    scanner.setCurrentPackageName(respectivePackageName);
                    tree.accept(scanner);
                }
            }
        }

        /*
         * 拿到全部 R classes
         * 然后 parseRClass(...) 解析 R class
         */
        for (Map.Entry<String, Set<String>> packageNameToRClassSet : scanner.getRClasses()
            .entrySet()) {
            String respectivePackageName = packageNameToRClassSet.getKey();
            for (String rClass : packageNameToRClassSet.getValue()) {
                parseRClass(respectivePackageName, rClass);
            }
        }

        // print id
        Deque<Map.Entry<QualifiedId, Id>> entries = new ArrayDeque<>(this.symbols.entrySet());
        while (!entries.isEmpty()) {
            Map.Entry<QualifiedId, Id> entry = entries.removeFirst();
            QualifiedId qualifiedId = entry.getKey();
            this.messager.printMessage(
                Diagnostic.Kind.NOTE,
                "[packageName] = " + qualifiedId.packageName +
                    "   [fullName] = " + qualifiedId.enclosingElementName + "." +
                    qualifiedId.elementSimpleName +
                    "   [simpleName] = " + qualifiedId.elementSimpleName +
                    "\n  [id] = " + qualifiedId.id
            );
        }

    }


    /**
     * 1.获取 该 R class 在 scanForRClasses(...) 时,生辰生成的那棵树
     * - 如果存在树,就解析编译好的 R class
     * - 不存在的话,创建一个 Id 扫描类,扫描 R class 内的所有 Id
     * 2.Id 扫描类,内还会让树调用 Var 扫描类,扫描全部 int 变量
     *
     * @param respectivePackageName R class package name
     * @param rClass R class
     */
    private void parseRClass(String respectivePackageName, String rClass) {
        Element element;

        try {
            element = elementUtils.getTypeElement(rClass);
        } catch (MirroredTypeException mte) {
            element = typeUtils.asElement(mte.getTypeMirror());
        }

        JCTree tree = (JCTree) trees.getTree(element);
        if (tree != null) { // tree can be null if the references are compiled types and not source
            IdScanner idScanner = new IdScanner(this.symbols,
                this.elementUtils.getPackageOf(element)
                    .getQualifiedName().toString(), respectivePackageName);
            tree.accept(idScanner);
        } else {
            parseCompiledR(respectivePackageName, (TypeElement) element);
        }
    }


    /**
     * 解析编译过的 R class
     *
     * @param respectivePackageName package name
     * @param rClass R class
     */
    private void parseCompiledR(String respectivePackageName, TypeElement rClass) {
        for (Element element : rClass.getEnclosedElements()) {
            String innerClassName = element.getSimpleName().toString();
            if (SUPPORTED_TYPES.contains(innerClassName)) {
                for (Element enclosedElement : element.getEnclosedElements()) {
                    if (enclosedElement instanceof VariableElement) {
                        VariableElement variableElement = (VariableElement) enclosedElement;
                        Object value = variableElement.getConstantValue();

                        if (value instanceof Integer) {
                            int id = (Integer) value;
                            ClassName rClassName =
                                ClassName.get(
                                    this.elementUtils.getPackageOf(variableElement).toString(),
                                    "R",
                                    innerClassName);
                            String resourceName = variableElement.getSimpleName().toString();
                            QualifiedId qualifiedId = new QualifiedId(
                                respectivePackageName,
                                element.getSimpleName().toString(),
                                enclosedElement.getSimpleName().toString(),
                                id
                            );
                            this.symbols.put(qualifiedId, new Id(id, rClassName, resourceName));
                        }
                    }
                }
            }
        }
    }

}




使用方法

private Set<Class<? extends Annotation>> getSupportedAnnotations() {  
    return new LinkedHashSet<Class<? extends Annotation>>() {
        {
            this.add(Save.class);
            this.add(SaveColor.class);
            this.add(SaveOnClick.class);
            this.add(SaveDimension.class);
        }
    };
}

/**
 * {@inheritDoc}
 */
@Override
public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {

    ...

    try {

        ...

        // R class 扫描
        ScannerManager scannerManager = ScannerManager.get(this.processingEnvironment);
        scannerManager.scanForRClasses(roundEnv, this.getSupportedAnnotations());

        ...

    } catch (Exception e) {
        this.e(e.getMessage());
        e.printStackTrace();
        return true;
    }

    ...

    return true;

}


the_compile_phase_scans_the_R_class_2




代码地址


scanner