AST应用-模块排序(依赖分析)

YSP13
141 阅读4分钟

模块排序(依赖分析)是基于模块的构建系统（webpack等）的重要部分。

======

1、介绍

1.1 模块排序

模块排序是构建工具的构建依赖图的过程，通过分析模块间的依赖关系来确定它们的处理和执行顺序。

1.2 主要目的

  1. 确保依赖关系的正确性
  2. 优化模块的加载和执行顺序
  3. 为代码分割(code splitting)和懒加载(lazy loading)提供基础

1.3 过程

  1. 依赖解析：从入口文件开始,递归地解析每个模块的依赖关系。
  2. 拓扑排序：通过拓扑排序解析出的依赖关系。确保了依赖模块总是在被依赖模块之前处理。
  3. 模块ID分配：排序后,每个模块分配一个唯一的ID。
  4. 执行顺序确定：最终构建工具根据排序和ID确定模块的执行顺序。

2、模块排序过程展示

2.1 文件结构

```text
src/
├── index.js
├── app.js
├── utils.js
└── components/
    ├── Header.js
    └── Footer.js
```

2.2 依赖关系

```
index.js → app.js
app.js   → components/Header.js
app.js   → components/Footer.js
app.js   → utils.js

utils.js             → null
components/Footer.js → null
components/Header.js → null


// JSON表示
{
  "index.js": ["app.js"],
  "app.js": [      "components/Header.js",       "components/Footer.js",       "utils.js"    ],
  "utils.js": [],
  "components/Footer.js": [],
  "components/Header.js": []
}
```

2.3 DAG表示

```text
        +----------+
        |  index.js|
        +----------+
             |
             v
        +----------+
        |  app.js  |
        +----------+
         /    |    \
        v     v     v
+----------+ +----------+ +----------+
| Header.js| | Footer.js| | utils.js |
+----------+ +----------+ +----------+
```

2.4 依赖图

根据以下的数组按照顺序处理就可以了，依赖问题已经被解决
```JSON
[
  {
    id: 0,
    name: './src/utils.js',
    dependencies: [],
  },
  {
    id: 1,
    name: './src/components/Header.js',
    dependencies: [],
  },
  {
    id: 2,
    name: './src/components/Footer.js',
    dependencies: [],
  },
  {
    id: 3,
    name: './src/app.js',
    dependencies: [
      { id: 1, name: './components/Header' },
      { id: 2, name: './components/Footer' },
      { id: 0, name: './utils' }
    ],
  },
  {
    id: 4,
    name: './src/styles.css',
    dependencies: [],
  },
  {
    id: 5,
    name: './src/index.js',
    dependencies: [
      { id: 3, name: './app' },
      { id: 4, name: './styles.css' }
    ],
  }
]
```

3、和依赖分析相关的AST节点

3.1 ImportSpecifier

  1. ImportSpecifier 节点类型
type ImportDeclaration {
    type: 'ImportDeclaration';
    specifiers: ImportSpecifier[];
    source: Literal;
}
interface ImportSpecifier {
    type: 'ImportSpecifier' | 'ImportDefaultSpecifier' | 'ImportNamespaceSpecifier';
    local: Identifier;
    imported?: Identifier;
}
  1. DEMO
# Code 
import Header from './components/Header';  
import Footer from './components/Footer';  
import { helper } from './utils';

# AST
Program
├── ImportDeclaration
│   ├── ImportDefaultSpecifier: Header
│   └── source: './components/Header'
├── ImportDeclaration
│   ├── ImportDefaultSpecifier: Footer
│   └── source: './components/Footer'
└── ImportDeclaration
    ├── ImportSpecifier
    │   └── imported: helper
    │   └── local: helper
    └── source: './utils'

# JSON数据
{
  "type": "Program",
  "body": [
    {
      "type": "ImportDeclaration",
      "specifiers": [...],
      "source": { "type": "Literal", "value": "./components/Header" }
    },
    {
      "type": "ImportDeclaration",
      "specifiers": [...],
      "source": { "type": "Literal", "value": "./components/Footer" }
    },
    {
      "type": "ImportDeclaration",
      "specifiers": [...],
      "source": { "type": "Literal", "value": "./utils" }
    }
  ]
}

3.2 require

  1. DEMO
const Header = require('./components/Header');  
const Footer = require('./components/Footer');  
const { helper } = require('./utils');

Program
├── VariableDeclaration (const)
│   └── VariableDeclarator
│       ├── id: Header
│       └── init: CallExpression (require)
│           └── Arguments: './components/Header'
├── VariableDeclaration (const)
│   └── VariableDeclarator
│       ├── id: Footer
│       └── init: CallExpression (require)
│           └── Arguments: './components/Footer'
└── VariableDeclaration (const)
    └── VariableDeclarator
        ├── id: ObjectPattern
        │   └── Property
        │       ├── key: helper
        │       └── value: helper
        └── init: CallExpression (require)
            └── Arguments: './utils'


{
  "type": "Program",
  "body": [
    {
      "type": "VariableDeclaration",
      "declarations": [
        {
          "type": "VariableDeclarator",
          ...
          "init": {
            "type": "CallExpression",
            "callee": {
              "type": "Identifier",
              "name": "require"
            },
            "arguments": [
              {
                "type": "Literal",
                "value": "./components/Header"
              }
            ]
          }
        }
      ],
      "kind": "const"
    },
    {
      "type": "VariableDeclaration",
      "declarations": [
        {
          "type": "VariableDeclarator",
          ...
          "init": {
            "type": "CallExpression",
            "callee": {
              "type": "Identifier",
              "name": "require"
            },
            "arguments": [
              {
                "type": "Literal",
                "value": "./components/Footer"
              }
            ]
          }
        }
      ],
      "kind": "const"
    },
    {
      "type": "VariableDeclaration",
      "declarations": [
        {
          "type": "VariableDeclarator",
          ...
          "init": {
            "type": "CallExpression",
            "callee": {
              "type": "Identifier",
              "name": "require"
            },
            "arguments": [
              {
                "type": "Literal",
                "value": "./utils"
              }
            ]
          }
        }
      ],
      "kind": "const"
    }
  ]
}

4、实现

4.1 依赖分析

核心思路：找到ImportDeclaration节点或者require节点，递归分析直到结束。

const esprima = require('esprima');
const fs = require('fs');
const path = require('path');

class DependencyAnalyzer {
    constructor(entryFile) {
        this.entryFile = entryFile;
        this.dependencyTree = {};
        this.circularDependencies = [];
        this.stack = [];
    }

    analyze() {
        this.analyzeDependencies(this.entryFile);
        this.detectCircularDependencies();
        return {
            tree: this.dependencyTree,
            circular: this.circularDependencies
        };
    }

    analyzeDependencies(filePath) {
        if (this.dependencyTree[filePath]) return;

        const content = fs.readFileSync(filePath, 'utf-8');
        const ast = esprima.parseModule(content, { jsx: true, range: true, comment: true });

        this.dependencyTree[filePath] = {
            dependencies: [],
            type: 'file',
            size: content.length
        };

        this.stack.push(filePath);

        ast.body.forEach(node => {
            // 处理ImportDeclaration节点
            if (node.type === 'ImportDeclaration') {
                this.handleImport(filePath, node.source.value);
            } 
            // 处理require节点
            else if (node.type === 'CallExpression' && node.callee.name === 'require') {
                this.handleRequire(filePath, node.arguments[0].value);
            }
        });

        this.stack.pop();
    }

    handleImport(currentFile, importPath) {
        const resolvedPath = this.resolveImportPath(currentFile, importPath);
        this.addDependency(currentFile, resolvedPath);
    }

    handleRequire(currentFile, requirePath) {
        const resolvedPath = this.resolveImportPath(currentFile, requirePath);
        this.addDependency(currentFile, resolvedPath);
    }

    addDependency(currentFile, dependencyPath) {
        this.dependencyTree[currentFile].dependencies.push(dependencyPath);
        if (!this.dependencyTree[dependencyPath]) {
            this.analyzeDependencies(dependencyPath);
        }
    }

    resolveImportPath(currentFile, importPath) {
        if (importPath.startsWith('.')) {
            return path.resolve(path.dirname(currentFile), importPath);
        }
        // 简化的node_modules解析
        return path.resolve(process.cwd(), 'node_modules', importPath);
    }

    detectCircularDependencies() {
        const visited = new Set();
        const recursionStack = new Set();

        const dfs = (node) => {
            visited.add(node);
            recursionStack.add(node);

            for (const dependency of this.dependencyTree[node].dependencies) {
                if (!visited.has(dependency)) {
                    if (dfs(dependency)) return true;
                } else if (recursionStack.has(dependency)) {
                    this.circularDependencies.push([node, dependency]);
                    return true;
                }
            }

            recursionStack.delete(node);
            return false;
        };

        Object.keys(this.dependencyTree).forEach(node => {
            if (!visited.has(node)) {
                dfs(node);
            }
        });
    }

    generateReport() {
        console.log('Dependency Analysis Report');
        console.log('==========================');
        this.printDependencyTree(this.entryFile);
        console.log('\nCircular Dependencies:');
        this.circularDependencies.forEach(([a, b]) => {
            console.log(`  ${a} <-> ${b}`);
        });
        console.log('\nFile Sizes:');
        Object.entries(this.dependencyTree).forEach(([file, info]) => {
            console.log(`  ${file}: ${info.size} bytes`);
        });
    }

    printDependencyTree(file, level = 0) {
        console.log('  '.repeat(level) + file);
        if (this.dependencyTree[file]) {
            this.dependencyTree[file].dependencies.forEach(dep => {
                this.printDependencyTree(dep, level + 1);
            });
        }
    }
}

const analyzer = new DependencyAnalyzer('./index.js');
analyzer.analyze();
analyzer.printDependencyTree();

解析结果

{
  "index.js": ["app.js"],
  "app.js": [
      "components/Header.js", 
      "components/Footer.js", 
      "utils.js"
    ],
  "utils.js": [],
  "components/Footer.js": [],
  "components/Header.js": []
}

4.2 拓扑排序

采用深度优先遍历

class ModuleOrderResolver {
  constructor(dependencyTree) {
      this.dependencyTree = dependencyTree;
      this.visited = new Set();
      this.order = [];
  }

  resolve() {
      const modules = Object.keys(this.dependencyTree);
      
      for (const module of modules) {
          if (!this.visited.has(module)) {
              this.dfs(module);
          }
      }

      return this.order.reverse();
  }

  dfs(module) {
      this.visited.add(module);

      const dependencies = this.dependencyTree[module].dependencies;
      for (const dep of dependencies) {
          if (!this.visited.has(dep)) {
              this.dfs(dep);
          }
      }

      this.order.push(module);
  }
}


const resolver = new ModuleOrderResolver(dependencyTree);
console.log(resolver.resolve());

输出

[
  "absoultepath/src/components/Header.js",
  "absoultepath/src/components/Footer.js",
  "absoultepath/src/utils.js",
  "absoultepath/src/app.js",
  "absoultepath/src/styles.css",
  "absoultepath/src/index.js"
]

5、循环依赖

5.1 demo

// 模块 A , a.js
const B = require('./B');
exports.foo = () => {
    console.log('A.foo');
    B.bar();
}

// 模块 B, b.js
const A = require('./A');
exports.bar = () => {
    console.log('B.bar');
    A.foo();
}
  1. 能进行依赖分析
{
    "a.js": {
        deps: [ "b.js" ]
    },
    "b.js": {
        deps: [ "a.js" ]
    },
}

  1. 无法进行拓扑排序 能进行拓扑排序的一定是DAG(有向无环图)

  2. 构建工具可以解决的循环依赖。例子如下

    1. 简单的循环引用
      例如：
    // a.js
import { b } from './b.js';
export const a = 1;
console.log(b);

// b.js
import { a } from './a.js';
export const b = 2;
console.log(a);

    这种情况下，大多数现代构建工具和模块系统可以处理，因为它们可以先执行模块的部分内容，然后再解析循环依赖。

    1. 延迟执行的循环依赖
      当循环依赖发生在函数内部或者使用动态导入时，构建工具通常可以处理：
    // a.js
import { b } from './b.js';
export function a() {
  return b() + 1;
}

// b.js
import { a } from './a.js';
export function b() {
  return a() + 1;
}
    1. 可拆分的循环依赖
      当循环依赖可以通过重构代码轻易拆分时，一些智能的构建工具可能会自动进行优化。

  3. 构建工具不能见解决的循环依赖，例子如下 5. 初始化时的相互依赖

    // a.js
import { B } from './b.js';
export class A {
  constructor() {
    new B();
  }
}

// b.js
import { A } from './a.js';
export class B {
  constructor() {
    new A();
  }
}

  4. 常见的解决方案

    • 构建工具报错提示
    • 开发者代码修改

6、DEMO代码

src/
├── index.js
├── index.css
├── app.js
├── utils.js
└── components/
    ├── Header.js
    └── Footer.js


// src/index.js
import App from './app';
import './styles.css';

console.log(App);

// src/app.js
import Header from './components/Header';
import Footer from './components/Footer';
import { helper } from './utils';

export default function App() {
  return helper(Header() + Footer());
}

// src/utils.js
export function helper(str) {
  return str.toUpperCase();
}

// components/Footer.js
export default function Header() {
  return '<header>Header</header>';
}

// components/Footer.js
export default function Footer() {
  return '<footer>Footer</footer>';
}
avatar
文章
阅读
粉丝