原作者：快手商业化前端温鑫
bit作为一个组件市场基础模块，远端元数据信息对于我们并不透明,了解元数据信息有利于我们更方便的对bit进行理解。
暂且看一下bit的远端scope目录结构，
- objects 所有的hash文件都存储与这个目录下。
- components 这个目前这个文件是空的暂时不知道事干吗的
- scope.json 来描述当前的scope信息
- index.json 来存储当前scope下的所有组件
既然index.json也就是数据源头了。内容大致如下。
[{
"id": {
"scope": "ad-fe",
"name": "ad-banner"
},
"isSymlink": false,
"hash": "b08db6a1550e8a14b7ed1206fd04fc5d49c8cd10"
},]
其中hash部分前两位对应objects的文件夹名，后面的是对应的文件。
Hash文件名存储
通过zlib 进行压缩，然后通过hash文件名存储的方式。
优点在于
- 存储数据的容量变小。
- 散列分布的数据文件更有利于文件系统操作文件。
hash生成方法
const crypto = require('crypto');
function sha1(data, encoding = 'hex') {
return crypto
.createHash('sha1')
.update(data)
.digest(encoding);
}
data 对应不同model实现的object的id方法
model 代码在src/scope/models
比如
//src/scope/models/model-component.ts
id(): string {
return this.scope ? [this.scope, this.name].join('/') : this.name;
}
提取parse方法
查看bit源码src/scope/objects/object.ts
const zlib = require('zlib');
const fs = require('fs');
const NULL_BYTE = '\u0000';
function parse(buffer, types) {
const firstNullByteLocation = buffer.indexOf(NULL_BYTE);
const headers = buffer.slice(0, firstNullByteLocation).toString();
const contents = buffer.slice(firstNullByteLocation + 1, buffer.length);
const [header] = headers.split(" ");
try{
console.log(header,JSON.stringify(JSON.parse(contents.toString()),null,2))
}catch(e){
console.log(header,contents.toString())
}
}
// 本地解析
let file = 'f73981c95c06908d6de07f9d12cd0482caf91d19';
let dir = "./ad-fe/objects/"+ file.substr(0,2);
let f = file.substr(2);
fs.readFile(path.resolve(dir,f), (err, data) => {
if (err) throw err;
zlib.inflate(data, (err, data) => {
parse(data)
});
// console.log(output);
});
存储层级
graph LR
Component-->Version
Version-->Source
Component
获取到第一层数据结构
对应源码对象src/scope/models/model-component.ts
Component {
"name": "ad-banner",
"scope": "ad-fe",
"versions": {
"0.0.1": "117f652e37217dd753a1b5b4cfb474547a15c691",
"0.0.2": "2b18288ec0161468ca2aa5c6d0cea19679eb1fd4",
"0.0.3": "713cb2a80153cecc9f91c5267986e3e466a7b476",
"0.0.4": "d41fa1b62f0085d32d83ad943716919b031ceb1b",
"0.0.5": "90960d4a65860e556a413e78ef7c8b7f5cd1544c",
"0.0.6": "75005b5c51dbeb11e55d22a93598ec022b45251c",
"0.0.7": "03fce2181b9fb8af516f74a96693e8b20d707cac",
"0.0.8": "532f21f071ac4072ddffdc7d07e8a54467ecae92",
"0.0.9": "0e9731bae3755db235ae93423bae3290c5089615",
"0.0.10": "f73981c95c06908d6de07f9d12cd0482caf91d19"
},
"lang": "javascript",
"deprecated": false,
"bindingPrefix": "@bit",
"remotes": [
{
"url": "ssh://bit@bit.xxxx.com:/data/bit/ad-fe",
"name": "ad-fe",
"date": "1590068611220"
}
]
}
这些数据结构还是比较清晰的。
Version
进一步获取第二层元数据
对应源码对象src/scope/models/version.ts
Version {
"files": [
{
"file": "ea3a806c29ff79c45dfdad4e67a61c2cea8e3d80",
"relativePath": "src/components/ad-banner/index.vue",
"name": "index.vue",
"test": false
}
],
"mainFile": "src/components/ad-banner/index.vue",
"bindingPrefix": "@bit",
"log": {
"message": "",
"date": "1590065100777",
"username": "xuhongzhi",
"email": "xuhongzhi@kuaishou.com"
},
"ci": {},
"docs": [],
"dependencies": [
{
"id": {
"scope": "ad-fe",
"name": "logger",
"version": "0.0.1"
},
"relativePaths": [
{
"sourceRelativePath": "src/logger/index.ts",
"destinationRelativePath": "src/logger/index.ts",
"importSpecifiers": [
{
"mainFile": {
"isDefault": false,
"name": "pageShowLog"
}
},
{
"mainFile": {
"isDefault": false,
"name": "showLog"
}
},
{
"mainFile": {
"isDefault": false,
"name": "clickLog"
}
}
],
"isCustomResolveUsed": true,
"importSource": "@/logger"
}
]
}
],
"devDependencies": [],
"compilerDependencies": [],
"testerDependencies": [],
"flattenedDependencies": [
{
"scope": "ad-fe",
"name": "logger",
"version": "0.0.1"
}
],
"flattenedDevDependencies": [],
"flattenedCompilerDependencies": [],
"flattenedTesterDependencies": [],
"packageDependencies": {
"@ks/ks-bridge": "2.0.0-alpha.5",
"vue-property-decorator": "^8.0.0"
},
"devPackageDependencies": {},
"peerPackageDependencies": {},
"compilerPackageDependencies": {},
"testerPackageDependencies": {},
"customResolvedPaths": [],
"overrides": {},
"packageJsonChangedProps": {},
"extensions": []
}
Source
通过file指定的hash我们可以获取到源文件数据。
源文件不仅仅是源码文件，还比如静态文件
Source <template>
<view :class="['ad-banner', 'needsclick', { isTeam }]">
<swiper
:class="['ad-banner-swiper', 'needsclick', { isTeam }]"
:indicator-dots="true"
indicator-color="rgba(255,255,255,0.8)"
indicator-active-color="#ff5000"
:autoplay="true"
:circular="true"
:interval="3000"
:duration="500"
@change="changeCurrentMonitor"
>
<swiper-item v-for="(i, index) in config" :key="index" @click="onClickBanner(i)" class="needsclick">
<!--...more -->
}
border-radius: 16rpx;
overflow: hidden;
}
</style>
总结
通过这些元信息我们可以更好的了解bit的数据格式，和获取数据的过程。bit存储没有什么魔法，解析起来也相对容易。
其hash文件存储的原因网上没找到太多介绍。只找到一个 File Name Hashing: Creating a Hashed Directory Structure
大致就是说能存的更多。
最后行文匆忙，如有错误，欢迎指正。
