临时不临时,chrome javascript 存储临时数据的五种方法大盘点

614 阅读6分钟

五种方法比较

sessionStorage 只对当前页面有效,关闭页面自动清理数据,但是可以通过恢复标签页找回。 localStorage 是持久存储,但存在大小限制。 indexDB 是数据库,但会在存储空间不足时自动清空。 书签名称适合用来存储简短的数据,比如存放当前页码,“书签”更名符其实。 外部数据库适合大量持久存储数据,类似indexedDB但永不删除,支持多索引排序。 油猴脚本自带的 GM_setValue 的后端是 levelDB,适合大量键值对的存放与读取。

其中,sessionStorage 虽然是临时数据,但保持标签开启就不会消失,如果配合tab持久化扩展,那么使用效果更佳。临时不临时,全在用户一念之间。

localStorage、indexDB 虽说是存储到磁盘,但都不太靠谱。GM_setValue 背后的 levelDB 是谷歌开发的开源项目。 跨浏览器不太方便。

“书签存储”由用户管理,新建书签代表需要存储页码,删除书签,那么页码数等据随之而去。书签需要全部读取到内存,所以书签很多时,导致新建删除书签有延时。外部数据库由完整 sqlite 数据库支持,适合自定扩展逻辑,比如还可用于下载音视频图片、导出某网站内部的历史记录等等,性能稳定,安全有效。 可跨浏览器使用。

“书签存储”、外部数据库是我发明的黑科技,专治浏览器各种不服。

以下就是 chrome 浏览器油猴脚本超长时间持久存储临时数据的办法五种的精彩内容,永不过期 ——

一、sessionStorage

方法一,sessionStorage,会话存储。和tab与网址关联,刷新、前进后退、重启、关闭标签页不会丢失数据,打开新的标签页或站点才开启新数据。

// 存储数据
sessionStorage.setItem('key', 'value');
// 获取数据
var data = sessionStorage.getItem('key');
// 删除数据
sessionStorage.removeItem('key');

二、localStorage、indexedDB

方法二,localStorage,本地存储。还有 indexedDB。需要注意的是,根据浏览器的不同,localStorage 存在大小限制,而 indexedDB 可能会在c盘空间不足时,自动清除全部数据,故此此类存储需要仔细测试后,才能放给用户使用。

// 存储数据
localStorage.setItem('key', 'value');

// 获取数据
var data = localStorage.getItem('key');

// 删除数据
localStorage.removeItem('key');

三、书签名称

方法三,将数据存储在书签名称之中。用户脚本没有相关api,因为用户脚本需要兼容各个浏览器,而浏览器扩展的书签api可能大相径庭。

那么如何在脚本中将一些短的数据永久存储在书签之中呢?可配合相关chrome扩展使用,涉及chrome 扩展背景页面与内容页面的通信,需要通过sendMessage与窗口监听器实现。需要注意的是,sendMessage 传的数据必须可序列化。

// content 向背景页面发送消息
chrome.runtime.sendMessage({ data: 'value' });

// 在背景页面监听消息
chrome.runtime.onMessage.addListener(function(message, sender, sendResponse) {
  // 处理接收到的消息
  // 各种处理,然后通过 sendResponse ,或 dispatchEvent 回调
});

content.js 运行于每个页面,可以直接在其中调用 扩展api, 向背景页面发送数据,然后回调。

content.js :

	window.addEventListener("my_bookmark_event", function(evt){
		chrome.runtime.sendMessage("my-event-xxx", function(data) { 
			if(e.detail && e.detail.cb) {
				var backEvt = new CustomEvent(e.detail.cb, {detail : data});
				window.dispatchEvent(backEvt, function(data) { });;
			}
		});
	});
	

content.js :

	
	window.addEventListener("my_bookmark_handler", function(evt){
		console.log("my_bookmark_handler:", evt, evt.detail);
		chrome.runtime.sendMessage({
			url:evt.detail.url
			,name:evt.detail.name
			,update:evt.detail.update
			,data:evt.detail
			,get:evt.detail.get
			,type:"bookmark-handler"
		}, function(data) { });
	});
	

后台消息处理示例:

// 后台消息处理
chrome.runtime.onMessage.addListener(function(message, sender, sendResponse) {
	if (message && message.type) {
		sender.pageUrl = sender.url;
		if(message.type == 'bookmark-handler'){
			
			if(message.get) { // 搜寻已有书签,并回调
				chrome.bookmarks.search({'url':message.url},
					function(bookmark_array) {
						// 如果有结果,
						……
						
						// 回调方法一
						sendResponse(...); 
						
						// 回调方法二
						chrome.tabs.executeScript(sender.tab.id, {
							code: `var backEvt = new CustomEvent("${message.data.cb}", {detail : ${JSON.stringify(e)}});
							window.dispatchEvent(backEvt, function(data) { });`
							}, () => {});
					}
				);
			} else if(message.update) {
				// 同理,当已存在书签时,更新而非重复创建。

			} else {
				// 同理,chrome允许创建重复url的书签。
			}
		
		} 
		
		
	}
});

四、外部数据库

比如java,就可以用jdbc驱动sqlite数据库,然后开启微型服务器,持久化存储信息。

以下是我造的轮子,名之曰FFDB:

/** Sqlite Database helper */
public class FFDB {
	static FFDB instance;
	final Connection database;
	Statement statement;
	HashMap<String, PreparedStatement> preparedStatements = new HashMap<>();
	static final HashMap<String, FHDB> FHDBs = new HashMap<>();

	ExecutorService threadPool = Executors.newFixedThreadPool(5);
	boolean bUniqueTable;
	
	public static FFDB getInstance() {
		if (instance==null) {
			synchronized (FFDB.class) {
				try {
					if (instance==null)
						instance = new FFDB();
				} catch (SQLException e) {
					CMN.Log(e);
				}
			}
		}
		return instance;
	}

	FFDB() throws SQLException {
		SQLiteConfig config = new SQLiteConfig();
		config.setSharedCache(true);
		database = DriverManager.getConnection("jdbc:sqlite:D:/sample.db", config.toProperties());
		statement=database.createStatement();
		database.setAutoCommit(false);
	}

	void getInitMap(JSONObject json, StringBuilder sb) {
		Object obj;
		int idx=0;
		Set<String> keys = json.keySet();
		Iterator<String> iter = keys.iterator();
		String key;
		boolean ftd = false;
		while (iter.hasNext()) {
			key=iter.next();
			obj = json.get(key);
			if (ftd) {
				sb.append(" , ");
			} else {
				ftd = true;
			}
			idx++;
			if (key.equals("rowId")) {
				sb.append("id INTEGER PRIMARY KEY AUTOINCREMENT");
			}
			else if (obj instanceof Integer
					|| obj instanceof Long
					|| obj instanceof Short
			) {
				sb.append(key).append(" INTEGER");
				if (IU.parsint(obj+"", -1)!=-1) {
					sb.append(" DEFAULT ").append(obj);
				}
			}
			else if (obj instanceof String) {
				sb.append(key).append(" TEXT");
				if (!obj.toString().equals("-1")) {
					sb.append(" DEFAULT ").append("'").append(obj).append("'");
				}
			}
			else {
				CMN.Log("getInitMap::不支持::", idx, obj);
			}
		}
	}
	
	public void openTable(String table, JSONObject map, JSONObject indexed) throws SQLException {
		StringBuilder sb = new StringBuilder();
		sb.append("CREATE TABLE IF NOT EXISTS ")
			.append(table)
			.append("(");
		getInitMap(map, sb);
		String sql = sb.append(")").toString();
		CMN.Log("openTable::sql::", sql);
		statement.execute(sql);
		if (indexed!=null) {
			Iterator<String> iter = indexed.keySet().iterator();
			String key;
			boolean ftd = false;
			while (iter.hasNext()) {
				key=iter.next();
				sql="CREATE "+(IU.parsint(indexed.getInteger(key), 0)==1?" UNIQUE ":"")+"INDEX if not exists "+table+"_"+key+"_index ON "+table+"("+key+")";
				statement.execute(sql);
			}
		}
	}

	void getValNames(JSONObject json, StringBuffer sb, boolean kuohao) {
		Set<String> keys = json.keySet();
		Iterator<String> iter = keys.iterator();
		String key;
		if(kuohao) sb.append("(");
		boolean ftd = false;
		while (iter.hasNext()) {
			key=iter.next();
			if (ftd) {
				sb.append(",");
			} else {
				ftd = true;
			}
			sb.append(key);
		}
		if(kuohao) sb.append(")");
	}

	void getValMaps(JSONObject json, StringBuffer sb) {
		Set<String> keys = json.keySet();
		Iterator<String> iter = keys.iterator();
		String key;
		boolean ftd = false;
		while (iter.hasNext()) {
			key=iter.next();
			if (ftd) {
				sb.append(" & ");
			} else {
				ftd = true;
			}
			sb.append(key)
				.append("=");
			Object obj = json.get(key);
			if (true) {
				sb.append("?");
			} else {
				if (obj instanceof Integer
						|| obj instanceof Long
						|| obj instanceof Short
				) {
					sb.append(obj);
				}
				else if (obj instanceof String) {
					sb.append("'").append(obj).append("'");
				}
				else {
					CMN.Log("setValues::不支持::", key, obj);
				}
			} 
		}
	}
	
	void getValues(ResultSet set, JSONObject json) throws SQLException {
		Iterator<String> iter = json.keySet().iterator();
		Object obj;
		int idx=0;
		String key;
		while (iter.hasNext()) {
			key=iter.next();
			idx++;
			json.put(key, set.getObject(idx));
		}
	}
	
	void setValues(PreparedStatement prepared, JSONObject json) throws SQLException {
		prepared.clearParameters();
		Collection<Object> vals = json.values();
		Iterator<Object> iter = vals.iterator();
		Object obj;
		int idx=0;
		while (iter.hasNext()) {
			obj=iter.next();
			idx++;
			if (obj instanceof Integer) {
				prepared.setInt(idx, (Integer) obj);
			}
			else if (obj instanceof Long) {
				prepared.setLong(idx, (Long) obj);
			}
			else if (obj instanceof String) {
				prepared.setString(idx, (String) obj);
			}
			else if (obj instanceof Blob) {
				prepared.setBlob(idx, (Blob) obj);
			} else {
				CMN.Log("setValues::不支持::", idx, obj);
			} 
		}
	}
	
	public void putBatch(String table, JSONArray array) {
		try {
			//statement.execute("insert into TEST(rid, fav) VALUES("+rid+", "+fav+")");
			JSONObject json = array.getJSONObject(0);
			StringBuffer sb = new StringBuffer();
			sb.append("REPLACE INTO ").append(table);
			getValNames(json, sb, true);
			sb.append(" VALUES(");
			boolean ftd = false;
			for (int i = 0; i < json.size(); i++) {
				if (ftd) {
					sb.append(",");
				} else {
					ftd = true;
				}
				sb.append("?");
			}
			sb.append(")");
			String sql = sb.toString();
			synchronized (database) {
				PreparedStatement prepared = preparedStatements.get(sql);
				if (prepared==null || prepared.isClosed()) {
					preparedStatements.put(sql, prepared=database.prepareStatement(sql));
				}
				int cc= 0;
				try {
					prepared.clearBatch();
					cc = 0;
					for (int i = 0; i < array.size(); i++) {
						json = array.getJSONObject(i);
						prepared.clearParameters();
						setValues(prepared, json);
						prepared.addBatch();
						if (i%1024==0) {
							database.commit();
							cc += prepared.executeBatch().length;
							prepared.clearBatch();
						}
					}
					cc += prepared.executeBatch().length;
				} catch (SQLException e) {
					CMN.Log(e);
					database.rollback();
				}
				CMN.Log("executeBatch::", cc, array.size());
			}
		} catch (Exception e) {
			CMN.Log(e);
		}
	}

	public void put(String table, JSONObject json) {
		try {
			//statement.execute("insert into TEST(rid, fav) VALUES("+rid+", "+fav+")");
			StringBuffer sb = new StringBuffer();
			sb.append("REPLACE INTO ").append(table);
			getValNames(json, sb, true);
			sb.append(" VALUES(");
			boolean ftd = false;
			for (int i = 0; i < json.size(); i++) {
				if (ftd) {
					sb.append(",");
				} else {
					ftd = true;
				}
				sb.append("?");
			}
			sb.append(")");
			String sql = sb.toString();
			synchronized (database) {
				PreparedStatement prepared = preparedStatements.get(sql);
				if (prepared==null || prepared.isClosed()) {
					preparedStatements.put(sql, prepared=database.prepareStatement(sql));
				}
				try {
					setValues(prepared, json);
					prepared.execute();
					database.commit();
				} catch (SQLException e) {
					CMN.Log(e);
					database.rollback();
				}
			}
		} catch (Exception e) {
			CMN.Log(e);
		}
	}
	
	public JSONObject get(String table, JSONObject ret, JSONObject where) {
		//try {
		//	//ResultSet set = statement.executeQuery("select fav from TEST where rid="+rid+" limit 1");
		//	synchronized (database) {
		//		PreparedStatement prepared = database.prepareStatement("select fav from TEST where rid=? limit 1");
		//		prepared.setInt(1, rid);
		//		ResultSet set =  prepared.executeQuery();
		//		if (set.next()) {
		//			defVal = set.getInt(1);
		//		}
		//		set.close();
		//	}
		//} catch (Exception e) {
		//	CMN.Log(e);
		//}
		//return defVal;
		try {
			//ResultSet set = statement.executeQuery("select fav from TEST where rid="+rid+" limit 1");
			StringBuffer sb = new StringBuffer();
			sb.append("select ");
			getValNames(ret, sb, false); // 模板
			sb.append(" FROM ").append(table).append(" WHERE ");
			getValMaps(where, sb);
			sb.append(" limit 1");
			String sql = sb.toString();
			CMN.Log("get::sql::", sql);
			synchronized (database) {
				PreparedStatement prepared = preparedStatements.get(sql);
				if (prepared==null || prepared.isClosed()) {
					prepared=database.prepareStatement(sql);
				}
				setValues(prepared, where);
				ResultSet set = prepared.executeQuery();
				if (set.next()) {
					getValues(set, ret);
				}
				set.close();
			}
		} catch (Exception e) {
			CMN.Log(e);
		}
		return ret;
	}

	public JSONArray getBatch(String table, JSONObject temp, JSONArray array) {
		JSONObject where = array.getJSONObject(0);
		JSONArray ret = new JSONArray();
		try {
			//ResultSet set = statement.executeQuery("select fav from TEST where rid="+rid+" limit 1");
			StringBuffer sb = new StringBuffer();
			sb.append("select ");
			getValNames(temp, sb, false); // 模板
			sb.append(" FROM ").append(table).append(" WHERE ");
			getValMaps(where, sb);
			sb.append(" limit 1");
			String sql = sb.toString();
			CMN.Log("getBatch::sql::", sql);
			synchronized (database) {
				PreparedStatement prepared = preparedStatements.get(sql);
				if (prepared==null || prepared.isClosed()) {
					prepared=database.prepareStatement(sql);
				}
				for (int i = 0; i < array.size(); i++) {
					where = array.getJSONObject(i);
					setValues(prepared, where);
					ResultSet set = prepared.executeQuery();
					if (set.next()) {
						JSONObject templet = (JSONObject) temp.clone();
						getValues(set, templet);
						ret.add(templet);
					}
					set.close();
				}
			}
		} catch (Exception e) {
			CMN.Log(e);
		}
		CMN.Log("getBatch::", ret.size(), array.size());
		return ret;
	}

	public void downloadUnique(JSONObject temp) {
		CMN.Log("downloadUnique::", temp);
//		filename: ""
//		finalUrl: "https://"
//		referrer: "https://"
		String filename = temp.getString("filename");
		String finalUrl = temp.getString("finalUrl");
		int idx = finalUrl.indexOf("?");
		if (idx>0) finalUrl = finalUrl.substring(0, idx);
		String referrer = temp.getString("referrer");
		if (!bUniqueTable) {
			try {
				openTable("files", FFDB.parseObject("{url:'', name:'', fav:0}"), FFDB.parseObject("{url:1}"));
				bUniqueTable = true;
			} catch (SQLException ignored) { }
		}
		JSONObject ret = new JSONObject();
		ret.put("name", null);
		get("files", ret, FFDB.parseObject("{url:'"+finalUrl+"'}"));
		if (ret.get("name") != null) {
			CMN.Log("已经下载过了!");
		} else {
			CMN.Log("下载…");
			ret.put("name", filename);
			ret.put("url", finalUrl);
			ret.put("fav", 0);
			threadPool.execute(new Runnable() {
				@Override
				public void run() {
					try {
						DownloadInfo info = new DownloadInfo(new URL(temp.getString("finalUrl")));
						info.setReferer(new URL(temp.getString("referrer")));
						String path = "C:\\Users\\Admin\\Downloads\\video";
						String fn = filename;
						String fix = ".mp4";
						int idx = fn.lastIndexOf(".");
						if (idx>0) {
							fix = fn.substring(idx);
							fn = fn.substring(0, idx);
						}
						File file = new File(path+fn+fix);
						int cc=0;
						while (file.exists()) {
							file = new File(path+fn+"."+(++cc)+fix);
						}
						WGet wGet = new WGet(info, file);
						wGet.download();
						put("files", ret);
					} catch (Exception e) {
						CMN.Log(e);
					}
				}
			});
		} 
	}



	public static JSONObject parseObject(String text) {
		if (text==null) {
			return null;
		}
		boolean nord = (JSON.DEFAULT_PARSER_FEATURE & Feature.OrderedField.getMask())==0;
		JSON.DEFAULT_PARSER_FEATURE |= Feature.OrderedField.getMask();
		JSONObject ret = JSONObject.parseObject(text);
		if (nord) {
			JSON.DEFAULT_PARSER_FEATURE &= ~Feature.OrderedField.getMask();
		}
		return ret;
	}

	static class FHDB{
		final String name;
		final HashSet<String> FHash = new HashSet<>();
		final StringBuilder FStr = new StringBuilder();
		boolean FHashPrepared;
		FHDB(String name) {
			this.name = name;
		}
		public void prepareHashset() {
			if (!FHashPrepared) {
				synchronized (FHash) {
					if (!FHashPrepared) {
						FHashPrepared = true;
						try {
							String key;
							BufferedReader reader = new BufferedReader(new FileReader("D:\\"+name+".txt"));
							while ((key=reader.readLine())!=null) {
								FStr.append(key).append("\n");
								FHash.add(key);
							}
							reader.close();
						} catch (Exception e) {
							CMN.Log(e);
						}
					}
				}
			}
		}
		public void putHashSet(JSONArray array) {
			prepareHashset();
			ArrayList<String> toSave = new ArrayList<>(array.size());
			for (int i = 0; i < array.size(); i++) {
				String key = array.getString(i);
				if (!FHash.contains(key)) {
					toSave.add(key);
				}
			}
			if (toSave.size()>0) {
				synchronized (FHash) {
					try {
						BufferedOutputStream output = new BufferedOutputStream(new FileOutputStream("D:\\"+name+".txt", true));
						for (String key:toSave) {
							FHash.add(key);
							FStr.append(key).append("\n");
							output.write(key.getBytes(StandardCharsets.UTF_8));
							output.write("\r\n".getBytes(StandardCharsets.UTF_8));
						}
						output.close();
					} catch (Exception e) {
						CMN.Log(e);
					}
				}
			}
		}
		public String getHashSet() {
			prepareHashset();
			return FStr.toString();
		}
	}


	public static Response handleRequest(HTTPSession session) {
		if (Method.POST.equals(session.getMethod())) {
			try {
				session.parseBody(null);
				//SU.Log("DB.jsp::", session.getHeaders());
				SU.Log("DB.jsp::", session.getParameters(), session.getMethod());
				FFDB db = FFDB.getInstance();
//				String text = session.getParameter("data");
				String text = session.getParms().get("data");
				//SU.Log("text::", text, text.length());
				//text = URLDecoder.decode(text);
				Objects.requireNonNull(text);
				JSONObject data;
				if (text.startsWith("[")) {
					data = JSONArray.parseArray(text).getJSONObject(0);
				} else {
					data = FFDB.parseObject(text);
				}
				if (data.containsKey("dwnld")) {
					getInstance().downloadUnique(data);
				} else {
					JSONArray fSet = data.getJSONArray("fSet");
					String tableName = data.getString("table");
					if (fSet != null) {
						if (tableName==null) {
							tableName = "sample";
						}
						if (tableName.startsWith("sample") && !tableName.contains("\\") && !tableName.contains("/")) {
							FHDB fSetter;
							synchronized (FHDBs) {
								fSetter = FHDBs.get(tableName);
								if (fSetter==null) {
									fSetter = new FHDB(tableName);
								}
							}
							if (fSet.size() == 0) {
								return newFixedLengthResponse(fSetter.getHashSet()) ;
							} else {
								fSetter.putHashSet(fSet);
							}
						}
					}
					else {
						//CMN.Log(data);
						JSONObject json = data.getJSONObject("json");
						JSONArray batch = data.getJSONArray("batch");
						JSONObject where = data.getJSONObject("where");
						JSONObject indexed = data.getJSONObject("indexed");
						if (json==null && batch!=null) {
							json = batch.getJSONObject(0);
						}
						CMN.Log(tableName, json, where, indexed);
						Objects.requireNonNull(tableName);
						Objects.requireNonNull(json);
						if (indexed != null) { // open
							db.openTable(tableName, json, indexed);
						}
						else if (where != null) { // get
							String ret;
							if (batch != null) {
								JSONArray result = db.getBatch(tableName, json, batch);
								ret = result.toString();
								CMN.Log("getBatch::", result.size(), json.size());
							} else {
								ret = db.get(tableName, json, where).toString();
							}
							return newFixedLengthResponse(ret);
						}
						else { // set
							if (batch != null) {
								db.putBatch(tableName, batch);
							} else {
								db.put(tableName, json);
							}
						}
					}
				} 
			} catch (Exception e) {
				CMN.Log(e);
			}
		}
		return emptyResponse;
	}

	
}

还包括了一个简易下载器,同样存储下载过的文件名到数据库,避免重复下载。

nanohttpd中使用:

	@Override
	public Response handle(HTTPSession session) throws IOException {
        
		String uri = session.getUri();
                ...
		if(key.startsWith("\\DB.jsp"))
			return app.handleFFDB(session);
		
       }
   

五、用户脚本自带功能 GM_setValue

GM_setValue("my_key", {my_val:1});

var myVal = GM_getValue("my_key", {});

需要权限 @require GM_setValue。数据存储由脚本管理器管理,一般是 LevelDB ,但也可能是 sqlite。