Overview
I/O in q is one of the most powerful and succinct features of the language. The names and behavior of the functions are idiosyncratic but the economy of expression is unrivaled.
I/O is realized via handles, which are symbolic names of resources such as files or machines on a network. One-and-done operations can be performed directly on the symbolic handle – e.g., you can read a file into memory in a single operation. For continuing operations, you open the symbolic handle to obtain an open handle. The open handle is a function that is applied to perform operations. When you have completed the desired operations, you close the open handle to free any allocated resources.
File Handles
`:[path]name
hcount and hdel
q)hcount `:/data/solong.txt
35
hdel instructs the OS to remove the file specified by its symbolic handle operand.
q)hdel `:/data/solong.txt
`:/data/solong.txt
Serializing and Deserializing q Entities
The magic is done by (an overload of) the binary set, whose left operand is a file handle and right operand is the entity to be written. The result is the symbolic handle of the written file. The file is automatically closed once the write is complete.
q)`:/data/a set 42
`:/data/a
q)`:/data/L set 10 20 30
_
q)`:/data/t set ([] c1:`a`b`c; c2:10 20 30)
_
q)get `:/data/a
42
q)get `:/data/L
_
q)get `:/data/t
_
An equivalent way to read a data file is with (an overload of) value
q)value `:/data/t
c1 c2
-----
a 10
b 20
c 30
Binary Data Files
As mentioned previously, q files come in two flavors, binary and text. Serialized q data persisted with set is written in binary form with a header at the beginning of the file. You can read it as raw binary data to inspect its internals.
Open a data file handle with open, whose result is a function called the open handle. This function should be stored in a variable, traditionally h, which is functionally applied to data to write it to the file.
q)`:/data/L set 10 20 30
`:/data/L
q)h:hopen `:/data/L
q)h[42]
3i
q)h 100 200
3i
We verify that the appends have been made.
q)hclose h
q)get `:/data/L
10 20 30 42 100 200
Writing and Reading Binary
Apply read on a file handle to read any file into q as a list of bytes. For example, we can read the previously serialized value L as bytes.
q)read1 `:/data/L set 10 20 30
0xfe2007000000000003000000000000000a0000000000000014000000000000001e..
If you want to write raw binary data, as opposed to the internal representation of a q entity containing the data, use the infelicitously named 1:. It takes a symbolic file handle as its left argument and a simple byte list as its right argument. Bytes in the right operand are essentially streamed to the file.
q)`:/data/answer.bin 1: 0x06072a
`:/data/answer.bin
q)read1 `:/data/answer.bin
0x06072a
Using Apply Amend
Fundamentalists can use Apply Amend in place of set to serialize q entities to files. To write the file, or overwrite an existing file, use assign :
q).[`:/data/raw; (); :; 1001 1002 1003]
`:/data/raw
q)get `:/data/raw
1001 1002 1003
To append to an existing file use ,
q).[`:/data/raw; (); ,; 42]
`:/data/raw
q)get `:/data/raw
1001 1002 1003 42
Save and Load on Tables
We have already seen that it is easy to write and read tables to/from persistent storage
q)`:/data/t set ([] c1:`a`b`c; c2:10 20 30; c3:1.1 2.2 3.3)
`:/data/t
q)get `:/data/t
_
In its simplest form save serializes a table in a global variable to a binary file having the same name as the variable. It overwrites an existing file.
q)t:([] c1:`a`b`c; c2:10 20 30; c3:1.1 2.2 3.3)
q)save `:/data/t
`:/data/t
q)get `:/data/t
c1 c2 c3
---------
a 10 1.1
b 20 2.2
c 30 3.3
As you might expect, load is the inverse of save meaning that it reads a serialized table from a file into a variable with the same name as the file. It creates the variable in the workspace or overwrites it if it already exists.
q)t / t doesn't exist
't
q)load `:/data/t
`t
q)t / now it does
c1 c2 c3
---------
a 10 1.1
b 20 2.2
c 30 3.3
q)save `:data/t.csv
`:data/t.csv
q)save `:data/t.xml
`:data/t.xml
q)save `:data/t.xls
`:data/t.xls
Splayed Tables
For larger tables that may not fit into memory on all machines, you can ask q to serialize each column of the table to its own file in a specified directory. A table persisted in this form is called a splayed table.
q))))`:/tmp/tsplay/ set ([] c1:10 20 30; c2:1.1 2.2 3.3)
`:/tmp/tsplay/
q))))\ls -ltrh /tmp/tsplay
"total 16"
"-rw-r--r-- 1 mac wheel 40B 3 10 00:57 c1"
"-rw-r--r-- 1 mac wheel 40B 3 10 00:57 c2"
Nearly all the metadata regarding the splayed table can be read from the file system – i.e., the name of table from directory and names of the columns from the files. The one missing bit is the order of the columns, which is stored as a serialized list in the hidden .d file.
q))))get hsym `$"/tmp/tsplay/.d"
`c1`c2
The following succeed, given all columns must be simple or compound lists. The latter means a list of simple lists of uniform type. An arbitrary general list column cannot be splayed.
q)`:/data/tok/ set ([] c1:2000.01.01+til 3; c2:1 2 3)
`:/data/tok/
q)`:/data/tok/ set ([] c1:1 2 3; c2:(1.1 2.2; enlist 3.3; 4.4 5.5))
`:/data/tok/
作者：Jeffry A. Borror
