io 顺序写与随机写之间的区别

698 阅读4分钟

image.png 一直以来,无论是kafka,还是redis,谈及原理时,都会提及顺序写会带来高性能等好处。偶然看见一篇说的通透又剪短的文章。本想彻底译为中文,但过程中发现反而词不表意了,所以只将比较难翻译的部分转为中文,意为便于英语不太好的伙伴们更简单清晰的理解。所以本文的目的是想做到既保持原汁原味又简单快速的理解,希望能有所帮助和收获

Difference between sequential write and random write

顺序写与随机写在下面的情况下的区别是什么,1、基于机械硬盘(HDD)的系统 2、基于固态硬盘(SSD)的系统 What is the difference between sequential write and random write in case of :- 1)Disk based systems 2)SSD [Flash Device ] based systems

When you write two blocks that are next to each-other on disk, you have a sequential write. When you write two blocks that are located far away from eachother on disk, you have random writes. With a spinning hard disk, the second pattern is much slower (can be magnitudes), because the head has to be moved around to the new position.

When people talk about sequential vs random writes to a file, they're generally drawing a distinction between writing without intermediate seeks ("sequential"), vs. a pattern of seek-write-seek-write-seek-write, etc. ("random").

The distinction is very important in traditional disk-based systems, where each disk seek will take around 10ms. Sequentially writing data to that same disk(依次将数据写入同一个磁盘) takes about 30ms per MB. So if you sequentially(依次) write 100MB of data to a disk, it will take around(大约) 3 seconds. But if you do 100(次) random writes of 1MB each(每次), that will take a total of 4 seconds (3 seconds for the actual writing, and 10ms*100 == 1 second for all the seeking).

当每次随机写越来越小时,你会为磁盘寻道付出了越来越多的代价。在极端的情况下你执行一亿次,每次随机的一个字节的写,整个的写入你仍将花费3秒,但是磁盘寻址你将花费11.57天! 所以你的写是顺序还是随机 的清晰程度确实会影响完成任务所需的时间。 As each random write gets smaller, you pay more and more of a penalty for the disk seeks. In the extreme case where you perform 100 million random 1-byte writes, you'll still net 3 seconds for all the actual writes, but you'd now have 11.57 days worth of seeking to do! So clearly the degree to which your writes are sequential vs. random can really affect the time it takes to accomplish your task.

The situation is a bit different when it comes to flash(当它是闪存). With flash, you don't have a physical disk head that you must move around. (This is where the 10ms seek cost comes from for a traditional disk). However, flash devices tend to have large page sizes (the smallest "typical" page size is around 512 bytes according to wikipedia, and 4K page sizes appear to be common as well(4K页面大小似乎也很常见 )). So if you're writing a small number of bytes, flash still has overhead(开销) in that you must read out an entire page, modify the bytes you're writing, and then write back the entire page.

我不知道你的(flash page)闪存页面的具体大小,但从经验上看,在闪存上,如果您的每次写入的大小通常与设备的页面大小相当。那么你不会看到随机写入和顺序写入之间有太大的性能差异。如果您的每次写入与设备页面大小相比都很小,那么在进行随机写入时您会看到一些开销。 I don't know the characteristic numbers for flash off the top of my head. But the rule of thumb is that on flash if each of your writes is generally comparable in size to the device's page size, then you won't see much performance difference between random and sequential writes. If each of your writes is small compared to the device page size, then you'll see some overhead when doing random writes.

结合以上,应用程序层往往会对你隐藏很多东西。例如,在kernel内核、硬盘/闪存控制器等,这可能会在您的“顺序”写作中插入不明显的寻址。但在大多数情况下,在应用层看起来是顺序写的(没有寻址,都是连续的io)将具有顺序写入的性能,而在应用层看起来是随机写的(没有寻址,都是连续的io)将具有随机写入的性能 Now for all of the above, it's true that at the application layer much is hidden from you. There are layers in the kernel, disk/flash controller, etc. that could for example interject non-obvious seeks in the middle of your "sequential" writing. But in most cases, writing that "looks" sequential at the application layer (no seeks, lots of continuous I/O) will have sequential-write performance while writing that "looks" random at the application layer will have the (generally worse) random-write performance.

Difference between sequential write and random write