Physical drive write cache state на что влияет
Перейти к содержимому

Physical drive write cache state на что влияет

  • автор:

Почему такая низкая скорость HP Smart Array P420/1GB?

К контроллеру HP Smart Array P420/1GB подключены 4 ssd samsung 850pro 1gb. Диски собраны в raid-10. Эта полка используется как local-lvm хранилище proxmox 4.2. В виртуальную машину установлена windows server 2012r2. Измеряю скорость и получаю

----------------------------------------------------------------------- CrystalDiskMark 5.2.1 x64 (C) 2007-2017 hiyohiyo Crystal Dew World : http://crystalmark.info/ ----------------------------------------------------------------------- * MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s] * KB = 1000 bytes, KiB = 1024 bytes Sequential Read (Q= 32,T= 1) : 995.651 MB/s Sequential Write (Q= 32,T= 1) : 153.450 MB/s Random Read 4KiB (Q= 32,T= 1) : 41.382 MB/s [ 10103.0 IOPS] Random Write 4KiB (Q= 32,T= 1) : 10.231 MB/s [ 2497.8 IOPS] Sequential Read (T= 1) : 833.864 MB/s Sequential Write (T= 1) : 55.364 MB/s Random Read 4KiB (Q= 1,T= 1) : 16.172 MB/s [ 3948.2 IOPS] Random Write 4KiB (Q= 1,T= 1) : 1.088 MB/s [ 265.6 IOPS] Test : 1024 MiB [C: 73.1% (23.1/31.7 GiB)] (x5) [Interval=5 sec] Date : 2017/04/11 21:33:44 OS : Windows Server 2012 R2 Server Standard (full installation) [6.3 Build 9600] (x64)

Диск подключен с параметром No cache.
Откуда такая низкая производительность?

P.S. подскажите, пожалуйста, как на debian замерить производительность диска в аналогичных попугаях.

  • Вопрос задан более трёх лет назад
  • 5453 просмотра

4 комментария

Оценить 4 комментария

Disk Cache wiki documentation

You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.

IsThisThingOn
Member

Nov 26, 2021 66 15 13

I was reading through the wiki
https://pve.proxmox.com/wiki/Performance_Tweaks#Disk_Cache
have some questions and maybe even suggestions for improvement. Sometimes it also contradicts the points from the article it references to.

I’m no expert by any means and not a native speaker, so please don’t take these suggestions as an insult. I could be very wrong on some basic understandings
Because it is a complex topic, I like to number the different points.

1. Maybe start with an exclaimer that for a secure writeback you need the barrier-passing feature. That feature is enabled by default on ext4
2. Do NTFS or ZFS guests also support write barriers by default?
3. The wiki says this only applies to RAW, but I was unable to find good documentation how these caching modes behave for qcow2. Anyone has some inputs?
4. The table is a little bit confusing to newcomers. I think simpler to understand by splitting up the Host Page Cache into read and write and what data can be lost in a power failure. Also I would replace «power loss» with «unexpected shutdown» because that covers more incidents than can happen.
5. Rename the mode «none» to «no cache» or «Default (no cache)», because that is how the GUI shows it
6. The description for cache=none says: «Because the actual storage device may report a write as completed when placed in its write queue only». For what device is that true? Are we talking about the drive firmware lying? I had this discussion in a TrueNAS forum, where I asked how we can trust HDDs to not have writes in a queue and the general consent was that there is no drive lying about having writes written while they are still in cache no matter if SSD or HDD. Otherwise the outrage would be even bigger than the SMR fiasko from WD. That is why I would ditch the «in case of power failure you can lose data»
7. «cache=none seems to be the best performance», can that really be true? Compared to other modes, it is just not using cache and directly talking to the disk, how can that be the best performance?
8. Reorder the modes to match the order of the GUI
9. According to the notes, directsync is the safest option. But why? Write through behaves the same but with a read cache, how is that less secure?

Here is how I as a beginner would write that wiki entry:

Mode Proxmox Page cache read Proxmox Page cache write Disk Write data loss from unexpected shutdowns? Notes
default (No Cache) disabled disabled normal behavior Only async writes Normal read performance

Normal write performance

Slow write performance,
because even async writes are written as sync writes.

Slow write performance, because even async writes are written as sync writes

Normal write performance

Good write performance for sync writes

  • host page cache is not used
  • In case of a power failure, you could loose async writes
  • You need to use the barrier option in your Linux guest to avoid FS corruption in case of power failure.

This mode causes qemu-kvm to interact with the disk image file or block device with O_DIRECT semantics, so the host page cache is bypassed and I/O happens directly between the qemu-kvm userspace buffers and the storage device. The guest is expected to send down flush commands as needed to manage data integrity. Equivalent to direct access to your hosts’ disk, performance wise.

  • host page cache is not used
  • guest disk cache mode is writethrough
  • similar to Write through, an fsync is made for each write.

This mode causes qemu-kvm to interact with the disk image file or block device with both O_DSYNC and O_DIRECT semantics, where writes are reported as completed only when the data has been committed to the storage device, and when it is also desirable to bypass the host page cache. Like cache=writethrough, it is helpful to guests that do not send flushes when needed. It was the last cache mode added, completing the possible combinations of caching and direct access semantics.

  • host page cache is used as read cache
  • guest disk cache mode is writethrough
  • similar to Direct Sync, an fsync is made for each write.

This mode causes qemu-kvm to interact with the disk image file or block device with O_DSYNC semantics, where writes are reported as completed only when the data has been committed to the storage device. The host page cache is used in what can be termed a writethrough caching mode. Guest virtual storage adapter is informed that there is no writeback cache, so the guest would not need to send down flush commands to manage data integrity. The storage behaves as if there is a writethrough cache.

  • host page cache is used as read & write cache
  • guest disk cache mode is writeback
  • In case of a power failure, you could loose async writes
  • You need to use the barrier option in your Linux guest to avoid FS corruption in case of power failure.

This mode causes qemu-kvm to interact with the disk image file or block device with neither O_DSYNC nor O_DIRECT semantics, so the host page cache is used and writes are reported to the guest as completed when placed in the host page cache, and the normal page cache management will handle commitment to the storage device. Additionally, the guest’s virtual storage adapter is informed of the writeback cache, so the guest would be expected to send down flush commands as needed to manage data integrity.

  • as Write Back, but ignores flush commands from the guest!
  • Warning: No data integrity even if the guest is sending flush commands. Not recommended for production use.

Конфигурация ADAPTEC 78165 на использование Write-Cache

Добрый день, уважаемые специалисты!
Домашняя система, данные некритичны, UPS есть, AFM-700 нет.
В данный момент подключил к 78165 два HDD в режиме Single Drive, цель рост производительности по сравнению с 3 Gbps портам контроллера Intel X79 за счет использования 1024 МБ кэша устройства.

Конфигурация сейчас такая, смущает вот этот вот момент «Write-cache status : Off» :

C:\Windows\system32>arcconf getconfig 1 Controllers found: 1 ---------------------------------------------------------------------- Controller information ---------------------------------------------------------------------- Controller Status : Optimal Controller Mode : Simple Volume Controller Cache Preservation : Enabled Channel description : SAS/SATA Controller Model : Adaptec ASR78165 Controller Serial Number : 3A32133C2FA Controller World Wide Name : 50000D1107221500 Controller Alarm : Enabled Physical Slot : 3 Temperature : 44 C/ 111 F (Normal) Installed memory : 1024 MB Global task priority : High Performance Mode : Default/Dynamic Host bus type : PCIe Host bus speed : 8000 MHz Host bus link width : 8 bit(s)/link(s) PCI Device ID : 652 Stayawake period : Disabled Spinup limit internal drives : 0 Spinup limit external drives : 0 Defunct disk drive count : 0 Logical devices/Failed/Degraded : 2/0/0 NCQ status : Enabled Statistics data collection mode : Enabled Global Physical Device Write Cache Policy: Enabled Monitor Log Severity Level : Informational -------------------------------------------------------- RAID Properties -------------------------------------------------------- Copyback : Disabled Automatic Failover : Enabled Background consistency check : Enabled Background consistency check period : 30 -------------------------------------------------------- Controller BIOS Setting Information -------------------------------------------------------- Runtime BIOS : Disabled Array BBS Support : Enabled Physical Drives Displayed during POST : Enabled Backplane Mode : IBPI BIOS Halt on Missing Drive Count : 255 -------------------------------------------------------- Controller Version Information -------------------------------------------------------- BIOS : 7.5-0 (32106) Firmware : 7.5-0 (32106) Driver : 7.5-0 (41063) Boot Flash : 7.5-0 (32106) CPLD (Load version/ Flash version) : 8/ 10 SEEPROM (Load version/ Flash version) : 1/ 1 -------------------------------------------------------- Controller Cache Backup Unit Information -------------------------------------------------------- Overall Backup Unit Status : Not Present ---------------------------------------------------------------------- Logical device information ---------------------------------------------------------------------- Logical Device number 0 Logical Device name : VOLUME 1 Block Size of member drives : 512 Bytes RAID level : Simple_volume Unique Identifier : A1484B2C Status of Logical Device : Optimal Size : 3814390 MB Parity space : Not Applicable Read-cache setting : Enabled Read-cache status : On Write-cache setting : Enabled Write-cache status : Off Partitioned : Yes Protected by Hot-Spare : Not Applicable Bootable : Yes Failed stripes : No Power settings : Disabled -------------------------------------------------------- Logical Device segment information -------------------------------------------------------- Segment 0 : Present (3815447MB, SATA, HDD, Connector:0, Device:0) Z1Z7W7TR Logical Device number 1 Logical Device name : VOLUME 2 Block Size of member drives : 512 Bytes RAID level : Simple_volume Unique Identifier : 8D2AC9AC Status of Logical Device : Optimal Size : 3814390 MB Parity space : Not Applicable Read-cache setting : Enabled Read-cache status : On Write-cache setting : Enabled Write-cache status : Off Partitioned : Yes Protected by Hot-Spare : Not Applicable Bootable : No Failed stripes : No Power settings : Disabled -------------------------------------------------------- Logical Device segment information -------------------------------------------------------- Segment 0 : Present (3815447MB, SATA, HDD, Connector:0, Device:1) W300M6DL ---------------------------------------------------------------------- Physical Device information ---------------------------------------------------------------------- Device #0 Device is a Hard drive State : Online Block Size : 512 Bytes Supported : Yes Transfer Speed : SATA 6.0 Gb/s Reported Channel,Device(T:L) : 0,0(0:0) Reported Location : Connector 0, Device 0 Vendor : ATA Model : ST4000NM0033-9ZM Firmware : SN04 Serial number : Z1Z7W7TR World-wide name : 5000C5007A858665 Reserved Size : 1072920 KB Used Size : 3814400 MB Unused Size : 64 KB Total Size : 3815447 MB Write Cache : Enabled (write-back) FRU : None S.M.A.R.T. : No S.M.A.R.T. warnings : 0 Power State : Full rpm Supported Power States : Full rpm,Powered off SSD : No Temperature : Not Supported NCQ status : Enabled ---------------------------------------------------------------- Device Phy Information ---------------------------------------------------------------- Phy #0 PHY Identifier : 0 SAS Address : 30000D1107221503 Attached PHY Identifier : 3 Attached SAS Address : 50000D1107221500 Device #1 Device is a Hard drive State : Online Block Size : 512 Bytes Supported : Yes Transfer Speed : SATA 6.0 Gb/s Reported Channel,Device(T:L) : 0,1(1:0) Reported Location : Connector 0, Device 1 Vendor : ATA Model : ST4000DM000-1F21 Firmware : CC52 Serial number : W300M6DL World-wide name : 5000C50069ECC48B Reserved Size : 1072920 KB Used Size : 3814400 MB Unused Size : 64 KB Total Size : 3815447 MB Write Cache : Enabled (write-back) FRU : None S.M.A.R.T. : No S.M.A.R.T. warnings : 0 Power State : Full rpm Supported Power States : Full rpm,Powered off,Reduced rpm SSD : No Temperature : Not Supported NCQ status : Enabled ---------------------------------------------------------------- Device Phy Information ---------------------------------------------------------------- Phy #0 PHY Identifier : 0 SAS Address : 30000D1107221500 Attached PHY Identifier : 0 Attached SAS Address : 50000D1107221500 ---------------------------------------------------------------------- Connector information ---------------------------------------------------------------------- Connector #0 Connector Name : CN0 ------------------------------------------------------------------- Lane Information ------------------------------------------------------------------- Lane #0 Channel ID : 0 Device ID : 0 SAS Address : 50000D1107221500 PHY Identifier : 3 ---------------------------------------------------------------- Lane SAS Phy Information ---------------------------------------------------------------- SAS Address : 50000D1107221500 Attached PHY Identifier : 0 Attached SAS Address : 30000D1107221503 Negotiated Logical Link Rate : PHY enabled - 6 Gbps Lane #1 Channel ID : 0 Device ID : 1 SAS Address : 50000D1107221500 PHY Identifier : 0 ---------------------------------------------------------------- Lane SAS Phy Information ---------------------------------------------------------------- SAS Address : 50000D1107221500 Attached PHY Identifier : 0 Attached SAS Address : 30000D1107221500 Negotiated Logical Link Rate : PHY enabled - 6 Gbps Command completed successfully.

Прошу подсказать команды — как заставить работать собственный кэш контроллера для записи, и как правильно выключить буферы дисков, как я понимаю одновременно два буфера использовать категорически неправильно.

Tert Advanced member Сообщения: 4233 Зарегистрирован: 19 янв 2003, 08:09 Откуда: Москва Контактная информация:

Как добиться максимальной скорости записи SSD на контроллере P420

При настройках по умолчанию скорость чтения SSD хорошая, но вот с записью явно что-то не то.

На этом скриншоте показана скорость 4 дисков SSD Samsung Evo 250GB в нулевом рейде

Побороть эту проблему можно если включить дисковый кеш, который драматически влияет на скорость записи.

​​​​​​​Для того чтобы его включить нужно запустить одну из утилит HP для конфигурования рейдов, в данном случае HP smart storage administrator. Выбрать нужный контроллер из списка и нажать кнопку «Modify controller settings»

В открывшемся окне нужно установить птичку на опции «Physical Drive Write Cache State» -> Enabled

Для применения новых настроек нажать кнопку «Save settings».

После этого скорость записи на том же логическом диске будет значительно быстрее:

Drive cache disabled Drive cache enabled

Тест проводился на сервере HP Proliant DL360 G7 с установленным контроллером HP Smart array P420 c 512MB кеша
​​​​​​​(кеш не используется при работе с SSD но требуется для работы контроллера).

ВАЖНО! Дисковый кэш на пользовательских SSD энергозависим. Если у сервера отключится питания, то содержимое такого кэша пропадет. Поэтому перед включением кэша нужно убедиться, что сервер защищен от аварийных отключений света.

Добавить комментарий

Ваш адрес email не будет опубликован. Обязательные поля помечены *