This time around I got the storage spaces bug and built my system around doing a tiered storage space to accommodate my VMware cluster backup, file sharing DFS replica and miscellaneous hobby data (media libraries, general clutter, etc).
I'm using a single 500GB Samsung 850 Pro for my boot disk, 2x 850 Pros for my SSD tier and 2x WD Red NAS 6TB for my HDD tier. Everything is connected via the Z97 SATA ports on my ASRock Z97 Extreme6 board (i4790 i7 CPU). I picked this board because it has 10 SATA ports, 4 from an ASMedia chip to go with the 6 native Z97 chipset ports.
There's lots to complain about my setup, from the desktop board to questions about bottlenecking SATA, inadequate SSD tier size to my use of the CPU graphics. Money ultimately has its say.
I'm fooling around to determine my storage strategy to maximize my utilization of SSD while having a lot of storage. Which I think boils down to determining what doesn't ever really need SSD tiering and putting it in a minimally tiered virtual disks so as to maximize other virtual disks with tiering.
Some crazy but not entirely so options include using iSCSI targets as loopback disks -- these can be unmounted and moved between virtual disks with different tiering allocations or moved to other storage locations completely, including to other Windows iSCSI storage servers.
Now for the things about spaces that bugs me:
- Pools shouldn't be limited to locally connected physical disks.
- Why leave out iSCSI disks? It seems not too crazy to be able to use spaces against an iSCSI SAN as volume. A pool with 8 iSCSI disks comprised of N volumes spread over X SANs provides pretty deep redundancy from failure. Each pool disk will have potentially hundreds to tens of thousands of IOPS (SSD) and gigs of write-back caching. Older SAN storage can be combined transparently to provide dev or archival storage. Latency may be a problem, but I hate when stuff is excluded because it may have some performance limitations but otherwise is functional.
NAS4Free will actually let you do this -- serve out via iSCSI a target comprised of a volume built from iSCSI mounts from other servers. - Maybe even allow for .vdhx virtual disk files to be used as pool members. This would allow a pool to be created with a logical structure that didn't need to align with a physical disk structure (you could build a double-parity virtual disk spread over .vdxh members located in any arbitrary location -- one disk, two disks, NAS volumes. A three SSD physical disk combo probably has enough throughput to host a double parity virtual disk for a lot of workloads. It might make for interesting whole-virtual disk backups that could fit on a small number of max capacity hard disks.
- This would also allow for nested pools. If you have 4 6 TB disks you could do a 12 TB two column mirror disk. If you could create 12 .vhdx files of 1 TB each and use them as disks for a pool, you could create a double parity pool. While it sounds less than useful, it would allow pools to be staged or moved across storage devices as physical storage devices became available.
- Why can't you move virtual disks between pools?
- This makes no sense. If I built a storage space server with a 12 disk, SAS 8 controller shelf and later wanted to migrate to a 24 disk SAS 12 controller with a different physical disk arrangement (SSD/HDD ratio, disk size, whatever) logic dictates that I would just create a new pool with the new shelf and then migrate my virtual disks to the new pool. Equallogic works something like this -- add a new shelf to a group, add it to a unique pool and you can move volumes between pools.
I think there is a more complicated way to do this command line, adding disks to the pool and then removing disks which accomplishes the same thing, but less elegantly and less selectively than migrating virtual disks especially if your intent was to, say, migrate a virtual disk to a new SSD based pool but not replace the existing pool.
- Why can't virtual disks change redundancies on the fly (provided adequate pool resources exist)? A pool with 10 TB free should allow me to convert a 1 TB stripe to a mirror or parity or vice versa.
- Why is the GUI so retarded? This is typical of MS of late -- GUIs are stripped down to tablet-levels of sophistication. They mostly do the sane defaults, but lack almost all the flexibility of the PowerShell commands. The PowerShell is nice, but PowerShell command documentation for spaces is there, but it's like placeholder language with few explanations or much detail.
Worse, monitoring of tiering is sketchy at best. There are a handful of real time WMI counters. Tiering reports are 1990's style text tables only generated if you modify the tiering scheduled task. I haven't found a way to get reporting on tier usage per file.
- Is tiering tunable? "Hot" data would seem to be data that is accessed frequently, but over what time scale? Since the last tiering analysis, or over some longer time span? Does it make sense to prefer a file accessed regularly over weeks or months over one busy only yesterday? I assume that access data is based on the delta between tuning runs, but it's not really documented.
- Could dedupe be a feature at the pool level? Maybe with a feature that allows SSDs to be dedicated pool chunk store. Might allow for greater pool storage efficiency, especially if all virtual disks could now dedupe each other.
- How do dedupe and tiering work for/against each other?
- They would seem to have some synergy. In theory, deduplicated data would tend to be hotter because it represents common chunks that span multiple files, leading to it being moved up a tier. This would speed up dedupe reads by reducing chunk assembly latency, as well as multiplying the benefit over multiple deduplicated files which share chunks. Some non-hot files gain a tiering advantage because they have hot deduplicated chunks that get tiered.
- But on the other hand, does the act of deduplicating files end up polluting the SSD tier with deduplication metadata that's hot only because of the disk intensive nature deduplication and not because the data itself is hot from regular (ie, not deduplication activity) access? Tiering may end up making deduplication activity more efficient, but reduce the value of tiering for normal file access.
- Columns are allocated at maximum possible column width when virtual disk are created, meaning drives have to be added to a pool at column numbers of drives. Would like to see a way to re-stripe columns to other column sizes if space permits. This would allow for pools to be expanded in a smaller granular fashion and allow re-columned disks to also be expanded to meet the pool size multiples. Shrinking a virtual disk column count reduces performance, but the cost may be negated by the effectiveness of tiering.
- Adding drives should be more flexible, such as re-balancing virtual disks over physical disks.
I'm sure I'll think of more things to come.
