Got a 3Par training going on at work since today. I have no experience with 3Pars so this is my first encounter with it really. I know a bit about storage thanks to having worked with HP LeftHands (now called HP StoreVirtual) and Windows Server 2012 Storage Spaces – but 3Par is a whole another beast!
Will post more notes on today’s material tomorrow (hopefully!) but here’s some bits and pieces before I go to sleep:
- You have disks.
- These disks are in magazines (in the 10000 series) – up to 4 per magazines if I remember correctly.
- The magazines are then put into cages. 10 magazines per cage?
- Disks can also be in cages directly (in the non-10000 series, such as the 7200 and 7400 series).
- Note: I could be getting the model numbers and figures wrong so take these with a pinch of salt!
- Important thing to remember is you have disks and these disks are in cages (either directly or as part of magazines).
- Let’s focus on the 7200/ 7400 series for now.
- A cage is a 2U enclosure.
- In the front you have the disks (directly, no magazines here).
- In the rear you have two nodes.
- Yes, two nodes! 3Pars always work in pairs. So you need two nodes.
- 7200 series can only have two nodes max/ min.
- 7400 series can have two nodes min, four nodes max.
- So it’s better to get a 7400 series even if you only want two nodes now as you can upgrade later on. With a 7200 series you are stuck with what you get.
- 7200 series is still in the market coz it’s a bit cheaper. That’s coz it also has lower specs (coz it’s never going to do as much as a 7400 series).
- What else? Oh yeah, drive shelves. (Not sure if I am getting the term correct here).
- Drive shelves are simply cages with only drives in them. No nodes!
- There are limits on how many shelves a node can control.
- 7200 series has the lowest limit.
- Followed by a two node 7400 series.
- Followed by a four node 7400 series. This dude has the most!
- A four node 7400 is 4Us of nodes (on the rear side).
- The rest of the 42U (rack size) minus 4U (node+disks) = 38U is all drive shelves!
- Number of drives in the shelf varies if I remember correctly. As in you can have larger size drives (physical size and storage) so there’s less per shelf.
- Or you could have more drives but smaller size/ lower storage.
- Sorry I don’t have clear numbers here! Most of this is from memory. Must get the slides and plug in more details later.
- Speaking of nodes, these are the brains behind the operation.
- A node contains an ASIC (Application Specific Integrated Circuit). Basically a chip that’s designed for a specific task. Cisco routers have ASICs. Brocade switches have ASICs. Many things have ASICs in them.
- A node contains a regular CPU – for management tasks – and also an ASIC. The ASIC does all the 3Par stuff. Like deduplication, handing metadata, optimizing traffic and writes (it skips zeroes when writing/ sending data – is a big deal).
- The ASIC and one more thing (TPxx – 3Par xx are the two 3Par innovations). Plus the fact that everything’s built for storage, unlike a LeftHand which is just a Proliant Server.
- Caching is a big deal with 3Pars.
- You have write caching. Which means whenever the 3Par is given a blob of data, the node that receives it (1) stores it in its cache, (2) sends to its partner, and (3) tells whoever gave it the blob that the data is now written. Note that in reality the data is only in the cache; but since both nodes have it now in their cache it can be considered somewhat safe, and so rather than wait for the disks to write data and reply with a success, the node assures whoever gave it the data that the data is written to disk.
- Write caching obviously improves performance tremendously! So it’s a big deal. 7400 series have larger caches than 7200.
- This also means if one node in a two-pair is down, caching won’t happen – coz now the remaining node can’t simply lie that the data is written. What happens if it too fails? There is no other node with a cached copy. So before the node replies with a confirmation it must actually write the data to disk. Hence if one node fails performance is affected.
- And then you have read caching. When data is read additional data around it is read-ahead. This improves performance if this additional data is required next. (If required then it’s a cache hit, else a cache miss).
- Caching is also involved in de-duplication.
- Oh, de-duplication only happens for SSDs.
- And it doesn’t happen if you want Adaptive Optimization (whatever that is).
- Oh, de-duplication only happens for SSDs.
- You have write caching. Which means whenever the 3Par is given a blob of data, the node that receives it (1) stores it in its cache, (2) sends to its partner, and (3) tells whoever gave it the blob that the data is now written. Note that in reality the data is only in the cache; but since both nodes have it now in their cache it can be considered somewhat safe, and so rather than wait for the disks to write data and reply with a success, the node assures whoever gave it the data that the data is written to disk.
- There is remote copying – which is like SRM for VMware. You can have data being replicated from one 3Par system to another. And this can happen between the various models.
- Speaking of which, all 3Par models have the same OS etc. So that’s why you can do copying and such easily. And manage via a common UI.
- There’s a GUI. And a command line interface (CLI). The CLI commands are also available in a regular command prompt. And there’s PowerShell cmdlets now in beta testing?
- Data is stored in chunklets. These are 1GB units. Spread across disks.
- There’s something called a CPG (Common Provisioning Group). This is something like a template or a definition.
- You define a CPG that says (for instance) you want RAID1 (mirroring) with 4 sets (i.e. 4 copies of the data) making use of a particular set of physical disks.
- Then you create Logical Disks (LDs) based on a CPG. You can have multiple logical disks – based on the same or different CPGs.
- Finally you create volumes on these Logical Disks. These are what you export to hosts via iSCSI & friends.
- We are not done yet! :) There’s also regions. These are 128 MB blocks of data. 8 x 128 MB = 1024 MB (1 GB) = size of a chunklet. So a chunklet has 8 regions.
- A region is per disk. So when I said above that a chunklet is spead across disks, what I really meant is that a chunklet is made up of 8 regions and each region is on a particular disk. That way a chunklet is spread across multiple disks. (Tada!)
- A chunklet is to a physical disk what a region is to a logical disk.
- So while a chunklet is 1 GB and doesn’t mean much else, a region has properties. If a logical disk is RAID1, the region has a twin with a copy of the same data. (Or does it? I am not sure really!)
- And lastly … we have steps! This is 128 KB (by default for a certain class of disks, it varies for different classes, exact number doesn’t matter much).
- Here’s what happens: when the 3Par gets some data to be written, it writes 128 KB (a step size) to one region, then 128 KB to another region, and so on. This way the data is spread across many regions.
- And somehow that ties in to chunklets and how 3Par is so cool and everything is redundant and super performant etc. Now that I think back I am not really sure what the step does. It was clear to me after I asked the instructor many questions and he explained it well – but now I forget it already! Sigh.
Some day soon I will update this post or write a new one that explains all these better. But for now this will have to suffice. Baby steps!