How Many Cats Can You Fit in an Igloo?

As more and more data is created, not just by business, science and education, but by individuals globally, data storage is increasingly becoming an essential part of our everyday lives.

At CERN, the European Organization for Nuclear Research, around 25GB of data is collected every second when they run their collective experiments; that's the equivalent to one whole Blu-ray disk - every second.

Compare this to the first computer with what we would consider a hard drive, IBM 305 RAMAC. Launched in September 1956, it weighed over a ton and provided only 5MB of capacity.

Uploading your latest selfie, cat picture or hipster style food photo to the RAMAC would have taken up most of the space, and likely most of the day to complete.

Now, as an individual reading this you may be thinking, this is no challenge today though; after all, how much storage does taking a few pictures of kittens really consume?

In the grand scheme of things, one image is pretty insignificant, but look at the bigger picture: mobile phones, laptops, tablets, smart home devices, and smart watches all generate and store data locally and often a backup in the cloud. One picture taken on a high resolution camera can be serval megabytes, perhaps as much as 10. Videos are exponentially larger too, particularly with the dawn of 4K capture in your pocket. This means that just a few high resolution snaps of your furry friend could cost you hundreds of megabytes.

The rise of the smart phone and mobile eco systems such as IOS/Android has essentially fuelled this online storage revolution. Facebook, Snapchat and Instagram were built on these platforms to create new innovative ways of sharing moments with friends and family, but with so many moments it's not just the users that are overwhelmed with the sheer volume.

For our personal and more private files, Dropbox and Google Drive have become much more popular as a way of accessing your data on the move. You can use it to share private or large files with friends and colleagues, as extra storage capacity for your phone or as a complete device backup. As a result, most modern tech users will have an extensive online data profile which only ever increases in total every day.

How your extensive personal online data profile is supported on the back end of the cloud is probably of little interest to you, so long as it remains ready for you as you demand it right?

For the providers of these storage services though, this represents a complex problem, how to match capacity for exponential data growth while making it quick to access, highly available and more cost effective than ever before?

Further highlighting this challenge, according to International Data Corp. (IDC), 4.4 zettabytes (4.4trn gigabytes) of digital data had already been created by 2013. The Industrial Internet is destined to boost that total to 44 zettabytes (ZB) by 2020.

That's a lot of cat pictures, or if you prefer -

Let's face it, perhaps your cat pic from 2012 isn't that important any more... to you or your service provider...

A selfie uploaded to Facebook within the last 24 hours will get a relatively high number of hits and be accessed more regularly than the picture from 2012, that is of course excluding Grumpy Cat who remains a worldwide phenomenon.

This is where the concept of storage tiering can be used effectively, for example, data that has been accessed recently and is being requested regularly would be considered 'hot'; as it's needed to hand often. Your tiering policy can also be tailored to your specific requirements, so 'hot' might be considered as data that has been requested anything from once an hour to once a day.

'Warm' data is the middle-man between hot and cold and might be considered to be accessed from daily all the way up to monthly.

Finally, 'Cold' data is very infrequently accessed and might not be required ever again. Cold data will effectively be everything outside of hot and warm, and if it's not stored at this level then it would likely be deleted.

This is a very simple categorisation, which is also dynamic in its nature too. For example, if data in cold storage is accessed once or twice in succession it would likely be promoted up to the next tier. If access continues it may even, make it back into the hot tier.

Now having explained the high level view of what storing tiering is, why does it matter?

Storage devices come in many sizes, performance levels and hugely different costs. Generally, the faster the performance and the more data that can be stored, the higher the price tag.

Flash storage (Solid State Drives) for example, are fast both in transfer speed and through low response times, but have smaller capacity in general and come at a premium cost over standard hard disk drives. In comparison, a 1TB hard disk drive is around £0.13 per GB, whereas a 1TB SSD is around £0.59 per GB.

As you can see for pure capacity HDD's win on cost, but in another very important factor, performance, SSD's are far superior.

Each technology on its own would either be very expensive or would not provide enough performance to satisfy the requirements alone. Therefore, using both types simultaneously in storage tiers can help provide you with the most available space for the lowest cost.

How does the above apply to you as an individual? You may be thinking at this stage what does it matter? Surely a few pictures cannot require high performance storage? Whilst true, remember you are not alone in this world. The Snapchat service has 150 million active users every day and each one of them expects instant access to their selfies.

Hopefully we have caught your attention on how effective tiering your storage can be to satisfy both performance and capacity.

At Boston we help our customers deal with the challenges of data storage on a daily basis. Whether the solution is one of our Igloo appliances that mix of RAM, SSD and HDD to deliver performance and capacity, or if it's a scale out solution; we have options from a wide range of accredited industry platforms which are tailorable to meet the most demanding requirements.

Some examples from our Igloo range which are perfect for storage tiering are below, but feel free to contact us any time to get our insights and feedback on your particular pain points.

Hot tier storage: Boston Igloo Plus

In partnership with AccelStor, the software-defined all-flash array provider, Boston brings to the market an ultra-high performance flash appliance, the Igloo Plus. A 1U rack-mount storage solution featuring up to 500K sustained IOPS performance with 5TB usable storage capacity and high-speed, 10GbE, InfiniBand or 16Gbit/s Fibre Channel connectivity.

Storm (with IB FDR interface) is a good fit for high-performance computing (HPC) applications, and Hurricane is a good fit for enterprises with existing Fibre Channel install base (e.g., with SAN switches). Tornado and Lightening come with mainstream, cost-effective 10GbE interfaces and are applicable to generic applications, including those commonly seen in enterprises/datacentres, and even for building (niche) industry solutions.

Warm-tier Storage: Boston Igloo 2U-19T-Stor

Boston's 2U-19T-Stor provides big capacity and fast, affordable storage. It is an innovative Hybrid Storage System which fuses the capacity of HDDs with the performance of SSDs in a single solution that offers high performance while lowering cost. Allowing you to target those mixed workloads between hot and cold. Additionally, by leveraging capacity optimization technologies and advanced tiered SSD and RAM caching, Boston Igloo 2U-19T-Stor provides an overall efficiency boost and increased cache performance.

Cold-tier Storage: Boston Igloo 2U-24T-Stor

Using the very latest storage operating system by Microsoft, the Boston Igloo 2U-24T-Stor is both a NAS and an iSCSI SAN appliance.

Windows Storage Server 2012 offers a wide and varied range of file serving and storage capabilities. It addresses heterogeneous environments, enables more efficient storage utilization and is delivered as a turnkey, file-based storage solution. It provides unified storage with support for both file (SMB and NFS) and block (ISCSI) protocols, giving flexibility as well as enabling efficient storage utilization with thin provisioning and data de-duplication technologies.

Multi-tier Storage solution: BOSTON DATASCALER-L SFF

Boston's dataScaler-L SFF (small form factor) appliance is the latest addition to Boston's Lustre based parallel file system portfolio and provides a lower cost / capacity entry point for Lustre storage systems. Traditional Lustre solutions on the market are focused on addressing the high end cluster users, while the Boston Lustre SFF is designed for small or mid-sized clusters with capacities as low as 18TB (and scaling to 720TB). This solution provides a high performance alternative to standard NFS and provides customers with the option to scale and grow as the cluster and user demands increase over time.

To help with your storage challenges, contact our sales team today to book a consultation with no cost or obligation.

So how many cats can you fit in an igloo? - Assuming that your high resolution cat photo takes 10mb of storage, you could fit 500,000 pictures of cats on your Boston Igloo Plus Storm... that's a lot of cats.

Links/reference:

http://www.techradar.com/news/computing-components/storage/the-data-capacity-gap-why-the-world-is-running-out-of-data-storage-1284024/3#

https://www.weforum.org/agenda/2015/11/are-we-running-out-of-data-storage-space/

http://searchstorage.techtarget.com/opinion/Whats-the-future-of-data-storage-in-2016

http://home.cern/about/computing/processing-what-record

http://www.ibmbigdatahub.com/blog/your-big-data-hot-warm-or-cold

http://www.ebuyer.com/500796-seagate-constellation-2tb-3-5-sata-enterprise-hard-drive-st2000nm0033 £128.99

http://www.ebuyer.com/718382-samsung-850-evo-2tb-2-5inch-ssd-mz-75e2t0b-eu £558.99

http://home.cern/sites/home.web.cern.ch/files/styles/medium/public/image/about_section_page/2013/01/cern-servers.jpg?itok=MyedI_8q

http://fortune.com/2016/04/13/facebook-poaches-google-tech-wiz/

https://storageservers.wordpress.com/2013/07/17/facts-and-stats-of-worlds-largest-data-centers/

http://thenextweb.com/shareables/2011/12/26/this-is-what-a-5mb-hard-drive-looked-like-is-1956-required-a-forklift/#gref