Shredded Storage in SharePoint 2013 Preview

As you probably know, documents stored in a library or as attachments are stored as binary large objects (BLOBs) in the content database, by default. Remote BLOB Storage (RBS) is a set of APIs that let you move BLOBs out of the SQL Server content database to another storage mechanism. 

You can read whitepaper on BLOB externalization for details. 

In SharePoint 2010, there was opportunity for improvement with both the storage utilization story and I/O performance for documents. In SharePoint 2010, if version history is enabled on a document library, each new version results in a new BLOB for that document. Conceptually, a 1MB file with 10 versions is consuming 10MB of storage. 

What a lot of people don’t consider is that a “new version” doesn’t mean just a change to the document-- it can mean a change to metadata. So if a user changes a metadata field, that is a new version, and a copy of the BLOB is created, even if no change was made to the document itself! 

So BLOBs can proliferate quickly and, to put it bluntly, “pointlessly.” By the way, it’s a best practice to set version retention limits on any library where version history is enabled. 

Second, I/O performance is problematic in SharePoint 2010. There’s an unnecessary file read that occurs when changes—at least to Office documents—are uploaded to the SharePoint web server. 

At the highest level, what SharePoint 2013 shredded storage does is “chunk” or “page” the BLOB into numerous smaller shreds. So a single BLOB is now a construct made up of numerous shreds. 

One result of this architecture is an effect similar to deduplication or single instancing: only differences are saved, not entire BLOBs. So, for example, if you have versioning enabled and a user makes a change to a document, only changed shreds are added to the storage footprint of that document. Shreds that have not changed from the previous version are simply “associated” with both versions. 

You can see significant improvements in storage utilization. That same 1MB file with 10 versions may be consuming 2.2MB of storage, for example. 

Shredded storage also reduces the amount of information about a file that has to be retrieved by the web server from the content database, so I/O improves.

With that conceptual introduction in place, let me punch out a couple of things you need to know, which I’ve found misrepresented in the community:

• Shredded storage is, on the whole, a good thing, and is on by default. 
• You can disable (or re-enable) shredded storage on a per-web application basis. 
• BLOBs are not shredded on an upgrade, but are shredded when uploaded or modified. 
• Shredded storage is SharePoint 2013, running on SQL Server 2008 R2 or SQL Server 2012. 
• Shredded storage is different than Cobalt. Cobalt is a framework that allows Office client applications to efficiently synchronize changes to SharePoint using the File Synchronization via SOAP of HTTP (FSSHTTP) API. Shredded storage is about how a document is shredded, stored, and reassembled by SQL Server. I’m hearing lots of people suggest that shredded storage works only on Office documents. Not true. Such statements are confounding shredded storage with Cobalt. When we look inside a content database on SharePoint 2013, we see PDFs and other file formats being shredded as well. 
• Shredded storage is independent of RBS. You can use RBS with or without shredded storage, and vice versa. Now whether you would use RBS with shredded storage is another question. Folks in the community are currently running tests to determine the performance implications of doing so. My guess is that many of the performance advantages of RBS that I saw with customers, and that both Microsoft and I have documented in white papers, will be reduced or eliminated due to shredded storage. There was also a benefit in SharePoint 2010 to running RBS to store BLOBs on SAN and NAS devices that support deduplication. Shredded storage might very well reduce or eliminate that benefit. However, RBS will continue to be critically important in hierarchical storage management, where you are managing tiers of storage (and therefore cost and other characteristics) based on business rules.

So that’s the “net net” of shredded storage. 

What is still not well documented are the inner workings, so I’ll strive in a future article to detail how it works. Even some of the “official” training and documentation I’ve seen has holes, particularly in relation to the interaction of Cobalt and shredded storage, and how shredded storage does (or does not) help I/O for non-office document formats. 

The good news is you don’t really have to understand how it works, just that it does work. The bottom line is that the SharePoint web server is doing more work, and SQL Server is doing more work, to reduce I/O bottlenecks. Because I/O is likely to be the number-one bottleneck in SharePoint performance, this is all quite desirable. And, along the way, the storage footprint of a document can be reduced—perhaps significantly. 

Each release of SharePoint offers a “feature” that is terribly named (does “shredded storage” sound like a good thing?), poorly documented, and misrepresented in the community. This is one of them for SharePoint 2013. 

Curtsey: www.sharepointpromag.com

SPC2012 - Session List & Additional Training Announced, SharePoint Conference. SharePoint Server 2010. SharePoint 2013.

You've been asking for it, and now it's here! The SharePoint Conference 2012 website just got updated with details of over 250 sessions full of SharePoint goodness (and there's a few more to come). 

Take a look at what you'll be missing if you don't REGISTER NOW!

• In addition to regular sessions we've got a great line up of pre and post conference training opportunities designed to give you a wider breadth of knowledge and add to your conference experience. The highlights include:
• SharePoint 2013 Deployment and Administration End-to-End
• Start Building Apps for Office and SharePoint Today
• Ignite your SharePoint Strategy with Visio Services: A day long technical deep dive
• Business Intelligence
• The Best of the New Project Ignite
• Building SharePoint 2013 Apps using Windows Azure

We're less than 2 months away from what promises to be the biggest SharePoint Conference ever, so don't wait, register now before seats sell out and get ready to 'share more and do more' at SharePoint Conference 2012.

Curtsey: www.sharepointconference.com

SharePoint Governance for the Masses: A Kevin Abel Tip

A comprehensive SharePoint governance plan – samples of which abound on the web – is simply overkill for the vast majority of Phase 1 SharePoint implementations. All too often a company takes one of these samples, edits where appropriate, fills in parameters where needed, and then prints out a tome for the bookshelf. The Governance Plan task is checked off, but woe is the world of the SharePoint platform owner who is questioned about the efficacy of this document.

 

SharePoint governance shouldn’t be all or nothing. It should be relevant. As SharePoint matures and takes hold in an organization, so the governance plan should grow to accommodate these changes. A Phase 1 SharePoint implementation should have a Phase 1 Governance Plan. We call it Governance 50, because time and again, we find that our typical Phase 1 implementations typically generate a Phase 1 Governance Plan with approximately 50 items. What follows is a sampling of the categories and types of items that are often relevant in a Governance 50 plan:

As depicted in the plan above, each governance category has a number of individual governance items. For each item we specify the policy for the specific client along with a frequency of activity, the approach for delivering the governance activity and, where applicable, a reference to a process for the specific governance activity.

Capturing the most important and relevant governance items as an actionable plan in a SharePoint list allows a company to realize SharePoint governance and mitigate the risks of deploying an ungoverned SharePoint platform. As the deployment grows and matures, so too can the governance plan be amended and enhanced as necessary.

Curtsey: Abel Solutions

Microsoft Announces System Center 2012 SP1 and Clarifies the Cloud OS

One of the primary focuses of the System Center (SC) 2012 SP1 release, is the integration with the latest Microsoft server technologies—especially Windows Server 2012 and SQL Server 2012. With the upcoming SP1 release, SC 2012 SP1 will fully support running on the new Windows Server 2012 OS. In addition, internally, SC SP1 will also support the use of SQL Server 2012 for its own database requirements. Not surprisingly, considering that the System Center 2012 Suite consists of multiple products, there’s way too many new features in the SP1 release to list in this brief newsletter. Some of the standout features in the SC 2012 SP1 release include System Center Configuration Manager (SCCM) 2012 SP1’s ability to deploy and manage Windows 8 and Windows Server 2012, the use of Windows Azure-based software distribution points to reduce infrastructure costs, and Service Manager (SM) 2012 SP1’s more granular chargeback information.

Virtual Machine Manager (VMM) 2012 SP1 provides improved multi-tenancy for hosting organizations and Windows Azure IaaS integration, with the ability to move on-premises virtual machines (VMs) to Windows Azure. Data Protection Manager (DPM) 2012 SP1 adds the option to provide on-line server backups to the Windows Azure cloud. However, considering the growing importance of the private cloud, I think the most important new feature in the SC SP1 announcement was VMM 2012 SP1’s enhanced virtual networking capability that Microsoft is calling Hyper-V Network Virtualization. This feature is similar to VMware’s Distributed Network Switch and it enables software defined networking (SDN) for the virtual infrastructure. With VMM 2012 SP1’s new SDN capabilities, the policies that define a Hyper-V VM’s networking characteristics are moved with the VM as it is migrated live between Hyper-V hosts. 

To help clear away the cloud around the Cloud OS, Microsoft wants to make it clear that when they refer to the Cloud OS they are not referring to Windows Server 2012. Although the two are often discussed together, Windows Server 2012 is more accurately referred to as the cornerstone of the Cloud OS. The Cloud OS is really a term describing a combination of Microsoft technologies. Satya Nadella, President, Server & Tools Business for Microsoft, in his blog post characterized Microsoft’s vision of the Cloud OS as “The Cloud OS does what operating systems have always done: manage hardware and provide a platform for applications. But it also expands to include services and technologies that have not previously been considered part of an operating system.” The Cloud OS concept describes a datacenter without physical boundaries that transcends today’s physical datacenters. Windows Server 2012 and Windows Azure deliver the core of the Cloud OS, while System Center 2012 provides a consistent management experience.

You can find out more about the new features in the SC 2012 SP1 release at What’s New in System Center 2012 SP1.  

Microsoft’s SC SP1 is currently in beta and you can download it and try it for yourself from the Microsoft Download Center. 

Curtsey: www.windowsitpro.com

Windows Azure for SharePoint: Combining SharePoint with Windows Azure.

Sick of constantly hassling people to clean up files on the server? Want more compute resource at peak times but don’t want the added overhead? Combine your SharePoint with Windows Azure and these problems, and more besides, are solved. Azure can handle your document storage, as well as holding huge amounts of SharePoint application data, all within the cloud.

Azure’s dynamically scalable hosting potential can bring practical service-based scenarios into SharePoint. And it’s not just added space that Azure offers; it’s also ease of use. You can create customer-facing interfaces in Azure, that can be easily integrated into SharePoint’s architecture, and your developers can continue to leverage their skills in Microsoft Visual Studio, Microsoft .NET, Java or other development environments.

Azure’s simple pay-as-you-go pricing model also streamlines budgeting and removes the need for capital expenditure on upfront purchases, provisioning and management of additional server infrastructure. In short, Windows Azure is about simplicity: creating an easier life for you, while improving the performance of your current SharePoint system.

An overview of Windows Azure

Azure is already improving businesses through the following key features and benefits:

• Simplified operations and maintenance of applications – by providing on-demand scalability of both compute and storage. 
• Reduced need for up-front purchases – with a pay-as-you-go pricing model, budgeted from Opex, not Capex.
• Increased compatibility – providing an open environment that supports multiple internet protocols, including HTTP/HTTPS, REST, SOAP, and XML. This enables your developers to easily create cloud-based applications in their preferred scripting language without further training.
• Responsive management – Azure handles load balancing and resource management automatically, giving users all the bandwidth and storage they need, the instant they need it.

StorSimple solved their forever increasing storage issues with Windows Azure.

Start-up software vendor StorSimple is using Windows Azure to solve the storage issues of companies with high-growth applications. StorSimple’s hybrid solution seamlessly integrates cloud storage services for Microsoft Exchange, SharePoint and Windows Server with on-premises servers to deliver a scalable, on-demand cloud storage model.

Read more 

Who’s using Windows Azure with SharePoint?

Point8020 - Learning content provider Point8020 uses Windows Azure to enhance the delivery of expert-led instructional videos that help businesses make the most of SharePoint.
Read more

“Our customers already have an investment in Microsoft technology. I’m sure they will be happy knowing our product is based on Windows Azure. It gives them confidence that the services will be reliable, available, and offer great performance.”

                                        Martin Harwar, Chief Executive Officer, Point8020

Hitachi - Global IT consulting firm Hitachi Consulting is streamlining reporting for U.S. Government contractors with their Program Management Framework combining the Microsoft SQL Azure cloud database service with Microsoft SharePoint Server 2010.

Read more 

“With SQL Azure and our solution, we anticipate that a typical client will save at least $200,000 annually in hardware and licensing costs alone. Plus, the reduced administrative burden of SQL Azure and Windows Azure will allow our clients to refocus scarce technical resources on higher value activities.”

Rob Rae, Director Microsoft Platform Practice, Hitachi Consulting

Curtsey: Microsoft Corp.