Recently, there’s been a lot of talk about built-in defragging systems. Is Windows®7 the best option? In the latest issue of Processor Magazine, experts weigh in, making the case for Diskeeper’s optimization in the enterprise. Read the whole article here: http://www.processor.com/articles//P3402/11p02/11p02.pdf?guid

In today’s well connected world of electronics and instant communications I received a text from a friend asking if I had seen the recent PC World magazine (February, 2012).  He said it had some tidbit of information concerning one of my favorite subjects; system performance, defragmentation, and SSDs.  I located a copy here at the office and found the article. As I read the first line I realized the debate on the virtues of defragmentation especially on SSDs will be one that goes on indefinitely as no one really talks about the issue with supporting hard facts and numbers.  Most articles are rehashing ideas and opinions long since debunked.  They continue to surface because very few truly understand the intricacies of the Windows NTFS file system and that of the storage media, whether it is rotating magnetic hard disks or electronic solid state disks.

So let’s set the record straight… Fragmentation is exponentially more of a problem with today’s data explosion. Defragmenting once a week will still cause the user to experience slowdowns from the degradation effects and doesn’t address the issue when files are initially being written.  And yes, never do a traditional defrag on SSDs.

NTFS file and free space fragmentation happens far more frequently than you might guess.  It has the potential to happen as soon as you install the operating system.  It can happen when you install applications or system updates, access the internet, download and save photos, create e-mail, office documents, etc…  It is a normal occurrence and behavior of the computer system, but does have a negative effect on over all application and system performance.  As fragmentation happens the computer system and underlying storage is performing more work than necessary.  Each I/O request takes a measurable amount of time.  Even in SSD environments there is no such thing as an “instant” I/O request.  Any time an application requests to read or write data and that request is split into additional I/O requests it causes more work to be done.   This extra work causes a delay right at that very moment in time.  Whoever thought that defragmenting once a month or weekly was good enough, simply didn’t understand fragmentation.

Disk drives have gotten faster over the years, but so have CPUs.  In fact, the gap between the difference in speed between hard disks and CPU has actually widened.  This means that applications can get plenty of CPU cycles, but they are still starving to get the data from the storage.  What’s more, the amount of data that is being stored has increased dramatically.  Just think of all those digital photos taken and shared over the holidays.  Each photo use to be approximately 1MB in size, now they are exceeding 15MB per photo and some go way beyond that.  Video editing and rendering and storage of digital movies have also become quite popular and as a result applications are manipulating hundreds of Gigabytes of data.  With typical disk cluster sizes of 4k, a 15MB size file could potentially be fragmented into nearly 4,000 extents.  This means an extra 4,000 disk I/O requests are required to read or write the file.  No matter what type of storage, it will simply take longer to complete the operation.

Suppose I chose to do some editing of my family videos on Tuesday evening.  Even the built-in defragmentation tool in Windows 7 doesn’t do me much good because it isn’t schedule to run until Wednesday morning at 1:00am.  This also means that quite a bit of fragmentation has built up since the previous week when it last ran.  Maybe I’ll manually run it, but that can take quite a while and I’ve wasted time that I would have rather spent on my project.  Unfortunately, the Windows built-in defragmentation utility doesn’t prevent fragmentation so even after running it manually; I still will wind up with fragmentation and slow access speed of my newly created files. 

I’ve often thought about why Wednesday at 1:00am was chosen as the time to schedule defragmentation.  Why isn’t it scheduled all the time?   It is because there could be system resource conflicts that either interfere with getting the task done or the defragmentation process has difficulty throttling back under a variety of conditions.  Regardless, this wait a week to clean up fragmentation doesn’t really help me when I need it most.

As pointed out in the article, the built-in defragmenter does not have the technology advancement to properly deal with fragmentation and SSDs. The physical placement of data on an SSD doesn’t really matter like it does on regular magnetic HDDs.  With an SSD there is no rotational latency or seek time to contend with.  Many experts assume that fragmentation is no longer a problem, but the application data access speed isn’t just defined in those terms.  Each and every I/O request performed takes a measurable amount of time.  SSD’s are fast, but they are not instantaneous.  Windows NTFS file system does not behave any differently because the underlying storage is an SSD vs. HDD and therefore fragmentation still occurs.  Reducing the unnecessary I/O’s by preventing and eradicating the fragmentation reduces the number of I/O requests and as a result speeds up application data response time and improve the overall lifespan of the SSD.  In essence, this makes for more sequential I/O operations which is generally faster and outperforms random writes.

In addition, SSD’s require that old data be erased before new data is written over it, rather than just writing over the old information as with HDDs.  This doubles the wear and tear and can cause major issues with the speed performance and lifespan of the SSD.  Most SSD manufactures have very sophisticated wear-leveling technologies to help with this. The principle issue is write speed degradation due to free space fragmentation.  Small free spaces scattered across the SSD causes the NTFS file system to write a file in fragmented pieces to those small available free spaces.  This has the effect of causing more random I/O traffic that is slower than sequential operations.

I think I have clearly made my point….  The built-in defragmenter in Windows 7 is not a solution for neither the consumer/home user, nor the enterprise business user.  Data access speeds are far more critical in the business world where time is money.  In the enterprise environment there are generally many more files that are used by higher number of users that are accessing data across shared type of storage such as SAN.  Even virtual platforms benefit from the same points covered.  This opens the door and is the reason why robust solutions such as Diskeeper exist.  More data about Diskeeper and the superior technology it offers can be found at http://www.diskeeper.com.

I felt compelled to write a little bit about this subject after reading recently about some new updates to software SANs. The glamour of the virtual platform layers and Cloud have somewhat overshadowed all of the virtualization already occurring within storage, and the extra levels that are added “below decks”. It’s a topic meriting some scrutiny from any storage administrator committed to high performance.

Outside of the physical data store itself, every element of the I/O path above it is virtual. It should also be noted that at essentially each step along this I/O path, infrastructure customization and proprietary technologies can (and often do) vary or add new virtual layers to the process. All of these logical abstractions have evolved from various sources in the storage ecosystem in order to drive scalability and agile responses to disaster or growth.

Let’s consider a common hypothetical path that an I/O request takes from a Windows client VM to the physical data store in a modern infrastructure. In this example, the storage for the client VM is a virtual RAID 5 configured of LUNs from a SAN. An I/O request originating at the top OS level, Windows in this case, will go through these underlying levels before getting to the actual physical storage device. Windows > volume manager > virtual RAID > SAN LUN > physical store (with the possibility of additional abstraction levels based on storage customization).

Based upon how the storage infrastructure has been established in this scenario, there is a virtual RAID 5 implemented above the SAN LUN layer. That being the case, the volume manager directs the request to the virtual RAID beneath it. Due to Striping, I/O at this stage can end up fractured (intentionally) by the RAID, based on how the array has been provisioned. The I/O Path has now become distributed, and may be even further replicated on its way to physical storage.

The RAID sends its request to the SAN LUN below, another abstraction from the physical storage itself slicing the store into basic logical units. The SAN LUN layer completes the request directly to the physical storage. The data is then returned along the same route to the requester.

Now, numerous solutions exist for managing communication and throughput within this data pipeline. Administrators can tailor their RAID presentation, ensure partition alignment, upgrade the underlying hardware, even add new software abstraction layers intended to organize data better at lower levels. However, an interesting concept emerges after review.

None of these solutions handle the most basic, and one of the most critical vulnerabilities in the existing ecosystem’s performance: assuring that the file request is as sequential and rapid as possible at the point of origin. Whether virtualized as is so common today or installed over direct-attached storage, Windows Read and Write performance is degraded by file and free space fragmentation at this top level as it causes more I/O requests to occur. Each request through all of the abstraction layers greets its first bottleneck at the outset, in how contiguous the file arrangement is within Windows. Optimizing reads and writes at this upper level helps ensure in most cases the fastest I/O path no matter how much or how little storage abstraction has been structured beneath.

In a recent white paper, Diskeeper Corporation tested a variety of I/O metrics over SAN storage with and without file fragmentation being intelligently prevented and handled. In one such test, Iometer (an open source I/O measurement tool) displayed over a 200% improvement in IOPS (I/Os per second) after Diskeeper 2011 had handled Windows volume fragmentation. Testing was performed on a SAN connected to a Windows Server 2008 R2 virtual host:

You can read the entire white paper here: http://downloads.diskeeper.com/pdf/improve-san-performance.pdf 

Thin provisioning is a fairly hot topic in the storage arena, and with good reason. Many zones within the business and enterprise see massive benefit from the scalability of thin provisioning, and it can be a cost saver besides. However, the principle of thin provisioning suffers some unique maladies at both client and storage levels.

Some storage arrays include a feature permitting thin provisioning for their LUNs. This storage layer thin provisioning occurs below the virtual platform storage stack, and essentially means scalable datastores. Horizontal scaling of data stores adds a new tier of agility to the storage ecosystem that some businesses absolutely require.

LUN thin provisioning shouldn’t be confused with Virtual Disk TP, which works at a file level (not array). Thin provisioned VMs can expand based on pre-determined use cases, adding an extra degree of flexibility to storage density. Intelligently combining TP at multiple tiers yields some pretty neat capacity results.

Datastore thin provisioning has been the source of some concern for storage administrators with regards to recovery from over-provisioning. When virtual disks are deleted or copied away from a datastore, the array itself is not led to understand that those storage blocks are now free. You can see how this can lead to needless storage consumption.

vSphere 5 from VMware introduced a solution for this issue. The new vSphere Storage APIs for Array Integration (VAAI) for TP uses the SCSI UNMAP command to tell the storage array that space previously occupied by a VM can be reclaimed. This addresses one aspect of the issue with thin VM growth.

Files are not simply being written to a virtual disk, they’re also deleted with regularity. Unfortunately, there is no associated feature within virtual platforms or Windows to inform the storage array that blocks can be recovered from a thin disk which should have contracted after deletions. Similar to the issue above, this leads to unnecessary storage waste.

With the release of V-locity 3 in 2011, we introduced a new Automatic Space Reclamation engine. This engine automatically zeroes out “dead” free space within thin virtual disks, without requiring that they be taken offline and with no impact on resource usage. So what does this mean? Thin VMs can be compacted, actually reclaiming the deleted space to the storage array for dynamic use elsewhere. The thin virtual disks themselves are kept slimmed down within datastores, giving more control back to the storage admins governing provisioning.

You can read more about VAAI for TP in vSphere 5 on the VMware blog here.

We’ve just returned from the Interop Expo in New York, and what a show! The recent release of V-locity® 3 was extremely well received and interest in its innovations was very high. The Diskeeper Corporation booth was constantly attended by groups of CIOs and storage administrators eager to hear about the benefits of the new virtual platform optimizer.

The lion’s share of energy and buzz at the show surrounded virtualization and cloud computing. Leading vendors across these markets as well as storage, networking, and information security exhibited for large groups of virtual admins and IT executives. Shows like Interop are critical for decision makers to stay apprised of the ever-evolving IT infrastructure landscape, and excellent opportunities to get educated about what is truly needed to grow and maintain a virtual environment that runs on all engines for them.

In addition to being asked by numerous IT analysts about the innovations underlying the incredible advantages of V-locity 3, I was interviewed by TMC (Technology Marketing Corporation) about it.

The need to meet higher Service Level Agreements and reduce Total Cost of Ownership for shared storage have reached a new plateau in virtualized networks and private clouds—what V-locity 3 does best.

If you’re reading this and you were at the event, we’d love to hear about your experiences at Interop this year.

Diskeeper Corporation will be exhibiting at the Gartner Symposium in Orlando, FL next week. If you’re planning on attending this IT Expo, stop by the booth to hear firsthand about how V-locity 3 is improving virtual systems in a whole new way.

Here's what's new in V-locity 3.0.9 (update) which will be broadly available next week: 

1. Full support for all V-locity features on vSphere 5.0.

2. Support for V-locity Guest installation on VMs hosted on Citrix XenServer. All optimization technology and features present in the V-locity Guest software, such as V-Aware, CogniSAN, IntelliWrite, Space Reclamation, are supported. Note: The only missing feature on Citrix XenServer is the availability of a "Host Agent" (which automatically informs the V-locity Guest component about the virtual disk types - e.g. snapshot, sparse disk, etc...). This is planned for a future release. In the interim, manual configuration may be required.

3. Fixed a rare problem with V-locity service crashing when installed on a non-boot volume or when installed to a short path.

4. Fixed a problem with connection data disappearing from Host Agent.

5. Fixed an MMC crash when remote connecting to V-locity Guest while local UI already open.

6. Fixed a warning message when opening multiple Guest UI instances.

7. Fixed some UI issues that cut-off text for English, Japanese, German and French. 

8. Improved the Installation instructions to provide additional recommendations and information. 

Samsung (the provider of the Win8 tablets that were given away to all attendees) also did a demonstration of ExpressCache at the Microsoft BUILD conference last week.

Their booth, which was at the front of the EXPO, showed many machines with ExpressCache on them and also had a demo table set up with 3 machines.

o   System with a HDD only

o   System with SSD and HDD ExpressCache (8GB SSD)

o   System with SSD only (128GB SSD)

There was a button to push, on the counter top display, that started all of the notebooks up at the same time and then auto-launched an application. The ExpressCache system smoked the HDD notebook and was almost as fast at the large (128GB) SSD-only system. And, it offered a large capacity HDD as well.

The demo proved that, for a fraction of the price of buying a large SSD, you can get all the performance that SSDs offer with ExpressCache technology (and still store all your stuff).

IT professional and author, Justin James of TechRepublic published a top 10 list of ways to speed up your PC. Number 9 was one very familar to us:

"9: Defrag. Defragging your hard drives is a great way to get some more performance. While modern Windows systems automatically defrag on a regular basis, I’ve found that the Windows defragging is fairly unaggressive. We’ve reviewed a lot of different defrag apps here at TechRepublic. I suggest that you check out your alternatives and find one that does a better job for you."

Their findings mimic what we see with many of our business customer seeking to maximize Windows 7 performance. The built in defragmenter sounds like an attractive option at first, but closer inspection and testing clearly demonstrates significant value (better ROI) in advanced third party optimization technology. 

Read the whole article here: http://www.techrepublic.com/blog/10things/10-things-you-can-do-to-boost-pc-performance/2712?tag=nl.e101 

This site is located on our website at: http://www.diskeeper.com/support/forum/ 

Customers, trialware users and any interested parties are all welcome to come to the site and look around or even join up and submit/contribute. Its purpose is to help you get the information you need to maximize the benefits of our solutions, so we look forward to chatting with you through this new portal.

You can read the overview and rules of the fourm here: http://www.diskeeper.com/support/forum/yaf_postst12_General-Rules-and-Guidelines.aspx

Cures Pains from I/O Bottlenecks, Slow VMs, Slow Migration, Resource Conflicts, 

Wasted Storage Space and Slow Backup Speeds

Diskeeper Corporation announced at VMworld the release of new V-locity^® 3 virtual platform disk optimizer for VMware^®. 

V-locity 3 delivers invisible background optimization to give maximum I/O performance on virtual servers. 

Diskeeper has long been a popular compliment for design and engineering software systems, including by the the most popular design and enginerring software company itself:

“Diskeeper indeed provided the ROI we expected. We are very satisfied. I have been using your product on my personal desktop and at work for the past few versions. It always proved to be a valuable product for transparent defrag and overall performance. 

Prior to installing Diskeeper our systems were almost unresponsive. This easy deployment was done right away as Diskeeper was much needed. It is a fine product that we rely upon.  It was reported to me that the automatic defrag did not take any visible resources or produce any slower performance which was really appreciated. 

The automatic defrag also helped overall performance greatly on the files sytem access. One of the folders has millions of files in it with no subfolders and it was almost inaccessible before the Diskeeper implementation due to the fragmentation level being very high” 

- Marc-Andre, Autodesk   

Autodesk, Inc., is a leader in 3D design, engineering and entertainment software. Customers across the manufacturing, architecture, building, construction, and media & entertainment industries—including the last 16 Academy Award winners for Best Visual Effects—use Autodesk software to design, visualize, and simulate their ideas before they’re ever built or created. From blockbuster visual effects and buildings that create their own energy to electric cars and the batteries that power them, the work of our 3D software customers is everywhere you look. Since its introduction of AutoCAD software in 1982, Autodesk continues to develop the broadest portfolio of state-of-the-art 3D software for global markets.

Diskeeper Corporation announced today the launch of a PC Speed Contest where contestants are asked to Show – via a picture or short video and Tell – the remarkable gains that they have received in accelerating PC performance using Diskeeper® 2011 performance software.

Diskeeper increases the speed and reliability of all PCs by improving system performance, faster boot-up times and increased speeds on everything from internet browsing to antivirus scans. Diskeeper 2011 gives PCs faster-than-new speed without any effort from the user.

The grand prize winner will receive a new Laptop computer and a copy of Diskeeper 2011 Pro-Premier software.

To enter the contest: http://tinyurl.com/3b37ayq

A major concern with virtual guest operating systems in a virtualized environment is the possible resource contention introduced as each virtual guest operates independent of the others.

With respect to storage, Diskeeper Corporation’s V-locity 3.0 virtual platform disk optimizer mitigates over-utilization in two ways:

First, each V-locity Guest installation coordinates resource scheduling with a centrally installed V-locity Host Agent.  These guests automatically discover the V-locity Host Agent following installation. Once connected together in this fashion, V-locity Guests cooperate to complete each of their defragmentation tasks in a manner most efficient to the virtual server's resources as a whole.  

Second, the automatic zeroing of free space feature ensures that unused space on virtual drives is zeroed out and compacted in such a way that when a virtual guest is migrated to a different virtual server via VMotion, only the allocated data is transferred.  This speeds up the VMotion process and decreases the load on the shared storage subsystem.

RainWorx specializes in Online Auction Software.  Our hosted web sites are mostly our Auction Software customers.  As such, every auction listing (for each customer, on each server) has one or more images associated, so we have a very high volume of images being uploaded which could potentially create a lot of fragmentation.

Bill Moller

RainWorx

Dashboard integrated. Coming soon... 

Dawn Richcreek, Vice President of Marketing has been recognized by Everything Channel’s CRN Magazine as one of the top Women of the Channel in 2011. This annual list recognizes female executives for their accomplishments over the past year, based on their achievements as executives and the amount of influence they wield over the technology channel.  This year’s Women of the Channel were chosen by the editors of CRN Magazine from a field of vendor channel organizations, distributors and solution providers. 

Over the past year, Richcreek successfully developed the Diskeeper Channel Awareness Campaign with Tech Data.   Additionally, she has launched two new products, Diskeeper^® 2011 data performance software and V-locity^®  2.0  virtual platform disk optimizer, that make the day-to-day lives of reseller customers easier and more productive.  Richcreek’s previous accomplishments include implementation of the Diskeeper Corporation Channel Marketing program where she provided profit generating training and sales tools for thousands of resellers on the Diskeeper Corporation suite of products enabling them to better service and provide solutions to their customers. She is a marketing veteran with over 25 years of experience in the industry. The complete article is located here:  http://tinyurl.com/3dakwmy
 

We'll be releasing V-locity 3.0, so stop by. Engineers and product managers will be there to answer your technical questions.

As usual, we'll be there all week.

I can verify that Diskeeper has kept fragmentation in check across our Windows environment. We have been using Diskeeper in our environment for almost two years with good results.  We started with version 2010, and installed it on all of our Windows servers, both physical and virtual, server 2003 and 2008 R2 - over 800 machines in all. 

That first version of Diskeeper generated some impressive numbers across our environment – 33 million files defragmented, 947 million fragments prevented\eliminated (we were over 1 billion at one point there), and 21.6% performance increase on defragmented files. 

Not having to worry about the fragmentation on all of our many servers has saved us a lot of management headache.    

Christopher M. McIntyre, UK Healthcare

University of Kentucky, Lexington, KY

We're in DC this coming week at government trade show FOSE. Stop by our booth #1137 to get the latest scoop on V-locity 3.0.

Microsoft's Worldwide Partner Conference (WPC) is starting today in Los Angeles. It is, per Microsoft, attracting over 12,000 partners (VARs, MSPs, consultants, ISVs) during the course of the week.

We have a booth in the expo hall, so if you are a current Diskeeper Corp partner or interested in becoming one, please stop by to get the latest news and info on recent and upcoming software releases. We have both Channel and Technical representatives at the booth, so we should be able to answer/assist with any questions or requests.

Our booth (#1248) is located near Windows Azure.