“We have been a Diskeeper customer for several years now and have not had any complaints. Installation is quick and easy with the admin console tool. The application does not have code bloat with each new version. I believe that the traditional hard drive in our environment has lasted longer since the drive heads have to do less work. I have 5 year+ old PCs with the original hard drives in them.  I have been very impressed with the new Intelliwrite prevention technology. I have seen 90% frag prevention on workstations and have a 380gb MS SQL database with 99% prevention. Our Domain Controllers and Fileservers range from 78% to 88%. This average has been done since we have installed the Diskeeper 2010 version in November of 2009.  Attached [below] is a screenshot of Diskeeper 2010 Server running MS Server 2003 R2 X64 with MS SQL. I hope to replace some aging servers with SSD drives and am looking forward to testing Diskeeper with HyperFast” .

Mike L., Information Technology Manager, Ohio Valley Community Credit Union

 

If you live in on the west coast (USA), you may have seen an ad campaign about happy cows coming from California. Given we're a CA-based software company we concur.

OK wait a minute. Why are we talking about cows? What does this have to do with defrag? Maybe you're thinking "Michael's talking bovine, he's had a few too many".

Well maybe yes, but... I do have a point.  

CoW, or "copy on write" refers to a low level technology that looks at block data changes and then takes a particular action. Every time there is a write, a CoW technology would seek to make a copy of just the changes - not the entire new file. That action may be part of inline data dedupe, may be a snapshot, etc. The basic point is that CoW is used to isolate changes to data at a granular level rather than at a file level.

The consideration is that most CoW technologies are unable to distinguish between changes to data or movement of pieces of data, due to a defragmentation job. In other words, if you run a defrag, CoW may think there are actual data changes to files, and then take action to make block-change copies, run a dedupe, etc...

Holy CoW you say!

So, it is possible that a defrag, any kind of defrag, causes a CoW based technology to go into hyperactive mode. That may mean a CDP (continuous data protection)/snapshot program may be triggered in to keeping/taking far more copies that it needs to. It is basically fooled into thinking that a defrag represents actual changes to data, and that they need to process those changes. Now this is a false processing but, because most CoW solutions function underneath the file system layer, they simply cannot differentiate that actions that take place at the file system are not changes to data, but rather data-movement-for-optimization reasons.

So, if we're talking about a CoW solution that takes snapshots, we could see an increase in the amount of copies that a snapshot solution would have to take. That would make for a fairly fat CoW (a lot of extra data storage demands created by not understanding defrag).

CoW based solutions that work at the file system level (NTFS), have the possibility to recognize changes due to defrag and differentiate them from changes due to new actual data to a file. One such CoW solution is Microsft's VSS (Volume Shadow Copy Service). However, as noted, most CoW technologies operate at storage layers that are abstracted from the local disk file system - they operate at levels underneath the file system. Again, that means defrag jobs in the file system will not be recognized by these solutions, and a defrag job of any kind, will cause unnecessary copies.

In comes IntelliWrite. Now, it makes sense to think that IntelliWrite was invented as a faster and, perhaps cooler, solution to fragmentation. But, the truth is that solving fragmentation at the source (when it is created) is vital to ensure compatibility with many of the CoW solutions that may be implemented at virtualization host or SAN layer (i.e. underneath and hence, unaware of the file system).

So, in a nutshell, IntelliWrite was very much designed to ensure that defrag offered full compatibility with modern storage technologies. That, we feel, makes for happy CoWs.

In anticipation of an upcoming presentation (March 10) from Microsoft promoting SSDs, and best practices for optimizing SSDs, is a HyperFast review by V3 (from last year). They tested HyperFast to see what benefit it would provide. Read it here. 

 

If you're looking to defragment SANs (Storage Area Networks) or VMs (virtual machines), you're on the right track. But, there are things you need to be aware of. Defragmenting SAN attached storage and VMs can require planning, implementation, and configuration that is generally unnecessary with direct attached storage (DAS). 

For example, what do you do if you use Thin Provisioning? What about Linked Clones? Does defrag still help if your multipathing? 

The paper sets out to address many considerations and technologies in modern storage infrastructures, and offers "best practices" guidance based on our decades of experience working with many of the largest companies in the world to help them maximize their system's performance and reliability.  

Click on the image below to read the paper.

It makes for fascinating reading on a Saturday night. Just get the fireplace firing, grab some milk and cookies, get a warm blanket and curl up in bed (or on the couch) and enjoy! ;-)

The chart below present results from testing our engineers did in conjuction with engineers at a longtime Diskeeper customer, Solvay Pharmaceuticals (now Abbott), last summer. The tests, done on VMware ESX 3.5 with direct attached storage (DAS), were specifically "proof of concept" that Windows Guest VMs need to be defragmented. A tool we frequently recommend called ReadFile was used to gauge speed of file access. Symantec AV was also run and timed.  

You can view a larger image of above graph here as well: VMware performance gains.jpg (319.27 kb)

“My first impression of Diskeeper 2010, AWESOME!  The ability to proactively prevent fragmentation BEFORE it happens is the Holy Grail of disk I/O management.” - Bruce McMillan, Solvay

 

With close to 400,000 systems, the Naval Marine Corp Intranet (NMCI), managed by EDS/HP, is "the largest internal computer network in the world" and has been using Diskeeper on their systems for close to a decade now: 

"We run Diskeeper on our two production web servers and the development server that we are using to host a web-based warfare systems interface management tool. The name of the application is Warfare Interface Systems Engineering (WISE). 

I am a big fan of Diskeeper because I am all too aware of the problems that  fragmentation can cause on a system. I use it at home as well. I specifically like the feature that keeps your drives defragmented automatically via a background process that does not rely on manual defragmentation of the drives. Because I know that if you leave that up to a user or even some administrators to do, it will never get done and people will be wondering why their system performance is starting to degrade.  

Our office systems are a part of the Navy/Marine Corps Intranet and all of these systems have Diskeeper running on them as well [almost 400,000]. I know all too well this group of users in our command. I used to work IT support and they would never defrag their systems. I do some PC work on the side with home users and I see the same thing there."

Mike M, US NavyNSWC Dam Neck

Virginia Beach, VA

 

"Not only does Diskeeper's software attempt to prevent fragmentation from ever occurring, but it cuts power consumption and unnecessary I/O operations."

"IT administrators who turn frequently to disk defragmentation tools should have a look at Diskeeper 2010." 

- Jason Brooks, eWeek

 

eWeek ran some tests to see if IntelliWrite really lived up to the promise and, no surprise to us, it clearly did. Their tests, done with Windows XP guests running on VMware vShpere 4, found IntelliWrite prevented 87% of fragmentation. Invisitasking quickly cleaned up the few pieces that slipped through.

Read the full article here.

A cure for cancer it is not, but in the category of performance (defrag) software, IntelliWrite is quite the bold, unorthodox step forward. Like many things new and revolutionary it is to be questioned, critiqued and qualified.

When we created IntelliWrite we had to first question the old way of doing things. Same goes for you, the user. You also need to form your own opinion on whether a new approach to eliminating fragmentation is warranted and worthwhile.

Now of course new does not necesarrily mean better, but new can, when done right change the way of things... for the better.

Our customers are our best and favorites critics. We hope we have provided you value and have earned your approval.

IntelliWrite technology is, as the name would imply, smart. Not only is it a better solution to removing fragmentation (it prevents fragmentation), but it actually self-learns; i.e. it gets smarter over time. Or perhaps more simply as one customer, who per his company policy must remain anonymous :-(, said, "crazy smart".

 

IntelliWrite learns how the various different applications on your system write data, much like advanced heuristic anti-virus solutions that learn "patterns" and can then proactively block new, as yet unreported, malware. IntelliWrite adapts to the applications on your system and how they write, and thereby fragment your files. That adaptive learning makes IntelliWrite increasingly more effective on your computer.  

Another part of the technology is an awareness of its surroundings. IntelliWrite automatically detects conditions where it might affect system performance and backs off, thus providing the best balance for performance – improves performance when fragmentation can be prevented and does not impact free space fragmentation when it might matter (i.e. when the available space is very low). One such easy-to-demonstrate case is on volumes with less than 2GB free space; IntelliWrite disables itself.

We began the process of securing the intellectual property rights to this revolutionary innovation prior to the release. You don't have to be a crazy Irishman to know that if you're looking for "The only way to prevent fragmentation before it happens(tm)", you've come to the right place.

 

The update to Undelete 2009 to support Windows 7 and Windows Server 2008r2 is available. It also includes some general fixes.

I recommend going to your Account Page and downloading from there.

 

The Undelete update that adds support for Win7 and Server 2008r2 has certified internally. It's now a matter of routing and uploading the software. Expect it to be available on your Diskeeper Account page early next week.

 

Diskeeper 2010 Administrator is now complete and available for broad use. You can download trialware from here. For customers previously using the Release Candidate (RC), you can go to your Diskeeper Account page and download your full licensed version. We made numerous user experience enhancements from the RC.

One improved UI example is the new start page:

The new major feature in Diskeeper 2010 Administrator is the “at-a-glance” network-wide overview of disk performance and system health. This network performance dashboard includes access for more details on noted issues and ability to directly initiate tasks such as remote control, deployment, and configuration to quickly remedy the situation. The Dashboard provides important customizable alerts on systems that have:

• Low free space

• Severe file fragmentation

• Extreme paging file or meta data fragmentation

• Failures to defragment or where Diskeeper is not operational

• Product activation expiration

• Diskeeper updates available for installation

The "alerts" are now available for both computers with Diskeeper and those without it. As with prior versions of Diskeeper Administrator, "reports" continue to be available for both systems without and without Diskeeper.

Ops Manager (both MOM and SCOM) Management Packs are included with the Diskeeper Administrator full version. If you require a trial of this solution, please contact our Corporate Sales dept.

I'll post more about this joint webinar (i.e. date/time, links) as we get closer to the date: 

“Windows 7 and Solid State Drives: Performance Myths and Facts

Presented by Microsoft & Diskeeper Corp.

What are the benefits of SSDs, above and beyond traditional hard drives? What are the myths around performance? Do file and free space fragmentation affect SSDs? How accurate are available benchmarking tools? How can you successfully work SSDs into your current customer offerings?  Join technical engineers from Microsoft and Diskeeper Corporation for this informative webinar.”

It is interesting to see the varying degrees of effort beta testers undertake. Some just kick the tires and make sure nothing obvious breaks. Others may run benchmark tools, test compatibility with production apps in production-recreated labs. Some even on production systems. Then there those, like Yu-Ji Hosokawa. Mr Hosokawa is a senior software engineer from Tokyo, Japan. When Diskeeper 2010 was in beta last fall, he decided to run side by side comparison tests to validate IntelliWrite, and he wrote a software program to automate it.

He ran tests on Windows XP and Windows 7 (below), preventing 99% of fragmentation and completing the benchmark 2-3x faster in many cases. Granted, this  environment is artificially created, but the results are compelling nevertheless. Real world results are quite similar (stay tuned for future blogs with more test routines).

Here are his results:

 

For those interested in duplicating these tests. Mr Hosokawa was kind enough to provide the software he wrote, including the source code! You can find it here.

The program is called IntelliWriteBench.exe. It also requires SysInternals Contig,exe program which can be found here. Contig.exe muct be located in the same directory as IntelliWriteBench.exe.  

IntelliWriteBench.exe parameters are;  

--workers

      # of parallel thread  

--file

      # of test files  

--bytes

      size of data for one file  

--total

      # of writing times for one file  

--directory

      directory for testing  

--verbose

      show status  

The default (if no parameters are selected) is: IntelliWriteBench.exe --workers 3 --files 10 --bytes 4096 --total 10000 --directory .\

Disclaimer: the software referenced on this page (with exception to Diskeeper with IntelliWrite) is not provided, guaranteed, or supported by Diskeeper Corp. Use at your own risk.

Rumack: Fragmentation slows down SD Card performance!

 

Striker: Surely you can't be serious.

 

Rumack: I am serious... and don't call me Shirley.    

 

The word is slowly [but surely :-)] getting out that fragmentation affects NAND Flash storage - specifically free space fragmentation. The SD Association website is a great resource for edification and development resources. On their page describing the SD Card Speed Classes, they even discuss fragmentation.

 

 

Fragmentation and Speed

The memory of a card is divided into minimum memory units. The host writes data onto memory units where no data is already stored. As available memory becomes divided into smaller units through normal use, this leads to an increase in non-linear, or fragmented storage. The amount of fragmentation can reduce write speeds so higher SD card speeds help compensate for fragmentation.

 

There are several methods to address this issue. One is, as mentioned, buy better performing storage and hope your requirements never exceed the cards ability to deliver on your needs (though fragmentation will still limit the storage's peak performance). However, a better approach is to fix the root issue, and there are two ways to do that.

1. Defragment the free space (e.g. HyperFast, Diskeeper)

2. Copy all the data off the SD Card and reformat the card

The best approach is likely to be determined by the device in which the SD Card is used. If it's from your digital camera, option 2 is probably pretty easy to undertake. If pulling all the data off the card is not feasible, optimize it.

The only other question then might be "how often do I run optimization?". I did a blog post on that, including some performance tests, a few years ago, Read that Here or Here.

Lastly, I thought I'd include a recent personal success from an IT professional who happens to also use Diskeeper at work:

"Happy new year!! I made an interesting test over the holidays using Diskeeper 2010. I was updating my GPS with new maps and discovered that my Garmin unit uses Fat32 file system so I figured I would run your software on it just for fun and see the results. I was able to defragment a large part of the files and it more than doubled the speed of the unit. The images render faster with less stutter, routes are recalculated almost instantly now and it finds points of interest much faster. I did the same test on 2 other GPS`s and got the same results." Regards, Carlo

As of December 31st, Diskeeper 2010 is certified. 

On the heels of a prior update comes a new incremental build. This update resolves rare cases the Terabyte Volume Engine (TVE) process may fail, and a second unrelated issue where Diskeeper does does not start (i.e. defragmenting) after the initial installation due to licensing. As usual, if you are not experiencing these issues, there is no need to update.

Here is the ReadMe note from the update:

D.  CHANGES IN THIS RELEASE
===========================

1. Fixed issue where defragmentation operation failed to start.

Undelete 2009 support for Windows 7/Server 2008R2 will be delayed until mid to late January.

This is a reposting of a prior (and popular) blog entry that unfortunately was "chopped" when we migrated to our new platform. Here it is in full:  

Preface: this article refers to making direct edits to the Windows Registry. If you are not experienced with this subject, ask your company's IT Administrator or a computer-expert friend/neighbor for help.

Our Tech Support group has seen a few reports of this error (Program not found - skipping AUTOCHECK) from customers when running Diskeeper's bootime defrag. The error starts early in the boot process while the Session Manager process (smss.exe) is busy getting the system up and running. Smss.exe is critical to loading the paging file, initializing the registry and loading kernel components. But, before it does any of that it looks to a registry key called BootExecute. At that location it launches any applications listed. Session Manager then looks to the Windows system32 folder for particular executables it has been instructed to launch. By default there is only one program listed here - autochk.exe, the boot-time version of chkdsk, which will run if there are any file system inconsistency flags detected (i.e. volume dirty bit is set).

Read on for the solution...

Software vendors who need exclusive access to a volume (such as a defragmenter) will name proprietary executables at this registry location (and place the programs in the Windows system32 folder). Using this system is how Diskeeper is able to safely defragment files that could not be defragmented when the system is up and running.

However, malware creators have also used this BootExecute location to load their spyware/virus crap.

If you uninstall a legitimate program that has written into this BootExecute registry you may see this message. The uninstall will typically delete the executable from the system32 program, but not edit the registry. In most cases, the registry change is only a temporary one. For example: if you set Diskeeper to run a Bootime defrag "on next reboot" but uninstall it before the reboot, you can create the same issue. Once the Diskeeper Bootime defrag completes, it removes this string from the registry. Other applications are likely to behave similarly. It is also possible that a program, during install, writes data into this key, but then does not remove it on uninstall.

Another possibility is if you have run an anti-malware program that has removed the referenced executable from the system32 folder, but not changed the BootExecute registry to remove the "pointer" to that file. In that case you will also see this message.

When the program named is from a removed third-party vendor, the message is harmless. You should only be concerned if the default autochk program itself does not run.

If you do get this error and want to clean it up, here's what you need to do:

Look in the registry at:

"HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager" and remove the string [the name referenced in the message on system startup] from the BootExecute value.

Under normal circumstances only the following would be present:

autocheck autochk *

This is what it might look like with added values:

autocheck autochk * autocheck stera

You can change the value back to the default (as shown in the first example above), but understand that it may possibly impact a legitimate program listed here.

Stera.exe is part of an adware program that pretends to be an anti-adware program!

As always, you need to be very careful editing the registry. If you see anything else listed here other than the executable named in the error message, look at named file's properties in the system32 folder or do a web search on it. A legitimate vendor can advise you on what to do to avoid potential conflict.

For Diskeeper it would look like:
autocheck autochk * autocheck AUTONTFS E: PAGE=KEEP DIRS=NONE MFT=MIN

(where E: represents the drive letter on which to run the bootime).

You can also reset it back to the default (autocheck autochk *) without issue. That is the safest bet with Diskeeper. You'll simply need to go back into Diskeeper and reset the Bootime job.

Diskeeper customers are always welcome to contact our support team for assistance.

A new build is available for Diskeeper 2010 users. It addresses the SQL Server CHECKDB false-positive reported previously. If you aren't using SQL Server, then you won't need to update at this time.

Here is the ReadMe text (noting the only change):

D.  CHANGES IN THIS RELEASE
===========================

1.  Fixed issue where Diskeeper was conflicting with SQL database
    consistency check resulting in errors in the event log.

We have an online FAQ that I thought I'd post here as well. It provides some valuable insight into what's going on with all those graphs and statistics related to our proprietary new IntelliWrite technology.

IntelliWrite Fragmentation Prevention

 

This section of the Dashboard tab explains that IntelliWrite increases system performance by preventing fragmentation before it happens.

IntelliWrite System Statistics for All Volumes Since the Previous Day

This section of the Dashboard Tab includes statistical information regarding fragmentation prevented by IntelliWrite, fragmentation eliminated by Automatic Defragmentation, and a sum of the two numbers since the previous day.

It is important to note that the amount of fragmentation prevented is an estimation determined by a large number of factors and extensive testing. The IntelliWrite graphs are designed to approximate fragments prevented across a wide range of applications and must incorporate various types of file writes and modifications into a singular display. In some cases the charts may over-estimate and in other cases, underestimate. For example, on SQL Server® the graphs may overestimate about 10% of the fragmentation prevented, on Microsoft® Office documents it may underestimate the number of fragments prevented by 80%. What is important is that with IntelliWrite enabled, fragmentation is being handled, before it happens. The end result of having Diskeeper with IntelliWrite and Automatic Defragmentation working is a system running at peak performance in the most efficient way possible (without fragmentation).

The statement: "Proactive prevention is the most energy efficient method to eliminate increased storage power consumption caused by fragmentation," points out that when IntelliWrite is handling your system’s fragmentation, it is reducing the amount of disk head movement previously needed to accomplish defragmentation and therefore is reducing Diskeeper's total system resource footprint.

System Fragmentation Prevention Graph

 

This section of the Dashboard tab shows system fragmentation prevented by IntelliWrite in real-time. In the graph, fragmentation prevented every second, within the last minute, for your entire system, is displayed in green. The scale on the left side of the graph pertains to the number of fragments that have been prevented and the scale along the top of the graph pertains to the seconds in the last minute.

Statistical Information for Selected Volume(s) Since the Previous Day

 

This section of the Dashboard tab shows statistical information for selected volume(s) since the previous day. The columns of the table include the name of the selected volume(s), whether or not IntelliWrite and Automatic Defragmentation are enabled, the number of fragments prevented, the number of fragments eliminated and file read/write time % improvement.