Malware Defense History and its Secrets
(An Unorthodox Approach to the History of Malware Defense)

by Dr. Nikolai Bezroukov.

Ch1: An Overview of Malware History

Standard disclaimer: The statements, views and opinions presented in the e-book are those of the author and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

To get a proper understanding of this topic, let's first review the history of this subject. Any serious researcher would agree that the antivirus industry (as well as now newly minted antispyware vendors) is to a large extent a by-product of Microsoft.  Of course Microsoft was not a single reason, but due to its dominance and inability (or lack of desire) to close some holes in Windows, until recently it played a major role in creating the industry.

The history of Av software closely corresponds to the history of viruses and worms themselves.

It can be viewed as consisting of the several overlapping stages:

• Boot and file viruses (1988-1996). Microsoft DOS was and early version of Windows were the most virus friendly OSes available.  First of all due to their mass deployment, but not only because of that. Almost until 1996 when macro viruses appears and started of threaten Microsoft bottom line it did absolutely nothing to make Windows more secure.  No attempt of patching major security holes present in its DOS and early Windows were undertaken as they might break compatibility with old software.
   
• Macro viruses (1996-2000). This period was very interesting as AV vendors proved to be completely unprepared to the new threat (the situation that will repeat itself in the future many times). Please remember that in 1995-1996 it took almost a year for AV vendors to (more or less adequately) react on the Concept macro virus. Each time a relatively new threat arise, AV vendors fall far behind the regular upgrade cycle. Before that the value of AV products for Ms Word macro virus protection was the same as  the value of a simple grep-style search utility, available for free from any good file repository ;-).
  
  In late 90th Microsoft started to understand that native Office formats are insecure and that can badly affect the revenue stream. So the company made some improvements in Office 97 and especially in Office 2000. But it was too little and too late. See Note on virus paranoia.
   
• Mail viruses and worms (1998-2004). Here Microsoft was in the game again and managed to create a few new category of mail worms (due to a simple and stupid decision of hiding the extension in files by default): "double extension attachment" worms and Trojans ;-).
   
• Network viruses and worms (2001-2005). This is a pretty new phenomena although in the past there was Morris worm, which is the oldest network worm known. Paradoxically it was written for Unix. Windows infection by network worms stated with Code Red( July 16, 2001), SQL Slammer (Jan, 2003), MSBlaster (Aug, 2003); Sasser (Apr 30, 2004) and Zotob (Aug 17, 2005). Probably those events will continue happen with the same interval in the future.

Unfortunately malware is not only about architectural limitations and security weaknesses. Sometimes even advantages of Microsoft platform lead to creation of a new class of worms. For example the fact that VBA/VBS became  a standard macro language in MS Windows environment and is used in MS Office let to the creation of a new type of viruses -- macro viruses. Unix/Linux cannot have this class of  viruses just because applications available for Unix does not have a common macrolanguage. they also have the problem with a tiny market share that prevents creating a critical mass of available targets essential for any successful virus of worm.

Similar situation repeated in mail. while Outlook proved to be a very good and innovative in some areas scriptable mail client. Partially because of the power of Outlook Microsoft had recurrent problems with the security of MAPI services and, especially, with the Microsoft Outlook implementation. 

While Microsoft Window has had and will continue to have vulnerabilities that lead to creation of viruses and worms including some new types I would like to stress that antivirus and antispyware products should not be considered as the only solution.

If history is any guidance that it is almost guaranteed that AV companies will be  late comers to fight each new threat: they always missed the treat by at least a year and then try to sell half-baked solutions to unsuspecting customers.  With any new threat it is better to use products from the companies who were first to understand the threat and who are not burdened by almost 20 years of AV malpractice. Now this is especially true about spyware 

They are partially solution partially a part of the problem for three main reasons:

• The relevant information about the viruses/worms/spyware necessary for correct defense is often is hidden or buried behind a smoke screen of FUD. For example, try to find out on which versions of MS Windows a particular virus can replicate. In most cases this is not an easy task and this information is often hidden by AV vendors in order to secure a commercial advantage and reach the widest spectrum of customers possible.

  Next try to find out how the virus detects its presence on the computer -- information that often can be used for simple "home-made" vaccines. 

  That's why you should never rely of a single vendor information. Try to correlate at least three description, or, better, do some own controlled experiments to verify the data.

  The sad truth about AV vendors is that they tend to exaggerate the threat and often withhold vital information about a particular virus implementation limitations from the users.

  
  
• Most AV products  systematically are behind the most prominent threats (for example all AV products were almost completely useless during MSBlaster epidemics, probably the most successful network worm). Sometimes algorithms used in most popular products are so outdated (for example in scanners), that they are part of the problem, not the part of the solution due to the number of false positives and additional load of PC due to on the fly scanning.  My impression is that "on the fly scanners" in a corporate environment are a hidden Trojan horse far more dangerous and disruptive then any recent worm epidemics, and also slowing PC down two times or more because the list of searchable extensions often includes RTF files,  Access databases or Excel spreadsheets ;-). 
   
• The damage that some AV products ("on the fly" scanners) cause in desktop, and especially server environment often overweight the benefits. On the fly scanning has such a negative effect on I/O that working with a large Excel spreadsheet and MS Access databases is best done with on-the fly scanning  turned off. Yes, there are some Excel viruses (and sometimes they can even propagate within an organization :-) , but none of them can compete with the Trojan horse effect of AV products ;-). The damage from stupidly written AV products far exceed damage from any Excel viruses. Those guys often outsmart virus writers and ensure a lasting damage to the organizations that are too fundamentalist about virus scanning :-). A simple non-resident scanners can often be as effective as resident scanners for a fraction of the cost (and damage). In a large organization the number of problems (and helpdesk tickets) connected with side affects of Av scanners often exceed number of problem coursed by actual virus infection by at least one order of magnitude.  In a sense current generation of virus scanners are extremely inferior products and definitely represent a mixed blessing.
   
• As the industry was built of hype and the most popular products might be "barking on the wrong tree"; For example "enterprise level' virus signature distribution often represent the most non-constructive approach to the problem that introduced another half-baked solution (say F-secure signature distribution) of pretty generic problem. I do not understand why any large company are buying one vendor oriented, buggy and unreliable solution, when generic software distribution mechanism (or even Netware login scripts) are pretty adequate for this type of problem. Also the threat is often pretty specific (compare MSBlaster type of infection with the  multipart IRC worms like Muma). Using generic AV scanner is less effective then distribution of worm specific "fighting module", which can be created using Perl or other the scripting language in no time.   The set of viruses typical for a large corporation is pretty stable and for a fraction of the cost it is possible to create "mini-modules"  that check for those viruses extremely efficiently and limit the usage of particular Ac product to periodic scans of the harddrive, the mode where it can do much less damage in comparison with "on the fly" scanning.  
   
• Low qualification and errors in judgment  of some corporate administrators who are responsible for the AV defense.  Security like a cleaning survives does not attract the brightest and most capable programmers and administrators. More often then not, the responsibility for AV defense one large company is offloaded to  a person with no specific knowledge or technical skills or even interest in his area. But such a "placeholder" approach can backfire. First of all AV companies are notorious for parasitizing on this ignorance of management in selling their wares. That is their standard selling practice. At the same time the damage can be greatly amplified by using a low quality AV product or stupid defense tactics (especially virus paranoia.). Often ignorant security admin are proactive in shutting down the whole corporate network without real justification (due to low level of understanding of the threat), thus serving as a perfect complimentary module to popular viruses and worms. Sometimes when the threat is not well covered by the existing scanner instead of quick deployment of a newer or custom worm specific solution, the extension list for  "on the fly scanners" is changed in the most  perverted way for the whole corporation (scan everything, just in case, even text files).  That just increases the damage from the virus adding to it damage from incompetence.  

Sources of information about AV defense

Despite information overload about all kind of virus and worms, there are not so many articles/books that are worth reading about computer viruses. Even in old days of MS-DOS and file and boot viruses and Trojans one probably was better off with reading a good book about MS DOS structure and BIOS programming than a book about viruses. And this is still true in new dimensions of malware like network worms. 

In mid 1990-th when macro viruses became prominent  one would be better off reading a book about VBA than all this 1largely misleading and conflicting information that AV vendors distributed about macro viruses. At this point (and even now) understanding of programming (especially scripting languages and VBA) is a must for in depth understanding this type of the treat. Corporate AV specialists, who do not want or are unable to learn programming usually cannot adequately react to new threats and can became more part of the problem, then a part of the solution.

In 2000-2003 SMTP-based worms (miMail) and RPC-based worms (MS Blaster) were completely missed by major AV products (and that explain the level of epidemics they caused) and one probably can benefit from learning the basics of SMTP and MIME as well as TCP/IP. 

Now spyware and network worms came into the prominence and it require more registry knowledge then before. For network worms some hardening of the registry and enabling PC firewall might represent a better protection then any AV product conceivable.

Virus Bulletin used to provide decent analytical articles about complex viruses, but they deteriorated and recently are not that interesting to read.   Also they are ridiculously expensive and too closely connected with Sophos (that in 2003 acquired and partially destroyed ActiveState). 

This information vacuum sometimes creates an impression that underground virus-related publications are more reliable source of information about viruses than AV vendors and AV-related press ;-).   For example in case of Sircam worm no major AV vendors mentioned that it's cannot work on Windows 2000 and Windows NT and needs MAPI properly configured.

I would like to stress it again: any information from virus vendors should be viewed with skepticism due to inherent conflict of interests of AV firms. For people who are responsible for the corporate AV defense it pays to compare several sources of information especially if you do not have a sample of a particular virus and unable to analyses it yourself. Often AV vendors overestimate the danger and do not mention a limitations of a particular virus.

Fake Epidemics

Funny, but changes in signature database of an antivirus scanner (when it suddenly stars to  detect some spyware/adware that it previous ignored) can produce an illusion of epidemics in large corporations, discovering cases that probably existed for months on corporate PC in one day and creating some sense of urgency to fight this newly discovered mass infection. Don't be fooled by such cases, they are just nuisance, not a real epidemic. Otherwise you can face the corresponding chaos and damage from dealing with too many cases of the  "non vital" threat at once.

Questions to be asked depend on the type of the virus/worm

 Generally it make sense to try to find an answers on the following questions: 

• Is it working on the version of Window that I (or my corporation) use ?  For example if a corporation is standardized on  NT or Windows 2000 environment it makes sense to experiment and see if it is able to function/replicate in this environment or is limited to a more popular Windows 9x series. It requires just a couple of PCs to do such an experiment.
   
• Does it require specific components or DLL to be present? Does it depends of some not so common in certain environments features present (for example Outlook installed or MAPI correctly configured). Does it uses some obscure DLL or some other program ?  Viruses are usually highly specialized program and virus authors often test them on their own, pretty idiosyncratic environment only; they usually do not test them in different environments and definitely not on a typical corporate desktop ;-)
   
• How does the virus/worm detects it's presence the computer to avoid multiple infection? Most Trojans, file viruses and worms can it be fooled by creating a "fake" identification, by some stub or registry entry. Often this part of the worm virus is programmed in a very primitive way and that provides a possibility to block the virus even without any AV programs using a "vaccine" approach.

For macro viruses/worms the questions to be asked include: 

• Is it really working on the version of MS Office that I/my organization use (most early macro viruses do not work with Office 2000)? 
• Is it a specific mailer program or version of MAPI dependent ?  Does the virus/worm propagate in a particular version of messaging software? For example if you use Netscape Messenger or Notes instead of Outlook as a mail client, then Melissa represents much less danger -- you still can pass it to your friends for forwarding a particular attachment, but that's it -- your computer will never redistribute the virus automatically as was intended by the virus author.
• Is it too buggy to propagate?

For mail worms additional questions are:

• Are the mail headers really random or can be easily filtered based of "FROM", "TO", and/or "SUBJECT" fields using simple filters like those that are present in Netscape Messenger (this was the case with Sircam and Hybris (hahaha, sexyfun) )
• Does the size of infected letter fall into a certain range ?
• Can we block the worm based on extensions of infected attachments (for example Sircam uses double extensions in infected files like dot.exe, xls.pif, doc.lnk, etc.).  It's a very bad idea to allow executable attachment into corporate environment anyway, but sometime it's difficult to block incoming .EXE attachment due to some stupid corporate subscription or whatever. You can filter double attachments using procmail or other tool. The corporation can also switch to "OKEXE  approach" (creation of special prefix (like OK in this example) that is used for *all* executable attachments that are send via external mail gateway and invoke batch file that run some AV program or Perl script to make common sense checks and then renames it to a regular extension).

The real problem here is that until recently Windows executables were not signed (and macros also belong to executables -- they are just hidden in MS Word document format) and thus were not protected against tampering with MD5 checksums or similar. Both NT and Unix needs MD5 checksums for executables ASAP. Actually NT has such a capability (Authenticode) but it's not widely used. It looks like Windows XP make some positive steps in this direction but I do not know details.

Do not jump into installing commercial AV tools each time you hear about a new nasty virus. In most case free scripts run via scheduler or via Netware login scripts (or similar mechanism) can be as good or even better. The same is true for the protection from email worms on the gateway level, but here free tools can serve complementary role; a commercial AV gateway filter has certain advantages. See my Overview of VB’97.

The False Positives and AV Scanner Effect on Corporate IT Environment

With all the hype about viruses worm and actual pretty brutal epidemics of network worms that we recently experienced, the main danger for corporate IT infrastructure and then source of lass are major AV products. It you calculate the number of helpdesk tickets devoted to this particular brand of software and thier cost for a typical multinational it is clear that this is a very dangerous software that creates as much if not more problems as they solve (or pretend to solve).

Moreover in commercial environment the loss leader is not viruses, its AV false alarms (or false positives as they are often called) -- despite AV-vendors  precautions they regularly appear and spoil the party. That mean that AV programs are much more close to Trojan Horse than one might suspect ;-) One self-quote would help:

Each day, most cases in corporate environment that are reported by customers that are not actual cases of virus infections. They are false alarms or false-positives.

The main problem here is AV products. Paranoid users and sometimes system administrators often blame on a computer virus effects that they cannot understand/explain.

In the past the main course of false positives in the corporate environment are such products as Inoculan, McAfee and F-secure.  The old version 4.0x sometimes recognized regular VBscript program as MS Word documents and produce messages like "Infection: 'Macro.Word97.Class.eb' [AVP]. Very distressing experience if one hundred remote and clueless users get this message in one day. Probably real virus would be less frustrating ;-)

Usually file viruses and macro viruses are reported as false positive, sometimes interesting combinations arise. For example now obsolete F-prot 2.xx   reported as infected any MS Word document that was disinfected by corresponding version of McAfee. That was very interesting experience if one part of the corporation uses F-prot and other McAfee ;-)

Another interesting combination arise if false positive detected but AV program unable to disinfect it and either renames it or put is some directory. That's when AV program became a real 100% pure Trojan Horse.

Inoculan is especially bad as it not only reports about files as infected when they are not, but moves them to the Infected directory. If configured that way Inoculan can be considered as a sort of Trojan Horse and may prevent installation of the packages on the network drive.

I do not want to go that far as to propose everybody move to Linux/FreeBSD (although they are not ideal and are susceptible to certain type of viruses and Trojans, they do provide a much better AV protection out of box than any flavor of Windows).

All I ask for is: please approach skeptically to installation of AV software package and try to minimize thier negative influence by avoiding typical configuration mistakes and limiting the use of on the fly scanning. So not jump into the AV infrastructure improvement mode with each new epidemics. For example for network worms improvement of AV infrastructure is

Windows 2000/XP because a new virus or  worm was discovered -- usually that does not significantly increase the level of your AV protection. Try to use built-in mechanisms first, specialized tools second and generic AV scanners only as the last resort. the latter should be used in of-line scanning mode, not in "on the fly" mode to minimizes negative influences on other software. If you chose to use "on the fly" protection use minimal set of extensions.

Still as  a successful new virus probably will use slightly different approach, not foreseen by AV vendor and signatures are always slightly behind the events, unless you use an automatic update system (and in this case risk troubles if update is buggy) Av scanner does not provide too much protection to count on. They are mostly cleaning tools for known threats. Understanding your environment is a better goal and other tools can provide multilevel protection which is always more affective that a single level, based on AV package.

Often in a corporate environment a lot can be done with adequate polities. If for example all Word documents and executable should be zipped before sending you can completely block corresponding attachments and thus diminish or eliminate related threats. 

Upgrade is also an interesting alternative, especially for home users. for example Windows 200 users can benefit form upgrade to XP SP2. If you are not ready for such a radical solution ;-) in any case one should upgrade to Office 2003 that has better protection in Outlook against typical mail threats.

It is obvious that anti-virus vendors will always be playing catch-up with the virus writers. Theoretically if anti-virus software updates were released quickly and people instantly installed them desktop just patch protection might be adequate. This is however impossible. The life cycle of a virus looks something like (this a self-quote again):

1. Virus is written, tested, possibly deployed on a test network (but usually not debugged for every Microsoft OS in existence; for example virus may not work on Windows 9x, if the only test platform was Windows XP, or if it was debugged on Windows 9x, or on Windows 2000.
2. Virus is released on a selected target (university campus, Usenet groups, etc.)
3. Virus (if "successful" in a biological sense) enter the stage of epidemics and spreads like wildfire, possibly causing damage (such as sending documents from folders on the hard drive, or even wiping motherboard BIOS chips ;-). Generally the more damage the virus cause the less changes are for its survival and the ability to reach the critical mass. 
4. The first samples get to AV vendor (someone notices a strange activity detect the virus and sends it to an anti-virus vendor ). At this time the virus might not still reach the critical mass.
5. AV vendor(s) analyzed, and possibly decompiled a virus and updated their product's signature file. Typically the anti-virus vendor share data with others, but they may or may not do this promptly. Anyway at this stage getting a virus sample is not a problem.
6. Press informs users about the threat and how to fight it. The anti-virus vendors issue bulletins, make the update to the first buggy solution, etc.
7. Large ISPs and some large corporations  install updated signature files and implement other defense measures fort of their mail gateways (or firewall in case of MSBlaster type of viruses)  then on the desktop  Some large customers with decent professionals or support contracts do the same, some have automated distribution systems for the update, resulting in a rapid deployment of the fix.
8. Even home users start catching the virus. This is a start of a "chronic period" of the virus life when it still manage to infect some machines but the number is shrinking....
9. Environment (for example version of OS or version of Office, etc)  changes to the extent that the virus is no longer is a viable threat. This is a clinical death. Complex viruses are more sensitive to env. changes and thus generally die much faster than simple one.

Please note that with some tuning for most email worms spam filters which can serve as a heuristic virus/worms protection tool.  That means that right now no home user  should access his ISP POP account with plain vanilla message client (like Netscape Messenger). One needs a spam filter either built as a POP retrieval tool or (in corporate environment and for Using/Linux) using message filter on the gateway level with additional spam filter installed. See Filtering Mail FAQ for more details.