Softpanorama
(slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-)

Google   


Malware Defense History and its Secrets
(An Unorthodox Approach to the History of Malware Defense)

by Dr. Nikolai Bezroukov.


Skeptical Foreword

The preoccupation with computer "hacking"
is a way for physically unattractive males
to enter the mainstream of society.
Anonymous

Viruses are to a certain extent a byproduct of Microsoft’s dominance in personal computers. MS-DOS, the first mass produced operating system for the first mass produced computer, is amazingly virus friendly. Essentially each new generation of computers leads to the creation and widespread adoption) of certain new types of programs. Here, Microsoft certainly made a significant contribution and it’s not accidental that among several new widespread programs that came to light, due to the PC revolution, are computer viruses.

I do not want to give an exact definition, but viruses generally need to replicate and attach themselves to other executable programs or executable content (scripts that may be part of the document or e-mail). Depending on the method of replication there are several types of virus -- boot viruses, file viruses, macro viruses and mail worms (actually worms are distinct from viruses, but very few people care; we can think about network propagation that is characteristic to the worm as an additional replication method). A virus does not need to belong to a single type: mixed, multi-environment viruses (for example boot+file, macro+file, macro+worm) also exist.  

Corporate AV protection became a branch of the software industry with a lot of vendors trying to hype and sell (or sell and hype) different antiviral products to corporations as well as to and the unsuspecting public. There are several annual conferences devoted to this problem, and in each up to a dozen vendors with anti-virus (AV) products are usually present.

The history of AV software closely corresponds to the history of viruses and worms themselves. It can be viewed as consisting of the several overlapping stages:

 

I also would like to stress that viruses represent both a computer science related problem and a social problem. So-called virus paranoia (with virus hoaxes as the most visible aspect of this problem), virus hype and the deception of the public by virus writers, mass media and antivirus vendors are part of the social dimension of the problem.

We can define several types of computer science research problems connected with computer viruses:

Social problems connected with viruses are as interesting as computer-science related problems. 

One of points stressed in this book is that the defense should be multilevel and one should not rely too much of traditional products from major AV companies. For each category of viruses/worms the are several relatively unknown methods that can provide a decent alternative or complement to the traditional methods of anti-virus defense used in a corporate environment.

Let's start with a skeptical analysis of the role of Microsoft and AV vendors play in this field.

Microsoft Role

Any serious researcher would agree that the antivirus industry (as well as now newly minted antispyware vendors) is to a large extent a product of Microsoft.  Of course Microsoft was not a single reason, but due to its dominance and inability (or lack of desire) to close some holes in Windows, until recently it played a major role in creating the industry... 

Microsoft DOS was and early version of Windows in its various incarnations remains the most virus friendly/spyware friendly OS available.  Almost until 2000 (and including Windows 2000) Microsoft did very little to patch security holes. It actually created a few new -- for example as we now know all too well, hiding the extension in files has had an interesting side effect, creating a whole new class of "double attachment" worms and Trojans ;-). Also with new networking capabilities new threat from RPC and other networking protocols surfaced creating a new category of worms network worms.   I would suggest that Microsoft definitely could make the life of virus authors more difficult even taking into account Windows architectural limitations. Until recently they just did not want to do anything because it does not affect MS bottom line. With spyware and networking worms the situation is different. Currently Microsoft provides a free tool to fight spyware and incorporates additional security mechanism in the future versions of Windows.  windows XP SP2 is more secure than windows 2000 and Longhorn will be more secure then Windows XP.

Unfortunately malware is not only about architectural limitations and security weaknesses. Sometimes even advantages of Microsoft platform lead to creation of a new class of worms. For example the fact that VBA/VBS became  a standard macro language in MS Windows environment and is used in MS Office let to the creation of a new type of viruses -- macro viruses. Unix/Linux cannot have this class of  viruses just because applications available for Unix does not have a common macrolanguage. they also have the problem with a tiny market share that prevents creating a critical mass of available targets essential for any successful virus of worm.

In late 90th Microsoft started to understand that native Office formats are insecure and that can badly affect the revenue stream. So the company made some improvements in Office 97 and especially in Office 2000. But it was too little and too late. See Note on virus paranoia.

Similar situation repeated in mail. while Outlook proved to be a very good and innovative in some areas scriptable mail client, Microsoft had recurrent problems with the security of MAPI services and, especially, with the Microsoft Outlook implementation.

AV Vendors 

While Microsoft Window has had and will continue to have vulnerabilities that lead to creation of viruses and worms including some new types I would like to stress that antivirus and antispyware products should not be considered as the only solution. They are partially solution partially a part of the problem for three main reasons:

Despite information overload about all kind of virus and worms, there are not so many articles/books that are worth reading about computer viruses. Even in old days of MS-DOS and file and boot viruses and Trojans one probably was better off with reading a good book about MS DOS structure and BIOS programming than a book about viruses. And this is still true in new dimensions of malware like network worms. 

In mid 1990-th when macro viruses became prominent  one would be better off reading a book about VBA than all this 1largely misleading and conflicting information that AV vendors distributed about macro viruses.

In 2000-2003 SMTP-based worms (miMail) and RPC-based worms (MS Blaster) were completely missed by major AV products (and that explain the level of epidemics they caused) and one probably can benefit from learning the basics of SMTP and MIME as well as TCP/IP. 

Now spyware and network worms came into the prominence and it require more memory and registry knowledge. 

And it is important to understand that with any new threat it is almost guaranteed that AV companies will be  late comers to this field. The history confirms this hypothesis pretty nicely: they always missed the treat by at least a year and then try to sell half-baked solutions to unsuspecting customers.  It is better to use products from the companies who were first to understand the threat and who are not burdened by 10 year of AV malpractice. for example for spyware protection Adaware is one of such products and it has free version on its scanner. See more about spyware scanners at special page   For network worms some hardening of the registry and enabling PC firewall might represent a better protection then any AV product conceivable.

Funny, but changes in signature database of an antivirus scanner (when it suddenly stars to  detect some spyware/adware that it previous ignored) can produce an illusion of epidemics in large corporations, discovering cases that probably existed for months on corporate PC in one day and creating some sense of urgency to fight this newly discovered mass infection. Don't be fooled by such cases, they are just nuisance, not a real epidemic. Otherwise you can face the corresponding chaos and damage from dealing with too many cases of the  "non vital" threat at once.

Again I would like to stress that at least a rudimentary understanding of programming (now, especially scripting languages and VBA) is a must for in depth understanding of malware including viruses,  worms and spyware. Corporate AV specialists, who do not want or are unable to learn programming usually cannot adequately react to new threats and can became more part of the problem, then a part of the solution. 

Please remember that in 1995-1996 it took almost a year for AV vendors to (more or less adequately) react on the Concept macro virus. Each time a relatively new threat arise, AV vendors fall far behind the regular upgrade cycle. Before that the value of AV products for Ms Word macro virus protection was the same as  the value of a simple grep-style search utility, available for free from any good file repository ;-). 

Virus Bulletin used to provide decent analytical articles about complex viruses, but they deteriorated and recently are not that interesting to read.   Also they are ridiculously expensive and too closely connected with Sophos (that in 2003 acquired and partially destroyed ActiveState). 

This information vacuum sometimes creates an impression that underground virus-related publications are more reliable source of information about viruses than AV vendors and AV-related press ;-).   For example in case of Sircam worm no major AV vendors mentioned that it's cannot work on Windows 2000 and Windows NT and needs MAPI properly configured.

I would like to stress it again: any information from virus vendors should be viewed with skepticism due to inherent conflict of interests of AV firms. For people who are responsible for the corporate AV defense it pays to compare several sources of information especially if you do not have a sample of a particular virus and unable to analyses it yourself. Often AV vendors overestimate the danger and do not mention a limitations of a particular virus. Questions to be asked depend on the type of the virus. Generally it make sense to try to find an answers on the following questions: 

For macro viruses/worms the questions to be asked include: 

For mail worms additional questions are:

The real problem here is that until recently Windows executables were not signed (and macros also belong to executables -- they are just hidden in MS Word document format) and thus were not protected against tampering with MD5 checksums or similar. Both NT and Unix needs MD5 checksums for executables ASAP. Actually NT has such a capability (Authenticode) but it's not widely used. It looks like Windows XP make some positive steps in this direction but I do not know details.

All systems executables should be signed with MD5 based signature
or similar and their integrity should be easily verifiable

Do not jump into installing commercial AV tools each time you hear about a new nasty virus. In most case free scripts run via scheduler or via Netware login scripts (or similar mechanism) can be as good or even better. The same is true for the protection from email worms on the gateway level, but here free tools can serve complementary role; a commercial AV gateway filter has certain advantages. See my Overview of VB’97.

Moreover in commercial environment the loss leader is not viruses, its AV false alarms (or false positives as they are often called) -- despite AV-vendors  precautions they regularly appear and spoil the party. That mean that AV programs are much more close to Trojan Horse than one might suspect ;-) One self-quote would help:

Each day, most cases in corporate environment that are reported by customers that are not actual cases of virus infections. They are false alarms or false-positives.

The main problem here is AV products. Paranoid users and sometimes system administrators often blame on a computer virus effects that they cannot understand/explain.

In the past the main course of false positives in the corporate environment are such products as Inoculan, McAfee and F-secure.  The old version 4.0x sometimes recognized regular VBscript program as MS Word documents and produce messages like "Infection: 'Macro.Word97.Class.eb' [AVP]. Very distressing experience if one hundred remote and clueless users get this message in one day. Probably real virus would be less frustrating ;-)

Usually file viruses and macro viruses are reported as false positive, sometimes interesting combinations arise. For example now obsolete F-prot 2.xx   reported as infected any MS Word document that was disinfected by corresponding version of McAfee. That was very interesting experience if one part of the corporation uses F-prot and other McAfee ;-)

Another interesting combination arise if false positive detected but AV program unable to disinfect it and either renames it or put is some directory. That's when AV program became a real 100% pure Trojan Horse.

Inoculan is especially bad as it not only reports about files as infected when they are not, but moves them to the Infected directory. If configured that way Inoculan can be considered as a sort of Trojan Horse and may prevent installation of the packages on the network drive.

I do not want to go that far as to propose everybody move to Linux/FreeBSD (although they are not ideal and are susceptible to certain type of viruses and Trojans, they do provide a much better AV protection out of box than any flavor of Windows). All I ask for is: please do not just jump and install some over-hyped AV software package on Windows 2000/XP because a new virus or  worm was discovered -- usually that does not significantly increase the level of your AV protection. Try to use built-in mechanisms first, specialised tools second and generic AV scanners only as the last resort. the latter should be used in of-line scanning mode, not in "on the fly" mode to minimizes negative influences on other software. If you shoce to use "on the fly" protection use minimal set of extentions.

Still as  a successful new virus probably will use slightly different approach, not foreseen by AV vendor and signatures are always slightly behind the events, unless you use an automatic update system (and in this case risk troubles if update is buggy) Av scanner does not provide too much protection to count on. They are mostly cleaning tools for known threats. Understanding your environment is a better goal and other tools can provide multilevel protection which is always more affective that a single level, based on AV package.

Understanding your environment is a much more important goal
 than getting super AV protection. In large corporation all attachments with executable extensions (and probably zip files with names shorted then 10 letters) should be blocked on the mail gateway. That is much more efficient measure against mail worms that any AV scanner upgrade or signature distribution bonanza

Often in a corpoate environemnt a lot can be done with adequate polities. If for example all Word documents and executable should be zipped before sending you can completly block correcponing attachments and thus diminih or eliminate related threats. 

Upgrade is also an intereting alternative, especially for home users. for example Windows 200 users can benefit form upgrane to XP SP2. If you are not ready for such a radical solution ;-) in any case one should upgrade to Office 2003 that has better protection in Outlook against typical mail threats.

It is obvious that anti-virus vendors will always be playing catch-up with the virus writers. Theoretically if anti-virus software updates were released quickly and people instantly installed them desktop just patch protection might be adequate. This is however impossible. The life cycle of a virus looks something like (this a self-quote again):

  1. Virus is written, tested, possibly deployed on a test network (but usually not debugged for every Microsoft OS in existence; for example virus may not work on Windows 9x, if the only test platform was Windows XP, or if it was debugged on Windows 9x, or on Windows 2000.
  2. Virus is released on a selected target (university campus, Usenet groups, etc.)
  3. Virus (if "successful" in a biological sense) enter the stage of epidemics and spreads like wildfire, possibly causing damage (such as sending documents from folders on the hard drive, or even wiping motherboard BIOS chips ;-). Generally the more damage the virus cause the less changes are for its survival and the ability to reach the critical mass. 
  4. The first samples get to AV vendor (someone notices a strange activity detect the virus and sends it to an anti-virus vendor ). At this time the virus might not still reach the critical mass.
  5. AV vendor(s) analyzed, and possibly decompiled a virus and updated their product's signature file. Typically the anti-virus vendor share data with others, but they may or may not do this promptly. Anyway at this stage getting a virus sample is not a problem.
  6. Press informs users about the threat and how to fight it. The anti-virus vendors issue bulletins, make the update to the first buggy solution, etc.
  7. Large ISPs and some large corporations  install updated signature files and implement other defense measures fort of their mail gateways (or firewall in case of Ms.Blaster type of viruses)  then on the desktop  Some large customers with decent professionals or support contracts do the same, some have automated distribution systems for the update, resulting in a rapid deployment of the fix.
  8. Even home users start catching the virus. This is a start of a "chronic period" of the virus life when it still manage to infect some machines but the number is shrinking....
  9. Environment (for example version of OS or version of Office, etc)  changes to the extent that the virus is no longer is a viable threat. This is a clinical death. Complex viruses are more sensitive to env. changes and thus generally die much faster than simple one.

Please note that with some tuning for most email worms spam filters which can serve as a heuristic virus/worms protection tool.  That means that right now no home user  should access his ISP POP account with plain vanilla message client (like Netscape Messenger). One needs a spam filter either built as a POP retrieval tool or (in corporate environment and for Using/Linux) using message filter on the gateway level with additional spam filter installed. See Filtering Mail FAQ for more details.

Of course the volume of information about viruses and antiviral programs is well beyond the capabilities of any single person. So despite the fact that the author checked all the facts, one needs to access information provided critically. It can and probably does contain errors.

This is an introductory corporative AV strategy book, but not a "for dummies" AV book. This book presupposes some level of understanding of the virus problem, assembler language and operating system and applications used by viruses. It is partially based on my Russian language book Computer Virology that was published in 1991, but I am too lazy to translate all this old material and this book is considerably shorter and more superficial that the old one. Due to the lack of objective/skeptical information about antivirus protection I feel that it make sense to publish all my notes "as is".  I just hope that some this information and the angle of attacking the problem might be useful.

 

Dr. Nikolai Bezroukov


Webliography



Copyright © 1996-2008 by Dr. Nikolai Bezroukov. www.softpanorama.org was created as a service to the UN Sustainable Development Networking Programme (SDNP) in the author free time. Submit comments This document is an industrial compilation designed and created exclusively for educational use and is placed under the copyright of the Open Content License(OPL). Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

Standard disclaimer: The statements, views and opinions presented on this web page are those of the author and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

Last modified: February 28, 2008