|
|
Home | Switchboard | Unix Administration | Red Hat | TCP/IP Networks | Neoliberalism | Toxic Managers |
| (slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and bastardization of classic Unix | |||||||
| News | Recommended | Debugging | Random Findings | Humor | Etc |
|
|
Reader beware. It's extremely difficult to find a decent book on testing and here I have no recommendations yet. One reader put it about one particular book, but what looks like pretty generic advice Although it is not software testing, I would recommend the book "Refactoring--improving the design of existing code" by Martin Fowler. It is a easier read and you can get more out of it immediately than with this one.
|
|
One more advice: just beware words "object-oriented" in title. Your chances to get the product of a snail oil salesman are really high for such titles ;-)
The main danger for a programmer is to buy a misleading book for managers. An anonymous Amazon reviewer put it very well:
Misleading book for managers.
Reviewer: A reader from Seattle, WA August 10, 1999
This book is entirely management with no technical content. Worse, the authors seem to think they know the technical side, and thus may mislead readers into thinking that they understand it as well. A major problem in software testing is that it is typically practiced by people who can't program and can't do math, and managed by people who not only can't do either, but don't realize that both skills are important to effective testing.
You need to be careful and do not blindly trust Amazon reviewers in this category of books. I really and absolutely sincerely do not recommend the book The Art of Software Testing. I consider it to be a pretty mediocre outdated book the has a really ridiculous price. Now try to read Amazon reviews for the book ;-). If this is not a "lemming effect" than I do not know...
This list is in early beta and is just for information.
|
You can use Honor System to make a contribution, supporting this site |
- Paperback: 384 pages ; Dimensions (in inches): 0.90 x 8.64 x 7.92
- Publisher: Manning Publications Company; (October 2003)
- ISBN: 1930110995
- Average Customer Review:
Based on 3 reviews. Write a review.
- Amazon.com Sales Rank: 2,625
Every programmer should own this book, December 17, 2003
I think we can all agree that unit testing is a good thing that can be painful to implement. The main problem with unit testing is building tests that are easy to run and maintain. That is where JUnit enters the picture. JUnit simplifies the task of creating unit tests and this book simplifies the task of understanding and using JUnit.
Reviewer: Thomas Paul (see more about me) from Plainview, NY USA The book starts with the basics of using JUnit but then moves on to explaining the intricacies of JUnit. UML diagrams clearly show the flow of control within JUnit and your unit tests. The author gives a good description of how to integrate JUnit with Ant, Maven, and Eclipse. The book also shows how to do unit testing of Servlets and JSPs which can be much more complex to test. The author demonstrates both the use of Cactus as well as the use of mock objects for J2EE testing. In fact, the best part of the book is that the author helps you determine how to decide what type of testing strategy is right for your application.
I thoroughly enjoyed this book and found the author's comments clear and insightful. The author supplies plenty of code samples with explanations of not just how but why we do our testing a certain way. The author's own experience is passed on to us with his "best practices". Anyone who is writing Java code should read and understand this book. The quality of the code you produce will vastly improve.
When I first started to read JUnit in Action, I was hoping that it wouldn't be a tutorial on the open source tool JUnit. I am glad to say that it is much more. I think the book's name could really be "Testing in Practice". Sure, JUnit is covered in a lot of detail, but so are other tools such as:
Reviewer: Dion G Almaer (see more about me) from Cambridge - Integration with: Ant, Maven, and Eclipse
- Mock Objects (via both EasyMock and DynaMock)
- Cactus for testing in a container
- And other small helper tools (nice ant tasks, etc.)What made me really enjoy this book is the way it is written, coupled with the practical look at the many technologies involved in testing. It is a fresh read, that doesn't get bogged down. The book flows really well, giving you best practices throughout. They don't just say "Do X", they actually show you where these best practices come from as they refactor their own code. You are really aware that these authors know their stuff, and are drawing from a lot of experience (compared to the online FAQs).
Testing Applications on the Web Test Planning for Internet-Based Systems by Hung Q. Nguyen
|
Grey box testing is based on a general understanding of a system's architecture and components. This understanding drives test strategy and identifies opportunities to test components in isolation.
Reviewer: Bret Pettichord (see more about me) from Austin, Texas The shade of grey can vary from white box testing (full review of source code) to black box testing (no review of source code). You choose what level of information to gather depending on your budget, capabilities and judgment.
This book provides the first detailed approach to grey box testing, focussing on web-based application architectures. These architectures are based on a heavy use of components: application servers, web servers, load balancers, databases and the like. This book describes these components, suggests how they can fail and what you can do to anticipate, trigger, or detect such failures.
This approach is supported by the author's extensive experience testing web-based (and other) applications as president of a software testing company. It is augmented by plenty of good advice on how to communicate test results clearly.
Testing Computer Software, Second Edition
by Cem Kaner, Jack Falk, and Hung Quoc Nguyen
reviewed by Diomidis Spinellis
by
John Robbins
Our Price: $39.99
Paperback - 466 pages Bk&Cd
Rom edition (January 26, 2000)
Microsoft Press; ISBN: 0735608865
; Dimensions (in inches): 1.37 x 9.23 x 7.35
Amazon.com Sales Rank: 5,640
Avg. Customer Review:
Number of Reviews: 10
 |
Do you aspire to be a serious developer? If so, read this bk |
March 11, 2000 |
|
| Reviewer: Osiris Pedroso (see more about me) from SF Bay Area | |||
| John Robbins did a great job on this book. I was very impressed
both on the depth and breadth of the issues treated.
At first, I though on skipping the chapter on Visual Basic debugging, since I am strictly a VC++ developer, but I am glad I did not. Even that chapter gave me insights that I can use on my day to day, for example, when he runs the VB compiler while in the debugger and is able to see how VB uses the C compiler's code generation (second pass). Some great insights and lots of good example on how to resolve problems and how to use all the capabilities of the debugger to ones best advantage. It will teach you what to do when the debugger gets you to a source line that before that line, everything is working properly, after that line the world has turned upside down. It will teach you enough X86 assembly to make you dangerous and be able to read between the (source) lines in the process. Even if you thought you knew it already. Could it be improved ? Yes, can't it always ? Coverage on tracking memory leaks could be expanded, for example, to cover MFC's shortcomings when reporting them, but this book is a close to perfection as I have seen them. And it is a great read too. His style is easy to follow, even though some of the subjects are deep and complex, but John transfers the knowledge so easily, it is amazing. Once I completed my first read, I really felt like I had just finished listening to a very good rendition of a Beethoven or Mozart simphony. Every developer that aspire to be a serious developer should read it and reread it. And thank you, John Robbins. I will be buying every book you write. |
|||
|
|
|||
| 7 of 8 people found the following review helpful: | ||
 |
Good, but could be even better |
March 5, 2000 |
| Reviewer: EJ Bartelds from Rotterdam, The Netherlands | ||
| When I first heard John Robbins was writing a book on Win32
debugging, I was delighted. I've been a fan of his MSJ Bugslayer articles
since the beginning, and John's debugging knowledge, displayed in those
articles, has helped me tremendously.
However, for someone who has read all his MSJ work, this book is a bit of a disappointment. The reason is that the second part of the book is a collection of his (slightly-rewritten) MSJ articles, with almost no new content added as far as I can see. The first part of the book, however, is worth every dollar, as other reviewers have already mentioned, even though I was missing coverage of the WinDbg debugger, and MS tools such as userdump. Maybe in a second edition? To summarize, I suspect this book to be a 5-star for anyone who is fairly new to debugging and has not read John's MSJ columns. For others, who have been exposed to his columns, and have some experience, I'd rate this book 3-stars. |
||
A useful book, not only for automated testing, March 2, 2000Mark Fewster and Dorothy Graham Software Test Automation Addison Wesley, 1999 ISBN 0-201-33140-3
A book for beginners in test automation. Everything you always wanted to know about test automation, but never dared to ask, and the tool suppliers probably wonуt tell you.
This book is a must for every serious test manager. It is not only about automation. It is really about test architecture and test suite design. Most other books are about how to design test cases, or how to manage testing. This book is about designing the whole stuff into such a form that you can maintain it, that it will survice maintenance, and that you can automate it. The background is test automation. Many examples show the testware design pitfalls in building automated tests. But it hit me how useful the book is to any testing effort, even manual testing.
The book has two parts. Part 1 is a detailed guide to designing automated (or not totally automated) tests. Questions of selecting and introducing test tools are also discussed. Part 2 contains 15 guest chapters, written by people who have experience in test automation, sharing their experience. These chapters vary in style and are rather short.
There is a readers guide, It shows what te read if you have different intentions and know more or less from before.
Overview of part 1, the Test Automation Design Part.
Chapter 1 gives the context of test automation. It places the different tools into a test model and warns us of naive belief in and application of automation.
Chapter 2 shows the results of capture and playback. At the end of this chapter, the reader will have understood the limited scope of application of this. Problems with script robustness and automatic results comparison are shown.
From chapter 3, the focus changes to scripting. The chapter presents the five levels of generating test scripts, from linear through structures, shared, data-driven to keyword based. The reader will soon see that investment in scripting can pay off. The benefits of data driven or keyword driven testing are clearly shown. The only negative comment is that keyword driven testing, the most advanced and modern approach, is not shown to the same detail as the other techniques. However, these techniques are presented by authors of methodologies and real users in chapters 22 to 25.
Chapter 4 shows the practical details of automated verification of test results. There are lots of methods and pitfalls. The main possibilities are dynamic verification (during running the tests) and static verification (afterwards. The other option is how much to verify. In principle, everything may be interesting. But this costs both work time, maintenance effort, storage, and makes tests less robust and more proine to maintenance. An interesting alternative is to design a broad smoke test where as much as possible is captures and verified, and then a large main test, where every test only vewwrifies its main objective. The authors also show different methods for comparing рexoticс output. One objection is the shallow section on testing embedded systems. At least for telecom systems, but also other рstandardс technical interfaces, lots of commercial tools exist that can be used to verify output in a problem oriented and intelligent way. There should also be some more mention about verifying time correctness of system outputs. There might also be some more focus on the use of check sums, statistical selection out of large amounts of data etc.
Chapter 5 shows a possible architecture of the test set. This chapter applies just as well to manual as to automated testing. It fills a hole in IEEE Standard 829, namely proposition as to how to structure the detailed test material, the scripts, input data, utilities, expected results, and actual results, togehter with all other information. The main idea behind the proposed directory structure and naming conventions is to make it easy to maintain and find through. It should also be easry to copy any needed test sets to a platform where they are needed.
There could be more different approaches, naturally, but the presented approach would work well and scale up. There could also be a cross reference to what IEEE 829 contains, and some example about how to use internet technology to document test structure, but all this is of minor disadvantage. This chapter is valuable to anyone who needs to structure test material.
Chapter 5 and 6 are a bit special, as they are as useful for automated, as for nonautomated testing. They concern the structure of the testing material, all the invoilved files, and the preprocessing and postprocessing. These chapters fill a hole in IEEE Standard 829, where no consideration is given to the practical details of building up and maintaining test material. A tool independent structure is given, and advice about how to connect the material with tools. Chapter 6 is an eye opener to the naive user of test automation tools: Preprocessing and postprocessing tasks must be automated, but are not in the scope of the commercial tools.
Chapter 7 presents other issues about test suite maintenance and maintainability. This is somehow good old material and knowledge. It one uses the good old rules for structured design and programing, and scheduled cleaning up of programs, the same could be applied to test amintenance and test material standards. Chapter 7 serves as a useful reminder about what you think you already knew, but maybe forgot in the concrete case.
Chapter 8 gives ideas about how to measure both testing and automated testing. For a metrics fan, this chapter is too short. It only outlines lots of examples of what can be done and why, but does not go into detail. On the other hand, there is enough literature elsewhere about this. Many of the proposed measurements are not very precise, but good advice to гthe poor manх who has not measured much and has to fight for any resources to measure anything. The idea is: Anything which is of interest can be made measurable in some way, maybe not the ideal way and maybe not precisely. But any measurement is better than none. The chapter does a useful job in that respect.
Chapter 9 is a collection of points that have no natural place elsewhere. It includes short discussions on what to automate and what not, how to select tests, the order of running them, tricks to minimize time for failure analysis, deciding the status of a test. Most of the points are well known and not discussed in much detail. A good section is about test status, where the authors present some good ideas about how to handle known bugs in the products that are not going to be fixed for a while (and failed tests for such bugs).
Chapter 10 and 11 concern selecting and introducing a test tool. Much of these chapters are general and could be read by anyone having to select and implement any software tool. What I am missing here is more specific information about concrete test automation tool features to ask for. The general questions to tool vendors are useful, but more concrete advice special for test automation tools is lacking. A few concrete references to test tool functionality overviews would have been helpful here.
In all, I found part 1 very useful, and well worth the time spent reading.
Part 2 is different. There are case studies about successful introduction of test automation, the problems solved, the benfits achieved, and the difficulties met. Chapter 12 to 18, as well as 20, 21, 23 and 25 to 28 are about successful cases. Most of these are short and regrettably general in nature. Not easy to read concrete advice for your own situation. The most interesting one was chapter 28, describing seveal unexpected uses of automated tests at Microsoft. Chapter 19 is a study about test automation failures, and this is useful. Probably it is always more easy to learn from failures. Chapters 20 and 22 are very special, they both describe keyword driven test automation design methods. Chapter 20 is about the action word method, now called TestFrame, by Hans Buwalda from CMG, whereas chapter 22 is about RadSTAR. Both chapters are useful, as they describe the most modern approach to maintainable test design. However, chapter 22 could be more detailed.
In all, I found less use for part 2 of the book, probably because I have heard some of the stories before, at conferences. It may be different to a person new to the field. I was missing more case studies about atuomated tests of other than business systems. There is a lot of automation to be done in this field, and very often there is no alternaitve to automated testing.
In all: Buy the book. In the worst case, chapters 5 and 6 only create benetitfs enough to make it worth the money.
Hans Schaefer
This is simply the Very Best Book. Period., June 13, 2000
Reviewer: A reader from Nashua, New Hampshire
Anyone who claims to be a Principle or Senior Software QA Engineer, or a QA
Automation Engineer, but has never read or used this book should be convicted
of professional malpractice.
Absolutely the best current reference book on why and how to analyze, architect, design and construct verification and validation vehicles that dramatically improve the probability of software defect detection - both manually and through later automation. Said vehicles can be easily audited and inspected to improve positive and negative software behavior coverage and equivalence classes. But most important, this great book describes how intelligently devised manual vehicles can be used to automatically drive future software builds and release regression assessments (to achieve automated regression testing).
Describes everything that the software test tool vendors refuse to tell buyers, but what every software QA professional must know (in spite of the vendors).
This book is simply a breath of fresh air.
It is a virtual bible to positive and negative software behavior verification and validation. My only displeasure is that the authors choose to continue to use obsolete terminology: specifically testing. Software testing is not what this book is about. This book is about how to create and achieve highly effective software behavior verification and validation.
Software QA professionals must clearly move way beyond ad hoc testing, because the need today is far greater than ancient testing paradigms. The need is for intelligently designed, architected, constructed frameworks that enable software behavior and performance verification and validation, whether performed manually, automatically, or through agents such as monkeys and oracles.
|
Misleading book for managers. |
|
|
Reviewer: A reader from
Seattle, WA
August 10, 1999 |
|
| This book is entirely management with no technical content. Worse, the authors seem to think they know the technical side, and thus may mislead readers into thinking that they understand it as well. A major problem in software testing is that it is typically practiced by people who can't program and can't do math, and managed by people who not only can't do either, but don't realize that both skills are important to effective testing. |
Best of Breed, October 24, 2000
Reviewer: Steve Paulovich (see more about me) from Boston, MA
As a Software Quality Assurance manager with almost 20 years experience in the field (and automated testing experience dating back to the DOS days) I highly recommend this book.Dustin, Rashka, and Paul have managed to articulate the best practices I've used throughout my SQA career. The ATLM (Automated Test Lifecycle Methodology) they describe in detail is valuable not only in successfully planning an *automated* testing project, but in the successful development of an overall testing strategy.
If you are looking for a deeper understanding of successful software quality assurance, especially automated testing, this should be the FIRST book you buy. The information is detailed enough that this may be the ONLY book you need to buy.
automated software testing, March 21, 2000
Reviewer: alen mitrovic from Ljubljana, Slovenia
This is excellent book. i use to cal it Automated Software testing bible. It's usable in practice. It helps you to develop your own strategy for automating tests in your own test environment. Guidelines presented in this book are real ones, and easy transferable to real Test world. Best book I have ever read about test automation.
Reviewer: Yuri Weinstein
If you think "Automation" you should get this one - "Automated Software Testing" book. Being a professional automation engineer for last 8 years, I am glad to see that this type of books start making their way to customers. Different automation approaches, example criteria are very helpful!
Reviewer: M.E. HOM from San Francisco
Software Testing is all about methods and technology- 80% methods, 20% technology. I find the methods that are listed in this book to be sound. ...Other topics (i.e., test project management, test cycle, test plan, etc) that are also listed in this book are well explained.
There isn't that many great books in automated software testing or general software testing .
However, the delinated concepts in this book and Graham's book (Software Test Automation) are well-written good. After reading these books, I feel most readers (beginning and intermediate SQA professionals) could have a better sense of what Automated Software Testing is about. Recommend this book very highly.
|
|
|
|
|
Okay Survey Book but Caveat
Programmer! |
|
|
Reviewer: A reader from
Mt. Clemens, Michigan, USA
April 17, 1999 |
|
| I can't help but think of
a scene from the movie "Casablanca" when I think of this book. In the
scene, Louie, the prefect of police, asks Rick what brought him to Casablanca.
"I came for the waters," Rick says. "But there are no waters in Casablanca,"
says Louie. "I was misinformed," Rick says dryly.
I, too, was misinformed about this book. From the reviews that I read, I thought that, even though the book was designed primarily for testers, a programmer could get a lot out of it. This, by and large, isn't true (unless you're a programmer for a software company). There were about four chapters in this book that were worth something and even these were more of a survey of stuff from other books rather than a comprehensive treatment. Unless you're a professional tester, find yourself another book. --This text refers to an out of print or unavailable edition of this title. |
|
|
|
|
|
Bad book, Bad
Reviewer: A reader
from California, USA
March 14, 2000
Up to now, I have been writing and designing software
have just started to look into software testing. Since the only review said
this was a good book for programmers, I bought it. I did not like this book.
It is very dry, sterile writing. The contents is too academic--lots of
theory and not anything which I could pick up and say "That's a good idea and
I can use that right away". The book covers topics such as Test requirement
checklist, test specifications, test suites, improving testing, etc... If you
already have a large test infrastructure in place then this book 'might' help
you understand some of that process (or not). If you don't and want to start
one then, this is not the place to go. Although it is not software testing,
I would recommend the book "Refactoring--improving the design of existing code"
by Martin Fowler. It is a easier read and you can get more out of it immediately
than with this one.
Society
Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers : Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism : The Iron Law of Oligarchy : Libertarian Philosophy
Quotes
War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda : SE quotes : Language Design and Programming Quotes : Random IT-related quotes : Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce : Bernard Shaw : Mark Twain Quotes
Bulletin:
Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 : Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law
History:
Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds : Larry Wall : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS : Programming Languages History : PL/1 : Simula 67 : C : History of GCC development : Scripting Languages : Perl history : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history
Classic books:
The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month : How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite
Most popular humor pages:
Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor
The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D
Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.
FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.
This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...
|
|
You can use PayPal to to buy a cup of coffee for authors of this site |
Disclaimer:
The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.
Last modified: March 12, 2019
