博客园  :: 首页  :: 新随笔  :: 联系 :: 订阅 订阅  :: 管理

预测测试的未来

Posted on 2005-05-10 15:58  Jackei  阅读(2868)  评论(5编辑  收藏  举报

不记得在哪里看到这篇文章的英文原文和中文译文了。从内容上看,应该是在2003年末写得,不过内容还是有很多值得我们思考的。在这里对作者和译者表示致敬。


 

Predicting the Future of Testing

By Harry Robinson

 

Summary: As the end of the year approaches, psychics and pundits alike will start making their predictions about what's in store for us in 2004 and beyond. In this week's column, industry veteran Harry Robinson gives us his forecast on the future of software testing.

 

"It is tough to make predictions, especially about the future."—Yogi Berra

 

Every December, tabloid fortune-tellers reveal what will happen in the coming year: “Madonna will fly on the space shuttle,” “the U.S. capital will move to Wichita, Kansas” and so forth. I’d like to jump on that bandwagon and make a few predictions of my own about where software testing is going in the next few years. And I can only hope my prophecies fare better than those of my esteemed tabloid colleagues!

 

My main prediction is that software testing in the future will look very different than it does today. My reasoning is straightforward: Software testing largely stinks today. It comes into a project too late, contributes too little, and costs too much. If we care about the quality of our products and the health of our bottom lines, we need to re-think our approach to testing and quality.

 

I will even go out on a limb and say that better methods, better training and a better appreciation for testers will revolutionize the software industry. To be specific, technologies such as executable specifications, model-based test generation, bug prevention, and system simulation will play important roles in the unfolding drama.

 

Here are some scenes we will see in the industry over the next few years. In fact, some of these trends are starting to emerge already.

 

Testers, Spec Writers, and Developers See Themselves as Partners

 

Testers Help Spec Writers 

Testers work with spec writers to review and understand the spec as it is being written. Early review and modeling exposes many consistency, completeness, and ambiguity bugs while they are still cheap to fix.

 

Spec Writers Help Testers

The test team creates models to generate tests of its applications' behaviors. Spec writers review the models to ensure that they adequately cover the feature set. The resulting test model becomes an "executable spec."

 

Testers Help Developers

Because the spec is clean and unambiguous, the developers understand better what their code should accomplish. Testers provide lightweight models that developers can run against their code before they officially hand it over for testing. 

 

Developers Help Testers

Proceeding on a feature-by-feature basis (to avoid the late-in-the-cycle, product-pounding approach of the past), developers and testers ensure that the code is easy to test with automated methods. Developer's code is now full of testability hooks, making errors more detectable.

 

Testers Help Testers

Tests are now modeled in a high-level language, so teams working on other features (and even other products) can help review and improve the test models. This creates a community of test generation experts.

 

Methods and Metrics Get Better

 

Bug Prevention and Early Detection

Because the emphasis is now on the quality of the deliverable (and not on how many bugs you found along the way), prevention practices and detection tools, such as static analyzers, are mainstream.

 

Simulation in Testing

Simulation tools become common, making it easy to “fake” computer environments. Testing for exceptions and error paths now happens early in the development process. After the code has stabilized, real environments verify that the simulations were accurate.

 

Just-in-time Test Cases

Massive test case management systems are a thing of the past. Most tests are generated on the fly. Test cases are no longer stored away like rotting inventory, so it is easy to keep tests up to date. 

 

Positive Metrics

Misleading metrics, such as bug counts and test case counts, are dead. Useful metrics, such as spec coverage, model coverage, and code coverage, drive the projects.

 

Fewer Testers, Better Testers

Machines now perform much of the mundane work that testers previously did creating tests. Teams require fewer testers, and the testers who remain are more highly trained. Their work is more interesting to them because they are focused on bigger issues in their tests rather than slogging through grunt work.

 

More Tests, Better Tests

Testers can now generate millions of tests on any day, so the challenge becomes how to run the most useful tests first. Combinatorial tools allow testers to prioritize their testing and aim their test runs at the areas most likely to have significant bugs.

 

Roles Will Change for Testers

 

Distinctions Blur in Testing

Work in the testing field blurs the line between people who only specialize in hands-on testing and those who only create test tools. A new specialty emerges that encompasses both "testers who like to break things via programs" and "programmers who like to create programs that break things" – people in newsgroups debate endlessly about what to call the new specialty.

 

Boundaries Blur Between Testing and Development

Testers and developers work in tandem to produce testable, high-quality code. Testers help iron out spec issues to make the developer's job easier, and developers create cleaner, more testable code to make the tester's job easier.

 

Feedback from Customers Becomes Integrated with Testing

The quality of deliverables becomes higher. Testers routinely conduct root cause analyses. We ask questions such as "How did we miss this bug?" and "How can we prevent this type of bug in the future?" We work to delight our customers.

 

New Challenges Emerge

The sophisticated and interconnected environment of the computing world guarantees that new problems such as security testing continue to keep testers running hard. This is OK—testers find these challenges invigorating.

 

Testers Get Respect

Testers are no longer called in at the last moment to "pound on the product." They provide a visible, vital, value-added service throughout the software development process. People realize that testing can be rewarding, interesting, and even fun. 

 

Testing Gets Trendy

Software testers start to hold their heads high. And, because breaking stuff is at least as much fun as building it, people begin to rotate between positions in development and testing. Everyone learns more about what makes good code.

 

Adrenaline Junkies Move On

The new process works so well that spec writers, developers, and testers end up having lives. This is disconcerting to some who were raised in the adrenaline-charged world of late-night, last-minute, firefighting sessions. These people gravitate to companies that remain out of control.

 

Elvis Presley Is Discovered Working as a Software Tester

The giveaway is his conference paper titled “Software Quality: It’s Now or Never”. 

 

Prepare for the future today

 

Whether or not my predictions come true, the future is on its way. Here are five ideas for how you can prepare to meet it:

 

1. Get Actively Dissatisfied

Don't accept the current state of testing. Look around and think, "What are we doing that makes no sense?"

 

2. Push the Envelope

Figure out how to test better and share that knowledge. Overall quality will improve only if everyone seeks to make the code they are working on the best it can be.

 

3. Learn More about Testing

At this moment, the industry is exploding with innovative software testing ideas. Go to conferences, join mailing lists, and scour the Web to see what is happening on the cutting edge of testing.

 

4. Learn More about Development

Take a programming class. Even if you don't plan to write significant amounts of code, view the class as a reconnaissance flight over bug territory.

 

5. Change the World!

As PC pioneer Alan Kay said: "The best way to predict the future is to invent it."

 

For more info:

 

On executable specs: “Executable Specifications: Creating Testable, Enforceable Designs”

 

On model-based testing: “Intelligent Test Automation”

 

On system simulation: “Software’s Invisible Users”

 

About the Author Harry Robinson is Test Architect for Microsoft's Enterprise Management Division. In addition to his day job, he teaches classes on advanced software test automation and is a driving force behind Microsoft's model-based testing initiative. He has been at Microsoft for five years and has a Bachelors and Masters in Electrical Engineering. You can reach him at harryr@microsoft.com.


 


 

测试未来的预测

By Harry Robinson

 

摘要:一年将尽,心理学家或者一些博学者们,又将对2004年或者更久的将来作出预测。在这次的周末专栏中,Harry Robinson将向我们讲述他对测试未来的预测。

 

 “预测是件很难的事情,尤其是预测未来” —Yogi Berra

 

每年十二月,小报的“未来预测者”们会向大家切揭示即将到来的一年将要发生的事情:“麦丹娜将要乘坐航天飞机,“美国将迁都 Wichita”,等等。我将加入这个潮流,对软件测试何去何从做一个我自己的预测。并且我希望,我的预测费用能够比我的那些值得尊敬的小报同事更高些。

 

我的主要预测就是,将来的软件测试与现在的软件测试看起来很不一样。原因很直接:今天的软件测试很大程度上是臭名昭著的:软件测试参与到项目中的时间太晚、贡献太少、花费太高。如果我们关心我们产品的质量以及我们的账本底线的话,我们就需要重新思考测试和质量的方法。

 

即使遭到一致反对,我也要说:更好的方法,对测试人员更好的培训、更好的欣赏将改革软件产业。具体地说,诸如可执行的说明书、基于模型的测试产生、BUG预防、系统模拟这些技术,将在这场演变过程中扮演重要的角色。

 

下面就是我们在将来的几年里可能看到的情形。事实上,某些趋势已经开始了。

 

测试人员,需求撰写人员和开发人员,都将看到自己是其中的一份子。

 

测试人员帮助需求撰写人员

测试人员与需求撰写人员共同工作,在需求完成以后,审查以及理解需求。早期的审查以及建模可以暴露很多关于一致性、完整性和模糊性的BUG,这个时候修补这些BUG付出的代价还十分小。

 

需求撰写人员帮助测试人员

测试小组建造模型,用于产生对其产品行为的测试。需求撰写人员审查模型,以确保他们充分覆盖了产品特征集。这样产生的测试模块将成为一个“可执行需求”。

 

测试人员帮助开发人员

因为需求清楚,毫不含糊,开发人员更好的理解了他们的代码将要完成什么。

在正式的将代码提交做测试之前,测试人员提供给开发人员一些模型,以便开发人员可以在自己的代码中实现它们。

 

开发人员帮助测试人员

基于特征对特征这样的方式(防止以往的“后期才介入开发,一股脑找出产品问题的方式),开发人员和测试人员共同保证代码易于实施自动测试.开发人员的代码中处处都是易测试性的开关,使得错误检测更加容易.

 

测试人员帮助测试人员

测试用一种高级语言来模拟,因此别的特征的测试小组(甚至别的产品的测试小组)可以复查和改进测试模型.这就形成了一个测试专家的共同体.

 

方法日趋完善

 

BUG预防和早期检测

因为现在把重点放在产品交付的质量上来了(而不是在于找到了多少BUG), 预防实践和静态分析仪这样的检测工具将成为主流.

 

仿真测试

仿真工具变得很普遍,使得仿造计算机环境变得容易起来.在开发过程的早期就可以进行意外和错误流程的测试.代码稳定后,再用真实环境验证仿真是否准确无误.

 

及时的测试用例

庞大的测试用例管理系统将成为昔日的东西,大量的测试用例生成了却没有被使用.测试用例将不再像腐烂的存货一样被收藏起来,因此,让测试用例保持最新变得容易起来.

 

积极的方法

误导人的方法,比如计算BUG的数量、计算测试用例的数量,将不复存在.有用的方法,比如需求覆盖、模型覆盖、代码覆盖将驱动项目开发.

 

更少更精的测试人员

机器将代替测试人员做大部分他们以往创建测试所做的繁琐工作,测试小组需要比以往更少的测试人员,留下来的测试人员将是经过更多高度培训过的.他们所做的工作将更加有趣,因为在测试中他们将致力于更大的问题,而不是在抱怨中艰难地开展工作.

 

更多更好的测试

测试人员将可以在一天中进行成千上万的测试,所以,如何首先运行最有用的测试将成为一大挑战.相关的工具将允许测试人员为他们的测试区分优先级,以及将测试目标放在那些最易出现重大BUG的地方.

 

测试人员的角色更换

 

测试中界限模糊

在测试领域工作使得专职测试的人员和专职创建测试工具的人员界限模糊,一个既是“通过程序破坏事物的测试员又是创建程序用于破坏事物的程序员的专业出现了,――关于如何称呼这个新的专业,新闻圈内的人们还在进行着无休止的争论。

 

测试与开发界限模糊

测试人员与开发人员一前一后,共同创造可测试的、高质量的代码。测试人员帮助开发人员消除需求中的问题,使得开发人员的工作更易完成,同时,开发人员写出更清晰、可测性更高的代码,使得测试人员的工作更易完成。

 

顾客反馈与测试合为一体

交付的产品质量更高。测试人员进行根本原因的分析,我们会问比如“我们怎么会遗漏了这个BUG呢?”或者“我们将来如何防止这类BUG?”这些问题,我们的工作就是使顾客满意。

 

新的挑战出现

复杂和相互关联的计算机世界使得了测试安全这一类的新问题让测试人员不断努力工作,但这没关系――因为这些挑战使测试人员精力充沛。

 

测试人员获得尊重

测试人员将不再是在最后时刻才被叫来“对产品狂轰烂炸”,他们将在整个软件开发过程中提供一个可见的、重要的、增值的服务。人们意识到,测试是有益的、有趣的甚至富有乐趣。

 

测试变得流行

软件测试人员开始扬眉吐气,而且,由于破坏事物至少可以带来创建事物一样的乐趣,人们开始在开发和测试角色之间转换,所有的人将学到更多关于如何得到良好代码的知识。

 

激情“吸毒者”继续存在

新的过程运行得如此良好,使得需求撰写者,开发人员以及测试人员不再具有生命力,这就使得那些在激情掌控的世界被提升的人惶惶不可终日,那样的世界意味着工作到深夜、最后一刻测试才参与,以及如同交战开火般的会议。而这些人对于那些还没有受新的运行过程控制的公司来说还具有吸引力。

 

Elvis Presley是一个软件测试员

他的会议发放材料的标题就是:“软件质量:就是现在,否则永远不可能”

 

 

今天就为将来准备

不管我的预测是否成为现实,未来也会按照它自己的方式到来,下面就是如何准备面临未来的五个意见:

1.积极地不满于现状

不要接受测试的现状,四处看看,并且思考“我们在做些什么毫无意义的事情?”

 

2.抛开人与人之间的封闭

领悟如何更好的测试,并且分享这些知识。只有每一个人都试图使他所写的代码达到最佳状态时,整体质量才会改进。

 

3.学习更多关于测试的东西

如今,行业受软件测试的创新思维激发。用参加会议,加入邮件列表,网上冲浪,这些方式来解在测试前沿发生的一切。

 

4.学习更多关于开发的东西

参加一个编程学习班,即使你不打算编写大量的代码。将学习班当作是在BUG领土上的一次侦察飞行。

 

5.挑战世界

正如PC先驱Alan Kay所言:“预测未来的最好方式就是开创未来”