bigpo.ru
добавить свой файл
  1 2 3 ... 8 9

Part 1: Definitions (5 points each)


Definitions should take 2-3 minutes each. In drafting the exam, I allow about 3 minutes per definition.

  1. Domain testing

  2. Equivalence class

  3. Boundary condition

  4. Best representative

  5. Fault vs. failure vs. defect

  6. Function testing

  7. Regression testing

  8. Specification-based testing

  9. Power of a test

  10. Public bugs vs private bugs

  11. Prevention costs

  12. Appraisal costs

  13. Internal vs. external failure costs

  14. Oracle

  15. Exploratory testing

  16. Waterfall lifecycle

  17. Lifecycle model

  18. Evolutionary development

  19. Line (or statement) coverage

  20. Boundary chart

  21. Software quality

  22. Black box testing

  23. Glass box / white box testing

  24. Risk-based testing

  25. Corner case

  26. Finite state machine

  27. Stochastic testing

  28. Dumb monkey

  29. State

  30. Combination testing

  31. All-pairs combination testing

  32. Input constraints

  33. Storage constraints

  34. Smart Monkey

  35. State variable

  36. Value of a state variable

  37. Testing project plan

  38. Test matrix

  39. Manual test script

  40. Attribute to be measured

  41. Surrogate measure

  42. Defect arrival rate

  43. Defect arrival rate curve (Weibull distribution)

  44. Stress testing

  45. Computation constraints

  46. Output constraints


Part 2: Short Answers (10 points each)


Short answers should take about 5 minutes to answer. In planning the timing of the exam, I allow about 6 minutes per short answer question.

  1. Give two examples of defects you are likely to discover and five examples of defects that you are unlikely to discover if you focus your testing on line-and-branch coverage.

  2. Give three different definitions of “software error.” Which do you prefer? Why?

  3. Ostrand & Balcer described the category-partition method for designing tests. Their first three steps are:

      1. Analyze

      2. Partition, and

      3. Determine constraints

Describe and explain these steps.

  1. A program asks you to enter a password, and then asks you to enter it again. The program compares the two entries and either accepts the password (if they match) or rejects it (if they don’t). You can enter letters or digits. How many valid entries could you test? (Please show and/or explain your calculations.)

  2. A program is structured as follows:

    • It starts with a loop, the index variable can run from 0 to 20. The program can exit the loop normally at any value of the index.

    • Coming out of the loop, there is a case statement that will branch to one of 10 places depending on the value of X. X is a positive, non-zero integer. It can have any value from 1 to MaxInt.

    • In 9 of the 10 cases, the program executes X statements and then goes into another loop. If X is even, the program can exit the loop normally at any value of its index, from 1 to X. If X is odd, the program goes through the loop 666 times and then exits. In the 10th case, the program exits.

Ignore the possibility of invalid values of the index variable or X. How many paths are there through this program? Please show and/or explain your calculations.

  1. Consider a program with two loops, controlled by index variables. The first variable increments (by 1 each iteration) from -3 to 20. The second variable increments (by 2 each iteration) from 10 to 20. The program can exit from either loop normally at any value of the loop index. (Ignore the possibility of invalid values of the loop index.)

    • If these were the only control structures in the program, how many paths are there through the program?

      • If the loops are nested

      • If the loops are in series, one after the other

    • If you could control the values of the index variables, what test cases would you run if you were using a domain testing approach?

    • Please explain your answers with enough detail that I can understand how you arrived at the numbers.

7. List and briefly explain three strengths of the waterfall lifecycle.

  1. List and briefly explain three strengths of the evolutionary lifecycle.

  2. Describe the characteristics of a good scenario test.

  3. List and explain four claimed strengths of manual scripted tests and four claimed weaknesses.

  4. List (and briefly describe) four different missions for a test group. How would your testing strategy differ across the four missions?

  5. Distinguish between using code coverage to highlight what has not been tested from using code coverage to measure what has been tested. Describe some benefits and some risks of each type of use. (In total, across the two uses, describe three benefits and three risks.)

  6. In lecture, I used a minefield analogy to argue that variable tests are better than repeated tests. Provide five counter-examples, contexts in which we are at least as well off reusing the same old tests.

  1. List and describe five different dimensions (different “goodnesses”) of “goodness of tests”.

  2. Describe two difficulties and two advantages of state-machine-model based testing.

  3. Can you represent a state machine graphically? If so, how? If not, why not?

  4. Explain the relationship between graph traversal and our ability to automate state-model-based tests.

  5. Compare and contrast the adjacency and incidence matrices. Why would you use one instead of the other?

  6. What does it tell us about the system under test if the model of system (accurately) shows weak connectivity?

  7. What is the state explosion problem and what are some of the ways that state-model-based test designers use to cope with this problem?

  8. Consider the variable, “synchronize ends” in this dialog from OpenOffice Presentation. “Synchronize ends” can be checked or unchecked. Are these two values distinct? Justify your answer.




Part 3: Long Answers (20 points each)

  1. Imagine testing a date field. The field is of the form MM/DD/YYYY (two digit month, two digit day, 4 digit year). Do an equivalence class analysis and identify the boundary tests that you would run in order to test the field. (Don’t bother with non-numeric values for these fields.)



2. I, J, and K are signed integers. The program calculates

K = IntegerPartOf (SquareRoot (I*J))

For this question, consider only cases in which you enter integer values into I and J. Do an equivalence class analysis from the point of view of the effects of I and J (jointly) on the variable K. Identify the boundary tests that you would run (the values you would enter into I and J).

NOTE: Variations on this question consider:

K = I * J

K = I / J

  1. Ostrand & Balcer described the category-partition method for designing tests. Their first three steps are:

      1. Analyze

      2. Partition, and

      3. Determine constraints

Apply their method to this function:

I, J, and K are unsigned integers. The program calculates K = IntegerPartOf (SquareRoot (I*J)).

For this question, consider only cases in which you enter integer values into I and J.

Do an equivalence class analysis from the point of view of the effects of I and J (jointly) on the variable K.

  1. The Spring and Fall changes between Standard and Daylight Savings time creates an interesting problem for telephone bills. Focus your thinking on the complications arising from the daylight savings time transitions. Create a table that shows risks, equivalence classes, boundary cases, and expected results for a long distance telephone service that bills calls at a flat rate of $0.05 per minute. Assume that the chargeable time of a call begins when the called party answers, and ends when the calling party disconnects.

  2. Describe a traceability matrix.

    • How would you build a traceability matrix for the word processor in OpenOffice?

    • What is the traceability matrix used for?

    • What are the advantages and risks associated with driving your testing using a traceability matrix?

    • Give examples of advantages and risks that you would expect to deal with if you used a traceability matrix for the word processor. Answer this in terms of two of the main features of the word processor. You can choose which two features.

  1. What is regression testing? What are some benefits and some risks associated with regression testing? Under what circumstances would you use regression tests?

  2. Why is it important to design maintainability into automated regression tests? Describe some design (of the test code) choices that will usually make automated regression tests more maintainable and explain (briefly) why each choice increases maintainability.

  3. Suppose that you find a reproducible failure that doesn’t look very serious.

    • Describe three tactics for testing whether the defect is more serious than it first appeared.

    • As a particular example, suppose that the display got a little corrupted (stray dots on the screen, an unexpected font change, that kind of stuff) in OpenOffice’s word processor when you drag the mouse across the screen. Describe three follow-up tests that you would run, one for each of the tactics that you listed above.

  1. Imagine testing a file name field. For example, go to an Open File dialog, you can enter something into the field. Do a domain testing analysis: List a risk, equivalence classes appropriate to that risk, and best representatives of the equivalence classes. For each test case (use a best representative), briefly explain why this is a best representative. Keep doing this until you have listed 12 best-representative test cases.

  2. Suppose that you had access to the OpenOffice source code and the time / opportunity to revise it. Suppose that you had access to the source code and that you decided to do a diagnostics-based high volume automated test strategy to test OpenOffice Presentation’s treatment of copying and pasting of slides.

    • What diagnostics would you add to the code, and why?

    • Describe 3 potential defects, defects that you could reasonably imagine would be in the software that handles copy/paste of slide, that would be easier to find using a diagnostics-based strategy than by using a lower-volume strategy such as exploratory testing, spec-based testing, or domain testing.

  1. Consider testing the OpenOffice function that lets you save a document in HTML format.

    • How would you develop a list of risks for this capability? (If you are talking to people, who would you ask and what would you ask them?) (If you are consulting books or records or databases, what are you consulting and what information are you looking for in it?)

    • Why is this a good approach for building a list of risks?

    • List 10 risks associated with this function.

    • For each risk, briefly (very briefly) describe a test that could determine whether there was an actual defect.

Note: In practice, I probably won’t ask for 10 risks on the exam. On the exam, I’ll ask for fewer, perhaps five or seven. The review questions ask for more because I would rather have students think of a longer list and then give their best several items on the exam. If I told them to prepare a list of seven and then asked for seven on the exam, many students would struggle to remember the last two or three examples. If they prepare for 10, they are more likely to be able to give 7 on the exam than if I ask them to prepare for 7.

  1. Consider testing the OpenOffice function that lets you enter data into a table in the word processor.

    • How would you develop a list of risks for this capability? (If you are talking to people, who would you ask and what would you ask them?) (If you are consulting books or records or databases, what are you consulting and what information are you looking for in it?)

    • Why is this a good approach for building a list of risks?

    • List 10 risks associated with this function.

    • For each risk, briefly (very briefly) describe a test that could determine whether there was an actual defect.

  2. Consider testing the OpenOffice function that lets you enter data into a spreadsheet on a Presentation slide.

  • How would you develop a list of risks for this capability? (If you are talking to people, who would you ask and what would you ask them?) (If you are consulting books or records or databases, what are you consulting and what information are you looking for in it?)

  • Why is this a good approach for building a list of risks?

  • List 10 risks associated with this function.

  • For each risk, briefly (very briefly) describe a test that could determine whether there was an actual defect.

  1. Imagine that you were testing the feature, Save With Password in the OpenOffice word processor.

  • Explain how you would develop a set of scenario tests that test this feature.

  • Describe a scenario test that you would use to test this feature.

  • Explain why this is a particularly good scenario test.

  1. Imagine that you were testing the feature, Save With Password in the OpenOffice word processor.

  • Explain how you would develop a set of soap operas that test this feature.

  • Describe one test that might qualify as a soap opera.

  • Explain why this is a good soap opera test.

  1. Imagine that you were testing the feature, Insert Object in the OpenOffice Presentation module.

  • Explain how you would develop a set of scenario tests for this feature.

  • Describe a scenario test that you would use to test this feature.

  • Explain why this is a particularly good scenario test.

  1. Define a scenario test and describe the characteristics of a good scenario test. Imagine developing a set of scenario tests for the Outlining feature of the word processing module of OpenOffice. What research would you do in order to develop a series of scenario tests for Outlining? Describe two scenario tests that you would use and explain why each is a good test. Explain how these tests would relate to your research.

  2. (The following statement is not true, but pretend it is for exam purposes.) Sun has just announced that they will include email support in Release 2.0 of the StarOffice product, which they will ship in November, 2003. They announce that in the first implementation, the lists of new and saved messages will be displayed in spreadsheet format, based on the existing spreadsheet code.

    • List and briefly explain 5 risk factors that you would expect to find associated with the spreadsheet interface to the email database. (Refer to Amland’s paper for discussion of risk factors.)

    • For each risk factor, predict 2 defects that could arise in the spreadsheet interface part of the 2.0 project. By “predict”, I mean name and describe the potential defect, and explain why that particular risk factor makes this defect more likely.

  1. We are going to do some configuration testing on the OpenOffice word processor. We want to test it on

    • Windows 95, 98, and 2000 (the latest service pack level of each)

    • Printing to an HP inkjet, a LexMark inkjet, and a Xerox laser printer

    • Connected to the web with a dial-up modem (28k), a DSL modem, and a cable modem

    • With a 640x480 display and a 1024x768 display

      • How many combinations are there of these variables?

      • Explain what an all-pairs combinations table is

      • Create an all-pairs combinations table. (Show at least some of your work.)

      • Explain why you think this table is correct.

  1. Imagine that you are an external test lab, and Sun comes to you with OpenOffice. They want you to test the product. When you ask them what test documentation they want, they say that they want something appropriate but they are relying on your expertise. To decide what test documentation to give them, what questions would you ask (up to 7 questions) and for each answer, how would the answer to that question guide you?

  2. (The following statement is not true, but pretend it is for exam purposes.) OpenOffice.Org has just announced that they will include built-in support for digital cameras and digital video recorders in the Word Processing module. They announce that the word processor will have features that allow users to download single images and digitally stored movies from supported cameras and recorders. Users will be able to place a picture or movie in a document and view the picture or a movie while editing the document and they will be able to see the picture and the starting frame of the movie on the document printout. Users will be able to edit the pictures or movies, such as by cropping them, stretching or resizing them, changing their colors, or imposing text or other graphics on the image.

    • List and briefly explain 5 risk factors that you would expect to find associated with the new support for digital pictures and video. (Refer to Amland’s paper for discussion of risk factors.)

    • For each risk factor, predict 2 defects that could arise in the support for digital pictures and video. By “predict”, I mean name and describe the potential defect, and explain why that particular risk factor makes this defect more likely.

  1. The oracle problem is the problem of finding a method that lets you determine whether a program passed or failed a test.

Suppose that you were doing automated testing of spell-checking in the OpenOffice word processor.

Describe three different oracles that you could use or create to determine whether this feature was working. For each of these oracles,

    • identify a bug that would be easy to detect using the oracle. Why would this bug be easy to detect with this oracle? and

    • identify another bug that your oracle would be more likely to miss. Why would this bug be harder to detect with this oracle?

  1. You are using a high-volume random testing strategy for the OpenOffice word processing program. You will evaluate results by using an oracle.

    • Consider testing the spell-checking feature using oracles. How would you create an oracle (or group of oracles)? What would the oracle(s) do?

    • Now consider the placement of footnotes at the bottom of the page. How would you create an oracle (or group of oracles) for this? What would the oracle(s) do?

    • Which oracle would be more challenging to create or use, and why?

  1. You are using a high-volume random testing strategy for the OpenOffice Presentation program will evaluate results by using an oracle.

  • Consider inserting a spreadsheet into a slide. When you enter values into the spreadsheet, you can insert functions into the cells of the spreadsheet. Think about testing those functions using oracles. How would you create an oracle (or group of oracles)? What would the oracle(s) do?

  • Consider entering a chart into a slide. Once you have entered data into the chart, OpenOffice draws the chart. Think about testing the chart as drawn to determine whether it properly shows the chart data. How would you create an oracle (or group of oracles)? What would the oracle(s) do?

  • Which oracle would be more challenging to create or use, and why?

  1. Imagine that you were testing the spellchecking feature of the OpenOffice Word Processor. Describe four examples of each of the following types of attacks that you could make on this feature, and for each one, explain why your example is a good attack of that kind.

    • Input constraint attacks

    • Output constraint attacks

    • Storage constraint attacks

    • Computation constraint attacks.

(Notes for you while you study. Refer to Jorgensen / Whittaker’s paper on how to break software. Don’t give me two examples of what is essentially the same attack. In the exam, I will not ask for all 16 examples, but I might ask for 4 examples of one type or two examples of two types, etc.)

  1. Imagine that you were testing the OpenOffice word processor feature that lets you save a document in HTML format.

Describe four examples of each of the following types of attacks that you could make on this feature, and for each one, explain why your example is a good attack of that kind.

    • Input constraint attacks

    • Output constraint attacks

    • Storage constraint attacks

    • Computation constraint attacks.

(Notes for you while you study. Refer to Jorgensen / Whittaker’s paper on how to break software. Don’t give me two examples of what is essentially the same attack. In the exam, I will not ask for all 16 examples, but I might ask for 4 examples of one type or two examples of two types, etc.)

  1. What is the Defect Arrival Rate? Some authors model the defect arrival rate using a Weibull probability distribution. Describe this curve and briefly explain three of the claimed strengths and three of the claimed weaknesses or risks of using this curve.

  2. The following group of slides are from Windows Paint 95. Please don’t spend your time replicating the steps or the bug. (You’re welcome to do so if you are curious, but I will design my marking scheme to not give extra credit for that extra work.)

Treat the steps that follow as fully reproducible. If you go back to ANY step, you can reproduce it.

For those of you who aren’t familiar with paint programs, the essential idea is that you lay down dots. For example, when you draw a circle, the result is a set of dots, not an object. If you were using a draw program, you could draw the circle and then later select the circle, move it, cut it, etc. In a paint program, you cannot select the circle once you’ve drawn it. You can select an area that includes the dots that make up the circle, but that area is simply a bitmap and none of the dots in it have any relationship to any of the others.

I strongly suggest that you do this question last because it can run you out of time if you have not thought it through carefully in advance.




Here’s the opening screen. The background is white. The first thing that we’ll do is select the Paint Can

We’ll use this to lay down a layer of grey paint on top of the background. Then, when we cut or move an area, we’ll see the white background behind what was moved.



Here’s the screen again, but the background has been painted gray.

The star in the upper left corner is a freehand selection tool. After you click on it, you can trace around any part of the picture. The tracing selects that part of the picture. Then you can cut it, copy it, move it, etc.




This shows an area selected with the freehand selection tool. The bottom right corner is selected. (The dashed line surrounds the selected area.)

NOTE: The actual area selected might not be perfectly rectangular. The freehand tool shows a rectangle that is just big enough to enclose the selected area. For our purposes, this is not a bug. This is a design decision by Microsoft.



Next, we’ll draw a circle (so you can see what’s selected), then use the freehand select tool to select the area around it.

When you use the freehand selection tool, you select an area by moving the mouse. The real area selected is not a perfect rectangle. The rectangle just shows us where the selected area is.




Now we cut the selection. (To do this, press Ctrl-X.)

The jagged border shows exactly the area that was selected.




Next, exit the program, restart it, color the background grey, draw the circle, select the area around the circle and drag it up and to the right.




This time, we’ll try the Rectangular Selection tool.

With this one, if you move the mouse to select an area, the area that is actually selected is the smallest rectangle that encloses the path that your mouse drew.

So, exit the program, start it up, color the background, draw a circle, click the Rectangular Selection tool, select the area around the circle and move it up. It works.

This works.


Well, this was just too boring, because everything is working. When you don’t find a bug while testing a feature, one tactic is to keep testing the feature but combine it with some other test.

In this case, we’ll try Zooming the image. When you zoom 200%, the picture itself doesn’t change size, but the display doubles in size. Every dot is displayed as twice as tall and twice as wide.

So we exit the program, start it up, color the background grey, draw the circle, and then . . .




Bring up the Custom Zoom dialog, and select 200% zoom, click OK.


<< предыдущая страница   следующая страница >>