Response to Stop 29119 Petition /
「29119 Petition(29119の発行中止請願)」への回答

(訳注:このページは、ISO/IEC JTC1/SC7/WG26のWebサイト(に掲載されている"Response to Stop 29119 Petition“の日本語訳です。文書の性質上なるべく正確さを期すようにしていますが、念のため英文を掲載して対訳形式にしています。この文書はWG26に公式に認められています。翻訳はASTER(ソフトウェアテスト技術振興協会)として、にしが行いました。翻訳部分の全ての文責は翻訳者に帰着します。もし誤訳や不適切な部分があれば、までお知らせ下さい。)

ISO 29119 Background / ISO 29119 の背景

Up until last year there was no comprehensive set of international software testing standards. There are standards that touch upon software testing, but many of these standards overlap and contain what appear to be contradictory requirements with conflicts in definitions, processes
and procedures. There are some useful IEEE testing standards (e.g. IEEE 829, IEEE 1028) and national standards (e.g. BS 7925-1/-2) but there were large gaps in the standards relating to
software testing, such as organizational-level testing, test management and non-functional testing, where no useful standards existed at all. This means that consumers of software testing services (and testers themselves) had no single source of information on good testing practice.

昨年まで、ソフトウェアテストに関する包括的な国際標準はありませんでした。ソフトウェアテストに触れている標準はいくつかありますが、多くは重複しており、また定義やプロセス、手順に一貫していないところがあり矛盾した要求を含んでいるようです。IEEE(米国電気電子学会)の標準(IEEE829、IEEE1028など)や国家標準(BS 7925-1/02など)といったテストに関する有用な標準もありますが、組織レベルのテスト、テストマネジメント、非機能テストなどに大きなギャップがあるため、本当に有用な標準とは言えませんでした。これはテストサービスを活用する顧客(そしてテスト技術者自身)にとって、よいテストに関する情報をまとめて提供してくれるものが無い、ということを意味しています。

Given these conflicts and gaps, developing an integrated set of international software testing standards that provide far wider coverage of the testing discipline provided a pragmatic solution to help organizations and testers. And ideally this initiative would not re-
invent the wheel, but build upon the best of the available standards;
thus the motivation for the ISO/IEC/IEEE 29119 set of standards.

こういった矛盾やギャップを考慮すると、テストのdisciplineをとても広くカバーするまとまった国際標準を作成することが、テストを行う組織や技術者の助けとなる実践的な解を提供することになるわけです。理想を言えばこの活動は車輪の再発明になるべきではないでしょうが、実際のところこれまでの標準の優れたところを踏まえて作成することになるでしょう。これが、ISO/IEC/IEEE 29119シリーズの国際標準を作成する動機です。

In May 2007 ISO formed a working group to develop new standards on software testing – a new area for ISO. The scope of this initial work (ISO/IEC/IEEE 29119 parts 1, 2, 3 & 4) was largely defined by the existing IEEE and BSI standards (which they would replace), although it was clear from the onset that a completely new 'Test Processes’ standard would be required, in particular to ensure that agile life cycles and exploratory testing were considered, as well as more traditional approaches to software projects and to testing. Subsequently, work on a 'Test Assessment’ standard (ISO/IEC 33063) based on the 'Test Processes’ standard started. Much later (in 2012) a standard on Keyword-Driven Testing (ISO/IEC/IEEE 29119-5) was begun, and a proposal for a standard on 'Work Product Reviews’ is currently under ballot.

2007年5月にISO(国際標準化機構)は、これまで手がけてこなかったソフトウェアテストの分野で新しい標準を作成するWG(ワーキンググループ)を設立しました。初期の成果物(ISO/IEC/IEEE 29119 part 1~4)の範囲は(後にISO 29119シリーズに置き換えられることになる)IEEEやBSI(英国規格協会)の標準で既に広く定義されているものでしたが、テストプロセスに関する全く新しい標準を手がけること、特に、伝統的なソフトウェア開発やテストだけでなく、アジャイル開発や探索的テストも考慮にいれることが必要なのは明白でした。続いて、テストプロセスに関する標準を基にして、テストアセスメントに関する標準(ISO/IEC 33063)の作成も開始されました。さらにその後(2012年)、キーワード駆動テスト(ISO/IEC/IEEE 29119-5)に関する標準の作成が開始されました。ワークプロダクトレビューに関する新規作成の提案も現在投票中です。

The current set of standards handled by WG26, the joint ISO/IEC/IEEE working group on software testing, is shown below:

  • Concepts and Definitions (ISO/IEC/IEEE 29119-1) – published August 2013
  • Test Processes (ISO/IEC/IEEE 29119-2) – published August 2013
  • Test Documentation (ISO/IEC/IEEE 29119-3) – published August 2013
  • Test Techniques (ISO/IEC/IEEE 29119-4) – due for publication in 2014
  • Keyword-Driven Testing (ISO/IEC/IEEE 29119-5) – due for publication in 2015
  • Test Assessment (ISO/IEC 33063) – due for publication in 2014/2015
  • Work Product Reviews (awaiting approval – working draft available)


  • コンセプトと定義(ISO/IEC/IEEE 29119-1) – 2013年8月発行
  • テストプロセス(ISO/IEC/IEEE 29119-2) – 2013年8月発行
  • テスト文書(ISO/IEC/IEEE 29119-3) – 2013年8月発行
  • テスト技法(ISO/IEC/IEEE 29119-4) – 2015年発行予定
  • キーワード駆動テスト(ISO/IEC/IEEE 29119-5) – 2015年発行予定
  • テストアセスメント(ISO/IEC 33063) – 2015年発行予定
  • ワークプロダクトレビュー(ISO/IEC 20246) – ワーキングドラフト準備中

The first three of these standards were published over a year ago, and were available as drafts for several years prior to that. Both the IEEE and BSI contributed existing standards, which were themselves developed by consensus over many years, as source documents to the project (these
standards will be retired as the new standards are published).


The availability of these international testing standards provides a
number of potential benefits:

  • Improved communication – through a common terminology.
  • Definition of good practice in the testing industry – a guideline for testing professionals, a benchmark for those buying testing services and a basis for future improvements to testing practices. Note that we do not claim that these standards define 'best practice’, which will change based on the specific context.
  • A baseline for the testing discipline – for instance, the standard on test techniques and measures provides an ideal baseline for comparison of the effectiveness of test design techniques (for instance, by academics performing research) and a means of ensuring consistency of test coverage measurement (which is useful for tool developers).
  • A starting point – for any organization that is looking to define their testing processes, templates, techniques or concepts for the first time.


  • コミュニケーションの改善 – 用語の共通化を通して可能になります。
  • テスト業界における優れた取り組みの明確化 – テストのプロフェッショナルのためのガイドラインにも、テストサービスを依頼する顧客のためのベンチマークにも、優れた取り組みに向かう改善の基盤としても役立ちます。ただし私たちは、29119シリーズが「ベストプラクティス」だと主張するつもりはありません。何がベストなのか、は状況によって異なるからです。
  • テストのdisciplineに対するベースライン – 例えばテスト技法の標準は、テスト設計技法の効果の比較のベースラインを提供します。これは学術的な実証研究などで可能になります。またテストの測定の標準は、テストカバレッジの測定の一貫性を保証する手段を提供します。これはツールベンダにとって有用です。
  • きっかけ – 自分たちのプロセスやテンプレート、技法、コンセプトを最初に定義するために何かいいものを探している組織に有用でしょう。

Petition and Response /

In August 2014 the following online petition was created and the
following is the response from the convenor of ISO Software Testing
Working Group (WG26), Dr Stuart Reid.

2014年8月に、以下のようなオンラインの反対請願が始められました。またそれらに対し、ISOのソフトウェアテストのワーキンググループWG26のコンビーナ(訳注:ワーキンググループのリーダのこと)であるStuart Reid博士が回答を以下のように行っています。

“To the President of the International Organization for Standardisation, We, the undersigned, hereby petition the International Organization for Standardisation to suspend publication of ISO/IEC/IEEE 29119-4 and ISO/ IEC/IEEE 29119-5, and to withdraw ISO/IEC/IEEE 29119-1, ISO/IEC/IEEE 29119-2 and ISO/IEC/IEEE 29119-3. It is our view that significant disagreement and sustained opposition
exists amongst professional testers as to the validity of these standards, and that there is no consensus (per definition 1.7 of ISO/IEC Guide 2:2004) as to their content."

「国際標準化機構(ISO: the International Organization for Standardisation)会長(President)宛: 我々この反対請願に署名する者は、ISO/IEC/IEEE29119-4, 5の発行の延期と、ISO/IEC/IEEE29119-1, 2, 3の廃止を請願します。我々は、これらの標準の妥当性についてプロフェッショナルのテスト技術者の間でかなりの反対(significant disagreement)と継続的な異議(sustained opposition)が存在し、内容についてコンセンサス(ISO/IEC Guide 2:2004のdefinition1.7に書かれています)が存在しないという見方をしています。」

Members of the ISO Software Testing Working Group (WG26) are well-versed in the definition of consensus. The six years spent in gaining consensus on the published testing standards provided us all with plenty of experience in the discussion, negotiation and resolution of technical disagreements – if nothing else, we are now experts at compromise and reaching consensus.


However, as a Working Group (WG) we can only gain consensus when those with substantial objections raise them via the ISO/IEC or IEEE processes. The petition talks of sustained opposition. A petition initiated a year after the publication of the first three standards (after over 6 years’ development) represents input to the standards after the fact and inputs can now only be included in futursussse maintenance versions of the standards as they evolve.


It is unclear if 'significant disagreement’ reflects the number of professional testers who don’t like these standards or their level of dissatisfaction with their content, however the petition comments raise a number of issues which deserve a considered response:

「かなりの反対(significant disagreement)」が、29119シリーズを好まないプロフェッショナルのテスト技術者の数、もしくは内容に対する不満足さを反映しているかどうかは定かではありません。とはいえ、熟慮を要する回答を行うべき論点(issues)は増えてきています。

“The standards are not free." /

Most ISO/IEC/IEEE standards cost money (ISO is partially funded by the sale of standards), and the testing standards are no different in this respect. Personally, I would prefer all standards to be made freely-available, but I am not in a position to make this change – and do not know where the costs of development would come from. I can see that the charge made for the standards has forced a large proportion of those commenting on them to base their commentary on
second-hand descriptions of the standards – and with fuller access and better information I expect many of these people would have refrained from 'signing’ the petition and making incorrect statements about the standards.


“The standards 'movement’ is politicized and driven by big business to the exclusion of others." /

A large proportion of the members of WG26 are listed at our About WG26 page along with their employers. This list does not support the assertion in the petition. The seven editors (who do the
majority of the work) are from a government department, a charity, two small testing consultancies, a mid-size testing consultancy, a university and one is semi-retired. All WG members give their time
freely and many use their own money to attend meetings. As all received comments have their resolution fully documented anyone who submits a comment on a draft standard can easily see how their suggested change was handled – thus even those who cannot afford the time to come to WG meetings can easily influence the content of the standards. IEEE balloting also extends the consensus. For example, the IEEE balloting group for ISO/IEC/IEEE 29119-4 contains 76 people: 5 identify themselves as academic; 11 consultants; 9 government and/or military; 5 process management; 7 software producers; 5 project managers; 12 software product users/acquirers, and 21 with general interest.


“The methods advocated haven’t been tried and the standards do not emerge from practice." /

The number of years’ and range of testing experience on the Working Group (and the number of countries represented) shows that a wide range of experiences have been drawn on to create the standards. Early drafts were widely distributed and many organizations started (and continue) to use the standards – and so provided important feedback on their use to allow improvements to be made. At least one academic study at PhD level has been performed on the use of these early drafts within 14 different software organizations by Jussi Kasurinen of Lappeenranta University of Technology. Three of the main contributions of the thesis were described as: The concepts presented in the ISO/IEC 29119 test process model enable better end-product quality. The ISO/IEC 29119 test standard is a feasible process model for a practical organization with some limitations. The ISO/IEC 29119 test standard is a feasible foundation for a test process development framework.

WGに集まった何ヶ国もの代表の経験を基に何年もかけて議論したことは、29119シリーズをつくるために幅広い経験が引き出されたことを示しています。初期の標準案は広く配布され、多くの組織が適用し始めました(そして使い続けています)。そして重要なフィードバックが得られ、改善につながりました。ラッペーンランタ大学(Lappeenranta University of Technology)のJussi Kasurinenは、14の異なるソフトウェア組織に対して初期の標準案を適用するという博士号レベルの研究を行っています。この論文の3つの主要な結論は、以下の通りです:29119シリーズのテストプロセスモデルで提示されているコンセプトは、最終製品の品質をより高くできます。いくつかの制約はありますが、実際の組織で実現可能なプロセスモデルです。テストプロセスを構築するフレームワークの実現可能な基礎になります。

“The standards represent an old-fashioned view and do not address testing on agile projects." /

The standards were being continually updated until 2013 and so are inclusive of most development life cycles, including agile. As an example, the test documentation standard (ISO/IEC/IEEE 29119-3) is largely made up of example test documentation and for each defined document type example documentation for both traditional and agile projects is provided. The standards will be regularly reviewed and changes based on feedback from use have already been documented for the next versions.


“The standards require users to create too much documentation." /

Unlike the IEEE 829 standard, which it replaces, the test documentation standard, ISO/IEC/IEEE 29119-3, does not require documentation to follow specific naming conventions nor does it require a specific set of documents – so users can decide how many documents to create and what to call them. The standard is information-based and simply requires the necessary test information to be recorded somewhere (e.g. on a Wiki). As stated above, it is fully aligned with agile development approaches and so users taking a lean approach to documentation can be fully compliant with the standard.


“The existence of these standards forces testers to start using them." /

According to ISO, standards are “Guideline documentation that reflects agreements on products, practices, or operations by nationally or internationally recognized industrial, professional, trade associations or governmental bodies".


They are guideline documents therefore they are not compulsory unless mandated by an individual or an organization – it is up to the organization (or individual) as to whether or not following the
standards is required, either in part or in their entirety. However, if specified in a contract then they can define requirements on the testing, but as with all contracts this depends on the signatories. Because the standards includes the ability to apply its requirements in a range of
lifecycles, including agile, if a company is required to use the standards it will not prevent an agile approach.


My view of the testing standards is that they will be of most use to testers who want to see a definition of good practice and see how close (or far) they are away from it. They can then decide if they wish to change their practices and if they wish to adopt the standard then they
are free to tailor it to suit their needs. One of the most common uses I make of the testing standards is to use them as checklists – I then have more confidence that I haven’t accidentally ignored an important aspect of the testing.


“The Testing Standards are simply another way of making money through certification." /

There is currently no certification scheme associated with the testing standards, and I am not aware of one being developed. There is also no link between the ISO/IEC/IEEE Testing Standards and the ISTQB tester certification scheme. The ISTQB scheme could align with the body of knowledge represented by the ISO/IEC/IEEE testing standards, but that is a decision for those who run ISTQB.


“The Testing Standards do not allow exploratory testing to be used." /

Exploratory testing is explicitly included as a valid approach to testing in the standards. The following is a quote from part 1:"When deciding whether to use scripted testing, unscripted testing or a hybrid of both, the primary consideration is the risk profile of the test item. For example, a hybrid practice might use scripted testing to test high risk test items and unscripted testing to test low risk test items on the same project."

探索的テストは、価値の高いアプローチだと29119シリーズで明記されています。以下の引用は29119-1からのものです: “記述的なテストを行うか非記述的なテストを行うか、もしくは両者を混合して行うかを決める際には、テスト対象のリスクプロファイルを考えるのが第一です。例えば同じプロジェクトで両者を混合して行う場合、リスクの高いテストは記述的なテストを行い、リスクの低いテストは非記述的なテストを行う、といった方法がありえるかもしれません。”

If using exploratory testing (and also wanting to show compliance with the standard) then a simple rationale for not documenting tests in advance would be required. The standard also does not require the tests executed during an exploratory testing session to be documented – it does require they be “recorded", but this could be with a screen reader or as notes in a session sheet, and even this requirement can be tailored out if an organization deemed such recording as unnecessary.


“No-one knew about the standards and the Working Group worked in isolation." /

From the outset the development of the testing standards has been well-publicised worldwide by members of the Working Group at conferences, in magazine articles and on the web. Early in the
development, in 2008, workshops were run at both the IEEE International Conference in Software Testing and the EuroSTAR conference where the content and structure of the set of standards were discussed and improved. The working group also went to great lengths to invite the broader testing community to comment on the standard and to voice their opinions at meetings of our software testing standards working group, by placing information on the website on how individuals can get involved in the development of the standard (see our How to Get Involved page). Since then I have lost count of the number of testing conferences where the standards were presented on and discussed. At every one of my presentations on the standards I have invited the audience to become involved in their development. Of course, I am not the only person who has spoken about the standards – experts from many countries have spoken about the standards and invited participation.


Testing experts from around the world were invited to take part in the development of the standards – these included a number of prominent members of the AST who were personally approached and asked to contribute to the standards early in their development, but they declined to take part. Other members of the AST have provided input, such as Jon Hagar, who is the IEEE-appointed Project Editor – and he has presented on the standards at the CAST conference.

29119シリーズの作成には、世界中からテストのエキスパートが参加しています。その中には、作成初期に個人的にアプローチし貢献したいと問い合わせてこられたAST(訳注:Association for Software Testing。この反対請願の原動力となっている団体)の著名なメンバもいます。その後辞退されましたが。また、29119シリーズに貢献しているASTのメンバもいます。IEEEから参加しているプロジェクトエディタのJon Hager氏です。CASTカンファレンス(訳注:ASTの年次カンファレンス)で29119シリーズについて発表も行っています。

“No-one outside the Working Group is allowed to participate." /

There are a number of options for those interested in contributing to the standards – and these are all still open. The Working Group (WG) meets twice a year for 6 days – and is made up of experts acting in a personal capacity, appointed by national standards bodies, such as ANSI, or liaising organizations (e.g. IEEE). Most experts at Working Group meetings are supported by a 'mirror panel’ in their home country who act as reviewers and submit comments on the drafts. Many of these mirror panel members represent specific industry areas (e.g. financial testing), testing roles (e.g. independent consultant) or testing interest groups. These comments are then collated and moderated by the mirror panel to remove duplicates and contradictory suggestions before being submitted to the WG. The Working Group also accepts comments directly from any individual with an interest in software testing. Each comment is individually handled and the response to the comment is agreed by the WG and is fully documented. As an example, the WG itself resolved well over 3000 comments on the 'Test Processes’ standard. All comment resolution files are then circulated back to each country’s mirror panels, so that individual contributors can see how their comments were handled by the WG. This makes it a very transparent process that is accessible to any tester, anywhere.


“Context-Driven Testing isn’t covered by the standards." /
「29119シリーズでは、コンテキスト駆動テスト(Context-Driven Testing)はカバーされていない」

I fully agree with the seven basic principles of the 'Context-Driven School’ at To me most of them are truisms, and I can see that to those new to software testing they are a useful starting point. I also have no problem with context-driven testers declaring themselves as a 'school’, however I am unhappy when they assign other testers to other deprecated 'schools’ of their own making.

私は、http://context-driven-testing.comに載っている「コンテキスト駆動学派(Context-Driven School)」の7つの基本原則は全く正しいと思います。私にとってほとんどは自明のことですが、ソフトウェアテストを始めたばかりの技術者には有用なきっかけになるという点も理解しています。コンテキスト駆動の立場を取る技術者が自分たちを「学派(school)」と宣言するのは別に構いませんが、そうでない技術者を「反対派」のように扱うのは残念ですね。

Jon Hagar, the Project Editor for the ISO/IEC/IEEE 29119 set of standards, is a supporter of the context-driven school and he, along with the rest of the Working Group, ensured that that many of the context-driven perspectives were considered. For instance, the following statement is early in part 1: “Software testing is performed as a context-managed process."

29119シリーズのプロジェクトエディタであるJon Hagerはコンテキスト駆動学派の支持者ですが、WGの他のメンバと同様に、29119シリーズにはコンテキスト駆動の考え方が考慮されていると考えています。例えば、29119-1の最初の方には「ソフトウェアテストは、コンテキストをきちんと考えたプロセス(context-managed process)で行われるものである」と書いてあります。

The standards do, however, also mandate that a risk-based approach is used, as can be seen in the following quote (also from part 1): “A key premise of this standard is the idea of performing the optimal testing within the given constraints and context using a risk-based approach." I see no problem in following both risk-based and context-driven approaches simultaneously as I believe the context and the risk-profile to be part of the same big picture which I would use to determine my
approach to the testing.


Dr Stuart Reid
Convenor, WG26
8th September 2014


WG26コンビーナ Stuart Reid博士