Diversitas.org Project

Status message

You are currently in UIA's online document archive. These pages are no longer maintained. To search the full archive click here.

UIA Response to NCDNHC public support evaluation

Methodology – evidence - questions

In response to the need for the analysis of public support for .org bids delegated to the Noncommercial Domain Name Holders Constituency (NCDNHC), the Union of International Associations (UIA) hopes that the NCDNHC keeps the following points in mind when reviewing the various applications:

  1. Methodological concerns and comments
  2. UIA consultation
  3. UIA survey
  4. Formal support for UIA registry activity
  5. UIA membership support
  6. Presentation of pattern of support in practice for UIA registry activity
  7. Letters of support for UIA bid
  8. Concluding questions

  1. Methodological concerns: The UIA proposal responds to the specific request in ICANN’s RFP to "Submit any evidence that demonstrates support for your proposal among registrants in the .org TLD, particularly those actually using .org domain names for noncommercial purposes". The UIA proposal therefore set out to demonstrate a long-term pattern of support for its registry activity amongst non-profit organizations -- notably amongst those with websites.

    Evidence of “support” comes in many forms, implicit and explicit. In analyzing the endorsement for proposals (whether “public support” or otherwise), the UIA recommends that the NCDNHC distinguish among categories of support such as:

    • Generic expressions of support to improve, enhance or distinguish the .org community in some way -- which may be just as applicable to other proposals as the bid named by the supporter;
    • Character references for an organization, unrelated to the contents of its proposal – which, whilst being a indication of the merits of the proposer, do not constitute clear support for the bid or indeed for the superior quality of the bid in comparison with other bids (or indeed in comparison with other bidders);
    • Endorsement for a proposal from an official of an organization writing in their personal capacity – which should not be construed as support from the organization (even if the organization’s letterhead or email is used);
    • Support for the notion of non commercial involvement in the running of the .org registry (e.g, “do you think the civil society should have a role in providing guidance to the registry running .org?”) – such demonstrations of “soft” support do not constitute support for a specific bid, even though they have in some cases been portrayed as such;
    • Authorized endorsement for a proposal from an organization without any operational commitment for future involvement or support – which is a useful indicator of the intention to support in the moment (indeed such support may be reciprocation for support previously received) but is weaker than support where subsequent obligations are incurred;
    • Endorsement for a proposal from an authorized person within an organization implying operational commitment to future involvement or support. This is one of the most valuable forms of endorsement, although a distinction (also relevant to the two previous cases) has to be made between:
      • a person with inadequate understanding of domain name and .org issues to distinguish between the merits and weaknesses of the respective bids
      • a person who does indeed have adequate understanding
    • Endorsement for a proposal from members of the bidding organization – which, even when numerous, must necessarily represent a very particular segment of the non-commercial constituency rather than reflecting a wide variety of cross-sectoral perspectives of bodies using the .org domain;
    • Endorsement of a proposal in the light of a bidder’s registry capability with other TLDs – without necessarily reflecting awareness of the challenges of the non-commercial domain or the bidders capacity to deal with them;
    • Endorsement of a proposal as a result of an extensive email solicitation for such support -- so demonstrating considerable campaigning capability on the part of the bidder but failing to demonstrate considered support (relative to the other bids) and ignoring the digital divide;
    • Implicit support revealed as long-term patterns of responsive conduct to the registry activity of the bidding organization by a large and representative sample of registered non- commercial organizations – which does not imply any policy endorsement, but it does constitute legitimate evidence of probable future involvement in the registry in practice. This kind of analysis can only be performed by a bidder with a direct registry relationship with members of the .org community. As in the case of the UIA, it is particularly significant where the maintenance of very thick registries depends on voluntary involvement of registrants rather than a contractual obligation.
    • Formal resolutions approved in support of the registry activity of an organization, notably by key international governmental or nongovernmental organizations – whilst recognizing that few such bodies are able to undertake informed debate on domain name issues in their plenary sessions and resulting resolutions may focus on registry activity based on older technologies

An analysis of support according to the above categories enables the qualitative weight of support to be made explicit.

Other methodological comments

    1. Other things being equal, there is another problem in weighting support. In a democratic process, are the views of a “large”, “important” or “competent” organization to be considered as weighing more heavily than those of a smaller, less important or less competent body, or an individual? Similarly how is weighting attached to a view expressed by a highly active .org registrant compared with a registrant who holds an inactive domain or no domain?

    2. Harold Feld, on behalf of the NCDNHC, indicated in his questionnaire on the Forum that the “the winning bidder should understand that the majority of registrants are unsophisticated users” (ICANN .org Forum, 30 July 2002, emphasis added). Having tracked the uptake of domains by non-commercial international organizations over the past 5 years, the UIA recognizes the fundamental methodological challenge for any survey that purports to reflect “majority opinion” regarding the respective merits of the bids. There is a real danger that explicit endorsements will only be received from an inner circle of bodies that are aware of the issues and may well already be interested parties in the bid process. Those who have recently acquired web domains may not feel sufficiently empowered to make an informed judgement (with or without the advice of their technical support). Support is unlikely to have been sought or received from those non-commercial bodies who have not yet acquired domains or do not have email, even though those beyond the digital divide are described as an important marketing “target” by many of the bids and part of the future .org constitutency.

    3. It is relevant to note that Harold Feld’s concern with respect to “unsophisticated users” might also be extended to public support for “unsophisticated registries” as bidders. If the bidder and its supporters have little knowledge of .org as a community distinct from other domains, then the support is not broadly reflective of the needs of the non-commercial constituency. The approach by many bidders to use conventional marketing techniques for .org is a symptom of this problem.

    4. It is normally assumed that the bid documents are the basis for comparative evaluation and the bid deadline precludes subsequent modifications until a new benchmark is established (in this case by the draft ICANN Staff Report). It is regrettable that weighting of unknown proportion has been given to support letters after the deadlines, thus encouraging bidders to demonstrate quantitative rather than qualitative support through the Forum. It seems that almost all public discussion of the relative merits of the bids has been stifled as a consequence. There was never any indication that the Forum should be used in this way by the bidders. If numbers of supporters were the issue, then voting would have been a more appropriate method of showing support.

      In retrospect, given the collective desire to move towards a more democratic, open decision process regarding the respective merits of the bids, a case could have been made for a quite different process. As in any conventional election, ICANN or NCDNHC could have presented the bids on a single web page (with links), and e- mailed all 2.6 million registrants and invited those organizations that are not-for- profits to vote. The fact that this was not done, and only a very small percentage of registrants are aware of the bid process (whether or not their lack of sophistication enables them to make a considered vote), needs to be considered in assessing the quality of public support received.

    5. Technical suppliers: It is important to be aware of the degree to which public support is reinforced or undermined on the basis of unstated criteria governing assessment of bodies associated with the bid (ie not the bidder itself). This issue has become particularly apparent in the case of using VeriSign as a subcontractor, which UIA has proposed to do for three years because of this company’s unquestioned technical capacities and in order to guarantee continuity of service for registrants during transition of the operator. Furthermore without this being the subject of open assessment, but given the time scale to the transition on 1 January 2003, other bidders may find themselves obliged to make use of subcontractors such as VeriSign.

      Aside from their technical capacity, the criteria influencing public opinion in their evaluation of technical suppliers to a bidder must necessarily be political. Such political issues need to be seen as forming part of a larger and highly controversial debate within the internet and non-commercial communities on unresolved issues such as “big business”, “Microsoft- style” monopolies, convenient registry-registrar relationships, questionable practices of registrars linked to bidders, criticism of ICANN, unresolved relationships with the US Department of Commerce, consideration of non-US registry operation and the preference of many for a non-profit operator for .org. Many technical partners and contractors involved in the various bids could be evaluated in the light of such unstated and unresolved issues, which are not part of the RFP process.

      The degree to which the .org bid process should be used as a surrogate for addressing issues that have not been explicitly placed on the table should be considered in assessing declarations of support. The UIA is extremely sensitive to the wider implications of these issues, but has taken a pragmatic approach (given the technical exigencies) and through its bid strategy has distanced the long-term operation of .org from outsourced functions which might be affected by such unresolved political debates.

  1. Consultation: The process of "consultation" is not specifically mentioned in ICANN’s RPF (whether or not it may be considered to have been implied). The short window provided between the issuance of the RFP and the due date for proposals made it impossible to perform a valid consultation process across the entire .org community, even for an organization like UIA that has exceptional contact information for that community. Its "failure" to “consult” with any organizations -- beyond the manner in which it "consults" them for information for its registries -- in no way means that it does not value the opinion of these organizations, nor does it imply that consultations will not be made in the future should it become the .org operator. The UIA also understands only too well that for larger organizations, with many levels of decision-makers, the correct process of consultation, to obtain an authoritative opinion, must be scheduled over a longer period than that available under the ICANN process.

    In this respect, a well-recognized methodological issue in any consultation process is whether the process results in (a) genuine consensus, amongst a representative cross-section of a community, that meaningful consultation has taken place rather than (b) the claim that genuine consultation has taken place.

  2. Survey: The UIA considered doing an email / web survey of some 25,000 organizations (with email and web access) on which it maintains thick registry entries (with their voluntary assistance). It rejected this for four reasons:

    • the UIA had just recently completed another survey on international meetings (with a response rate noted elsewhere as further concrete evidence of support for its approach "across the global internet community"); it considered that a second survey would be intrusive and insensitive to the limited resources that many organizations have to respond to such requests;

    • having tracked the uptake by organizations of websites, the UIA was very aware of the abstruse nature of .org domain operation for many senior decision -makers, even amongst organizations with websites. Additionally we did not know, in terms of social survey methodology, how to construct an unbiased questionnaire for people who may well know nothing about the politico-technical issues on which their support was requested -- especially if they were "beyond the digital divide";

    • even though bodies with email may have websites, the technical relationship between the two applications often precludes convenient web surveys and so the result would have been biased towards those organizations with more sophisticated systems (or to people within those bodies with the necessary skills);

    Concern was also felt that for support to be operationally significant rather than well-meaning, a much more complex process of interaction would have been required with the more authoritative levels of any organization consulted rather than relying on interaction with those having the necessary technical understanding of the issues.

  3. Formal support for UIA registry activity: The UIA bid pointed to the fact that its registry activity is the subject of a formal UN/ECOSOC resolution dating back to 1950 – support from a body that formally represents “we the peoples”. The UIA’s registry activity is the basis for its continued formal (and periodically reviewed) relations with various UN agencies -- and many more such bodies currently subscribe to the online version of one of its thick registries. This activity, and its future electronic development, has also been formally acknowledged in two major project proposals evaluated positively by the World Bank and the European Commission.

  4. UIA membership support: The bid pointed to the fact that the UIA constitution (dating back in various forms to 1910) embodied a formal commitment to registry activity and that it was on this basis that its Members supported that work. The UIA Full Members are characterized by the importance they attach to non-profit associative action, especially as represented through international associations. Whether or not they are currently associated with one or more such bodies, they are highly sensitive to the issues preoccupying the non-commercial world and are thus representative, to a valuable degree, of the views of that sector. The bid was prepared by the UIA Secretariat under powers delegated to it by those Members.

  5. Presentation of pattern of support in practice: In the light of the above issues, the UIA therefore chose to present data indicative of a pattern of implicit, long-term support in practice for UIA thick registry activity through regular communications with thousands of non-profits globally. The UIA submitted in the bid, and in subsequent comments to this Forum, that this was a telling indicator of how its registry process was viewed in practice. It was further argued that this was evidence, in the sense requested by ICANN, for the probability of such support in future in relation to its operation of the .org domain.

    This indication of support for UIA registry activity in practice takes the following forms:

    The methodological question for ICANN (and in this case for the NCDNHC) is whether evidence for such support in past and continuing practice is to be considered effective support for what is envisaged in the proposal. This needs to be evaluated by ICANN against the significance for future practice of letters of support which are in no way legally binding on the bodies supplying them as a commitment to a pattern of practice -- and may indeed, given the time constraints, have been produced without reference to the competent decision-making authority from which they have been received.

  6. Letters of support: The 15 letters of support appended to the UIA proposal were from organizations which could respond quickly to personal requests from UIA staff. They include organizations ranging from the International Development Research Corporation (a Crown corporation of Canada supported by their government) to the Centre for Civil Society of the London School of Economics.. They were included to exemplify the range of bodies from which such letters of support could be obtained given greater time. We understand that a similar number of letters were also sent directly to ICANN in support of our bid. We were not aware of any criteria on which organizational support was to be evaluated in terms of importance. We also were not aware that letters of support arriving after the application deadline would be weighed as part of the formal evaluation process. Whether in a democratic process of consultation, the weight of some registrants is to be considered greater than others is again a matter for ICANN to decide.

    This information is provided in two forms:

    • Appendix F to the bid document
    • Summary of Support on Diversitas site

  7. Concluding questions

    The UIA would find it most helpful to better understand the following:

    1. What is the overall weighting assigned to the criteria related to support level from the non-commercial community in the evaluation process? What is the weighting associated with the other criteria that the NCDNHC will be assessing?

    2. Will there be a methodological scheme for assessing the relative value of various levels of support? Shouldn’t it have been decided at the front end of the bid process? How was the methodology constructed and what is it? Or will it simply be a quantitative assessment?

    3. Will various “letters of support” arriving after the application deadline be included in the evaluation? If so, will they be provided with the same weighting as indicators of support provided within the deadlines set by ICANN?

    4. How much weight will be given to the greatest strength of the UIA/Diversitas bid – the patterns of support in practice?

    We are very supportive that the NCDNHC – with the greatest stake and knowledge of civil society within ICANN -- is playing a significant role in the evaluation of the various bids. We also recognize that the NCDNHC may not have been involved in the front-end in laying out the bid process, and may have a limited ability at this stage in the process to move forward in the fashion it would under other circumstances have liked to pursue.