Research Design

Phase III: Cataloging the Contents of Constitutions

The third, and most important, phase of the project involves coding each constitutional event. Although each of the other phases is useful in its own right, the main reason for the other aspects of the project is to facilitate the coding of constitutions. Details about the procedures we use to code constitutions are provided in the remainder of this section.

Survey Instrument

A critical task in our project was the review and selection of those attributes of constitutions to be included in the survey, which now includes more than 650 different questions. In order to create a survey that captures the full breadth of constitutional content, we developed a comprehensive process of reviewing the literature on comparative constitutionalism, reading a large sample of actual constitutions, and consulting with scholars of constitutional design. In this section, you can read more about the creation of the survey instrument as well as view the full survey instrument.

Section of the CCP survey instrument

Section of the CCP survey instrument

Designing the Survey Instrument — We began by reviewing the limited number of data projects on the topic. In this regard, the Maarseveen and van der Tang (1978) volume served as a particularly useful foundation from which to build, as did the key system that was developed by Tschentscher (2004). Other comparative sources helped us catalog the various ingredients of historical constitutions, from the simple United States document to the lengthy Brazilian one to Soviet-inspired documents. After several months of editing, we settled on a fairly stable draft of the survey instrument. We then formed an advisory board of leading scholars on comparative constitutional law to review our draft and provide general guidance on substantive and operational strategy. Based on the helpful comments and suggestions of these experienced constitutional scholars, we refined our question set considerably.

Question Categories — The survey questions are organized into the following categories:

  • General Characteristics
  • Amendment Provisions
  • Executive
  • Legislature
  • Judiciary
  • Federalism
  • Elections
  • Regulatory and Oversight Institutions
  • International Provisions
  • Duties
  • Criminal Procedures
  • Rights
  • Special Issues Domains (e.g. protection of the environment, treatment of the mass media, and regulation of the military).

Question Types — A crucial step was to transform our concepts and indicators into survey items and accompanying instructions that are clear and comprehensible for our coders. Our instrument includes three types of questions: (1) open-ended responses, (2) single-answer multiple choice, and (3) multiple-answer multiple choice. When possible we drafted closed-ended questions of the second and third type. Even these questions, however, allow the coder to respond “other” or “not specified,” followed by an open-ended response. For more fine-grained topics or those that we judged to be better dealt with inductively, we used open-ended response formats.

Citation and Comment Information — In addition to the actual survey response, the survey instrument asks for two pieces of information for each question: (1) the section or article of the constitution that served as the source of the coder’s response, and (2) the coder’s comments. While coders are required to provide the first piece of information, they are expected to use the comments only to identify any problems of interpretation or to add responses that are not included among the pre-established choices.

Instructions to Coders — Below most questions on the survey, we provide instructions that clarify the question or offer help with respect to known issues of interpretation.

Coding Procedures

We hire select undergraduate, graduate, and law students to code the constitutional events. In order to minimize errors in the final data set, each constitutional event is surveyed twice by different coders working independently. After this double-coding, each constitution is reviewed by a principal investigator or a senior research assistant (whom we call a reconciler) to check for errors and to resolve discrepancies among multiple coders. The final data set includes those responses for which the two coders agree and a “reconciled” answer for questions in which they disagree. Every record is subject to this checking and reconciliation process before being considered a part of the data set ready for public distribution. We believe the procedure described above maximizes reliability while providing a reasonably efficient coding method as well.

Interpretation Issues

Constitutions are often unclear in their meaning, which means one of the primary jobs of our employees is constitutional interpretation. We have created a message board system for keeping track of our interpretation decisions and bringing future employees up to date on past decisions. If the coders or reconcilers have questions about how to interpret specific provisions, they can ask the principal investigators directly by posting a question to the message board. A searchable archive of all posts to the message board is maintained, so that “precedents” established through earlier interpretations are available to coders and primary decision-makers alike.

In addition to the message board, we have created a glossary of constitutional terms and a list of our coders’ most Frequently Asked Question (FAQ’s). These documents give examples of our coding protocol and answer fundamental questions that are commonly asked by new coders. Some might be interested in this article we published that explored the extent and sources of errors in the data.

Survey Software

CCP coding software

CCP coding software

In order to facilitate the coding process, we designed and developed special survey software that is customized to our particular needs. Coders enter data in what is essentially a custom web survey. Particular answers branch to applicable (or skip around non-applicable) follow-up questions and users may navigate the survey in any order. If coders are coding a constitutional event that amounts to only a small amendment of an existing constitution, they may use an “import” feature that allows them to import the answers from an existing event and modify the answers accordingly.

One of the unique features of the survey engine is its reconciliation capability. During reconciliation, a PI or reconciler reviews the answers of multiple coders of a single constitutional event and produces a “final” row of data for an observation. The software application highlights discrepant answers and provides the reconciler with all information entered by the coders, allowing the reconciler to render a final judgement on each discrepancy. In this way, the software provides a convenient and reliable method for producing a final set of data from multiple codings.

ERRORS AND OMISSIONS

Although we have done our best to ensure that all data released by the CCP are reliable, data from a project of this magnitude will inevitably suffer from some errors and omissions. Moreover, constitutions often lend themselves to various interpretations. We kindly ask users of these data to help improve the data’s reliability by reporting such mistakes and/or differences of interpretation that they encounter to us via the link below. This will ensure the data contained in future releases are as reliable as possible.

Report an Error


< Phase II: The Repository of Constitutional Texts Back to Summary >